Kubernetes部署DNS和Dashboard

Kubernetes部署DNS和Dashboard

一、部署DNS

k8s的github 有很多组件的安装文档,但是,安装需要核对自己的k8s版本,找到对应版本的github,否则安装过程中会有很多问题。

本文k8s机器基于kubernetes1.5.x + centos集群,安装方式见:Centos7搭建kubernetes搭建

安装

获取/cluster/addons/dns/kubedns-controller.yaml.sed, 修改domain和镜像(如果能翻墙,可以不修改镜像)。
dns 基础配置:

$DNS_DOMAIN=cluster.local
$DNS_SERVER_IP=10.254.0.100

deployment文件skydns-rc.yaml如下:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: kube-dns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
spec:
  # replicas: not specified here:
  # 1. In order to make Addon Manager do not reconcile this replicas parameter.
  # 2. Default is 1.
  # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
  strategy:
    rollingUpdate:
      maxSurge: 10%
      maxUnavailable: 0
  selector:
    matchLabels:
      k8s-app: kube-dns
  template:
    metadata:
      labels:
        k8s-app: kube-dns
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
        scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]'
    spec:
      containers:
      - name: kubedns
        # image: gcr.io/google_containers/kubedns-amd64:1.9
        image: registry.cn-hangzhou.aliyuncs.com/google-containers/kubedns-amd64:1.9
        resources:
          # TODO: Set memory limits when we've profiled the container for large
          # clusters, then set request = limit to keep this container in
          # guaranteed class. Currently, this container falls into the
          # "burstable" category so the kubelet doesn't backoff from restarting it.
          limits:
            memory: 170Mi
          requests:
            cpu: 100m
            memory: 70Mi
        livenessProbe:
          httpGet:
            path: /healthz-kubedns
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        readinessProbe:
          httpGet:
            path: /readiness
            port: 8081
            scheme: HTTP
          # we poll on pod startup for the Kubernetes master service and
          # only setup the /readiness HTTP server once that's available.
          initialDelaySeconds: 3
          timeoutSeconds: 5
        args:
        # - --domain=cluster.local.
        - --domain=cluster.local.
        - --dns-port=10053
        - --config-map=kube-dns
        # This should be set to v=2 only after the new image (cut from 1.5) has
        # been released, otherwise we will flood the logs.
        - --v=2
        - --kube-master-url=http://192.168.99.100:8080 # 指定api
        env:
        - name: PROMETHEUS_PORT
          value: "10055"
        ports:
        - containerPort: 10053
          name: dns-local
          protocol: UDP
        - containerPort: 10053
          name: dns-tcp-local
          protocol: TCP
        - containerPort: 10055
          name: metrics
          protocol: TCP
      - name: dnsmasq
        # image: gcr.io/google_containers/kube-dnsmasq-amd64:1.4.1
        image: registry.cn-hangzhou.aliyuncs.com/google-containers/kube-dnsmasq-amd64:1.4
        livenessProbe:
          httpGet:
            path: /healthz-dnsmasq
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        args:
        - --cache-size=1000
        - --no-resolv
        - --server=127.0.0.1#10053
        # - --log-facility=-
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        # see: https://github.com/kubernetes/kubernetes/issues/29055 for details
        resources:
          requests:
            cpu: 150m
            memory: 10Mi
      - name: dnsmasq-metrics
        # image: gcr.io/google_containers/dnsmasq-metrics-amd64:1.0.1
        image: registry.cn-hangzhou.aliyuncs.com/google-containers/dnsmasq-metrics-amd64:1.0
        livenessProbe:
          httpGet:
            path: /metrics
            port: 10054
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        args:
        - --v=2
        - --logtostderr
        ports:
        - containerPort: 10054
          name: metrics
          protocol: TCP
        resources:
          requests:
            memory: 10Mi
      - name: healthz
        # image: gcr.io/google_containers/exechealthz-amd64:1.2
        image: registry.cn-hangzhou.aliyuncs.com/google-containers/exechealthz-amd64:1.2
        resources:
          limits:
            memory: 50Mi
          requests:
            cpu: 10m
            # Note that this container shouldn't really need 50Mi of memory. The
            # limits are set higher than expected pending investigation on #29688.
            # The extra memory was stolen from the kubedns container to keep the
            # net memory requested by the pod constant.
            memory: 50Mi
        args:
        - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null
        - --url=/healthz-dnsmasq
        - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null
        - --url=/healthz-kubedns
        - --port=8080
        - --quiet
        ports:
        - containerPort: 8080
          protocol: TCP
      dnsPolicy: Default  # Don't use cluster DNS.

修改service文件skydns-svc.yaml:

apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "KubeDNS"
spec:
  selector:
    k8s-app: kube-dns
  # clusterIP: $DNS_SERVER_IP
  clusterIP: 10.254.0.100
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP

启动

kubectl create -f  skydns-rc.yaml
kubectl create -f  skydns-svc.yaml

修改各个node节点的kubelet配置并重启:

vim /etc/kubernetes/kubelet
# 添加这一行
KUBELET_ARGS="--cluster_dns=10.254.0.100 --cluster_domain=cluster.local"
systemctl restart kubelet

验证

cat nginx-deploy.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2 # tells deployment to run 2 pods matching the template
  template: # create pods using pod definition in this template
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

启动查看

$ kubectl create -f nginx-deploy.yaml
nginx-deployment-4087004473-k35hw   1/1       Running   0          10m
nginx-deployment-4087004473-z081c   1/1       Running   0          10m

cat nginx-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
  selector:
    app: nginx
$ kubectl create -f nginx-svc.yaml
$ kubectl get svc
NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   10.254.0.1       <none>        443/TCP   4d
nginx        10.254.126.209   <none>        80/TCP    2h

现在,部署了nginx pods和对应的service,启动一个容器测试下:

$ kubectl run curl --image=radial/busyboxplus:curl -i --tty
[ root@curl-2421989462-w9lhz:/ ]$ cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local DHCP
nameserver 10.254.0.100
nameserver 10.0.2.3
options ndots:5

[ root@curl-2421989462-w9lhz:/ ]$ nslookup nginx
Server:    10.254.0.100
Address 1: 10.254.0.100 kube-dns.kube-system.svc.cluster.local

Name:      nginx
Address 1: 10.254.126.209 nginx.default.svc.cluster.local

[ root@curl-2421989462-w9lhz:/ ]$ curl nginx
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

至此,dns部署成功。可以做服务发现,比如新加一个nginx pod,可以直接访问service name nginx 作负载均衡转发给对应pod。

二、部署Dashboard

wget https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml 

修改

 args:
          - --apiserver-host=http://192.168.99.100:8080

启动

$ kubectl create -f kubernetes-dashboard.yaml 
$ kubectl get po -n kube-system
NAME                                    READY     STATUS             RESTARTS   AGE
kube-dns-1607457672-0bmxp               4/4       Running            8          1d
kubernetes-dashboard-3697257891-h2fcx   1/1       Running            2          20h
$ kubectl get svc -n kube-system
NAME                   CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
kube-dns               10.254.0.100    <none>        53/UDP,53/TCP   2d
kubernetes-dashboard   10.254.224.65   <nodes>       80:30732/TCP    20h

浏览器访问http://nodeIP:30732

但是,每次重启kubernetes-dashboard,CLUSTER-IP和PORT都会变,为了方便暴露给外部访问,可以结合DNS使用。

编辑任意一个node的/etc/resolv.conf,添加nameserver

search default.svc.cluster.local svc.cluster.local cluster.local DHCP
nameserver 10.254.0.100

修改kube-proxy: /etc/kubernetes/proxy

KUBE_PROXY_ARGS="--cluster-cidr=172.30.0.0/16"

–cluster-cidr: 用于桥接来自群集外部的流量。 如果没有配置,则不执行群外桥接。

修改后重启kube-proxy,然后通过service name.namespace访问dashbord:

& curl kubernetes-dashboard.kube-system
 <!doctype html> <html ng-app="kubernetesDashboard"> <head> <meta charset="utf-8"> <title ng-controller="kdTitle as $ctrl" ng-bind="$ctrl.title()"></title> <link rel="icon" type="image/png" href="assets/images/kubernetes-logo.png"> <meta name="viewport" content="width=device-width"> <link rel="stylesheet" href="static/vendor.4f4b705f.css"> <link rel="stylesheet" href="static/app.93b90a74.css"> </head> <body> <!--[if lt IE 10]>
      <p class="browsehappy">You are using an <strong>outdated</strong> browser.
      Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your
      experience.</p>
    <![endif]--> <kd-chrome layout="column" layout-fill> </kd-chrome> <script src="static/vendor.6952e31e.js"></script> <script src="api/appConfig.json"></script> <script src="static/app.8a6b8127.js"></script> </body> </html>

这样,可以配置nginx代理。集群外就能访问dashboard,也不用担心虚拟ip和端口改变。

location / {
		proxy_set_header    Host    $host;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
	  	proxy_pass http://kubernetes-dashboard.kube-system;
        }
}
CONTENTS