kubernetes环境下私有仓库搭建

前期在客户那里搭建了基本运行环境，鉴于很多企业的环境都是内部网无法连接外部，因此搭建私有仓库是逃避不开的问题，按照网上的步骤搭建，虽然遇到一些问题，但还好都算容易解决了，下面大致把步骤记录一下便于下次去客户那里更新。

首先在需要在启动registry Pod的机器上把registry images获取下来

docker pull registry

然后在生成几个构建persistence volumn(pv), persistence volumn claim(pvc),以及registry rc和service的文件

[root@k8s-master registry]# cat pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01
  release: stable
spec:
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    path: /k8s/test
    server: 10.182.168.99

[root@k8s-master registry]# cat pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim2
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20G

[root@k8s-master registry]# cat registry-rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: registry
  labels:
    name: registry
spec:
  replicas:
  selector:
    name: registry
  template:
    metadata:
      labels:
        name: registry
    spec:
      containers:
      - name: registry
        image: registry
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort:
        volumeMounts:
        - mountPath: "/var/lib/registry"
          name: mypd
      volumes:
      - name: mypd
        persistentVolumeClaim:
          claimName: myclaim2

[root@k8s-master registry]# cat registry-srv.yaml
apiVersion: v1
kind: Service
metadata:
    name: registry
    labels:
      name: registry
spec:
    type: NodePort
    ports:
    - port:
      nodePort:
    selector:
      name: registry

10.182.168.99是k8s-node-1的地址,非flannel集群地址

一个一个建立起来，当然需要在k8s-node-1上打个标签

kubectl label node k8s-node- name=registry

在建立registry-rc的时候遇到一些问题。

registry pod总是处于container creating的状态。

• 首先需要在启动registry pod的机器上创建相关的目录，我的是在k8s-node-1上创建/k8s/test

通过describe pods 一看，基本都是nfs mount的问题，解决方式如下：

• 启动nfs service

systemctl start nfs

• 遇到

Output: mount.nfs: access denied by server while mounting 10.182.168.99:/k8s/test错误时,修改配置文件/etc/exports，加入 insecure 选项

/k8s/test  *(insecure,rw,async,no_root_squash)

启动完成，一切顺利，

[root@k8s-master registry]# kubectl get pods
NAME                       READY     STATUS    RESTARTS   AGE
helloworld-service-62wl1   /       Running             88d
helloworld-service-8cbt2   /       Running             88d
registry-7nj8q             /       Running             1h
[root@k8s-master registry]# kubectl get services
NAME            CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
helloworldsvc   10.254.247.84   <nodes>       :/TCP   88d
kubernetes      10.254.0.1      <none>        /TCP          120d
registry        10.254.174.54   <nodes>       :/TCP   1h

=====================================================================

接下来验证。

在k8s-node-1上有一大堆的本地images

[root@k8s-node- v2]# docker images
REPOSITORY                                             TAG                      IMAGE ID            CREATED             SIZE
docker.io/registry                                     latest                   3ebefe7c539b         days ago          33.19 MB
-domain                                            v2                       326bf14bb29f         months ago        2.055 GB
oracle/coherence                                       12.2.1.0.-cacheserver   57a90e86e1d2         months ago         MB
oracle/coherence                                       12.2.1.0.-proxy         238c85d61468         months ago         MB
gcr.io/google_containers/nginx-ingress-controller      0.9.-beta.             2c3d45bb8cb9         months ago        130.6 MB
gcr.io/google_containers/k8s-dns-sidecar-amd64         1.14.                   7c4034e4ffa4         months ago        44.5 MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64        1.14.                   ca8759c215c9         months ago        52.36 MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64   1.14.                   e5c335701995         months ago        44.84 MB
-helloworld                                        v1                       351691157b77         months ago        2.064 GB

找一个小的，然后重新tag一下,这里我认为应该指向node的外部地址和端口而不是集群地址

docker tag docker.io/nginx  k8s-node-:/nginx

然后修改/etc/sysconfig/docker文件，主要是修改两行（在需要用到registry的节点上都需要修改)

ADD_REGISTRY='--add-registry k8s-node-1'

INSECURE_REGISTRY='--insecure-registry k8s-node-1:30002'

然后重启docker

service docker restart

将images push到本地的registry,然后我们查看/k8s/test目录下就有内容了.

docker push k8s-node-:/nginx

在其他机器运行docker pull,看到已经从本地拉回images.

[root@k8s-master registry]# docker pull k8s-node-:/nginx
Using default tag: latest
Trying to pull repository k8s-node-:/nginx ...
sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316: Pulling from k8s-node-:/nginx
36a46ebd5019: Pull complete
57168433389f: Pull complete
332ec8285c50: Pull complete
Digest: sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316
Status: Downloaded newer image for k8s-node-:/nginx:latest
[root@k8s-master registry]# docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
k8s-node-:/nginx   latest              46102226f2fd         months ago        109.4 MB