Harbor作为Docker的镜像中心

转载于网络

我们采用Harbor作为Docker的镜像中心。
有几个原因：

• Harbor采用Docker Compose拉起维护，简单方便。
• 采用Nginx作为入口网关，各种参数配置相对熟悉。
• 基于Nginx的HTTPS证书配置相对方便。
• Harbor已支持在线清理废弃的镜像历史，这点很重要。
  ...

一句话，够简单，够方便。

环境准备

Host List

IP Address	Hosts	Disk	Comment
192.168.0.21	harbor	1TB	Docker Image Registry

OS

并将内核升级到最新稳定版本4.20.

[root@localhost ~]# uname -sr
Linux 4.20.0-1.el7.elrepo.x86_64
[root@localhost ~]#
[root@localhost ~]#

安装步骤

下载harbor安装包

Harbor提供两种安装方式：在线安装和离线安装，由于GitHub服务器是在国外，国内的很多服务器都是在内网，即使可以访问公网，下载速度也不快，推荐外部下载，然后上传到内网。
本人的服务器速度还可以，直接通过服务器下载，并解压

[root@localhost harbor]# wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.1.tgz
--2019-01-07 15:16:27--  https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.1.tgz
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.160.112, 2404:6800:4012:1::2010
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.160.112|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 597857483 (570M) [application/x-tar]
Saving to: ‘harbor-offline-installer-v1.7.1.tgz.1’

100%[====================================================================>] 597,857,483 4.64MB/s   in 5m 23s 

2019-01-07 15:21:51 (1.77 MB/s) - ‘harbor-offline-installer-v1.7.1.tgz.1’ saved [597857483/597857483]

[root@localhost harbor]#
[root@localhost harbor]# tar -zxvf  harbor-offline-installer-v1.7.1.tgz

准备SSL证书

参考Docker的安全策略推荐，我们对我们的Docker镜像中心采用TLS证书验证的HTTPS访问方式。

准备证书目录

[root@localhost harbor]# mkdir -p data/cert
[root@localhost harbor]# cd data/cert
[root@localhost cert]# pwd
/home/harbor/data/cert

生成证书

生成根证书

生成CA证书。

[root@localhost cert]# openssl genrsa -out ca.key 4096
Generating RSA private key, 4096 bit long modulus
......................++
..................................++
e is 65537 (0x10001)
[root@localhost cert]#

生成CA Key。

[root@localhost cert]# openssl req -x509 -new -nodes -sha512 -days 3650 \
>   -subj "/C=CN/ST=Shanghai/L=Shanghai/O=example/OU=Personal/CN=hub.twikle.net" \
>   -key ca.key \
>   -out ca.crt
[root@localhost cert]#
[root@localhost cert]# ls -al
total 8
drwxr-xr-x 2 root root   44 Jan  7 15:35 .
drwxr-xr-x 3 root root   17 Jan  7 15:26 ..
-rw-r--r-- 1 root root 2041 Jan  7 15:35 ca.crt
-rw-r--r-- 1 root root 3247 Jan  7 15:33 ca.key
[root@localhost cert]#

生成服务器证书

生成私有Key

[root@localhost cert]# openssl genrsa -out hub.twikle.net.key 4096
Generating RSA private key, 4096 bit long modulus
............................................++
................................................++
e is 65537 (0x10001)
[root@localhost cert]# ls -al
total 12
drwxr-xr-x 2 root root   73 Jan  7 15:40 .
drwxr-xr-x 3 root root   17 Jan  7 15:26 ..
-rw-r--r-- 1 root root 2041 Jan  7 15:35 ca.crt
-rw-r--r-- 1 root root 3247 Jan  7 15:33 ca.key
-rw-r--r-- 1 root root 3243 Jan  7 15:40 hub.twikle.net.key
[root@localhost cert]#

生成证书的签名。

[root@localhost cert]# openssl req -sha512 -new \
>   -subj "/C=CN/ST=Shanghai/L=Shanghai/O=example/OU=Personal/CN=hub.twikle.net" \
>   -key hub.twikle.net.key \
>   -out hub.twikle.net.csr
[root@localhost cert]#
[root@localhost cert]#
[root@localhost cert]# ls -al
total 16
drwxr-xr-x 2 root root  102 Jan  7 15:43 .
drwxr-xr-x 3 root root   17 Jan  7 15:26 ..
-rw-r--r-- 1 root root 2041 Jan  7 15:35 ca.crt
-rw-r--r-- 1 root root 3247 Jan  7 15:33 ca.key
-rw-r--r-- 1 root root 1712 Jan  7 15:43 hub.twikle.net.csr
-rw-r--r-- 1 root root 3243 Jan  7 15:40 hub.twikle.net.key
[root@localhost cert]#

生成证书。

[root@localhost cert]# cat > v3.ext <<-EOF
> authorityKeyIdentifier=keyid,issuer
> basicConstraints=CA:FALSE
> keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
> extendedKeyUsage = serverAuth
> subjectAltName = @alt_names
>
> [alt_names]
> DNS.1=hub.twikle.net
> DNS.2=hub.twikle
> DNS.3=xxx.xxx.xxx.xxx #注意替换为自己的主机名
> EOF
[root@localhost cert]#
[root@localhost cert]# openssl x509 -req -sha512 -days 3650 \
>     -extfile v3.ext \
>     -CA ca.crt -CAkey ca.key -CAcreateserial \
>     -in hub.twikle.net.csr \
>     -out hub.twikle.net.crt
Signature ok
subject=/C=CN/ST=Shanghai/L=Shanghai/O=example/OU=Personal/CN=hub.twikle.net
Getting CA Private Key
[root@localhost cert]# ls -al
total 32
drwxr-xr-x 2 root root 4096 Jan  7 15:49 .
drwxr-xr-x 3 root root   17 Jan  7 15:26 ..
-rw-r--r-- 1 root root 2041 Jan  7 15:35 ca.crt
-rw-r--r-- 1 root root 3247 Jan  7 15:33 ca.key
-rw-r--r-- 1 root root   17 Jan  7 15:49 ca.srl
-rw-r--r-- 1 root root 2114 Jan  7 15:49 hub.twikle.net.crt
-rw-r--r-- 1 root root 1712 Jan  7 15:43 hub.twikle.net.csr
-rw-r--r-- 1 root root 3243 Jan  7 15:40 hub.twikle.net.key
-rw-r--r-- 1 root root  270 Jan  7 15:47 v3.ext
[root@localhost cert]#

证书格式调整。

[root@localhost cert]# openssl x509 -inform PEM -in hub.twikle.net.crt -out hub.twikle.net.cert
[root@localhost cert]# ls -al
total 36
drwxr-xr-x 2 root root 4096 Jan  7 15:51 .
drwxr-xr-x 3 root root   17 Jan  7 15:26 ..
-rw-r--r-- 1 root root 2041 Jan  7 15:35 ca.crt
-rw-r--r-- 1 root root 3247 Jan  7 15:33 ca.key
-rw-r--r-- 1 root root   17 Jan  7 15:49 ca.srl
-rw-r--r-- 1 root root 2114 Jan  7 15:51 hub.twikle.net.cert
-rw-r--r-- 1 root root 2114 Jan  7 15:49 hub.twikle.net.crt
-rw-r--r-- 1 root root 1712 Jan  7 15:43 hub.twikle.net.csr
-rw-r--r-- 1 root root 3243 Jan  7 15:40 hub.twikle.net.key
-rw-r--r-- 1 root root  270 Jan  7 15:47 v3.ext
[root@localhost cert]#

配置Harbor安装参数

修改harbor.cfg文件中的相关安装参数。在第一步中的解压目录中找到要修改的harbor.cfg。

[root@localhost harbor]# vi harbor.cfg
......
#set hostname
hostname = hub.twikle.net:8443
#set ui_url_protocol
ui_url_protocol = https
......
#The path of cert and key files for nginx, they are applied, pls use your own crt path here.
ssl_cert = /home/harbor/data/cert/hub.twikle.net.crt
ssl_cert_key = /home/harbor/data/cert/hub.twikle.net.key
......
#Change the admin password from UI after launching Harbor.
harbor_admin_password = xxxxx
......
#Turn on or off the self-registration feature
self_registration = off
......
#Set to "adminonly" so that only admin user can create project.
project_creation_restriction = adminonly
......
#######Harbor DB configuration section#######

#The address of the Harbor database. Only need to change when using external db.
db_host = ***.***.***.***

#The password for the root user of Harbor DB. Change this before any production use.
db_password = xxxxxx

#The port of Harbor database host
db_port = 5432

#The user name of Harbor database
db_user = harbor
......

注意，请勿修改，这个是Harbor的一个bug，修改过后，admin server会一直启动失败。

#The path of secretkey storage
secretkey_path = /data

报错：

adminserver[14789]: 2017-05-04T03:09:55Z [FATAL] [main.go:46]: failed to initialize the system: read /etc/adminserver/key: is a directory

修改默认启动端口

修改docker-compose的脚本，进入harbor的解压目录，找到docker-compose.yml，修改nginx相关的映射端口。

......
    ports:
      - 8080:80
      - 8443:443
......

修改存储路径

依旧是修改docker-compose.yml文件，替换所有的/data目录为自己的目录。或者就用默认的/data路径

......
    volumes:
      - /home/harbor/harbor/data/registry:/storage:z
      - ./common/config/registry/:/etc/registry/:z
      - ./common/config/custom-ca-bundle.crt:/harbor_cust_cert/custom-ca-bundle.crt:z
    networks:
......

执行环境准备脚本

进入harbor的解压目录。

[root@localhost harbor]# cd harbor/
[root@localhost harbor]# ls -al
total 590240
drwxr-xr-x 3 root root      4096 Jan  8 09:55 .
drwxr-xr-x 4 root root        88 Jan  7 15:26 ..
drwxr-xr-x 3 root root        30 Jan  7 15:23 common
-rw-r--r-- 1 root root       939 Jan  4 19:23 docker-compose.chartmuseum.yml
-rw-r--r-- 1 root root       975 Jan  4 19:23 docker-compose.clair.yml
-rw-r--r-- 1 root root      1434 Jan  4 19:23 docker-compose.notary.yml
-rw-r--r-- 1 root root      5608 Jan  4 19:23 docker-compose.yml
-rw-r--r-- 1 root root      8088 Jan  9 10:53 harbor.cfg
-rw-r--r-- 1 root root 603562385 Jan  4 19:24 harbor.v1.7.1.tar.gz
-rwxr-xr-x 1 root root      5739 Jan  4 19:23 install.sh
-rw-r--r-- 1 root root     11347 Jan  4 19:23 LICENSE
-rw-r--r-- 1 root root    748160 Jan  4 19:23 open_source_license
-rwxr-xr-x 1 root root     36337 Jan  4 19:23 prepare
[root@localhost harbor]# ./prepare
Generated and saved secret to file: /home/harbor/data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/core/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/config.yml
Generated configuration file: ./common/config/log/logrotate.conf
Generated configuration file: ./common/config/registryctl/env
Generated configuration file: ./common/config/core/app.conf
Generated certificate, key file: ./common/config/core/private_key.pem, cert file: ./common/config/registry/root.crt
The configuration files are ready, please use docker-compose to start the service.
[root@localhost harbor]#
[root@localhost harbor]#

启动Harbor

[root@localhost harbor]# docker-compose up -d
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating registry           ... done
Creating harbor-adminserver ... done
Creating redis              ... done
Creating registryctl        ... done
Creating harbor-db          ... done
Creating harbor-core        ... done
Creating harbor-jobservice  ... done
Creating harbor-portal      ... done
Creating nginx              ... done
[root@localhost harbor]#

宿主机防火墙开放端口

[root@localhost harbor]# firewall-cmd --zone=public --add-port=8443/tcp --permanent
success
[root@localhost harbor]# firewall-cmd --reload
success
[root@localhost harbor]#

检查安装结果

 

image.png

[root@localhost ~]# docker login xxx.xxx.xxx:8443
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@localhost ~]# 

报错：

x509: cannot validate certificate because of not containing any IP SANs  这是因为使用IP地址的原因，如使用域名做为地址应该不会

解决方法：

参考：

https://blog.csdn.net/zsd498537806/article/details/79290732

log：

harbor 运行时产生的文件、目录

harbor 将日志打印到 /var/log/harbor 的相关目录下，使用 docker logs XXX 或 docker-compose logs XXX 将看不到容器的日志。

$ # 日志目录
$ ls /var/log/harbor
adminserver.log  jobservice.log  mysql.log  proxy.log  registry.log  ui.log
$ # 数据目录，包括数据库、镜像仓库
$ ls /data/
ca_download  config  database  job_logs registry  secretkey
请参考  https://github.com/opsnull/follow-me-install-kubernetes-cluster/blob/master 的harbot11.yaml

docker 解决 x509: certificate signed by unknown authority

 

 

添加如下配置

# vim /etc/docker/daemon.json
{
  "insecure-registries": ["registry.svc.xxx.cn"]
}

本机拉本机仓库，那直接把crt证书拉本地，放

/etc/pki/ca-trust/source/anchors/

然后执行

update-ca-trust

一定要重启docker，即可。