部署harbor以https模式和k8s对接
集群时间同步
我们在之前的kubeasz部署高可用kubernetes1.17.2 并实现traefik2.1.2部署篇已经实现了基于chrony的时间同步
[root@bs-k8s-master01 ~]# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
server 20.0.0.202 iburst
[root@bs-k8s-master01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* bs-k8s-master02 -15ms[ -17ms] +/- 21ms
[root@bs-k8s-master01 ~]# scp /etc/chrony.conf 20.0.0.207:/etc/chrony.conf
root@20.0.0.207's password:
chrony.conf % .7KB/s :
[root@bs-k8s-harbor01 ~]# systemctl restart chronyd.service
[root@bs-k8s-harbor01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 20.0.0.202 +25us[ -546us] +/- 36ms
注：时间同步的概念应该深入每一个搞IT人员的心中
部署docker
#安装服务器必备软件
# yum -y install wget vim iftop iotop net-tools nmon telnet lsof iptraf nmap httpd-tools lrzsz mlocate ntp ntpdate strace libpcap nethogs iptraf iftop nmon bridge-utils bind-utils telnet nc nfs-utils rpcbind nfs-utils dnsmasq python python-devel yum-utils device-mapper-persistent-data lvm2 tcpdump mlocate tree
#添加docker源信息
[root@bs-k8s-harbor01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#列出所有docker版本，选择和Kubernetes集群一致的docker版本[不一致能否正常，我没尝试]
[root@bs-k8s-harbor01 ~]# yum list docker-ce --showduplicates | sort -r
[root@bs-k8s-master01 ~]# docker -v
Docker version 19.03., build 633a0ea838 #kubernetes集群版本
#安装对应版本
[root@bs-k8s-harbor01 ~]# yum -y install docker-ce-19.03.-.el7
#为了保持集群环境一致，daemon.json也应该一致
[root@bs-k8s-master01 docker]# cat daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["127.0.0.1/8"],
"max-concurrent-downloads": ,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": ""
},
"data-root": "/var/lib/docker"
}
[root@bs-k8s-harbor01 ~]# mkdir /etc/docker
[root@bs-k8s-master01 docker]# scp daemon.json 20.0.0.207:/etc/docker/
root@20.0.0.207's password:
[root@bs-k8s-harbor01 docker]# systemctl restart docker && systemctl enable docker
[root@bs-k8s-harbor01 docker]# docker version
Client: Docker Engine - Community
Version: 19.03.
......
Server: Docker Engine - Community
Engine:
Version: 19.03.
部署harbor
harbor的管理是基于docker-compose的
# yum install -y docker-compose
# docker-compose version
docker-compose version 1.18., build 8dd22a9
docker-py version: 2.6.
CPython version: 3.6.
OpenSSL version: OpenSSL 1.0.2k-fips Jan
harbor的 git地址：https://github.com/goharbor/harbor
这里我使用的版本是1.8
[root@bs-k8s-harbor01 data]# pwd
/data
[root@bs-k8s-harbor01 data]# ll
总用量
-rw-r--r-- root root 5月 harbor-offline-installer-v1.8.0.tgz
root@bs-k8s-harbor01 data]# tar xf harbor-offline-installer-v1.8.0.tgz
[root@bs-k8s-harbor01 data]# mv harbor /usr/local/
[root@bs-k8s-harbor01 data]# cd /usr/local/harbor/
[root@bs-k8s-harbor01 harbor]# ls
harbor.v1.8.0.tar.gz harbor.yml install.sh LICENSE prepare
#创建证书
# mkdir /data/ca
# openssl genrsa -out /data/ca/harbor-ca.key
Generating RSA private key, bit long modulus
....................+++
..................................................................................+++
e is (0x10001)
# openssl req -x509 -new -nodes -key /data/ca/harbor-ca.key -subj "/CN=harbor.linux.com" -days -out /data/ca/harbor-ca.crt
#修改配置文件
# cp harbor.yml{,.bak}
# vim harbor.yml
# diff harbor.yml{,.bak}
5c5
5c5
< hostname: harbor.linux.com
---
> hostname: reg.mydomain.com
8c8
< #http:
---
> http:
10c10
< # port:
---
> port:
13c13
< https:
---
> # https:
15c15
< port:
---
> # port:
,18c17,
< certificate: /data/ca/harbor-ca.crt
< private_key: /data/ca/harbor-ca.key
---
> # certificate: /your/certificate/path
> # private_key: /your/private/key/path
27c27
< harbor_admin_password: zisefeizhu
---
> harbor_admin_password: Harbor12345
35c35
< data_volume: /data/harbor
---
> data_volume: /data
#部署
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
# cp /data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/
# ./install.sh
# docker-compose start
Starting log ... done
Starting registry ... done
Starting registryctl ... done
Starting postgresql ... done
Starting core ... done
Starting portal ... done
Starting redis ... done
Starting jobservice ... done
Starting proxy ... done
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
287136c60b95 goharbor/nginx-photon:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) 0.0.0.0:->/tcp, 0.0.0.0:->/tcp nginx
66a07d42818c goharbor/harbor-jobservice:v1.8.0 "/harbor/start.sh" seconds ago Up seconds harbor-jobservice
e4bb415fd236 goharbor/harbor-portal:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) /tcp harbor-portal
1530c4b4c604 goharbor/harbor-core:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) harbor-core
adc160874fef goharbor/redis-photon:v1.8.0 "docker-entrypoint.s…" seconds ago Up seconds /tcp redis
300165f93782 goharbor/harbor-db:v1.8.0 "/entrypoint.sh post…" seconds ago Up seconds (healthy) /tcp harbor-db
a81c3d53eb2e goharbor/registry-photon:v2.7.1-patch--v1.8.0 "/entrypoint.sh /etc…" seconds ago Up seconds (healthy) /tcp registry
1a7cf72c6433 goharbor/harbor-registryctl:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) registryctl
6be2b10b733d goharbor/harbor-log:v1.8.0 "/bin/sh -c /usr/loc…" seconds ago Up seconds (healthy) 127.0.0.1:->/tcp harbor-log
# ss -lntup
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN *: *:* users:(("systemd",pid=,fd=))
udp UNCONN *: *:* users:(("chronyd",pid=,fd=))
udp UNCONN 127.0.0.1: *:* users:(("chronyd",pid=,fd=))
udp UNCONN ::: :::* users:(("chronyd",pid=,fd=))
tcp LISTEN *: *:* users:(("sshd",pid=,fd=))
tcp LISTEN 127.0.0.1: *:* users:(("docker-proxy",pid=,fd=))
tcp LISTEN *: *:* users:(("systemd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
tcp LISTEN ::: :::* users:(("sshd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
# docker login harbor.linux.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded 配置开机自启
# cat /etc/rc.d/rc.local
cd /usr/local/harbor && docker-compose start
注：登陆失败的原因可能有：1. hosts没有域名解析 2.密码错误
客户端配置
以bs-k8s-master01为例
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.
ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.
ECDSA key fingerprint is MD5:a2:3a::bc:e7:7a:f8:c3:ef:db:6c:d5:d2::e1:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.
root@20.0.0.207's password:
harbor-ca.crt % .7KB/s :
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeededxxxxxxxxxx31 # mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com2mkdir: 已创建目录 "/etc/docker/certs.d"3mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"4The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.5ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.6ECDSA key fingerprint is MD5:a2:3a:03:bc:e7:7a:f8:c3:ef:db:6c:d5:d2:34:e1:3c.7Are you sure you want to continue connecting (yes/no)? yes8Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.9root@20.0.0.207's password: 10harbor-ca.crt % .7KB/s : 11Username: admin12Password: 13WARNING! Your password will be stored unencrypted in /root/.docker/config.json.14Configure a credential helper to remove this warning. See15https://docs.docker.com/engine/reference/commandline/login/#credentials-store1617Login Succeeded18# mkdir -pv /etc/docker/certs.d/harbor.linux.com/19mkdir: 已创建目录 "/etc/docker/certs.d"20mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"21# scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/22root@20.0.0.207's password: 23harbor-ca.crt 100% 1115 690.7KB/s 00:00 24# docker login harbor.linux.com25Username: admin26Password: 27WARNING! Your password will be stored unencrypted in /root/.docker/config.json.28Configure a credential helper to remove this warning. See29https://docs.docker.com/engine/reference/commandline/login/#credentials-store3031Login Succeededshell
# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["harbor.linux.com", "20.0.0.207:443"],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
# systemctl restart docker
# docker login harbor.linux.com
# docker login 20.0.0.207:443
# cat /root/.docker/config.json
{
"auths": {
"auths": {
"20.0.0.207:443": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
},
"harbor.linux.com": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (linux)"
}
}
注：其他集群机同样操作
测试
以ceph rbd 为例
#kubernetes master节点拉取镜像上传到harbor仓库，worker节点拉取镜像
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker tag quay.io/external_storage/rbd-provisioner:latest harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker push harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-node01 ~]# docker pull harbor.linux.com/rbd/rbd-provisioner:latest
对接kubernetes
#用户名密码存放
# cat ~/.docker/config.json |base64 -w
ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
#创建secret
# cat secret-harbor.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: secret-harbor.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
name: k8s-harbor-login
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# pwd
/data/k8s/harbor
# kubectl apply -f secret-harbor.yaml
secret/k8s-harbor-login created
#部署rbd pod测试
[root@bs-k8s-master01 harbor]# cat external-storage-rbd-provisioner.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: external-storage-rbd-provisioner.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-provisioner
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["kube-dns"]
verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: rbd-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rbd-provisioner
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rbd-provisioner
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rbd-provisioner
namespace: default
spec:
replicas:
selector:
matchLabels:
app: rbd-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: rbd-provisioner
spec:
containers:
- name: rbd-provisioner
image: "harbor.linux.com/rbd/rbd-provisioner:latest"
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
value: ceph.com/rbd
imagePullSecrets:
- name: k8s-harbor-login
serviceAccount: rbd-provisioner
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
rbd: "true"
#节点打标签
[root@bs-k8s-master01 harbor]# kubectl label nodes 20.0.0.204 rbd=true
node/20.0.0.204 labeled
#删除bs-k8s-node01节点上的rbd镜像
[root@bs-k8s-master01 harbor]# kubectl apply -f external-storage-rbd-provisioner.yaml
serviceaccount/rbd-provisioner created
clusterrole.rbac.authorization.k8s.io/rbd-provisioner created
clusterrolebinding.rbac.authorization.k8s.io/rbd-provisioner created
role.rbac.authorization.k8s.io/rbd-provisioner created
rolebinding.rbac.authorization.k8s.io/rbd-provisioner created
deployment.apps/rbd-provisioner created
[root@bs-k8s-master01 harbor]# kubectl get pods -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rbd-provisioner-9cf46c856-bl454 / ContainerCreating 6s <none> 20.0.0.204 <none> <none>
rbd-provisioner-9cf46c856-bl454 / Running 37s 172.20.46.82 20.0.0.204 <none> <none>
测试完成
部署harbor以https模式和k8s对接的更多相关文章
- kubernetes实战(十三)：k8s使用helm持久化部署harbor集成openLDAP登录
1.基本概念 上节在k8s中部署了harbor和ldap,本节将部署harbor使用openLDAP验证,部署方式与之前相同,只是改了adminserver-cm.yaml的AUTH_MODE: &q ...
- 部署 harbor 私有仓库
安装下载依赖包 安装docker-compose 从 docker compose 发布页面下载最新的 docker-compose 二进制文件,本文以1.25.4为例 cd /opt/k8s/wor ...
- 群晖DS218+部署Harbor(1.10.3)
欢迎访问我的GitHub https://github.com/zq2599/blog_demos 内容:所有原创文章分类汇总及配套源码,涉及Java.Docker.Kubernetes.DevOPS ...
- Flink 集群运行原理兼部署及Yarn运行模式深入剖析
1 Flink的前世今生(生态很重要) 原文:https://blog.csdn.net/shenshouniu/article/details/84439459 很多人可能都是在 2015 年才听到 ...
- centos7部署harbor
官网 https://github.com/goharbor/harbor 1.升级系统内核 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrep ...
- CentOS部署Harbor镜像仓库
关于Harbor Harbor是用于存储和分发Docker镜像的镜像仓库服务,相比Docker Registry,Harbor在安全.标识.管理等方面做了增强,更适合企业使用: 官方网站:https: ...
- 部署http访问SVN模式出现403问题
部署http访问SVN模式到阿里云服务器 参考连接地址 https://help.aliyun.com/document_detail/52864.html 设置好账号进行访问 http://ip/s ...
- docker部署harbor私有镜像库（3）
一.harbor介绍 在实际生产运维中,往往需要把镜像发布到几十.上百台或更多的节点上.这时单台Docker主机上镜像已无法满足,项目越来越多,镜像就越来越多,都放到一台Docker主机上是不行的,我 ...
- 在nginx上部署vue项目(history模式)；
在nginx上部署vue项目(history模式): vue-router 默认是hash模式,使用url的hash来模拟一个完整的url,当url改变的时候,页面不会重新加载.但是如果我们不想has ...
随机推荐
- <USACO07JAN>解决问题Problem Solvingの思路
日常为dp贡献脑细胞 #include<iostream> #include<cmath> #include<cstdio> #include<cstdlib ...
- PyCharm 介绍、安装、入门使用
一.Pycharm介绍 前面几年的时间,我一直用的eclipse,后面开始听同事说用IntelliJ IDEA了,而且说是目前业界最好用的java开发工具,IDE(集成开发环境),没有之一.PyCha ...
- Annotation标注
# View more python tutorials on my Youtube and Youku channel!!! # Youtube video tutorial: https://ww ...
- drbd配置
DRBD就是网络RAID1,可以通过网络同步两个主机上的硬盘,drbd每次只允许对一个节点进行读写访问. 一.安装DRBD CentOS 6.x rpm -ivh http://www.elrepo. ...
- 从wordpress换hexo博客后
之前用wordpress做blog, 为什么换为hexo呢? 第一 wordpress的文章都保存在服务器的数据库, 维护不是很直观. 而hexo是自己编写markdown文章,本地一份,而b ...
- SpringBoot入门系列（一）如何快速创建SpringBoot项目
这段时间也没什么事情,所以就重新学习整理了Spring Boot的相关内容.今天开始整理更新Spring Boot学习笔记,感兴趣的朋友可以关注我的博客:https://www.cnblogs.com ...
- 疫情期，如何用A/B测试快速迭代你的产品？
作者:友盟+数据科学家 杨玉莲.陆子骏 冠状病毒来袭牵动着每个人的心,但是病毒影响的不仅仅是我们的健康,也以极快的速度极深远地影响了整个移动互联网的发展.主流阵地原本在线下的需求,如医疗和生鲜电商,快 ...
- Dart 运行速度测评与比较
引言 Dart 是一门优秀的跨平台语言,尽管生态方面略有欠缺,但无疑作为一门编程语言来说,Dart 是很优美,很健壮的,同时也引入了一些先进的编程范式,值得去学习. 测试内容 现在,我们就来测评一下D ...
- CF 1305E. Kuroni and the Score Distribution
题目大意:题目给定两个数n和m(1<=n<=5000,0<=m<=1e9)要求构造一个数列A,A中元素 大于等于1,小于等于1e9且满足严格递增 满足ai+aj=ak的(i,j ...
- 关于CSS设置页面背景图的一些疑问
关于背景图片的位置其background-position设置背景图片的位置有两种方式,一种是是根据像素设置,第二种根据百分比设置,第一种根据像素的位置是很简单的,只是关于百分比这个设置理解特别容易出 ...