部署harbor以https模式和k8s对接
集群时间同步
我们在之前的kubeasz部署高可用kubernetes1.17.2 并实现traefik2.1.2部署篇已经实现了基于chrony的时间同步
[root@bs-k8s-master01 ~]# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
server 20.0.0.202 iburst
[root@bs-k8s-master01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* bs-k8s-master02 -15ms[ -17ms] +/- 21ms
[root@bs-k8s-master01 ~]# scp /etc/chrony.conf 20.0.0.207:/etc/chrony.conf
root@20.0.0.207's password:
chrony.conf % .7KB/s :
[root@bs-k8s-harbor01 ~]# systemctl restart chronyd.service
[root@bs-k8s-harbor01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 20.0.0.202 +25us[ -546us] +/- 36ms
注：时间同步的概念应该深入每一个搞IT人员的心中
部署docker
#安装服务器必备软件
# yum -y install wget vim iftop iotop net-tools nmon telnet lsof iptraf nmap httpd-tools lrzsz mlocate ntp ntpdate strace libpcap nethogs iptraf iftop nmon bridge-utils bind-utils telnet nc nfs-utils rpcbind nfs-utils dnsmasq python python-devel yum-utils device-mapper-persistent-data lvm2 tcpdump mlocate tree
#添加docker源信息
[root@bs-k8s-harbor01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#列出所有docker版本，选择和Kubernetes集群一致的docker版本[不一致能否正常，我没尝试]
[root@bs-k8s-harbor01 ~]# yum list docker-ce --showduplicates | sort -r
[root@bs-k8s-master01 ~]# docker -v
Docker version 19.03., build 633a0ea838 #kubernetes集群版本
#安装对应版本
[root@bs-k8s-harbor01 ~]# yum -y install docker-ce-19.03.-.el7
#为了保持集群环境一致，daemon.json也应该一致
[root@bs-k8s-master01 docker]# cat daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["127.0.0.1/8"],
"max-concurrent-downloads": ,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": ""
},
"data-root": "/var/lib/docker"
}
[root@bs-k8s-harbor01 ~]# mkdir /etc/docker
[root@bs-k8s-master01 docker]# scp daemon.json 20.0.0.207:/etc/docker/
root@20.0.0.207's password:
[root@bs-k8s-harbor01 docker]# systemctl restart docker && systemctl enable docker
[root@bs-k8s-harbor01 docker]# docker version
Client: Docker Engine - Community
Version: 19.03.
......
Server: Docker Engine - Community
Engine:
Version: 19.03.
部署harbor
harbor的管理是基于docker-compose的
# yum install -y docker-compose
# docker-compose version
docker-compose version 1.18., build 8dd22a9
docker-py version: 2.6.
CPython version: 3.6.
OpenSSL version: OpenSSL 1.0.2k-fips Jan
harbor的 git地址：https://github.com/goharbor/harbor
这里我使用的版本是1.8
[root@bs-k8s-harbor01 data]# pwd
/data
[root@bs-k8s-harbor01 data]# ll
总用量
-rw-r--r-- root root 5月 harbor-offline-installer-v1.8.0.tgz
root@bs-k8s-harbor01 data]# tar xf harbor-offline-installer-v1.8.0.tgz
[root@bs-k8s-harbor01 data]# mv harbor /usr/local/
[root@bs-k8s-harbor01 data]# cd /usr/local/harbor/
[root@bs-k8s-harbor01 harbor]# ls
harbor.v1.8.0.tar.gz harbor.yml install.sh LICENSE prepare
#创建证书
# mkdir /data/ca
# openssl genrsa -out /data/ca/harbor-ca.key
Generating RSA private key, bit long modulus
....................+++
..................................................................................+++
e is (0x10001)
# openssl req -x509 -new -nodes -key /data/ca/harbor-ca.key -subj "/CN=harbor.linux.com" -days -out /data/ca/harbor-ca.crt
#修改配置文件
# cp harbor.yml{,.bak}
# vim harbor.yml
# diff harbor.yml{,.bak}
5c5
5c5
< hostname: harbor.linux.com
---
> hostname: reg.mydomain.com
8c8
< #http:
---
> http:
10c10
< # port:
---
> port:
13c13
< https:
---
> # https:
15c15
< port:
---
> # port:
,18c17,
< certificate: /data/ca/harbor-ca.crt
< private_key: /data/ca/harbor-ca.key
---
> # certificate: /your/certificate/path
> # private_key: /your/private/key/path
27c27
< harbor_admin_password: zisefeizhu
---
> harbor_admin_password: Harbor12345
35c35
< data_volume: /data/harbor
---
> data_volume: /data
#部署
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
# cp /data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/
# ./install.sh
# docker-compose start
Starting log ... done
Starting registry ... done
Starting registryctl ... done
Starting postgresql ... done
Starting core ... done
Starting portal ... done
Starting redis ... done
Starting jobservice ... done
Starting proxy ... done
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
287136c60b95 goharbor/nginx-photon:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) 0.0.0.0:->/tcp, 0.0.0.0:->/tcp nginx
66a07d42818c goharbor/harbor-jobservice:v1.8.0 "/harbor/start.sh" seconds ago Up seconds harbor-jobservice
e4bb415fd236 goharbor/harbor-portal:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) /tcp harbor-portal
1530c4b4c604 goharbor/harbor-core:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) harbor-core
adc160874fef goharbor/redis-photon:v1.8.0 "docker-entrypoint.s…" seconds ago Up seconds /tcp redis
300165f93782 goharbor/harbor-db:v1.8.0 "/entrypoint.sh post…" seconds ago Up seconds (healthy) /tcp harbor-db
a81c3d53eb2e goharbor/registry-photon:v2.7.1-patch--v1.8.0 "/entrypoint.sh /etc…" seconds ago Up seconds (healthy) /tcp registry
1a7cf72c6433 goharbor/harbor-registryctl:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) registryctl
6be2b10b733d goharbor/harbor-log:v1.8.0 "/bin/sh -c /usr/loc…" seconds ago Up seconds (healthy) 127.0.0.1:->/tcp harbor-log
# ss -lntup
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN *: *:* users:(("systemd",pid=,fd=))
udp UNCONN *: *:* users:(("chronyd",pid=,fd=))
udp UNCONN 127.0.0.1: *:* users:(("chronyd",pid=,fd=))
udp UNCONN ::: :::* users:(("chronyd",pid=,fd=))
tcp LISTEN *: *:* users:(("sshd",pid=,fd=))
tcp LISTEN 127.0.0.1: *:* users:(("docker-proxy",pid=,fd=))
tcp LISTEN *: *:* users:(("systemd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
tcp LISTEN ::: :::* users:(("sshd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
# docker login harbor.linux.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded 配置开机自启
# cat /etc/rc.d/rc.local
cd /usr/local/harbor && docker-compose start
注：登陆失败的原因可能有：1. hosts没有域名解析 2.密码错误
客户端配置
以bs-k8s-master01为例
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.
ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.
ECDSA key fingerprint is MD5:a2:3a::bc:e7:7a:f8:c3:ef:db:6c:d5:d2::e1:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.
root@20.0.0.207's password:
harbor-ca.crt % .7KB/s :
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeededxxxxxxxxxx31 # mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com2mkdir: 已创建目录 "/etc/docker/certs.d"3mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"4The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.5ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.6ECDSA key fingerprint is MD5:a2:3a:03:bc:e7:7a:f8:c3:ef:db:6c:d5:d2:34:e1:3c.7Are you sure you want to continue connecting (yes/no)? yes8Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.9root@20.0.0.207's password: 10harbor-ca.crt % .7KB/s : 11Username: admin12Password: 13WARNING! Your password will be stored unencrypted in /root/.docker/config.json.14Configure a credential helper to remove this warning. See15https://docs.docker.com/engine/reference/commandline/login/#credentials-store1617Login Succeeded18# mkdir -pv /etc/docker/certs.d/harbor.linux.com/19mkdir: 已创建目录 "/etc/docker/certs.d"20mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"21# scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/22root@20.0.0.207's password: 23harbor-ca.crt 100% 1115 690.7KB/s 00:00 24# docker login harbor.linux.com25Username: admin26Password: 27WARNING! Your password will be stored unencrypted in /root/.docker/config.json.28Configure a credential helper to remove this warning. See29https://docs.docker.com/engine/reference/commandline/login/#credentials-store3031Login Succeededshell
# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["harbor.linux.com", "20.0.0.207:443"],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
# systemctl restart docker
# docker login harbor.linux.com
# docker login 20.0.0.207:443
# cat /root/.docker/config.json
{
"auths": {
"auths": {
"20.0.0.207:443": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
},
"harbor.linux.com": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (linux)"
}
}
注：其他集群机同样操作
测试
以ceph rbd 为例
#kubernetes master节点拉取镜像上传到harbor仓库，worker节点拉取镜像
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker tag quay.io/external_storage/rbd-provisioner:latest harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker push harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-node01 ~]# docker pull harbor.linux.com/rbd/rbd-provisioner:latest
对接kubernetes
#用户名密码存放
# cat ~/.docker/config.json |base64 -w
ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
#创建secret
# cat secret-harbor.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: secret-harbor.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
name: k8s-harbor-login
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# pwd
/data/k8s/harbor
# kubectl apply -f secret-harbor.yaml
secret/k8s-harbor-login created
#部署rbd pod测试
[root@bs-k8s-master01 harbor]# cat external-storage-rbd-provisioner.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: external-storage-rbd-provisioner.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-provisioner
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["kube-dns"]
verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: rbd-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rbd-provisioner
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rbd-provisioner
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rbd-provisioner
namespace: default
spec:
replicas:
selector:
matchLabels:
app: rbd-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: rbd-provisioner
spec:
containers:
- name: rbd-provisioner
image: "harbor.linux.com/rbd/rbd-provisioner:latest"
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
value: ceph.com/rbd
imagePullSecrets:
- name: k8s-harbor-login
serviceAccount: rbd-provisioner
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
rbd: "true"
#节点打标签
[root@bs-k8s-master01 harbor]# kubectl label nodes 20.0.0.204 rbd=true
node/20.0.0.204 labeled
#删除bs-k8s-node01节点上的rbd镜像
[root@bs-k8s-master01 harbor]# kubectl apply -f external-storage-rbd-provisioner.yaml
serviceaccount/rbd-provisioner created
clusterrole.rbac.authorization.k8s.io/rbd-provisioner created
clusterrolebinding.rbac.authorization.k8s.io/rbd-provisioner created
role.rbac.authorization.k8s.io/rbd-provisioner created
rolebinding.rbac.authorization.k8s.io/rbd-provisioner created
deployment.apps/rbd-provisioner created
[root@bs-k8s-master01 harbor]# kubectl get pods -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rbd-provisioner-9cf46c856-bl454 / ContainerCreating 6s <none> 20.0.0.204 <none> <none>
rbd-provisioner-9cf46c856-bl454 / Running 37s 172.20.46.82 20.0.0.204 <none> <none>
测试完成
部署harbor以https模式和k8s对接的更多相关文章
- kubernetes实战(十三)：k8s使用helm持久化部署harbor集成openLDAP登录
1.基本概念 上节在k8s中部署了harbor和ldap,本节将部署harbor使用openLDAP验证,部署方式与之前相同,只是改了adminserver-cm.yaml的AUTH_MODE: &q ...
- 部署 harbor 私有仓库
安装下载依赖包 安装docker-compose 从 docker compose 发布页面下载最新的 docker-compose 二进制文件,本文以1.25.4为例 cd /opt/k8s/wor ...
- 群晖DS218+部署Harbor(1.10.3)
欢迎访问我的GitHub https://github.com/zq2599/blog_demos 内容:所有原创文章分类汇总及配套源码,涉及Java.Docker.Kubernetes.DevOPS ...
- Flink 集群运行原理兼部署及Yarn运行模式深入剖析
1 Flink的前世今生(生态很重要) 原文:https://blog.csdn.net/shenshouniu/article/details/84439459 很多人可能都是在 2015 年才听到 ...
- centos7部署harbor
官网 https://github.com/goharbor/harbor 1.升级系统内核 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrep ...
- CentOS部署Harbor镜像仓库
关于Harbor Harbor是用于存储和分发Docker镜像的镜像仓库服务,相比Docker Registry,Harbor在安全.标识.管理等方面做了增强,更适合企业使用: 官方网站:https: ...
- 部署http访问SVN模式出现403问题
部署http访问SVN模式到阿里云服务器 参考连接地址 https://help.aliyun.com/document_detail/52864.html 设置好账号进行访问 http://ip/s ...
- docker部署harbor私有镜像库（3）
一.harbor介绍 在实际生产运维中,往往需要把镜像发布到几十.上百台或更多的节点上.这时单台Docker主机上镜像已无法满足,项目越来越多,镜像就越来越多,都放到一台Docker主机上是不行的,我 ...
- 在nginx上部署vue项目(history模式)；
在nginx上部署vue项目(history模式): vue-router 默认是hash模式,使用url的hash来模拟一个完整的url,当url改变的时候,页面不会重新加载.但是如果我们不想has ...
随机推荐
- [转]<版本一>写代码的小女孩
天冷极了,下着雪,又快黑了.这是NOIP的前夜.在这又冷又黑的晚上,一个衣衫破烂的小女孩在机房敲着代码.她从班里逃出来的时候还拿着一本算导,但是有什么用呢?那是一本很破旧的书——那么大,一向是她妈妈垫 ...
- 红灯区：DevOps 建设的思考和实践
点击关注"有赞coder" 获取更多技术干货哦- 作者:费解 团队:效能改进 背景 众所周知,在丰田精益生产中,核心观念包含对人的尊重.消除浪费.持续改善,只有这样,企业才能保持良 ...
- 我们为什么不愿意相信AI？
人工智能--即AI已经变得越来越聪明,甚至能够预测未来.比如警察可以用AI来提前预判出犯罪的时间或地点,医生可以用AI预测病人最有可能心脏病发作或是中风.甚至研究人员还试图为AI添加上更多的想象力,因 ...
- ant:如何用ant将web project打包成war包
说明:src中的文件将不会呈现出来,诸位可以自己写一个简单的web项目,然后依照我所提供的ant脚本配置来设置. 文件结构如图所示: 配置为下: build.xml < ...
- PS4游戏将登陆PC：一曲属于主机的悲歌
曾经,红白机.PS游戏机等成为一代人难以磨灭的记忆.而随后的索尼PS3.微软Xbox 360.任天堂Wii U等,也称霸了次时代主机时代,成为家庭娱乐的中心.但面对着依托于PC和智能移动终端 ...
- SpringBoot入门系列（二）如何返回统一的数据格式
前面介绍了Spring Boot的优点,然后介绍了如何快速创建Spring Boot 项目.不清楚的朋友可以看看之前的文章:https://www.cnblogs.com/zhangweizhong/ ...
- c/s用户与服务器之间的传输
####第一页 <%@ page language="java" contentType="text/html; charset=UTF-8" pageE ...
- 这些Zepto中实用的方法集
前言 时间过得可真快,转眼间2017年已去大半有余,你就说吓不吓人,这一年你成长了多少,是否荒度了很多时光,亦或者天天向上,收获满满.今天主要写一些看Zepto基础模块时,比较实用的部分内部方法,在我 ...
- css手写一个表头固定
Bootstrap,layui等前端框架里面都对表头固定,表格滚动有实现,偏偏刚入职的公司选择了手动渲染表格,后期又觉得表格数据拉太长想要做表头固定.为了避免对代码改动太大,所以决定手写表头固定 主要 ...
- LeetCode 33.Search in Rotated Sorted Array（M）
题目: Suppose an array sorted in ascending order is rotated at some pivot unknown to you beforehand. ( ...