部署harbor以https模式和k8s对接
集群时间同步
我们在之前的kubeasz部署高可用kubernetes1.17.2 并实现traefik2.1.2部署篇已经实现了基于chrony的时间同步
[root@bs-k8s-master01 ~]# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
server 20.0.0.202 iburst
[root@bs-k8s-master01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* bs-k8s-master02 -15ms[ -17ms] +/- 21ms
[root@bs-k8s-master01 ~]# scp /etc/chrony.conf 20.0.0.207:/etc/chrony.conf
root@20.0.0.207's password:
chrony.conf % .7KB/s :
[root@bs-k8s-harbor01 ~]# systemctl restart chronyd.service
[root@bs-k8s-harbor01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 20.0.0.202 +25us[ -546us] +/- 36ms
注：时间同步的概念应该深入每一个搞IT人员的心中
部署docker
#安装服务器必备软件
# yum -y install wget vim iftop iotop net-tools nmon telnet lsof iptraf nmap httpd-tools lrzsz mlocate ntp ntpdate strace libpcap nethogs iptraf iftop nmon bridge-utils bind-utils telnet nc nfs-utils rpcbind nfs-utils dnsmasq python python-devel yum-utils device-mapper-persistent-data lvm2 tcpdump mlocate tree
#添加docker源信息
[root@bs-k8s-harbor01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#列出所有docker版本，选择和Kubernetes集群一致的docker版本[不一致能否正常，我没尝试]
[root@bs-k8s-harbor01 ~]# yum list docker-ce --showduplicates | sort -r
[root@bs-k8s-master01 ~]# docker -v
Docker version 19.03., build 633a0ea838 #kubernetes集群版本
#安装对应版本
[root@bs-k8s-harbor01 ~]# yum -y install docker-ce-19.03.-.el7
#为了保持集群环境一致，daemon.json也应该一致
[root@bs-k8s-master01 docker]# cat daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["127.0.0.1/8"],
"max-concurrent-downloads": ,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": ""
},
"data-root": "/var/lib/docker"
}
[root@bs-k8s-harbor01 ~]# mkdir /etc/docker
[root@bs-k8s-master01 docker]# scp daemon.json 20.0.0.207:/etc/docker/
root@20.0.0.207's password:
[root@bs-k8s-harbor01 docker]# systemctl restart docker && systemctl enable docker
[root@bs-k8s-harbor01 docker]# docker version
Client: Docker Engine - Community
Version: 19.03.
......
Server: Docker Engine - Community
Engine:
Version: 19.03.
部署harbor
harbor的管理是基于docker-compose的
# yum install -y docker-compose
# docker-compose version
docker-compose version 1.18., build 8dd22a9
docker-py version: 2.6.
CPython version: 3.6.
OpenSSL version: OpenSSL 1.0.2k-fips Jan
harbor的 git地址：https://github.com/goharbor/harbor
这里我使用的版本是1.8
[root@bs-k8s-harbor01 data]# pwd
/data
[root@bs-k8s-harbor01 data]# ll
总用量
-rw-r--r-- root root 5月 harbor-offline-installer-v1.8.0.tgz
root@bs-k8s-harbor01 data]# tar xf harbor-offline-installer-v1.8.0.tgz
[root@bs-k8s-harbor01 data]# mv harbor /usr/local/
[root@bs-k8s-harbor01 data]# cd /usr/local/harbor/
[root@bs-k8s-harbor01 harbor]# ls
harbor.v1.8.0.tar.gz harbor.yml install.sh LICENSE prepare
#创建证书
# mkdir /data/ca
# openssl genrsa -out /data/ca/harbor-ca.key
Generating RSA private key, bit long modulus
....................+++
..................................................................................+++
e is (0x10001)
# openssl req -x509 -new -nodes -key /data/ca/harbor-ca.key -subj "/CN=harbor.linux.com" -days -out /data/ca/harbor-ca.crt
#修改配置文件
# cp harbor.yml{,.bak}
# vim harbor.yml
# diff harbor.yml{,.bak}
5c5
5c5
< hostname: harbor.linux.com
---
> hostname: reg.mydomain.com
8c8
< #http:
---
> http:
10c10
< # port:
---
> port:
13c13
< https:
---
> # https:
15c15
< port:
---
> # port:
,18c17,
< certificate: /data/ca/harbor-ca.crt
< private_key: /data/ca/harbor-ca.key
---
> # certificate: /your/certificate/path
> # private_key: /your/private/key/path
27c27
< harbor_admin_password: zisefeizhu
---
> harbor_admin_password: Harbor12345
35c35
< data_volume: /data/harbor
---
> data_volume: /data
#部署
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
# cp /data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/
# ./install.sh
# docker-compose start
Starting log ... done
Starting registry ... done
Starting registryctl ... done
Starting postgresql ... done
Starting core ... done
Starting portal ... done
Starting redis ... done
Starting jobservice ... done
Starting proxy ... done
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
287136c60b95 goharbor/nginx-photon:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) 0.0.0.0:->/tcp, 0.0.0.0:->/tcp nginx
66a07d42818c goharbor/harbor-jobservice:v1.8.0 "/harbor/start.sh" seconds ago Up seconds harbor-jobservice
e4bb415fd236 goharbor/harbor-portal:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) /tcp harbor-portal
1530c4b4c604 goharbor/harbor-core:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) harbor-core
adc160874fef goharbor/redis-photon:v1.8.0 "docker-entrypoint.s…" seconds ago Up seconds /tcp redis
300165f93782 goharbor/harbor-db:v1.8.0 "/entrypoint.sh post…" seconds ago Up seconds (healthy) /tcp harbor-db
a81c3d53eb2e goharbor/registry-photon:v2.7.1-patch--v1.8.0 "/entrypoint.sh /etc…" seconds ago Up seconds (healthy) /tcp registry
1a7cf72c6433 goharbor/harbor-registryctl:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) registryctl
6be2b10b733d goharbor/harbor-log:v1.8.0 "/bin/sh -c /usr/loc…" seconds ago Up seconds (healthy) 127.0.0.1:->/tcp harbor-log
# ss -lntup
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN *: *:* users:(("systemd",pid=,fd=))
udp UNCONN *: *:* users:(("chronyd",pid=,fd=))
udp UNCONN 127.0.0.1: *:* users:(("chronyd",pid=,fd=))
udp UNCONN ::: :::* users:(("chronyd",pid=,fd=))
tcp LISTEN *: *:* users:(("sshd",pid=,fd=))
tcp LISTEN 127.0.0.1: *:* users:(("docker-proxy",pid=,fd=))
tcp LISTEN *: *:* users:(("systemd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
tcp LISTEN ::: :::* users:(("sshd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
# docker login harbor.linux.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded 配置开机自启
# cat /etc/rc.d/rc.local
cd /usr/local/harbor && docker-compose start
注：登陆失败的原因可能有：1. hosts没有域名解析 2.密码错误
客户端配置
以bs-k8s-master01为例
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.
ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.
ECDSA key fingerprint is MD5:a2:3a::bc:e7:7a:f8:c3:ef:db:6c:d5:d2::e1:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.
root@20.0.0.207's password:
harbor-ca.crt % .7KB/s :
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeededxxxxxxxxxx31 # mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com2mkdir: 已创建目录 "/etc/docker/certs.d"3mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"4The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.5ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.6ECDSA key fingerprint is MD5:a2:3a:03:bc:e7:7a:f8:c3:ef:db:6c:d5:d2:34:e1:3c.7Are you sure you want to continue connecting (yes/no)? yes8Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.9root@20.0.0.207's password: 10harbor-ca.crt % .7KB/s : 11Username: admin12Password: 13WARNING! Your password will be stored unencrypted in /root/.docker/config.json.14Configure a credential helper to remove this warning. See15https://docs.docker.com/engine/reference/commandline/login/#credentials-store1617Login Succeeded18# mkdir -pv /etc/docker/certs.d/harbor.linux.com/19mkdir: 已创建目录 "/etc/docker/certs.d"20mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"21# scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/22root@20.0.0.207's password: 23harbor-ca.crt 100% 1115 690.7KB/s 00:00 24# docker login harbor.linux.com25Username: admin26Password: 27WARNING! Your password will be stored unencrypted in /root/.docker/config.json.28Configure a credential helper to remove this warning. See29https://docs.docker.com/engine/reference/commandline/login/#credentials-store3031Login Succeededshell
# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["harbor.linux.com", "20.0.0.207:443"],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
# systemctl restart docker
# docker login harbor.linux.com
# docker login 20.0.0.207:443
# cat /root/.docker/config.json
{
"auths": {
"auths": {
"20.0.0.207:443": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
},
"harbor.linux.com": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (linux)"
}
}
注：其他集群机同样操作
测试
以ceph rbd 为例
#kubernetes master节点拉取镜像上传到harbor仓库，worker节点拉取镜像
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker tag quay.io/external_storage/rbd-provisioner:latest harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker push harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-node01 ~]# docker pull harbor.linux.com/rbd/rbd-provisioner:latest
对接kubernetes
#用户名密码存放
# cat ~/.docker/config.json |base64 -w
ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
#创建secret
# cat secret-harbor.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: secret-harbor.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
name: k8s-harbor-login
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# pwd
/data/k8s/harbor
# kubectl apply -f secret-harbor.yaml
secret/k8s-harbor-login created
#部署rbd pod测试
[root@bs-k8s-master01 harbor]# cat external-storage-rbd-provisioner.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: external-storage-rbd-provisioner.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-provisioner
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["kube-dns"]
verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: rbd-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rbd-provisioner
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rbd-provisioner
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rbd-provisioner
namespace: default
spec:
replicas:
selector:
matchLabels:
app: rbd-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: rbd-provisioner
spec:
containers:
- name: rbd-provisioner
image: "harbor.linux.com/rbd/rbd-provisioner:latest"
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
value: ceph.com/rbd
imagePullSecrets:
- name: k8s-harbor-login
serviceAccount: rbd-provisioner
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
rbd: "true"
#节点打标签
[root@bs-k8s-master01 harbor]# kubectl label nodes 20.0.0.204 rbd=true
node/20.0.0.204 labeled
#删除bs-k8s-node01节点上的rbd镜像
[root@bs-k8s-master01 harbor]# kubectl apply -f external-storage-rbd-provisioner.yaml
serviceaccount/rbd-provisioner created
clusterrole.rbac.authorization.k8s.io/rbd-provisioner created
clusterrolebinding.rbac.authorization.k8s.io/rbd-provisioner created
role.rbac.authorization.k8s.io/rbd-provisioner created
rolebinding.rbac.authorization.k8s.io/rbd-provisioner created
deployment.apps/rbd-provisioner created
[root@bs-k8s-master01 harbor]# kubectl get pods -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rbd-provisioner-9cf46c856-bl454 / ContainerCreating 6s <none> 20.0.0.204 <none> <none>
rbd-provisioner-9cf46c856-bl454 / Running 37s 172.20.46.82 20.0.0.204 <none> <none>
测试完成
部署harbor以https模式和k8s对接的更多相关文章
- kubernetes实战(十三)：k8s使用helm持久化部署harbor集成openLDAP登录
1.基本概念 上节在k8s中部署了harbor和ldap,本节将部署harbor使用openLDAP验证,部署方式与之前相同,只是改了adminserver-cm.yaml的AUTH_MODE: &q ...
- 部署 harbor 私有仓库
安装下载依赖包 安装docker-compose 从 docker compose 发布页面下载最新的 docker-compose 二进制文件,本文以1.25.4为例 cd /opt/k8s/wor ...
- 群晖DS218+部署Harbor(1.10.3)
欢迎访问我的GitHub https://github.com/zq2599/blog_demos 内容:所有原创文章分类汇总及配套源码,涉及Java.Docker.Kubernetes.DevOPS ...
- Flink 集群运行原理兼部署及Yarn运行模式深入剖析
1 Flink的前世今生(生态很重要) 原文:https://blog.csdn.net/shenshouniu/article/details/84439459 很多人可能都是在 2015 年才听到 ...
- centos7部署harbor
官网 https://github.com/goharbor/harbor 1.升级系统内核 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrep ...
- CentOS部署Harbor镜像仓库
关于Harbor Harbor是用于存储和分发Docker镜像的镜像仓库服务,相比Docker Registry,Harbor在安全.标识.管理等方面做了增强,更适合企业使用: 官方网站:https: ...
- 部署http访问SVN模式出现403问题
部署http访问SVN模式到阿里云服务器 参考连接地址 https://help.aliyun.com/document_detail/52864.html 设置好账号进行访问 http://ip/s ...
- docker部署harbor私有镜像库（3）
一.harbor介绍 在实际生产运维中,往往需要把镜像发布到几十.上百台或更多的节点上.这时单台Docker主机上镜像已无法满足,项目越来越多,镜像就越来越多,都放到一台Docker主机上是不行的,我 ...
- 在nginx上部署vue项目(history模式)；
在nginx上部署vue项目(history模式): vue-router 默认是hash模式,使用url的hash来模拟一个完整的url,当url改变的时候,页面不会重新加载.但是如果我们不想has ...
随机推荐
- Luogu1681_ 最大正方形II
题目背景 忙完了学校的事,v神终于可以做他的"正事":陪女朋友散步.一天,他和女朋友走着走着,不知不觉就来到了一个千里无烟的地方.v神正要往回走,如发现了一块牌子,牌子上有有一行小 ...
- 疯狂补贴的4G+ 会是又一个资费陷阱吗？
会是又一个资费陷阱吗?" title="疯狂补贴的4G+ 会是又一个资费陷阱吗?"> 常言说得好,防火防盗防运营商--具有垄断性质的中国移动.联通.电信三大基础 ...
- C++走向远洋——46（教师兼干部类、多重继承、派生）
*/ * Copyright (c) 2016,烟台大学计算机与控制工程学院 * All rights reserved. * 文件名:text.cpp * 作者:常轩 * 微信公众号:Worldhe ...
- NumPy的随机函数子库——numpy.random
NumPy的随机函数子库numpy.random 导入模块:import numpy as np 1.numpy.random.rand(d0,d1,...,dn) 生成一个shape为(d0,d1, ...
- Intellij IDEA 干货分享
更多视频详情:https://www.bilibili.com/video/av89385013/ Intellij IDEA 真是越用越强大 它总是在我们写代码的时候 不时给我们来个小惊喜 出于对 ...
- python基础-基本概念
python概念介绍 python是一门动态解释型的强类型定义语言,创始人吉多·范罗苏姆(Guido van Rossum) #编译型语言 编译型:一次性将所有程序编译成二进制文件 缺点:开发效率低, ...
- Ubuntu 16.04.5部署Django环境
1.安装python环境 使用如下命令安装的是3.5.x版本 sudo apt-get install python3-pip 如果使用下面的命令,会安装2.x版本的python sudo apt-g ...
- Spring的工作原理
一.什么是Spring (1).Spring真正的精华是它的Ioc模式实现的BeanFactory和AOP,它自己在这个基础上延伸的功能有些画蛇添足. (2). Spring它是一个开源的项目,而且目 ...
- 微信小程序状态管理工具 JStore
微信小程序状态管理工具 JStore 闲着没事做,就想着给微信小程序写一个状态管理工具,名叫 JStore,这个状态管理工具是仿照 vuex 的几个方法来写的,所以有 vuex 的基础同学很容易理解. ...
- 伪元素 before 和 after 初探
伪元素 before 和 after 初探 使用了 CodePen 做演示,欢迎点击预览 定义 首先来看 MDN 的定义: ::before 创建一个伪元素,作为已选中元素的第一个子元素,常通过 co ...