部署harbor以https模式和k8s对接
集群时间同步
我们在之前的kubeasz部署高可用kubernetes1.17.2 并实现traefik2.1.2部署篇已经实现了基于chrony的时间同步
[root@bs-k8s-master01 ~]# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
server 20.0.0.202 iburst
[root@bs-k8s-master01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* bs-k8s-master02 -15ms[ -17ms] +/- 21ms
[root@bs-k8s-master01 ~]# scp /etc/chrony.conf 20.0.0.207:/etc/chrony.conf
root@20.0.0.207's password:
chrony.conf % .7KB/s :
[root@bs-k8s-harbor01 ~]# systemctl restart chronyd.service
[root@bs-k8s-harbor01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 20.0.0.202 +25us[ -546us] +/- 36ms
注：时间同步的概念应该深入每一个搞IT人员的心中
部署docker
#安装服务器必备软件
# yum -y install wget vim iftop iotop net-tools nmon telnet lsof iptraf nmap httpd-tools lrzsz mlocate ntp ntpdate strace libpcap nethogs iptraf iftop nmon bridge-utils bind-utils telnet nc nfs-utils rpcbind nfs-utils dnsmasq python python-devel yum-utils device-mapper-persistent-data lvm2 tcpdump mlocate tree
#添加docker源信息
[root@bs-k8s-harbor01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#列出所有docker版本，选择和Kubernetes集群一致的docker版本[不一致能否正常，我没尝试]
[root@bs-k8s-harbor01 ~]# yum list docker-ce --showduplicates | sort -r
[root@bs-k8s-master01 ~]# docker -v
Docker version 19.03., build 633a0ea838 #kubernetes集群版本
#安装对应版本
[root@bs-k8s-harbor01 ~]# yum -y install docker-ce-19.03.-.el7
#为了保持集群环境一致，daemon.json也应该一致
[root@bs-k8s-master01 docker]# cat daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["127.0.0.1/8"],
"max-concurrent-downloads": ,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": ""
},
"data-root": "/var/lib/docker"
}
[root@bs-k8s-harbor01 ~]# mkdir /etc/docker
[root@bs-k8s-master01 docker]# scp daemon.json 20.0.0.207:/etc/docker/
root@20.0.0.207's password:
[root@bs-k8s-harbor01 docker]# systemctl restart docker && systemctl enable docker
[root@bs-k8s-harbor01 docker]# docker version
Client: Docker Engine - Community
Version: 19.03.
......
Server: Docker Engine - Community
Engine:
Version: 19.03.
部署harbor
harbor的管理是基于docker-compose的
# yum install -y docker-compose
# docker-compose version
docker-compose version 1.18., build 8dd22a9
docker-py version: 2.6.
CPython version: 3.6.
OpenSSL version: OpenSSL 1.0.2k-fips Jan
harbor的 git地址：https://github.com/goharbor/harbor
这里我使用的版本是1.8
[root@bs-k8s-harbor01 data]# pwd
/data
[root@bs-k8s-harbor01 data]# ll
总用量
-rw-r--r-- root root 5月 harbor-offline-installer-v1.8.0.tgz
root@bs-k8s-harbor01 data]# tar xf harbor-offline-installer-v1.8.0.tgz
[root@bs-k8s-harbor01 data]# mv harbor /usr/local/
[root@bs-k8s-harbor01 data]# cd /usr/local/harbor/
[root@bs-k8s-harbor01 harbor]# ls
harbor.v1.8.0.tar.gz harbor.yml install.sh LICENSE prepare
#创建证书
# mkdir /data/ca
# openssl genrsa -out /data/ca/harbor-ca.key
Generating RSA private key, bit long modulus
....................+++
..................................................................................+++
e is (0x10001)
# openssl req -x509 -new -nodes -key /data/ca/harbor-ca.key -subj "/CN=harbor.linux.com" -days -out /data/ca/harbor-ca.crt
#修改配置文件
# cp harbor.yml{,.bak}
# vim harbor.yml
# diff harbor.yml{,.bak}
5c5
5c5
< hostname: harbor.linux.com
---
> hostname: reg.mydomain.com
8c8
< #http:
---
> http:
10c10
< # port:
---
> port:
13c13
< https:
---
> # https:
15c15
< port:
---
> # port:
,18c17,
< certificate: /data/ca/harbor-ca.crt
< private_key: /data/ca/harbor-ca.key
---
> # certificate: /your/certificate/path
> # private_key: /your/private/key/path
27c27
< harbor_admin_password: zisefeizhu
---
> harbor_admin_password: Harbor12345
35c35
< data_volume: /data/harbor
---
> data_volume: /data
#部署
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
# cp /data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/
# ./install.sh
# docker-compose start
Starting log ... done
Starting registry ... done
Starting registryctl ... done
Starting postgresql ... done
Starting core ... done
Starting portal ... done
Starting redis ... done
Starting jobservice ... done
Starting proxy ... done
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
287136c60b95 goharbor/nginx-photon:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) 0.0.0.0:->/tcp, 0.0.0.0:->/tcp nginx
66a07d42818c goharbor/harbor-jobservice:v1.8.0 "/harbor/start.sh" seconds ago Up seconds harbor-jobservice
e4bb415fd236 goharbor/harbor-portal:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) /tcp harbor-portal
1530c4b4c604 goharbor/harbor-core:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) harbor-core
adc160874fef goharbor/redis-photon:v1.8.0 "docker-entrypoint.s…" seconds ago Up seconds /tcp redis
300165f93782 goharbor/harbor-db:v1.8.0 "/entrypoint.sh post…" seconds ago Up seconds (healthy) /tcp harbor-db
a81c3d53eb2e goharbor/registry-photon:v2.7.1-patch--v1.8.0 "/entrypoint.sh /etc…" seconds ago Up seconds (healthy) /tcp registry
1a7cf72c6433 goharbor/harbor-registryctl:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) registryctl
6be2b10b733d goharbor/harbor-log:v1.8.0 "/bin/sh -c /usr/loc…" seconds ago Up seconds (healthy) 127.0.0.1:->/tcp harbor-log
# ss -lntup
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN *: *:* users:(("systemd",pid=,fd=))
udp UNCONN *: *:* users:(("chronyd",pid=,fd=))
udp UNCONN 127.0.0.1: *:* users:(("chronyd",pid=,fd=))
udp UNCONN ::: :::* users:(("chronyd",pid=,fd=))
tcp LISTEN *: *:* users:(("sshd",pid=,fd=))
tcp LISTEN 127.0.0.1: *:* users:(("docker-proxy",pid=,fd=))
tcp LISTEN *: *:* users:(("systemd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
tcp LISTEN ::: :::* users:(("sshd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
# docker login harbor.linux.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded 配置开机自启
# cat /etc/rc.d/rc.local
cd /usr/local/harbor && docker-compose start
注：登陆失败的原因可能有：1. hosts没有域名解析 2.密码错误
客户端配置
以bs-k8s-master01为例
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.
ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.
ECDSA key fingerprint is MD5:a2:3a::bc:e7:7a:f8:c3:ef:db:6c:d5:d2::e1:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.
root@20.0.0.207's password:
harbor-ca.crt % .7KB/s :
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeededxxxxxxxxxx31 # mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com2mkdir: 已创建目录 "/etc/docker/certs.d"3mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"4The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.5ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.6ECDSA key fingerprint is MD5:a2:3a:03:bc:e7:7a:f8:c3:ef:db:6c:d5:d2:34:e1:3c.7Are you sure you want to continue connecting (yes/no)? yes8Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.9root@20.0.0.207's password: 10harbor-ca.crt % .7KB/s : 11Username: admin12Password: 13WARNING! Your password will be stored unencrypted in /root/.docker/config.json.14Configure a credential helper to remove this warning. See15https://docs.docker.com/engine/reference/commandline/login/#credentials-store1617Login Succeeded18# mkdir -pv /etc/docker/certs.d/harbor.linux.com/19mkdir: 已创建目录 "/etc/docker/certs.d"20mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"21# scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/22root@20.0.0.207's password: 23harbor-ca.crt 100% 1115 690.7KB/s 00:00 24# docker login harbor.linux.com25Username: admin26Password: 27WARNING! Your password will be stored unencrypted in /root/.docker/config.json.28Configure a credential helper to remove this warning. See29https://docs.docker.com/engine/reference/commandline/login/#credentials-store3031Login Succeededshell
# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["harbor.linux.com", "20.0.0.207:443"],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
# systemctl restart docker
# docker login harbor.linux.com
# docker login 20.0.0.207:443
# cat /root/.docker/config.json
{
"auths": {
"auths": {
"20.0.0.207:443": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
},
"harbor.linux.com": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (linux)"
}
}
注：其他集群机同样操作
测试
以ceph rbd 为例
#kubernetes master节点拉取镜像上传到harbor仓库，worker节点拉取镜像
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker tag quay.io/external_storage/rbd-provisioner:latest harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker push harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-node01 ~]# docker pull harbor.linux.com/rbd/rbd-provisioner:latest
对接kubernetes
#用户名密码存放
# cat ~/.docker/config.json |base64 -w
ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
#创建secret
# cat secret-harbor.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: secret-harbor.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
name: k8s-harbor-login
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# pwd
/data/k8s/harbor
# kubectl apply -f secret-harbor.yaml
secret/k8s-harbor-login created
#部署rbd pod测试
[root@bs-k8s-master01 harbor]# cat external-storage-rbd-provisioner.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: external-storage-rbd-provisioner.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-provisioner
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["kube-dns"]
verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: rbd-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rbd-provisioner
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rbd-provisioner
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rbd-provisioner
namespace: default
spec:
replicas:
selector:
matchLabels:
app: rbd-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: rbd-provisioner
spec:
containers:
- name: rbd-provisioner
image: "harbor.linux.com/rbd/rbd-provisioner:latest"
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
value: ceph.com/rbd
imagePullSecrets:
- name: k8s-harbor-login
serviceAccount: rbd-provisioner
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
rbd: "true"
#节点打标签
[root@bs-k8s-master01 harbor]# kubectl label nodes 20.0.0.204 rbd=true
node/20.0.0.204 labeled
#删除bs-k8s-node01节点上的rbd镜像
[root@bs-k8s-master01 harbor]# kubectl apply -f external-storage-rbd-provisioner.yaml
serviceaccount/rbd-provisioner created
clusterrole.rbac.authorization.k8s.io/rbd-provisioner created
clusterrolebinding.rbac.authorization.k8s.io/rbd-provisioner created
role.rbac.authorization.k8s.io/rbd-provisioner created
rolebinding.rbac.authorization.k8s.io/rbd-provisioner created
deployment.apps/rbd-provisioner created
[root@bs-k8s-master01 harbor]# kubectl get pods -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rbd-provisioner-9cf46c856-bl454 / ContainerCreating 6s <none> 20.0.0.204 <none> <none>
rbd-provisioner-9cf46c856-bl454 / Running 37s 172.20.46.82 20.0.0.204 <none> <none>
测试完成
部署harbor以https模式和k8s对接的更多相关文章
- kubernetes实战(十三)：k8s使用helm持久化部署harbor集成openLDAP登录
1.基本概念 上节在k8s中部署了harbor和ldap,本节将部署harbor使用openLDAP验证,部署方式与之前相同,只是改了adminserver-cm.yaml的AUTH_MODE: &q ...
- 部署 harbor 私有仓库
安装下载依赖包 安装docker-compose 从 docker compose 发布页面下载最新的 docker-compose 二进制文件,本文以1.25.4为例 cd /opt/k8s/wor ...
- 群晖DS218+部署Harbor(1.10.3)
欢迎访问我的GitHub https://github.com/zq2599/blog_demos 内容:所有原创文章分类汇总及配套源码,涉及Java.Docker.Kubernetes.DevOPS ...
- Flink 集群运行原理兼部署及Yarn运行模式深入剖析
1 Flink的前世今生(生态很重要) 原文:https://blog.csdn.net/shenshouniu/article/details/84439459 很多人可能都是在 2015 年才听到 ...
- centos7部署harbor
官网 https://github.com/goharbor/harbor 1.升级系统内核 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrep ...
- CentOS部署Harbor镜像仓库
关于Harbor Harbor是用于存储和分发Docker镜像的镜像仓库服务,相比Docker Registry,Harbor在安全.标识.管理等方面做了增强,更适合企业使用: 官方网站:https: ...
- 部署http访问SVN模式出现403问题
部署http访问SVN模式到阿里云服务器 参考连接地址 https://help.aliyun.com/document_detail/52864.html 设置好账号进行访问 http://ip/s ...
- docker部署harbor私有镜像库（3）
一.harbor介绍 在实际生产运维中,往往需要把镜像发布到几十.上百台或更多的节点上.这时单台Docker主机上镜像已无法满足,项目越来越多,镜像就越来越多,都放到一台Docker主机上是不行的,我 ...
- 在nginx上部署vue项目(history模式)；
在nginx上部署vue项目(history模式): vue-router 默认是hash模式,使用url的hash来模拟一个完整的url,当url改变的时候,页面不会重新加载.但是如果我们不想has ...
随机推荐
- ASP.NET MVC4网站搭建与发布【最新】
ASP.NET MVC4网站搭建与发布 一些往事 2015年,仅仅大二的我怀着一颗创业之心,加入了常熟派英特,成为阳光职场平台的创始之一,并肩负了公司技术部的大梁,当时阳光职场正在从线下服务向互联网化 ...
- php 克隆 clone
php 克隆 clone 在实际编程过程中,我们常常要遇到这种情况:有一个对象A,在某一时刻A中已经包含了一些有效值,此时可能会需要一个和A完全相同新对象B,并且此后对B任何改动都不会影响到A中的值, ...
- Git相关命令总结
准备类型命令 生成ssh秘钥(密码可以留空): ssh-keygen 配置用户信息(安装后第一件事): # --systen代表配置系统全局,--global代表配置当前用户全局# 在当前项目中使用其 ...
- legend图例
import matplotlib.pyplot as plt import numpy as np x=np.linspace(-3,3,50) y1=x*2+1 y2=x**2 plt.plot( ...
- kNN算法 Demo
项目链接: https://github.com/WES6/kNN
- Python的Flask框架开发RESTful API
web框架选择 Django,流行但是笨重,还麻烦,人生苦短,肯定不选 web.py,轻量,但据说作者仙逝无人维护,好吧,先pass tornado,据说倡导自己造轮子,虽然是facebook开源的吧 ...
- Microsoft Translator发布粤语文本翻译
今天,Microsoft Translator发布了粤语的文本翻译,新的语言增加将继续丰富微软翻译产品的生态系统*,让更多组织和个人能够快速且高效地实现翻译应用.在中国,大有约5500万人使用粤语(语 ...
- webpack的基本配置(初识)
webpack能根据模块的依赖关系递归地构建一个依赖关系图,当中包含了应用程序所需要的所有模块,最后打包成一个或多个bundle.它有四个核心概念entry.output .loader.plugin ...
- JVM性能优化系列-(7) 深入了解性能优化
7. 深入了解性能优化 7.1 影响系统性能的方方面面 影响系统性能的因素有很多,以下列举了常见的一些系统性能优化的方向: 7.2 常用的性能评价和测试指标 响应时间 提交请求和返回该请求的响应之间使 ...
- LeetCode【面试题 16.17. 连续数列】
------------恢复内容开始------------ 题目描述 给定一个整数数组(有正数有负数),找出总和最大的连续数列,并返回总和. 示例: 输入: [-2,1,-3,4,-1,2,1,-5 ...