部署harbor以https模式和k8s对接
集群时间同步
我们在之前的kubeasz部署高可用kubernetes1.17.2 并实现traefik2.1.2部署篇已经实现了基于chrony的时间同步
[root@bs-k8s-master01 ~]# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
server 20.0.0.202 iburst
[root@bs-k8s-master01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* bs-k8s-master02 -15ms[ -17ms] +/- 21ms
[root@bs-k8s-master01 ~]# scp /etc/chrony.conf 20.0.0.207:/etc/chrony.conf
root@20.0.0.207's password:
chrony.conf % .7KB/s :
[root@bs-k8s-harbor01 ~]# systemctl restart chronyd.service
[root@bs-k8s-harbor01 ~]# chronyc sources -v
Number of sources =
.-- Source mode '^' = server, '=' = peer, '#' = local clock.
/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
|| .- xxxx [ yyyy ] +/- zzzz
|| Reachability register (octal) -. | xxxx = adjusted offset,
|| Log2(Polling interval) --. | | yyyy = measured offset,
|| \ | | zzzz = estimated error.
|| | | \
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 20.0.0.202 +25us[ -546us] +/- 36ms
注：时间同步的概念应该深入每一个搞IT人员的心中
部署docker
#安装服务器必备软件
# yum -y install wget vim iftop iotop net-tools nmon telnet lsof iptraf nmap httpd-tools lrzsz mlocate ntp ntpdate strace libpcap nethogs iptraf iftop nmon bridge-utils bind-utils telnet nc nfs-utils rpcbind nfs-utils dnsmasq python python-devel yum-utils device-mapper-persistent-data lvm2 tcpdump mlocate tree
#添加docker源信息
[root@bs-k8s-harbor01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#列出所有docker版本，选择和Kubernetes集群一致的docker版本[不一致能否正常，我没尝试]
[root@bs-k8s-harbor01 ~]# yum list docker-ce --showduplicates | sort -r
[root@bs-k8s-master01 ~]# docker -v
Docker version 19.03., build 633a0ea838 #kubernetes集群版本
#安装对应版本
[root@bs-k8s-harbor01 ~]# yum -y install docker-ce-19.03.-.el7
#为了保持集群环境一致，daemon.json也应该一致
[root@bs-k8s-master01 docker]# cat daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["127.0.0.1/8"],
"max-concurrent-downloads": ,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": ""
},
"data-root": "/var/lib/docker"
}
[root@bs-k8s-harbor01 ~]# mkdir /etc/docker
[root@bs-k8s-master01 docker]# scp daemon.json 20.0.0.207:/etc/docker/
root@20.0.0.207's password:
[root@bs-k8s-harbor01 docker]# systemctl restart docker && systemctl enable docker
[root@bs-k8s-harbor01 docker]# docker version
Client: Docker Engine - Community
Version: 19.03.
......
Server: Docker Engine - Community
Engine:
Version: 19.03.
部署harbor
harbor的管理是基于docker-compose的
# yum install -y docker-compose
# docker-compose version
docker-compose version 1.18., build 8dd22a9
docker-py version: 2.6.
CPython version: 3.6.
OpenSSL version: OpenSSL 1.0.2k-fips Jan
harbor的 git地址：https://github.com/goharbor/harbor
这里我使用的版本是1.8
[root@bs-k8s-harbor01 data]# pwd
/data
[root@bs-k8s-harbor01 data]# ll
总用量
-rw-r--r-- root root 5月 harbor-offline-installer-v1.8.0.tgz
root@bs-k8s-harbor01 data]# tar xf harbor-offline-installer-v1.8.0.tgz
[root@bs-k8s-harbor01 data]# mv harbor /usr/local/
[root@bs-k8s-harbor01 data]# cd /usr/local/harbor/
[root@bs-k8s-harbor01 harbor]# ls
harbor.v1.8.0.tar.gz harbor.yml install.sh LICENSE prepare
#创建证书
# mkdir /data/ca
# openssl genrsa -out /data/ca/harbor-ca.key
Generating RSA private key, bit long modulus
....................+++
..................................................................................+++
e is (0x10001)
# openssl req -x509 -new -nodes -key /data/ca/harbor-ca.key -subj "/CN=harbor.linux.com" -days -out /data/ca/harbor-ca.crt
#修改配置文件
# cp harbor.yml{,.bak}
# vim harbor.yml
# diff harbor.yml{,.bak}
5c5
5c5
< hostname: harbor.linux.com
---
> hostname: reg.mydomain.com
8c8
< #http:
---
> http:
10c10
< # port:
---
> port:
13c13
< https:
---
> # https:
15c15
< port:
---
> # port:
,18c17,
< certificate: /data/ca/harbor-ca.crt
< private_key: /data/ca/harbor-ca.key
---
> # certificate: /your/certificate/path
> # private_key: /your/private/key/path
27c27
< harbor_admin_password: zisefeizhu
---
> harbor_admin_password: Harbor12345
35c35
< data_volume: /data/harbor
---
> data_volume: /data
#部署
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
# cp /data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/
# ./install.sh
# docker-compose start
Starting log ... done
Starting registry ... done
Starting registryctl ... done
Starting postgresql ... done
Starting core ... done
Starting portal ... done
Starting redis ... done
Starting jobservice ... done
Starting proxy ... done
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
287136c60b95 goharbor/nginx-photon:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) 0.0.0.0:->/tcp, 0.0.0.0:->/tcp nginx
66a07d42818c goharbor/harbor-jobservice:v1.8.0 "/harbor/start.sh" seconds ago Up seconds harbor-jobservice
e4bb415fd236 goharbor/harbor-portal:v1.8.0 "nginx -g 'daemon of…" seconds ago Up seconds (healthy) /tcp harbor-portal
1530c4b4c604 goharbor/harbor-core:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) harbor-core
adc160874fef goharbor/redis-photon:v1.8.0 "docker-entrypoint.s…" seconds ago Up seconds /tcp redis
300165f93782 goharbor/harbor-db:v1.8.0 "/entrypoint.sh post…" seconds ago Up seconds (healthy) /tcp harbor-db
a81c3d53eb2e goharbor/registry-photon:v2.7.1-patch--v1.8.0 "/entrypoint.sh /etc…" seconds ago Up seconds (healthy) /tcp registry
1a7cf72c6433 goharbor/harbor-registryctl:v1.8.0 "/harbor/start.sh" seconds ago Up seconds (healthy) registryctl
6be2b10b733d goharbor/harbor-log:v1.8.0 "/bin/sh -c /usr/loc…" seconds ago Up seconds (healthy) 127.0.0.1:->/tcp harbor-log
# ss -lntup
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN *: *:* users:(("systemd",pid=,fd=))
udp UNCONN *: *:* users:(("chronyd",pid=,fd=))
udp UNCONN 127.0.0.1: *:* users:(("chronyd",pid=,fd=))
udp UNCONN ::: :::* users:(("chronyd",pid=,fd=))
tcp LISTEN *: *:* users:(("sshd",pid=,fd=))
tcp LISTEN 127.0.0.1: *:* users:(("docker-proxy",pid=,fd=))
tcp LISTEN *: *:* users:(("systemd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
tcp LISTEN ::: :::* users:(("sshd",pid=,fd=))
tcp LISTEN ::: :::* users:(("docker-proxy",pid=,fd=))
# docker login harbor.linux.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded 配置开机自启
# cat /etc/rc.d/rc.local
cd /usr/local/harbor && docker-compose start
注：登陆失败的原因可能有：1. hosts没有域名解析 2.密码错误
客户端配置
以bs-k8s-master01为例
# mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com
mkdir: 已创建目录 "/etc/docker/certs.d"
mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"
The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.
ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.
ECDSA key fingerprint is MD5:a2:3a::bc:e7:7a:f8:c3:ef:db:6c:d5:d2::e1:3c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.
root@20.0.0.207's password:
harbor-ca.crt % .7KB/s :
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeededxxxxxxxxxx31 # mkdir -pv /etc/docker/certs.d/harbor.linux.com/ && scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/ && docker login harbor.linux.com2mkdir: 已创建目录 "/etc/docker/certs.d"3mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"4The authenticity of host '20.0.0.207 (20.0.0.207)' can't be established.5ECDSA key fingerprint is SHA256:EqqNfQ6sVyEO5yRX8E2plLlEaaeTyLbXhocH4uxhvJw.6ECDSA key fingerprint is MD5:a2:3a:03:bc:e7:7a:f8:c3:ef:db:6c:d5:d2:34:e1:3c.7Are you sure you want to continue connecting (yes/no)? yes8Warning: Permanently added '20.0.0.207' (ECDSA) to the list of known hosts.9root@20.0.0.207's password: 10harbor-ca.crt % .7KB/s : 11Username: admin12Password: 13WARNING! Your password will be stored unencrypted in /root/.docker/config.json.14Configure a credential helper to remove this warning. See15https://docs.docker.com/engine/reference/commandline/login/#credentials-store1617Login Succeeded18# mkdir -pv /etc/docker/certs.d/harbor.linux.com/19mkdir: 已创建目录 "/etc/docker/certs.d"20mkdir: 已创建目录 "/etc/docker/certs.d/harbor.linux.com/"21# scp 20.0.0.207:/data/ca/harbor-ca.crt /etc/docker/certs.d/harbor.linux.com/22root@20.0.0.207's password: 23harbor-ca.crt 100% 1115 690.7KB/s 00:00 24# docker login harbor.linux.com25Username: admin26Password: 27WARNING! Your password will be stored unencrypted in /root/.docker/config.json.28Configure a credential helper to remove this warning. See29https://docs.docker.com/engine/reference/commandline/login/#credentials-store3031Login Succeededshell
# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://dockerhub.azk8s.cn", "https://docker.mirrors.ustc.edu.cn"],
"insecure-registries": ["harbor.linux.com", "20.0.0.207:443"],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
# systemctl restart docker
# docker login harbor.linux.com
# docker login 20.0.0.207:443
# cat /root/.docker/config.json
{
"auths": {
"auths": {
"20.0.0.207:443": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
},
"harbor.linux.com": {
"auth": "YWRtaW46emlzZWZlaXpodQ=="
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (linux)"
}
}
注：其他集群机同样操作
测试
以ceph rbd 为例
#kubernetes master节点拉取镜像上传到harbor仓库，worker节点拉取镜像
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker pull quay.io/external_storage/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker tag quay.io/external_storage/rbd-provisioner:latest harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-master01 k8s]# docker push harbor.linux.com/rbd/rbd-provisioner:latest
[root@bs-k8s-node01 ~]# docker pull harbor.linux.com/rbd/rbd-provisioner:latest
对接kubernetes
#用户名密码存放
# cat ~/.docker/config.json |base64 -w
ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
#创建secret
# cat secret-harbor.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: secret-harbor.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: Secret
metadata:
name: k8s-harbor-login
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubGludXguY29tIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NmVtbHpaV1psYVhwb2RRPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xpZW50LzE5LjAzLjUgKGxpbnV4KSIKCX0KfQ==
# pwd
/data/k8s/harbor
# kubectl apply -f secret-harbor.yaml
secret/k8s-harbor-login created
#部署rbd pod测试
[root@bs-k8s-master01 harbor]# cat external-storage-rbd-provisioner.yaml
##########################################################################
#Author: zisefeizhu
#QQ: ********
#Date: --
#FileName: external-storage-rbd-provisioner.yaml
#URL: https://www.cnblogs.com/zisefeizhu/
#Description: The test script
#Copyright (C): All rights reserved
###########################################################################
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-provisioner
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["kube-dns"]
verbs: ["list", "get"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: rbd-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rbd-provisioner
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rbd-provisioner
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rbd-provisioner
subjects:
- kind: ServiceAccount
name: rbd-provisioner
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rbd-provisioner
namespace: default
spec:
replicas:
selector:
matchLabels:
app: rbd-provisioner
strategy:
type: Recreate
template:
metadata:
labels:
app: rbd-provisioner
spec:
containers:
- name: rbd-provisioner
image: "harbor.linux.com/rbd/rbd-provisioner:latest"
imagePullPolicy: IfNotPresent
env:
- name: PROVISIONER_NAME
value: ceph.com/rbd
imagePullSecrets:
- name: k8s-harbor-login
serviceAccount: rbd-provisioner
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
rbd: "true"
#节点打标签
[root@bs-k8s-master01 harbor]# kubectl label nodes 20.0.0.204 rbd=true
node/20.0.0.204 labeled
#删除bs-k8s-node01节点上的rbd镜像
[root@bs-k8s-master01 harbor]# kubectl apply -f external-storage-rbd-provisioner.yaml
serviceaccount/rbd-provisioner created
clusterrole.rbac.authorization.k8s.io/rbd-provisioner created
clusterrolebinding.rbac.authorization.k8s.io/rbd-provisioner created
role.rbac.authorization.k8s.io/rbd-provisioner created
rolebinding.rbac.authorization.k8s.io/rbd-provisioner created
deployment.apps/rbd-provisioner created
[root@bs-k8s-master01 harbor]# kubectl get pods -o wide -w
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
rbd-provisioner-9cf46c856-bl454 / ContainerCreating 6s <none> 20.0.0.204 <none> <none>
rbd-provisioner-9cf46c856-bl454 / Running 37s 172.20.46.82 20.0.0.204 <none> <none>
测试完成
部署harbor以https模式和k8s对接的更多相关文章
- kubernetes实战(十三)：k8s使用helm持久化部署harbor集成openLDAP登录
1.基本概念 上节在k8s中部署了harbor和ldap,本节将部署harbor使用openLDAP验证,部署方式与之前相同,只是改了adminserver-cm.yaml的AUTH_MODE: &q ...
- 部署 harbor 私有仓库
安装下载依赖包 安装docker-compose 从 docker compose 发布页面下载最新的 docker-compose 二进制文件,本文以1.25.4为例 cd /opt/k8s/wor ...
- 群晖DS218+部署Harbor(1.10.3)
欢迎访问我的GitHub https://github.com/zq2599/blog_demos 内容:所有原创文章分类汇总及配套源码,涉及Java.Docker.Kubernetes.DevOPS ...
- Flink 集群运行原理兼部署及Yarn运行模式深入剖析
1 Flink的前世今生(生态很重要) 原文:https://blog.csdn.net/shenshouniu/article/details/84439459 很多人可能都是在 2015 年才听到 ...
- centos7部署harbor
官网 https://github.com/goharbor/harbor 1.升级系统内核 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrep ...
- CentOS部署Harbor镜像仓库
关于Harbor Harbor是用于存储和分发Docker镜像的镜像仓库服务,相比Docker Registry,Harbor在安全.标识.管理等方面做了增强,更适合企业使用: 官方网站:https: ...
- 部署http访问SVN模式出现403问题
部署http访问SVN模式到阿里云服务器 参考连接地址 https://help.aliyun.com/document_detail/52864.html 设置好账号进行访问 http://ip/s ...
- docker部署harbor私有镜像库（3）
一.harbor介绍 在实际生产运维中,往往需要把镜像发布到几十.上百台或更多的节点上.这时单台Docker主机上镜像已无法满足,项目越来越多,镜像就越来越多,都放到一台Docker主机上是不行的,我 ...
- 在nginx上部署vue项目(history模式)；
在nginx上部署vue项目(history模式): vue-router 默认是hash模式,使用url的hash来模拟一个完整的url,当url改变的时候,页面不会重新加载.但是如果我们不想has ...
随机推荐
- web虚拟主机的三种配置方法
- javaScript系列 [28]
本文介绍JavaScript事件相关的知识点,主要包括事件流.事件处理程序.事件对象(event)以及常见事件类型和事件委托等相关内容. 在网页开发涉及的三种基础技术(HTML CSS JavaScr ...
- 等效燃油消耗ECMS与庞德里亚金最小值原理PMP中协同状态的关系
今儿阅读了一篇文献——<车联网环境下并联混合动力客车控制策略优化研究>,是北理的博士所写的,内容比较翔实.主要是里面的关于ECMS和庞德里亚金最小值原理(PMP)的关系推导很让人印象深刻, ...
- HTTP&ServletContext&Response对象_文件上传
今日内容 1. HTTP协议:响应消息 2. Response对象 3. ServletContext对象 HTTP协议 1. 请求消息:客户端发送给服务器端的数据 * 数据格式: 1. 请求行 2. ...
- php表单提交后再后退 内容则默认清空的解决方法
转载原文地址: http://www.jquerycn.cn/a_14422 在session_start()之后,字符输出之前加上header("Cache-control: privat ...
- 前端实现html转pdf方法总结
最近要搞前端html转pdf的功能.折腾了两天,略有所收,踩了一些坑,所以做些记录,为后来的兄弟做些提示,也算是回馈社区.经过一番调(sou)研(suo)发现html导出pdf一般有这几种方式,各有各 ...
- Layabox UI 笔记
1.UI List 官方文档是 https://ldc2.layabox.com/doc/?language=zh&nav=zh-js-3-3-7 通常用List的时候会出现滚动条按钮比例拉 ...
- jQuery上传文件按钮美化
效果图如下: 思路: 1:打开文件设置为透明,外面包一层标签,给标签设置颜色背景,给人点击浏览其实是点击打开文件的错觉.(给外标签相对定位,打开文件标签绝对定位). 2:点击浏览后,选择了文件,就把文 ...
- JDK8内存模型—消失的PermGen
一.JVM 内存模型 根据 JVM 规范,JVM 内存共分为虚拟机栈.堆.方法区.程序计数器.本地方法栈五个部分. 1.虚拟机栈:每个线程有一个私有的栈,随着线程的创建而创建.栈里面存着的是一种叫“栈 ...
- 前端传json数组 ，后端的接收
前端传输: var updateGoodsId=$(this).val();//get id var updateGoodsPrice=$("#IngoodsPrice"+upda ...