实战项目部署应用到kubernetes流程(jenkins+docker+k8s)
说明
通过jenkins构建java应用程序发布到k8s集群中
本文已一个大数据的java项目来演示构建部署过程
支持发布和回滚
支持一套模板应用不同项目
k8s基础准备
创建项目名称空间
[root@k8s-master ~]# kubectl create ns architect
namespace/architect created [root@k8s-master ~]# kubectl label namespace architect istio-injection=enabled
namespace/architect labeled
[root@k8s-master ~]# kubectl get namespace -L istio-injection
NAME STATUS AGE ISTIO-INJECTION
architect Active 87s enabled
default Active 39d enabled
istio-system Active 7d17h
kube-public Active 39d
kube-system Active 39d
[root@k8s-master ~]#
创建全局配置文件
＃创建大数据配置文件
[root@k8s-master ~]# ls bigdata-counter
core-site.xml dmp_operator1.keytab hbase-site.xml hdfs-site.xml krb5.conf
[root@k8s-master ~]#
[root@k8s-master ~]# kubectl create configmap bigdata-config --from-file=/root/bigdata-counter/ -n architect
configmap/bigdata-config created
[root@k8s-master ~]#
#全局环境变量
[root@k8s-master ~]# cat yaml/global-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: global-config
namespace: architect
data:
CONFIG_SERVER_URL: http://config.ms.xx.com
CONFIG_SERVER_USER: myconfig
ILOG_BOOTSTRAP_SERVERS: bootstrap.servers=192.168.1.122:9092
CONFIG_SERVER_PWDS: mc123456 [root@k8s-master ~]#
[root@k8s-master ~]# kubectl apply -f yaml/global-config.yaml
configmap/global-config created
[root@k8s-master ~]# kubectl get configmap -n architect
NAME DATA AGE
bigdata-config 4 2m33s
global-config 4 3s
[root@k8s-master ~]#
创建habror认证信息
[root@k8s-master ~]# kubectl create secret docker-registry registry-secret --docker-server=dev-hub.xx.net --docker-username=admin --docker-password=Harbor12345 --docker-email=admin@dev-hub.xx.net -n architect
secret/registry-secret created
[root@k8s-master ~]# kubectl get secret -n architect
NAME TYPE DATA AGE
default-token-5wsxl kubernetes.io/service-account-token 3 2m24s
istio.default istio.io/key-and-cert 3 2m24s
registry-secret kubernetes.io/dockerconfigjson 1 10s
项目deployment文件
可以做成模本，通过变量替换形式．后面有讲到
[root@k8s-master ~]# cat bigdata-interface-deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: bigdata-interface
version: default
name: bigdata-interface
namespace: architect
spec:
replicas: 1
selector:
matchLabels:
app: bigdata-interface
version: default
template:
metadata:
labels:
app: bigdata-interface
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: bigdata-interface
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b63
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: bigdata-interface
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
[root@k8s-master ~]#
jenkins pipeline 模板文件
node {
//项目名称
String APPNAME='bigdata-interface'
//名称空间
String NAMESPACE='architect'
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(name:'GIT_BRANCH_TAG', defaultValue: 'master',description:'', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.GIT_BRANCH_TAG]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git']]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
进一步改造pipeline
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: 'git@git.jiatuiyun.net:bigdata-center/bigdata-interface.git', description: 'git仓库地址', name: 'XBUILD_REPO', trim: false),
string(defaultValue: 'master', description: '版本名称', name: 'XBUILD_BRANCHE', trim: false)
])])
stage('git clone') {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bc', url: params.XBUILD_REPO]]])
}
stage('maven build') {
sh 'mvn clean package -U'
}
stage('docker build && push habror'){
sh "cd ${WORKSPACE}"
restapi = '`ls -d *-restapi||echo .`'
sh "ls -l ${restapi}/target/"
sh "echo ${restapi}"
sh "curl -o ${restapi}/target/Dockerfile http://192.168.1.55:8089/files/Dockerfile-java.v1"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
sh "docker build -t ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${restapi}/target"
sh "docker tag ${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
sh "docker login ${HarborUrl} -u ${HarborUser} -p ${HarborPasswd}"
sh "docker push ${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER}"
}
stage('deploy'){
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
echo "${params.GIT_BRANCH_TAG}"
echo "${BUILD_NUMBER}"
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins上创建自由风格项目
效果如下 支持参数化构建。但每项目单独JOB
查看镜像仓库已经推送正常
项目回滚
node {
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
//updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
//test
}
}
jenkins效果如下：输入namespace和deployment 名称和上次构建版本
查看k8s集群
可以灵活的在已知版本回滚操作
[root@k8s-master ~]# kubectl get deploy -n architect -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
bigdata-interface 0/1 1 0 3h12m bigdata-interface dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b1 app=bigdata-interface,version=default
[root@k8s-master ~]# kubectl get pods -n architect
NAME READY STATUS RESTARTS AGE
bigdata-interface-6bf7b5646d-sf9j5 1/2 Running 0 24s
[root@k8s-master ~]#
通过以上实践，可以将deployment做成模板
deployment模板文件
-bash-4.2$ cat k8s-deployment.tpl
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: {APPNAME}
version: default
name: {APPNAME}
namespace: {NAMESPACE}
spec:
replicas: 1
selector:
matchLabels:
app: {APPNAME}
version: default
template:
metadata:
labels:
app: {APPNAME}
version: default
spec:
containers:
- env:
- name: izone
- name: MY_SERVICE_NAME
value: {APPNAME}
- name: ilogEnvs
value: izone,buildNumber
- name: ILOG_BOOTSTRAP_SERVERS
valueFrom:
configMapKeyRef:
key: ILOG_BOOTSTRAP_SERVERS
name: global-config
- name: CONFIG_SERVER_URL
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_URL
name: global-config
- name: CONFIG_SERVER_USER
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_USER
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: CONFIG_SERVER_PWDS
valueFrom:
configMapKeyRef:
key: CONFIG_SERVER_PWDS
name: global-config
- name: JAR_BOOT_ARG
value: --spring.cloud.config.fail-fast=true
- name: JDK_HEAP_OPTS
value: -Xms256m -Xmx256m
image: {HarborUrl}/{NAMESPACE}/{APPNAME}:{NAMESPACE}_b{BUILD_NUMBER}
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 2
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 300
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: {APPNAME}
ports:
- containerPort: 80
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /actuator/info
port: 80
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
volumeMounts:
- mountPath: /opt/jdk
name: jdk-path
- mountPath: /srv/applogs
name: applogs
imagePullSecrets:
- name: registry-secret
restartPolicy: Always
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /usr/local/jdk/default
type: ""
name: jdk-path
- hostPath:
path: /data/applogs
type: ""
name: applogs
-bash-4.2$
通过模板动态生成项目的deploy文件．进行升级回滚．
比如下面的：
node {
String HarborUrl='dev-hub.jiatuiyun.net'
String HarborUser='admin'
String HarborPasswd='Harbor12345'
String KubernetHost = 'root@192.168.100.101'
properties([
parameters([
string(defaultValue: 'architect', description: 'git组', name: 'XBUILD_TEAM', trim: false),
string(defaultValue: 'bigdata-interface', description: '项目名', name: 'XBUILD_DEPLOY', trim: false),
string(defaultValue: '1', description: '镜像标签', name: 'XBUILD_NUMBER', trim: false)
])])
stage('deploy'){
echo "${BUILD_NUMBER}"
NAMESPACE = params.XBUILD_TEAM
APPNAME = params.XBUILD_DEPLOY
BUILD_NUMBER = params.XBUILD_NUMBER
echo "----${params.XBUILD_NUMBER}---${params.XBUILD_DEPLOY} ----${params.XBUILD_TEAM} "
//kubectl set image deployment/bigdata-interface bigdata-interface=dev-hub.jiatuiyun.net/architect/bigdata-interface:architect_b48 -n architect
updateImage="kubectl set image deployment/${APPNAME} --namespace=${NAMESPACE} ${APPNAME}=${HarborUrl}/${NAMESPACE}/${APPNAME}:${NAMESPACE}_b${BUILD_NUMBER} --record"
command="sed -e 's#{APPNAME}#${APPNAME}#g;s#{NAMESPACE}#${NAMESPACE}#g;s#{HarborUrl}#${HarborUrl}#g;s#{BUILD_NUMBER}#${BUILD_NUMBER}#g' /data/jenkins_slave/k8s-deployment.tpl > /data/jenkins_slave/${APPNAME}-deploy.yaml"
sh "${command}"
sh "scp -P 22 /data/jenkins_slave/${APPNAME}-deploy.yaml ${KubernetHost}:/root/ "
//sh "ssh -p 22 ${KubernetHost} ${updateImage}"
sh "ssh -p 22 ${KubernetHost} kubectl apply -f /root/${APPNAME}-deploy.yaml"
//test
}
}
pipeline参数化构建
示例：
pipeline {
agent any
parameters {
gitParameter branch: '', branchFilter: '.*', defaultValue: '', description: '', name: 'XBUILD_BRANCHE', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH_TAG'
string defaultValue: 'git@git.xx.net:company/xx-xx.git', description: '', name: 'XBUILD_REPO', trim: false
}
stages {
stage('Example') {
steps {
checkout([$class: 'GitSCM', branches: [[name: params.XBUILD_BRANCHE]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'b6a484b7-cbab-4215-9e93-2271d7a333bb', url: params.XBUILD_REPO]]])
}
}
}
}
构建效果如下：
实战项目部署应用到kubernetes流程(jenkins+docker+k8s)的更多相关文章
- docker 实战-项目部署
前面学习了 docker 的命令和实际的用法,现在来实战一下.编排一个服务,也就是项目部署. 目前我们在一个闭源环境下工作,介绍一些工作流程和部署项目的方法. 该工作流程比较特殊 所谓闭源环境就是 没 ...
- jenkins+docker+k8s项目发布
目录 一.简介 二.新建docker-build工程 三.项目部署 四.访问测试 一.简介 1.该章节基于jenkins.Harbor.pipeline.k8s来做发布,如对这些不熟悉,请按以下进入学 ...
- .Net 6 WebApi 项目部署到 Linux 系统上的 Docker 容器
1.创建一个基础的WebApi项目 注意:因为发布时候,Dockerfile文件必须和解决方案.cspro文件放在同级,所以建议勾上这个,当时遇到这个问题,导致打包镜像时找不到.cspro文件,搞了好 ...
- 第四十四章 微服务CICD（6）- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- gitlab + jenkins + docker + k8s
总体流程: 在开发机开发代码后提交到gitlab 之后通过webhook插件触发jenkins进行构建,jenkins将代码打成docker镜像,push到docker-registry 之后将在k8 ...
- kubernetes下jenkins实战maven项目编译构建
关于kubernetes环境的jenkins集群 在kubernetes环境部署的jenkins集群,执行任务时会新建pod,任务完成后pod被销毁,架构如下所示: 在kubernetes搭建jenk ...
- Kubernetes笔记（三）：Gitlab+Jenkins Pipeline+Docker+k8s+Helm自动化部署实践（干货分享！）
通过前面两篇文章,我们已经有了一个"嗷嗷待哺"的K8s集群环境,也对相关的概念与组件有了一个基本了解(前期对概念有个印象即可,因为只有实践了才能对其有深入理解,所谓"纸上 ...
- 【k8s实战一】Jenkins 部署应用到 Kubernetes
[k8s实战一]Jenkins 部署应用到 Kubernetes 01 本文主旨 目标是演示整个Jenkins从源码构建镜像到部署镜像到Kubernetes集群过程. 为了简化流程与容易重现文中效果, ...
- kubernetes部署jenkins(Docker in Docker)及认证
引言 Jenkins是一款开源 CI&CD 软件,用于自动化各种任务,包括构建.测试和部署软件. 本文将Jenkins的master与slave置于Pod中,部署在namespace:jenk ...
随机推荐
- Centos7修改Docker默认存储位置
一.前言 Centos7安装docker之后,默认的镜像及容器存储路径为/var/lib/docker,可以使用命令docker info查看. 但是该路径默认使用的是系统盘的存储,如果挂载了数据盘, ...
- Python3.x 基础练习题100例（11-20）
练习11: 题目: 古典问题:有一对兔子,从出生后第3个月起每个月都生一对兔子,小兔子长到第三个月后每个月又生一对兔子,假如兔子都不死,问每个月的兔子总数为多少? 分析: 兔子的规律为数列1,1,2, ...
- “蚂蚁牙黑”太火，想玩就用ModelArts做一个！
摘要:本文将介绍如何借力一站式 AI 开发平台,"傻瓜式"操作实现生成"蚂蚁牙黑"小视频. 作者:华为云EI专家胡琦 一夜之间,朋友圈都在"蚂蚁牙黑& ...
- AQS源码解读（ReentrankLock的公平锁和非公平锁）
构建Debug代码: 1 package com.hl.interview.lock; 2 3 import java.util.Scanner; 4 import java.util.concurr ...
- 剑指 Offer 34. 二叉树中和为某一值的路径 + 记录所有路径
剑指 Offer 34. 二叉树中和为某一值的路径 Offer_34 题目详情 题解分析 本题是二叉树相关的题目,但是又和路径记录相关. 在记录路径时,可以使用一个栈来存储一条符合的路径,在回溯时将进 ...
- 通过穷举法快速破解excel或word加密文档最高15位密码
1.打开文件 2.工具 --- 宏 ---- 录制新宏 --- 输入名字如 :aa 3.停止录制 ( 这样得到一个空宏 ) 4.工具 --- 宏 ---- 宏 , 选 aa, 点编辑按钮 5.删除窗口 ...
- C#中的字段与属性的区别及属性的作用
C#中的字段与属性的区别及属性的作用 先上代码 public class Employee { //字段 private string name; //属性 public string Name { ...
- 2018.12-2019.1 TO-DO LIST
AC自动机 P3808 [模板]AC自动机(简单版)(完成时间:2018.12.06) P3796 [模板]AC自动机(加强版)(完成时间:2018.12.06) P2444 [POI2000]病毒( ...
- 设计模式之建造者模式(BuilderPattern)
一.意义 将一个复杂的对象的构建与它的表示分离,使得同样的构建过程可以创建不同的表示. 说明:复杂对象的构建,比如一个对象有几十个成员属性,那么我们在创建这个对象,并给成员属性赋值时,就会很麻烦.采用 ...
- 解决linux sudo apt-get install xx是2出现无法定位软件包方法
解决办法: 在etc/apt/sources.list最后一行添加 deb http://archive.ubuntu.com/ubuntu/ trusty main universe restric ...