Jenkins和Gitlab CI/CD自动更新k8s中pod使用的镜像说明

Jenkins

使用Jenkins的话，完成的工作主要有如下步骤：

1.从Gogs或Gitlab仓库上拉取代码

2.使用Maven编译代码，打包成jar文件

3.根据jar文件使用相对应的Dockerfile文件制作成Docker镜像

4.把Docker镜像推送到Nexus上的Docker仓库(或者Harbor仓库)

5.运行shell脚本，给k8s的master主机上执行更新pod的脚本命令 (这一步不想自动实现的话可以采取手动操作)

前四步的操作：

地址：https://www.cnblogs.com/sanduzxcvbnm/p/11803368.html

https://www.cnblogs.com/sanduzxcvbnm/p/11800008.html

https://www.cnblogs.com/sanduzxcvbnm/p/11799583.html

https://www.cnblogs.com/sanduzxcvbnm/p/11797660.html

第5步的操作说明：

k8s使用的是kuboard面板提供的有一个CI/CD集成功能，主要使用的就是这个

官方网址：https://kuboard.cn/guide/cicd/

直接在 shell 脚本中调用 Kuboard/Kubernetes API，更新对应容器镜像的标签。完成此接口调用后，Kubernetes 将自动完成滚动更新。

界面中提供的脚本里，容器的版本是当前 Kubernetes 中已部署的版本，通常在 Jenkins/GitlabRunner 之类的工具里，需要将该脚本的镜像标签参数化。

Gitlab Runner

使用Jenkins的话，完成的工作主要有如下步骤：

1.代码提交到Gitlab仓库后根据检测到的.gitlab-ci.yml文件自动执行CI/CD操作

2.CI/CD操作

2.1使用Maven编译代码，打包成jar文件

2.2根据jar文件使用相对应的Dockerfile文件制作成Docker镜像

2.3把Docker镜像推送到Nexus上的Docker仓库(或者Harbor仓库) (这一步使用的变量可以在Gitlab中进行设置)

2.4运行shell脚本，给k8s的master主机上执行更新pod的脚本命令 (还有另一种办法，详看下面) (这一步不想自动实现的话可以采取手动操作)

.gitlab-ci.yml文件内容示例：

stages:
  - test
  - build
  - release
  - review
  - deploy

test:
  stage: test
  tags:
    - test
  script:
    - sleep 3
    - echo "We did it! Something else runs in parallel!"

compile:
  image: maven:3.5-jdk-8-alpine
  tags:
    - maven
  stage: build
  only:
    - dev
  script:
    - mvn clean package -Dmaven.test.skip=true
  artifacts:
    paths:
      - target/*.jar
      - target/lib

image_build:
  stage: release
  image: docker:latest
  tags:
    - docker
  services:
    - docker:dind
  variables:
    TAG: v0.9 # 镜像版本，需要想办法设置灵活
  script:
    - docker build -t "${CI_REGISTRY_IMAGE}:$TAG" .
    - docker rm -f test || true
    - docker login -u "${CI_REGISTRY_USER}" -p "${CI_REGISTRY_PASSWORD}" "${CI_REGISTRY}"
    - docker tag "${CI_REGISTRY_IMAGE}:$TAG" "${CI_REGISTRY}/${CI_REGISTRY_IMAGE}:$TAG"
    - docker push "${CI_REGISTRY}/${CI_REGISTRY_IMAGE}:$TAG"
  only:
    - dev

pod_update:
  stage: deploy
  tags:
    - k8s
  script:
    - echo "=============== 开始执行更新pod使用的镜像任务  ==============="
    - pwd
    - chmod a+x ./scripts/build/update.sh
    # 这一步执行报错：不是找不到文件就是找不到命令，原因采用的注册runner时的docker镜像，里面没有curl命令，解决办法：在这一步找一个含有curl命令的镜像即可
    # - ./scripts/build/update.sh
    - echo "=============== 更新任务执行结束 ============================"

默认在注册runner的时候需要填写一个基础的镜像，只要使用执行器为docker类型的runner所有的操作运行都会在容器中运行。 如果全局指定了images则所有作业使用此image创建容器并在其中运行。 全局未指定image，再次查看job中是否有指定，如果有此job按照指定镜像创建容器并运行，没有则使用注册runner时指定的默认镜像。

update.sh文件内容跟Jenkins中的第五步操作一样

Gitlab Runner上跟K8S上更新pod的另一个操作：

在.gitlab-ci.yml文件安装kubectl工具，然后使用这个工具客户端跟k8s进行通信，从而执行更新pod的操作

安装kubectl工具地址：https://www.cnblogs.com/sanduzxcvbnm/p/13865238.html

示例内容如下：

image:
  name: golang:1.10.3-stretch
  entrypoint: ["/bin/sh", "-c"]

# The problem is that to be able to use go get, one needs to put
# the repository in the $GOPATH. So for example if your gitlab domain
# is mydomainperso.com, and that your repository is repos/projectname, and
# the default GOPATH being /go, then you'd need to have your
# repository in /go/src/mydomainperso.com/repos/projectname
# Thus, making a symbolic link corrects this.
before_script:
  - mkdir -p "/go/src/git.qikqiak.com/${CI_PROJECT_NAMESPACE}"
  - ln -sf "${CI_PROJECT_DIR}" "/go/src/git.qikqiak.com/${CI_PROJECT_PATH}"
  - cd "/go/src/git.qikqiak.com/${CI_PROJECT_PATH}/"

stages:
  - test
  - build
  - release
  - review
  - deploy

test:
  stage: test
  script:
    - make test

test2:
  stage: test
  script:
    - sleep 3
    - echo "We did it! Something else runs in parallel!"

compile:
  stage: build
  script:
    # Add here all the dependencies, or use glide/govendor/...
    # to get them automatically.
    - make build
  artifacts:
    paths:
      - app

image_build:
  stage: release
  image: docker:latest
  variables:
    DOCKER_DRIVER: overlay
    DOCKER_HOST: tcp://localhost:2375
  services:
    - name: docker:17.03-dind
      command: ["--insecure-registry=registry.qikqiak.com"]
  script:
    - docker info
    - docker login -u "${CI_REGISTRY_USER}" -p "${CI_REGISTRY_PASSWORD}" registry.qikqiak.com
    - docker build -t "${CI_REGISTRY_IMAGE}:latest" .
    - docker tag "${CI_REGISTRY_IMAGE}:latest" "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}"
    - test ! -z "${CI_COMMIT_TAG}" && docker push "${CI_REGISTRY_IMAGE}:latest"
    - docker push "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}"

deploy_review:
  image: cnych/kubectl
  stage: review
  only:
    - branches
  except:
    - tags
  environment:
    name: dev
    url: https://dev-gitlab-k8s-demo.qikqiak.com
    on_stop: stop_review
  script:
    - kubectl version
    - cd manifests/
    - sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml ingress.yaml service.yaml
    - sed -i "s/__VERSION__/${CI_COMMIT_REF_NAME}/" deployment.yaml ingress.yaml service.yaml
    - |
      if kubectl apply -f deployment.yaml | grep -q unchanged; then
          echo "=> Patching deployment to force image update."
          kubectl patch -f deployment.yaml -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"ci-last-updated\":\"$(date +'%s')\"}}}}}"
      else
          echo "=> Deployment apply has changed the object, no need to force image update."
      fi
    - kubectl apply -f service.yaml || true
    - kubectl apply -f ingress.yaml
    - kubectl rollout status -f deployment.yaml
    - kubectl get all,ing -l ref=${CI_ENVIRONMENT_SLUG}

stop_review:
  image: cnych/kubectl
  stage: review
  variables:
    GIT_STRATEGY: none
  when: manual
  only:
    - branches
  except:
    - master
    - tags
  environment:
    name: dev
    action: stop
  script:
    - kubectl version
    - kubectl delete ing -l ref=${CI_ENVIRONMENT_SLUG}
    - kubectl delete all -l ref=${CI_ENVIRONMENT_SLUG}

deploy_live:
  image: cnych/kubectl
  stage: deploy
  environment:
    name: live
    url: https://live-gitlab-k8s-demo.qikqiak.com
  only:
    - tags
  when: manual
  script:
    - kubectl version
    - cd manifests/
    - sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml ingress.yaml service.yaml
    - sed -i "s/__VERSION__/${CI_COMMIT_REF_NAME}/" deployment.yaml ingress.yaml service.yaml
    - kubectl apply -f deployment.yaml
    - kubectl apply -f service.yaml
    - kubectl apply -f ingress.yaml
    - kubectl rollout status -f deployment.yaml
    - kubectl get all,ing -l ref=${CI_ENVIRONMENT_SLUG}