使用Kubernetes的java-client实现Deployment的部署及更新操作

1. 背景介绍

需求: 针对多种协议SDK构造探针,测试公司接入机服务状况(每一个探针应对单一接入机,接入机数量可能会动态变化).

难点: 大多数协议SDK均不支持多实例运行,且部分SDK通过生成文件保存内部状态;各协议SDK处于迭代状态,不能对其进行魔改.

分析:

(1) 基于以上原因,无法选择多线程或者多进程,在单一物理机或容器内完成探针相应功能;

(2) 尝试通过Kubernetes部署探针容器,通过容器完成不同协议SDK的进程隔离乃至文件隔离;

(3) 通过Deployment设置容器环境参数的方式,给不同容器设置对应的启动参数;

(4) Kubernetes控制程序通过Apollo动态获取配置,更新相应协议的Deployment,从而更新相关容器镜像.

2. 依赖设置

pom文件中添加kubernetes以及apollo依赖:

        <dependency>
            <groupId>io.kubernetes</groupId>
            <artifactId>client-java</artifactId>
            <version>4.0.0</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>com.ctrip.framework.apollo</groupId>
            <artifactId>apollo-client</artifactId>
            <version>1.1.0</version>
        </dependency>

3. 部署Deployment

3.1 获取api client

注意,此处我们使用ExtensionsV1beta1Api ,对应api version: extensions/v1beta1.

在实际操作中,可以通过kubectl version获取api server支持的版本.

        ApiClient client;
        try {
            client = Config.defaultClient();
        } catch (IOException ex) {
            log.error("get k8s default client failed, error: [{}]", ex.getMessage());
            throw new UserException(AppStatus.INTERNAL_SERVER_ERROR, "can not get k8s default client.");
        }
        Configuration.setDefaultApiClient(client);
        ExtensionsV1beta1Api api = new ExtensionsV1beta1Api(client);

3.2 创建Deployment

Deployment创建需要设置如下内容:

• api version;
• kind;
• meta data;
• spec info.

不熟悉以上元素的,可以查阅官方文档,或者通过Kubernetes inAction一书了解.

        ExtensionsV1beta1Deployment edpDeployment = new ExtensionsV1beta1Deployment();
        edpDeployment.setApiVersion("extensions/v1beta1");
        edpDeployment.setKind("Deployment");
        edpDeployment.setMetadata(createDeploymentMeta(namespace, ProtocolType.EDP, config));
        edpDeployment.setSpec(createDeploymentSpec(ProtocolType.EDP, config));
        try {
            appsV1Api.createNamespacedDeployment(Common.INSPECTOR_NAMESPACE, edpDeployment, false, null, null);
        } catch (ApiException e) {
            log.error(e.getMessage());
        }

至此,可以通过kubectl相关命令查看Deployment的创建情况.

4. 更新Deployment

Deployment的更新操作比较晦涩,需要先构建ArrayList<JsonObject>存放更新操作以及相应数值(此处使用Gson),进而调用相应接口完成操作.

    /**
     * 构造Deployment的更新信息(json格式,需要指定操作类型,更新元素路径,以及更新后的数值)
     * 此处,仅示范如何更新pod中第一个容器(filebeta)的镜像ID
     * @return json信息
     */
    public ArrayList<JsonObject> getInspectorImagePatchElements() {
        // k8s java-client官方示例使用Gson
        Gson gson = new Gson();
        ArrayList<JsonObject> result = new ArrayList<>();
        // 更新Deployment中的filebeta容器镜像ID
        DeploymentPatchJson patchJson = new DeploymentPatchJson("replace",
                "/spec/template/spec/containers/0/image", appConfig.geFilebetaImageId());
        result.add((gson.fromJson(gson.toJson(patchJson), JsonElement.class)).getAsJsonObject());
        return result;
    }

    /**
     * 更新指定的Deployment
     *
     */
    public void patchCurrentDeployment(ExtensionsV1beta1Api api, ExtensionsV1beta1Deployment deployment) {
        ArrayList<JsonObject> patchElements = inspectorService.getInspectorImagePatchElements();
        try {
            api.patchNamespacedDeployment(deployment.getMetadata().getName(),
                    deployment.getMetadata().getNamespace(), patchElements, "true", null);
        } catch (ApiException e) {
            log.error("patch deployment failed, error: [{}]", e.getMessage());
        }
    }

5. 手动更新Deployment

(1) kubectl仿照调用api

kubectl patch deployment your_deployment -p \
  '{"spec":{"template":{"spec":{"terminationGracePeriodSeconds":31}}}}'

(2) 通过HTTP请求

curl --header "Content-Type: application/json-patch+json" \
--request PATCH \
--data '[{"op": "add", "path": "/status/capacity/example.com~1foo", "value": "5"}]' \
http://k8s-master:8080/api/v1/nodes/k8s-node-1/status

(3) kubectl设置

带*信息,请根据实际情况进行替换.

kubectl set image deployment/composer app=hub.***.com/online/composer:pro2019051602 --context=devops* -n qos*

(4) 直接编辑Deployment

kubectl edit deployment composer -n qos* --context=devops*

PS:

如果您觉得我的文章对您有帮助，请关注我的微信公众号，谢谢!