sqler 集成 terraform v0.12 生成资源部署文件

terraform v0.12 发布了，有好多新功能的添加，包括语法的增强，新函数的引入，更好的开发提示
只是当前对于一些老版本的provider 暂时还不兼容，但是大部分官方的provider 都是可以使用的

这片文章只是一个简单的demo，使用sqler 提供rest api，集合tf 0.12 的新功能，使用模版函数以及 json 解码函数
实现一个部署资源的动态生成

环境准备

tf 使用本地软件，sqler 以及数据库使用docker-compose 运行

• 软件下载

 

tf 下载 https://www.terraform.io/downloads.html

• 项目结构

├── main.tf

├── restapi

│ ├── config

│ │ └── config-example.hcl

│ ├── docker-compose.yaml

│ └── user-app.sql

├── terraform.tfstate

├── tf-result

└── user_apps.tmpl

 

 

• sqler 环境
  docker-compose 文件

 

version: "3"

services:

  sqler:

    image: dalongrong/sqler:2.0

    volumes:

    - "./config/config-example.hcl:/app/config.example.hcl"

    environment:

    - "DRIVER=postgres" 

    - "DSN=postgresql://postgres:dalong@postgres:5432/postgres?sslmode=disable"

    ports:

    - "3678:3678"

    - "8025:8025"

  postgres:

    image: postgres:9.6.11

    ports:

    - "5432:5432"

    environment:

    - "POSTGRES_PASSWORD:dalong"

 

代码说明

• sqler 数据库查询dsl

  config/config-example.hcl

apps {

    exec = <<SQL

    select * from apps

    SQL

}

users {

    exec = <<SQL

     select * from users;

    SQL

}

user_apps {

    aggregate = ["users","apps"]

}

• pg sql

  几个测试的数据表

CREATE TABLE apps (

    id SERIAL PRIMARY KEY,

    appname text,

    appversion text,

    content text

);

CREATE TABLE users (

    id SERIAL PRIMARY KEY,

    username text,

    age integer,

    content text

);

​

INSERT INTO "public"."apps"("appname", "appversion", "content") VALUES('login', 'v1.0', 'login') RETURNING "id", "appname", "appversion", "content";

INSERT INTO "public"."apps"("appname", "appversion") VALUES('logo', 'v2.0') RETURNING "id", "appname", "appversion", "content";

​

INSERT INTO "public"."users"("username", "age", "content") VALUES('dalong', 22, 'demo') RETURNING "id", "username", "age", "content";

INSERT INTO "public"."users"("username", "age") VALUES('rong', 29) RETURNING "id", "username", "age", "content";

 

• tf main.tf

  主要是集成模版函数以及http provider 还有jsondecode 函数

main.tf

variable filename {

  type = string

  default = "tf-result/main.tf"

}

data "http" "user_apps" {

  url = "http://localhost:8025/user_apps"

}

resource "local_file" "main_tf" {

    content = templatefile("user_apps.tmpl", jsondecode(data.http.user_apps.body))

    filename = var.filename

}

user_apps.tmpl

%{ for item in data.apps ~}

resource "users" "${item.appname}" {

    appname = "${item.appname}"

    version = "${item.appversion}"

    content = "${item.content != null ? item.content : ""}"

}

%{ endfor ~}

​

%{ for item in data.users ~}

resource "apps" "${item.username}" {

    username = "${item.username}"

    age = "${item.age}"

    content = "${item.content != null ? item.content : ""}"

}

%{ endfor ~}

运行&&测试

• 启动sqler

cd restapi && docker-compose up -d

• 运行tf

init ：

下载依赖的provider

terraform init 

 

plan：

查询tf 的任务

terraform  plan

效果

Refreshing Terraform state in-memory prior to plan...

The refreshed state will be used to calculate this plan, but will not be

persisted to local or remote state storage.

​

data.http.user_apps: Refreshing state...

​

------------------------------------------------------------------------

​

An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  + create

​

Terraform will perform the following actions:

​

  # local_file.main_tf will be created

  + resource "local_file" "main_tf" {

      + content = "resource \"users\" \"login\" {\n appname = \"login\"\n version = \"v1.0\"\n content = \"login\"\n}\nresource \"users\" \"logo\" {\n appname = \"logo\"\n version = \"v2.0\"\n content = \"\"\n}\n\nresource \"apps\" \"dalong\" {\n username = \"dalong\"\n age = \"22\"\n content = \"demo\"\n}\nresource \"apps\" \"rong\" {\n username = \"rong\"\n age = \"29\"\n content = \"\"\n}\n"

      + filename = "tf-result/main.tf"

      + id = (known after apply)

    }

​

Plan: 1 to add, 0 to change, 0 to destroy.

​

------------------------------------------------------------------------

​

Note: You didn't specify an "-out" parameter to save this plan, so Terraform

can't guarantee that exactly these actions will be performed if

"terraform apply" is subsequently run.

apply:

data.http.user_apps: Refreshing state...

​

An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  + create

​

Terraform will perform the following actions:

​

  # local_file.main_tf will be created

  + resource "local_file" "main_tf" {

      + content = "resource \"users\" \"login\" {\n appname = \"login\"\n version = \"v1.0\"\n content = \"login\"\n}\nresource \"users\" \

"logo\" {\n appname = \"logo\"\n version = \"v2.0\"\n content = \"\"\n}\n\nresource \"apps\" \"dalong\" {\n username = \"dalong\"\n ag

e = \"22\"\n content = \"demo\"\n}\nresource \"apps\" \"rong\" {\n username = \"rong\"\n age = \"29\"\n content = \"\"\n}\n"

      + filename = "tf-result/main.tf"

      + id = (known after apply)

    }

​

Plan: 1 to add, 0 to change, 0 to destroy.

​

Do you want to perform these actions?

  Terraform will perform the actions described above.

  Only 'yes' will be accepted to approve.

​

  Enter a value: yes

​

local_file.main_tf: Creating...

local_file.main_tf: Creation complete after 0s [id=ee0a2ad2f12e1e21723c46248af40d5cc53e46cd]

​

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

 

• 生成的模版内容

resource "users" "login" {

    appname = "login"

    version = "v1.0"

    content = "login"

}

resource "users" "logo" {

    appname = "logo"

    version = "v2.0"

    content = ""

}

​

resource "apps" "dalong" {

    username = "dalong"

    age = "22"

    content = "demo"

}

resource "apps" "rong" {

    username = "rong"

    age = "29"

    content = ""

}

说明

以上只是一个简单的集成，实际上我们可以集合数据库的管理，做为配置管理，同时集成安全认证，使用slqer 快速提供数据接口
使用tf 进行资源部署，同时可以使用tf 的状态管理，实现技术设施资源的快速部署以及管理，还是很不错的

参考资料

https://github.com/rongfengliang/sqler-docker-compose
https://www.terraform.io/docs/cli-index.html
https://www.terraform.io/docs/configuration/functions/templatefile.html
https://www.terraform.io/docs/providers/http/index.html
https://github.com/rongfengliang/sqler-terraform-resource-demo