redis,docker和kubernetes留言簿案例实战

kubernetes实战-基于redis和docker的留言簿案例

[置顶] kubernetes实战-基于redis和docker的留言簿案例

标签： dockerkubernetesredis

2015-06-22 19:41 7570人阅读 评论(5) 收藏 举报

本文章已收录于：

 Docker知识库

 分类：

docker（7） 

版权声明：本文为博主原创文章，未经博主允许不得转载。

一、介绍

本案例基于Kubernetes和Docker，其中包括

1、web前端

2、redis master

3、redis slave

其中web前端通过javascript redis api和redis master交互

 

kubernetes体系架构

 

二、配置

0、先决条件

Kubernetes 集群

 

1、启动redis master

使用replication controller确保只有一个pod在运行（当某个节点down了，rc会在另一个健康的node启动redis master），但可能会有数据丢失。

[root@centos1 example]# kubectl create -f redis-master-controller.json

replicationcontrollers/redis-master

[root@centos1 example]# kubectl get rc

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

redis-master master redis name=redis-master 1

验证master运行成功，如下展示了pod运行在centos2/192.168.1.112这台机器上    

[root@centos1 example]# kubectl get pods

POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE

redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running 55 seconds

master redis Running 55 seconds

SSH到centos2/192.168.1.112查看docker状态

[root@centos2 yum.repos.d]# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

91689ce56668 redis:latest "/entrypoint.sh redi 3 minutes ago Up 3 minutes k8s_master.52732b08_redis-master-svar7_default_5b6d5485-1894-11e5-b3ad-000c293c8c19_97e79b7b

38c3180813c3 gcr.io/google_containers/pause:0.8.0 "/pause" 3 minutes ago Up 3 minutes k8s_POD.49eee8c2_redis-master-svar7_default_5b6d5485-1894-11e5-b3ad-000c293c8c19_298e038f

注意：kubectl create执行后，如果镜像不存在，会执行docker pull，根据网络情况，下载中的pods 在kubertnetes UI上会显示pending状态

 

2、启动master service

一个kubernetes service会对一个或多个container进行负载均衡，这是通过我们上面redis-master中定义的labels元数据实现的，值得注意的是，在redis中只有一个master，但是我们依然为它创建一个service，这是因为这样我们就能使用一个elastic IP来路由到具体某一个master。

kubernetes集群中的service是通过container中的环境变量实现服务发现的，service基于pod label实现container的负载均衡。

在第一步中创建的pod包含了一个label“name=redis-master”，service的selector字段决定了service将流量转发给哪个pod，port和targetPort信息定义了service proxy运行在什么端口。

[root@centos1 example]# kubectl create -f redis-master-service.json

services/redis-master

[root@centos1 example]# kubectl get services

NAME LABELS SELECTOR IP(S) PORT(S)

redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP

上面的运行成功后，所有pods都能发现redis master运行在6379端口，从salve到master流量走向会有以下两步：

1、一个redis slave会连接到redis master service的port上

2、流量会从service节点上的port到targetPort

如果targetPort未指定，默认和port一致

 

3、启动replicated slave pod

虽然redis master是一个单独的pod，redis slaves是一个replicated pod，在Kubernetes中，一个Replication Controller负责管理一个replicated pod的多个实例，RC会自动拉起down掉的replica（可以通过杀死docker 进程方式简单测试）

[root@centos1 example]# kubectl create -f redis-slave-controller.json

replicationcontrollers/redis-slave

[root@centos1 example]# kubectl get rc

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

redis-master master redis name=redis-master 1

redis-slave slave kubernetes/redis-slave:v2 name=redis-slave 2

[root@centos1 example]# kubectl get pods

POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE

redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running 41 minutes

master redis Running 41 minutes

redis-slave-31tkb 172.17.0.10 centos2/192.168.1.112 name=redis-slave Running 29 seconds

slave kubernetes/redis-slave:v2 Running 28 seconds

redis-slave-uk8nu 172.17.0.11 centos2/192.168.1.112 name=redis-slave Running 29 seconds

slave kubernetes/redis-slave:v2 Running 28 seconds

可以看到一个master pod和两个slave pod

 

4、启动slave service

和master一样，我们希望有一个代理服务连接到redis slave，除了服务发现之外，slave service还为web app client提供了透明代理。

这次service 的selector是name=redis-slave，我们可以方便的使用kubectl get services -l "label=value"命令来定位这些服务

[root@centos1 example]# kubectl create -f redis-slave-service.json

services/redis-slave

[root@centos1 example]# kubectl get services

NAME LABELS SELECTOR IP(S) PORT(S)

redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP

redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP

 

5、创建frontend pod

这是一个简单的PHP 服务，用来和master service（写请求）或slave service（读请求）交互

[root@centos1 example]# kubectl create -f frontend-controller.json

replicationcontrollers/frontend

[root@centos1 example]# kubectl get rc

CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS

frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3

redis-master master redis name=redis-master 1

redis-slave slave kubernetes/redis-slave:v2 name=redis-slave 2

运行成功后，查看当前pod运行状态

[root@centos1 example]# kubectl get pods

POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE

frontend-fr5z1 172.17.0.13 centos2/192.168.1.112 name=frontend Running 2 minutes

php-redis kubernetes/example-guestbook-php-redis:v2 Running 2 minutes

frontend-gjx3t 172.17.0.14 centos2/192.168.1.112 name=frontend Running 2 minutes

php-redis kubernetes/example-guestbook-php-redis:v2 Running 2 minutes

frontend-v608r 172.17.0.12 centos2/192.168.1.112 name=frontend Running 2 minutes

php-redis kubernetes/example-guestbook-php-redis:v2 Running 2 minutes

redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running 53 minutes

master redis Running 53 minutes

redis-slave-31tkb 172.17.0.10 centos2/192.168.1.112 name=redis-slave Running 12 minutes

slave kubernetes/redis-slave:v2 Running 12 minutes

redis-slave-uk8nu 172.17.0.11 centos2/192.168.1.112 name=redis-slave Running 12 minutes

slave kubernetes/redis-slave:v2 Running 12 minutes

可以看到一个redis master，两个redis slave和三个frontend pods

 

6、创建guestbook service

和其他service一样，你可以创建一个service管理frontend pods

[root@centos1 example]# kubectl create -f frontend-service.json

services/frontend

[root@centos1 example]# kubectl get services

NAME LABELS SELECTOR IP(S) PORT(S)

frontend name=frontend name=frontend 10.254.154.111 80/TCP

redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP

redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP

我们可以通过frontend service（10.254.154.111）访问pods，但是这个IP明显是无法在外部访问的，下一节讲解如何在外部网络访问guestbook

 

7、外部网络访问guestbook

kubernetes 支持两种访问暴露一个服务到外部IP地址NodePorts 和LoadBalancers，https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/services.md#external-services

另外可以查看防火墙，找到service对应的端口，如下

[root@centos1 example]# kubectl get pods,services

POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE

frontend-fr5z1 172.17.0.13 centos2/192.168.1.112 name=frontend Running 22 minutes

php-redis kubernetes/example-guestbook-php-redis:v2 Running 22 minutes

frontend-gjx3t 172.17.0.14 centos2/192.168.1.112 name=frontend Running 22 minutes

php-redis kubernetes/example-guestbook-php-redis:v2 Running 22 minutes

frontend-v608r 172.17.0.12 centos2/192.168.1.112 name=frontend Running 22 minutes

php-redis kubernetes/example-guestbook-php-redis:v2 Running 22 minutes

redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running About an hour

master redis Running About an hour

redis-slave-31tkb 172.17.0.10 centos2/192.168.1.112 name=redis-slave Running 32 minutes

slave kubernetes/redis-slave:v2 Running 32 minutes

redis-slave-uk8nu 172.17.0.11 centos2/192.168.1.112 name=redis-slave Running 32 minutes

slave kubernetes/redis-slave:v2 Running 32 minutes

NAME LABELS SELECTOR IP(S) PORT(S)

frontend name=frontend name=frontend 10.254.154.111 80/TCP

kubernetes component=apiserver,provider=kubernetes <none> 10.254.0.2 443/TCP

kubernetes-ro component=apiserver,provider=kubernetes <none> 10.254.0.1 80/TCP

redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP

redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP

[root@centos1 example]# kubectl get services

NAME LABELS SELECTOR IP(S) PORT(S)

frontend name=frontend name=frontend 10.254.154.111 80/TCP

kubernetes component=apiserver,provider=kubernetes <none> 10.254.0.2 443/TCP

kubernetes-ro component=apiserver,provider=kubernetes <none> 10.254.0.1 80/TCP

redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP

redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP

发现redis-master是在10.254.154.90上，登录到centos2上，执行iptables-save，发现其中有这样一条规则

-A KUBE-PORTALS-HOST -d 10.254.154.90/32 -p tcp -m comment --comment "default/redis-master:" -m tcp --dport 6379 -j DNAT --to-destination 192.168.1.112:49038

说明，centos2本机的49038端口映射到master container内的6379了，当然我们就能执行在本机访问redis了

[root@centos2 yum.repos.d]# redis-cli -p 49038

127.0.0.1:49038> set a b

OK

127.0.0.1:49038> get a

"b"

 

8、使用curl简单测试

提交数据

curl "localhost:8000/index.php?cmd=set&key=messages&value=jay_sais_hi"

查询数据

curl "localhost:8000/index.php?cmd=get&key=messages"

 

附本案例用到的6个.json文件

1、redis-master-controller.json

{

"kind":"ReplicationController",

"apiVersion":"v1beta3",

"metadata":{

"name":"redis-master",

"labels":{

"name":"redis-master"

}

},

"spec":{

"replicas":1,

"selector":{

"name":"redis-master"

},

"template":{

"metadata":{

"labels":{

"name":"redis-master"

}

},

"spec":{

"containers":[

{

"name":"master",

"image":"redis",

"ports":[

{

"containerPort":6379

}

]

}

]

}

}

}

}

2、redis-master-service.json

{

"kind":"Service",

"apiVersion":"v1beta3",

"metadata":{

"name":"redis-master",

"labels":{

"name":"redis-master"

}

},

"spec":{

"ports": [

{

"port":6379,

"targetPort":6379

}

],

"selector":{

"name":"redis-master"

}

}

}

3、redis-slave-controller.json

{

"kind":"ReplicationController",

"apiVersion":"v1beta3",

"metadata":{

"name":"redis-slave",

"labels":{

"name":"redis-slave"

}

},

"spec":{

"replicas":2,

"selector":{

"name":"redis-slave"

},

"template":{

"metadata":{

"labels":{

"name":"redis-slave"

}

},

"spec":{

"containers":[

{

"name":"slave",

"image":"kubernetes/redis-slave:v2",

"ports":[

{

"containerPort":6379

}

]

}

]

}

}

}

}

4、redis-slave-service.json

{

"kind":"Service",

"apiVersion":"v1beta3",

"metadata":{

"name":"redis-slave",

"labels":{

"name":"redis-slave"

}

},

"spec":{

"ports": [

{

"port":6379

}

],

"selector":{

"name":"redis-slave"

}

}

}

5、frontend-controller.json

{

"kind":"ReplicationController",

"apiVersion":"v1beta3",

"metadata":{

"name":"frontend",

"labels":{

"name":"frontend"

}

},

"spec":{

"replicas":3,

"selector":{

"name":"frontend"

},

"template":{

"metadata":{

"labels":{

"name":"frontend"

}

},

"spec":{

"containers":[

{

"name":"php-redis",

"image":"kubernetes/example-guestbook-php-redis:v2",

"ports":[

{

"containerPort":80

}

]

}

]

}

}

}

}

6、frontend-service.json

{

"kind":"Service",

"apiVersion":"v1beta3",

"metadata":{

"name":"frontend",

"labels":{

"name":"frontend"

}

},

"spec":{

"ports": [

{

"port":80

}

],

"selector":{

"name":"frontend"

}

}

}