Docker-常用基建的安装与部署

一：Docker安装

　　1：通过yum安装docker

yum -y install gcc
yum -y install gcc-c++
yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum list docker-ce --showduplicates | sort -r
yum -y install docker-ce-18.06.3.ce

2：为docker设置源

vim /etc/docker/daemon.json

{
  "registry-mirrors": ["https://r5yuqwmu.mirror.aliyuncs.com"],
  "data-root": "/home/data/docker",
  "log-driver": "json-file",
  "log-opts": {"max-size":"50m", "max-file":"1"}
}

启动docker：systemctl start docker

二：docker镜像

https://docs.docker.com/engine/

获取docker镜像的两种方式：（以mysql为例）

1：docker pull mysql:5.7

从dockerhub中去拉取网上公开的可用的镜像，可以自己去搜索自己想要的：https://hub.docker.com/search?q=&type=image&category=database

2：docker build -t mysql_self .

自定义Dockerfile，可以基于公开的镜像，自定义自己的镜像。

#基础镜像使用 mysql:5.7
FROM mysql:5.7

#作者
MAINTAINER author

ENV WORK_PATH /usr/local/work

#定义会被容器自动执行的目录
ENV AUTO_RUN_DIR /docker-entrypoint-initdb.d

#定义sql文件名
ENV FILE_0 myself.sql
ENV LANG en_GB.utf8

#配置文件复制进去
COPY my.cnf /etc

RUN yes|cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

#定义shell文件名
ENV INSTALL_DB_SHELL install_db.sh

#创建文件夹
RUN mkdir -p $WORK_PATH

#把数据库初始化数据的文件复制到工作目录下
COPY ./$FILE_0 $WORK_PATH/

#把要执行的shell文件放到/docker-entrypoint-initdb.d/目录下，容器会自动执行这个shell
COPY ./$INSTALL_DB_SHELL $AUTO_RUN_DIR/
WORKDIR $WORK_PATH

#给执行文件增加可执行权限
RUN chmod a+x $AUTO_RUN_DIR/$INSTALL_DB_SHELL

三：docker容器

当镜像都下载完成后，就可以创建docker容器并运行使用，下面就以实例测试mysql、canal、kafka和zookeeper这几个基建的部署。

# 创建自定义network
docker network create --subnet=172.18.0.0/16 mynetwork

docker run options 可以参考：https://docs.docker.com/engine/reference/commandline/run/

1：运行mysql容器

docker run -p 3306:3306 --name mysql_self --net mynetwork --ip 172.18.0.36  -v /home/data/docker/mysql/logs:/logs -v /home/data/docker/mysql/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=Abcd12345 -d --restart=always mysql_self:latest

2：canal

docker run -d --name canal -p 11111:11111 --net mynetwork --ip 172.18.0.41 -e canal.destinations=self -e canal.instance.mysql.slaveId=12 -e  canal.auto.scan=false -e canal.instance.master.address=172.18.0.36:3306 -e canal.instance.dbUsername=canal -e canal.instance.dbPassword=canal -e  canal.instance.filter.regex=esen_approval.apt_approval --restart=always canal/canal-server:v1.1.4

canal要想搭配好mysql，其中有几点需要注意；
1：canal.instance.master.address的地址必须指向mysql容器的ip和port

2：canal.instance.mysql.slaveId 不能与mysql的一致，mysql的log_bin要开启 ，查看命令为：show variables like 'log_bin'

3：canal.instance.dbUsername和canal.instance.dbPassword 要在启动前设置好。

3：zookeeper

docker run -d --name zookeeper -p 2181:2181 --net mynetwork --ip 172.18.0.37 -v /home/data/docker/zookeeper/logs:/datalog -v /home/data/docker/zookeeper/data:/data --restart=always -t  zookeeper:3.4.14

4：kafka

docker run -d --name kafka -p 9092:9092 -p 19092:19092 --net mynetwork --ip 172.18.0.38  -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.37:2181 -e KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT -e KAFKA_ADVERTISED_LISTENERS=INTERNAL://172.18.0.38:19092,EXTERNAL://外网IP:9092 -e KAFKA_LISTENERS=INTERNAL://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092 --restart=always -t wurstmeister/kafka

四：docker-compose

1：docker-compose安装

sudo curl -L "https://get.daocloud.io/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

docker-compose file 和docker版本之间的对照关系：https://docs.docker.com/compose/compose-file/

2：docker-compose.yml

version: '3.7'
services:
  mysql_hermes:
    build: /root/data/docker/mysql/mysql_hermes
    container_name: mysql_hermes
    environment:
      - "MYSQL_ROOT_PASSWORD=Abcd12345"
    ports:
      - "3306:3306"
    restart: always
    volumes:
        - "/root/data/docker/mysql/logs:/logs"
        - "/root/data/docker/mysql/data:/var/lib/mysql"
  zookeeper:
    image: zookeeper:3.4.14
    container_name: zookeeper
    restart: always
    hostname: zookeeper
    ports:
      - 2181:2181
    volumes:
      - /root/data/docker/zookeeper/logs:/datalog
      - /root/data/docker/zookeeper/data:/data
    networks:
        default:
            ipv4_address: 172.18.0.37
  kafka:
    image: wurstmeister/kafka:latest
    container_name: kafka
    restart: always
    ports:
      - 9092:9092
      - 19092:19092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 172.18.0.37:2181
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://172.18.0.38:19092,EXTERNAL://47.99.72.114:9092
      KAFKA_LISTENERS: INTERNAL://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092
    volumes:
      - /root/data/docker/kafka/data:/kafka
    depends_on:
      - zookeeper
    networks:
        default:
            ipv4_address: 172.18.0.38
  canal:
    image: canal/canal-server:v1.1.4
    restart: always
    container_name: canal
    depends_on:
      - mysql_hermes
    ports:
      - 11111:11111
    environment:
      - canal.instance.mysql.slaveId=12
      - canal.auto.scan=false
      - canal.destinations=hermes
      - canal.instance.master.address=mysql_hermes:3306
      - canal.instance.dbUsername=canal
      - canal.instance.dbPassword=canal
      - canal.instance.filter.regex=esen_approval.apt_approval
networks:
   default:
      ipam:
         config:
         - subnet: 172.18.0.0/16

启动命令：docker-compose -f xx.yml up -d

停止命令：docker-compose -f xx.yml down