容器基础(五): 实现一个简单容器sdocker

在前面几部分的基础上, 我们更新一下代码，实现一个简单容器 sdocker.

sdocker目录构成

linux: # tree
.
├── Makefile
├── cpu-test.c       # 由cpu.c重命名
├── memory-test.cpp  # 由memory.cpp重命名
├── resource.c       # 新增用于资源管理
├── resource.h
├── sdocker_exec.c   # 模拟 docker exec
└── sdocker_run.c    # 由namespace.c更新而来, 模拟 docker run

 directories,  files
linux: #

修改点

. 更新rootfs:
  a) 拷贝cpu-test/memory-test到rootfs/bin目录下, 对应的依赖库也按照先前文章介绍的方法拷贝到rootfs对应路径，方便后续进行cpu和memory测试;
  b) rootfs增加拷贝top命令, 方便查看cpu使用率;
  c) 增加rootfs/root/.bashrc, 用于设置容器的提示符, 内容如下所示:
    linux:~ # cat /var/lib/sdocker/rootfs/root/.bashrc
    export PS1="\h:\w $ "
    linux:~ #

  sdocker启动后的终端提示符如下所示(主机名@容器id: 路径)：
  sdocker@:/ $ 

. 增加sdocker_run程序:
  a) 增加cgroup资源管理文件resource.c, 可以设置memory.limit_in_bytes, memory.swappiness, cpu.cfs_period_us, cpu.cfs_quota_us.
  资源管理会在启动容器进程后在/sys/fs/cgroup/{cpu,memory}下建立目录名为容器id的目录, 程序结束后删除.
  b) 增加信号量和共享内存, 用于主进程和容器进程间传递数据(比如容器id和rootfs路径);
  c) 把proc, sys, dev, dev/pts这些都挂载到容器;
  d) 默认启动的容器mount-propagation type是MS_SHARED, 容器进程mount操作在主机可见, 为了让容器mount操作主机不可见, 设置容器的propagation type为MS_PRIVATE;

. 增加sdocker_exec程序, 通过注入namespace方式实现类似docker exec功能

基础验证

########### 启动容器, 默认执行bash ###########
linux: # ./sdocker_run -h

Usage:  sdocker [OPTIONS] IMAGE

Start a container using bash default

Options:
-c, --cpu-test               start cpu test with high cpu program
-m, --memory-test            start memory test with high memory program
-n, --hostname string        docker hostname, default is sdocker
    --memory-limit bytes     Memory limit, file: memory.limit_in_bytes
    --memory-swappiness int  Tune container memory swappiness ( to ) , file: memory.swappiness
    --cpu-period int         Limit CPU CFS (Completely Fair Scheduler) period, file: cpu.cfs_period_us
    --cpu-quota int          Limit CPU CFS (Completely Fair Scheduler) quota, file: cpu.cfs_quota_us
-h, --help                   show help

linux: # ./sdocker_run /var/lib/sdocker/rootfs/
[+] start container with id
sdocker@:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
                    : pts/    :: /bin/bash
                    : pts/    :: ps -ef
sdocker@:/ $

########### 启动另一个终端查看 ###########
linux: # ls /sys/fs/cgroup/{cpu,memory}
/sys/fs/cgroup/cpu:
                  cpu.cfs_period_us  cpu.shares     cpuacct.usage_percpu
cgroup.clone_children  cpu.cfs_quota_us   cpu.stat       notify_on_release
cgroup.procs           cpu.rt_period_us   cpuacct.stat   release_agent
cgroup.sane_behavior   cpu.rt_runtime_us  cpuacct.usage  tasks

/sys/fs/cgroup/memory:
                  memory.low_limit_in_bytes        memory.swappiness
cgroup.clone_children  memory.max_usage_in_bytes        memory.usage_in_bytes
cgroup.event_control   memory.move_charge_at_immigrate  memory.use_hierarchy
cgroup.procs           memory.numa_stat                 notify_on_release
cgroup.sane_behavior   memory.oom_control               release_agent
memory.failcnt         memory.pressure_level            tasks
memory.force_empty     memory.soft_limit_in_bytes
memory.limit_in_bytes  memory.stat
linux: # ./sdocker_exec  /var/lib/sdocker/rootfs/
sdocker@:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
                    : pts/    :: /bin/bash
                    : pts/    :: bash
                    : pts/    :: ps -ef
sdocker@:/ $ exit
exit
linux: #

无资源限制测试cpu-test

########### 启动容器执行cpu-test, 理论cpu占用要到100% ###########
linux: # ./sdocker_run -c /var/lib/sdocker/rootfs/
[+] start container with id 

########### 启动另一个终端查看 ###########
linux: # ./sdocker_exec  /var/lib/sdocker/rootfs/
sdocker@:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
                   : pts/    :: /bin/cpu-test
                    : pts/    :: bash
                    : pts/    :: ps -ef
$sdocker@:/ $ top
top - :: up  day,  min,   users,  load average: 0.96, 0.47, 0.19
Tasks:    total,    running,    sleeping,    stopped,    zombie
%Cpu(s):  0.1 us,  0.0 sy,  0.0 ni, 99.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   total,   used,  free,      buffers
KiB Swap:  total,         used,  free.    cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S   %CPU  %MEM     TIME+ COMMAND
                              R 100.00 0.001   :03.84 cpu-test
                           S  0.000 0.009   :00.00 bash
                           R  0.000 0.006   :00.00 top
sdocker@:/ $ exit
exit
linux: #

有资源限制测试cpu-test, 限制cpu使用率20%

########### 启动容器执行cpu-test, 设置cpu占用率不超过20% ###########
linux: # ./sdocker_run -c --cpu-quota  /var/lib/sdocker/rootfs/
[+] start container with id 

########### 启动另一个终端查看 ###########
linux: # ./sdocker_exec  /var/lib/sdocker/rootfs/
sdocker@:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
                   : pts/    :: /bin/cpu-test
                    : pts/    :: bash
                    : pts/    :: ps -ef
sdocker@:/ $ top
top - :: up  day,  min,   users,  load average: 0.31, 0.66, 0.40
Tasks:    total,    running,    sleeping,    stopped,    zombie
%Cpu(s):  1.2 us,  0.0 sy,  0.0 ni, 98.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   total,   used,  free,      buffers
KiB Swap:  total,         used,  free.    cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S   %CPU  %MEM     TIME+ COMMAND
                              R 20.333 0.001   :06.46 cpu-test
                           S  0.000 0.009   :00.00 bash
                           R  0.000 0.006   :00.00 top
sdocker@:/ $ exit
exit
linux: #

有资源限制测试memory-test, 设置内存使用上限10M, 并关闭 swap

########### 启动容器执行memory-test, 设置memory使用不超过10M, 一段时间后容器触发oom退出 ###########
linux: # ./sdocker_run --memory-test --memory-limit   --memory-swappiness  /var/lib/sdocker/rootfs/
[+] start container with id
[+] malloc size:
[+] malloc size:
[+] malloc size:
[+] malloc size:
[+] malloc size:
[+] malloc size:
[+] malloc size:
[+] malloc size:
[+] malloc size:
linux: # ps -ef | grep sdocker | grep -v grep
linux: #

########### 启动另一个终端查看, 一段时间后容器退出 ###########
linux: # ./sdocker_exec  /var/lib/sdocker/rootfs/
sdocker@:/ $ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
                    : pts/    :: /bin/memory-test
                    : pts/    :: bash
                    : pts/    :: ps -ef
sdocker@:/ $