kolla-build镜像时，问题汇总

记录下kolla-build镜像时，遇到的一些问题，既为了方便自己以后问题的查找，也为了帮助别人避免踩这些坑。遇到的问题会持续更新在博客里面。

问题1:使用的kolla 版本是ocata版本，本地已经搭建好yum源，并且在以前制作镜像的时候，一直是好的，今天再次编译镜像，一直报如下的错误

INFO:kolla.image.build.base:---> Package pciutils-libs.x86_64 0:3.5.1-3.el7 will be installed
INFO:kolla.image.build.base:---> Package perl-HTTP-Tiny.noarch 0:0.033-3.el7 will be installed
INFO:kolla.image.build.base:---> Package perl-parent.noarch 1:0.225-244.el7 will be installed
INFO:kolla.image.build.base:---> Package sysvinit-tools.x86_64 0:2.88-14.dsf.el7 will be installed
INFO:kolla.image.build.base:--> Finished Dependency Resolution
INFO:kolla.image.build.base:Error: Package: 7:device-mapper-event-1.02.146-4.el7.x86_64 (base)
INFO:kolla.image.build.base:           Requires: device-mapper = 7:1.02.146-4.el7
INFO:kolla.image.build.base:           Installed: 7:device-mapper-1.02.149-10.el7_6.2.x86_64 (@Updates)
INFO:kolla.image.build.base:               device-mapper = 7:1.02.149-10.el7_6.2
INFO:kolla.image.build.base:           Available: 7:device-mapper-1.02.146-4.el7.x86_64 (base)
INFO:kolla.image.build.base:               device-mapper = 7:1.02.146-4.el7
INFO:kolla.image.build.base:
INFO:kolla.image.build.base: You could try using --skip-broken to work around the problem
INFO:kolla.image.build.base: You could try running: rpm -Va --nofiles --nodigest

原因分析：遇到这个问题的时候，一开始以为自己的yun源出现了问题，所以去本地yum源里面去查找是否有device-mapper-event-1.02.146-4.el7.x86_64这个rpm包，在base/update里面，检查了一遍都存在这个包。后来怀疑是centos基础镜像有问题，就使用docker run -it centos启动了一个容器，在容器里面使用rpm -qa |grep device命令查看的时候，发现系统里面安装了device-mapper-1.02.149-10.el7_6.2.x86_64。kolla build编译进行需要的是device-mapper-event-1.02.146-4.el7.x86_64这个rpm包，但是系统已经安装了149的版本，造成了这种问题。处理方式时，幸亏以前搭建的kolla build服务器还在，把对应的centos基础镜像拷贝过来，导入到该环境里，启动一个容器以后，再次检查他的devcie相关包，发现确实是146的版本，所以以前的编译没有问题。

[root@localhost base]# docker run -it 75835a67d134 /bin/bash
[root@730fd71696a2 /]# rpm -qa |grep device
device-mapper-1.02.146-4.el7.x86_64
device-mapper-libs-1.02.146-4.el7.x86_64
[root@730fd71696a2 /]#

导入基础centos镜像以后，在编译镜像时，使用特定的私有仓库中的基础镜像

kolla-build -b centos --base-tag 7 --base-image 172.19.146.34:5001/centos/centos --tag ocata --type source keystone

问题的根因是kolla-build -b centos --base-tag 7 --tag ocata --type source keystone命令行执行的时候，每次都去官网拉去一遍centos基础镜像，再进行下一步的操作。而官网的centos基础镜像几乎每天都有新的rpm更新，导致与本地的yum源不匹配，造成异常

教训：在kolla-build镜像的时候，搭建自己的yum源，并且成功编译出镜像时，一定要把基础镜像保存一份，以后一直用这个基础镜像，否则一直拉取官网的最新镜像，在编译openstack镜像时，会造成各种异常的情况，并且一直更新yum源也很费事

问题2：kolla-ansible -i ./all-in-one pull 编译的镜像时，一直提示如下的错误

TASK [common : include] *******************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/common/tasks/bootstrap.yml for localhost

TASK [common : Creating log volume] *****************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n  
File \"/tmp/ansible_7mwoMR/ansible_module_kolla_docker.py\", line 802, in main\\n    dw = DockerWorker(module)\\n  
File \"/tmp/ansible_7mwoMR/ansible_module_kolla_docker.py\", line 221, in __init__\\n    self.dc = get_docker_client()(**options)\\n  
File \"/usr/lib/python2.7/site-packages/docker/client.py\", line 99, in __init__\\n    self._version = self._retrieve_server_version()\\n  
File \"/usr/lib/python2.7/site-packages/docker/client.py\", line 124, in _retrieve_server_version\\n    
\\'Error while fetching server API version: {0}\\'.format(e)\\n
DockerException: Error while fetching server API version: Timeout value connect was Timeout(connect=60, read=60, total=None), 
but it must be an int, float or None.\\n'"}
	to retry, use: --limit @/usr/share/kolla-ansible/ansible/site.retry

PLAY RECAP *******************************************************************************************************************************************
localhost                  : ok=19   changed=8    unreachable=0    failed=1

　原因分析：第一次拉取不管是官方镜像还是自己源码编译的镜像，都能成功的部署起来，但是执行命令kolla-ansible destroy --include-images --yes-i-really-really-mean-it把

部署的容器和缓存的镜像去掉，重新部署的时候，一直报以前的错误。后来谷歌了一下，是requests版本太高的问题。方法是把requests的版本降到requests==2.17.3就可以了

过程如下：

[root@controller ~]# pip list |grep requests
requests                         2.19.1
requestsexceptions               1.4.0
[root@controller ~]#
[root@controller ~]# pip uninstall requests
Uninstalling requests-2.19.1:
  Would remove:
    /usr/lib/python2.7/site-packages/requests-2.19.1.dist-info/*
    /usr/lib/python2.7/site-packages/requests/*
  Would not remove (might be manually added):
    /usr/lib/python2.7/site-packages/requests/packages/__init__.py
    /usr/lib/python2.7/site-packages/requests/packages/__init__.pyo
Proceed (y/n)? y
  Successfully uninstalled requests-2.19.1
[root@controller ~]#
[root@controller ~]# pip install requests==2.17.3

　备注：requests 2.18.xx　系列也有这个问题，为了保险起见，最好直接使用requests==2.17.3或者使用2.20.1

[root@localhost ~]# pip list |grep requests
requests                         2.20.1
requestsexceptions               1.4.0
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# python
Python 2.7.5 (default, Jul 13 2018, 13:06:57)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import docker
>>> c = docker.Client(base_url='unix://var/run/docker.sock', version='1.15', timeout=2.0)
>>> l = c.containers(all=True)
>>> quit()
[root@localhost ~]# 

问题3：kolla-ansible部署完以后，执行openstack 命令报如下错误

[root@controller ~]# source /etc/kolla/admin-openrc.sh
[root@controller ~]# openstack service list
Traceback (most recent call last):
  File "/usr/bin/openstack", line 7, in <module>
    from openstackclient.shell import main
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 23, in <module>
    from osc_lib import shell
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 33, in <module>
    from osc_lib.cli import client_config as cloud_config
  File "/usr/lib/python2.7/site-packages/osc_lib/cli/client_config.py", line 18, in <module>
    from openstack.config import exceptions as sdk_exceptions
  File "/usr/lib/python2.7/site-packages/openstack/__init__.py", line 17, in <module>
    import openstack.connection
  File "/usr/lib/python2.7/site-packages/openstack/connection.py", line 166, in <module>
    from openstack import cloud as _cloud
  File "/usr/lib/python2.7/site-packages/openstack/cloud/__init__.py", line 20, in <module>
    from openstack.cloud.openstackcloud import OpenStackCloud
  File "/usr/lib/python2.7/site-packages/openstack/cloud/openstackcloud.py", line 49, in <module>
    from openstack.cloud import _utils
  File "/usr/lib/python2.7/site-packages/openstack/cloud/_utils.py", line 28, in <module>
    from decorator import decorator
ImportError: No module named decorator

　　原因是 decorator包为3.4.0版本太低，导致，升级完以后，就没有问题了

[root@controller ~]# pip list |grep decorator
decorator                        3.4.0
[root@controller ~]# pip install -U decorator
Looking in indexes: http://172.19.146.225:8081/
Collecting decorator
  Downloading http://172.19.146.225:8081/packages/decorator-4.3.0-py2.py3-none-any.whl
Installing collected packages: decorator
  Found existing installation: decorator 3.4.0
    Uninstalling decorator-3.4.0:
      Successfully uninstalled decorator-3.4.0
Successfully installed decorator-4.3.0

　　问题4：kolla-ansible部署完以后，执行openstack 命令报如下错误

Traceback (most recent call last):
  File "/usr/bin/openstack", line 7, in <module>
    from openstackclient.shell import main
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 22, in <module>
    from osc_lib.api import auth
  File "/usr/lib/python2.7/site-packages/osc_lib/api/auth.py", line 22, in <module>
    from osc_lib.i18n import _
  File "/usr/lib/python2.7/site-packages/osc_lib/i18n.py", line 16, in <module>
    import oslo_i18n
  File "/usr/lib/python2.7/site-packages/oslo_i18n/__init__.py", line 14, in <module>
    from ._gettextutils import *
  File "/usr/lib/python2.7/site-packages/oslo_i18n/_gettextutils.py", line 24, in <module>
    from babel import localedata
ImportError: No module named babel

　　原因是babel包 为2.3.4版本太低导致，升级完包以后，就可以了

[root@controller ~]# pip unintall Babel
[root@controller ~]# pip install Babel
Looking in indexes: http://172.19.146.50:8081/
Requirement already satisfied: Babel in /usr/lib/python2.7/site-packages (2.3.4)
Requirement already satisfied: pytz in /usr/lib/python2.7/site-packages (from Babel) (2016.10)
[root@controller ~]# pip install Babel==2.6.0
Looking in indexes: http://172.19.146.50:8081/
Collecting Babel==2.6.0

　问题5：有时升级包时出现如下错误

[root@controller ~]# pip install -U wrapt
Looking in indexes: http://172.19.146.225:8081/simple/
Collecting wrapt
  Downloading http://172.19.146.225:8081/packages/simple/wrapt/wrapt-1.10.11.tar.gz
Installing collected packages: wrapt
  Found existing installation: wrapt 1.10.8
Cannot uninstall 'wrapt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to 
only a partial uninstall.

　解决方法：

[root@controller ~]# pip install --ignore-installed -U wrapt
Looking in indexes: http://172.19.146.225:8081/simple/
Collecting wrapt
  Downloading http://172.19.146.225:8081/packages/simple/wrapt/wrapt-1.10.11.tar.gz
Installing collected packages: wrapt
  Running setup.py install for wrapt ... done
Successfully installed wrapt-1.10.11
[root@controller ~]#