We use Ansible to deploy Docker containers on our systems and the infrastructures of our customers. This works pretty well if your system is configured properly.
Here’s a simple playbook to run the hello-world Docker container on one of your systems:
---
- hosts: docker.confirm.ch
tasks:
- name: run hello-world docker container
docker:
name: hello-world
image: hello-world
state: present
sudo: yes
Dependency to docker-py
Ansible is written in Python, so is the docker module. Therefor it makes use of the docker-py Python library to manage containers.
To use Ansible’s docker module, you’ve to make sure the docker-py library is installed. We do that by creating a new Ansible role named docker and then add the installation of the required packages (docker, docker-py) as a task. Now every role which uses Ansible’s docker module adds our docker role as dependency, therefor the docker and docker-py packages are installed automatically.
This is where the problem starts on RHEL or CentOS 7, because docker-py is available via PyPi, therefor pip and finally via yum / RHN channels as well.
The RPM
When you install the docker-py module via yum, you’ll end up with something like this:
# rpm -qi docker-python Name : docker-python Version : 1.4.0 Release : 108.el7 Architecture: x86_64 Install Date: Thu 03 Sep 2015 09:47:59 AM CEST Group : Unspecified Size : 180199 License : ASL 2.0 Signature : RSA/SHA256, Wed 29 Jul 2015 07:15:07 AM CEST, Key ID 199e2f91fd431d51 Source RPM : docker-1.7.1-108.el7.src.rpm Build Date : Tue 28 Jul 2015 10:09:38 PM CEST Build Host : x86-030.build.eng.bos.redhat.com Relocations : (not relocatable) Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla> Vendor : Red Hat, Inc. URL : http://www.docker.com Summary : An API client for docker written in Python Description : An API client for docker written in Python
As you can see the version 1.4.0 is installed. This can also be verified via pip and python’s help() or pydoc:
# pip list | grep docker
docker-py (1.4.0-dev)
# pydoc docker
Help on package docker:
NAME
docker - # Copyright 2013 dotCloud inc.
FILE
/usr/lib/python2.7/site-packages/docker/__init__.py
PACKAGE CONTENTS
auth (package)
client
clientbase
constants
errors
ssladapter (package)
tls
unixconn (package)
utils (package)
version
DATA
__title__ = 'docker-py'
__version__ = '1.4.0-dev'
version = '1.4.0-dev'
version_info = (1, 4, 0)
VERSION
1.4.0-dev
This looks quite ok, but there’s something really strange, because there is no official 1.4.0(-dev) version on PyPi or even github.
There’s a 1.4.0-dev version available on PyPi, but it’s an inofficial version from a user called sunadm.
But let’s ignore that for now, because here’s the real issue with the official docker-py RPM from Red Hat:
If you run the Ansible playbook above with the RPM docker-py-1.4.0-108.el7 installed, you’ll get a Python error / traceback:
Traceback (most recent call last):
File "/home/ansible/.ansible/tmp/ansible-tmp-1441276343.36-208631949891601/docker", line 3132, in <module>
main()
File "/home/ansible/.ansible/tmp/ansible-tmp-1441276343.36-208631949891601/docker", line 1464, in main
check_dependencies(module)
File "/home/ansible/.ansible/tmp/ansible-tmp-1441276343.36-208631949891601/docker", line 482, in check_dependencies
versioninfo = get_docker_py_versioninfo()
File "/home/ansible/.ansible/tmp/ansible-tmp-1441276343.36-208631949891601/docker", line 458, in get_docker_py_versioninfo
version.append(int(digit))
ValueError: invalid literal for int() with base 10: '0-de'
The PyPi package
Now you might just want to remove the RPM and install docker-py via PyPi / pip .
That’s a good idea, let’s see how that works out:
# remove RPM yum -y remove docker-python # install pip package (1.2.3 is tested and works with Ansible) pip install docker-py==1.2.3
Run Ansible again and you’ll catch another traceback:
Traceback (most recent call last): File "/home/ansible/.ansible/tmp/ansible-tmp-1441276636.72-115693703488445/docker", line 3132, in <module> main() File "/home/ansible/.ansible/tmp/ansible-tmp-1441276636.72-115693703488445/docker", line 1436, in main docker_api_version = dict(required=False, default=DEFAULT_DOCKER_API_VERSION, type='str'), NameError: global name 'DEFAULT_DOCKER_API_VERSION' is not defined
While the traceback was raised by Ansible, the root cause is within the docker.client module.
Though Ansible isn’t handling exceptions properly there – especially if docker-py isn’t installed at all 😉
Let’s dig into the problem by having a look at Ansible’s docker module.
Around line 428 you’ll see the import statements for the Python docker modules / classes:
import sys
import json
import os
import shlex
from urlparse import urlparse
try:
import docker.client
import docker.utils
import docker.errors
from requests.exceptions import RequestException
except ImportError:
HAS_DOCKER_PY = False
if HAS_DOCKER_PY:
try:
from docker.errors import APIError as DockerAPIError
except ImportError:
from docker.client import APIError as DockerAPIError
try:
# docker-py 1.2+
import docker.constants
DEFAULT_DOCKER_API_VERSION = docker.constants.DEFAULT_DOCKER_API_VERSION
except (ImportError, AttributeError):
# docker-py less than 1.2
DEFAULT_DOCKER_API_VERSION = docker.client.DEFAULT_DOCKER_API_VERSION
Later in that file around line 1638 you’ll see a dict which makes use of the DEFAULT_DOCKER_API_VERSION variable:
docker_api_version = dict(required=False, default=DEFAULT_DOCKER_API_VERSION, type='str'),
And exactly this line will raise our exception above, because DEFAULT_DOCKER_API_VERSION isn’t defined.
This happens because there was an ImportError while importing the docker.client class.
You can reproduce that by executing:
# python
Python 2.7.5 (default, Apr 9 2015, 11:03:32)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import docker.client
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/docker/__init__.py", line 20, in <module>
from .client import Client, AutoVersionClient # flake8: noqa
File "/usr/lib/python2.7/site-packages/docker/client.py", line 37, in <module>
import websocket
File "/usr/lib/python2.7/site-packages/websocket/__init__.py", line 22, in <module>
from ._core import *
File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 41, in <module>
from ._url import *
File "/usr/lib/python2.7/site-packages/websocket/_url.py", line 23, in <module>
from six.moves.urllib.parse import urlparse
ImportError: No module named urllib.parse
Python six dependency
The root cause is within the dependencies of the docker-py module / docker.client class.
Docker-py requires six, which is another Python library. By default RHEL / CentOS 7 provide six via RPM:
# rpm -qi python-six Name : python-six Version : 1.3.0 Release : 4.el7 Architecture: noarch Install Date: Mon 31 Aug 2015 11:08:51 AM CEST Group : Development/Languages Size : 51382 License : MIT Signature : RSA/SHA256, Wed 02 Apr 2014 09:11:29 PM CEST, Key ID 199e2f91fd431d51 Source RPM : python-six-1.3.0-4.el7.src.rpm Build Date : Sun 29 Dec 2013 03:36:45 PM CET Build Host : x86-020.build.eng.bos.redhat.com Relocations : (not relocatable) Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla> Vendor : Red Hat, Inc. URL : http://pypi.python.org/pypi/six/ Summary : Python 2 and 3 compatibility utilities Description : python-six provides simple utilities for wrapping over differences between Python 2 and Python 3. This is the Python 2 build of the module. # pip list | grep six six (1.3.0)
When you’ve a look at PyPi, you’ll see that the version 1.3.0 is relatively old (2013-03-18).
Though RedHat isn’t currently providing any newer version of the package and unfortunately it’s required for other RPMs as well:
===================================================================================================================== Package Arch Version Repository Size ===================================================================================================================== Removing: python-six noarch 1.3.0-4.el7 @rhel-server-7 50 k Removing for dependencies: abrt x86_64 2.1.11-22.el7_1 @rhel-server-7 2.2 M abrt-addon-ccpp x86_64 2.1.11-22.el7_1 @rhel-server-7 331 k abrt-addon-kerneloops x86_64 2.1.11-22.el7_1 @rhel-server-7 37 k abrt-addon-pstoreoops x86_64 2.1.11-22.el7_1 @rhel-server-7 14 k abrt-addon-python x86_64 2.1.11-22.el7_1 @rhel-server-7 30 k abrt-addon-vmcore x86_64 2.1.11-22.el7_1 @rhel-server-7 41 k abrt-addon-xorg x86_64 2.1.11-22.el7_1 @rhel-server-7 17 k abrt-cli x86_64 2.1.11-22.el7_1 @rhel-server-7 0.0 abrt-console-notification x86_64 2.1.11-22.el7_1 @rhel-server-7 1.3 k abrt-python x86_64 2.1.11-22.el7_1 @rhel-server-7 56 k abrt-tui x86_64 2.1.11-22.el7_1 @rhel-server-7 24 k docker-python x86_64 1.4.0-108.el7 @rhel-server-extras-7 176 k python-requests noarch 2.6.0-1.el7_1 @rhel-server-7 343 k python-urllib3 noarch 1.10.2-2.el7_1 @rhel-server-7 369 k sos noarch 3.2-15.el7_1.5 @rhel-server-7 1.0 M Transaction Summary ===================================================================================================================== Remove 1 Package (+15 Dependent packages)
The Workaround
But here’s the workaround to get Ansible with Docker up & running is the following procedure:
- remove the OS python-docker package
- install the official PyPi docker-py package version 1.2.3 via pip
- upgrade the six package to version >= 1.4.0 via pip without removing the OS package
So let’s update our playbook to meet the new requirements:
---
- hosts: docker.confirm.ch
tasks:
- name: make sure docker-python RPM is not installed
yum:
name: docker-python
state: absent
- name: make sure required PyPi packages are installed
pip:
name: '{{ item }}'
state: present
with_items:
- docker-py==1.2.3
- six>=1.4.0
- name: run hello-world docker container
docker:
name: hello-world
image: hello-world
state: present
sudo: yes
Though there are some drawbacks. OS packages with dependencies to six (e.g. sos, abrt) might not be tested against a newer version of it.
To look on the bright side, there’s already an official docker-py commit which requires six>=1.4.0. So in the end it’s just a matter of time until Red Hat will fix this. Hopefully we don’t have to wait another 2 years 😉
6 Comments