[Buildbot] #3048: ConnectionError when the Docker host is not available

[Buildbot trac]

#3048: ConnectionError when the Docker host is not available
-------------------+-----------------------------------
Reporter:  Ben     |      Owner:
    Type:  defect  |     Status:  new
Priority:  major   |  Milestone:  undecided
 Version:  master  |   Keywords:  docker, slave, latent
-------------------+-----------------------------------
 {{{
 2014-11-18 12:34:20+0100 [-] starting build <Build Trunk_ospd> using slave
 <LatentSlaveBuilder builder=u'Trunk_ospd' slave=u'ospd-python-slave'>
 2014-11-18 12:34:20+0100 [-] acquireLocks(slave <DockerLatentBuildSlave u
 'ospd-python-slave'>, locks [])
 2014-11-18 12:34:20+0100 [-] substantiating slave <LatentSlaveBuilder
 builder=u'Trunk_ospd' slave=u'ospd-python-slave'>
 2014-11-18 12:34:20+0100 [-] Buildslave ospd-python-slave detached from
 Trunk_ospd
 2014-11-18 12:34:20+0100 [-] while preparing slavebuilder:
         Traceback (most recent call last):
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/internet/defer.py", line 577, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/internet/defer.py", line 1155, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
             result = result.throwExceptionIntoGenerator(g)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/failure.py", line 389, in
 throwExceptionIntoGenerator
             return g.throw(self.type, self.value, self.tb)
         --- <exception caught here> ---
           File
 "/home/benoit/buildbot/buildbot/master/buildbot/process/builder.py", line
 349, in _startBuildFor
             ready = yield slavebuilder.prepare(self.builder_status, build)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/threadpool.py", line 196, in _worker
             result = context.call(ctx, function, *args, **kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/context.py", line 118, in callWithContext
             return self.currentContext().callWithContext(ctx, func, *args,
 **kw)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/context.py", line 81, in callWithContext
             return func(*args,**kw)
           File "dockerslave.py", line 74, in _start_instance
             found = self._image_exists(docker_client)
           File "dockerslave.py", line 65, in _image_exists
             for container in client.images():
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/docker/client.py", line 634, in images
             res = self._result(self._get(self._url("/images/json"),
 params=params),
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/docker/client.py", line 76, in _get
             return self.get(url, **self._set_request_timeout(kwargs))
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/sessions.py", line 469, in get
             return self.request('GET', url, **kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/sessions.py", line 457, in request
             resp = self.send(prep, **send_kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/sessions.py", line 569, in send
             r = adapter.send(request, **kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/adapters.py", line 407, in send
             raise ConnectionError(err, request=request)
         requests.exceptions.ConnectionError: ('Connection aborted.',
 error(111, 'Connection refused'))

 2014-11-18 12:34:20+0100 [-] slave <Build Trunk_ospd> can't build
 <LatentSlaveBuilder builder=u'Trunk_ospd'> after all; re-queueing the
 request
 2014-11-18 12:34:20+0100 [-] releaseLocks(<DockerLatentBuildSlave u'ospd-
 python-slave'>): []
 2014-11-18 12:34:20+0100 [-] while substantiating
         Traceback (most recent call last):
           File "/usr/lib/python2.7/threading.py", line 525, in __bootstrap
             self.__bootstrap_inner()
           File "/usr/lib/python2.7/threading.py", line 552, in
 __bootstrap_inner
             self.run()
           File "/usr/lib/python2.7/threading.py", line 505, in run
             self.__target(*self.__args, **self.__kwargs)
         --- <exception caught here> ---
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/threadpool.py", line 196, in _worker
             result = context.call(ctx, function, *args, **kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/context.py", line 118, in callWithContext
             return self.currentContext().callWithContext(ctx, func, *args,
 **kw)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/twisted/python/context.py", line 81, in callWithContext
             return func(*args,**kw)
           File "dockerslave.py", line 74, in _start_instance
             found = self._image_exists(docker_client)
           File "dockerslave.py", line 65, in _image_exists
             for container in client.images():
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/docker/client.py", line 634, in images
             res = self._result(self._get(self._url("/images/json"),
 params=params),
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/docker/client.py", line 76, in _get
             return self.get(url, **self._set_request_timeout(kwargs))
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/sessions.py", line 469, in get
             return self.request('GET', url, **kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/sessions.py", line 457, in request
             resp = self.send(prep, **send_kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/sessions.py", line 569, in send
             r = adapter.send(request, **kwargs)
           File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7
 /site-packages/requests/adapters.py", line 407, in send
             raise ConnectionError(err, request=request)
         requests.exceptions.ConnectionError: ('Connection aborted.',
 error(111, 'Connection refused'))

 }}}

 We should at least catch the exception.

--
Ticket URL: <http://trac.buildbot.net/ticket/3048>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation