[Buildbot-devel] Network access problems on OS X 10.5 buildslave

[C. E. Ball]

Hi,

I've been running a buildslave successfully on OS X 10.4 for some
time. Recently, however, I upgraded to OS X 10.5, and now have a
problem.

I start the buildslave from an ssh session, as usual ("buildbot start
..."), but I find that the buildslave only seems to have network
access while I remain logged in to that ssh session! If I log out of
the ssh session, any network command run on the slave fails.

I created a simple build that just executes 'ping -c 1
topographica.svn.sourceforge.net'. This works fine until I log out of
the ssh session from which I started the buildslave. Here is stdout
for a failure:

ping -c 1 topographica.svn.sourceforge.net
 in dir /Users/bb/buildslave/tests_ppc_darwin8.10.0/build (timeout 1200 secs)
 watching logfiles {}
 argv: ['ping', '-c', '1', 'topographica.svn.sourceforge.net']
 environment:
  HOME=/Users/bb
  INFOPATH=/sw/share/info:/sw/info:/usr/share/info
  LOGNAME=bb
  MAIL=/var/mail/bb
  MANPATH=/sw/share/man:/usr/share/man:/usr/local/share/man:/usr/X11/man:/sw/lib/perl5/5.8.8/man:/usr/X11R6/man
  OLDPWD=/Users/bb
  PATH=/sw/bin:/sw/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/X11R6/bin
  PERL5LIB=/sw/lib/perl5:/sw/lib/perl5/darwin
  PWD=/Users/bb/buildslave
  SGML_CATALOG_FILES=/sw/etc/sgml/catalog
  SHELL=/bin/bash
  SHLVL=1
  SSH_CLIENT=xxx.xxx.xxx.xxx 63589 22
  SSH_CONNECTION=xxx.xxx.xxx.xxx 63589 yyy.yyy.yyy.yyy 22
  SSH_TTY=/dev/ttys001
  TERM=xterm-color
  USER=bb
  XML_CATALOG_FILES=/sw/etc/xml/catalog
  _=/sw/bin/buildbot
ping: cannot resolve topographica.svn.sourceforge.net: Unknown host
program finished with exit code 68

(I've also appended the relevant sections of buildbot's log files to
this message, although they do not seem to illuminate the problem any
further to me.)

Commands that require no network access work without any problems. I
am puzzled; has anyone else had a problem like this? Can anyone
suggest what I can try to solve this problem?

I installed buildbot with fink ("fink install buildbot-py25"); I think
that under 10.4 I installed buildbot with macports. Maybe I should
update twisted (I've included printouts of versions of buildbot and
twisted on the master and slave below)?

Thanks very much,
Chris Ball


On the slave:
$ buildbot --version
Buildbot version: 0.7.6
Twisted version: 2.1.0

On the master:
$ buildbot --version
Buildbot version: 0.7.6
Twisted version: 2.5.0

The slave's twistd.log:
2008/03/27 06:02 CDT [Broker,client]
SlaveBuilder.remote_print(mac_tests): message from master: ping
2008/03/27 06:02 CDT [Broker,client]
SlaveBuilder.remote_ping(<SlaveBuilder 'mac_tests' at 7096200>)
2008/03/27 06:02 CDT [Broker,client] <SlaveBuilder 'mac_tests' at
7096200>.startBuild
2008/03/27 06:02 CDT [Broker,client]  startCommand:shell [id 4]
2008/03/27 06:02 CDT [Broker,client] ShellCommand._startCommand
2008/03/27 06:02 CDT [Broker,client]  ping -c 1 topographica.svn.sourceforge.net
2008/03/27 06:02 CDT [Broker,client]   in dir
/Users/bb/buildslave/tests_ppc_darwin8.10.0/build (timeout 1200 secs)
2008/03/27 06:02 CDT [Broker,client]   watching logfiles {}
2008/03/27 06:02 CDT [Broker,client]   argv: ['ping', '-c', '1',
'topographica.svn.sourceforge.net']
2008/03/27 06:02 CDT [Broker,client]  environment: {'PERL5LIB':
'/sw/lib/perl5:/sw/lib/perl5/darwin', 'TERM': 'xterm-color', 'SHELL':
'/bin/bash', \
'INFOPATH': '/sw/share/info:/sw/info:/usr/share/info', 'SHLVL': '1',
'SSH_TTY': '/dev/ttys001', 'OLDPWD': '/Users/bb', 'SSH_CLIENT':
'xxx.xxx.\
xxx.xxx 63589 22', 'XML_CATALOG_FILES': '/sw/etc/xml/catalog', 'PWD':
'/Users/bb/buildslave', 'SGML_CATALOG_FILES': '/sw/etc/sgml/catalog',
'LOGN\
AME': 'bb', 'USER': 'bb', 'MANPATH':
'/sw/share/man:/usr/share/man:/usr/local/share/man:/usr/X11/man:/sw/lib/perl5/5.8.8/man:/usr/X11R6/man\
', 'MAIL': '/var/mail/bb', 'PATH':
'/sw/bin:/sw/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/X11R6/bin',
'SSH_CONNECTION\
': 'xxx.xxx.xxx.xxx 63589 yyy.yyy.yyy.yyy 22', 'HOME': '/Users/bb',
'_': '/sw/bin/buildbot'}
2008/03/27 06:02 CDT [-] command finished with signal None, exit code 68
2008/03/27 06:02 CDT [-] SlaveBuilder.commandComplete
<buildbot.slave.commands.SlaveShellCommand instance at 0x6c4c60>

The master's twistd.log:
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] ping finished: success
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] <Build mac_tests>.startBuild
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] acquireLocks(step
<Build mac_tests>, locks [<SlaveLock(sake, 1)[ppc_darwin8.10.0]
146994828>])
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] acquireLocks(step
<buildbot.steps.shell.ShellCommand instance at 0x8e05e2c>, locks [])
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy]
ShellCommand.startCommand(cmd=%s) (<RemoteShellCommand '['ping', '-c',
'1', 'topographica.svn.sourceforge.net']'>,)
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] <RemoteShellCommand
'['ping', '-c', '1', 'topographica.svn.sourceforge.net']'>:
RemoteCommand.run [4]
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] command '['ping',
'-c', '1', 'topographica.svn.sourceforge.net']' in dir 'build'
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] LoggedRemoteCommand.start
2008/03/27 12:02 +0100 [Broker,0,yyy.yyy.yyy.yyy] <RemoteShellCommand
'['ping', '-c', '1', 'topographica.svn.sourceforge.net']'> rc=68
2008/03/27 12:02 +0100 [-] closing log
<buildbot.status.builder.LogFile instance at 0x8e26f0c>
2008/03/27 12:02 +0100 [-]
releaseLocks(<buildbot.steps.shell.ShellCommand instance at
0x8e05e2c>): []
2008/03/27 12:02 +0100 [-]  step 'shell' complete: failure