[Buildbot-devel] Severe issues with buildbot 0.7.4

[Kenneth Lareau]

Brian Warner wrote:
> Kenneth Lareau <Ken.Lareau at nominum.com> writes:
> 
>>     buildbot 0.7.4
> 
>> "/usr/local/lib/python2.4/site-packages/buildbot/slave/commands.py", line 322, in _startCommand
>>              argv = (" ".join(self.command) % os.environ).split()
> 
> These line numbers don't match up.. is your version of commands.py patched?

No, it's not patched; I just checked a system on which there's no
problem and the file is precisely the same.

> 
>>            File "/usr/local/lib/python2.4/UserDict.py", line 17, in __getitem__
>>              def __getitem__(self, key): return self.data[key]
>>          exceptions.KeyError: 'workdir'
> 
> Hrm, where did that UserDict come from? os.environ is a regular built-in dict
> type. And why would 'workdir' be present in os.environ? In ShellCommand that
> usually comes from self.workdir .
> 
> And.. I can't find that argv= source line anywhere in commands.py .. it looks
> like you may have had to modify your version.. could you forward a copy to us?

UserDict.py is a standard file for Python 2.4.x, AFAIK, or are you
questioning why it's being called?  An 'env' on the system shows
no setting of 'workdir' that I see, so I'm not sure why os.environ
is being used here.  I will include a copy of the commands.py file
here.

>> 2007/04/19 12:08 -0700 [-] BuildStep.failed, traceback follows
>>
>>
>> At this point buildbot stops logging completely on the master,
>> though it still seems to be responsive to other builders being
>> run.
> 
> My hunch here is that something in the traceback caused twisted's logging
> module to die as it attempted to log the error on the buildmaster side. Once
> upon a time, unicode characters in log messages could do this, although I
> believe that with twisted-2.5.0 there is at least some kind of message to
> help you spot the problem (like "hey, somebody tried to log unicode, please
> don't do that"). Something which has a customized __repr__ that might throw
> an exception could be involved.

As far as I know, there is no Unicode involved, and as I mentioned
before, this system was operating fine until last night and I know
of no changes that have been made on it, unfortunately.

> It is quite likely that the buildmaster will continue on fine, although
> obviously with logging disabled it becomes awfully hard to investigate the
> problem further :).
> 
> So questions for you: what does your commands.py look like, how are those
> extra keys getting put into os.environ, is there any unicode involved, and
> what platforms are your buildmaster/buildslave running on?

Including the commands.py file, no clue how the extra keys are being
put into os.environ, no Unicode involved that I know, the buildmaster
is running on RHEL 3 with python 2.4.3 and the buildslave is running
on FreeBSD 6.2 with Python 2.4.4.

> thanks,
>  -Brian
> 

And thank you for your quick response!


Ken
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: commands.py
URL: <http://buildbot.net/pipermail/devel/attachments/20070419/2e9be7ae/attachment.ksh>