[Buildbot-devel] logging

[exarkun at twistedmatrix.com]

On 26 Mar, 06:57 pm, a.nikolov at drweb.com wrote:
>exarkun at twistedmatrix.com wrote:
>>Programming is hard.  Debugging, moreso.  Can you be more specific 
>>about
>>what you're looking for?
>
>Ok, when i just force one builder from the web interface, it goes ok.

Sorry, when I said "more specific" I was talking about your requirements 
for a debugging tool, not about the specifics of the case you want to 
debug.
>
>When i trigger a couple of builders by sending a change to them, it 
>goes ok.
>
>But when i trigger several hundred of builders by a Nighty scheduler 
>(this
>implies an excessive locks using: 8 master locks and 2 slave locks) 
>then
>some of builds perform well and some of them - just as i said before:
>
>- master and slave lock aquired by Build
>- first step aquires lock in [] (no locking) successes and releases 
>lock
>- no other steps done ('Not started', 'Not finished')
>- Build releases master and slave lock
>
>Are there any ideas how to debug this?

I'd start by making it really easy to reproduce, probably by wrapping it 
up in an automated test of some sort.  Then I'd instrument the 
implementation with logging and use pdb to step through execution and 
examine local state until I knew what the problem was.

I know that's not very specific, but it's also not terribly unusual. 
It's how I approach most problems like this.

Jean-Paul