[Buildbot-devel] Re: Locking stacktrace

[Dobes Vandermeer]

Also, between the two builders that are not mutually locking against
each other, one of them was added using a "reload configuration".  It
seems likely that the two builders ended up with a different  lock
instance on the reload - is this a little design bug I have unveiled ?


On 11/7/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> I've confirmed that this assertion failed on a lock, and now like all
> the locks on that slave are failing to lock each other out.  I don't
> know why this is happening, but I'm getting steps and builders failing
> to acquire the appropriate locks, and I even have a redundant level of
> locks (locks on the steps and on the builders each which should be
> blocking each other but now are not).
>
> On 11/4/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> > Actually its not necessarily caused by moving a builder from one slave
> > to another, I'm still getting it after I restarted the master; maybe
> > it is caused by reloading the configuration, it is hard to say:
> >
> > 2005/11/04 19:07 Pacific Standard Time [-] acquireLocks(step
> > <buildbot.process.step.ShellCommand instance at 0x0184A440>
> > , locks [<SlaveLock(ps2_tool)[slave4] 26707848>])
> > 2005/11/04 19:07 Pacific Standard Time [-]
> > <SlaveLock(ps2_tool)[slave4] 26707848> isAvailable: self.owner=None
> > 2005/11/04 19:07 Pacific Standard Time [-]
> > <SlaveLock(ps2_tool)[slave4] 26707848>
> > claim(<buildbot.process.step.ShellComm
> > and instance at 0x0184A440>)
> > 2005/11/04 19:07 Pacific Standard Time [-]
> > <SlaveLock(ps2_tool)[slave4] 26707848> is claimed
> > 2005/11/04 19:07 Pacific Standard Time [-] ShellCommand.start using
> > log <buildbot.status.builder.LogFile instance at 0x0
> > 18C3FA8>
> > 2005/11/04 19:07 Pacific Standard Time [-]  for cmd
> > <RemoteShellCommand '['...']'>
> > 2005/11/04 19:07 Pacific Standard Time [-] <RemoteShellCommand '['...']'>: Rem
> > oteCommand.run [265]
> > 2005/11/04 19:07 Pacific Standard Time [-] command '['..']' in dir '.'
> > 2005/11/04 19:07 Pacific Standard Time [-] LoggedRemoteCommand.start
> > <buildbot.status.builder.LogFile instance at 0x018C
> > 3FA8>
> > 2005/11/04 19:07 Pacific Standard Time [-]
> > <SlaveLock(ps2_tool)[slave4] 26707848> nowAvailable
> > 2005/11/04 19:07 Pacific Standard Time [-] Traceback (most recent call last):
> >           File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py",
> > line 44, in runApp
> >             app.runReactorWithLogging(config, oldstdout, oldstderr)
> >           File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py",
> > line 128, in runReactorWithLogging
> >             reactor.run()
> >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > line 200, in run
> >             self.mainLoop()
> >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > line 208, in mainLoop
> >             self.runUntilCurrent()
> >         --- <exception caught here> ---
> >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py",
> > line 533, in runUntilCurrent
> >             call.func(*call.args, **call.kw)
> >           File "Z:\Tools\PythonLibs\buildbot\locks.py", line 43, in nowAvailable
> >             assert not self.owner
> >         exceptions.AssertionError:
> >
> >
> > On 11/4/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> > > I seem to get a stacktrace like this sometimes:
> > >
> > > 2005/11/04 15:55 Pacific Standard Time [-] Traceback (most recent call last):
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py",
> > > line 44, in runApp
> > >             app.runReactorWithLogging(config, oldstdout, oldstderr)
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py",
> > > line 128, in runReactorWithLogging
> > >             reactor.run()
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > > line 200, in run
> > >             self.mainLoop()
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > > line 208, in mainLoop
> > >             self.runUntilCurrent()
> > >         --- <exception caught here> ---
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py",
> > > line 533, in runUntilCurrent
> > >             call.func(*call.args, **call.kw)
> > >           File "Z:\Tools\PythonLibs\buildbot\locks.py", line 43, in nowAvailable
> > >             assert not self.owner
> > >         exceptions.AssertionError:
> > >
> > >
> > > I think it may be caused by switching a builder from one slave to
> > > another by reloading the configuration, when the builder has a
> > > SlaveLock (although I don't think the SlaveLock is held, so its a bit
> > > hard to debug).
> > >
> > > Probably i could get more information if the status page showed which
> > > lock(s) a builder was waiting for, on which slaves... hopefully
> > > that'll be ready sometime soon? :-) :-)
> > >
> >
>