[Buildbot-devel] Re: Locking stacktrace

[Dobes Vandermeer]

I've confirmed that this assertion failed on a lock, and now like all
the locks on that slave are failing to lock each other out.  I don't
know why this is happening, but I'm getting steps and builders failing
to acquire the appropriate locks, and I even have a redundant level of
locks (locks on the steps and on the builders each which should be
blocking each other but now are not).

On 11/4/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> Actually its not necessarily caused by moving a builder from one slave
> to another, I'm still getting it after I restarted the master; maybe
> it is caused by reloading the configuration, it is hard to say:
>
> 2005/11/04 19:07 Pacific Standard Time [-] acquireLocks(step
> <buildbot.process.step.ShellCommand instance at 0x0184A440>
> , locks [<SlaveLock(ps2_tool)[slave4] 26707848>])
> 2005/11/04 19:07 Pacific Standard Time [-]
> <SlaveLock(ps2_tool)[slave4] 26707848> isAvailable: self.owner=None
> 2005/11/04 19:07 Pacific Standard Time [-]
> <SlaveLock(ps2_tool)[slave4] 26707848>
> claim(<buildbot.process.step.ShellComm
> and instance at 0x0184A440>)
> 2005/11/04 19:07 Pacific Standard Time [-]
> <SlaveLock(ps2_tool)[slave4] 26707848> is claimed
> 2005/11/04 19:07 Pacific Standard Time [-] ShellCommand.start using
> log <buildbot.status.builder.LogFile instance at 0x0
> 18C3FA8>
> 2005/11/04 19:07 Pacific Standard Time [-]  for cmd
> <RemoteShellCommand '['...']'>
> 2005/11/04 19:07 Pacific Standard Time [-] <RemoteShellCommand '['...']'>: Rem
> oteCommand.run [265]
> 2005/11/04 19:07 Pacific Standard Time [-] command '['..']' in dir '.'
> 2005/11/04 19:07 Pacific Standard Time [-] LoggedRemoteCommand.start
> <buildbot.status.builder.LogFile instance at 0x018C
> 3FA8>
> 2005/11/04 19:07 Pacific Standard Time [-]
> <SlaveLock(ps2_tool)[slave4] 26707848> nowAvailable
> 2005/11/04 19:07 Pacific Standard Time [-] Traceback (most recent call last):
>           File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py",
> line 44, in runApp
>             app.runReactorWithLogging(config, oldstdout, oldstderr)
>           File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py",
> line 128, in runReactorWithLogging
>             reactor.run()
>           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> line 200, in run
>             self.mainLoop()
>           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> line 208, in mainLoop
>             self.runUntilCurrent()
>         --- <exception caught here> ---
>           File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py",
> line 533, in runUntilCurrent
>             call.func(*call.args, **call.kw)
>           File "Z:\Tools\PythonLibs\buildbot\locks.py", line 43, in nowAvailable
>             assert not self.owner
>         exceptions.AssertionError:
>
>
> On 11/4/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> > I seem to get a stacktrace like this sometimes:
> >
> > 2005/11/04 15:55 Pacific Standard Time [-] Traceback (most recent call last):
> >           File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py",
> > line 44, in runApp
> >             app.runReactorWithLogging(config, oldstdout, oldstderr)
> >           File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py",
> > line 128, in runReactorWithLogging
> >             reactor.run()
> >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > line 200, in run
> >             self.mainLoop()
> >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > line 208, in mainLoop
> >             self.runUntilCurrent()
> >         --- <exception caught here> ---
> >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py",
> > line 533, in runUntilCurrent
> >             call.func(*call.args, **call.kw)
> >           File "Z:\Tools\PythonLibs\buildbot\locks.py", line 43, in nowAvailable
> >             assert not self.owner
> >         exceptions.AssertionError:
> >
> >
> > I think it may be caused by switching a builder from one slave to
> > another by reloading the configuration, when the builder has a
> > SlaveLock (although I don't think the SlaveLock is held, so its a bit
> > hard to debug).
> >
> > Probably i could get more information if the status page showed which
> > lock(s) a builder was waiting for, on which slaves... hopefully
> > that'll be ready sometime soon? :-) :-)
> >
>