[Buildbot-devel] Re: Locking stacktrace

[Dobes Vandermeer]

Actually by reading the logs I can clearly see that one is locking one
copy of the SlaveLock, and the other is locking a different SlaveLock.
 I guess that's the disadvantage of using instances, huh? :^(

For now I'll have to put the locks (and schedulers, come to think of
it) into another file and import it so that it is only loaded once.

On 11/7/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> Also, between the two builders that are not mutually locking against
> each other, one of them was added using a "reload configuration".  It
> seems likely that the two builders ended up with a different  lock
> instance on the reload - is this a little design bug I have unveiled ?
>
>
> On 11/7/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> > I've confirmed that this assertion failed on a lock, and now like all
> > the locks on that slave are failing to lock each other out.  I don't
> > know why this is happening, but I'm getting steps and builders failing
> > to acquire the appropriate locks, and I even have a redundant level of
> > locks (locks on the steps and on the builders each which should be
> > blocking each other but now are not).
> >
> > On 11/4/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> > > Actually its not necessarily caused by moving a builder from one slave
> > > to another, I'm still getting it after I restarted the master; maybe
> > > it is caused by reloading the configuration, it is hard to say:
> > >
> > > 2005/11/04 19:07 Pacific Standard Time [-] acquireLocks(step
> > > <buildbot.process.step.ShellCommand instance at 0x0184A440>
> > > , locks [<SlaveLock(ps2_tool)[slave4] 26707848>])
> > > 2005/11/04 19:07 Pacific Standard Time [-]
> > > <SlaveLock(ps2_tool)[slave4] 26707848> isAvailable: self.owner=None
> > > 2005/11/04 19:07 Pacific Standard Time [-]
> > > <SlaveLock(ps2_tool)[slave4] 26707848>
> > > claim(<buildbot.process.step.ShellComm
> > > and instance at 0x0184A440>)
> > > 2005/11/04 19:07 Pacific Standard Time [-]
> > > <SlaveLock(ps2_tool)[slave4] 26707848> is claimed
> > > 2005/11/04 19:07 Pacific Standard Time [-] ShellCommand.start using
> > > log <buildbot.status.builder.LogFile instance at 0x0
> > > 18C3FA8>
> > > 2005/11/04 19:07 Pacific Standard Time [-]  for cmd
> > > <RemoteShellCommand '['...']'>
> > > 2005/11/04 19:07 Pacific Standard Time [-] <RemoteShellCommand '['...']'>: Rem
> > > oteCommand.run [265]
> > > 2005/11/04 19:07 Pacific Standard Time [-] command '['..']' in dir '.'
> > > 2005/11/04 19:07 Pacific Standard Time [-] LoggedRemoteCommand.start
> > > <buildbot.status.builder.LogFile instance at 0x018C
> > > 3FA8>
> > > 2005/11/04 19:07 Pacific Standard Time [-]
> > > <SlaveLock(ps2_tool)[slave4] 26707848> nowAvailable
> > > 2005/11/04 19:07 Pacific Standard Time [-] Traceback (most recent call last):
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py",
> > > line 44, in runApp
> > >             app.runReactorWithLogging(config, oldstdout, oldstderr)
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py",
> > > line 128, in runReactorWithLogging
> > >             reactor.run()
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > > line 200, in run
> > >             self.mainLoop()
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > > line 208, in mainLoop
> > >             self.runUntilCurrent()
> > >         --- <exception caught here> ---
> > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py",
> > > line 533, in runUntilCurrent
> > >             call.func(*call.args, **call.kw)
> > >           File "Z:\Tools\PythonLibs\buildbot\locks.py", line 43, in nowAvailable
> > >             assert not self.owner
> > >         exceptions.AssertionError:
> > >
> > >
> > > On 11/4/05, Dobes Vandermeer <dobesv at gmail.com> wrote:
> > > > I seem to get a stacktrace like this sometimes:
> > > >
> > > > 2005/11/04 15:55 Pacific Standard Time [-] Traceback (most recent call last):
> > > >           File "Z:\Tools\PythonLibs\Python24\twisted\scripts\_twistw.py",
> > > > line 44, in runApp
> > > >             app.runReactorWithLogging(config, oldstdout, oldstderr)
> > > >           File "Z:\Tools\PythonLibs\Python24\twisted\application\app.py",
> > > > line 128, in runReactorWithLogging
> > > >             reactor.run()
> > > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > > > line 200, in run
> > > >             self.mainLoop()
> > > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\posixbase.py",
> > > > line 208, in mainLoop
> > > >             self.runUntilCurrent()
> > > >         --- <exception caught here> ---
> > > >           File "Z:\Tools\PythonLibs\Python24\twisted\internet\base.py",
> > > > line 533, in runUntilCurrent
> > > >             call.func(*call.args, **call.kw)
> > > >           File "Z:\Tools\PythonLibs\buildbot\locks.py", line 43, in nowAvailable
> > > >             assert not self.owner
> > > >         exceptions.AssertionError:
> > > >
> > > >
> > > > I think it may be caused by switching a builder from one slave to
> > > > another by reloading the configuration, when the builder has a
> > > > SlaveLock (although I don't think the SlaveLock is held, so its a bit
> > > > hard to debug).
> > > >
> > > > Probably i could get more information if the status page showed which
> > > > lock(s) a builder was waiting for, on which slaves... hopefully
> > > > that'll be ready sometime soon? :-) :-)
> > > >
> > >
> >
>