[Buildbot-devel] Periodic build problems after upgrade

Thu Nov 3 16:31:25 UTC 2005

So just yesterday I upgraded my buildbot-0.6.6 install to 0.7.0 because I'd
found that I wanted a behaviour out of some of my builds that could only be
accomplished with interlocks and it seemed like a bad idea to invest too
much time in something I knew was already gone in the latest version. That
aspect of the builds are working great now, but now I'm observing my
periodic builds are somewhat less periodic than they once were.

In 0.6.6 I had a periodicBuildTime set on all of my builds for about an hour
and a half. Since each of the builds generally took about two and a half
hours and I wanted essentially continuous builds, this seemed to be a
reasonable compromise between having the periodic build timer almost always
finding another build in progress and having it wait too long at the end of
one build before starting the next. Turns out there was an uncaught
exception (somewhere, I can find it if anyone's interested but I don't have
the code on hand right now) that would cause the periodic build timer to not
reset itself if it expired and attempted to start another build while one
was already in progress. Once I fixed that everything was humming along
nicely.

Now, though, none of my periodic builds are running and looking at the logs
I have a pretty good idea why, but I need someone to tell me if I'm just not
reading the documentation correctly on the way the new schedulers work. I
dumbed my test case down to the following code which works:

b1 = {
"name": "foo-periodic",
"slavename": "lpgbuild01",
"builddir": "foo-periodic",
"factory": GNUAutoconf(s(step.CVS,
cvsroot=":pserver:anonymous at 147.11.25.18:/master/linux",
cvsmodule="foo", mode="update"), configure=None),
}
s1=Scheduler(
name="foo-sched-sched",
branch="HEAD",
treeStableTimer=3*60,
builderNames=["foo-periodic"]
)
c['builders'] = [b1]
c['schedulers'] = [s1]

and this code which does not:

b1 = {
"name": "foo-periodic",
"slavename": "lpgbuild01",
"builddir": "foo-periodic",
"factory": GNUAutoconf(s(step.CVS,
cvsroot=":pserver:anonymous at 147.11.25.18:/master/linux",
cvsmodule="foo", mode="update"), configure=None),
}
s1=Periodic(
name="foo-periodic-sched",
branch="HEAD",
periodicBuildTimer=3600,
builderNames=["foo-periodic"]
)
c['builders'] = [b1]
c['schedulers'] = [s1]

The failing case generates this in the log:

2005/11/03 08:06 PST [-] Log opened.
2005/11/03 08:06 PST [-] twistd 2.1.0 (/usr/bin/python 2.3.4) starting up
2005/11/03 08:06 PST [-] reactor class:
twisted.internet.selectreactor.SelectReactor
2005/11/03 08:06 PST [-] Loading buildbot.tac...
2005/11/03 08:06 PST [-]
/usr/lib/python2.3/site-packages/twisted/manhole/telnet.py:8:
exceptions.DeprecationWarning: As of Twisted 2.1,
twisted.protocols.telnetis deprecated. See
twisted.conch.telnet for the current, supported API.
2005/11/03 08:06 PST [-] Loaded.
2005/11/03 08:06 PST [-] loading configuration from
/home/buildadmin/bbmaster/master.cfg
2005/11/03 08:06 PST [-]
/usr/lib/python2.3/site-packages/twisted/web/woven/interfaces.py:60:
twisted.python.components.ComponentsDeprecationWarning: Please don't use
__adapt__ on Interface subclasses
2005/11/03 08:06 PST [-]
/usr/lib/python2.3/site-packages/twisted/web/woven/interfaces.py:135:
twisted.python.components.ComponentsDeprecationWarning: Please don't use
__adapt__ on Interface subclasses
2005/11/03 08:06 PST [-] unknown key 'cvsRoot' defined in config dictionary
2005/11/03 08:06 PST [-] loadConfig_Sources, change_svc is <
buildbot.changes.changes.ChangeMaster instance at 0xb7a094ac> <
buildbot.master.BuildMaster instance at 0xb7b5e58c>
2005/11/03 08:06 PST [-] Traceback (most recent call last):
File "/usr/lib/python2.3/site-packages/twisted/application/service.py", line
226, in addService
service.startService()
File "/usr/lib/python2.3/site-packages/twisted/application/service.py", line
199, in startService
service.startService()
File "/usr/lib/python2.3/site-packages/twisted/application/internet.py",
line 169, in startService
self._loop.start(self.step, now=True).addErrback(self._failed)
File "/usr/lib/python2.3/site-packages/twisted/internet/task.py", line 64,
in start
self()
--- <exception caught here> ---
File "/usr/lib/python2.3/site-packages/twisted/internet/task.py", line 84,
in __call__
self.f(*self.a, **self.kw <http://self.kw>)
File "/usr/lib/python2.3/site-packages/buildbot/scheduler.py", line 329, in
doPeriodicBuild
self.submit(bs)
File "/usr/lib/python2.3/site-packages/buildbot/scheduler.py", line 54, in
submit
self.parent.submitBuildSet(bs)
File "/usr/lib/python2.3/site-packages/buildbot/master.py", line 998, in
submitBuildSet
raise KeyError("no such builder named '%s'" % name)
exceptions.KeyError: "no such builder named 'foo-periodic'"

2005/11/03 08:06 PST [-] adding new builder foo-periodic for category None
2005/11/03 08:06 PST [-] trying to load status pickle from
/home/buildadmin/bbmaster/foo-periodic/builder
2005/11/03 08:06 PST [-] added builder foo-periodic in category None
2005/11/03 08:06 PST [-] Botmaster.addBuilder(foo-periodic)
2005/11/03 08:06 PST [-] adding IStatusReceiver <Waterfall on port 8080>
2005/11/03 08:06 PST [-] twisted.web.server.Site starting on 8080
2005/11/03 08:06 PST [-] Starting factory <twisted.web.server.Site instance
at 0xb78cc82c>
2005/11/03 08:06 PST [-] adding IStatusReceiver <
buildbot.status.client.PBListener instance at 0xb7904dcc>
2005/11/03 08:06 PST [-] twisted.spread.pb.PBServerFactory starting on 8008
2005/11/03 08:06 PST [-] Starting factory
<twisted.spread.pb.PBServerFactoryinstance at 0xb79202cc>
2005/11/03 08:06 PST [-] adding IStatusReceiver <
buildbot.status.mail.MailNotifier instance at 0xb79253ac>
2005/11/03 08:06 PST [-] twisted.spread.pb.PBServerFactory starting on 8007
2005/11/03 08:06 PST [-] Starting factory
<twisted.spread.pb.PBServerFactoryinstance at 0xb7b5e64c>
2005/11/03 08:06 PST [-] BuildMaster listening on port 8007
2005/11/03 08:06 PST [-] configuration updated

Which I guess means the periodic scheduler is trying to spawn a build
immediately (which, I realize, is exactly as advertised in the documentation
and behaviour I really want) and happens before the builder's __init__, or
at the very least before the builder shows up in the list of builders. Is
this expected/desired behaviour or have I just done something painfully
stupid with my master.cfg? Even these scheduler-less builds are all okay if
I kick one off from the web interface.

For what it's worth, this is bb-0.7.0, Twisted 2.1.0, TwistedMail 0.2.0,
TwistedWeb 0.5.0, TwistedWords 0.3.0 and ZopeInterface 3.1.0b1 on the
offending system.

--
-Joe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20051103/07707bfa/attachment.html>