[Buildbot-devel] Scheduler issues after upgrade from 0.8.5 to 0.8.6p1

[Iustin Pop]

Hi all,

I see two strange things after upgrade to 0.8.6p1 but not sure if they
are bugs or just issues with our config. Both seem to be related to
schedulers, btw.

First, changes generated by our custom git poller don't seem to be
picked up anymore. One scheduler example:

  AnyBranchScheduler(name="all",
                     branches=UNITTEST_BRANCHES,
                     treeStableTimer=30,
                     builderNames=builder_tests),

And the change appears in the master log:

2012-06-08 11:29:16+0000 [-] added change Change(revision=u'656db618b5690b4f655856875933fac9692fb7c7', who=u'Iustin Pop <iustin at google.com>', branch=u'stable-2.6', comments=u'Remove one obsolete hlint override', when=1339154956, category=None, project=u'', repository=u'') to database

But even though 'stable-2.6' is in the UNITTEST_BRANCHES list, it's
never picked up.

Second issue is that it seem builds generated via 'try' or even
timed.Nightly sometime are 'stuck' in pending and do not go away for
hours, until master restart. The only thing I see in the logs is:

2012-06-08 10:07:14+0000 [-] WARN attached_slaves: 0 7

because:
2012-06-08 10:17:14+0000 [-] Counter BotMaster.attached_slaves: 0
2012-06-08 10:17:14+0000 [-] Counter AbstractBuildSlave.attached_slaves: 7

Thoughts? Is it due to a broken upgrade? Could I reinitialise/cleanup
build pickles, or the database, etc.? Actually cleaning the saved
pickles doesn't help…

Other strange things in the logs are messages like:

2012-06-08 10:00:09+0000 [HTTPChannel,0,127.0.0.1] WARNING: serializing
ephemeral <DelayedCall 0x3e873b0 [-164.540410995s] called=0 cancelled=0
LoopingCall<10>(JobdirService.poll, *(), **{})()> …

and

2012-06-08 10:00:14+0000 [HTTPChannel,0,127.0.0.1] unable to save build qa-kvm-tiny-#3
2012-06-08 10:00:14+0000 [HTTPChannel,0,127.0.0.1] Unhandled Error
…
  File "/usr/local/lib/python2.6/dist-packages/buildbot-0.8.6p1-py2.6.egg/buildbot/status/build.py", line 408, in saveYourself
   dump(self, open(tmpfilename, "wb"), -1)
  cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Tested with both Twisted 10.0 and 12.1.0, no difference. 

thanks,
iustin