[Buildbot-devel] slow buildbot startup

[Vitali Lovich]

Hi,

I’ve noticed that over time our buildbot takes a really long time to start.
At this point it now takes roughly a minute to get to “BuildMaster is running”.

I’ve noticed what look to be substantive delays as soon as it hits loading the builders:

15839 2015-06-04 23:19:52-0700 [-] Setting up database with URL 'sqlite:///state.sqlite'
15840 2015-06-04 23:19:52-0700 [-] setting database journal mode to 'wal'
15841 2015-06-04 23:19:52-0700 [-] adding 5 new slaves, removing 0
15842 2015-06-04 23:19:52-0700 [-] adding 14 new builders, removing 0
15843 2015-06-04 23:19:52-0700 [-] trying to load status pickle from /var/buildbot/apple-ci/master/workdir/A/builder
15844 2015-06-04 23:19:52-0700 [-] added builder A in category
15845 2015-06-04 23:20:03-0700 [-] trying to load status pickle from /var/buildbot/apple-ci/master/workdir/B/builder
15846 2015-06-04 23:20:03-0700 [-] added builder B in category
15847 2015-06-04 23:20:03-0700 [-] trying to load status pickle from /var/buildbot/apple-ci/master/workdir/C/builder
15848 2015-06-04 23:20:03-0700 [-] added builder C in category
15849 2015-06-04 23:20:12-0700 [-] trying to load status pickle from /var/buildbot/apple-ci/master/workdir/D/builder
15850 2015-06-04 23:20:12-0700 [-] added builder D in category
15851 2015-06-04 23:20:17-0700 [-] trying to load status pickle from /var/buildbot/apple-ci/master/workdir/E/builder
15852 2015-06-04 23:20:17-0700 [-] added builder E in category
15853 2015-06-04 23:20:17-0700 [-] trying to load status pickle from /var/buildbot/apple-ci/master/workdir/F/builder
15854 2015-06-04 23:20:17-0700 [-] added builder F Stage in category

We can see adding 6 builders here takes ~25 seconds.

Any suggestions (or tips on how to profile it) are welcome.

Thanks,
Vitali
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20150604/81c1152d/attachment.html>