[Buildbot-devel] Scaling buildbot

[Jim Rowan]

Use your favorite sysadmin tools to figure out if it’s cpu, disk, or memory bound…  (Is this a small machine?  Does it have enough memory?  Is everything on local disk, or is NFS involved?   Is your disk “fast”?)

On Jun 11, 2015, at 12:16 PM, Dan Kegel <dank at kegel.com> wrote:

> I'm facing long response times on our buildbot instance (a naive one
> configured with
> the default local database).  It can take a minute or more to refresh
> the waterfall
> or to wget a log (it just now took 78 seconds to fetch a 22 megabyte log file),
> and its response time to webhooks is so slow that gitlab by default resends
> them, causing hilarity.  And the git polling time has to be set high
> (six minutes)
> to avoid swamping the git server... and even then, buildbot master grinds to a
> halt any time it's doing git polling (that's when the long response times hit).
> 
> Ideas so far, in order of easiest to hardest:
> - stop running a slave on the same machine as the master (already
> done, but didn't help much)
> - stop using git polling and switch everything to webhooks (will have
> to wait for our gitlab instance to come back from the dead first)
> - switch to MySQL
> - add a master (doc suggests splitting duties where one master handles
> slaves, 2nd master handles everything else)
> 
> Am I missing anything?
> 
> I'm looking forward to buildbot getting better at reporting metrics on
> response times.  (Or does it already do that?)
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel