[Buildbot-devel] Inconsistent UI performance?
ayust at yelp.com
Tue Feb 8 22:21:39 UTC 2011
Buildbot is single threaded (but asynchronous). This means that when the
master has lots of load due to a new build request or the like, it will
temporarily slow down UI responsiveness due to other tasks being processed
and eating up slices of processing time.
There are plans to move the UI to a separate process, so that Buildbot build
processing would not have an impact on UI load times, but that has not yet
On Tue, Feb 8, 2011 at 1:47 PM, David Coppit <david at coppit.org> wrote:
> I forgot to mention that there are 45k pickle files (Files of the form
> "<builder name>/<build number>".)
> On Tue, 8 Feb 2011, David Coppit wrote:
> > Sorry for the shot-in-the-dark question...
> > We've set up a UI responsiveness monitor on the same machine as our
> > master. We load the UI once every minute. Less than 1% of the time the
> > response takes more than 10 seconds, sometimes as long as 60 or 120
> > seconds.
> > We're concerned that this will become a bigger issue as load, builders,
> > builds, history, etc. increase in the future. We're also concerned that
> > we've somehow compromised the performance of our master.
> > Can anyone confirm whether this is "normal behavior"?
> > We've dumped stacks for threads during one of these stalls and saw that
> > the waterfall page is pretty slow. It appears to be reading a lot of
> > files. But we haven't done page rendering measurements to confirm, and
> > threading model makes us wonder if we're even catching a good snapshot of
> > the state at the time of the stall.
> > Here are some stats:
> > Version: Heavily modified 0.8.1
> > Hardware: Enterprise-class Dell server. 12 GB RAM. Load < .25
> > Slaves: about 30.
> > buildrequests: 29k rows
> > builds: 31k rows
> > buildset_properties: 17k rows
> > buildsets: 8k rows
> > changes: 90k rows
> > scheduler_changes: 160k rows (I believe there is a fix after 0.8.1)
> > schedulers: 19 (2 nightly, 1 try, 16 a subclass of AnyBranch)
> > sourcestamp_changes: 38k rows
> > sourcestamps: 8k rows
> > submissions: 18k rows
> > Two logs part of every build are around 5mb in size.
> > Build outputs are being stored on a separate filer that is mounted on the
> > slaves and master.
> David Coppit http://coppit.org/
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the devel