[Buildbot-devel] Inconsistent UI performance?

[Philippe McLean]

is that 45k = 45000 pickle files?

maybe try deleting the buildmaster directory and create it from scratch, if
you don't care about old build logs. see if that helps

On Tue, Feb 8, 2011 at 2:21 PM, Amber Yust <ayust at yelp.com> wrote:

> Buildbot is single threaded (but asynchronous). This means that when the
> master has lots of load due to a new build request or the like, it will
> temporarily slow down UI responsiveness due to other tasks being processed
> and eating up slices of processing time.
>
> There are plans to move the UI to a separate process, so that Buildbot
> build processing would not have an impact on UI load times, but that has not
> yet been implemented.
>
> ~Amber
>
>
> On Tue, Feb 8, 2011 at 1:47 PM, David Coppit <david at coppit.org> wrote:
>
>> I forgot to mention that there are 45k pickle files (Files of the form
>> "<builder name>/<build number>".)
>>
>> On Tue, 8 Feb 2011, David Coppit wrote:
>>
>> > Sorry for the shot-in-the-dark question...
>> >
>> > We've set up a UI responsiveness monitor on the same machine as our
>> > master. We load the UI once every minute. Less than 1% of the time the
>> > response takes more than 10 seconds, sometimes as long as 60 or 120
>> > seconds.
>> >
>> > We're concerned that this will become a bigger issue as load, builders,
>> > builds, history, etc. increase in the future. We're also concerned that
>> > we've somehow compromised the performance of our master.
>> >
>> > Can anyone confirm whether this is "normal behavior"?
>> >
>> > We've dumped stacks for threads during one of these stalls and saw that
>> > the waterfall page is pretty slow. It appears to be reading a lot of
>> pickle
>> > files. But we haven't done page rendering measurements to confirm, and
>> the
>> > threading model makes us wonder if we're even catching a good snapshot
>> of
>> > the state at the time of the stall.
>> >
>> > Here are some stats:
>> >
>> > Version: Heavily modified 0.8.1
>> > Hardware: Enterprise-class Dell server. 12 GB RAM. Load < .25
>> > Slaves: about 30.
>> >
>> > buildrequests: 29k rows
>> > builds: 31k rows
>> > buildset_properties: 17k rows
>> > buildsets: 8k rows
>> > changes: 90k rows
>> > scheduler_changes: 160k rows (I believe there is a fix after 0.8.1)
>> > schedulers: 19 (2 nightly, 1 try, 16 a subclass of AnyBranch)
>> > sourcestamp_changes: 38k rows
>> > sourcestamps: 8k rows
>> > submissions: 18k rows
>> >
>> > Two logs part of every build are around 5mb in size.
>> >
>> > Build outputs are being stored on a separate filer that is mounted on
>> the
>> > slaves and master.
>> >
>>
>> _____________________________________________________________________
>> David Coppit                                       http://coppit.org/
>>
>>
>> ------------------------------------------------------------------------------
>> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
>> Pinpoint memory and threading errors before they happen.
>> Find and fix more than 250 security defects in the development cycle.
>> Locate bottlenecks in serial and parallel code that limit performance.
>> http://p.sf.net/sfu/intel-dev2devfeb
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110208/60723e1c/attachment.html>