[users at bb.net] buildbot CPU usage

[Pierre Tardy]

Hi,

Pollers are usually indeed not  scaling as they, hmm, poll.
What you are describing here is hints that the twisted reactor thread is
always busy, which should not happen if you only start 10 builds.
You might have some custom steps which are doing something heavily cpu
bound in the main thread.
What I usually do is to use statprof:
https://pypi.python.org/pypi/statprof/

in order to know what the cpu is doing.
You could create a builder which you can trig whenever you need, and which
would start the profiling, wait a few minutes, and then save profiling to a
file.



Le mar. 2 août 2016 à 17:53, Francesco Di Mizio <francescodimizio at gmail.com>
a écrit :

> Hey Dan,
>
> I am using a p4 poller. Maybe it's suffering from the same problems?
>
> On Tue, Aug 2, 2016 at 5:45 PM, Francesco Di Mizio <
> francescodimizio at gmail.com> wrote:
>
>> I'd like to provide a bit more context.Right after restarting the master
>> and kicking off 10 builds CPU was at 110-120%. This made the UI unusable
>> and basically all the services were stuck, including the REST API.
>> After 3-4 minutes like this and WITH all the 10 builds still running the
>> CPU usage went down to 5%, stayed there for 5 minutes and all was smooth
>> and quick again. From then on it keps oscillating, I've seen spikes of 240%
>> :(
>>
>>
>>
>>
>>
>> On Tue, Aug 2, 2016 at 4:12 PM, Francesco Di Mizio <
>> francescodimizio at gmail.com> wrote:
>>
>>> Sometimes it goes up to 140%. I was not able to relate this with a
>>> particular builds condition - seems like it can happen any time and is not
>>> related to how many builds are going on.
>>>
>>> I usually realize the server got into this state because the web UI gets
>>> stuck. As soon as the CPU% goes back to normal values (2-3% most times) the
>>> web finishes loading just instantly.
>>>
>>> Any pointers as to what might be causing this? Only reason I can think
>>> of is too many people trying to access the web UI simultaniously - may I be
>>> right?
>>>
>>>
>>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160802/d303bf53/attachment.html>