[users at bb.net] buildbot CPU usage

[Pierre Tardy]

Cool!
I can indeed see 3 spikes.

Looks related to logs and logs compression.

What is the HW spec of your master machine?
How much log does your build generate?

Pierre

Le ven. 26 août 2016 à 11:42, Francesco Di Mizio <francescodimizio at gmail.com>
a écrit :

> Pierre,
>
> I enabled it, waited 1 min and saw the spike, then stopped after a few
> secs. Attached the json.
> Awesome tool btw, work wonders!
>
> On Thu, Aug 25, 2016 at 1:14 PM, Pierre Tardy <tardyp at gmail.com> wrote:
>
>> You can try to hit the button in the morning with a two hours gather
>> period, and hope that you see the spike during that period..
>>
>>
>> Le jeu. 25 août 2016 à 12:17, Francesco Di Mizio <
>> francescodimizio at gmail.com> a écrit :
>>
>>> Thanks a lot! Pierre I will def will give it a shot. I am not sure I'll
>>> be able to smash that 'start recording' button as the UI isusuallystuck
>>> when the CPU spikes. Updates to come!
>>>
>>> On Thu, Aug 25, 2016 at 10:45 AM, Pierre Tardy <tardyp at gmail.com> wrote:
>>>
>>>> Hi Francesco,
>>>>
>>>> I spent some time in order to implement a profiler plugin for buildbot
>>>>
>>>> You can give it a look, and send your profile.json file if you need
>>>> more analysis from me.
>>>> https://github.com/tardyp/buildbot_profiler
>>>>
>>>> Regards,
>>>> Pierre
>>>>
>>>>
>>>> Le mer. 24 août 2016 à 22:43, Francesco Di Mizio <
>>>> francescodimizio at gmail.com> a écrit :
>>>>
>>>>> I've tried and it's not an easy task because of my Win into Vagrant
>>>>> into Docker setup.
>>>>> I'll try again soon when I get a Linux  box!
>>>>>
>>>>> On Fri, Aug 19, 2016 at 5:54 PM, Vasily <just.one.man at yandex.ru>
>>>>> wrote:
>>>>>
>>>>>> Umm, no. VTune has Python support starting 2017 Beta, and, well, it
>>>>>> was my team (at Intel) work actually :-)
>>>>>>
>>>>>> P.S. I'm from Intel, too.
>>>>>>
>>>>>> Thanks,
>>>>>> Vasily
>>>>>> 19 авг. 2016 г. 18:17 пользователь "Francesco Di Mizio" <
>>>>>> francescodimizio at gmail.com> написал:
>>>>>>
>>>>>> I had thought you were making fun of Intel somehow ;)
>>>>>>>
>>>>>>> On Aug 19, 2016 5:07 PM, "Pierre Tardy" <tardyp at gmail.com> wrote:
>>>>>>>
>>>>>>> ahah
>>>>>>>
>>>>>>> I though this was a taunt on me being employed by Intel.
>>>>>>> I actually had mitigated experience with vtune few years ago, and
>>>>>>> didn't know they had python support until then.
>>>>>>> Being an opensource guy, I usually neglegate to look at proprietary
>>>>>>> stuff.
>>>>>>>
>>>>>>> Pierre
>>>>>>>
>>>>>>> Le ven. 19 août 2016 à 12:18, Vasily <just.one.man at yandex.ru> a
>>>>>>> écrit :
>>>>>>>
>>>>>>>> I'm again suggesting to look into Python profiling capabilities of
>>>>>>>> Intel® VTune™ Amplifier. It could run statistical profiling for a long time
>>>>>>>> and display CPU usage over time, so the developer can look at specific time
>>>>>>>> range where CPU usage was too high and see which functions were executed.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Vasily
>>>>>>>> 19 авг. 2016 г. 11:57 пользователь "Pierre Tardy" <tardyp at gmail.com>
>>>>>>>> написал:
>>>>>>>>
>>>>>>>> Hi Francesco,
>>>>>>>>>
>>>>>>>>> Your described setup looks sane to me.
>>>>>>>>>
>>>>>>>>> The problems we are trying to catch are cpu spikes, as far as I
>>>>>>>>> understand, which does not happen for very long, but are very annoying for
>>>>>>>>> users, as it is blocking the reactor.
>>>>>>>>>
>>>>>>>>> This problem is not easy to see in the profile you sent, as this
>>>>>>>>> profile is over long time, so we see the average of each method during the
>>>>>>>>> day and not the spikes.
>>>>>>>>>
>>>>>>>>> What would really be needed is a on-demand profiler which would
>>>>>>>>> detect cpu spikes and only log the stack traces during those times.
>>>>>>>>>
>>>>>>>>> Here is a nice blog pst explaining why statistic profiling is cool
>>>>>>>>> and easy to implement in python.
>>>>>>>>> https://nylas.com/blog/performance
>>>>>>>>>
>>>>>>>>> For 0.9.1 I want to concentrate on scalability, and write a
>>>>>>>>> debugging ui plugin based on those ideas (and probably code)
>>>>>>>>>
>>>>>>>>> That would be great if your team can help on that matter.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Pierre
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160826/7c467537/attachment.html>