[users at bb.net] buildbot CPU usage

[Pierre Tardy]

Hi Francesco,

I spent some time in order to implement a profiler plugin for buildbot

You can give it a look, and send your profile.json file if you need more
analysis from me.
https://github.com/tardyp/buildbot_profiler

Regards,
Pierre


Le mer. 24 août 2016 à 22:43, Francesco Di Mizio <francescodimizio at gmail.com>
a écrit :

> I've tried and it's not an easy task because of my Win into Vagrant into
> Docker setup.
> I'll try again soon when I get a Linux  box!
>
> On Fri, Aug 19, 2016 at 5:54 PM, Vasily <just.one.man at yandex.ru> wrote:
>
>> Umm, no. VTune has Python support starting 2017 Beta, and, well, it was
>> my team (at Intel) work actually :-)
>>
>> P.S. I'm from Intel, too.
>>
>> Thanks,
>> Vasily
>> 19 авг. 2016 г. 18:17 пользователь "Francesco Di Mizio" <
>> francescodimizio at gmail.com> написал:
>>
>> I had thought you were making fun of Intel somehow ;)
>>>
>>> On Aug 19, 2016 5:07 PM, "Pierre Tardy" <tardyp at gmail.com> wrote:
>>>
>>> ahah
>>>
>>> I though this was a taunt on me being employed by Intel.
>>> I actually had mitigated experience with vtune few years ago, and didn't
>>> know they had python support until then.
>>> Being an opensource guy, I usually neglegate to look at proprietary
>>> stuff.
>>>
>>> Pierre
>>>
>>> Le ven. 19 août 2016 à 12:18, Vasily <just.one.man at yandex.ru> a écrit :
>>>
>>>> I'm again suggesting to look into Python profiling capabilities of
>>>> Intel® VTune™ Amplifier. It could run statistical profiling for a long time
>>>> and display CPU usage over time, so the developer can look at specific time
>>>> range where CPU usage was too high and see which functions were executed.
>>>>
>>>> Thanks,
>>>> Vasily
>>>> 19 авг. 2016 г. 11:57 пользователь "Pierre Tardy" <tardyp at gmail.com>
>>>> написал:
>>>>
>>>> Hi Francesco,
>>>>>
>>>>> Your described setup looks sane to me.
>>>>>
>>>>> The problems we are trying to catch are cpu spikes, as far as I
>>>>> understand, which does not happen for very long, but are very annoying for
>>>>> users, as it is blocking the reactor.
>>>>>
>>>>> This problem is not easy to see in the profile you sent, as this
>>>>> profile is over long time, so we see the average of each method during the
>>>>> day and not the spikes.
>>>>>
>>>>> What would really be needed is a on-demand profiler which would detect
>>>>> cpu spikes and only log the stack traces during those times.
>>>>>
>>>>> Here is a nice blog pst explaining why statistic profiling is cool and
>>>>> easy to implement in python.
>>>>> https://nylas.com/blog/performance
>>>>>
>>>>> For 0.9.1 I want to concentrate on scalability, and write a debugging
>>>>> ui plugin based on those ideas (and probably code)
>>>>>
>>>>> That would be great if your team can help on that matter.
>>>>>
>>>>> Regards,
>>>>> Pierre
>>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160825/49a2d071/attachment.html>