[users at bb.net] buildbot CPU usage

[Francesco Di Mizio]

Thanks for that Pierre. I have it on on production and the performances do
not degrade indeed.

buildbot at e2123a50d1fb:~$ cat profile.txt
No samples recorded.

I need to study statprof a bit now ;)

On Wed, Aug 10, 2016 at 11:20 AM, Vasily <vassnlit at gmail.com> wrote:

> One can also use Intel® VTune™, you know... :-)
>
> Thanks,
> Vasily
> 09 авг. 2016 г. 23:56 пользователь "Pierre Tardy" <tardyp at gmail.com>
> написал:
>
>> You can add following snippet to your master.cfg. It looks statprof does
>> not work on osx, so I cannot test it to the end, but I know similar did
>> work on my prod
>>
>>
>> import statprof
>> from  twisted.application.internet import TimerService
>> from buildbot.util.service import BuildbotService
>> class BuildbotTimerService(TimerService, BuildbotService):
>>     name = "timer"
>>     def __init__(self):
>>         BuildbotService.__init__(self)
>>         TimerService.__init__(self, 10, self.dump_stats)
>>     def dump_stats(self):
>>         statprof.stop()
>>         with open("profile.txt", "w") as f:
>>             statprof.display(f)
>>         statprof.start()
>>
>> c['services'] = [ BuildbotTimerService() ]
>>
>>
>> Le mar. 9 août 2016 à 21:44, Francesco Di Mizio <
>> francescodimizio at gmail.com> a écrit :
>>
>>> Pierre,
>>>
>>> if you can enlighten on how you intend to use statprof, I will be happy
>>> to give it a try.
>>> This problem here is on top of my list right now as it's making my prod
>>> env unusable.
>>>
>>> On Tue, Aug 9, 2016 at 8:25 PM, Pierre Tardy <tardyp at gmail.com> wrote:
>>>
>>>> Please note that cprofile is very intrusive and will lead to about 3x
>>>> slower code.
>>>>
>>>> So I would not run that in prod. This is why I prefer statprof which is
>>>> using a non intrusive method to get the stats
>>>>
>>>> Le mar. 9 août 2016 19:15, Francesco Di Mizio <
>>>> francescodimizio at gmail.com> a écrit :
>>>>
>>>>> On a side note I have managed to get twistd to dump a blob I can then
>>>>> load and analyze. Got to see if I can run this in production.
>>>>> If anybody is curious the following worked for me:
>>>>> twistd --savestats -n --profiler=cprofile  --profile=/crcdata/
>>>>> profile.stats -y ./buildbot.tac
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Aug 9, 2016 at 5:21 PM, Francesco Di Mizio <
>>>>> francescodimizio at gmail.com> wrote:
>>>>>
>>>>>> Well that's expected to an extent - p4 poller is running 'p4 changes'
>>>>>> every pollinterval seconds.
>>>>>>
>>>>>> Anyway just as a test I've tried to disable the poller and still at
>>>>>> times I am seeing the CPU spiking up to above 100%. I believe I really need
>>>>>> to profile this somehow.
>>>>>>
>>>>>> On Tue, Aug 9, 2016 at 5:08 PM, Dan Kegel <dank at kegel.com> wrote:
>>>>>>
>>>>>>> Also watch 'top' and see if poller processes are hogging resources.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>> _______________________________________________
>> users mailing list
>> users at buildbot.net
>> https://lists.buildbot.net/mailman/listinfo/users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160810/73461a0f/attachment.html>