[users at bb.net] buildbot CPU usage

[Pierre Tardy]

Hi,

After carefully looking at your code, it does not look to have obvious
performance flaw.
I think it would have been cheaper for you to just use the forcescheduler
REST api, but if that code works for you this is totally fine.

I saw that you did not manage to get sample in your prod as well, so I did
look at my code again, and now I am able to see the samples.

statprof does not like to stop before start

Please try again with this code:


import statprof
from  twisted.application.internet import TimerService
from buildbot.util.service import BuildbotService
class BuildbotTimerService(TimerService, BuildbotService):
    name = "timer"
    def __init__(self):
        BuildbotService.__init__(self)
        TimerService.__init__(self, 10, self.dump_stats)
        self.started = False
    def dump_stats(self):
        if not self.started:
            statprof.start()
            self.started = True
            return
        statprof.stop()
        with open("profile.txt", "w") as f:
            statprof.display(f)
            statprof.display()
        statprof.start()

c['services'] = [ BuildbotTimerService() ]



Le mer. 10 août 2016 à 14:14, Francesco Di Mizio <francescodimizio at gmail.com>
a écrit :

> Many thanks. Find it attached.
>
> It's a tiny http server. People use a web page to send an HTTP request to
> it to kick off builds.
>
>
> On Wed, Aug 10, 2016 at 1:55 PM, Pierre Tardy <tardyp at gmail.com> wrote:
>
>> I'll take a look.
>>
>> Le mer. 10 août 2016 12:45, Francesco Di Mizio <
>> francescodimizio at gmail.com> a écrit :
>>
>>> I am starting to think it may be due to my own custom scheduler. If I
>>> uploaded the code somewhere could anybody take a look? it's a 150 lines py
>>> script.
>>>
>>> On Wed, Aug 10, 2016 at 12:10 PM, Francesco Di Mizio <
>>> francescodimizio at gmail.com> wrote:
>>>
>>>> Thanks for that Pierre. I have it on on production and the performances
>>>> do not degrade indeed.
>>>>
>>>> buildbot at e2123a50d1fb:~$ cat profile.txt
>>>> No samples recorded.
>>>>
>>>> I need to study statprof a bit now ;)
>>>>
>>>> On Wed, Aug 10, 2016 at 11:20 AM, Vasily <vassnlit at gmail.com> wrote:
>>>>
>>>>> One can also use Intel® VTune™, you know... :-)
>>>>>
>>>>> Thanks,
>>>>> Vasily
>>>>> 09 авг. 2016 г. 23:56 пользователь "Pierre Tardy" <tardyp at gmail.com>
>>>>> написал:
>>>>>
>>>>>> You can add following snippet to your master.cfg. It looks statprof
>>>>>> does not work on osx, so I cannot test it to the end, but I know similar
>>>>>> did work on my prod
>>>>>>
>>>>>>
>>>>>> import statprof
>>>>>> from  twisted.application.internet import TimerService
>>>>>> from buildbot.util.service import BuildbotService
>>>>>> class BuildbotTimerService(TimerService, BuildbotService):
>>>>>>     name = "timer"
>>>>>>     def __init__(self):
>>>>>>         BuildbotService.__init__(self)
>>>>>>         TimerService.__init__(self, 10, self.dump_stats)
>>>>>>     def dump_stats(self):
>>>>>>         statprof.stop()
>>>>>>         with open("profile.txt", "w") as f:
>>>>>>             statprof.display(f)
>>>>>>         statprof.start()
>>>>>>
>>>>>> c['services'] = [ BuildbotTimerService() ]
>>>>>>
>>>>>>
>>>>>> Le mar. 9 août 2016 à 21:44, Francesco Di Mizio <
>>>>>> francescodimizio at gmail.com> a écrit :
>>>>>>
>>>>>>> Pierre,
>>>>>>>
>>>>>>> if you can enlighten on how you intend to use statprof, I will be
>>>>>>> happy to give it a try.
>>>>>>> This problem here is on top of my list right now as it's making my
>>>>>>> prod env unusable.
>>>>>>>
>>>>>>> On Tue, Aug 9, 2016 at 8:25 PM, Pierre Tardy <tardyp at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Please note that cprofile is very intrusive and will lead to about
>>>>>>>> 3x slower code.
>>>>>>>>
>>>>>>>> So I would not run that in prod. This is why I prefer statprof
>>>>>>>> which is using a non intrusive method to get the stats
>>>>>>>>
>>>>>>>> Le mar. 9 août 2016 19:15, Francesco Di Mizio <
>>>>>>>> francescodimizio at gmail.com> a écrit :
>>>>>>>>
>>>>>>>>> On a side note I have managed to get twistd to dump a blob I can
>>>>>>>>> then load and analyze. Got to see if I can run this in production.
>>>>>>>>> If anybody is curious the following worked for me:
>>>>>>>>> twistd --savestats -n --profiler=cprofile  --profile=/crcdata/
>>>>>>>>> profile.stats -y ./buildbot.tac
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Aug 9, 2016 at 5:21 PM, Francesco Di Mizio <
>>>>>>>>> francescodimizio at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Well that's expected to an extent - p4 poller is running 'p4
>>>>>>>>>> changes' every pollinterval seconds.
>>>>>>>>>>
>>>>>>>>>> Anyway just as a test I've tried to disable the poller and still
>>>>>>>>>> at times I am seeing the CPU spiking up to above 100%. I believe I really
>>>>>>>>>> need to profile this somehow.
>>>>>>>>>>
>>>>>>>>>> On Tue, Aug 9, 2016 at 5:08 PM, Dan Kegel <dank at kegel.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Also watch 'top' and see if poller processes are hogging
>>>>>>>>>>> resources.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> users at buildbot.net
>>>>>> https://lists.buildbot.net/mailman/listinfo/users
>>>>>>
>>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160810/fc70e9fb/attachment.html>