[users at bb.net] buildbot CPU usage

Francesco Di Mizio francescodimizio at gmail.com
Mon Feb 6 17:27:31 UTC 2017


Pierre,

9.3 seems to have done the trick. Weird I was the only one having such a
huge perf degradation.

Francesco

On Fri, Feb 3, 2017 at 4:55 PM, Pierre Tardy <tardyp at gmail.com> wrote:

> Buildbot <0.9.3 did compress the whole log at once inside a transaction. I
> can imagine this taking very long with 100% CPU.
>
>
> Le mer. 1 févr. 2017 18:12, Francesco Di Mizio <francescodimizio at gmail.com>
> a écrit :
>
>> Hey Pierre,
>>
>> what I noticed is that the query does not take long at all when I run it
>> myself.
>> When it is issued through buildbot the state though, it is in a 'idle in
>> transaction' state, which seems to be something bad according to many blogs
>> on the web. As if bbot is waiting for postgres to do something but postgres
>> isnt actually doing anything.
>>
>> Will give 9.3 but at this point I doubt it'll help.
>>
>> Thanks!
>>
>>
>> On Wed, Feb 1, 2017 at 4:56 PM, Pierre Tardy <tardyp at gmail.com> wrote:
>>
>>
>>
>> Le lun. 30 janv. 2017 à 17:49, Francesco Di Mizio <
>> francescodimizio at gmail.com> a écrit :
>>
>> Digging this up again. Havent moved to a physical server yet. I will
>> first be splitting the bbot server and postgres on different hosts.
>>
>> What I've noticed the CPU spiking is always (and I rea) due to queries
>> like:
>> SELECT logchunks.first_line, logchunks.last_line, logchunks.content,
>> logchunks.compressed FROM logchunks WHERE logchunks.logid = 56343 ORDER BY
>> logchunks.first_line
>>
>> If I am correct that call can only come from CompressLog
>> <https://github.com/buildbot/buildbot/blob/cfe60e31206748bcf01232eb13c0d6177cd04d5d/master/buildbot/db/logs.py#L252> .
>> They do take a very long time (even 1 minute). While running all the system
>> is stuck. I wonder if I should use some custom postgres config, I am simply
>> running the official postgres 9.5 container.
>>
>> Hi Francesco,
>> If your logs are not anormally large, like you said already this should
>> not take one minute.
>>
>> That said, I have released in 0.9.3 an optimization that removes that
>> particular request so that the compression happens on a stream of smaller
>> requests.
>>
>> You might consider an upgrade in order to see if the situation improves.
>>
>> Pierre
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170206/259bceee/attachment.html>


More information about the users mailing list