[users at bb.net] buildbot CPU usage

Francesco Di Mizio francescodimizio at gmail.com
Wed Feb 1 17:12:55 UTC 2017


Hey Pierre,

what I noticed is that the query does not take long at all when I run it
myself.
When it is issued through buildbot the state though, it is in a 'idle in
transaction' state, which seems to be something bad according to many blogs
on the web. As if bbot is waiting for postgres to do something but postgres
isnt actually doing anything.

Will give 9.3 but at this point I doubt it'll help.

Thanks!


On Wed, Feb 1, 2017 at 4:56 PM, Pierre Tardy <tardyp at gmail.com> wrote:

>
>
> Le lun. 30 janv. 2017 à 17:49, Francesco Di Mizio <
> francescodimizio at gmail.com> a écrit :
>
>> Digging this up again. Havent moved to a physical server yet. I will
>> first be splitting the bbot server and postgres on different hosts.
>>
>> What I've noticed the CPU spiking is always (and I rea) due to queries
>> like:
>> SELECT logchunks.first_line, logchunks.last_line, logchunks.content,
>> logchunks.compressed FROM logchunks WHERE logchunks.logid = 56343 ORDER BY
>> logchunks.first_line
>>
>> If I am correct that call can only come from CompressLog
>> <https://github.com/buildbot/buildbot/blob/cfe60e31206748bcf01232eb13c0d6177cd04d5d/master/buildbot/db/logs.py#L252> .
>> They do take a very long time (even 1 minute). While running all the system
>> is stuck. I wonder if I should use some custom postgres config, I am simply
>> running the official postgres 9.5 container.
>>
> Hi Francesco,
> If your logs are not anormally large, like you said already this should
> not take one minute.
>
> That said, I have released in 0.9.3 an optimization that removes that
> particular request so that the compression happens on a stream of smaller
> requests.
>
> You might consider an upgrade in order to see if the situation improves.
>
> Pierre
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170201/2aae71f0/attachment.html>


More information about the users mailing list