[Buildbot-devel] buildmaster consuming a lot of memory
l10n.moz at googlemail.com
Thu Jan 31 11:07:23 UTC 2008
Ouch, that explains why my waterfall is soooooo slow. Just a single
check-in in en-US triggers 47 builds in the one builder I have :-)
Any suggestion on how to increase that limit without patching my
install? Given all the custom status classes I have, I might be able
to do it from there, but 'least hacky' would be good.
For those curious, I added a database to my own buildbot at
http://l10n.mozilla.org/buildbot/, I'm not using it for a waterfall,
though. I trust Brian in that that's hard to write, so I don't rewrite
it. The code for that uses sqlalchemy, so likely not suitable for
upstream, but that's as far as I grok sql. Code is in
2008/1/29, Brian Warner <warner-buildbot at lothar.com>:
> On Thu, 10 Jan 2008 21:40:36 -0500
> "Dustin J. Mitchell" <dustin at zmanda.com> wrote:
> > Yes, for all of the reasons you gave, and: a buildmaster requires a
> > lot of RAM (~500M in my case) and CPU time, so running lots of
> > buildmasters will quickly eat up system resources.
> So it isn't just me? A couple of my buildmasters seem to eat more and more
> RAM over a period of weeks or months, and I haven't been able to figure out
> why. The CPU usage surprises me: I'm not seeing that, and there's nothing I
> can think of that would really make the buildmaster use a whole lot of CPU
> (unless yours is rendering a big waterfall every few seconds or something).
> I suspect that the memory usage is related to status handling, specifically
> loading in lots of build pickles to render the waterfall. There's a cache
> that limits the number of builds held in memory at once (to something like 10
> or 20 per Builder, I think), but that doesn't feel like it explains the
> It'd be great to figure this out one. The buildmaster ought not to consume
> more than, say, 30 or 40MB. Perhaps saving status in a database would help.
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
More information about the devel