[Buildbot-devel] buildmaster consuming a lot of memory

[Brian Warner]

On Thu, 10 Jan 2008 21:40:36 -0500
"Dustin J. Mitchell" <dustin at zmanda.com> wrote:
> 
> Yes, for all of the reasons you gave, and: a buildmaster requires a
> lot of RAM (~500M in my case) and CPU time, so running lots of
> buildmasters will quickly eat up system resources.

So it isn't just me? A couple of my buildmasters seem to eat more and more
RAM over a period of weeks or months, and I haven't been able to figure out
why. The CPU usage surprises me: I'm not seeing that, and there's nothing I
can think of that would really make the buildmaster use a whole lot of CPU
(unless yours is rendering a big waterfall every few seconds or something).

I suspect that the memory usage is related to status handling, specifically
loading in lots of build pickles to render the waterfall. There's a cache
that limits the number of builds held in memory at once (to something like 10
or 20 per Builder, I think), but that doesn't feel like it explains the
problem.

It'd be great to figure this out one. The buildmaster ought not to consume
more than, say, 30 or 40MB. Perhaps saving status in a database would help.

cheers,
 -Brian