[Buildbot-devel] Playing nice on loaded buildslaves - sharing buildmasters between projects?

[tom fogal]

My advice: don't run all the slaves at the same time.

http://buildbot.net/buildbot/docs/current/manual/cfg-buildslaves.html#latent-buildslaves

Note the 'libvirt' section.  LXC has some libvirt hooks, AFAIK.

I'm exploring this option myself now, albeit with KVM.  It'd be the 
same.  I think it's better to serialize, not parallelize: when a commit 
comes and you need to run 5 builds (on 5 slaves/VMs), start a VM, build, 
kill it, start next VM, build ... lather, rinse, repeat.

This scheme allows me to give my VMs absurd amounts of memory, which is 
important for windows because VStudio builds run like molasses if you 
don't give windows enough memory.

One thing that's on my mind right now is automatic update of slave OSs. 
  Ping me if you're working on that, too.

-tom

On 10/19/2012 09:48 AM, Dan Kegel wrote:
> Perhaps unwisely, I have a whole bunch of buildslaves
> running in lxc containers (very lightweight vm's) on one
> machine, all talking to different buildmasters.
> If I'm not careful, I can run out of memory because too
> many builds happen at once.
>
> A few ideas for how to prevent this:
>
> - don't run so many buildslaves on one physical machine
> - use a single buildmaster
> - vary the hour and minute parameters to timed.Nightly() in master.cfg.
> - make the build script use -j1 instead of -n$ncpus when load is high
> - make build script terminate on startup if the load is really high.
> - use distcc and let it handle load balancing
>
> I'm not sure how I would go about using a single buildmaster
> for all projects, and still have separate waterfalls, one
> for each project.  Is that possible?
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_sfd2d_oct
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel