[Buildbot-devel] Buildbot with virtual machines

[Trevor Strohman]

> Does this sound like what you had in mind?

Yes, it does.  I think I could live with serial execution, but I'd be  
interested in the "more complex scheduling rules" eventually.

I probably need to be running something that does this by October 1st  
at the very latest.  If your approach isn't done by then, I will  
cobble something together on my own.

Trevor

On Aug 18, 2006, at 10:18 AM, Michael March wrote:

> On 8/18/06, Trevor Strohman <strohman at cs.umass.edu> wrote:
>> I am building a "virtual" build farm, where almost all of my target
>> platforms are virtual machines running on the same hardware.  I
>> expect that eventually I will have 10 to 15 operating system images
>> on a single dual processor machine (and perhaps more).  This machine
>> also needs to have enough spare capacity to run a few other low-load
>> services.
>>
>> Because there are so many virtual machines, I don't want them all to
>> run at the same time.  Ideally, when a build is submitted, buildbot
>> could launch some of the virtual machines, let them complete the
>> build, then shut them down, then start a new batch of machines.
>>
>> Is this possible?  I don't mind modifying some code since I'll be
>> writing my own system if buildbot doesn't work for me.
>>
>> Trevor
>
>
> Trevor.. Please resist the temptation to roll your own solution. I
> need, and have began working on (very early), the same solution. I
> don't have time to write out a detailed explanation of what my plan is
> so I'll try to get the gist out there...
>
> Here are the highlights:
>
> 1. The system will be based on VMWare Server at first.. (but it
> doesn't have to be..)
> 2. On each server that VMWare Server is running there will be a 'slave
> proxy'. That proxy will read a conf file that will match every VM in
> that VMWare Server's "inventory" (a VMWare Server term) to BuildBot
> buildmaster.
> 3. Periodically, the 'slave master' will read its config file and for
> every virtual slave that has an entry AND VMWare says its available,
> it will make a TCP connection to the BuildBot slave to provide that "I
> am here" heart beat.  So if there are 30 VMs in the config file, 30
> TCP connections will be created.
> 4. If the BuildBot master requests a build, the 'slave proxy' will
> start up (or resume) a VM. That BuildSlave will have the 'slave proxy'
> set as its master.
> 5. The 'slave proxy' will relay all traffic to the master, watching
> for the end of the build. At that time the VM slave will be suspended
> to disk again.
> 6. At first the proxy will run builds serially, but later I plan to
> add more complex scheduling rules.
>
> This setup, if implemented, will accomplish two goals:
>
> 1. Be able to do builds on virtually unlimited amout of VMs regardless
> of resources (except disk space.)
> 2. Little of no modifications to the BuildBot base code.
>
> Does this sound like what you had in mind?
>
> -- 
> <admiral>
>
> Michael F. March ----- mmarch at gmail.com
>          "Seriously" - HSR