[Buildbot-devel] slave level locks

[Robert Echlin]

-----Original Message-----
From: buildbot-devel-bounces at lists.sourceforge.net
[mailto:buildbot-devel-bounces at lists.sourceforge.net] On Behalf Of Brian
Warner
Sent: Tuesday, April 17, 2007 3:56 AM
To: Stefan Seefeld
Cc: Timothee Besset; buildbot-devel at lists.sourceforge.net
Subject: Re: [Buildbot-devel] slave level locks


> Another question, then, is how to react to high loads. Using locks
> suggest to simply serialize access to the machine, i.e. to queue
builders.
> However, in a build farm multiple equivalent machines may be
available,
> so a real dispatcher is needed that can send build requests to one
among
> many hosts.

Good point.

I had been thinking of the slave-availability thing as being more
advisory
than strict, i.e. put the heavily-loaded buildslave on the bottom of the
list, but not actually forbid using it unless the buildslave admin marks
it
as being out-of-service. But I can see how a load-average-measuring
process
could tell us that no builds should be started on that slave at all
until the
load dropped.

Of course, we'd have to do something more clever than just waiting until
the
load drops below some threshold: I can just imagine 10 builds ready to
go,
all waiting on the same thing, then all being started at the same time,
swamping the machine. There would have to be some better mechanism to
allow
only one build at a time on that machine (spread across all the slaves
that
are running there), and that build could only start when the system load
(possibly caused by non-buildbot processes) goes low enough.

-- rme
Don't we have slave locks already that can be used to serialize all
builds on a single machine? I know we have on our system. That only
serializes stuff through one buildbot master, though. Are you talking
about multiple slaves from multiple masters?

We considered the opposite approach of having two locks per slave and
having half of the builds on a slave use each lock, so as to potentially
complete the builds faster.

Rob
- - -
Relevant master.cfg snippets, with names somewhat sanitized:

  from buildbot import locks

  ...

  lock_rh90 = locks.SlaveLock("bot-rh90slave")
  lock_winxp = locks.SlaveLock("bot-winxpslave")

  ...

  c['builders'].append( b( bot="bot-winxpslave", stream="Branch_name",
locks=[lock_winxp] ) )


<stuff deleted>


--
Rob Echlin
Software Development Environment Prime
Espial IPTV
rechlin at espial.com
Phone: +1 613-230-4770 ext 1150
www.espial.com

-------- Espial Group Inc. Confidential --------
Important Notice: This communication is intended to be received only by
the individual or entity to whom or to which it is addressed and may
contain information that is privileged, confidential and/or subject to
copyright. Any unauthorized use, copying, review or disclosure of this
communication is strictly prohibited. If you have received this
communication in error, please delete the message and notify the sender
by reply email. Thank you for your cooperation.