[Buildbot-commits] buildbot/buildbot interfaces.py,1.32,1.33 master.py,1.81,1.82

Brian Warner warner at users.sourceforge.net
Fri Oct 14 19:42:41 UTC 2005


Update of /cvsroot/buildbot/buildbot/buildbot
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32254/buildbot

Modified Files:
	interfaces.py master.py 
Log Message:
Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-326
Creator:  Brian Warner <warner at lothar.com>

implement multiple slaves per Builder, allowing concurrent Builds

	* lots: implement multiple slaves per Builder, which means multiple
	current builds per Builder. Some highlights:
	* buildbot/interfaces.py (IBuilderStatus.getState): return a tuple
	of (state,currentBuilds) instead of (state,currentBuild)
	(IBuilderStatus.getCurrentBuilds): replace getCurrentBuild()
	(IBuildStatus.getSlavename): new method, so you can tell which
	slave got used. This only gets set when the build completes.
	(IBuildRequestStatus.getBuilds): new method

	* buildbot/process/builder.py (SlaveBuilder): add a .state
	attribute to track things like ATTACHING and IDLE and BUILDING,
	instead of..
	(Builder): .. the .slaves attribute here, which has been turned
	into a simple list of available slaves. Added a separate
	attaching_slaves list to track ones that are not yet ready for
	builds.
	(Builder.fireTestEvent): put off the test-event callback for a
	reactor turn, to make tests a bit more consistent.
	(Ping): cleaned up the slaveping a bit, now it disconnects if the
	ping fails due to an exception. This needs work, I'm worried that
	a code error could lead to a constantly re-connecting slave.
	Especially since I'm trying to move to a distinct remote_ping
	method, separate from the remote_print that we currently use.
	(BuilderControl.requestBuild): return a convenience Deferred that
	provides an IBuildStatus when the build finishes.
	(BuilderControl.ping): ping all connected slaves, only return True
	if they all respond.

	* buildbot/slave/bot.py (BuildSlave.stopService): stop trying to
	reconnect when we shut down.

	* buildbot/status/builder.py: implement new methods, convert
	one-build-at-a-time methods to handle multiple builds
	* buildbot/status/*.py: do the same in all default status targets
	* buildbot/status/html.py: report the build's slavename in the
	per-Build page, report all buildslaves on the per-Builder page

	* buildbot/test/test_run.py: update/create tests
	* buildbot/test/test_slaves.py: same
	* buildbot/test/test_scheduler.py: remove stale test

	* docs/buildbot.texinfo: document the new builder-specification
	'slavenames' parameter


Index: interfaces.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v
retrieving revision 1.32
retrieving revision 1.33
diff -u -d -r1.32 -r1.33
--- interfaces.py	1 Sep 2005 20:53:21 -0000	1.32
+++ interfaces.py	14 Oct 2005 19:42:39 -0000	1.33
@@ -171,6 +171,10 @@
         pass
     def getBuilderName():
         pass
+    def getBuilds():
+        """Return a list of IBuildStatus objects for each Build that has been
+        started in an attempt to satify this BuildRequest."""
+
     def subscribe(observer):
         """Register a callable that will be invoked (with a single
         IBuildStatus object) for each Build that is created to satisfy this
@@ -211,25 +215,24 @@
 
     def getState():
         # TODO: this isn't nearly as meaningful as it used to be
-        """Return a tuple (state, build=None) for this Builder. 'state' is
-        the so-called 'big-status', indicating overall status (as opposed to
+        """Return a tuple (state, builds) for this Builder. 'state' is the
+        so-called 'big-status', indicating overall status (as opposed to
         which step is currently running). It is a string, one of 'offline',
-        'idle', or 'building'. In the 'building' state, 'build' may be an
-        IBuildStatus object representing the current build (or None if the
-        Builder is in the pre-build ping-the-slave phase)."""
+        'idle', or 'building'. 'builds' is a list of IBuildStatus objects
+        (possibly empty) representing the currently active builds."""
 
-    def getSlave():
-        """Return an ISlaveStatus object for the buildslave that is used by
-        this builder."""
+    def getSlaves():
+        """Return a list of ISlaveStatus objects for the buildslaves that are
+        used by this builder."""
 
     def getPendingBuilds():
         """Return an IBuildRequestStatus object for all upcoming builds
         (those which are ready to go but which are waiting for a buildslave
         to be available."""
 
-    def getCurrentBuild():
-        """Return an IBuildStatus object for the current build in progress.
-        If the state is not 'building', this will be None."""
+    def getCurrentBuilds():
+        """Return a list containing an IBuildStatus object for each build
+        currently in progress."""
         # again, we could probably provide an object for 'waiting' and
         # 'interlocked' too, but things like the Change list might still be
         # subject to change
@@ -360,6 +363,9 @@
     # Once you know the build has finished, the following methods are legal.
     # Before ths build has finished, they all return None.
 
+    def getSlavename():
+        """Return the name of the buildslave which handled this build."""
+
     def getText():
         """Returns a list of strings to describe the build. These are
         intended to be displayed in a narrow column. If more space is
@@ -807,7 +813,9 @@
 
     def requestBuild(request):
         """Queue a L{buildbot.process.base.BuildRequest} object for later
-        building."""
+        building. This returns a Deferred that fires (with an L{IBuildStatus}
+        instance) when the BuildRequest finishes, just as if you did
+        req.waitUntilFinished."""
 
     def getPendingBuilds():
         """Return a list of L{IBuildRequestControl} objects for this Builder.

Index: master.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/master.py,v
retrieving revision 1.81
retrieving revision 1.82
diff -u -d -r1.81 -r1.82
--- master.py	7 Oct 2005 18:45:42 -0000	1.81
+++ master.py	14 Oct 2005 19:42:39 -0000	1.82
@@ -24,7 +24,7 @@
 from buildbot.twcompat import implements
 from buildbot.util import now
 from buildbot.pbutil import NewCredPerspective
-from buildbot.process.builder import Builder
+from buildbot.process.builder import Builder, IDLE
 from buildbot.status.builder import BuilderStatus, SlaveStatus, Status
 from buildbot.changes.changes import Change, ChangeMaster
 from buildbot import interfaces
@@ -289,21 +289,28 @@
         # which is the master-side object that defines and controls a build.
         # They are added by calling botmaster.addBuilder() from the startup
         # code.
+
+        # self.slaves contains a ready BotPerspective instance for each
+        # potential buildslave, i.e. all the ones listed in the config file.
+        # If the slave is connected, self.slaves[slavename].slave will
+        # contain a RemoteReference to their Bot instance. If it is not
+        # connected, that attribute will hold None.
         self.slaves = {} # maps slavename to BotPerspective
         self.statusClientService = None
         self.watchers = {}
 
+
+    # these four are convenience functions for testing
+
     def waitUntilBuilderAttached(self, name):
-        # convenience function for testing
         b = self.builders[name]
-        if b.slaves:
-            return defer.succeed(None)
+        #if b.slaves:
+        #    return defer.succeed(None)
         d = defer.Deferred()
         b.watchers['attach'].append(d)
         return d
 
     def waitUntilBuilderDetached(self, name):
-        # convenience function for testing
         b = self.builders.get(name)
         if not b or not b.slaves:
             return defer.succeed(None)
@@ -311,11 +318,20 @@
         b.watchers['detach'].append(d)
         return d
 
+    def waitUntilBuilderFullyDetached(self, name):
+        b = self.builders.get(name)
+        # TODO: this looks too deeply inside the Builder object
+        if not b or not b.slaves:
+            return defer.succeed(None)
+        d = defer.Deferred()
+        b.watchers['detach_all'].append(d)
+        return d
+
     def waitUntilBuilderIdle(self, name):
-        # convenience function for testing
         b = self.builders[name]
-        for sb in b.slaves.keys():
-            if b.slaves[sb] != "idle":
+        # TODO: this looks way too deeply inside the Builder object
+        for sb in b.slaves:
+            if sb.state != IDLE:
                 d = defer.Deferred()
                 b.watchers['idle'].append(d)
                 return d
@@ -349,17 +365,18 @@
 
         if builder.name in self.builderNames:
             raise KeyError("muliply defined builder '%s'" % builder.name)
-        slavename = builder.slavename
-        if not self.slaves.has_key(slavename):
-            raise KeyError("builder %s uses undefined slave %s" % \
-                           (builder.name, slavename))
+        for slavename in builder.slavenames:
+            if not self.slaves.has_key(slavename):
+                raise KeyError("builder %s uses undefined slave %s" % \
+                               (builder.name, slavename))
 
         self.builders[builder.name] = builder
         self.builderNames.append(builder.name)
         builder.setBotmaster(self)
 
-        slave = self.slaves[slavename]
-        return slave.addBuilder(builder)
+        dl = [self.slaves[slavename].addBuilder(builder)
+              for slavename in builder.slavenames]
+        return defer.DeferredList(dl)
 
     def removeBuilder(self, builder):
         """Stop using a Builder.
@@ -373,9 +390,10 @@
         b = self.builders[builder.name]
         del self.builders[builder.name]
         self.builderNames.remove(builder.name)
-        slave = self.slaves.get(builder.slavename)
-        if slave:
-            return slave.removeBuilder(builder)
+        for slavename in builder.slavenames:
+            slave = self.slaves.get(slavename)
+            if slave:
+                return slave.removeBuilder(builder)
         return defer.succeed(None)
 
     def getPerspective(self, slavename):
@@ -734,9 +752,13 @@
             if type(b) is tuple:
                 raise ValueError("builder %s must be defined with a dict, "
                                  "not a tuple" % b[0])
-            if b['slavename'] not in slavenames:
+            if b.has_key('slavename') and b['slavename'] not in slavenames:
                 raise ValueError("builder %s uses undefined slave %s" \
                                  % (b['name'], b['slavename']))
+            for n in b.get('slavenames', []):
+                if n not in slavenames:
+                    raise ValueError("builder %s uses undefined slave %s" \
+                                     % (b['name'], n))
             if b['name'] in buildernames:
                 raise ValueError("duplicate builder name %s"
                                  % b['name'])





More information about the Commits mailing list