From warner at users.sourceforge.net Mon Jul 2 09:23:12 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 09:23:12 +0000
Subject: [Buildbot-commits] buildbot/contrib hg_buildbot.py,1.1,1.2
Message-ID:
Update of /cvsroot/buildbot/buildbot/contrib
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7972/contrib
Modified Files:
hg_buildbot.py
Log Message:
[project @ hg_buildbot.py: patch from Frederic Leroy to make it work better]
Original author: warner at lothar.com
Date: 2007-07-02 09:22:27+00:00
Index: hg_buildbot.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/contrib/hg_buildbot.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -d -r1.1 -r1.2
--- hg_buildbot.py 21 May 2006 19:07:27 -0000 1.1
+++ hg_buildbot.py 2 Jul 2007 09:23:10 -0000 1.2
@@ -22,25 +22,14 @@
# TODO: consider doing 'import mercurial.hg' and extract this information
# using the native python
-out = commands.getoutput("hg -v log -r %s" % CHANGESET_ID)
-# TODO: or maybe use --template instead of trying hard to parse everything
-#out = commands.getoutput("hg --template SOMETHING log -r %s" % CHANGESET_ID)
+out = commands.getoutput("hg log -r %s --template '{author}\n{files}\n{desc}'" % CHANGESET_ID)
s = StringIO(out)
-while True:
- line = s.readline()
- if not line:
- break
- if line.startswith("user:"):
- user = line[line.find(":")+1:].strip()
- elif line.startswith("files:"):
- files = line[line.find(":")+1:].strip().split()
- elif line.startswith("description:"):
- comments = "".join(s.readlines())
- if comments[-1] == "\n":
- # this removes the additional newline that hg emits
- comments = comments[:-1]
- break
+user = s.readline().strip()
+# NOTE: this fail when filenames contain spaces. I cannot find a way to get
+# hg to use some other filename separator.
+files = s.readline().strip().split()
+comments = "".join(s.readlines())
change = {
'master': MASTER,
@@ -53,5 +42,5 @@
'files': files,
}
-runner.sendchange(c, True)
+runner.sendchange(change, True)
From warner at users.sourceforge.net Mon Jul 2 09:23:12 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 09:23:12 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.857,1.858
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv7972
Modified Files:
ChangeLog
Log Message:
[project @ hg_buildbot.py: patch from Frederic Leroy to make it work better]
Original author: warner at lothar.com
Date: 2007-07-02 09:22:27+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.857
retrieving revision 1.858
diff -u -d -r1.857 -r1.858
--- ChangeLog 18 Jun 2007 03:58:22 -0000 1.857
+++ ChangeLog 2 Jul 2007 09:23:07 -0000 1.858
@@ -1,3 +1,8 @@
+2007-07-02 Brian Warner
+
+ * contrib/hg_buildbot.py: patch from Frederic Leroy to make this
+ work better.
+
2007-06-17 Brian Warner
* buildbot/test/test_config.py (Factories.testAllSteps): make sure
From warner at users.sourceforge.net Mon Jul 2 17:44:51 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:44:51 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave bot.py,1.25,1.26
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv26865/buildbot/slave
Modified Files:
bot.py
Log Message:
[project @ in the buildslave .tac file, rename 'host' to 'buildmaster_host']
Original author: warner at lothar.com
Date: 2007-07-02 17:37:12+00:00
Index: bot.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/bot.py,v
retrieving revision 1.25
retrieving revision 1.26
diff -u -d -r1.25 -r1.26
--- bot.py 12 Dec 2006 03:24:03 -0000 1.25
+++ bot.py 2 Jul 2007 17:44:48 -0000 1.26
@@ -463,8 +463,9 @@
# debugOpts['failPingOnce'] can be set to True to make the slaveping fail
# exactly once.
- def __init__(self, host, port, name, passwd, basedir, keepalive,
- usePTY, keepaliveTimeout=30, umask=None, debugOpts={}):
+ def __init__(self, buildmaster_host, port, name, passwd, basedir,
+ keepalive, usePTY, keepaliveTimeout=30, umask=None,
+ debugOpts={}):
log.msg("Creating BuildSlave")
service.MultiService.__init__(self)
self.debugOpts = debugOpts.copy()
@@ -476,7 +477,7 @@
self.umask = umask
bf = self.bf = BotFactory(keepalive, keepaliveTimeout)
bf.startLogin(credentials.UsernamePassword(name, passwd), client=bot)
- self.connection = c = internet.TCPClient(host, port, bf)
+ self.connection = c = internet.TCPClient(buildmaster_host, port, bf)
c.setServiceParent(self)
def waitUntilDisconnected(self):
From warner at users.sourceforge.net Mon Jul 2 17:44:52 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:44:52 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_runner.py, 1.13, 1.14
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv26865/buildbot/test
Modified Files:
test_runner.py
Log Message:
[project @ in the buildslave .tac file, rename 'host' to 'buildmaster_host']
Original author: warner at lothar.com
Date: 2007-07-02 17:37:12+00:00
Index: test_runner.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_runner.py,v
retrieving revision 1.13
retrieving revision 1.14
diff -u -d -r1.13 -r1.14
--- test_runner.py 6 Sep 2006 00:41:55 -0000 1.13
+++ test_runner.py 2 Jul 2007 17:44:49 -0000 1.14
@@ -163,12 +163,13 @@
self.failUnless(os.path.exists(tac))
tacfile = open(tac,"rt").read()
self.failUnlessIn("basedir", tacfile)
- self.failUnlessIn("host = 'buildmaster'", tacfile)
+ self.failUnlessIn("buildmaster_host = 'buildmaster'", tacfile)
self.failUnlessIn("port = 1234", tacfile)
self.failUnlessIn("slavename = 'botname'", tacfile)
self.failUnlessIn("passwd = 'passwd'", tacfile)
self.failUnlessIn("keepalive = 600", tacfile)
- self.failUnlessIn("BuildSlave(host, port, slavename", tacfile)
+ self.failUnlessIn("BuildSlave(buildmaster_host, port, slavename",
+ tacfile)
makefile = os.path.join(basedir, "Makefile.sample")
self.failUnlessExists(makefile)
@@ -211,12 +212,13 @@
self.failUnlessExists(os.path.join(basedir, "buildbot.tac.new"))
tacfile = open(os.path.join(basedir, "buildbot.tac.new"),"rt").read()
self.failUnlessIn("basedir", tacfile)
- self.failUnlessIn("host = 'buildmaster'", tacfile)
+ self.failUnlessIn("buildmaster_host = 'buildmaster'", tacfile)
self.failUnlessIn("port = 9999", tacfile)
self.failUnlessIn("slavename = 'newbotname'", tacfile)
self.failUnlessIn("passwd = 'passwd'", tacfile)
self.failUnlessIn("keepalive = 30", tacfile)
- self.failUnlessIn("BuildSlave(host, port, slavename", tacfile)
+ self.failUnlessIn("BuildSlave(buildmaster_host, port, slavename",
+ tacfile)
make = open(os.path.join(basedir, "Makefile.sample"), "rt").read()
self.failUnlessEqual(make, oldmake, "*should* rewrite Makefile.sample")
From warner at users.sourceforge.net Mon Jul 2 17:44:50 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:44:50 +0000
Subject: [Buildbot-commits] buildbot/buildbot/scripts runner.py,1.51,1.52
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/scripts
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv26865/buildbot/scripts
Modified Files:
runner.py
Log Message:
[project @ in the buildslave .tac file, rename 'host' to 'buildmaster_host']
Original author: warner at lothar.com
Date: 2007-07-02 17:37:12+00:00
Index: runner.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/runner.py,v
retrieving revision 1.51
retrieving revision 1.52
diff -u -d -r1.51 -r1.52
--- runner.py 6 Feb 2007 20:36:09 -0000 1.51
+++ runner.py 2 Jul 2007 17:44:48 -0000 1.52
@@ -252,7 +252,7 @@
from buildbot.slave.bot import BuildSlave
basedir = r'%(basedir)s'
-host = '%(host)s'
+buildmaster_host = '%(host)s'
port = %(port)d
slavename = '%(name)s'
passwd = '%(passwd)s'
@@ -261,8 +261,8 @@
umask = %(umask)s
application = service.Application('buildslave')
-s = BuildSlave(host, port, slavename, passwd, basedir, keepalive, usepty,
- umask=umask)
+s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
+ keepalive, usepty, umask=umask)
s.setServiceParent(application)
"""
From warner at users.sourceforge.net Mon Jul 2 17:44:50 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:44:50 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.858,1.859
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv26865
Modified Files:
ChangeLog
Log Message:
[project @ in the buildslave .tac file, rename 'host' to 'buildmaster_host']
Original author: warner at lothar.com
Date: 2007-07-02 17:37:12+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.858
retrieving revision 1.859
diff -u -d -r1.858 -r1.859
--- ChangeLog 2 Jul 2007 09:23:07 -0000 1.858
+++ ChangeLog 2 Jul 2007 17:44:47 -0000 1.859
@@ -1,5 +1,12 @@
2007-07-02 Brian Warner
+ * buildbot/slave/bot.py (BuildSlave.__init__): rename the 'host'
+ argument to 'buildmaster_host', to make it more obvious that this
+ points to the buildmaster. Thanks to Bob Proulx for the
+ suggestion.
+ * buildbot/scripts/runner.py (slaveTAC): same
+ * buildbot/test/test_runner.py (Create.testSlave): match the change
+
* contrib/hg_buildbot.py: patch from Frederic Leroy to make this
work better.
From warner at users.sourceforge.net Mon Jul 2 17:50:19 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:50:19 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.859,1.860
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29340
Modified Files:
ChangeLog
Log Message:
[project @ choose slaves randomly rather than always picking the first one, closes #36]
Original author: warner at lothar.com
Date: 2007-07-02 17:49:12+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.859
retrieving revision 1.860
diff -u -d -r1.859 -r1.860
--- ChangeLog 2 Jul 2007 17:44:47 -0000 1.859
+++ ChangeLog 2 Jul 2007 17:50:16 -0000 1.860
@@ -1,5 +1,14 @@
2007-07-02 Brian Warner
+ * buildbot/process/builder.py (Builder.maybeStartBuild): choose
+ the slave randomly rather than always taking the first one. Thanks
+ to Pike for the patch. Closes #36.
+ * buildbot/test/test_slaves.py: match the change. to avoid huge
+ changes to the tests, I added a CHOOSE_SLAVES_RANDOMLY flag which
+ enables/disables the round-robin-ness (enabled by default), and
+ some unit tests disable it.
+ * docs/buildbot.texinfo (Buildmaster Architecture): document it
+
* buildbot/slave/bot.py (BuildSlave.__init__): rename the 'host'
argument to 'buildmaster_host', to make it more obvious that this
points to the buildmaster. Thanks to Bob Proulx for the
From warner at users.sourceforge.net Mon Jul 2 17:50:19 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:50:19 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process builder.py,1.43,1.44
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29340/buildbot/process
Modified Files:
builder.py
Log Message:
[project @ choose slaves randomly rather than always picking the first one, closes #36]
Original author: warner at lothar.com
Date: 2007-07-02 17:49:12+00:00
Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/builder.py,v
retrieving revision 1.43
retrieving revision 1.44
diff -u -d -r1.43 -r1.44
--- builder.py 23 Jan 2007 08:01:45 -0000 1.43
+++ builder.py 2 Jul 2007 17:50:17 -0000 1.44
@@ -1,5 +1,5 @@
-#! /usr/bin/python
+import random
from zope.interface import implements
from twisted.python import log, components
from twisted.spread import pb
@@ -224,6 +224,7 @@
expectations = None # this is created the first time we get a good build
START_BUILD_TIMEOUT = 10
+ CHOOSE_SLAVES_RANDOMLY = True # disabled for determinism during tests
def __init__(self, setup, builder_status):
"""
@@ -502,15 +503,18 @@
if not self.buildable:
self.updateBigStatus()
return # nothing to do
- # find the first idle slave
- for sb in self.slaves:
- if sb.isAvailable():
- break
- else:
+
+ # pick an idle slave
+ available_slaves = [sb for sb in self.slaves if sb.isAvailable()]
+ if not available_slaves:
log.msg("%s: want to start build, but we don't have a remote"
% self)
self.updateBigStatus()
return
+ if self.CHOOSE_SLAVES_RANDOMLY:
+ sb = random.choice(available_slaves)
+ else:
+ sb = available_slaves[0]
# there is something to build, and there is a slave on which to build
# it. Grab the oldest request, see if we can merge it with anything
From warner at users.sourceforge.net Mon Jul 2 17:50:19 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:50:19 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_slaves.py,1.6,1.7
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29340/buildbot/test
Modified Files:
test_slaves.py
Log Message:
[project @ choose slaves randomly rather than always picking the first one, closes #36]
Original author: warner at lothar.com
Date: 2007-07-02 17:49:12+00:00
Index: test_slaves.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slaves.py,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -d -r1.6 -r1.7
--- test_slaves.py 11 Dec 2006 08:23:29 -0000 1.6
+++ test_slaves.py 2 Jul 2007 17:50:17 -0000 1.7
@@ -68,6 +68,7 @@
# rotate or attempt any sort of load-balancing, two builds in
# sequence should both use the first slave. This may change later if
# we move to a more sophisticated scheme.
+ b.CHOOSE_SLAVES_RANDOMLY = False
d = self.doBuild("b1")
d.addCallback(self._testSequence_1)
@@ -91,12 +92,16 @@
return d1
def _testSimultaneous_1(self, res, d2):
self.failUnlessEqual(res.getResults(), SUCCESS)
- self.failUnlessEqual(res.getSlavename(), "bot1")
- d2.addCallback(self._testSimultaneous_2)
+ b1_slavename = res.getSlavename()
+ d2.addCallback(self._testSimultaneous_2, b1_slavename)
return d2
- def _testSimultaneous_2(self, res):
+ def _testSimultaneous_2(self, res, b1_slavename):
self.failUnlessEqual(res.getResults(), SUCCESS)
- self.failUnlessEqual(res.getSlavename(), "bot2")
+ b2_slavename = res.getSlavename()
+ # make sure the two builds were run by different slaves
+ slavenames = [b1_slavename, b2_slavename]
+ slavenames.sort()
+ self.failUnlessEqual(slavenames, ["bot1", "bot2"])
def testFallback1(self):
# detach the first slave, verify that a build is run using the second
@@ -125,6 +130,8 @@
# reduce the ping time so we'll failover faster
self.master.botmaster.builders["b1"].START_BUILD_TIMEOUT = 1
+ assert self.master.botmaster.builders["b1"].CHOOSE_SLAVES_RANDOMLY
+ self.master.botmaster.builders["b1"].CHOOSE_SLAVES_RANDOMLY = False
self.disappearSlave("bot1", "b1")
d = self.doBuild("b1")
d.addCallback(self._testFallback2_1)
@@ -133,7 +140,9 @@
self.failUnlessEqual(res.getResults(), SUCCESS)
self.failUnlessEqual(res.getSlavename(), "bot2")
b1slaves = self.master.botmaster.builders["b1"].slaves
- self.failUnlessEqual(len(b1slaves), 1)
+ # TODO: this check fails sometimes, sometimes len(b1slaves)==2,
+ # sometimes it is empty
+ self.failUnlessEqual(len(b1slaves), 1, b1slaves)
self.failUnlessEqual(b1slaves[0].slave.slavename, "bot2")
@@ -152,6 +161,7 @@
self.slaves['bot1'].debugOpts["stallPings"] = (10, timers)
br = BuildRequest("forced", SourceStamp())
d1 = br.waitUntilFinished()
+ self.master.botmaster.builders["b1"].CHOOSE_SLAVES_RANDOMLY = False
self.control.getBuilder("b1").requestBuild(br)
s1 = br.status # this is a BuildRequestStatus
# give it a chance to start pinging
From warner at users.sourceforge.net Mon Jul 2 17:50:19 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 02 Jul 2007 17:50:19 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.99,1.100
Message-ID:
Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29340/docs
Modified Files:
buildbot.texinfo
Log Message:
[project @ choose slaves randomly rather than always picking the first one, closes #36]
Original author: warner at lothar.com
Date: 2007-07-02 17:49:12+00:00
Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.99
retrieving revision 1.100
diff -u -d -r1.99 -r1.100
--- buildbot.texinfo 18 Jun 2007 02:52:49 -0000 1.99
+++ buildbot.texinfo 2 Jul 2007 17:50:17 -0000 1.100
@@ -478,8 +478,8 @@
may be multiple requests to build the current HEAD revision). These
requests are merged into a single Build instance, which includes the
SourceStamp that describes what exact version of the source code
-should be used for the build. The Build is then assigned to a
-SlaveBuilder and the build begins.
+should be used for the build. The Build is then randomly assigned to a
+free SlaveBuilder and the build begins.
@node Status Delivery Architecture, , Buildmaster Architecture, System Architecture
From warner at users.sourceforge.net Tue Jul 3 02:22:21 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 02:22:21 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py, 1.19,
1.20 test_slaves.py, 1.7, 1.8
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30105/buildbot/test
Modified Files:
runutils.py test_slaves.py
Log Message:
[project @ test_slaves.py: fix an intermittent failure, disappeared slaves were reconnecting]
Original author: warner at lothar.com
Date: 2007-07-03 02:21:43+00:00
Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.19
retrieving revision 1.20
diff -u -d -r1.19 -r1.20
--- runutils.py 18 Jun 2007 02:52:49 -0000 1.19
+++ runutils.py 3 Jul 2007 02:22:19 -0000 1.20
@@ -206,6 +206,9 @@
broker = bot.builders[buildername].remote.broker
broker.dataReceived = discard # seal its ears
broker.transport.write = discard # and take away its voice
+ # also discourage it from reconnecting once the connection goes away
+ assert self.slaves[slavename].bf.continueTrying
+ self.slaves[slavename].bf.continueTrying = False
def ghostSlave(self):
# the slave thinks it has lost the connection, and initiated a
Index: test_slaves.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slaves.py,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -d -r1.7 -r1.8
--- test_slaves.py 2 Jul 2007 17:50:17 -0000 1.7
+++ test_slaves.py 3 Jul 2007 02:22:19 -0000 1.8
@@ -128,10 +128,11 @@
# claimed for the build, and that the failing one is removed from the
# list.
+ b1 = self.master.botmaster.builders["b1"]
# reduce the ping time so we'll failover faster
- self.master.botmaster.builders["b1"].START_BUILD_TIMEOUT = 1
- assert self.master.botmaster.builders["b1"].CHOOSE_SLAVES_RANDOMLY
- self.master.botmaster.builders["b1"].CHOOSE_SLAVES_RANDOMLY = False
+ b1.START_BUILD_TIMEOUT = 1
+ assert b1.CHOOSE_SLAVES_RANDOMLY
+ b1.CHOOSE_SLAVES_RANDOMLY = False
self.disappearSlave("bot1", "b1")
d = self.doBuild("b1")
d.addCallback(self._testFallback2_1)
@@ -140,9 +141,7 @@
self.failUnlessEqual(res.getResults(), SUCCESS)
self.failUnlessEqual(res.getSlavename(), "bot2")
b1slaves = self.master.botmaster.builders["b1"].slaves
- # TODO: this check fails sometimes, sometimes len(b1slaves)==2,
- # sometimes it is empty
- self.failUnlessEqual(len(b1slaves), 1, b1slaves)
+ self.failUnlessEqual(len(b1slaves), 1, "whoops: %s" % (b1slaves,))
self.failUnlessEqual(b1slaves[0].slave.slavename, "bot2")
From warner at users.sourceforge.net Tue Jul 3 02:22:21 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 02:22:21 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.860,1.861
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30105
Modified Files:
ChangeLog
Log Message:
[project @ test_slaves.py: fix an intermittent failure, disappeared slaves were reconnecting]
Original author: warner at lothar.com
Date: 2007-07-03 02:21:43+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.860
retrieving revision 1.861
diff -u -d -r1.860 -r1.861
--- ChangeLog 2 Jul 2007 17:50:16 -0000 1.860
+++ ChangeLog 3 Jul 2007 02:22:18 -0000 1.861
@@ -1,5 +1,16 @@
2007-07-02 Brian Warner
+ * buildbot/test/runutils.py (RunMixin.disappearSlave): once a
+ slave has been disappeared, don't let it reconnect. This was
+ causing an intermittent failure in test_slaves.py, when the slave
+ that was supposed to be gone managed to come back by the end of
+ the test and affect the count.
+ * buildbot/test/test_slaves.py (Slave.testFallback2): minor
+ refactorings
+ * buildbot/process/builder.py (SlaveBuilder.__repr__): added some
+ diagnostic messages to track down this problem
+ (Builder.startBuild): same
+
* buildbot/process/builder.py (Builder.maybeStartBuild): choose
the slave randomly rather than always taking the first one. Thanks
to Pike for the patch. Closes #36.
From warner at users.sourceforge.net Tue Jul 3 02:22:21 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 02:22:21 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process builder.py,1.44,1.45
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30105/buildbot/process
Modified Files:
builder.py
Log Message:
[project @ test_slaves.py: fix an intermittent failure, disappeared slaves were reconnecting]
Original author: warner at lothar.com
Date: 2007-07-03 02:21:43+00:00
Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/builder.py,v
retrieving revision 1.44
retrieving revision 1.45
diff -u -d -r1.44 -r1.45
--- builder.py 2 Jul 2007 17:50:17 -0000 1.44
+++ builder.py 3 Jul 2007 02:22:19 -0000 1.45
@@ -26,6 +26,17 @@
self.ping_watchers = []
self.state = ATTACHING
self.remote = None
+ self.slave = None
+ self.builder_name = None
+
+ def __repr__(self):
+ r = ""
+ return r
def setBuilder(self, b):
self.builder = b
@@ -551,7 +562,7 @@
self.building.append(build)
self.updateBigStatus()
- log.msg("starting build %s.. pinging the slave" % build)
+ log.msg("starting build %s.. pinging the slave %s" % (build, sb))
# ping the slave to make sure they're still there. If they're fallen
# off the map (due to a NAT timeout or something), this will fail in
# a couple of minutes, depending upon the TCP timeout. TODO: consider
From warner at users.sourceforge.net Tue Jul 3 17:25:10 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 17:25:10 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py, 1.20,
1.21 test_run.py, 1.43, 1.44 test_slaves.py, 1.8, 1.9
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv11483/buildbot/test
Modified Files:
runutils.py test_run.py test_slaves.py
Log Message:
[project @ RunMixin.disappearSlave: allow callers to control the allowReconnect behavior]
Original author: warner at lothar.com
Date: 2007-07-03 17:24:36+00:00
Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.20
retrieving revision 1.21
diff -u -d -r1.20 -r1.21
--- runutils.py 3 Jul 2007 02:22:19 -0000 1.20
+++ runutils.py 3 Jul 2007 17:25:08 -0000 1.21
@@ -193,7 +193,8 @@
broker.transport.loseConnection()
del self.slaves['bot1']
- def disappearSlave(self, slavename="bot1", buildername="dummy"):
+ def disappearSlave(self, slavename="bot1", buildername="dummy",
+ allowReconnect=False):
# the slave's host has vanished off the net, leaving the connection
# dangling. This will be detected quickly by app-level keepalives or
# a ping, or slowly by TCP timeouts.
@@ -206,9 +207,10 @@
broker = bot.builders[buildername].remote.broker
broker.dataReceived = discard # seal its ears
broker.transport.write = discard # and take away its voice
- # also discourage it from reconnecting once the connection goes away
- assert self.slaves[slavename].bf.continueTrying
- self.slaves[slavename].bf.continueTrying = False
+ if not allowReconnect:
+ # also discourage it from reconnecting once the connection goes away
+ assert self.slaves[slavename].bf.continueTrying
+ self.slaves[slavename].bf.continueTrying = False
def ghostSlave(self):
# the slave thinks it has lost the connection, and initiated a
Index: test_run.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_run.py,v
retrieving revision 1.43
retrieving revision 1.44
diff -u -d -r1.43 -r1.44
--- test_run.py 11 Dec 2006 08:23:29 -0000 1.43
+++ test_run.py 3 Jul 2007 17:25:08 -0000 1.44
@@ -186,8 +186,7 @@
def testIdle2(self):
# now suppose the slave goes missing
- self.slaves['bot1'].bf.continueTrying = 0
- self.disappearSlave()
+ self.disappearSlave(allowReconnect=False)
# forcing a build will work: the build detect that the slave is no
# longer available and will be re-queued. Wait 5 seconds, then check
@@ -356,8 +355,7 @@
self.failUnlessEqual(res, True)
# now, before any build is run, make the slave disappear
- self.slaves['bot1'].bf.continueTrying = 0
- self.disappearSlave()
+ self.disappearSlave(allowReconnect=False)
# at this point, a ping to the slave should timeout
d = bc.ping(1)
@@ -375,8 +373,7 @@
self.failUnlessEqual(ss.getAdmin(), "one")
# now, before any build is run, make the first slave disappear
- self.slaves['bot1'].bf.continueTrying = 0
- self.disappearSlave()
+ self.disappearSlave(allowReconnect=False)
d = self.master.botmaster.waitUntilBuilderDetached("dummy")
# now let the new slave take over
@@ -447,7 +444,7 @@
d = self.master.botmaster.waitUntilBuilderDetached("dummy")
# whoops! how careless of me.
- self.disappearSlave()
+ self.disappearSlave(allowReconnect=True)
# the slave will realize the connection is lost within 2 seconds, and
# reconnect.
d.addCallback(self._testSlaveTimeout_2)
Index: test_slaves.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slaves.py,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -d -r1.8 -r1.9
--- test_slaves.py 3 Jul 2007 02:22:19 -0000 1.8
+++ test_slaves.py 3 Jul 2007 17:25:08 -0000 1.9
@@ -133,7 +133,7 @@
b1.START_BUILD_TIMEOUT = 1
assert b1.CHOOSE_SLAVES_RANDOMLY
b1.CHOOSE_SLAVES_RANDOMLY = False
- self.disappearSlave("bot1", "b1")
+ self.disappearSlave("bot1", "b1", allowReconnect=False)
d = self.doBuild("b1")
d.addCallback(self._testFallback2_1)
return d
From warner at users.sourceforge.net Tue Jul 3 17:25:10 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 17:25:10 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.861,1.862
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv11483
Modified Files:
ChangeLog
Log Message:
[project @ RunMixin.disappearSlave: allow callers to control the allowReconnect behavior]
Original author: warner at lothar.com
Date: 2007-07-03 17:24:36+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.861
retrieving revision 1.862
diff -u -d -r1.861 -r1.862
--- ChangeLog 3 Jul 2007 02:22:18 -0000 1.861
+++ ChangeLog 3 Jul 2007 17:25:07 -0000 1.862
@@ -1,3 +1,12 @@
+2007-07-03 Brian Warner
+
+ * buildbot/test/runutils.py (RunMixin.disappearSlave): oops, add
+ an allowReconnect= argument to let callers control whether they
+ want this don't-let-it-reconnect behavior, since some tests care.
+ Factor out the .continueTrying=False lines from those callers.
+ * buildbot/test/test_slaves.py: same
+ * buildbot/test/test_run.py: same
+
2007-07-02 Brian Warner
* buildbot/test/runutils.py (RunMixin.disappearSlave): once a
From warner at users.sourceforge.net Tue Jul 3 17:59:05 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 17:59:05 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.80,1.81
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv24963/buildbot/slave
Modified Files:
commands.py
Log Message:
[project @ rmdirRecursive: chmod files to 0700 before deleting them. Closes #29.]
Original author: warner at lothar.com
Date: 2007-07-03 17:51:51+00:00
Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.80
retrieving revision 1.81
diff -u -d -r1.80 -r1.81
--- commands.py 17 Apr 2007 06:38:50 -0000 1.80
+++ commands.py 3 Jul 2007 17:59:03 -0000 1.81
@@ -66,17 +66,24 @@
os.remove(dir)
return
+ # Verify the directory is read/write/execute for the current user
+ os.chmod(dir, 0700)
+
for name in os.listdir(dir):
full_name = os.path.join(dir, name)
# on Windows, if we don't have write permission we can't remove
# the file/directory either, so turn that on
if os.name == 'nt':
if not os.access(full_name, os.W_OK):
+ # I think this is now redundant, but I don't have an NT
+ # machine to test on, so I'm going to leave it in place
+ # -warner
os.chmod(full_name, 0600)
+
if os.path.isdir(full_name):
rmdirRecursive(full_name)
else:
- # print "removing file", full_name
+ os.chmod(full_name, 0700)
os.remove(full_name)
os.rmdir(dir)
From warner at users.sourceforge.net Tue Jul 3 17:59:05 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 17:59:05 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.862,1.863
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv24963
Modified Files:
ChangeLog
Log Message:
[project @ rmdirRecursive: chmod files to 0700 before deleting them. Closes #29.]
Original author: warner at lothar.com
Date: 2007-07-03 17:51:51+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.862
retrieving revision 1.863
diff -u -d -r1.862 -r1.863
--- ChangeLog 3 Jul 2007 17:25:07 -0000 1.862
+++ ChangeLog 3 Jul 2007 17:59:02 -0000 1.863
@@ -1,5 +1,14 @@
2007-07-03 Brian Warner
+ * buildbot/slave/commands.py (rmdirRecursive): chmod everything to
+ 0700 before removing it, to deal with situations where a build will
+ leave files around without write permissions and the 'copy' or
+ 'clobber' VC checkout modes need to blow away the tree first.
+ Thanks to Steve Milner for the suggestion and the patch (which I
+ mangled horribly). Closes #29.
+ * buildbot/test/test_slavecommand.py (Utilities.test_rmdirRecursive):
+ test for it
+
* buildbot/test/runutils.py (RunMixin.disappearSlave): oops, add
an allowReconnect= argument to let callers control whether they
want this don't-let-it-reconnect behavior, since some tests care.
From warner at users.sourceforge.net Tue Jul 3 17:59:05 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 17:59:05 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_slavecommand.py, 1.24,
1.25
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv24963/buildbot/test
Modified Files:
test_slavecommand.py
Log Message:
[project @ rmdirRecursive: chmod files to 0700 before deleting them. Closes #29.]
Original author: warner at lothar.com
Date: 2007-07-03 17:51:51+00:00
Index: test_slavecommand.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slavecommand.py,v
retrieving revision 1.24
retrieving revision 1.25
diff -u -d -r1.24 -r1.25
--- test_slavecommand.py 11 Dec 2006 08:23:29 -0000 1.24
+++ test_slavecommand.py 3 Jul 2007 17:59:03 -0000 1.25
@@ -14,6 +14,42 @@
# test slavecommand.py by running the various commands with a fake
# SlaveBuilder object that logs the calls to sendUpdate()
+class Utilities(unittest.TestCase):
+ def mkdir(self, basedir, path, mode=None):
+ fn = os.path.join(basedir, path)
+ os.makedirs(fn)
+ if mode is not None:
+ os.chmod(fn, mode)
+
+ def touch(self, basedir, path, mode=None):
+ fn = os.path.join(basedir, path)
+ f = open(fn, "w")
+ f.write("touch\n")
+ f.close()
+ if mode is not None:
+ os.chmod(fn, mode)
+
+ def test_rmdirRecursive(self):
+ basedir = "slavecommand/Utilities/test_rmdirRecursive"
+ os.makedirs(basedir)
+ d = os.path.join(basedir, "doomed")
+ self.mkdir(d, "a/b")
+ self.touch(d, "a/b/1.txt")
+ self.touch(d, "a/b/2.txt", 0444)
+ self.touch(d, "a/b/3.txt", 0)
+ self.mkdir(d, "a/c")
+ self.touch(d, "a/c/1.txt")
+ self.touch(d, "a/c/2.txt", 0444)
+ self.touch(d, "a/c/3.txt", 0)
+ os.chmod(os.path.join(d, "a/c"), 0444)
+ self.mkdir(d, "a/d")
+ self.touch(d, "a/d/1.txt")
+ self.touch(d, "a/d/2.txt", 0444)
+ self.touch(d, "a/d/3.txt", 0)
+ os.chmod(os.path.join(d, "a/d"), 0)
+
+ commands.rmdirRecursive(d)
+ self.failIf(os.path.exists(d))
class ShellBase(SignalMixin):
From warner at users.sourceforge.net Tue Jul 3 19:17:51 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 19:17:51 +0000
Subject: [Buildbot-commits] buildbot/buildbot/steps source.py,1.6,1.7
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/steps
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv26243/buildbot/steps
Modified Files:
source.py
Log Message:
[project @ when using a patch in a Source step, add [patch] to the description. Closes #49]
Original author: warner at lothar.com
Date: 2007-07-03 19:14:45+00:00
Index: source.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/steps/source.py,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -d -r1.6 -r1.7
--- source.py 18 Jun 2007 02:52:49 -0000 1.6
+++ source.py 3 Jul 2007 19:17:49 -0000 1.7
@@ -465,6 +465,8 @@
revstuff.append("[branch]")
if revision is not None:
revstuff.append("r%s" % revision)
+ if patch is not None:
+ revstuff.append("[patch]")
self.description.extend(revstuff)
self.descriptionDone.extend(revstuff)
From warner at users.sourceforge.net Tue Jul 3 19:17:51 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 19:17:51 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.863,1.864
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv26243
Modified Files:
ChangeLog
Log Message:
[project @ when using a patch in a Source step, add [patch] to the description. Closes #49]
Original author: warner at lothar.com
Date: 2007-07-03 19:14:45+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.863
retrieving revision 1.864
diff -u -d -r1.863 -r1.864
--- ChangeLog 3 Jul 2007 17:59:02 -0000 1.863
+++ ChangeLog 3 Jul 2007 19:17:49 -0000 1.864
@@ -1,5 +1,9 @@
2007-07-03 Brian Warner
+ * buildbot/steps/source.py (SVN.startVC): when applying a patch,
+ add "[patch]" to the step's display. Thanks to Dustin Mitchell for
+ the patch. Closes #49.
+
* buildbot/slave/commands.py (rmdirRecursive): chmod everything to
0700 before removing it, to deal with situations where a build will
leave files around without write permissions and the 'copy' or
From warner at users.sourceforge.net Tue Jul 3 19:21:31 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 19:21:31 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.864,1.865
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv27853
Modified Files:
ChangeLog
Log Message:
[project @ buildbot.texinfo: correct the description of Dependent schedulers]
Original author: warner at lothar.com
Date: 2007-07-03 19:21:12+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.864
retrieving revision 1.865
diff -u -d -r1.864 -r1.865
--- ChangeLog 3 Jul 2007 19:17:49 -0000 1.864
+++ ChangeLog 3 Jul 2007 19:21:29 -0000 1.865
@@ -1,5 +1,8 @@
2007-07-03 Brian Warner
+ * docs/buildbot.texinfo (Scheduler Types): correct the description
+ of Dependent schedulers. Thanks to Greg Ward for the patch.
+
* buildbot/steps/source.py (SVN.startVC): when applying a patch,
add "[patch]" to the step's display. Thanks to Dustin Mitchell for
the patch. Closes #49.
From warner at users.sourceforge.net Tue Jul 3 19:21:32 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 03 Jul 2007 19:21:32 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.100,1.101
Message-ID:
Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv27853/docs
Modified Files:
buildbot.texinfo
Log Message:
[project @ buildbot.texinfo: correct the description of Dependent schedulers]
Original author: warner at lothar.com
Date: 2007-07-03 19:21:12+00:00
Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.100
retrieving revision 1.101
diff -u -d -r1.100 -r1.101
--- buildbot.texinfo 2 Jul 2007 17:50:17 -0000 1.100
+++ buildbot.texinfo 3 Jul 2007 19:21:29 -0000 1.101
@@ -2096,11 +2096,10 @@
follows multiple branches at once. Each branch gets a separate timer.
@item Dependent
-This scheduler watches an ``upstream'' Builder. When that Builder
-successfully builds a particular set of Changes, it triggers builds of
-the same code on a configured set of ``downstream'' builders. The next
-section (@pxref{Build Dependencies}) describes this scheduler in more
-detail.
+This scheduler watches an ``upstream'' Scheduler. When all the
+Builders launched by that Scheduler successfully finish, the Dependent
+scheduler is triggered. The next section (@pxref{Build Dependencies})
+describes this scheduler in more detail.
@item Periodic
This simple scheduler just triggers a build every N seconds.
From warner at users.sourceforge.net Wed Jul 4 05:09:43 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:09:43 +0000
Subject: [Buildbot-commits] buildbot/buildbot/steps shell.py,1.5,1.6
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/steps
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv31546/buildbot/steps
Modified Files:
shell.py
Log Message:
[project @ ShellCommand: allow workdir= to be a WithProperties. Closes #43.]
Original author: warner at lothar.com
Date: 2007-07-04 05:08:34+00:00
Index: shell.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/steps/shell.py,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -d -r1.5 -r1.6
--- shell.py 18 Jun 2007 02:52:49 -0000 1.5
+++ shell.py 4 Jul 2007 05:09:41 -0000 1.6
@@ -170,6 +170,11 @@
command_argv.append(argv)
return command_argv
+ def _interpolateWorkdir(self, workdir):
+ if isinstance(workdir, WithProperties):
+ return workdir.render(self.build)
+ return workdir
+
def setupEnvironment(self, cmd):
# merge in anything from Build.slaveEnvironment . Earlier steps
# (perhaps ones which compile libraries or sub-projects that need to
@@ -213,6 +218,7 @@
assert isinstance(command, (list, tuple, str))
# create the actual RemoteShellCommand instance now
kwargs = self.remote_kwargs
+ kwargs['workdir'] = self._interpolateWorkdir(kwargs['workdir'])
kwargs['command'] = command
kwargs['logfiles'] = self.logfiles
cmd = RemoteShellCommand(**kwargs)
From warner at users.sourceforge.net Wed Jul 4 05:09:44 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:09:44 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.101,1.102
Message-ID:
Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv31546/docs
Modified Files:
buildbot.texinfo
Log Message:
[project @ ShellCommand: allow workdir= to be a WithProperties. Closes #43.]
Original author: warner at lothar.com
Date: 2007-07-04 05:08:34+00:00
Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.101
retrieving revision 1.102
diff -u -d -r1.101 -r1.102
--- buildbot.texinfo 3 Jul 2007 19:21:29 -0000 1.101
+++ buildbot.texinfo 4 Jul 2007 05:09:42 -0000 1.102
@@ -3970,7 +3970,10 @@
You can use build properties in ShellCommands by using the
@code{WithProperties} wrapper when setting the arguments of the
ShellCommand. This interpolates the named build properties into the
-generated shell command.
+generated shell command. You can also use a @code{WithProperties} as
+the @code{workdir=} argument: this allows the working directory for a
+command to be varied for each build, depending upon various build
+properties.
@example
from buildbot.steps.shell import ShellCommand, WithProperties
From warner at users.sourceforge.net Wed Jul 4 05:09:43 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:09:43 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.865,1.866
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv31546
Modified Files:
ChangeLog
Log Message:
[project @ ShellCommand: allow workdir= to be a WithProperties. Closes #43.]
Original author: warner at lothar.com
Date: 2007-07-04 05:08:34+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.865
retrieving revision 1.866
diff -u -d -r1.865 -r1.866
--- ChangeLog 3 Jul 2007 19:21:29 -0000 1.865
+++ ChangeLog 4 Jul 2007 05:09:41 -0000 1.866
@@ -1,5 +1,11 @@
2007-07-03 Brian Warner
+ * buildbot/steps/shell.py (ShellCommand): allow workdir= to be a
+ WithProperties instance. Thanks to Axel Hecht for the patch.
+ Closes #43.
+ * buildbot/test/test_properties.py (Interpolate.testWorkdir): test it
+ * docs/buildbot.texinfo (Build Properties): document it
+
* docs/buildbot.texinfo (Scheduler Types): correct the description
of Dependent schedulers. Thanks to Greg Ward for the patch.
From warner at users.sourceforge.net Wed Jul 4 05:09:44 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:09:44 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_properties.py, 1.7,
1.8
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv31546/buildbot/test
Modified Files:
test_properties.py
Log Message:
[project @ ShellCommand: allow workdir= to be a WithProperties. Closes #43.]
Original author: warner at lothar.com
Date: 2007-07-04 05:08:34+00:00
Index: test_properties.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_properties.py,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -d -r1.7 -r1.8
--- test_properties.py 18 Jun 2007 02:52:49 -0000 1.7
+++ test_properties.py 4 Jul 2007 05:09:41 -0000 1.8
@@ -60,6 +60,15 @@
self.failUnlessEqual(cmd,
["tar", "czf", "build-47.tar.gz", "source"])
+ def testWorkdir(self):
+ self.build.setProperty("revision", 47)
+ self.failUnlessEqual(self.build_status.getProperty("revision"), 47)
+ c = ShellCommand(command=["tar", "czf", "foo.tar.gz", "source"])
+ c.setBuild(self.build)
+ workdir = WithProperties("workdir-%d", "revision")
+ workdir = c._interpolateWorkdir(workdir)
+ self.failUnlessEqual(workdir, "workdir-47")
+
def testWithPropertiesDict(self):
self.build.setProperty("other", "foo")
self.build.setProperty("missing", None)
From warner at users.sourceforge.net Wed Jul 4 05:09:49 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:09:49 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.81,1.82
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv31577/buildbot/slave
Modified Files:
commands.py
Log Message:
[project @ ShellCommand: create the workdir if it didn't already exist]
Original author: warner at lothar.com
Date: 2007-07-04 05:09:08+00:00
Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.81
retrieving revision 1.82
diff -u -d -r1.81 -r1.82
--- commands.py 3 Jul 2007 17:59:03 -0000 1.81
+++ commands.py 4 Jul 2007 05:09:47 -0000 1.82
@@ -309,6 +309,9 @@
return self.deferred
def _startCommand(self):
+ # ensure workdir exists
+ if not os.path.isdir(self.workdir):
+ os.makedirs(self.workdir)
log.msg("ShellCommand._startCommand")
if self.notreally:
self.sendStatus({'header': "command '%s' in dir %s" % \
From warner at users.sourceforge.net Wed Jul 4 05:09:49 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:09:49 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.866,1.867
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv31577
Modified Files:
ChangeLog
Log Message:
[project @ ShellCommand: create the workdir if it didn't already exist]
Original author: warner at lothar.com
Date: 2007-07-04 05:09:08+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.866
retrieving revision 1.867
diff -u -d -r1.866 -r1.867
--- ChangeLog 4 Jul 2007 05:09:41 -0000 1.866
+++ ChangeLog 4 Jul 2007 05:09:47 -0000 1.867
@@ -5,6 +5,8 @@
Closes #43.
* buildbot/test/test_properties.py (Interpolate.testWorkdir): test it
* docs/buildbot.texinfo (Build Properties): document it
+ * buildbot/slave/commands.py (ShellCommand._startCommand): create the
+ workdir if it didn't already exist.
* docs/buildbot.texinfo (Scheduler Types): correct the description
of Dependent schedulers. Thanks to Greg Ward for the patch.
From warner at users.sourceforge.net Wed Jul 4 05:52:06 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:52:06 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.867,1.868
Message-ID:
Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15887
Modified Files:
ChangeLog
Log Message:
[project @ ShellCommand: log the environment with one variable per line, instead of a single very long line]
Original author: warner at lothar.com
Date: 2007-07-04 05:43:04+00:00
Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.867
retrieving revision 1.868
diff -u -d -r1.867 -r1.868
--- ChangeLog 4 Jul 2007 05:09:47 -0000 1.867
+++ ChangeLog 4 Jul 2007 05:52:03 -0000 1.868
@@ -1,5 +1,12 @@
2007-07-03 Brian Warner
+ * buildbot/slave/commands.py (ShellCommand._startCommand): when
+ logging the environment variables, put one variable on each line,
+ rather than having one really long line with all of them. This
+ should make them a bit more readable and not trigger the
+ horizontal scrollbar (when viewing it in a web browser) quite so
+ much. Thanks to Albert Hofkamp for the patch.
+
* buildbot/steps/shell.py (ShellCommand): allow workdir= to be a
WithProperties instance. Thanks to Axel Hecht for the patch.
Closes #43.
From warner at users.sourceforge.net Wed Jul 4 05:52:06 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 04 Jul 2007 05:52:06 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.82,1.83
Message-ID:
Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15887/buildbot/slave
Modified Files:
commands.py
Log Message:
[project @ ShellCommand: log the environment with one variable per line, instead of a single very long line]
Original author: warner at lothar.com
Date: 2007-07-04 05:43:04+00:00
Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.82
retrieving revision 1.83
diff -u -d -r1.82 -r1.83
--- commands.py 4 Jul 2007 05:09:47 -0000 1.82
+++ commands.py 4 Jul 2007 05:52:04 -0000 1.83
@@ -362,9 +362,13 @@
self.sendStatus({'header': msg+"\n"})
# then the environment, since it sometimes causes problems
- msg = " environment: %s" % (self.environ,)
- log.msg(" " + msg)
- self.sendStatus({'header': msg+"\n"})
+ msg = " environment:\n"
+ env_names = self.environ.keys()
+ env_names.sort()
+ for name in env_names:
+ msg += " %s=%s\n" % (name, self.environ[name])
+ log.msg(" environment: %s" % (self.environ,))
+ self.sendStatus({'header': msg})
# this will be buffered until connectionMade is called
if self.initialStdin:
From warner at users.sourceforge.net Fri Jul 13 07:21:12 2007
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 13 Jul 2007 07:21:12 +0000
Subject: [Buildbot-commits] site index.html,1.72,1.73
Message-ID:
Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15612
Modified Files:
index.html
Log Message:
direct folks from buildbot.sf.net to visit buildbot.net instead
Index: index.html
===================================================================
RCS file: /cvsroot/buildbot/site/index.html,v
retrieving revision 1.72
retrieving revision 1.73
diff -u -d -r1.72 -r1.73
--- index.html 7 Aug 2006 00:25:31 -0000 1.72
+++ index.html 13 Jul 2007 07:21:08 -0000 1.73
@@ -3,607 +3,17 @@
The BuildBot
-
The BuildBot
-
-
-The BuildBot is a system to automate the compile/test cycle required by
-most software projects to validate code changes. By automatically rebuilding
-and testing the tree each time something has changed, build problems are
-pinpointed quickly, before other developers are inconvenienced by the
-failure. The guilty developer can be identified and harassed without human
-intervention. By running the builds on a variety of platforms, developers who
-do not have the facilities to test their changes everywhere before checkin
-will at least know shortly afterwards whether they have broken the build or
-not. Warning counts, lint checks, image size, compile time, and other build
-parameters can be tracked over time, are more visible, and are therefore
-easier to improve.
-
-The overall goal is to reduce tree breakage and provide a platform to run
-tests or code-quality checks that are too annoying or pedantic for any human
-to waste their time with. Developers get immediate (and potentially public)
-feedback about their changes, encouraging them to be more careful about
-testing before checkin.
-
-Current contents:
-
-
- - The current release is buildbot-0.7.3 . You can download the source
- from the sf.net download page here.
- The release is signed with my GPG public key, available here.
-
- - The latest code is available from CVS for browsing
- or read-only
- checkout. There are also Arch and Darcs repositories which track the
- main CVS tree and provide lower-latency access than anonymous CVS, details
- are here. There is also a copy of the CVS
- user's manual available, which may give you
- an idea of what sorts of features are coming up in the next release.
-
- - The README file contains an overview, while the User's Manual contains complete documentation.
- The API Reference may be useful when
- writing the configuration file: the various Steps and Schedulers are
- documented there.
-
- - Recent changes are summarized in the NEWS file,
- while the complete details are in the ChangeLog.
-
- - The paper delivered at
- PyCon-2003, along with the slides.
-
- - The SourceForge
- Project Page
-
- - Please join the buildbot-devel
- mailing
- list to be notified about new releases, discuss how to configure and
- use the buildbot, and to hear about new development. This is both a user
- and a developer mailing list.
-
-
- - You can also join the buildbot-commits
- mailing
- list to be notified about changes in CVS.
-
-
- - Older versions (0.6.6 and earlier) require some pre-packaged VC
- repositories to run certain unit tests, available in this tarball: buildbot-test-vc-1.tar.gz
-
-
-Getting Buildbot
-
- - From Source: download the latest version from the SourceForge
- Project Page,
- run setup.py install as per the README,
- and read the User's Manual about configuring
- your new buildbot.
-
- - Ubuntu Users: install the 'buildbot' package from
- the Dapper 'universe' distribution. (thanks to Matthias Klose for making
- this happen!)
-
- - Darwin Users: run port install buildbot (thanks to
- Olivier Bonnet for making this
- happen!)
-
-
-
-Success Stories
-
-
-
-
- Twisted |
-
-
- |
-
-
- |
-
- The original publically-visible Buildbot installation runs the
- Twisted unit test suite.
- |
-
-
-
- Id Software |
-
- |
-
- |
-
- TTimo (at Id software) is using a Buildbot for testing Wolfenstein:
- Enemy Territory Linux.
- |
-
-
-
- Nunatek |
-
- |
-
- |
-
- Christopher Armstrong (radix) has set up a Buildbot to do web testing
- (using MaxQ) for one of his clients, as well as internally at Nunatak
- to run python unit tests.
- |
-
-
-
- ? |
-
- |
-
- |
-
- 'nbm' reports that the first-or-second-largest media company
- in South Africa uses buildbot , testing PHP code.
- |
-
-
-
- GStreamer |
-
-
- |
-
-
- |
-
- Thomas Vander Stichele at Fluendo
- is using a Buildbot to run multi-distribution tests for their
- GStreamer-based streaming multimedia framework.
- |
-
-
-
- ILM |
-
- |
-
- |
- Dave Peticolas at Industrial Light & Magic is using a Buildbot
- internally to test lots of python code
- |
-
-
-
- Free Geek |
-
-
- |
-
- |
-
- The Free Geek project has a buildbot for the diagnostic tools they use
- to test donated hardware.
- |
-
-
-
- PBP |
-
- |
-
-
- |
-
- Cory Dodt has created a BuildStep which allows Buildbots to drive
- tests. PBP is a web testing tool that (as I understand it) behaves as
- an HTTP client and makes sure a web application responds to queries
- correctly. With this BuildStep, you can incorporate PBP tests into your
- compile/build/test cycle.
- |
-
-
-
- Mono |
-
-
- |
-
-
- |
-
- According to Fawad Halim, the Mono project has a buildbot running to
- test the HEAD and release branches of the main project on several
- architectures.
- |
-
-
-
- SpamAssassin |
-
-
- |
-
-
- |
-
- Justin Mason reports that the SpamAssassin project is running a
- buildbot too.
- |
-
-
-
- Boost |
-
-
- |
-
-
- |
-
- Rene Rivera says that the well-known Boost C++ project is moving all
- their testing to run under a buildbot, but it does not currently (aug
- 2005) appear to be online yet.
- |
-
-
-
- Monotone |
-
-
- |
-
-
- |
-
- The Monotone version control system is using a buildbot to run unit
- tests. Their buildbot has been extended to handle Monotone's
- somewhat-unique VC scheme by Nathaniel Smith, and hopefully the patches
- will be merged into the main release soon.
- |
-
-
-
- ETLab |
-
- |
-
-
- |
-
- Mark Dillavou reports that he's installed a Buildbot in the Enabling
- Technology Laboratory at the University of Alabama, Birmingham, to
- maintain their visualization and virtual-environment projects.
- |
-
-
-
- Netbox Blue |
-
- |
-
- |
-
- Stephen Thorne says that his company, Netbox Blue, uses a buildbot to
- build and test their network security appliance.
- |
-
-
-
- Ethereal |
-
-
- |
-
-
- |
-
- Ulf Lamping reports that the Ethereal project, a well-known packet
- sniffer, uses a buildbot for their cross-platform compatibility
- testing. Joerg Mayer reports that they've changed their name to Wireshark and their new buildbot
- is here.
- |
-
-
-
- VideoLAN |
-
-
- |
-
-
- |
-
- Cl?ment Stenac writes in to say that the VideoLAN Project has a
- Buildbot to do their continuous-integration and nightly builds.
- |
-
-
-
- Nuxeo |
-
-
- |
-
-
- |
-
- Stefane Fermigier reports that Nuxeo SAS is running this buildbot to
- test their ECM project named CPS.
- |
-
-
-
- Zope |
-
-
- |
-
-
- |
-
- The Zope project (a large and well-known application server for
- building web Content Management Systems and the like) is using a
- buildbot to track the status of Zope3.
- |
-
-
-
- OpenID |
-
- |
-
-
- |
-
- Kevin Turner reports that JanRain, Inc. is using a buildbot for their
- multi-language OpenID libraries.
- |
-
-
-
- Python |
-
-
- |
-
-
- |
-
- Python, the language that the Buildbot is written in, has recently set
- up its own Buildbot. Guido van Rossum mentioned it in his blog,
- and again in his keynote at PyCon2006.
- |
-
-
-
- KDE |
-
-
- |
-
-
- |
-
- The KDE Project has a buildbot, working on their unstable code branch.
- |
-
-
-
- WebKit |
-
-
- |
-
-
- |
-
- The OpenDarwin WebKit component (which includes the HTML renderer used
- by the Safari browser in OS-X) has a buildbot too.
- |
-
-
-
- ASCEND |
-
-
- |
-
-
- |
-
- John Pye writes to say that the ASCEND
- project is using a buildbot.
- |
-
-
-
- Logicalware |
-
-
- |
-
-
- |
-
- Kevin Campbell reports that his company is using a buildbot to help
- them provide commercial support of a product over a large range of
- architectures.
- |
-
-
-
- CodeSourcery |
-
- |
-
-
- |
-
- Stefan Seefeld is using a buildbot for internal builds and tests at his
- company.
- |
-
-
-
- Xenomai |
-
- |
-
-
- |
-
- Niklaus Giger has set up a buildbot for the Xenomai project.
- |
-
-
-
- Aqsis |
-
-
- |
-
-
- |
- Paul Gregory has set up a buildbot for the Aqsis rendering project
- |
-
-
-
- Enfold Systems |
-
-
- |
-
-
- |
- Sidnei da Silva reports that his company has a buildbot to test and
- drive automated builds. One of his buildslaves is installed as a Windows
- Service (using py2exe). |
-
-
-
- Auger Observatory |
-
-
- |
-
-
- |
- Tom Paul reports that a buildbot is in use at the Pierre Auger
- Observatory to test the data processing and analysis software in use
- there. "The observatory is designed to unveil the origins and
- composition of the highest energy cosmic rays, and is operated by a
- collaboration of about 300 physicists (running code on quite a few
- different platforms)". |
-
-
-
- Cheesecake |
-
-
- |
-
-
- |
- Michal Kwiatkowski reports that the pycheesecake project is using
- a buildbot to run all their functional and unit tests. This project
- ranks many python packages according to such quality metrics like
- pylint scores, percentage of documented methods, and
- downloadability. |
-
-
-
- ReactOS |
-
-
- |
-
-
- |
- Aleksey Bragin has set up a buildbot for the ReactOS project, to
- automate the creation of installation CD images. This project is
- dedicated to creating a GPLed windows-compatible operating system
- from the ground up. |
-
-
-
- |
-
- |
-
- |
- install a Buildbot today and get your name added here!
- |
-
-
-
-
-
-
-
-
- |
- link to the buildbot's main status page |
-
-
- |
- link to the project's home page |
-
-
-
-
-
-
-
-
-
-
-
-
-
- Brian Warner
- <warner @ lothar.com>
-
-
-
-
+This page has moved! Please check out our new Trac instance at http://buildbot.net. It has everything you
+want to know about Buildbot configuration and development.
-The pretty home page icon comes from the Tango Project, and is
-used under the Creative Commons
-Attribution Share-Alike license.
+thanks!
+ -Brian
-Last modified: Sun Aug 6 17:24:52 PDT 2006
+Last modified: Fri Jul 13 00:21:01 PDT 2007