From warner at users.sourceforge.net  Fri Jun  2 03:32:25 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 03:32:25 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.86,1.87
Message-ID: <E1Fm0Og-0001MN-8N@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv4681/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ set ShellCommand.flunkOnFailure=True by default]

Original author: warner at lothar.com
Date: 2006-06-02 03:23:23

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.86
retrieving revision 1.87
diff -u -d -r1.86 -r1.87
--- step.py	22 May 2006 00:50:31 -0000	1.86
+++ step.py	2 Jun 2006 03:32:23 -0000	1.87
@@ -879,6 +879,9 @@
     the exit code of that command is non-zero, SUCCESS otherwise. To change
     this behavior, override my .evaluateCommand method.
 
+    By default, a failure of this step will mark the whole build as FAILURE.
+    To override this, give me an argument of flunkOnFailure=False .
+
     I create a single Log named 'log' which contains the output of the
     command. To create additional summary Logs, override my .createSummary
     method.
@@ -900,6 +903,10 @@
     descriptionDone = None # alternate description when the step is complete
     command = None # set this to a command, or set in kwargs
 
+    # override this on a specific ShellCommand if you want to let it fail
+    # without dooming the entire build to a status of FAILURE
+    flunkOnFailure = True
+
     def __init__(self, workdir,
                  description=None, descriptionDone=None,
                  command=None,




From warner at users.sourceforge.net  Fri Jun  2 03:32:25 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 03:32:25 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.639,1.640
Message-ID: <E1Fm0Og-0001MO-AE@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv4681

Modified Files:
	ChangeLog 
Log Message:
[project @ set ShellCommand.flunkOnFailure=True by default]

Original author: warner at lothar.com
Date: 2006-06-02 03:23:23

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.639
retrieving revision 1.640
diff -u -d -r1.639 -r1.640
--- ChangeLog	30 May 2006 07:07:40 -0000	1.639
+++ ChangeLog	2 Jun 2006 03:32:23 -0000	1.640
@@ -1,3 +1,11 @@
+2006-06-01  Brian Warner  <warner at lothar.com>
+
+	* buildbot/process/step.py (ShellCommand): set flunkOnFailure=True
+	by default, so that any ShellCommand which fails marks the overall
+	build as a failure. I should have done this from the beginning.
+	Add flunkOnFailure=False to the arguments if you want to turn off
+	this behavior.
+
 2006-05-30  Brian Warner  <warner at lothar.com>
 
 	* buildbot/clients/gtkPanes.py: add a third row: now it shows




From warner at users.sourceforge.net  Fri Jun  2 03:32:31 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 03:32:31 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.640,1.641
Message-ID: <E1Fm0Om-0001NE-0a@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv4716

Modified Files:
	ChangeLog 
Log Message:
[project @ statusgui improvements]

Original author: warner at lothar.com
Date: 2006-06-02 03:28:54

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.640
retrieving revision 1.641
diff -u -d -r1.640 -r1.641
--- ChangeLog	2 Jun 2006 03:32:23 -0000	1.640
+++ ChangeLog	2 Jun 2006 03:32:29 -0000	1.641
@@ -1,5 +1,12 @@
 2006-06-01  Brian Warner  <warner at lothar.com>
 
+	* buildbot/clients/gtkPanes.py (Box.setColor): ignore color=None
+	(Box.setETA): handle ETA=None (by stopping the timer)
+	(Box.update): make the [soon] text less different than the usual
+	text, so the rest of the text doesn't flop around so much. It
+	would be awfully nice to figure out how to center this stuff.
+	(ThreeRowBuilder.stepETAUpdate): more debugging printouts
+
 	* buildbot/process/step.py (ShellCommand): set flunkOnFailure=True
 	by default, so that any ShellCommand which fails marks the overall
 	build as a failure. I should have done this from the beginning.




From warner at users.sourceforge.net  Fri Jun  2 03:32:31 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 03:32:31 +0000
Subject: [Buildbot-commits] buildbot/buildbot/clients gtkPanes.py, 1.11, 1.12
Message-ID: <E1Fm0Om-0004Zj-AK@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/clients
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv4716/buildbot/clients

Modified Files:
	gtkPanes.py 
Log Message:
[project @ statusgui improvements]

Original author: warner at lothar.com
Date: 2006-06-02 03:28:54

Index: gtkPanes.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/clients/gtkPanes.py,v
retrieving revision 1.11
retrieving revision 1.12
diff -u -d -r1.11 -r1.12
--- gtkPanes.py	30 May 2006 07:07:40 -0000	1.11
+++ gtkPanes.py	2 Jun 2006 03:32:29 -0000	1.12
@@ -275,11 +275,16 @@
         self.label.set_text(text)
 
     def setColor(self, color):
+        if not color:
+            return
         self.box.modify_bg(gtk.STATE_NORMAL, gtk.gdk.color_parse(color))
 
     def setETA(self, eta):
-        self.when = now() + eta
-        self.startTimer()
+        if eta:
+            self.when = now() + eta
+            self.startTimer()
+        else:
+            self.stopTimer()
 
     def startTimer(self):
         self.stopTimer()
@@ -300,7 +305,7 @@
             return True # restart timer
         else:
             # done
-            self.label.set_text("%s\n[soon]" % (self.text,))
+            self.label.set_text("%s\n[soon]\n[overdue]" % (self.text,))
             self.timer = None
             return False
 
@@ -376,6 +381,7 @@
         self.step.setColor("white")
         self.step.stopTimer()
     def stepETAUpdate(self, stepname, eta):
+        print "[%s] stepETAUpdate: %s %s" % (self.name, stepname, eta)
         self.step.setETA(eta)
 
 




From warner at users.sourceforge.net  Fri Jun  2 05:15:48 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 05:15:48 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.641,1.642
Message-ID: <E1Fm20j-0001Rv-5l@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17002

Modified Files:
	ChangeLog 
Log Message:
[project @ add a 'reason' tooltip to the yellow start-of-build box]

Original author: warner at lothar.com
Date: 2006-06-02 05:14:45

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.641
retrieving revision 1.642
diff -u -d -r1.641 -r1.642
--- ChangeLog	2 Jun 2006 03:32:29 -0000	1.641
+++ ChangeLog	2 Jun 2006 05:15:46 -0000	1.642
@@ -1,5 +1,11 @@
 2006-06-01  Brian Warner  <warner at lothar.com>
 
+	* buildbot/status/html.py (BuildBox.getBox): set the 'title='
+	attribute of the "Build #NN" link in the yellow start-the-build
+	box to the build's reason. This means that you get a little
+	tooltip explaining why the build was done when you hover over the
+	yellow box. Thanks to Zandr Milewski for the suggestion.
+
 	* buildbot/clients/gtkPanes.py (Box.setColor): ignore color=None
 	(Box.setETA): handle ETA=None (by stopping the timer)
 	(Box.update): make the [soon] text less different than the usual




From warner at users.sourceforge.net  Fri Jun  2 05:15:48 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 05:15:48 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.84,1.85
Message-ID: <E1Fm20j-00060Z-73@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17002/buildbot/status

Modified Files:
	html.py 
Log Message:
[project @ add a 'reason' tooltip to the yellow start-of-build box]

Original author: warner at lothar.com
Date: 2006-06-02 05:14:45

Index: html.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v
retrieving revision 1.84
retrieving revision 1.85
diff -u -d -r1.84 -r1.85
--- html.py	21 May 2006 21:28:26 -0000	1.84
+++ html.py	2 Jun 2006 05:15:46 -0000	1.85
@@ -919,7 +919,9 @@
         name = b.getBuilder().getName()
         number = b.getNumber()
         url = "%s/builds/%d" % (urllib.quote(name, safe=''), number)
-        text = '<a href="%s">Build %d</a>' % (url, number)
+        reason = b.getReason()
+        text = ('<a title="Reason: %s" href="%s">Build %d</a>'
+                % (html.escape(reason), url, number))
         color = "yellow"
         class_ = "start"
         if b.isFinished() and not b.getSteps():




From warner at users.sourceforge.net  Fri Jun  2 06:26:30 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 06:26:30 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_web.py,1.29,1.30
Message-ID: <E1Fm378-0003WJ-Tw@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17222/buildbot/test

Modified Files:
	test_web.py 
Log Message:
[project @ fix test_web.py to match the title= change]

Original author: warner at lothar.com
Date: 2006-06-02 06:25:38

Index: test_web.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_web.py,v
retrieving revision 1.29
retrieving revision 1.30
diff -u -d -r1.29 -r1.30
--- test_web.py	21 May 2006 22:38:19 -0000	1.29
+++ test_web.py	2 Jun 2006 06:26:28 -0000	1.30
@@ -398,6 +398,7 @@
         bs.addStep(step1)
         bs.buildStarted(build1)
         step1.step_status.stepStarted()
+        bs.setReason("reason")
 
         log1 = step1.addLog("output")
         log1.addStdout("some stdout\n")




From warner at users.sourceforge.net  Fri Jun  2 06:26:30 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 06:26:30 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.642,1.643
Message-ID: <E1Fm378-00007Y-U3@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17222

Modified Files:
	ChangeLog 
Log Message:
[project @ fix test_web.py to match the title= change]

Original author: warner at lothar.com
Date: 2006-06-02 06:25:38

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.642
retrieving revision 1.643
diff -u -d -r1.642 -r1.643
--- ChangeLog	2 Jun 2006 05:15:46 -0000	1.642
+++ ChangeLog	2 Jun 2006 06:26:28 -0000	1.643
@@ -1,5 +1,8 @@
 2006-06-01  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/test_web.py (Logfile.setUp): set the .reason on
+	the fake build, so that title= has something to be set to
+
 	* buildbot/status/html.py (BuildBox.getBox): set the 'title='
 	attribute of the "Build #NN" link in the yellow start-the-build
 	box to the build's reason. This means that you get a little




From warner at users.sourceforge.net  Fri Jun  2 15:54:58 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 15:54:58 +0000
Subject: [Buildbot-commits] buildbot/contrib svn_buildbot.py,1.11,1.12
Message-ID: <E1FmBzG-0004aU-TB@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/contrib
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv23397/contrib

Modified Files:
	svn_buildbot.py 
Log Message:
[project @ contrib/svn_buildbot.py: handle property changes correctly]

Original author: warner at lothar.com
Date: 2006-06-02 15:53:04

Index: svn_buildbot.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/contrib/svn_buildbot.py,v
retrieving revision 1.11
retrieving revision 1.12
diff -u -d -r1.11 -r1.12
--- svn_buildbot.py	22 Mar 2006 20:53:30 -0000	1.11
+++ svn_buildbot.py	2 Jun 2006 15:54:56 -0000	1.12
@@ -145,7 +145,8 @@
         changed = commands.getoutput('svnlook changed %s "%s"' % (rev_arg,
                                                                   repo)
                                      ).split('\n')
-        changed = [x[1:].strip() for x in changed]
+        # the first 6 columns can contain status information
+        changed = [x[6:].strip() for x in changed]
 
         message = commands.getoutput('svnlook log %s "%s"' % (rev_arg, repo))
         who = commands.getoutput('svnlook author %s "%s"' % (rev_arg, repo))




From warner at users.sourceforge.net  Fri Jun  2 15:54:58 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 02 Jun 2006 15:54:58 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.643,1.644
Message-ID: <E1FmBzH-00057O-4n@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv23397

Modified Files:
	ChangeLog 
Log Message:
[project @ contrib/svn_buildbot.py: handle property changes correctly]

Original author: warner at lothar.com
Date: 2006-06-02 15:53:04

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.643
retrieving revision 1.644
diff -u -d -r1.643 -r1.644
--- ChangeLog	2 Jun 2006 06:26:28 -0000	1.643
+++ ChangeLog	2 Jun 2006 15:54:56 -0000	1.644
@@ -1,3 +1,10 @@
+2006-06-02  Brian Warner  <warner at lothar.com>
+
+	* contrib/svn_buildbot.py (ChangeSender.getChanges): ignore the
+	first six columns of 'svnlook' output, not just the first column,
+	since property changes appear in the other five. Thanks to Olivier
+	Bonnet for the patch.
+
 2006-06-01  Brian Warner  <warner at lothar.com>
 
 	* buildbot/test/test_web.py (Logfile.setUp): set the .reason on




From warner at users.sourceforge.net  Sat Jun  3 17:29:15 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 17:29:15 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.644,1.645
Message-ID: <E1FmZw4-0002qL-MB@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv9215

Modified Files:
	ChangeLog 
Log Message:
[project @ add bug number to the svn-property-change fix]

Original author: warner at lothar.com
Date: 2006-06-03 17:28:39

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.644
retrieving revision 1.645
diff -u -d -r1.644 -r1.645
--- ChangeLog	2 Jun 2006 15:54:56 -0000	1.644
+++ ChangeLog	3 Jun 2006 17:29:13 -0000	1.645
@@ -3,7 +3,7 @@
 	* contrib/svn_buildbot.py (ChangeSender.getChanges): ignore the
 	first six columns of 'svnlook' output, not just the first column,
 	since property changes appear in the other five. Thanks to Olivier
-	Bonnet for the patch.
+	Bonnet for the patch. Fixes SF#1398174.
 
 2006-06-01  Brian Warner  <warner at lothar.com>
 




From warner at users.sourceforge.net  Sat Jun  3 18:23:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 18:23:06 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.645,1.646
Message-ID: <E1FmamC-0001Pp-TL@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv32174

Modified Files:
	ChangeLog 
Log Message:
[project @ MailNotifier: don't double-escape the build URL, fixes SF#1452801]

Original author: warner at lothar.com
Date: 2006-06-03 18:20:34

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.645
retrieving revision 1.646
diff -u -d -r1.645 -r1.646
--- ChangeLog	3 Jun 2006 17:29:13 -0000	1.645
+++ ChangeLog	3 Jun 2006 18:23:04 -0000	1.646
@@ -1,3 +1,9 @@
+2006-06-03  Brian Warner  <warner at lothar.com>
+
+	* buildbot/status/mail.py (MailNotifier.buildMessage): don't
+	double-escape the build URL. Thanks to Olivier Bonnet for the
+	patch. Fixes SF#1452801.
+
 2006-06-02  Brian Warner  <warner at lothar.com>
 
 	* contrib/svn_buildbot.py (ChangeSender.getChanges): ignore the




From warner at users.sourceforge.net  Sat Jun  3 18:23:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 18:23:06 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status mail.py,1.24,1.25
Message-ID: <E1FmamE-00025h-9i@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv32174/buildbot/status

Modified Files:
	mail.py 
Log Message:
[project @ MailNotifier: don't double-escape the build URL, fixes SF#1452801]

Original author: warner at lothar.com
Date: 2006-06-03 18:20:34

Index: mail.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/mail.py,v
retrieving revision 1.24
retrieving revision 1.25
diff -u -d -r1.24 -r1.25
--- mail.py	17 Apr 2006 19:22:34 -0000	1.24
+++ mail.py	3 Jun 2006 18:23:04 -0000	1.25
@@ -233,8 +233,7 @@
             text += "The Buildbot has detected a new failure of %s.\n" % name
         buildurl = self.status.getURLForThing(build)
         if buildurl:
-            text += ("Full details are available at:\n %s\n" %
-                     urllib.quote(buildurl, '/:'))
+            text += "Full details are available at:\n %s\n" % buildurl
         text += "\n"
 
         url = self.status.getBuildbotURL()




From warner at users.sourceforge.net  Sat Jun  3 19:40:39 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 19:40:39 +0000
Subject: [Buildbot-commits] site ChangeLog, 1.35, 1.36 source-Arch.html, 1.8,
	1.9
Message-ID: <E1Fmbzf-00068b-G1@mail.sourceforge.net>

Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv32052

Modified Files:
	ChangeLog source-Arch.html 
Log Message:
update source-Arch.html

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/site/ChangeLog,v
retrieving revision 1.35
retrieving revision 1.36
diff -u -d -r1.35 -r1.36
--- ChangeLog	24 May 2006 18:20:33 -0000	1.35
+++ ChangeLog	3 Jun 2006 19:40:37 -0000	1.36
@@ -1,3 +1,7 @@
+2006-06-03  Brian Warner  <warner at lothar.com>
+
+	* source-Arch.html: update cvs-vs-arch-vs-darcs stuff
+
 2006-05-24  Brian Warner  <warner at lothar.com>
 
 	* index.html: add "Getting Buildbot" section to point at

Index: source-Arch.html
===================================================================
RCS file: /cvsroot/buildbot/site/source-Arch.html,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -d -r1.8 -r1.9
--- source-Arch.html	12 Mar 2006 08:21:01 -0000	1.8
+++ source-Arch.html	3 Jun 2006 19:40:37 -0000	1.9
@@ -11,16 +11,11 @@
 <p>In addition to CVS, the Buildbot source code is available via <a
 href="http://www.gnu.org/software/gnu-arch/">Arch</a>. The archive is
 experimental, but I figure this may make it slightly easier for contributors
-to track the upstream sources. More significantly, the Arch archive is
-synchronized from the CVS repository within a few minutes after each commit,
-as opposed to the 5-plus-hour delay between a commit and the time the sf.net
-anonymous CVS repository gets updated.</p>
-
-<p>The result is that if you wanted to (say) set up a metabuildbot (a
-Buildbot which runs the Buildbot unit test suite), and you wanted it to be
-able to test code that was a few minutes old instead of several hours old,
-you'd want to configure it to get its source code from the Arch archive
-instead of from anoncvs.</p>
+to track the upstream sources. At the moment (03-Jun-2006), this branch is
+manually synchronized from the CVS repository, so it may be a few hours to a
+few days behind, depending upon how quickly I get back to my desk after doing
+a commit. Check the ChangeLogs in both trees to make sure they're
+up-to-date.</p>
 
 <p>The Arch coordinates are as follows:</p>
 
@@ -120,21 +115,20 @@
 -----END PGP PUBLIC KEY BLOCK-----
 </pre>
 
-<p>Also note that this is a two-way sync, and that when I sometimes do
-Buildbot work from a local Arch repository, those changes can get pushed into
-CVS via the same gateway. Still experimental, but it means that if you use
-Arch and want to contribute, I'll be able to pull from your archive instead
-of having you mail me patches all the time.</p>
-
 <h1>Buildbot source code via Darcs</h1>
 
-<p>I also have a one-way Darcs mirror which tracks the CVS repository. To
+<p>The Buildbot source code is also available from a Darcs repository. To
 pull a tree this way, use the following command:</p>
 
 <pre>
-darcs get http://buildbot.sf.net/darcs-repos/trunk
+darcs get --partial http://buildbot.sf.net/darcs-repos/trunk
 </pre>
 
+<p>The Darcs tree is the most up-to-date available. The CVS repository on
+sf.net is actually slaved to the Darcs tree (and updated automatically upon
+each Darcs checkin). The Arch repository is also slaved to the Darcs tree
+(but is updated by hand).</p>
+
 <hr />
 
 
@@ -146,5 +140,5 @@
  <a href="http://www.lothar.com/warner.html">Brian Warner</a>
  &lt;warner&nbsp;&#64;&nbsp;lothar.com&gt;
 </address>
-<!-- hhmts start -->Last modified: Sat Mar 11 22:42:03 PST 2006 <!-- hhmts end -->
+<!-- hhmts start -->Last modified: Sat Jun  3 12:38:44 PDT 2006 <!-- hhmts end -->
 </body> </html>




From warner at users.sourceforge.net  Sat Jun  3 19:46:40 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 19:46:40 +0000
Subject: [Buildbot-commits] site index.html,1.64,1.65 ChangeLog,1.36,1.37
Message-ID: <E1Fmc58-0006xa-SZ@mail.sourceforge.net>

Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv1829

Modified Files:
	index.html ChangeLog 
Log Message:
add the Aqsis buildbot

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/site/ChangeLog,v
retrieving revision 1.36
retrieving revision 1.37
diff -u -d -r1.36 -r1.37
--- ChangeLog	3 Jun 2006 19:40:37 -0000	1.36
+++ ChangeLog	3 Jun 2006 19:46:38 -0000	1.37
@@ -1,5 +1,7 @@
 2006-06-03  Brian Warner  <warner at lothar.com>
 
+	* index.html: add the Aqsis buildbot
+
 	* source-Arch.html: update cvs-vs-arch-vs-darcs stuff
 
 2006-05-24  Brian Warner  <warner at lothar.com>

Index: index.html
===================================================================
RCS file: /cvsroot/buildbot/site/index.html,v
retrieving revision 1.64
retrieving revision 1.65
diff -u -d -r1.64 -r1.65
--- index.html	24 May 2006 18:20:33 -0000	1.64
+++ index.html	3 Jun 2006 19:46:38 -0000	1.65
@@ -480,6 +480,18 @@
   </tr>
 
   <tr>
+    <td>Aqsis</td>
+    <td><a href="http://build.aqsis.org:8010/">
+      <img src="buildbot.png" alt="Buildbot" height="16" width="16"></a>
+    </td>
+    <td><a href="http://www.aqsis.org/">
+      <img src="home.png" alt="home page" height="16" width="16"></a>
+    </td>
+    <td>Paul Gregory has set up a buildbot for the Aqsis rendering project
+    </td>
+  </tr>
+
+  <tr>
     <td></td>
     <td>
     </td>
@@ -531,5 +543,5 @@
 href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons
 Attribution Share-Alike license</a>.</font></p>
 
-<!-- hhmts start -->Last modified: Wed May 24 09:52:39 PDT 2006 <!-- hhmts end -->
+<!-- hhmts start -->Last modified: Sat Jun  3 12:44:18 PDT 2006 <!-- hhmts end -->
 </body> </html>




From warner at users.sourceforge.net  Sat Jun  3 20:00:11 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 20:00:11 +0000
Subject: [Buildbot-commits] site index.html,1.65,1.66 ChangeLog,1.37,1.38
Message-ID: <E1FmcID-0007hn-Mm@mail.sourceforge.net>

Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv6815

Modified Files:
	index.html ChangeLog 
Log Message:
add the Enfold Systems buildbot

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/site/ChangeLog,v
retrieving revision 1.37
retrieving revision 1.38
diff -u -d -r1.37 -r1.38
--- ChangeLog	3 Jun 2006 19:46:38 -0000	1.37
+++ ChangeLog	3 Jun 2006 20:00:08 -0000	1.38
@@ -1,6 +1,7 @@
 2006-06-03  Brian Warner  <warner at lothar.com>
 
-	* index.html: add the Aqsis buildbot
+	* index.html: add the Aqsis buildbot, and the Enfold Systems
+	buildbot
 
 	* source-Arch.html: update cvs-vs-arch-vs-darcs stuff
 

Index: index.html
===================================================================
RCS file: /cvsroot/buildbot/site/index.html,v
retrieving revision 1.65
retrieving revision 1.66
diff -u -d -r1.65 -r1.66
--- index.html	3 Jun 2006 19:46:38 -0000	1.65
+++ index.html	3 Jun 2006 20:00:08 -0000	1.66
@@ -492,6 +492,19 @@
   </tr>
 
   <tr>
+    <td>Enfold Systems</td>
+    <td><a href="http://buildbot.enfoldsystems.com/">
+      <img src="buildbot.png" alt="Buildbot" height="16" width="16"></a>
+    </td>
+    <td><a href="http://www.enfoldsystems.com/">
+      <img src="home.png" alt="home page" height="16" width="16"></a>
+    </td>
+    <td>Sidnei da Silva reports that his company has a buildbot to test and
+    drive automated builds. One of his buildslaves is installed as a Windows
+    Service (using py2exe). </td>
+  </tr>
+
+  <tr>
     <td></td>
     <td>
     </td>
@@ -543,5 +556,5 @@
 href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons
 Attribution Share-Alike license</a>.</font></p>
 
-<!-- hhmts start -->Last modified: Sat Jun  3 12:44:18 PDT 2006 <!-- hhmts end -->
+<!-- hhmts start -->Last modified: Sat Jun  3 12:57:59 PDT 2006 <!-- hhmts end -->
 </body> </html>




From warner at users.sourceforge.net  Sat Jun  3 20:21:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 20:21:06 +0000
Subject: [Buildbot-commits] buildbot/contrib/windows buildbot_service.py,
	NONE, 1.1 setup.py, NONE, 1.1
Message-ID: <E1FmccP-0002qX-IJ@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/contrib/windows
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15165/contrib/windows

Added Files:
	buildbot_service.py setup.py 
Log Message:
[project @ add py2exe support for windows, SF#1401121]

Original author: warner at lothar.com
Date: 2006-06-03 20:18:42

--- NEW FILE: buildbot_service.py ---
# Runs the build-bot as a Windows service.
# To use:
# * Install and configure buildbot as per normal (ie, running
#  'setup.py install' from the source directory).
#
# * Configure any number of build-bot directories (slaves or masters), as
#   per the buildbot instructions.  Test these directories normally by
#   using the (possibly modified) "buildbot.bat" file and ensure everything
#   is working as expected.
#
# * Install the buildbot service.  Execute the command:
#   % python buildbot_service.py
#   To see installation options.  You probably want to specify:
#   + --username and --password options to specify the user to run the
#   + --startup auto to have the service start at boot time.
#
#   For example:
#   % python buildbot_service.py --user mark --password secret \
#     --startup auto install
#   Alternatively, you could execute:
#   % python buildbot_service.py install
#   to install the service with default options, then use Control Panel
#   to configure it.
#
# * Start the service specifying the name of all buildbot directories as
#   service args.  This can be done one of 2 ways:
#   - Execute the command:
#     % python buildbot_service.py start "dir_name1" "dir_name2"
#   or:
#   - Start Control Panel->Administrative Tools->Services
#   - Locate the previously installed buildbot service.
#   - Open the "properties" for the service.
#   - Enter the directory names into the "Start Parameters" textbox.  The
#     directory names must be fully qualified, and surrounded in quotes if
#    they include spaces.
#   - Press the "Start"button.
#   Note that the service will automatically use the previously specified
#   directories if no arguments are specified. This means the directories
#   need only be specified when the directories to use have changed (and
#   therefore also the first time buildbot is configured)
#
# * The service should now be running.  You should check the Windows
#   event log.  If all goes well, you should see some information messages
#   telling you the buildbot has successfully started.
#
# * If you change the buildbot configuration, you must restart the service.
#   There is currently no way to ask a running buildbot to reload the
#   config.  You can restart by executing:
#   % python buildbot_service.py restart
#
# Troubleshooting:
# * Check the Windows event log for any errors.
# * Check the "twistd.log" file in your buildbot directories - once each
#   bot has been started it just writes to this log as normal.
# * Try executing:
#   % python buildbot_service.py debug
#   This will execute the buildbot service in "debug" mode, and allow you to
#   see all messages etc generated. If the service works in debug mode but
#   not as a real service, the error probably relates to the environment or
#   permissions of the user configured to run the service (debug mode runs as
#   the currently logged in user, not the service user)
# * Ensure you have the latest pywin32 build available, at least version 206.

# Written by Mark Hammond, 2006.

import sys, os, threading

import pywintypes
import winerror, win32con
import win32api, win32event, win32file, win32pipe, win32process, win32security
import win32service, win32serviceutil, servicemanager

# Are we running in a py2exe environment?
is_frozen = hasattr(sys, "frozen")

# Taken from the Zope service support - each "child" is run as a sub-process
# (trying to run multiple twisted apps in the same process is likely to screw
# stdout redirection etc).
# Note that unlike the Zope service, we do *not* attempt to detect a failed
# client and perform restarts - buildbot itself does a good job
# at reconnecting, and Windows itself provides restart semantics should
# everything go pear-shaped.

# We execute a new thread that captures the tail of the output from our child
# process. If the child fails, it is written to the event log.
# This process is unconditional, and the output is never written to disk
# (except obviously via the event log entry)
# Size of the blocks we read from the child process's output.
CHILDCAPTURE_BLOCK_SIZE = 80
# The number of BLOCKSIZE blocks we keep as process output.
CHILDCAPTURE_MAX_BLOCKS = 200

class BBService(win32serviceutil.ServiceFramework):    
    _svc_name_ = 'BuildBot'
    _svc_display_name_ = _svc_name_
    _svc_description_ = 'Manages local buildbot slaves and masters - ' \
                        'see http://buildbot.sourceforge.net'

    def __init__(self, args):
        win32serviceutil.ServiceFramework.__init__(self, args)

        # Create an event which we will use to wait on. The "service stop" 
        # request will set this event.
        # * We must make it inheritable so we can pass it to the child 
        #   process via the cmd-line
        # * Must be manual reset so each child process and our service
        #   all get woken from a single set of the event.
        sa = win32security.SECURITY_ATTRIBUTES()
        sa.bInheritHandle = True
        self.hWaitStop = win32event.CreateEvent(sa, True, False, None)

        self.args = args
        self.dirs = None
        self.runner_prefix = None

        # Patch up the service messages file in a frozen exe.
        # (We use the py2exe option that magically bundles the .pyd files
        # into the .zip file - so servicemanager.pyd doesn't exist.)
        if is_frozen and servicemanager.RunningAsService():
            msg_file = os.path.join(os.path.dirname(sys.executable),
                                    "buildbot.msg")
            if os.path.isfile(msg_file):
                servicemanager.Initialize("BuildBot", msg_file)
            else:
                self.warning("Strange - '%s' does not exist" % (msg_file,))

    def _checkConfig(self):
        # Locate our child process runner (but only when run from source)
        if not is_frozen:
            # Running from source
            python_exe = os.path.join(sys.prefix, "python.exe")
            if not os.path.isfile(python_exe):
                # for ppl who build Python itself from source.
                python_exe = os.path.join(sys.prefix, "PCBuild", "python.exe")
            if not os.path.isfile(python_exe):
                self.error("Can not find python.exe to spawn subprocess")
                return False

            me = __file__
            if me.endswith(".pyc") or me.endswith(".pyo"):
                me = me[:-1]

            self.runner_prefix = '"%s" "%s"' % (python_exe, me)
        else:
            # Running from a py2exe built executable - our child process is
            # us (but with the funky cmdline args!)
            self.runner_prefix = '"' + sys.executable + '"'

        # Now our arg processing - this may be better handled by a
        # twisted/buildbot style config file - but as of time of writing,
        # MarkH is clueless about such things!

        # Note that the "arguments" you type into Control Panel for the
        # service do *not* persist - they apply only when you click "start"
        # on the service. When started by Windows, args are never presented.
        # Thus, it is the responsibility of the service to persist any args.
        
        # so, when args are presented, we save them as a "custom option". If
        # they are not presented, we load them from the option.
        self.dirs = []
        if len(self.args) > 1:
            dir_string = os.pathsep.join(self.args[1:])
            save_dirs = True
        else:
            dir_string = win32serviceutil.GetServiceCustomOption(self,
                                                            "directories")
            save_dirs = False

        if not dir_string:
            self.error("You must specify the buildbot directories as "
                       "parameters to the service.\nStopping the service.")
            return False

        dirs = dir_string.split(os.pathsep)
        for d in dirs:
            d = os.path.abspath(d)
            sentinal = os.path.join(d, "buildbot.tac")
            if os.path.isfile(sentinal):
                self.dirs.append(d)
            else:
                msg = "Directory '%s' is not a buildbot dir - ignoring" \
                      % (d,)
                self.warning(msg)
        if not self.dirs:
            self.error("No valid buildbot directories were specified.\n"
                       "Stopping the service.")
            return False
        if save_dirs:
            dir_string = os.pathsep.join(self.dirs).encode("mbcs")
            win32serviceutil.SetServiceCustomOption(self, "directories",
                                                    dir_string)
        return True

    def SvcStop(self):
        # Tell the SCM we are starting the stop process.
        self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
        # Set the stop event - the main loop takes care of termination.
        win32event.SetEvent(self.hWaitStop)

    # SvcStop only gets triggered when the user explictly stops (or restarts)
    # the service.  To shut the service down cleanly when Windows is shutting
    # down, we also need to hook SvcShutdown.
    SvcShutdown = SvcStop

    def SvcDoRun(self):
        if not self._checkConfig():
            # stopped status set by caller.
            return

        self.logmsg(servicemanager.PYS_SERVICE_STARTED)

        child_infos = []

        for bbdir in self.dirs:
            self.info("Starting BuildBot in directory '%s'" % (bbdir,))
            hstop = self.hWaitStop

            cmd = '%s --spawn %d start %s' % (self.runner_prefix, hstop, bbdir)
            #print "cmd is", cmd
            h, t, output = self.createProcess(cmd)
            child_infos.append((bbdir, h, t, output))

        while child_infos:
            handles = [self.hWaitStop] + [i[1] for i in child_infos]

            rc = win32event.WaitForMultipleObjects(handles,
                                                   0, # bWaitAll
                                                   win32event.INFINITE)
            if rc == win32event.WAIT_OBJECT_0:
                # user sent a stop service request
                break
            else:
                # A child process died.  For now, just log the output
                # and forget the process.
                index = rc - win32event.WAIT_OBJECT_0 - 1
                bbdir, dead_handle, dead_thread, output_blocks = \
                                                        child_infos[index]
                status = win32process.GetExitCodeProcess(dead_handle)
                output = "".join(output_blocks)
                if not output:
                    output = "The child process generated no output. " \
                             "Please check the twistd.log file in the " \
                             "indicated directory."

                self.warning("BuildBot for directory %r terminated with "
                             "exit code %d.\n%s" % (bbdir, status, output))

                del child_infos[index]

                if not child_infos:
                    self.warning("All BuildBot child processes have "
                                 "terminated.  Service stopping.")

        # Either no child processes left, or stop event set.
        self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)

        # The child processes should have also seen our stop signal
        # so wait for them to terminate.
        for bbdir, h, t, output in child_infos:
            for i in range(10): # 30 seconds to shutdown...
                self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
                rc = win32event.WaitForSingleObject(h, 3000)
                if rc == win32event.WAIT_OBJECT_0:
                    break
            # Process terminated - no need to try harder.
            if rc == win32event.WAIT_OBJECT_0:
                break

            self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
            # If necessary, kill it
            if win32process.GetExitCodeProcess(h)==win32con.STILL_ACTIVE:
                self.warning("BuildBot process at %r failed to terminate - "
                             "killing it" % (bbdir,))
                win32api.TerminateProcess(h, 3)
            self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)

            # Wait for the redirect thread - it should have died as the remote
            # process terminated.
            # As we are shutting down, we do the join with a little more care,
            # reporting progress as we wait (even though we never will <wink>)
            for i in range(5):
                t.join(1)
                self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
                if not t.isAlive():
                    break
            else:
                self.warning("Redirect thread did not stop!")

        # All done.
        self.logmsg(servicemanager.PYS_SERVICE_STOPPED)

    #
    # Error reporting/logging functions.
    #
    def logmsg(self, event):
        # log a service event using servicemanager.LogMsg
        try:
            servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
                                  event,
                                  (self._svc_name_,
                                   " (%s)" % self._svc_display_name_))
        except win32api.error, details:
            # Failed to write a log entry - most likely problem is
            # that the event log is full.  We don't want this to kill us
            try:
                print "FAILED to write INFO event", event, ":", details
            except IOError:
                # No valid stdout!  Ignore it.
                pass

    def _dolog(self, func, msg):
        try:
            func(msg)
        except win32api.error, details:
            # Failed to write a log entry - most likely problem is
            # that the event log is full.  We don't want this to kill us
            try:
                print "FAILED to write event log entry:", details
                print msg
            except IOError:
                pass

    def info(self, s):
        self._dolog(servicemanager.LogInfoMsg, s)

    def warning(self, s):
        self._dolog(servicemanager.LogWarningMsg, s)

    def error(self, s):
        self._dolog(servicemanager.LogErrorMsg, s)
    
    # Functions that spawn a child process, redirecting any output.
    # Although builtbot itself does this, it is very handy to debug issues
    # such as ImportErrors that happen before buildbot has redirected.
    def createProcess(self, cmd):
        hInputRead, hInputWriteTemp = self.newPipe()
        hOutReadTemp, hOutWrite = self.newPipe()
        pid = win32api.GetCurrentProcess()
        # This one is duplicated as inheritable.
        hErrWrite = win32api.DuplicateHandle(pid, hOutWrite, pid, 0, 1,
                                       win32con.DUPLICATE_SAME_ACCESS)

        # These are non-inheritable duplicates.
        hOutRead = self.dup(hOutReadTemp)
        hInputWrite = self.dup(hInputWriteTemp)
        # dup() closed hOutReadTemp, hInputWriteTemp

        si = win32process.STARTUPINFO()
        si.hStdInput = hInputRead
        si.hStdOutput = hOutWrite
        si.hStdError = hErrWrite
        si.dwFlags = win32process.STARTF_USESTDHANDLES | \
                     win32process.STARTF_USESHOWWINDOW
        si.wShowWindow = win32con.SW_HIDE

        # pass True to allow handles to be inherited.  Inheritance is
        # problematic in general, but should work in the controlled
        # circumstances of a service process.
        create_flags = win32process.CREATE_NEW_CONSOLE
        # info is (hProcess, hThread, pid, tid)
        info = win32process.CreateProcess(None, cmd, None, None, True,
                                          create_flags, None, None, si)
        # (NOTE: these really aren't necessary for Python - they are closed
        # as soon as they are collected)
        hOutWrite.Close()
        hErrWrite.Close()
        hInputRead.Close()
        # We don't use stdin
        hInputWrite.Close()

        # start a thread collecting output
        blocks = []
        t = threading.Thread(target=self.redirectCaptureThread,
                             args = (hOutRead,blocks))
        t.start()
        return info[0], t, blocks

    def redirectCaptureThread(self, handle, captured_blocks):
        # One of these running per child process we are watching.  It
        # handles both stdout and stderr on a single handle. The read data is
        # never referenced until the thread dies - so no need for locks
        # around self.captured_blocks.
        #self.info("Redirect thread starting")
        while 1:
            try:
                ec, data = win32file.ReadFile(handle, CHILDCAPTURE_BLOCK_SIZE)
            except pywintypes.error, err:
                # ERROR_BROKEN_PIPE means the child process closed the
                # handle - ie, it terminated.
                if err[0] != winerror.ERROR_BROKEN_PIPE:
                    self.warning("Error reading output from process: %s" % err)
                break
            captured_blocks.append(data)
            del captured_blocks[CHILDCAPTURE_MAX_BLOCKS:]
        handle.Close()
        #self.info("Redirect capture thread terminating")

    def newPipe(self):
        sa = win32security.SECURITY_ATTRIBUTES()
        sa.bInheritHandle = True
        return win32pipe.CreatePipe(sa, 0)

    def dup(self, pipe):
        # create a duplicate handle that is not inherited, so that
        # it can be closed in the parent.  close the original pipe in
        # the process.
        pid = win32api.GetCurrentProcess()
        dup = win32api.DuplicateHandle(pid, pipe, pid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        pipe.Close()
        return dup

# Service registration and startup
def RegisterWithFirewall(exe_name, description):
    # Register our executable as an exception with Windows Firewall.
    # taken from http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ics/ics/wf_adding_an_application.asp
    from win32com.client import Dispatch
    #  Set constants
    NET_FW_PROFILE_DOMAIN = 0
    NET_FW_PROFILE_STANDARD = 1
    
    # Scope
    NET_FW_SCOPE_ALL = 0
    
    # IP Version - ANY is the only allowable setting for now
    NET_FW_IP_VERSION_ANY = 2
    
    fwMgr = Dispatch("HNetCfg.FwMgr")
    
    # Get the current profile for the local firewall policy.
    profile = fwMgr.LocalPolicy.CurrentProfile
    
    app = Dispatch("HNetCfg.FwAuthorizedApplication")
    
    app.ProcessImageFileName = exe_name
    app.Name = description
    app.Scope = NET_FW_SCOPE_ALL
    # Use either Scope or RemoteAddresses, but not both
    #app.RemoteAddresses = "*"
    app.IpVersion = NET_FW_IP_VERSION_ANY
    app.Enabled = True
    
    # Use this line if you want to add the app, but disabled.
    #app.Enabled = False
    
    profile.AuthorizedApplications.Add(app)

# A custom install function.
def CustomInstall(opts):
    # Register this process with the Windows Firewaall
    import pythoncom
    try:
        RegisterWithFirewall(sys.executable, "BuildBot")
    except pythoncom.com_error, why:
        print "FAILED to register with the Windows firewall"
        print why

#
# Magic code to allow shutdown.  Note that this code is executed in
# the *child* process, by way of the service process executing us with
# special cmdline args (which includes the service stop handle!)
def _RunChild():
    del sys.argv[1] # The --spawn arg.
    # Create a new thread that just waits for the event to be signalled.
    t = threading.Thread(target=_WaitForShutdown, 
                         args = (int(sys.argv[1]),)
                         )
    del sys.argv[1] # The stop handle
    # This child process will be sent a console handler notification as
    # users log off, or as the system shuts down.  We want to ignore these
    # signals as the service parent is responsible for our shutdown.
    def ConsoleHandler(what):
        # We can ignore *everything* - ctrl+c will never be sent as this
        # process is never attached to a console the user can press the
        # key in!
        return True
    win32api.SetConsoleCtrlHandler(ConsoleHandler, True)
    t.setDaemon(True) # we don't want to wait for this to stop!
    t.start()
    if hasattr(sys, "frozen"):
        # py2exe sets this env vars that may screw our child process - reset
        del os.environ["PYTHONPATH"]

    # Start the buildbot app
    from buildbot.scripts import runner
    runner.run()
    print "Service child process terminating normally."

def _WaitForShutdown(h):
    win32event.WaitForSingleObject(h, win32event.INFINITE)
    print "Shutdown requested"

    from twisted.internet import reactor
    reactor.callLater(0, reactor.stop)

# This function is also called by the py2exe startup code.
def HandleCommandLine():
    if len(sys.argv)>1 and sys.argv[1] == "--spawn":
        # Special command-line created by the service to execute the
        # child-process.
        # First arg is the handle to wait on
        _RunChild()
    else:
        win32serviceutil.HandleCommandLine(BBService,
                                           customOptionHandler=CustomInstall)

if __name__ == '__main__':
    HandleCommandLine()

--- NEW FILE: setup.py ---
# setup.py
# A distutils setup script to create py2exe binaries for buildbot.
# Both a service and standard executable are created.
# Usage:
# % setup.py py2exe

import sys, os, tempfile, shutil
from os.path import dirname, join, abspath, exists, splitext

this_dir = abspath(dirname(__file__))
bb_root_dir = abspath(join(this_dir, "..", ".."))

from distutils.core import setup
import py2exe

includes = []

# We try and bundle *all* modules in the following packages:
for package in ["buildbot.changes", "buildbot.process", "buildbot.status"]:
    __import__(package)
    p = sys.modules[package]
    for fname in os.listdir(p.__path__[0]):
        base, ext = splitext(fname)
        if not fname.startswith("_") and ext == ".py":
            includes.append(p.__name__ + "." + base)

# Other misc modules dynamically imported, so missed by py2exe
includes.extend("""
            buildbot.scheduler
            buildbot.slave.bot
            buildbot.master
            twisted.internet.win32eventreactor
            twisted.web.resource""".split())

# Turn into "," sep py2exe requires
includes = ",".join(includes)

py2exe_options = {"bundle_files": 1,
                  "includes": includes,
                 }

# Each "target" executable we create
buildbot_target = {
    "script": join(bb_root_dir, "bin", "buildbot")
}
# Due to the way py2exe works, we need to rebuild the service code as a
# normal console process - this will be executed by the service itself.

service_target = {
    "modules": ["buildbot_service"],
    "cmdline_style": "custom",
}

# We use the py2exe "bundle" option, so servicemanager.pyd
# (which has the message resources) does not exist.  Take a copy
# of it with a "friendlier" name.  The service runtime arranges for this
# to be used.
import servicemanager

msg_file = join(tempfile.gettempdir(), "buildbot.msg")
shutil.copy(servicemanager.__file__, msg_file)

data_files = [
    ["", [msg_file]],
    ["", [join(bb_root_dir, "buildbot", "status", "classic.css")]],
    ["", [join(bb_root_dir, "buildbot", "buildbot.png")]],
]

try:
    setup(name="buildbot",
          # The buildbot script as a normal executable
          console=[buildbot_target],
          service=[service_target],
          options={'py2exe': py2exe_options},
          data_files = data_files,
          zipfile = "buildbot.library", # 'library.zip' invites trouble :)
    )
finally:
    os.unlink(msg_file)




From warner at users.sourceforge.net  Sat Jun  3 20:21:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 20:21:06 +0000
Subject: [Buildbot-commits] buildbot ChangeLog, 1.646, 1.647 setup.py, 1.35,
	1.36
Message-ID: <E1FmccV-0003lu-2K@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15165

Modified Files:
	ChangeLog setup.py 
Log Message:
[project @ add py2exe support for windows, SF#1401121]

Original author: warner at lothar.com
Date: 2006-06-03 20:18:42

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.646
retrieving revision 1.647
diff -u -d -r1.646 -r1.647
--- ChangeLog	3 Jun 2006 18:23:04 -0000	1.646
+++ ChangeLog	3 Jun 2006 20:21:04 -0000	1.647
@@ -1,5 +1,14 @@
 2006-06-03  Brian Warner  <warner at lothar.com>
 
+	* contrib/windows/{setup.py, buildbot_service.py}: add support for
+	running py2exe on windows, contributed by Mark Hammond. Addresses
+	SF#1401121, but I think we still need to include
+	buildbot/scripts/sample.cfg
+	* setup.py: include buildbot_service.py as a script under windows
+	* buildbot/status/html.py: when sys.frozen (i.e. we're running in
+	a py2exe application), get the icon/css datafiles from a different
+	place than usual.
+
 	* buildbot/status/mail.py (MailNotifier.buildMessage): don't
 	double-escape the build URL. Thanks to Olivier Bonnet for the
 	patch. Fixes SF#1452801.

Index: setup.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/setup.py,v
retrieving revision 1.35
retrieving revision 1.36
diff -u -d -r1.35 -r1.36
--- setup.py	17 May 2005 20:04:21 -0000	1.35
+++ setup.py	3 Jun 2006 20:21:04 -0000	1.36
@@ -35,6 +35,7 @@
 scripts = ["bin/buildbot"]
 if sys.platform == "win32":
     scripts.append("contrib/windows/buildbot.bat")
+    scripts.append("contrib/windows/buildbot_service.py")
 
 setup(name="buildbot",
       version=version,




From warner at users.sourceforge.net  Sat Jun  3 20:21:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 20:21:06 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status html.py,1.85,1.86
Message-ID: <E1FmccU-0003lq-On@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15165/buildbot/status

Modified Files:
	html.py 
Log Message:
[project @ add py2exe support for windows, SF#1401121]

Original author: warner at lothar.com
Date: 2006-06-03 20:18:42

Index: html.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/html.py,v
retrieving revision 1.85
retrieving revision 1.86
diff -u -d -r1.85 -r1.86
--- html.py	2 Jun 2006 05:15:46 -0000	1.85
+++ html.py	3 Jun 2006 20:21:04 -0000	1.86
@@ -16,7 +16,7 @@
 
 from buildbot.twcompat import implements, Interface
 
-import string, types, time, os.path
+import sys, string, types, time, os.path
 
 from buildbot import interfaces, util
 from buildbot import version
@@ -1582,11 +1582,19 @@
 
         return NoResource("No such Builder '%s'" % path)
 
-# the icon is sibpath(__file__, "../buildbot.png") . This is for portability.
-up = os.path.dirname
-buildbot_icon = os.path.abspath(os.path.join(up(up(__file__)),
-                                             "buildbot.png"))
-buildbot_css = os.path.abspath(os.path.join(up(__file__), "classic.css"))
+if hasattr(sys, "frozen"):
+    # all 'data' files are in the directory of our executable
+    here = os.path.dirname(sys.executable)
+    buildbot_icon = os.path.abspath(os.path.join(here, "buildbot.png"))
+    buildbot_css = os.path.abspath(os.path.join(here, "classic.css"))
+else:
+    # running from source
+    # the icon is sibpath(__file__, "../buildbot.png") . This is for
+    # portability.
+    up = os.path.dirname
+    buildbot_icon = os.path.abspath(os.path.join(up(up(__file__)),
+                                                 "buildbot.png"))
+    buildbot_css = os.path.abspath(os.path.join(up(__file__), "classic.css"))
 
 class Waterfall(base.StatusReceiverMultiService):
     """I implement the primary web-page status interface, called a 'Waterfall




From warner at users.sourceforge.net  Sat Jun  3 20:48:02 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Sat, 03 Jun 2006 20:48:02 +0000
Subject: [Buildbot-commits] site index.html,1.66,1.67 ChangeLog,1.38,1.39
Message-ID: <E1Fmd2W-0003sm-0a@mail.sourceforge.net>

Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30510

Modified Files:
	index.html ChangeLog 
Log Message:
add the Auger buildbot

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/site/ChangeLog,v
retrieving revision 1.38
retrieving revision 1.39
diff -u -d -r1.38 -r1.39
--- ChangeLog	3 Jun 2006 20:00:08 -0000	1.38
+++ ChangeLog	3 Jun 2006 20:48:00 -0000	1.39
@@ -1,7 +1,7 @@
 2006-06-03  Brian Warner  <warner at lothar.com>
 
 	* index.html: add the Aqsis buildbot, and the Enfold Systems
-	buildbot
+	buildbot, and the Auger Observatory buildbot.
 
 	* source-Arch.html: update cvs-vs-arch-vs-darcs stuff
 

Index: index.html
===================================================================
RCS file: /cvsroot/buildbot/site/index.html,v
retrieving revision 1.66
retrieving revision 1.67
diff -u -d -r1.66 -r1.67
--- index.html	3 Jun 2006 20:00:08 -0000	1.66
+++ index.html	3 Jun 2006 20:48:00 -0000	1.67
@@ -505,6 +505,21 @@
   </tr>
 
   <tr>
+    <td>Auger Observatory</td>
+    <td>
+    </td>
+    <td><a href="http://www.auger.org/">
+      <img src="home.png" alt="home page" height="16" width="16"></a>
+    </td>
+    <td>Tom Paul reports that a buildbot is in use at the Pierre Auger
+      Observatory to test the data processing and analysis software in use
+      there. "The observatory is designed to unveil the origins and
+      composition of the highest energy cosmic rays, and is operated by a
+      collaboration of about 300 physicists (running code on quite a few
+      different platforms)". </td>
+</tr>
+
+  <tr>
     <td></td>
     <td>
     </td>
@@ -556,5 +571,5 @@
 href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons
 Attribution Share-Alike license</a>.</font></p>
 
-<!-- hhmts start -->Last modified: Sat Jun  3 12:57:59 PDT 2006 <!-- hhmts end -->
+<!-- hhmts start -->Last modified: Sat Jun  3 13:47:05 PDT 2006 <!-- hhmts end -->
 </body> </html>




From warner at users.sourceforge.net  Mon Jun  5 00:47:52 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 05 Jun 2006 00:47:52 +0000
Subject: [Buildbot-commits] site ChangeLog,1.39,1.40 index.html,1.67,1.68
Message-ID: <E1FnPPD-0002Jz-Rv@mail.sourceforge.net>

Update of /cvsroot/buildbot/site
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16512

Modified Files:
	ChangeLog index.html 
Log Message:
add the Auger waterfall too

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/site/ChangeLog,v
retrieving revision 1.39
retrieving revision 1.40
diff -u -d -r1.39 -r1.40
--- ChangeLog	3 Jun 2006 20:48:00 -0000	1.39
+++ ChangeLog	5 Jun 2006 00:47:49 -0000	1.40
@@ -1,3 +1,7 @@
+2006-06-04  Brian Warner  <warner at lothar.com>
+
+	* index.html: add the Auger waterfall too
+
 2006-06-03  Brian Warner  <warner at lothar.com>
 
 	* index.html: add the Aqsis buildbot, and the Enfold Systems

Index: index.html
===================================================================
RCS file: /cvsroot/buildbot/site/index.html,v
retrieving revision 1.67
retrieving revision 1.68
diff -u -d -r1.67 -r1.68
--- index.html	3 Jun 2006 20:48:00 -0000	1.67
+++ index.html	5 Jun 2006 00:47:49 -0000	1.68
@@ -506,7 +506,8 @@
 
   <tr>
     <td>Auger Observatory</td>
-    <td>
+    <td><a href="http://hepgis.physics.neu.edu:8010"/>
+      <img src="buildbot.png" alt="Buildbot" height="16" width="16"></a>
     </td>
     <td><a href="http://www.auger.org/">
       <img src="home.png" alt="home page" height="16" width="16"></a>
@@ -571,5 +572,5 @@
 href="http://creativecommons.org/licenses/by-sa/2.5/">Creative Commons
 Attribution Share-Alike license</a>.</font></p>
 
-<!-- hhmts start -->Last modified: Sat Jun  3 13:47:05 PDT 2006 <!-- hhmts end -->
+<!-- hhmts start -->Last modified: Sun Jun  4 17:44:56 PDT 2006 <!-- hhmts end -->
 </body> </html>




From warner at users.sourceforge.net  Tue Jun  6 19:01:14 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 06 Jun 2006 19:01:14 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.87,1.88
Message-ID: <E1Fni7C-0004Oe-8a@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17948/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ make WithProperties inherit from ComparableMixin, to avoid spurious config changes]

Original author: warner at lothar.com
Date: 2006-06-06 18:58:46

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.87
retrieving revision 1.88
diff -u -d -r1.87 -r1.88
--- step.py	2 Jun 2006 03:32:23 -0000	1.87
+++ step.py	6 Jun 2006 19:01:12 -0000	1.88
@@ -9,6 +9,7 @@
 from twisted.python.failure import Failure
 from twisted.web.util import formatFailure
 
+from buildbot import util
 from buildbot.interfaces import BuildSlaveTooOldError
 from buildbot.util import now
 from buildbot.status import progress, builder
@@ -852,11 +853,13 @@
             p = ""
         return p
 
-class WithProperties:
+class WithProperties(util.ComparableMixin):
     """This is a marker class, used in ShellCommand's command= argument to
     indicate that we want to interpolate a build property.
     """
 
+    compare_attrs = ('fmtstring', 'args')
+
     def __init__(self, fmtstring, *args):
         self.fmtstring = fmtstring
         self.args = args




From warner at users.sourceforge.net  Tue Jun  6 19:01:14 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 06 Jun 2006 19:01:14 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.647,1.648
Message-ID: <E1FnkB8-00026S-FX@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17948

Modified Files:
	ChangeLog 
Log Message:
[project @ make WithProperties inherit from ComparableMixin, to avoid spurious config changes]

Original author: warner at lothar.com
Date: 2006-06-06 18:58:46

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.647
retrieving revision 1.648
diff -u -d -r1.647 -r1.648
--- ChangeLog	3 Jun 2006 20:21:04 -0000	1.647
+++ ChangeLog	6 Jun 2006 19:01:12 -0000	1.648
@@ -1,3 +1,9 @@
+2006-06-06  Brian Warner  <warner at lothar.com>
+
+	* buildbot/process/step.py (WithProperties): make this inherit
+	from ComparableMixin, so that reloading an unchanged config file
+	doesn't cause us to spuriously reload any Builders which use them.
+
 2006-06-03  Brian Warner  <warner at lothar.com>
 
 	* contrib/windows/{setup.py, buildbot_service.py}: add support for




From warner at users.sourceforge.net  Wed Jun  7 16:53:21 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 07 Jun 2006 16:53:21 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.648,1.649
Message-ID: <E1Fo2RR-0007GH-4S@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17114

Modified Files:
	ChangeLog 
Log Message:
[project @ add a test for the WithProperties/ComparableMixin fix]

Original author: warner at lothar.com
Date: 2006-06-07 16:50:17

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.648
retrieving revision 1.649
diff -u -d -r1.648 -r1.649
--- ChangeLog	6 Jun 2006 19:01:12 -0000	1.648
+++ ChangeLog	7 Jun 2006 16:53:18 -0000	1.649
@@ -3,6 +3,8 @@
 	* buildbot/process/step.py (WithProperties): make this inherit
 	from ComparableMixin, so that reloading an unchanged config file
 	doesn't cause us to spuriously reload any Builders which use them.
+	* buildbot/test/test_config.py (ConfigTest.testWithProperties):
+	add a test for it
 
 2006-06-03  Brian Warner  <warner at lothar.com>
 




From warner at users.sourceforge.net  Wed Jun  7 16:53:21 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 07 Jun 2006 16:53:21 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_config.py, 1.34, 1.35
Message-ID: <E1Fo2Te-0006rL-8X@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17114/buildbot/test

Modified Files:
	test_config.py 
Log Message:
[project @ add a test for the WithProperties/ComparableMixin fix]

Original author: warner at lothar.com
Date: 2006-06-07 16:50:17

Index: test_config.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_config.py,v
retrieving revision 1.34
retrieving revision 1.35
diff -u -d -r1.34 -r1.35
--- test_config.py	13 Jan 2006 08:34:29 -0000	1.34
+++ test_config.py	7 Jun 2006 16:53:19 -0000	1.35
@@ -81,6 +81,27 @@
                    'builddir': 'workdir2', 'factory': f1 }]
 """
 
+wpCfg1 = buildersCfg + \
+"""
+from buildbot.process import step
+f1 = BasicBuildFactory('cvsroot', 'cvsmodule')
+f1.addStep(step.ShellCommand, command=[step.WithProperties('echo')])
+c['builders'] = [{'name':'builder1', 'slavename':'bot1',
+                  'builddir':'workdir1', 'factory': f1}]
+"""
+
+wpCfg2 = buildersCfg + \
+"""
+from buildbot.process import step
+f1 = BasicBuildFactory('cvsroot', 'cvsmodule')
+f1.addStep(step.ShellCommand,
+           command=[step.WithProperties('echo %s', 'revision')])
+c['builders'] = [{'name':'builder1', 'slavename':'bot1',
+                  'builddir':'workdir1', 'factory': f1}]
+"""
+
+
+
 ircCfg1 = emptyCfg + \
 """
 from buildbot.status import words
@@ -686,6 +707,28 @@
         self.failUnlessEqual(master.botmaster.builders, {})
         #self.failUnlessEqual(master.client_svc.statusbags, {}) # TODO
 
+    def testWithProperties(self):
+        master = self.buildmaster
+        master.loadConfig(wpCfg1)
+        self.failUnlessEqual(master.botmaster.builderNames, ["builder1"])
+        self.failUnlessEqual(master.botmaster.builders.keys(), ["builder1"])
+        b1 = master.botmaster.builders["builder1"]
+
+        # reloading the same config should leave the builder unchanged
+        master.loadConfig(wpCfg1)
+        b2 = master.botmaster.builders["builder1"]
+        self.failUnlessIdentical(b1, b2)
+
+        # but changing the parameters of the WithProperties should change it
+        master.loadConfig(wpCfg2)
+        b3 = master.botmaster.builders["builder1"]
+        self.failIf(b1 is b3)
+
+        # again, reloading same config should leave the builder unchanged
+        master.loadConfig(wpCfg2)
+        b4 = master.botmaster.builders["builder1"]
+        self.failUnlessIdentical(b3, b4)
+
     def checkIRC(self, m, expected):
         ircs = {}
         for irc in self.servers(m, words.IRC):




From warner at users.sourceforge.net  Thu Jun  8 21:42:36 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 08 Jun 2006 21:42:36 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status client.py,1.25,1.26
Message-ID: <E1FoSH6-00033s-Mz@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv4088/buildbot/status

Modified Files:
	client.py 
Log Message:
[project @ fix an oversight in remote_getCurrentBuilds]

Original author: warner at lothar.com
Date: 2006-06-08 21:39:47

Index: client.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/client.py,v
retrieving revision 1.25
retrieving revision 1.26
diff -u -d -r1.25 -r1.26
--- client.py	13 Jan 2006 08:34:28 -0000	1.25
+++ client.py	8 Jun 2006 21:42:34 -0000	1.26
@@ -84,7 +84,7 @@
         return makeRemote(self.b.getLastFinishedBuild())
 
     def remote_getCurrentBuilds(self):
-        return makeRemote(self.b.getCurrentBuilds())
+        return [IRemote(b) for b in self.b.getCurrentBuilds()]
 
     def remote_getBuild(self, number):
         return makeRemote(self.b.getBuild(number))




From warner at users.sourceforge.net  Thu Jun  8 21:42:36 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 08 Jun 2006 21:42:36 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.649,1.650
Message-ID: <E1FoSH9-0005UW-0U@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv4088

Modified Files:
	ChangeLog 
Log Message:
[project @ fix an oversight in remote_getCurrentBuilds]

Original author: warner at lothar.com
Date: 2006-06-08 21:39:47

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.649
retrieving revision 1.650
diff -u -d -r1.649 -r1.650
--- ChangeLog	7 Jun 2006 16:53:18 -0000	1.649
+++ ChangeLog	8 Jun 2006 21:42:34 -0000	1.650
@@ -1,3 +1,12 @@
+2006-06-08  Brian Warner  <warner at allmydata.com>
+
+	* buildbot/status/client.py
+	(RemoteBuilder.remote_getCurrentBuilds): oops, I screwed up when
+	changing this from getCurrentBuild() to getCurrentBuilds(). Each
+	build needs to be IRemote'd separately, rather than IRemote'ing
+	the whole list at once. I can't wait until newpb's serialization
+	adapters make this unnecessary.
+
 2006-06-06  Brian Warner  <warner at lothar.com>
 
 	* buildbot/process/step.py (WithProperties): make this inherit




From warner at users.sourceforge.net  Mon Jun 12 08:35:56 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:35:56 +0000
Subject: [Buildbot-commits] buildbot ChangeLog, 1.650, 1.651 setup.py, 1.36,
	1.37
Message-ID: <E1Fphtt-0005Rc-9N@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16596

Modified Files:
	ChangeLog setup.py 
Log Message:
[project @ setup.py: add Trove classifiers for PyPI]

Original author: warner at lothar.com
Date: 2006-06-12 07:18:01

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.650
retrieving revision 1.651
diff -u -d -r1.650 -r1.651
--- ChangeLog	8 Jun 2006 21:42:34 -0000	1.650
+++ ChangeLog	12 Jun 2006 08:35:53 -0000	1.651
@@ -1,3 +1,7 @@
+2006-06-12  Brian Warner  <warner at lothar.com>
+
+	* setup.py: add Trove classifiers for PyPI
+
 2006-06-08  Brian Warner  <warner at allmydata.com>
 
 	* buildbot/status/client.py

Index: setup.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/setup.py,v
retrieving revision 1.36
retrieving revision 1.37
diff -u -d -r1.36 -r1.37
--- setup.py	3 Jun 2006 20:21:04 -0000	1.36
+++ setup.py	12 Jun 2006 08:35:53 -0000	1.37
@@ -45,6 +45,16 @@
       author_email="warner-buildbot at lothar.com",
       url="http://buildbot.sourceforge.net/",
       license="GNU GPL",
+      classifiers=[
+    'Development Status :: 4 - Beta',
+    'Environment :: No Input/Output (Daemon)',
+    'Environment :: Web Environment',
+    'Intended Audience :: Developers',
+    'License :: OSI Approved :: GNU General Public License (GPL)',
+    'Topic :: Software Development :: Build Tools',
+    'Topic :: Software Development :: Testing',
+    ],
+
       packages=["buildbot",
                 "buildbot.status",
                 "buildbot.changes",




From warner at users.sourceforge.net  Mon Jun 12 08:36:10 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:10 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.88,1.89
Message-ID: <E1FphuA-0005kF-5T@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16958/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ applied patch SF#1473939, initial Perforce support]

Original author: warner at lothar.com
Date: 2006-06-12 07:01:00

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.88
retrieving revision 1.89
diff -u -d -r1.88 -r1.89
--- step.py	6 Jun 2006 19:01:12 -0000	1.88
+++ step.py	12 Jun 2006 08:36:08 -0000	1.89
@@ -1822,24 +1822,69 @@
         self.startCommand(cmd)
 
 
-class todo_P4(Source):
+class P4(Source):
+    """ P4 is a class for accessing perforce revision control"""
     name = "p4"
 
-    # to create the working directory for the first time:
-    #  need to create a View. The 'Root' parameter will have to be filled
-    #  in by the buildslave with the abspath of the basedir. Then the
-    #  setup process involves 'p4 client' to set up the view. After
-    #  that, 'p4 sync' does all the necessary updating.
-    #  P4PORT=P4PORT P4CLIENT=name p4 client
+    def __init__(self, p4base, defaultBranch=None, p4port=None, p4user=None,
+                 p4passwd=None, p4extra_views=[],
+                 p4client='buildbot_%(slave)s_%(builder)s', **kwargs):
+        """
+        @type  p4base: string
+        @param p4base: A view into a perforce depot, typically
+                       "//depot/proj/"
 
-    def __init__(self, p4port, view, **kwargs):
+        @type  defaultBranch: string
+        @param defaultBranch: Identify a branch to build by default. Perforce
+                              is a view based branching system. So, the branch
+                              is normally the name after the base. For example,
+                              branch=1.0 is view=//depot/proj/1.0/...
+                              branch=1.1 is view=//depot/proj/1.1/...
+
+        @type  p4port: string
+        @param p4port: Specify the perforce server to connection in the format
+                       <host>:<port>. Example "perforce.example.com:1666"
+
+        @type  p4user: string
+        @param p4user: The perforce user to run the command as.
+
+        @type  p4passwd: string
+        @param p4passwd: The password for the perforce user.
+
+        @type  p4extra_views: list of tuples
+        @param p4extra_views: Extra views to be added to
+                              the client that is being used.
+
+        @type  p4client: string
+        @param p4client: The perforce client to use for this buildslave.
+        """
+
+        self.branch = defaultBranch
         Source.__init__(self, **kwargs)
-        self.args.update({'p4port': p4port,
-                          'view': view,
-                          })
+        self.args['p4port'] = p4port
+        self.args['p4user'] = p4user
+        self.args['p4passwd'] = p4passwd
+        self.args['p4base'] = p4base
+        self.args['p4extra_views'] = p4extra_views
+        self.args['p4client'] = p4client % {
+            'slave': self.build.slavename,
+            'builder': self.build.builder.name,
+        }
+
+    def computeSourceRevision(self, changes):
+        if not changes:
+            return None
+        lastChange = max([int(c.revision) for c in changes])
+        return lastChange
 
     def startVC(self, branch, revision, patch):
-        cmd = LoggedRemoteCommand("p4", self.args)
+        slavever = self.slaveVersion("p4")
+        assert slavever, "slave is too old, does not know about p4"
+        args = dict(self.args)
+        args['branch'] = branch or self.branch
+        args['revision'] = revision
+        args['patch'] = patch
+        cmd = LoggedRemoteCommand("p4", args)
         self.startCommand(cmd)
 
 class P4Sync(Source):




From warner at users.sourceforge.net  Mon Jun 12 08:36:10 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:10 +0000
Subject: [Buildbot-commits] buildbot/buildbot/changes p4poller.py,1.5,1.6
Message-ID: <E1FphuA-0003do-D8@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/changes
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16958/buildbot/changes

Modified Files:
	p4poller.py 
Log Message:
[project @ applied patch SF#1473939, initial Perforce support]

Original author: warner at lothar.com
Date: 2006-06-12 07:01:00

Index: p4poller.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/changes/p4poller.py,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -d -r1.5 -r1.6
--- p4poller.py	7 Apr 2006 04:14:57 -0000	1.5
+++ p4poller.py	12 Jun 2006 08:36:08 -0000	1.6
@@ -1,28 +1,51 @@
-#! /usr/bin/python
+# -*- test-case-name: buildbot.test.test_p4poller -*-
 
 # Many thanks to Dave Peticolas for contributing this module
 
-from twisted.internet import defer
+import re
+import time
+
+from twisted.python import log, failure
+from twisted.internet import defer, reactor
 from twisted.internet.utils import getProcessOutput
 from twisted.internet.task import LoopingCall
 
 from buildbot import util
 from buildbot.changes import base, changes
 
+def get_simple_split(branchfile):
+    """Splits the branchfile argument and assuming branch is 
+       the first path component in branchfile, will return
+       branch and file else None."""
+
+    index = branchfile.find('/')
+    if index == -1: return None, None
+    branch, file = branchfile.split('/', 1)
+    return branch, file
+
 class P4Source(base.ChangeSource, util.ComparableMixin):
     """This source will poll a perforce repository for changes and submit
     them to the change master."""
 
-    compare_attrs = ["p4port", "p4user", "p4passwd", "p4client", "p4base",
+    compare_attrs = ["p4port", "p4user", "p4passwd", "p4base",
                      "p4bin", "pollinterval", "histmax"]
 
+    changes_line_re = re.compile(
+            r"Change (?P<num>\d+) on \S+ by \S+@\S+ '.+'$")
+    describe_header_re = re.compile(
+            r"Change \d+ by (?P<who>\S+)@\S+ on (?P<when>.+)$")
+    file_re = re.compile(r"^\.\.\. (?P<path>[^#]+)#\d+ \w+$")
+    datefmt = '%Y/%m/%d %H:%M:%S'
+
     parent = None # filled in when we're added
     last_change = None
     loop = None
     volatile = ['loop']
+    working = False
 
-    def __init__(self, p4port, p4user, p4passwd=None, p4client=None,
-                 p4base='//...', p4bin='p4',
+    def __init__(self, p4port=None, p4user=None, p4passwd=None,
+                 p4base='//', p4bin='p4',
+                 split_file=lambda branchfile: (None, branchfile),
                  pollinterval=60 * 10, histmax=100):
         """
         @type  p4port:       string
@@ -31,13 +54,13 @@
         @param p4user:       p4 user
         @type  p4passwd:     string
         @param p4passwd:     p4 passwd
-        @type  p4client:     string
-        @param p4client:     name of p4 client to poll
         @type  p4base:       string
         @param p4base:       p4 file specification to limit a poll to
-                             (i.e., //...)
+                             without the trailing '...' (i.e., //)
         @type  p4bin:        string
         @param p4bin:        path to p4 binary, defaults to just 'p4'
+        @type  split_file:   func
+        $param split_file:   splits a filename into branch and filename.
         @type  pollinterval: int
         @param pollinterval: interval in seconds between polls
         @type  histmax:      int
@@ -47,28 +70,49 @@
         self.p4port = p4port
         self.p4user = p4user
         self.p4passwd = p4passwd
-        self.p4client = p4client
         self.p4base = p4base
         self.p4bin = p4bin
+        self.split_file = split_file
         self.pollinterval = pollinterval
         self.histmax = histmax
 
     def startService(self):
         self.loop = LoopingCall(self.checkp4)
-        self.loop.start(self.pollinterval)
         base.ChangeSource.startService(self)
 
+        # Don't start the loop just yet because the reactor isn't running.
+        # Give it a chance to go and install our SIGCHLD handler before
+        # spawning processes.
+        reactor.callLater(0, self.loop.start, self.pollinterval)
+
     def stopService(self):
         self.loop.stop()
         return base.ChangeSource.stopService(self)
 
     def describe(self):
-        return "p4source %s-%s %s" % (self.p4port, self.p4client, self.p4base)
+        return "p4source %s %s" % (self.p4port, self.p4base)
 
     def checkp4(self):
-        d = self._get_changes()
-        d.addCallback(self._process_changes)
-        d.addCallback(self._handle_changes)
+        # Our return value is only used for unit testing.
+        if self.working:
+            log.msg("Skipping checkp4 because last one has not finished")
+            return defer.succeed(None)
+        else:
+            self.working = True
+            d = self._get_changes()
+            d.addCallback(self._process_changes)
+            d.addBoth(self._finished)
+            return d
+
+    def _finished(self, res):
+        assert self.working
+        self.working = False
+
+        # Again, the return value is only for unit testing.
+        # If there's a failure, log it so it isn't lost.
+        if isinstance(res, failure.Failure):
+            log.msg('P4 poll failed: %s' % res)
+        return res
 
     def _get_changes(self):
         args = []
@@ -78,9 +122,7 @@
             args.extend(['-u', self.p4user])
         if self.p4passwd:
             args.extend(['-P', self.p4passwd])
-        if self.p4client:
-            args.extend(['-c', self.p4client])
-        args.extend(['changes', '-m', str(self.histmax), self.p4base])
+        args.extend(['changes', '-m', str(self.histmax), self.p4base + '...'])
         env = {}
         return getProcessOutput(self.p4bin, args, env)
 
@@ -90,18 +132,26 @@
         for line in result.split('\n'):
             line = line.strip()
             if not line: continue
-            _, num, _, date, _, user, _ = line.split(' ', 6)
+            m = self.changes_line_re.match(line)
+            assert m, "Unexpected 'p4 changes' output: %r" % result
+            num = m.group('num')
             if last_change is None:
+                log.msg('P4Poller: starting at change %s' % num)
                 self.last_change = num
                 return []
-            if last_change == num: break
-            change = {'num' : num, 'date' : date, 'user' : user.split('@')[0]}
-            changelists.append(change)
+            if last_change == num:
+                break
+            changelists.append(num)
         changelists.reverse() # oldest first
-        ds = [self._get_change(c) for c in changelists]
-        return defer.DeferredList(ds)
 
-    def _get_change(self, change):
+        # Retrieve each sequentially.
+        d = defer.succeed(None)
+        for c in changelists:
+            d.addCallback(self._get_describe, c)
+            d.addCallback(self._process_describe, c)
+        return d
+
+    def _get_describe(self, dummy, num):
         args = []
         if self.p4port:
             args.extend(['-p', self.p4port])
@@ -109,34 +159,44 @@
             args.extend(['-u', self.p4user])
         if self.p4passwd:
             args.extend(['-P', self.p4passwd])
-        if self.p4client:
-            args.extend(['-c', self.p4client])
-        args.extend(['describe', '-s', change['num']])
+        args.extend(['describe', '-s', num])
         env = {}
         d = getProcessOutput(self.p4bin, args, env)
-        d.addCallback(self._process_change, change)
         return d
 
-    def _process_change(self, result, change):
+    def _process_describe(self, result, num):
         lines = result.split('\n')
+        m = self.describe_header_re.match(lines[0])
+        assert m, "Unexpected 'p4 describe -s' result: %r" % result
+        who = m.group('who')
+        when = time.mktime(time.strptime(m.group('when'), self.datefmt))
         comments = ''
         while not lines[0].startswith('Affected files'):
             comments += lines.pop(0) + '\n'
-        change['comments'] = comments
         lines.pop(0) # affected files
-        files = []
+
+        branch_files = {} # dict for branch mapped to file(s)
         while lines:
             line = lines.pop(0).strip()
             if not line: continue
-            files.append(line.split(' ')[1])
-        change['files'] = files
-        return change
+            m = self.file_re.match(line)
+            assert m, "Invalid file line: %r" % line
+            path = m.group('path')
+            if path.startswith(self.p4base):
+                branch, file = self.split_file(path[len(self.p4base):])
+                if (branch == None and file == None): continue
+                if branch_files.has_key(branch):
+                    branch_files[branch].append(file)
+                else:
+                    branch_files[branch] = [file]
 
-    def _handle_changes(self, result):
-        for success, change in result:
-            if not success: continue
-            c = changes.Change(change['user'], change['files'],
-                               change['comments'],
-                               revision=change['num'])
+        for branch in branch_files:
+            c = changes.Change(who=who,
+                               files=branch_files[branch],
+                               comments=comments,
+                               revision=num,
+                               when=when,
+                               branch=branch)
             self.parent.addChange(c)
-            self.last_change = change['num']
+
+        self.last_change = num




From warner at users.sourceforge.net  Mon Jun 12 08:36:10 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:10 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.53,1.54
Message-ID: <E1Fphu9-0005k6-V2@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16958/docs

Modified Files:
	buildbot.texinfo 
Log Message:
[project @ applied patch SF#1473939, initial Perforce support]

Original author: warner at lothar.com
Date: 2006-06-12 07:01:00

Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.53
retrieving revision 1.54
diff -u -d -r1.53 -r1.54
--- buildbot.texinfo	29 May 2006 00:10:20 -0000	1.53
+++ buildbot.texinfo	12 Jun 2006 08:36:08 -0000	1.54
@@ -128,6 +128,7 @@
 * CVSToys - mail notification::  
 * Other mail notification ChangeSources::  
 * PBChangeSource::              
+* P4Source::                    
 
 Build Process
 
@@ -150,7 +151,7 @@
 * Mercurial::                   
 * Arch::                        
 * Bazaar::                      
-* P4Sync::                      
+* P4::                          
 
 Simple ShellCommand Subclasses
 
@@ -1165,6 +1166,13 @@
 @code{branch} are simply concatenated together to derive the
 @code{svnurl} to use for the checkout.
 
+ at uref{http://www.perforce.com/, Perforce} is similar. The server
+is specified through a @code{P4PORT} parameter. Module and branch
+are specified in a single depot path, and revisions are
+depot-wide. When branches are used, the @code{p4base} and
+ at code{defaultBranch} are concatenated together to produce the depot
+path.
+
 @uref{http://wiki.gnuarch.org/, Arch} and
 @uref{http://bazaar.canonical.com/, Bazaar} specify a repository by
 URL, as well as a @code{version} which is kind of like a branch name.
@@ -2302,6 +2310,7 @@
 * CVSToys - mail notification::  
 * Other mail notification ChangeSources::  
 * PBChangeSource::              
+* P4Source::                    
 @end menu
 
 @node Choosing ChangeSources, CVSToys - PBService, Change Sources, Change Sources
@@ -2447,7 +2456,7 @@
 
 @code{BonsaiMaildirSource} parses messages sent out by Bonsai.
 
- at node PBChangeSource,  , Other mail notification ChangeSources, Change Sources
+ at node PBChangeSource, P4Source, Other mail notification ChangeSources, Change Sources
 @subsection PBChangeSource
 
 The last kind of ChangeSource actually listens on a TCP port for
@@ -2500,6 +2509,55 @@
 @end table
 
 
+ at node P4Source, , PBChangeSource, Change Sources
+ at subsection P4Source
+
+The @code{P4Source} periodically polls a @uref{http://www.perforce.com/,
+Perforce} depot for changes. It accepts the following arguments:
+
+ at table @samp
+ at item @code{p4base}
+The base depot path to watch, without the trailing '/...'.
+
+ at item @code{p4port}
+The Perforce server to connect to (as host:port).
+
+ at item @code{p4user}
+The Perforce user.
+
+ at item @code{p4passwd}
+The Perforce password.
+
+ at item @code{split_file}
+A function that maps a pathname, without the leading @code{p4base}, to a
+(branch, filename) tuple. The default just returns (None, branchfile),
+which effectively disables branch support. You should supply a function
+which understands your repository structure.
+
+ at item @code{pollinterval}
+How often to poll, in seconds. Defaults to 600 (10 minutes).
+
+ at item @code{histmax}
+The maximum number of changes to inspect at a time. If more than this
+number occur since the last poll, older changes will be silently
+ignored.
+ at end table
+
+ at heading Example
+
+This configuration uses the @code{P4PORT}, @code{P4USER}, and @code{P4PASSWD}
+specified in the buildmaster's environment. It watches a project in which the
+branch name is simply the next path component, and the file is all path
+components after.
+
+ at example
+import buildbot.changes.p4poller
+c['sources'].append(p4poller.P4Source(
+        p4base='//depot/project/',
+        split_file=lambda branchfile: branchfile.split('/',1)
+))
+ at end example
+
 @node Build Process, Status Delivery, Getting Source Code Changes, Top
 @chapter Build Process
 
@@ -2733,7 +2791,7 @@
 * Mercurial::                   
 * Arch::                        
 * Bazaar::                      
-* P4Sync::                      
+* P4::                          
 @end menu
 
 @node CVS, SVN, Source Checkout, Source Checkout
@@ -3005,7 +3063,7 @@
 
 @end table
 
- at node Bazaar, P4Sync, Arch, Source Checkout
+ at node Bazaar, P4, Arch, Source Checkout
 @subsubsection Bazaar
 
 @cindex Bazaar Checkout
@@ -3021,22 +3079,42 @@
 ourselves).
 
 
- at node P4Sync,  , Bazaar, Source Checkout
- at subsubsection P4Sync
+ at node P4,  , Bazaar, Source Checkout
+ at subsubsection P4
 
 @cindex Perforce Update
 
-The @code{P4Sync} build step performs a
- at uref{http://www.perforce.com/, Perforce} update. It is a temporary
-facility: a more complete P4 checkout step (named @code{P4}) will
-eventually replace it. This step requires significant manual setup on
-each build slave. It takes the following arguments.
+The @code{P4} build step creates a @uref{http://www.perforce.com/,
+Perforce} client specification and performs an update.
 
 @table @code
+ at item p4base
+A view into the Perforce depot without branch name or trailing "...".
+Typically "//depot/proj/".
+ at item defaultBranch
+A branch name to append on build requests if none is specified.
+Typically "trunk".
 @item p4port
-(required): the host:port string describing how to get to the P4 Depot
-(repository), used as the P4PORT environment variable for all p4
-commands
+(optional): the host:port string describing how to get to the P4 Depot
+(repository), used as the -p argument for all p4 commands.
+ at item p4user
+(optional): the Perforce user, used as the -u argument to all p4
+commands.
+ at item p4passwd
+(optional): the Perforce password, used as the -p argument to all p4
+commands.
+ at item p4extra_views
+(optional): a list of (depotpath, clientpath) tuples containing extra
+views to be mapped into the client specification. Both will have
+"/..." appended automatically. The client name and source directory
+will be prepended to the client path.
+ at item p4client
+(optional): The name of the client to use. In mode='copy' and
+mode='update', it's particularly important that a unique name is used
+for each checkout directory to avoid incorrect synchronization. For
+this reason, Python percent substitution will be performed on this value
+to replace %(slave)s with the slave name and %(builder)s with the
+builder name. The default is "buildbot_%(slave)s_%(build)s".
 @end table
 
 @node ShellCommand, Simple ShellCommand Subclasses, Source Checkout, Build Steps




From warner at users.sourceforge.net  Mon Jun 12 08:36:10 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:10 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.51,1.52
Message-ID: <E1FphuA-0003fP-Jt@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16958/buildbot/slave

Modified Files:
	commands.py 
Log Message:
[project @ applied patch SF#1473939, initial Perforce support]

Original author: warner at lothar.com
Date: 2006-06-12 07:01:00

Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.51
retrieving revision 1.52
diff -u -d -r1.51 -r1.52
--- commands.py	22 May 2006 18:02:32 -0000	1.51
+++ commands.py	12 Jun 2006 08:36:08 -0000	1.52
@@ -1439,6 +1439,128 @@
 registerSlaveCommand("hg", Mercurial, cvs_ver)
 
 
+class P4(SourceBase):
+    """A P4 source-updater.
+
+    ['p4port'] (required): host:port for server to access
+    ['p4user'] (optional): user to use for access
+    ['p4passwd'] (optional): passwd to try for the user
+    ['p4client'] (optional): client spec to use
+    ['p4views'] (optional): client views to use
+    """
+
+    header = "p4"
+
+    def setup(self, args):
+        SourceBase.setup(self, args)
+        self.p4port = args['p4port']
+        self.p4client = args['p4client']
+        self.p4user = args['p4user']
+        self.p4passwd = args['p4passwd']
+        self.p4base = args['p4base']
+        self.p4extra_views = args['p4extra_views']
+        self.p4mode = args['mode']
+        self.p4branch = args['branch']
+        self.p4logname = os.environ['LOGNAME']
+
+        self.sourcedata = str([
+            # Perforce server.
+            self.p4port,
+
+            # Client spec.
+            self.p4client,
+
+            # Depot side of view spec.
+            self.p4base,
+            self.p4branch,
+            self.p4extra_views,
+
+            # Local side of view spec (srcdir is made from these).
+            self.builder.basedir,
+            self.mode,
+            self.workdir
+        ])
+
+
+    def sourcedirIsUpdateable(self):
+        if os.path.exists(os.path.join(self.builder.basedir,
+                                       self.srcdir, ".buildbot-patched")):
+            return False
+        # We assume our client spec is still around.
+        # We just say we aren't updateable if the dir doesn't exist so we
+        # don't get ENOENT checking the sourcedata.
+        return os.path.isdir(os.path.join(self.builder.basedir,
+                                          self.srcdir))
+
+    def doVCUpdate(self):
+        return self._doP4Sync(force=False)
+
+    def _doP4Sync(self, force):
+        command = ['p4']
+
+        if self.p4port:
+            command.extend(['-p', self.p4port])
+        if self.p4user:
+            command.extend(['-u', self.p4user])
+        if self.p4passwd:
+            command.extend(['-P', self.p4passwd])
+        if self.p4client:
+            command.extend(['-c', self.p4client])
+        command.extend(['sync'])
+        if force:
+            command.extend(['-f'])
+        if self.revision:
+            command.extend(['@' + str(self.revision)])
+        env = {}
+        c = ShellCommand(self.builder, command, self.builder.basedir,
+                         environ=env, sendRC=False, timeout=self.timeout,
+                         keepStdout=True)
+        self.command = c
+        d = c.start()
+        d.addCallback(self._abandonOnFailure)
+        return d
+
+
+    def doVCFull(self):
+        env = {}
+        command = ['p4']
+        client_spec = ''
+        client_spec += "Client: %s\n\n" % self.p4client
+        client_spec += "Owner: %s\n\n" % self.p4logname
+        client_spec += "Description:\n\tCreated by %s\n\n" % self.p4logname
+        client_spec += "Root:\t%s\n\n" % self.builder.basedir
+        client_spec += "Options:\tallwrite rmdir\n\n"
+        client_spec += "LineEnd:\tlocal\n\n"
+
+        # Setup a view
+        client_spec += "View:\n\t%s" % (self.p4base)
+        if self.p4branch:
+            client_spec += "%s/" % (self.p4branch)
+        client_spec += "... //%s/%s/...\n" % (self.p4client, self.srcdir)
+        if self.p4extra_views:
+            for k, v in self.p4extra_views:
+                client_spec += "\t%s/... //%s/%s%s/...\n" % (k, self.p4client,
+                                                             self.srcdir, v)
+        if self.p4port:
+            command.extend(['-p', self.p4port])
+        if self.p4user:
+            command.extend(['-u', self.p4user])
+        if self.p4passwd:
+            command.extend(['-P', self.p4passwd])
+        command.extend(['client', '-i'])
+        log.msg(client_spec)
+        c = ShellCommand(self.builder, command, self.builder.basedir,
+                         environ=env, sendRC=False, timeout=self.timeout,
+                         stdin=client_spec)
+        self.command = c
+        d = c.start()
+        d.addCallback(self._abandonOnFailure)
+        d.addCallback(lambda _: self._doP4Sync(force=True))
+        return d
+
+registerSlaveCommand("p4", P4, cvs_ver)
+
+
 class P4Sync(SourceBase):
     """A partial P4 source-updater. Requires manual setup of a per-slave P4
     environment. The only thing which comes from the master is P4PORT.
@@ -1463,7 +1585,7 @@
     def sourcedirIsUpdateable(self):
         return True
 
-    def doVCUpdate(self):
+    def _doVC(self, force):
         d = os.path.join(self.builder.basedir, self.srcdir)
         command = [self.vcexe]
         if self.p4port:
@@ -1475,6 +1597,8 @@
         if self.p4client:
             command.extend(['-c', self.p4client])
         command.extend(['sync'])
+        if force:
+            command.extend(['-f'])
         if self.revision:
             command.extend(['@' + self.revision])
         env = {}
@@ -1483,7 +1607,10 @@
         self.command = c
         return c.start()
 
+    def doVCUpdate(self):
+        return self._doVC(force=False)
+
     def doVCFull(self):
-        return self.doVCUpdate()
+        return self._doVC(force=True)
 
 registerSlaveCommand("p4sync", P4Sync, cvs_ver)




From warner at users.sourceforge.net  Mon Jun 12 08:36:10 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:10 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_p4poller.py, NONE,
	1.1 test_vc.py, 1.55, 1.56
Message-ID: <E1FphuA-0003fe-Ik@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16958/buildbot/test

Modified Files:
	test_vc.py 
Added Files:
	test_p4poller.py 
Log Message:
[project @ applied patch SF#1473939, initial Perforce support]

Original author: warner at lothar.com
Date: 2006-06-12 07:01:00

--- NEW FILE: test_p4poller.py ---
import sys
import time

from twisted.python import log, failure
from twisted.internet import defer
from twisted.trial import unittest

from buildbot.twcompat import maybeWait
from buildbot.changes.changes import Change
from buildbot.changes.p4poller import P4Source, get_simple_split

#log.startLogging(sys.stderr)

first_p4changes = \
"""Change 1 on 2006/04/13 by slamb at testclient 'first rev'
"""

second_p4changes = \
"""Change 3 on 2006/04/13 by bob at testclient 'short desc truncated'
Change 2 on 2006/04/13 by slamb at testclient 'bar'
"""

third_p4changes = \
"""Change 5 on 2006/04/13 by mpatel at testclient 'first rev'
"""

change_4_log = \
"""Change 4 by mpatel at testclient on 2006/04/13 21:55:39

	short desc truncated because this is a long description.
"""
change_3_log = \
"""Change 3 by bob at testclient on 2006/04/13 21:51:39

	short desc truncated because this is a long description.
"""

change_2_log = \
"""Change 2 by slamb at testclient on 2006/04/13 21:46:23

	creation
"""

p4change = {
    '3': change_3_log +
"""Affected files ...

... //depot/myproject/branch_b/branch_b_file#1 add
... //depot/myproject/branch_b/whatbranch#1 branch
... //depot/myproject/branch_c/whatbranch#1 branch
""",
    '2': change_2_log +
"""Affected files ...

... //depot/myproject/trunk/whatbranch#1 add
... //depot/otherproject/trunk/something#1 add
""",
    '5': change_4_log +
"""Affected files ...

... //depot/myproject/branch_b/branch_b_file#1 add
... //depot/myproject/branch_b#75 edit
... //depot/myproject/branch_c/branch_c_file#1 add
""",
}


class MockP4Source(P4Source):
    """Test P4Source which doesn't actually invoke p4."""
    invocation = 0

    def __init__(self, p4changes, p4change, *args, **kwargs):
        P4Source.__init__(self, *args, **kwargs)
        self.p4changes = p4changes
        self.p4change = p4change

    def _get_changes(self):
        assert self.working
        result = self.p4changes[self.invocation]
        self.invocation += 1
        return defer.succeed(result)

    def _get_describe(self, dummy, num):
        assert self.working
        return defer.succeed(self.p4change[num])

class TestP4Poller(unittest.TestCase):
    def setUp(self):
        self.changes = []
        self.addChange = self.changes.append

    def testCheck(self):
        """successful checks"""
        self.t = MockP4Source(p4changes=[first_p4changes, second_p4changes],
                              p4change=p4change,
                              p4port=None, p4user=None,
                              p4base='//depot/myproject/',
                              split_file=lambda x: x.split('/', 1))
        self.t.parent = self

        # The first time, it just learns the change to start at.
        self.assert_(self.t.last_change is None)
        self.assert_(not self.t.working)
        return maybeWait(self.t.checkp4().addCallback(self._testCheck2))

    def _testCheck2(self, res):
        self.assertEquals(self.changes, [])
        self.assertEquals(self.t.last_change, '1')

        # Subsequent times, it returns Change objects for new changes.
        return self.t.checkp4().addCallback(self._testCheck3)

    def _testCheck3(self, res):
        self.assertEquals(len(self.changes), 3)
        self.assertEquals(self.t.last_change, '3')
        self.assert_(not self.t.working)

        # They're supposed to go oldest to newest, so this one must be first.
        self.assertEquals(self.changes[0].asText(),
            Change(who='slamb',
                   files=['whatbranch'],
                   comments=change_2_log,
                   revision='2',
                   when=time.mktime((2006, 4, 13, 21, 46, 23, 3, 103, -1)),
                   branch='trunk').asText())

        # These two can happen in either order, since they're from the same
        # Perforce change.
        self.assertIn(
            Change(who='bob',
                   files=['branch_b_file',
                          'whatbranch'],
                   comments=change_3_log,
                   revision='3',
                   when=time.mktime((2006, 4, 13, 21, 51, 39, 3, 103, -1)),
                   branch='branch_b').asText(),
            [c.asText() for c in self.changes])
        self.assertIn(
            Change(who='bob',
                   files=['whatbranch'],
                   comments=change_3_log,
                   revision='3',
                   when=time.mktime((2006, 4, 13, 21, 51, 39, 3, 103, -1)),
                   branch='branch_c').asText(),
            [c.asText() for c in self.changes])

    def testFailedChanges(self):
        """'p4 changes' failure is properly reported"""
        self.t = MockP4Source(p4changes=['Perforce client error:\n...'],
                              p4change={},
                              p4port=None, p4user=None)
        self.t.parent = self
        d = self.t.checkp4()
        d.addBoth(self._testFailedChanges2)
        return maybeWait(d)

    def _testFailedChanges2(self, f):
        self.assert_(isinstance(f, failure.Failure))
        self.assertIn('Perforce client error', str(f))
        self.assert_(not self.t.working)

    def testFailedDescribe(self):
        """'p4 describe' failure is properly reported"""
        c = dict(p4change)
        c['3'] = 'Perforce client error:\n...'
        self.t = MockP4Source(p4changes=[first_p4changes, second_p4changes],
                              p4change=c, p4port=None, p4user=None)
        self.t.parent = self
        d = self.t.checkp4()
        d.addCallback(self._testFailedDescribe2)
        return maybeWait(d)

    def _testFailedDescribe2(self, res):
        # first time finds nothing; check again.
        return self.t.checkp4().addBoth(self._testFailedDescribe3)

    def _testFailedDescribe3(self, f):
        self.assert_(isinstance(f, failure.Failure))
        self.assertIn('Perforce client error', str(f))
        self.assert_(not self.t.working)
        self.assertEquals(self.t.last_change, '2')

    def testAlreadyWorking(self):
        """don't launch a new poll while old is still going"""
        self.t = P4Source()
        self.t.working = True
        self.assert_(self.t.last_change is None)
        d = self.t.checkp4()
        d.addCallback(self._testAlreadyWorking2)

    def _testAlreadyWorking2(self, res):
        self.assert_(self.t.last_change is None)

    def testSplitFile(self):
        """Make sure split file works on branch only changes"""
        self.t = MockP4Source(p4changes=[third_p4changes],
                              p4change=p4change,
                              p4port=None, p4user=None,
                              p4base='//depot/myproject/',
                              split_file=get_simple_split)
        self.t.parent = self
        self.t.last_change = 50
        d = self.t.checkp4()
        d.addCallback(self._testSplitFile)

    def _testSplitFile(self, res):
        self.assertEquals(len(self.changes), 2)
        self.assertEquals(self.t.last_change, '5')

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.55
retrieving revision 1.56
diff -u -d -r1.55 -r1.56
--- test_vc.py	19 May 2006 07:44:22 -0000	1.55
+++ test_vc.py	12 Jun 2006 08:36:08 -0000	1.56
@@ -6,7 +6,35 @@
 from email.Utils import mktime_tz, parsedate_tz
 
 from twisted.trial import unittest
-from twisted.internet import defer, reactor, utils
+from twisted.internet import defer, reactor, utils, protocol, error
+try:
+    from twisted.python.procutils import which
+except ImportError:
+    # copied from Twisted circa 2.2.0
+    def which(name, flags=os.X_OK):
+        """Search PATH for executable files with the given name.
+
+        @type name: C{str}
+        @param name: The name for which to search.
+
+        @type flags: C{int}
+        @param flags: Arguments to L{os.access}.
+
+        @rtype: C{list}
+        @param: A list of the full paths to files found, in the
+        order in which they were found.
+        """
+        result = []
+        exts = filter(None, os.environ.get('PATHEXT', '').split(os.pathsep))
+        for p in os.environ['PATH'].split(os.pathsep):
+            p = os.path.join(p, name)
+            if os.access(p, flags):
+                result.append(p)
+            for e in exts:
+                pext = p + e
+                if os.access(pext, flags):
+                    result.append(pext)
+        return result
 
 #defer.Deferred.debug = True
 
@@ -47,6 +75,49 @@
 # small web server to provide access to the repository files while the test
 # is running).
 
+# Perforce starts the daemon running on localhost. Unfortunately, it must
+# use a predetermined Internet-domain port number, unless we want to go
+# all-out: bind the listen socket ourselves and pretend to be inetd.
+
+try:
+    import cStringIO as StringIO
+except ImportError:
+    import StringIO
+
+class _PutEverythingGetter(protocol.ProcessProtocol):
+    def __init__(self, deferred, stdin):
+        self.deferred = deferred
+        self.outBuf = StringIO.StringIO()
+        self.errBuf = StringIO.StringIO()
+        self.outReceived = self.outBuf.write
+        self.errReceived = self.errBuf.write
+        self.stdin = stdin
+
+    def connectionMade(self):
+        if self.stdin is not None:
+            self.transport.write(self.stdin)
+            self.transport.closeStdin()
+
+    def processEnded(self, reason):
+        out = self.outBuf.getvalue()
+        err = self.errBuf.getvalue()
+        e = reason.value
+        code = e.exitCode
+        if e.signal:
+            self.deferred.errback((out, err, e.signal))
+        else:
+            self.deferred.callback((out, err, code))
+
+def myGetProcessOutputAndValue(executable, args=(), env={}, path='.',
+                               reactor=None, stdin=None):
+    """Like twisted.internet.utils.getProcessOutputAndValue but takes
+    stdin, too."""
+    if reactor is None:
+        from twisted.internet import reactor
+    d = defer.Deferred()
+    p = _PutEverythingGetter(d, stdin)
+    reactor.spawnProcess(p, executable, (executable,)+tuple(args), env, path)
+    return d
 
 config_vc = """
 from buildbot.process import factory, step
@@ -313,7 +384,7 @@
         self.branch.append(rev)
         self.allrevs.append(rev)
 
-    def runCommand(self, basedir, command, failureIsOk=False):
+    def runCommand(self, basedir, command, failureIsOk=False, stdin=None):
         # all commands passed to do() should be strings or lists. If they are
         # strings, none of the arguments may have spaces. This makes the
         # commands less verbose at the expense of restricting what they can
@@ -323,8 +394,9 @@
         #print "do %s" % command
         env = os.environ.copy()
         env['LC_ALL'] = "C"
-        d = utils.getProcessOutputAndValue(command[0], command[1:],
-                                           env=env, path=basedir)
+        d = myGetProcessOutputAndValue(command[0], command[1:],
+                                       env=env, path=basedir,
+                                       stdin=stdin)
         def check((out, err, code)):
             #print
             #print "command: %s" % command
@@ -342,14 +414,15 @@
         d.addCallback(check)
         return d
 
-    def do(self, basedir, command, failureIsOk=False):
-        d = self.runCommand(basedir, command, failureIsOk=failureIsOk)
+    def do(self, basedir, command, failureIsOk=False, stdin=None):
+        d = self.runCommand(basedir, command, failureIsOk=failureIsOk,
+                            stdin=stdin)
         return waitForDeferred(d)
 
-    def dovc(self, basedir, command, failureIsOk=False):
+    def dovc(self, basedir, command, failureIsOk=False, stdin=None):
         """Like do(), but the VC binary will be prepended to COMMAND."""
         command = self.vcexe + " " + command
-        return self.do(basedir, command, failureIsOk)
+        return self.do(basedir, command, failureIsOk, stdin)
 
 class VCBase(SignalMixin):
     metadir = None
@@ -1315,6 +1388,163 @@
 
 VCS.registerVC(SVN.vc_name, SVNHelper())
 
+
+class P4Support(VCBase):
+    metadir = None
+    branchname = "branch"
+    vctype = "step.P4"
+    p4port = 'localhost:1666'
+    pid = None
+    base_descr = 'Change: new\nDescription: asdf\nFiles:\n'
+
+    class _P4DProtocol(protocol.ProcessProtocol):
+        def __init__(self):
+            self.started = defer.Deferred()
+            self.ended = defer.Deferred()
+
+        def outReceived(self, data):
+            # When it says starting, it has bound to the socket.
+            if self.started:
+                if data.startswith('Perforce Server starting...'):
+                    self.started.callback(None)
+                else:
+                    try:
+                        raise Exception('p4d said %r' % data)
+                    except:
+                        self.started.errback(failure.Failure())
+                self.started = None
+
+        def processEnded(self, status_object):
+            if status_object.check(error.ProcessDone):
+                self.ended.callback(None)
+            else:
+                self.ended.errback(status_object)
+
+    def _start_p4d(self):
+        proto = self._P4DProtocol()
+        reactor.spawnProcess(proto, self.p4dexe, ['p4d', '-p', self.p4port],
+                             env=os.environ, path=self.p4rep)
+        return proto.started, proto.ended
+
+    def capable(self):
+        global VCS
+        if not VCS.has_key("p4"):
+            VCS["p4"] = False
+            p4paths = which('p4')
+            p4dpaths = which('p4d')
+            if p4paths and p4dpaths:
+                self.vcexe = p4paths[0]
+                self.p4dexe = p4dpaths[0]
+                VCS["p4"] = True
+        if not VCS["p4"]:
+            raise unittest.SkipTest("No usable Perforce was found")
+
+    def vc_create(self):
+        tmp = os.path.join(self.repbase, "p4tmp")
+        self.p4rep = os.path.join(self.repbase, 'P4-Repository')
+        os.mkdir(self.p4rep)
+
+        # Launch p4d.
+        started, self.p4d_shutdown = self._start_p4d()
+        w = waitForDeferred(started)
+        yield w; w.getResult()
+
+        # Create client spec.
+        os.mkdir(tmp)
+        clispec = 'Client: creator\n'
+        clispec += 'Root: %s\n' % tmp
+        clispec += 'View:\n'
+        clispec += '\t//depot/... //creator/...\n'
+        w = self.dovc(tmp, '-p %s client -i' % self.p4port, stdin=clispec)
+        yield w; w.getResult()
+
+        # Create first rev (trunk).
+        self.populate(os.path.join(tmp, 'trunk'))
+        files = ['main.c', 'version.c', 'subdir/subdir.c']
+        w = self.do(tmp, ['sh', '-c',
+                          "p4 -p %s -c creator add " % self.p4port
+                          + " ".join(['trunk/%s' % f for f in files])])
+        yield w; w.getResult()
+        descr = self.base_descr
+        for file in files:
+            descr += '\t//depot/trunk/%s\n' % file
+        w = self.dovc(tmp, "-p %s -c creator submit -i" % self.p4port,
+                      stdin=descr)
+        yield w; out = w.getResult()
+        m = re.search(r'Change (\d+) submitted.', out)
+        assert m.group(1) == '1'
+        self.addTrunkRev(m.group(1))
+
+        # Create second rev (branch).
+        w = self.dovc(tmp, '-p %s -c creator integrate ' % self.p4port
+                      + '//depot/trunk/... //depot/branch/...')
+        yield w; w.getResult()
+        w = self.do(tmp, ['sh', '-c',
+                          "p4 -p %s -c creator edit branch/main.c"
+                          % self.p4port])
+        yield w; w.getResult()
+        self.populate_branch(os.path.join(tmp, 'branch'))
+        descr = self.base_descr
+        for file in files:
+            descr += '\t//depot/branch/%s\n' % file
+        w = self.dovc(tmp, "-p %s -c creator submit -i" % self.p4port,
+                      stdin=descr)
+        yield w; out = w.getResult()
+        m = re.search(r'Change (\d+) submitted.', out)
+        self.addBranchRev(m.group(1))
+    vc_create = deferredGenerator(vc_create)
+
+    def vc_revise(self):
+        tmp = os.path.join(self.repbase, "p4tmp")
+        self.version += 1
+        version_c = VERSION_C % self.version
+        w = self.do(tmp, ['sh', '-c',
+                          'p4 -p %s -c creator edit trunk/version.c'
+                           % self.p4port])
+        yield w; w.getResult()
+        open(os.path.join(tmp, "trunk/version.c"), "w").write(version_c)
+        descr = self.base_descr + '\t//depot/trunk/version.c\n'
+        w = self.dovc(tmp, "-p %s -c creator submit -i" % self.p4port,
+                      stdin=descr)
+        yield w; out = w.getResult()
+        m = re.search(r'Change (\d+) submitted.', out)
+        self.addTrunkRev(m.group(1))
+    vc_revise = deferredGenerator(vc_revise)
+
+    def setUp2(self, res):
+        if self.p4d_shutdown is None:
+            started, self.p4d_shutdown = self._start_p4d()
+            return started
+
+    def tearDown2(self):
+        self.p4d_shutdown = None
+        d = self.runCommand(self.repbase, '%s -p %s admin stop'
+                            % (self.vcexe, self.p4port))
+        return d.addCallback(lambda _: self.p4d_shutdown)
+
+class P4(P4Support, unittest.TestCase):
+
+    def testCheckout(self):
+        self.vcargs = { 'p4port': self.p4port, 'p4base': '//depot/',
+                        'defaultBranch': 'trunk' }
+        d = self.do_vctest(testRetry=False)
+        # TODO: like arch and darcs, sync does nothing when server is not
+        # changed.
+        return maybeWait(d)
+
+    def testPatch(self):
+        self.vcargs = { 'p4port': self.p4port, 'p4base': '//depot/',
+                        'defaultBranch': 'trunk' }
+        d = self.do_patch()
+        return maybeWait(d)
+
+    def testBranch(self):
+        self.vcargs = { 'p4port': self.p4port, 'p4base': '//depot/',
+                        'defaultBranch': 'trunk' }
+        d = self.do_branch()
+        return maybeWait(d)
+
+
 class DarcsHelper(BaseHelper):
     branchname = "branch"
     try_branchname = "branch"




From warner at users.sourceforge.net  Mon Jun 12 08:36:19 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:19 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.56,1.57
Message-ID: <E1FphuG-0003jC-8i@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17024/buildbot/test

Modified Files:
	test_vc.py 
Log Message:
[project @ test_vc (P4): make it work]

Original author: warner at lothar.com
Date: 2006-06-12 07:50:18

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.56
retrieving revision 1.57
diff -u -d -r1.56 -r1.57
--- test_vc.py	12 Jun 2006 08:36:08 -0000	1.56
+++ test_vc.py	12 Jun 2006 08:36:17 -0000	1.57
@@ -7,34 +7,6 @@
 
 from twisted.trial import unittest
 from twisted.internet import defer, reactor, utils, protocol, error
-try:
-    from twisted.python.procutils import which
-except ImportError:
-    # copied from Twisted circa 2.2.0
-    def which(name, flags=os.X_OK):
-        """Search PATH for executable files with the given name.
-
-        @type name: C{str}
-        @param name: The name for which to search.
-
-        @type flags: C{int}
-        @param flags: Arguments to L{os.access}.
-
-        @rtype: C{list}
-        @param: A list of the full paths to files found, in the
-        order in which they were found.
-        """
-        result = []
-        exts = filter(None, os.environ.get('PATHEXT', '').split(os.pathsep))
-        for p in os.environ['PATH'].split(os.pathsep):
-            p = os.path.join(p, name)
-            if os.access(p, flags):
-                result.append(p)
-            for e in exts:
-                pext = p + e
-                if os.access(pext, flags):
-                    result.append(pext)
-        return result
 
 #defer.Deferred.debug = True
 
@@ -1389,14 +1361,23 @@
 VCS.registerVC(SVN.vc_name, SVNHelper())
 
 
-class P4Support(VCBase):
-    metadir = None
+class P4Helper(BaseHelper):
     branchname = "branch"
-    vctype = "step.P4"
     p4port = 'localhost:1666'
     pid = None
     base_descr = 'Change: new\nDescription: asdf\nFiles:\n'
 
+    def capable(self):
+        p4paths = which('p4')
+        p4dpaths = which('p4d')
+        if not p4paths:
+            return (False, "p4 is not installed")
+        if not p4dpaths:
+            return (False, "p4d is not installed")
+        self.vcexe = p4paths[0]
+        self.p4dexe = p4dpaths[0]
+        return (True, None)
+
     class _P4DProtocol(protocol.ProcessProtocol):
         def __init__(self):
             self.started = defer.Deferred()
@@ -1426,20 +1407,14 @@
                              env=os.environ, path=self.p4rep)
         return proto.started, proto.ended
 
-    def capable(self):
-        global VCS
-        if not VCS.has_key("p4"):
-            VCS["p4"] = False
-            p4paths = which('p4')
-            p4dpaths = which('p4d')
-            if p4paths and p4dpaths:
-                self.vcexe = p4paths[0]
-                self.p4dexe = p4dpaths[0]
-                VCS["p4"] = True
-        if not VCS["p4"]:
-            raise unittest.SkipTest("No usable Perforce was found")
+    def dop4(self, basedir, command, failureIsOk=False, stdin=None):
+        command = "-p " + self.p4port + " " + command
+        return self.dovc(basedir, command, failureIsOk, stdin)
 
-    def vc_create(self):
+    def createRepository(self):
+        # this is only called once per VC system, so start p4d here.
+
+        self.createBasedir()
         tmp = os.path.join(self.repbase, "p4tmp")
         self.p4rep = os.path.join(self.repbase, 'P4-Repository')
         os.mkdir(self.p4rep)
@@ -1455,31 +1430,32 @@
         clispec += 'Root: %s\n' % tmp
         clispec += 'View:\n'
         clispec += '\t//depot/... //creator/...\n'
-        w = self.dovc(tmp, '-p %s client -i' % self.p4port, stdin=clispec)
+        w = self.dop4(tmp, 'client -i', stdin=clispec)
         yield w; w.getResult()
 
         # Create first rev (trunk).
         self.populate(os.path.join(tmp, 'trunk'))
         files = ['main.c', 'version.c', 'subdir/subdir.c']
-        w = self.do(tmp, ['sh', '-c',
+        w = self.do(tmp, ['sh', '-c', # TODO: ???
                           "p4 -p %s -c creator add " % self.p4port
                           + " ".join(['trunk/%s' % f for f in files])])
+        #self.dop4(tmp, "-c creator add "
+        #          + " ".join(['trunk/%s' % f for f in files]))
         yield w; w.getResult()
         descr = self.base_descr
         for file in files:
             descr += '\t//depot/trunk/%s\n' % file
-        w = self.dovc(tmp, "-p %s -c creator submit -i" % self.p4port,
-                      stdin=descr)
+        w = self.dop4(tmp, "-c creator submit -i", stdin=descr)
         yield w; out = w.getResult()
         m = re.search(r'Change (\d+) submitted.', out)
         assert m.group(1) == '1'
         self.addTrunkRev(m.group(1))
 
         # Create second rev (branch).
-        w = self.dovc(tmp, '-p %s -c creator integrate ' % self.p4port
+        w = self.dop4(tmp, '-c creator integrate '
                       + '//depot/trunk/... //depot/branch/...')
         yield w; w.getResult()
-        w = self.do(tmp, ['sh', '-c',
+        w = self.do(tmp, ['sh', '-c', # TODO: again?
                           "p4 -p %s -c creator edit branch/main.c"
                           % self.p4port])
         yield w; w.getResult()
@@ -1487,63 +1463,66 @@
         descr = self.base_descr
         for file in files:
             descr += '\t//depot/branch/%s\n' % file
-        w = self.dovc(tmp, "-p %s -c creator submit -i" % self.p4port,
-                      stdin=descr)
+        w = self.dop4(tmp, "-c creator submit -i", stdin=descr)
         yield w; out = w.getResult()
         m = re.search(r'Change (\d+) submitted.', out)
         self.addBranchRev(m.group(1))
-    vc_create = deferredGenerator(vc_create)
+    createRepository = deferredGenerator(createRepository)
 
     def vc_revise(self):
         tmp = os.path.join(self.repbase, "p4tmp")
         self.version += 1
         version_c = VERSION_C % self.version
-        w = self.do(tmp, ['sh', '-c',
+        w = self.do(tmp, ['sh', '-c', # TODO
                           'p4 -p %s -c creator edit trunk/version.c'
                            % self.p4port])
         yield w; w.getResult()
         open(os.path.join(tmp, "trunk/version.c"), "w").write(version_c)
         descr = self.base_descr + '\t//depot/trunk/version.c\n'
-        w = self.dovc(tmp, "-p %s -c creator submit -i" % self.p4port,
-                      stdin=descr)
+        w = self.dop4(tmp, "-c creator submit -i", stdin=descr)
         yield w; out = w.getResult()
         m = re.search(r'Change (\d+) submitted.', out)
         self.addTrunkRev(m.group(1))
     vc_revise = deferredGenerator(vc_revise)
 
-    def setUp2(self, res):
-        if self.p4d_shutdown is None:
-            started, self.p4d_shutdown = self._start_p4d()
-            return started
-
-    def tearDown2(self):
-        self.p4d_shutdown = None
+    def shutdown_p4d(self):
         d = self.runCommand(self.repbase, '%s -p %s admin stop'
                             % (self.vcexe, self.p4port))
         return d.addCallback(lambda _: self.p4d_shutdown)
 
-class P4(P4Support, unittest.TestCase):
+class P4(VCBase, unittest.TestCase):
+    metadir = None
+    vctype = "step.P4"
+    vc_name = "p4"
+
+    def tearDownClass(self):
+        return maybeWait(self.helper.shutdown_p4d())
 
     def testCheckout(self):
-        self.vcargs = { 'p4port': self.p4port, 'p4base': '//depot/',
-                        'defaultBranch': 'trunk' }
+        self.helper.vcargs = { 'p4port': self.helper.p4port,
+                               'p4base': '//depot/',
+                               'defaultBranch': 'trunk' }
         d = self.do_vctest(testRetry=False)
         # TODO: like arch and darcs, sync does nothing when server is not
         # changed.
         return maybeWait(d)
 
     def testPatch(self):
-        self.vcargs = { 'p4port': self.p4port, 'p4base': '//depot/',
-                        'defaultBranch': 'trunk' }
+        self.helper.vcargs = { 'p4port': self.helper.p4port,
+                               'p4base': '//depot/',
+                               'defaultBranch': 'trunk' }
         d = self.do_patch()
         return maybeWait(d)
 
     def testBranch(self):
-        self.vcargs = { 'p4port': self.p4port, 'p4base': '//depot/',
-                        'defaultBranch': 'trunk' }
+        self.helper.vcargs = { 'p4port': self.helper.p4port,
+                               'p4base': '//depot/',
+                               'defaultBranch': 'trunk' }
         d = self.do_branch()
         return maybeWait(d)
 
+VCS.registerVC(P4.vc_name, P4Helper())
+
 
 class DarcsHelper(BaseHelper):
     branchname = "branch"




From warner at users.sourceforge.net  Mon Jun 12 08:36:24 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:24 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.57,1.58
Message-ID: <E1FphuN-0005uy-Ow@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17041/buildbot/test

Modified Files:
	test_vc.py 
Log Message:
[project @ test_vc.py: conditionalize some debug prints]

Original author: warner at lothar.com
Date: 2006-06-12 08:17:00

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.57
retrieving revision 1.58
diff -u -d -r1.57 -r1.58
--- test_vc.py	12 Jun 2006 08:36:17 -0000	1.57
+++ test_vc.py	12 Jun 2006 08:36:22 -0000	1.58
@@ -363,17 +363,24 @@
         # specify.
         if type(command) not in (list, tuple):
             command = command.split(" ")
-        #print "do %s" % command
+        DEBUG = False
+        if DEBUG:
+            print "do %s" % command
+            print " in basedir %s" % basedir
+            if stdin:
+                print " STDIN:\n", stdin, "\n--STDIN DONE"
         env = os.environ.copy()
         env['LC_ALL'] = "C"
         d = myGetProcessOutputAndValue(command[0], command[1:],
                                        env=env, path=basedir,
                                        stdin=stdin)
         def check((out, err, code)):
-            #print
-            #print "command: %s" % command
-            #print "out: %s" % out
-            #print "code: %s" % code
+            if DEBUG:
+                print
+                print "command was: %s" % command
+                if out: print "out: %s" % out
+                if err: print "err: %s" % err
+                print "code: %s" % code
             if code != 0 and not failureIsOk:
                 log.msg("command %s finished with exit code %d" %
                         (command, code))




From warner at users.sourceforge.net  Mon Jun 12 08:36:29 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:29 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.58,1.59
Message-ID: <E1FphuQ-0003q9-2Z@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17071/buildbot/test

Modified Files:
	test_vc.py 
Log Message:
[project @ test_vc.py (P4): avoid use of 'sh -c' by passing '-d' to p4 so it won't use PWD]

Original author: warner at lothar.com
Date: 2006-06-12 08:18:17

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.58
retrieving revision 1.59
diff -u -d -r1.58 -r1.59
--- test_vc.py	12 Jun 2006 08:36:22 -0000	1.58
+++ test_vc.py	12 Jun 2006 08:36:27 -0000	1.59
@@ -1415,7 +1415,10 @@
         return proto.started, proto.ended
 
     def dop4(self, basedir, command, failureIsOk=False, stdin=None):
-        command = "-p " + self.p4port + " " + command
+        # p4 looks at $PWD instead of getcwd(), which causes confusion when
+        # we spawn commands without an intervening shell (sh -c). We can
+        # override this with a -d argument.
+        command = "-p %s -d %s %s" % (self.p4port, basedir, command)
         return self.dovc(basedir, command, failureIsOk, stdin)
 
     def createRepository(self):
@@ -1443,11 +1446,8 @@
         # Create first rev (trunk).
         self.populate(os.path.join(tmp, 'trunk'))
         files = ['main.c', 'version.c', 'subdir/subdir.c']
-        w = self.do(tmp, ['sh', '-c', # TODO: ???
-                          "p4 -p %s -c creator add " % self.p4port
-                          + " ".join(['trunk/%s' % f for f in files])])
-        #self.dop4(tmp, "-c creator add "
-        #          + " ".join(['trunk/%s' % f for f in files]))
+        w = self.dop4(tmp, "-c creator add "
+                      + " ".join(['trunk/%s' % f for f in files]))
         yield w; w.getResult()
         descr = self.base_descr
         for file in files:
@@ -1462,9 +1462,7 @@
         w = self.dop4(tmp, '-c creator integrate '
                       + '//depot/trunk/... //depot/branch/...')
         yield w; w.getResult()
-        w = self.do(tmp, ['sh', '-c', # TODO: again?
-                          "p4 -p %s -c creator edit branch/main.c"
-                          % self.p4port])
+        w = self.dop4(tmp, "-c creator edit branch/main.c")
         yield w; w.getResult()
         self.populate_branch(os.path.join(tmp, 'branch'))
         descr = self.base_descr
@@ -1480,9 +1478,7 @@
         tmp = os.path.join(self.repbase, "p4tmp")
         self.version += 1
         version_c = VERSION_C % self.version
-        w = self.do(tmp, ['sh', '-c', # TODO
-                          'p4 -p %s -c creator edit trunk/version.c'
-                           % self.p4port])
+        w = self.dop4(tmp, '-c creator edit trunk/version.c')
         yield w; w.getResult()
         open(os.path.join(tmp, "trunk/version.c"), "w").write(version_c)
         descr = self.base_descr + '\t//depot/trunk/version.c\n'




From warner at users.sourceforge.net  Mon Jun 12 08:36:34 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:34 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.59,1.60
Message-ID: <E1FphuV-000627-FU@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17088/buildbot/test

Modified Files:
	test_vc.py 
Log Message:
[project @ test_vc.py (P4): print something useful when we can't start the p4d server]

Original author: warner at lothar.com
Date: 2006-06-12 08:18:39

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.59
retrieving revision 1.60
diff -u -d -r1.59 -r1.60
--- test_vc.py	12 Jun 2006 08:36:27 -0000	1.59
+++ test_vc.py	12 Jun 2006 08:36:32 -0000	1.60
@@ -1396,12 +1396,16 @@
                 if data.startswith('Perforce Server starting...'):
                     self.started.callback(None)
                 else:
+                    print "p4d said %r" % data
                     try:
                         raise Exception('p4d said %r' % data)
                     except:
                         self.started.errback(failure.Failure())
                 self.started = None
 
+        def errReceived(self, data):
+            print "p4d stderr: %s" % data
+
         def processEnded(self, status_object):
             if status_object.check(error.ProcessDone):
                 self.ended.callback(None)




From warner at users.sourceforge.net  Mon Jun 12 08:36:41 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:41 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.651,1.652
Message-ID: <E1Fphub-00067H-Bt@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17109

Modified Files:
	ChangeLog 
Log Message:
[project @ ChangeLog: add entry for Perforce patch]

Original author: warner at lothar.com
Date: 2006-06-12 08:28:22

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.651
retrieving revision 1.652
diff -u -d -r1.651 -r1.652
--- ChangeLog	12 Jun 2006 08:35:53 -0000	1.651
+++ ChangeLog	12 Jun 2006 08:36:38 -0000	1.652
@@ -1,5 +1,25 @@
 2006-06-12  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step.py (P4): merge in patch SF#1473939, adding
+	proper Perforce (P4) support. Many many thanks to Scott Lamb for
+	contributing such an excellent patch, including docs and unit
+	tests! This makes it *so* much easier to apply. I had to update
+	test_vc.py to handle some recent refactorings, but everything else
+	applied smoothly. The only remaining thing I'd like to fix would
+	be to remove the hard-wired port 1666 used by p4d, and allow it to
+	claim any unused port. This would allow two copies of the test
+	suite to run on the same host at the same time, as well as
+	allowing the test suite to run while a real (production) p4d was
+	running on the same host. Oh, and maybe we should add a warning to
+	step.P4 that gets emitted if the slave is too old to provide the
+	'p4' SlaveCommand. Otherwise it looks great. (closes: SF#1473939).
+	* buildbot/slave/commands.py (P4): same
+	(P4Sync): same, some minor updates
+	* buildbot/changes/p4poller.py: same
+	* docs/buildbot.texinfo: same
+	* buildbot/test/test_p4poller.py: same
+	* buildbot/test/test_vc.py (P4): same
+
 	* setup.py: add Trove classifiers for PyPI
 
 2006-06-08  Brian Warner  <warner at allmydata.com>




From warner at users.sourceforge.net  Mon Jun 12 08:36:46 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Mon, 12 Jun 2006 08:36:46 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.60,1.61
Message-ID: <E1Fphug-0006BE-GR@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv17120/buildbot/test

Modified Files:
	test_vc.py 
Log Message:
[project @ test_vc (P4): skip test properly if p4 is not installed]

Original author: warner at lothar.com
Date: 2006-06-12 08:34:13

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.60
retrieving revision 1.61
diff -u -d -r1.60 -r1.61
--- test_vc.py	12 Jun 2006 08:36:32 -0000	1.60
+++ test_vc.py	12 Jun 2006 08:36:44 -0000	1.61
@@ -408,6 +408,7 @@
     createdRepository = False
     master = None
     slave = None
+    helper = None
     httpServer = None
     httpPort = None
     skip = None
@@ -1503,7 +1504,8 @@
     vc_name = "p4"
 
     def tearDownClass(self):
-        return maybeWait(self.helper.shutdown_p4d())
+        if self.helper:
+            return maybeWait(self.helper.shutdown_p4d())
 
     def testCheckout(self):
         self.helper.vcargs = { 'p4port': self.helper.p4port,




From warner at users.sourceforge.net  Wed Jun 14 06:49:21 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 14 Jun 2006 06:49:21 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.652,1.653
Message-ID: <E1FqPBs-0008LI-JT@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16095

Modified Files:
	ChangeLog 
Log Message:
[project @ test_p4poller: fix python2.2 compatibility]

Original author: warner at lothar.com
Date: 2006-06-14 06:47:51

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.652
retrieving revision 1.653
diff -u -d -r1.652 -r1.653
--- ChangeLog	12 Jun 2006 08:36:38 -0000	1.652
+++ ChangeLog	14 Jun 2006 06:49:19 -0000	1.653
@@ -1,3 +1,17 @@
+2006-06-13  Brian Warner  <warner at lothar.com>
+
+	* buildbot/test/test_p4poller.py (TestP4Poller.failUnlessIn): fix
+	compatibility with python2.2, which doesn't have the 'substr in
+	str' feature.
+	(TestP4Poller.makeTime): utility function to construct the
+	timestamp using the same strptime() approach as p4poller does. It
+	turns out that time.mktime() behaves slightly differently under
+	python2.2, probably something to do with the DST flag, and that
+	causes the test to fail under python2.2. (changing the mktime()
+	arguments to have dst=0 instead of -1 caused it to fail under
+	python2.3. Go figure.)
+	(TestP4Poller._testCheck3): use our makeTime() instead of mktime()
+
 2006-06-12  Brian Warner  <warner at lothar.com>
 
 	* buildbot/process/step.py (P4): merge in patch SF#1473939, adding




From warner at users.sourceforge.net  Wed Jun 14 06:49:21 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 14 Jun 2006 06:49:21 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_p4poller.py, 1.1, 1.2
Message-ID: <E1FqPBs-0008LG-Jw@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv16095/buildbot/test

Modified Files:
	test_p4poller.py 
Log Message:
[project @ test_p4poller: fix python2.2 compatibility]

Original author: warner at lothar.com
Date: 2006-06-14 06:47:51

Index: test_p4poller.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_p4poller.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -d -r1.1 -r1.2
--- test_p4poller.py	12 Jun 2006 08:36:08 -0000	1.1
+++ test_p4poller.py	14 Jun 2006 06:49:19 -0000	1.2
@@ -89,6 +89,13 @@
         self.changes = []
         self.addChange = self.changes.append
 
+    def failUnlessIn(self, substr, string):
+        # this is for compatibility with python2.2
+        if isinstance(string, str):
+            self.failUnless(string.find(substr) != -1)
+        else:
+            self.assertIn(substr, string)
+
     def testCheck(self):
         """successful checks"""
         self.t = MockP4Source(p4changes=[first_p4changes, second_p4changes],
@@ -121,29 +128,34 @@
                    files=['whatbranch'],
                    comments=change_2_log,
                    revision='2',
-                   when=time.mktime((2006, 4, 13, 21, 46, 23, 3, 103, -1)),
+                   when=self.makeTime("2006/04/13 21:46:23"),
                    branch='trunk').asText())
 
         # These two can happen in either order, since they're from the same
         # Perforce change.
-        self.assertIn(
+        self.failUnlessIn(
             Change(who='bob',
                    files=['branch_b_file',
                           'whatbranch'],
                    comments=change_3_log,
                    revision='3',
-                   when=time.mktime((2006, 4, 13, 21, 51, 39, 3, 103, -1)),
+                   when=self.makeTime("2006/04/13 21:51:39"),
                    branch='branch_b').asText(),
             [c.asText() for c in self.changes])
-        self.assertIn(
+        self.failUnlessIn(
             Change(who='bob',
                    files=['whatbranch'],
                    comments=change_3_log,
                    revision='3',
-                   when=time.mktime((2006, 4, 13, 21, 51, 39, 3, 103, -1)),
+                   when=self.makeTime("2006/04/13 21:51:39"),
                    branch='branch_c').asText(),
             [c.asText() for c in self.changes])
 
+    def makeTime(self, timestring):
+        datefmt = '%Y/%m/%d %H:%M:%S'
+        when = time.mktime(time.strptime(timestring, datefmt))
+        return when
+
     def testFailedChanges(self):
         """'p4 changes' failure is properly reported"""
         self.t = MockP4Source(p4changes=['Perforce client error:\n...'],
@@ -156,7 +168,7 @@
 
     def _testFailedChanges2(self, f):
         self.assert_(isinstance(f, failure.Failure))
-        self.assertIn('Perforce client error', str(f))
+        self.failUnlessIn('Perforce client error', str(f))
         self.assert_(not self.t.working)
 
     def testFailedDescribe(self):
@@ -176,7 +188,7 @@
 
     def _testFailedDescribe3(self, f):
         self.assert_(isinstance(f, failure.Failure))
-        self.assertIn('Perforce client error', str(f))
+        self.failUnlessIn('Perforce client error', str(f))
         self.assert_(not self.t.working)
         self.assertEquals(self.t.last_change, '2')
 




From warner at users.sourceforge.net  Thu Jun 15 05:46:58 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:46:58 +0000
Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.41,1.42
Message-ID: <E1Fqkh1-0004qP-0F@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30161/buildbot

Modified Files:
	interfaces.py 
Log Message:
[project @ move STDOUT/STDERR channel constants to buildbot.interfaces]

Original author: warner at lothar.com
Date: 2006-06-15 01:05:18

Index: interfaces.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v
retrieving revision 1.41
retrieving revision 1.42
diff -u -d -r1.41 -r1.42
--- interfaces.py	22 May 2006 08:16:25 -0000	1.41
+++ interfaces.py	15 Jun 2006 05:46:56 -0000	1.42
@@ -664,6 +664,10 @@
         0 for stdout, 1 for stderr, 2 for header. (note that stderr is merged
         into stdout if PTYs are in use)."""
 
+LOG_CHANNEL_STDOUT = 0
+LOG_CHANNEL_STDERR = 1
+LOG_CHANNEL_HEADER = 2
+
 class IStatusLogConsumer(Interface):
     """I am an object which can be passed to IStatusLog.subscribeConsumer().
     I represent a target for writing the contents of an IStatusLog. This
@@ -771,7 +775,8 @@
         log finishes."""
 
     def logChunk(build, step, log, channel, text):
-        """Some text has been added to this log. 'channel' is 0, 1, or 2, as
+        """Some text has been added to this log. 'channel' is one of
+        LOG_CHANNEL_STDOUT, LOG_CHANNEL_STDERR, or LOG_CHANNEL_HEADER, as
         defined in IStatusLog.getChunks."""
 
     def logFinished(build, step, log):




From warner at users.sourceforge.net  Thu Jun 15 05:46:58 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:46:58 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.79,1.80
Message-ID: <E1Fqkh1-00023c-8Z@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30161/buildbot/status

Modified Files:
	builder.py 
Log Message:
[project @ move STDOUT/STDERR channel constants to buildbot.interfaces]

Original author: warner at lothar.com
Date: 2006-06-15 01:05:18

Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v
retrieving revision 1.79
retrieving revision 1.80
diff -u -d -r1.79 -r1.80
--- builder.py	24 Apr 2006 06:45:37 -0000	1.79
+++ builder.py	15 Jun 2006 05:46:56 -0000	1.80
@@ -39,9 +39,9 @@
 #  startBuild
 #  finishBuild
 
-STDOUT = 0
-STDERR = 1
-HEADER = 2
+STDOUT = interfaces.LOG_CHANNEL_STDOUT
+STDERR = interfaces.LOG_CHANNEL_STDERR
+HEADER = interfaces.LOG_CHANNEL_HEADER
 ChunkTypes = ["stdout", "stderr", "header"]
 
 class LogFileScanner(basic.NetstringReceiver):




From warner at users.sourceforge.net  Thu Jun 15 05:46:58 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:46:58 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.653,1.654
Message-ID: <E1Fqkh1-0004qN-Ip@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30161

Modified Files:
	ChangeLog 
Log Message:
[project @ move STDOUT/STDERR channel constants to buildbot.interfaces]

Original author: warner at lothar.com
Date: 2006-06-15 01:05:18

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.653
retrieving revision 1.654
diff -u -d -r1.653 -r1.654
--- ChangeLog	14 Jun 2006 06:49:19 -0000	1.653
+++ ChangeLog	15 Jun 2006 05:46:55 -0000	1.654
@@ -1,3 +1,9 @@
+2006-06-14  Brian Warner  <warner at lothar.com>
+
+	* buildbot/interfaces.py (LOG_CHANNEL_*): move STDOUT / STDERR /
+	HEADER constants here ..
+	* buildbot/status/builder.py (STDOUT): .. from here
+
 2006-06-13  Brian Warner  <warner at lothar.com>
 
 	* buildbot/test/test_p4poller.py (TestP4Poller.failUnlessIn): fix




From warner at users.sourceforge.net  Thu Jun 15 05:47:16 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:16 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.80,1.81
Message-ID: <E1FqkhI-0002Am-Ms@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30613/buildbot/status

Modified Files:
	builder.py 
Log Message:
[project @ add ILogFile/ILogObserver]

Original author: warner at lothar.com
Date: 2006-06-15 01:10:55

Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v
retrieving revision 1.80
retrieving revision 1.81
diff -u -d -r1.80 -r1.81
--- builder.py	15 Jun 2006 05:46:56 -0000	1.80
+++ builder.py	15 Jun 2006 05:47:14 -0000	1.81
@@ -201,9 +201,9 @@
     logs."""
 
     if implements:
-        implements(interfaces.IStatusLog)
+        implements(interfaces.IStatusLog, interfaces.ILogFile)
     else:
-        __implements__ = interfaces.IStatusLog,
+        __implements__ = (interfaces.IStatusLog, interfaces.ILogFile)
 
     finished = False
     length = 0




From warner at users.sourceforge.net  Thu Jun 15 05:47:15 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:15 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.654,1.655
Message-ID: <E1FqkhI-0004wg-LG@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30613

Modified Files:
	ChangeLog 
Log Message:
[project @ add ILogFile/ILogObserver]

Original author: warner at lothar.com
Date: 2006-06-15 01:10:55

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.654
retrieving revision 1.655
diff -u -d -r1.654 -r1.655
--- ChangeLog	15 Jun 2006 05:46:55 -0000	1.654
+++ ChangeLog	15 Jun 2006 05:47:13 -0000	1.655
@@ -1,5 +1,10 @@
 2006-06-14  Brian Warner  <warner at lothar.com>
 
+	* buildbot/interfaces.py (ILogFile): add the Interface used from
+	the BuildStep towards the LogFile
+	(ILogObserver): and the one provided by a LogObserver
+	* buildbot/status/builder.py (LogFile): implement it
+
 	* buildbot/interfaces.py (LOG_CHANNEL_*): move STDOUT / STDERR /
 	HEADER constants here ..
 	* buildbot/status/builder.py (STDOUT): .. from here




From warner at users.sourceforge.net  Thu Jun 15 05:47:16 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:16 +0000
Subject: [Buildbot-commits] buildbot/buildbot interfaces.py,1.42,1.43
Message-ID: <E1FqkhL-0004xG-Nr@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30613/buildbot

Modified Files:
	interfaces.py 
Log Message:
[project @ add ILogFile/ILogObserver]

Original author: warner at lothar.com
Date: 2006-06-15 01:10:55

Index: interfaces.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v
retrieving revision 1.42
retrieving revision 1.43
diff -u -d -r1.42 -r1.43
--- interfaces.py	15 Jun 2006 05:46:56 -0000	1.42
+++ interfaces.py	15 Jun 2006 05:47:13 -0000	1.43
@@ -893,3 +893,33 @@
     def stopBuild(reason="<no reason given>"):
         """Halt the build. This has no effect if the build has already
         finished."""
+
+class ILogFile(Interface):
+    """This is the internal interface to a LogFile, used by the BuildStep to
+    write data into the log.
+    """
+    def addStdout(data):
+        pass
+    def addStderr(data):
+        pass
+    def addHeader(data):
+        pass
+    def finish():
+        """The process that is feeding the log file has finished, and no
+        further data will be added. This closes the logfile."""
+
+class ILogObserver(Interface):
+    """Objects which provide this interface can be used in a BuildStep to
+    watch the output of a LogFile and parse it incrementally.
+    """
+
+    # internal methods
+    def setStep(step):
+        pass
+    def setLog(log):
+        pass
+
+    # methods called by the LogFile
+    def logChunk(build, step, log, channel, text):
+        pass
+




From warner at users.sourceforge.net  Thu Jun 15 05:47:37 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:37 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.81,1.82
Message-ID: <E1Fqkhe-0004zP-7K@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30693/buildbot/status

Modified Files:
	builder.py 
Log Message:
[project @ prepare for multiple LogFiles, add LogObservers]

Original author: warner at lothar.com
Date: 2006-06-15 01:23:23

Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v
retrieving revision 1.81
retrieving revision 1.82
diff -u -d -r1.81 -r1.82
--- builder.py	15 Jun 2006 05:47:14 -0000	1.81
+++ builder.py	15 Jun 2006 05:47:35 -0000	1.82
@@ -207,7 +207,6 @@
 
     finished = False
     length = 0
-    progress = None
     chunkSize = 10*1000
     runLength = 0
     runEntries = [] # provided so old pickled builds will getChunks() ok
@@ -353,9 +352,6 @@
         p.resumeProducing()
 
     # interface used by the build steps to add things to the log
-    def logProgressTo(self, progress, name):
-        self.progress = progress
-        self.progressName = name
 
     def merge(self):
         # merge all .runEntries (which are all of the same type) into a
@@ -391,8 +387,6 @@
         for w in self.watchers:
             w.logChunk(self.step.build, self.step, self, channel, text)
         self.length += len(text)
-        if self.progress:
-            self.progress.setProgress(self.progressName, self.length)
 
     def addStdout(self, text):
         self.addEntry(STDOUT, text)
@@ -416,10 +410,6 @@
         self.finishedWatchers = []
         for w in watchers:
             w.callback(self)
-        if self.progress:
-            self.progress.setProgress(self.progressName, self.length)
-            del self.progress
-            del self.progressName
 
     # persistence stuff
     def __getstate__(self):
@@ -430,9 +420,6 @@
         d['entries'] = [] # let 0.6.4 tolerate the saved log. TODO: really?
         if d.has_key('finished'):
             del d['finished']
-        if d.has_key('progress'):
-            del d['progress']
-            del d['progressName']
         if d.has_key('openfile'):
             del d['openfile']
         return d




From warner at users.sourceforge.net  Thu Jun 15 05:47:37 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:37 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.655,1.656
Message-ID: <E1Fqkhe-0002DS-Bk@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30693

Modified Files:
	ChangeLog 
Log Message:
[project @ prepare for multiple LogFiles, add LogObservers]

Original author: warner at lothar.com
Date: 2006-06-15 01:23:23

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.655
retrieving revision 1.656
diff -u -d -r1.655 -r1.656
--- ChangeLog	15 Jun 2006 05:47:13 -0000	1.655
+++ ChangeLog	15 Jun 2006 05:47:35 -0000	1.656
@@ -1,5 +1,36 @@
 2006-06-14  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step.py (LoggedRemoteCommand): use a dict of
+	LogFiles, instead of just a single one. The old single logfile is
+	now called "stdio". LoggedRemoteCommand no longer creates a
+	LogFile for you (the code to do that was broken anyway). If you
+	don't create a "stdio" LogFile, then stdout/stderr will be
+	discarded.
+	(LogObserver): implement "LogObservers", which a BuildStep can add
+	to parse the output of a command in real-time. The primary use is
+	to provide more useful information to the Progress code, allowing
+	better ETA estimates.
+	(LogLineObserver): utility subclass which feeds complete lines to
+	the parser instead of bytes.
+	(BuildStep.progressMetrics): this is safer as a tuple
+	(BuildStep.setProgress): utility method, meant to be called by
+	LogObservers
+	(BuildStep.addLogObserver): new method, to be called at any time
+	during the BuildStep (even before any LogFiles have been created),
+	to attach (or schedule for eventual attachment) a LogObserver to a
+	LogFile.
+	(StdioProgressObserver): new LogObserver which replaces the old
+	"output" progress gatherer
+	(LoggingBuildStep.__init__): same
+	(LoggingBuildStep.startCommand): set up the "stdio" LogFile
+	(LoggingBuildStep._commandComplete): must use logs['stdio']
+	instead of the old single ".log" attribute.
+	* buildbot/status/builder.py (LogFile): remove old logProgressTo
+	functionality, now subsumed into StdioProgressObserver
+	* buildbot/test/test_status.py (Subscription._testSlave_2): the
+	log name changed from "output" to "stdio".
+
+
 	* buildbot/interfaces.py (ILogFile): add the Interface used from
 	the BuildStep towards the LogFile
 	(ILogObserver): and the one provided by a LogObserver




From warner at users.sourceforge.net  Thu Jun 15 05:47:37 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:37 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_status.py, 1.30, 1.31
Message-ID: <E1Fqkhe-0004zW-FS@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30693/buildbot/test

Modified Files:
	test_status.py 
Log Message:
[project @ prepare for multiple LogFiles, add LogObservers]

Original author: warner at lothar.com
Date: 2006-06-15 01:23:23

Index: test_status.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_status.py,v
retrieving revision 1.30
retrieving revision 1.31
diff -u -d -r1.30 -r1.31
--- test_status.py	13 Mar 2006 08:21:42 -0000	1.30
+++ test_status.py	15 Jun 2006 05:47:35 -0000	1.31
@@ -915,7 +915,7 @@
         self.failUnlessEqual(st2.getExpectations(), [('output', 38, None)])
         logs = st2.getLogs()
         self.failUnlessEqual(len(logs), 1)
-        self.failUnlessEqual(logs[0].getName(), "log")
+        self.failUnlessEqual(logs[0].getName(), "stdio")
         self.failUnlessEqual(logs[0].getText(), "data")
 
         self.eta = eta




From warner at users.sourceforge.net  Thu Jun 15 05:47:37 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:37 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.89,1.90
Message-ID: <E1Fqkhe-0002DW-JQ@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30693/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ prepare for multiple LogFiles, add LogObservers]

Original author: warner at lothar.com
Date: 2006-06-15 01:23:23

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.89
retrieving revision 1.90
diff -u -d -r1.89 -r1.90
--- step.py	12 Jun 2006 08:36:08 -0000	1.89
+++ step.py	15 Jun 2006 05:47:35 -0000	1.90
@@ -4,11 +4,14 @@
 from email.Utils import formatdate
 
 from twisted.internet import reactor, defer, error
+from twisted.protocols import basic
 from twisted.spread import pb
 from twisted.python import log
 from twisted.python.failure import Failure
 from twisted.web.util import formatFailure
 
+from buildbot import interfaces
+from buildbot.twcompat import implements, providedBy
 from buildbot import util
 from buildbot.interfaces import BuildSlaveTooOldError
 from buildbot.util import now
@@ -44,7 +47,7 @@
     @cvar  commandCounter: provides a unique value for each
                            RemoteCommand executed across all slaves
     @type  active:         boolean
-    @cvar  active:         whether the command is currently running
+    @ivar  active:         whether the command is currently running
     """
     commandCounter = [0] # we use a list as a poor man's singleton
     active = False
@@ -229,41 +232,66 @@
 
 class LoggedRemoteCommand(RemoteCommand):
     """
-    I am a L{RemoteCommand} which expects the slave to send back
-    stdout/stderr/rc updates. I gather these updates into a
-    L{buildbot.status.builder.LogFile} named C{self.log}. You can give me a
-    LogFile to use by calling useLog(), or I will create my own when the
-    command is started. Unless you tell me otherwise, I will close the log
-    when the command is complete.
+
+    I am a L{RemoteCommand} which gathers output from the remote command into
+    one or more local log files. These L{buildbot.status.builder.Logfile}
+    instances live in C{self.logs}. If the slave sends back
+    stdout/stderr/header updates, these will be put into
+    C{self.logs['stdio']}, if present. If the remote command uses other log
+    channels, they will go into other entries in C{self.logs}.
+
+    If you want to use stdout, you should create a LogFile named 'stdio' and
+    pass it to my useLog() message. Otherwise stdout/stderr will be ignored,
+    which is probably not what you want.
+
+    Unless you tell me otherwise, I will close all logs when the command is
+    complete.
+
+    @ivar logs: maps logname to a LogFile instance
+    @ivar _closeWhenFinished: maps logname to a boolean. If true, this
+                              LogFile will be closed when the RemoteCommand
+                              finishes. LogFiles which are shared between
+                              multiple RemoteCommands should use False here.
+
     """
 
-    log = None
-    closeWhenFinished = False
     rc = None
     debug = False
 
+    def __init__(self, *args, **kwargs):
+        self.logs = {}
+        self._closeWhenFinished = {}
+        RemoteCommand.__init__(self, *args, **kwargs)
+
     def __repr__(self):
         return "<RemoteCommand '%s' at %d>" % (self.remote_command, id(self))
 
     def useLog(self, loog, closeWhenFinished=False):
-        self.log = loog
-        self.closeWhenFinished = closeWhenFinished
+        assert providedBy(loog, interfaces.ILogFile)
+        name = loog.getName()
+        assert name not in self.logs
+        self.logs[name] = loog
+        self._closeWhenFinished[name] = closeWhenFinished
 
     def start(self):
-        if self.log is None:
-            # orphan LogFile, cannot be subscribed to
-            self.log = builder.LogFile(None)
-            self.closeWhenFinished = True
+        log.msg("LoggedRemoteCommand.start")
+        if 'stdio' not in self.logs:
+            log.msg("LoggedRemoteCommand (%s) is running a command, but "
+                    "it isn't being logged to anything. This seems unusual."
+                    % self)
         self.updates = {}
-        log.msg("LoggedRemoteCommand.start", self.log)
         return RemoteCommand.start(self)
 
     def addStdout(self, data):
-        self.log.addStdout(data)
+        if 'stdio' in self.logs:
+            self.logs['stdio'].addStdout(data)
     def addStderr(self, data):
-        self.log.addStderr(data)
+        if 'stdio' in self.logs:
+            self.logs['stdio'].addStderr(data)
     def addHeader(self, data):
-        self.log.addHeader(data)
+        if 'stdio' in self.logs:
+            self.logs['stdio'].addHeader(data)
+
     def remoteUpdate(self, update):
         if self.debug:
             for k,v in update.items():
@@ -278,6 +306,7 @@
             rc = self.rc = update['rc']
             log.msg("%s rc=%s" % (self, rc))
             self.addHeader("program finished with exit code %d\n" % rc)
+        # TODO: other log channels
         for k in update:
             if k not in ('stdout', 'stderr', 'header', 'rc'):
                 if k not in self.updates:
@@ -285,19 +314,72 @@
                 self.updates[k].append(update[k])
 
     def remoteComplete(self, maybeFailure):
-        if self.closeWhenFinished:
-            if maybeFailure:
-                self.addHeader("\nremoteFailed: %s" % maybeFailure)
-            else:
-                log.msg("closing log")
-            self.log.finish()
+        for name,loog in self.logs.items():
+            if self._closeWhenFinished[name]:
+                if maybeFailure:
+                    loog.addHeader("\nremoteFailed: %s" % maybeFailure)
+                else:
+                    log.msg("closing log %s" % loog)
+                loog.finish()
         return maybeFailure
 
+
+class LogObserver:
+    if implements:
+        implements(interfaces.ILogObserver)
+    else:
+        __implements__ = interfaces.ILogObserver,
+
+    def setStep(self, step):
+        self.step = step
+
+    def setLog(self, loog):
+        assert providedBy(loog, interfaces.IStatusLog)
+        loog.subscribe(self, True)
+
+    def logChunk(self, build, step, log, channel, text):
+        if channel == interfaces.LOG_CHANNEL_STDOUT:
+            self.outReceived(text)
+        elif channel == interfaces.LOG_CHANNEL_STDERR:
+            self.errReceived(text)
+
+    # TODO: add a logEnded method? er, stepFinished?
+
+class LogLineObserver(LogObserver):
+    def __init__(self):
+        self.stdoutParser = basic.LineOnlyReceiver()
+        self.stdoutParser.delimiter = "\n"
+        self.stdoutParser.lineReceived = self.outLineReceived
+        self.stdoutParser.transport = self # for the .disconnecting attribute
+        self.disconnecting = False
+
+        self.stderrParser = basic.LineOnlyReceiver()
+        self.stderrParser.delimiter = "\n"
+        self.stderrParser.lineReceived = self.errLineReceived
+        self.stderrParser.transport = self
+
+    def outReceived(self, data):
+        self.stdoutParser.dataReceived(data)
+
+    def errReceived(self, data):
+        self.stderrParser.dataReceived(data)
+
+    def outLineReceived(self, line):
+        """This will be called with complete stdout lines (not including the
+        delimiter). Override this in your observer."""
+        pass
+
+    def errLineReceived(self, line):
+        """This will be called with complete lines of stderr (not including
+        the delimiter). Override this in your observer."""
+        pass
+
+
 class RemoteShellCommand(LoggedRemoteCommand):
     """This class helps you run a shell command on the build slave. It will
-    accumulate all the command's output into a Log. When the command is
-    finished, it will fire a Deferred. You can then check the results of the
-    command and parse the output however you like."""
+    accumulate all the command's output into a Log named 'stdio'. When the
+    command is finished, it will fire a Deferred. You can then check the
+    results of the command and parse the output however you like."""
 
     def __init__(self, workdir, command, env=None, 
                  want_stdout=1, want_stderr=1,
@@ -434,7 +516,7 @@
 
     name = "generic"
     locks = []
-    progressMetrics = [] # 'time' is implicit
+    progressMetrics = () # 'time' is implicit
     useProgress = True # set to False if step is really unpredictable
     build = None
     step_status = None
@@ -455,6 +537,7 @@
             why = "%s.__init__ got unexpected keyword argument(s) %s" \
                   % (self, kwargs.keys())
             raise TypeError(why)
+        self._pendingLogObservers = []
 
     def setupProgress(self):
         if self.useProgress:
@@ -464,6 +547,12 @@
             return sp
         return None
 
+    def setProgress(self, metric, value):
+        """BuildSteps can call self.setProgress() to announce progress along
+        some metric."""
+        if self.progress:
+            self.progress.setProgress(metric, value)
+
     def getProperty(self, propname):
         return self.build.getProperty(propname)
 
@@ -558,6 +647,12 @@
           self.step_status.setText(['compile', 'failed'])
           self.step_status.setText2(['4', 'warnings'])
 
+        To have some code parse stdio (or other log stream) in realtime, add
+        a LogObserver subclass. This observer can use self.step.setProgress()
+        to provide better progress notification to the step.::
+
+          self.addLogObserver('stdio', MyLogObserver())
+
         To add a LogFile, use self.addLog. Make sure it gets closed when it
         finishes. When giving a Logfile to a RemoteShellCommand, just ask it
         to close the log when the command completes::
@@ -675,6 +770,7 @@
 
     def addLog(self, name):
         loog = self.step_status.addLog(name)
+        self._connectPendingLogObservers()
         return loog
 
     def addCompleteLog(self, name, text):
@@ -684,22 +780,54 @@
         for start in range(0, len(text), size):
             loog.addStdout(text[start:start+size])
         loog.finish()
+        self._connectPendingLogObservers()
 
     def addHTMLLog(self, name, html):
         log.msg("addHTMLLog(%s)" % name)
         self.step_status.addHTMLLog(name, html)
+        self._connectPendingLogObservers()
+
+    def addLogObserver(self, logname, observer):
+        assert providedBy(observer, interfaces.ILogObserver)
+        observer.setStep(self)
+        self._pendingLogObservers.append((logname, observer))
+        self._connectPendingLogObservers()
+
+    def _connectPendingLogObservers(self):
+        if not self._pendingLogObservers:
+            return
+        if not self.step_status:
+            return
+        current_logs = {}
+        for loog in self.step_status.getLogs():
+            current_logs[loog.getName()] = loog
+        for logname, observer in self._pendingLogObservers[:]:
+            if logname in current_logs:
+                observer.setLog(current_logs[logname])
+                self._pendingLogObservers.remove((logname, observer))
 
     def runCommand(self, c):
         d = c.run(self, self.remote)
         return d
 
 
+class StdioProgressObserver(LogObserver):
+    length = 0
+
+    def logChunk(self, build, step, log, channel, text):
+        self.length += len(text)
+        self.step.setProgress("output", self.length)
+
 
 class LoggingBuildStep(BuildStep):
     # This is an abstract base class, suitable for inheritance by all
     # BuildSteps that invoke RemoteCommands which emit stdout/stderr messages
 
-    progressMetrics = ['output']
+    progressMetrics = ('output',)
+
+    def __init__(self, *args, **kwargs):
+        BuildStep.__init__(self, *args, **kwargs)
+        self.addLogObserver('stdio', StdioProgressObserver())
 
     def describe(self, done=False):
         raise NotImplementedError("implement this in a subclass")
@@ -707,18 +835,17 @@
     def startCommand(self, cmd, errorMessages=[]):
         """
         @param cmd: a suitable RemoteCommand which will be launched, with
-                    all output being put into a LogFile named 'log'
+                    all output being put into a LogFile named 'stdio'
         """
         self.cmd = cmd # so we can interrupt it
         self.step_status.setColor("yellow")
         self.step_status.setText(self.describe(False))
-        loog = self.addLog("log")
-        for em in errorMessages:
-            loog.addHeader(em)
+        loog = self.addLog("stdio")
         log.msg("ShellCommand.start using log", loog)
         log.msg(" for cmd", cmd)
         cmd.useLog(loog, True)
-        loog.logProgressTo(self.progress, "output")
+        for em in errorMessages:
+            loog.addHeader(em)
         d = self.runCommand(cmd)
         d.addCallbacks(self._commandComplete, self.checkDisconnect)
         d.addErrback(self.failed)
@@ -741,7 +868,7 @@
 
     def _commandComplete(self, cmd):
         self.commandComplete(cmd)
-        self.createSummary(cmd.log)
+        self.createSummary(cmd.logs['stdio'])
         results = self.evaluateCommand(cmd)
         self.setStatus(cmd, results)
         return self.finished(results)
@@ -1220,7 +1347,7 @@
 
     name = "cvs"
 
-    #progressMetrics = ['output']
+    #progressMetrics = ('output',)
     #
     # additional things to track: update gives one stderr line per directory
     # (starting with 'cvs server: Updating ') (and is fairly stable if files
@@ -2020,7 +2147,7 @@
     descriptionDone = ["compile"]
     command = ["make", "all"]
 
-    OFFprogressMetrics = ['output']
+    OFFprogressMetrics = ('output',)
     # things to track: number of files compiled, number of directories
     # traversed (assuming 'make' is being used)
 




From warner at users.sourceforge.net  Thu Jun 15 05:47:44 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:44 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.656,1.657
Message-ID: <E1Fqkhl-0002Ev-Sa@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30731

Modified Files:
	ChangeLog 
Log Message:
[project @ update step_twisted to new logs['stdio'] scheme]

Original author: warner at lothar.com
Date: 2006-06-15 01:26:11

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.656
retrieving revision 1.657
diff -u -d -r1.656 -r1.657
--- ChangeLog	15 Jun 2006 05:47:35 -0000	1.656
+++ ChangeLog	15 Jun 2006 05:47:42 -0000	1.657
@@ -1,5 +1,10 @@
 2006-06-14  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step_twisted.py (HLint.commandComplete): update
+	to new cmd.logs['stdio'] scheme
+	(Trial.commandComplete): same
+	(BuildDebs.commandComplete): same
+
 	* buildbot/process/step.py (LoggedRemoteCommand): use a dict of
 	LogFiles, instead of just a single one. The old single logfile is
 	now called "stdio". LoggedRemoteCommand no longer creates a




From warner at users.sourceforge.net  Thu Jun 15 05:47:44 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:44 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py, 1.78,
	1.79
Message-ID: <E1Fqkhl-00050n-T2@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30731/buildbot/process

Modified Files:
	step_twisted.py 
Log Message:
[project @ update step_twisted to new logs['stdio'] scheme]

Original author: warner at lothar.com
Date: 2006-06-15 01:26:11

Index: step_twisted.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v
retrieving revision 1.78
retrieving revision 1.79
diff -u -d -r1.78 -r1.79
--- step_twisted.py	30 May 2006 07:07:33 -0000	1.78
+++ step_twisted.py	15 Jun 2006 05:47:42 -0000	1.79
@@ -62,7 +62,7 @@
         # submitted to hlint) because it is available in the logfile and
         # mostly exists to give the user an idea of how long the step will
         # take anyway).
-        lines = cmd.log.getText().split("\n")
+        lines = cmd.logs['stdio'].getText().split("\n")
         warningLines = filter(lambda line:':' in line, lines)
         if warningLines:
             self.addCompleteLog("warnings", "".join(warningLines))
@@ -436,7 +436,7 @@
         # figure out all status, then let the various hook functions return
         # different pieces of it
 
-        output = cmd.log.getText()
+        output = cmd.logs['stdio'].getText()
         counts = countFailedTests(output)
 
         total = counts['total']
@@ -704,7 +704,7 @@
 
     def commandComplete(self, cmd):
         errors, warnings = 0, 0
-        output = cmd.log.getText()
+        output = cmd.logs['stdio'].getText()
         summary = ""
         sio = StringIO.StringIO(output)
         for line in sio.readlines():




From warner at users.sourceforge.net  Thu Jun 15 05:47:51 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:51 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_twisted.py,1.6,1.7
Message-ID: <E1Fqkhr-0002G3-Gc@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30765/buildbot/test

Modified Files:
	test_twisted.py 
Log Message:
[project @ implement TrialTestCaseCounter, a LogObserver that counts test cases run]

Original author: warner at lothar.com
Date: 2006-06-15 01:29:02

Index: test_twisted.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_twisted.py,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -d -r1.6 -r1.7
--- test_twisted.py	15 May 2005 23:43:57 -0000	1.6
+++ test_twisted.py	15 Jun 2006 05:47:48 -0000	1.7
@@ -2,6 +2,8 @@
 
 from twisted.trial import unittest
 
+from buildbot import interfaces
+from buildbot.process import step_twisted
 from buildbot.process.step_twisted import countFailedTests, Trial
 from buildbot.status import builder
 
@@ -131,6 +133,39 @@
         count = countFailedTests(out5)
         self.assertEquals(count, self.count(total=None))
 
+class Counter(unittest.TestCase):
+
+    def setProgress(self, metric, value):
+        self.progress = (metric, value)
+
+    def testCounter(self):
+        self.progress = (None,None)
+        c = step_twisted.TrialTestCaseCounter()
+        c.setStep(self)
+        STDOUT = interfaces.LOG_CHANNEL_STDOUT
+        def add(text):
+            c.logChunk(None, None, None, STDOUT, text)
+        add("\n\n")
+        self.failUnlessEqual(self.progress, (None,None))
+        add("bogus line\n")
+        self.failUnlessEqual(self.progress, (None,None))
+        add("buildbot.test.test_config.ConfigTest.testBots ... [OK]\n")
+        self.failUnlessEqual(self.progress, ("tests", 1))
+        add("buildbot.test.test_config.ConfigTest.tes")
+        self.failUnlessEqual(self.progress, ("tests", 1))
+        add("tBuilders ... [OK]\n")
+        self.failUnlessEqual(self.progress, ("tests", 2))
+        # confirm alternative delimiters work too.. ptys seem to emit
+        # something different
+        add("buildbot.test.test_config.ConfigTest.testIRC ... [OK]\r\n")
+        self.failUnlessEqual(self.progress, ("tests", 3))
+        add("===============================================================================\n")
+        self.failUnlessEqual(self.progress, ("tests", 3))
+        add("buildbot.test.test_config.IOnlyLookLikeA.testLine ... [OK]\n")
+        self.failUnlessEqual(self.progress, ("tests", 3))
+
+
+
 class Parse(unittest.TestCase):
     def failUnlessIn(self, substr, string):
         self.failUnless(string.find(substr) != -1)




From warner at users.sourceforge.net  Thu Jun 15 05:47:50 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:50 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.657,1.658
Message-ID: <E1Fqkhr-00051c-8n@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30765

Modified Files:
	ChangeLog 
Log Message:
[project @ implement TrialTestCaseCounter, a LogObserver that counts test cases run]

Original author: warner at lothar.com
Date: 2006-06-15 01:29:02

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.657
retrieving revision 1.658
diff -u -d -r1.657 -r1.658
--- ChangeLog	15 Jun 2006 05:47:42 -0000	1.657
+++ ChangeLog	15 Jun 2006 05:47:48 -0000	1.658
@@ -1,5 +1,11 @@
 2006-06-14  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step_twisted.py (TrialTestCaseCounter):
+	implement a LogObserver that counts how many unit tests have been
+	run so far
+	(Trial.__init__): wire it in
+	* buildbot/test/test_twisted.py (Counter): unit test for it
+
 	* buildbot/process/step_twisted.py (HLint.commandComplete): update
 	to new cmd.logs['stdio'] scheme
 	(Trial.commandComplete): same




From warner at users.sourceforge.net  Thu Jun 15 05:47:50 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:50 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py, 1.79,
	1.80
Message-ID: <E1Fqkhr-0002Fy-BR@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30765/buildbot/process

Modified Files:
	step_twisted.py 
Log Message:
[project @ implement TrialTestCaseCounter, a LogObserver that counts test cases run]

Original author: warner at lothar.com
Date: 2006-06-15 01:29:02

Index: step_twisted.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v
retrieving revision 1.79
retrieving revision 1.80
diff -u -d -r1.79 -r1.80
--- step_twisted.py	15 Jun 2006 05:47:42 -0000	1.79
+++ step_twisted.py	15 Jun 2006 05:47:48 -0000	1.80
@@ -2,6 +2,7 @@
 
 from twisted.python import log, failure
 
+from buildbot import interfaces
 from buildbot.status import tests, builder
 from buildbot.status.builder import SUCCESS, FAILURE, WARNINGS, SKIPPED
 from buildbot.process import step
@@ -130,6 +131,33 @@
 
     return res
 
+
+class TrialTestCaseCounter(step.LogLineObserver):
+    _line_re = re.compile(r'^([\w\.]+) \.\.\. \[([^\]]+)\]$')
+    numTests = 0
+    finished = False
+
+    def outLineReceived(self, line):
+        # different versions of Twisted emit different per-test lines with
+        # the bwverbose reporter.
+        #  2.0.0: testSlave (buildbot.test.test_runner.Create) ... [OK]
+        #  2.1.0: buildbot.test.test_runner.Create.testSlave ... [OK]
+        #  2.4.0: buildbot.test.test_runner.Create.testSlave ... [OK]
+        # Let's just handle the most recent version, since it's the easiest.
+
+        if self.finished:
+            return
+        if line.startswith("=" * 40):
+            self.finished = True
+            return
+
+        m = self._line_re.search(line.strip())
+        if m:
+            testname, result = m.groups()
+            self.numTests += 1
+            self.step.setProgress('tests', self.numTests)
+
+
 UNSPECIFIED=() # since None is a valid choice
 
 class Trial(ShellCommand):
@@ -202,6 +230,7 @@
     """
 
     name = "trial"
+    progressMetrics = ('output', 'tests')
 
     flunkOnFailure = True
     python = None
@@ -370,6 +399,10 @@
             self.description = ["testing"]
             self.descriptionDone = ["tests"]
 
+        # this counter will feed Progress along the 'test cases' metric
+        counter = TrialTestCaseCounter()
+        self.addLogObserver('stdio', counter)
+
     def setupEnvironment(self, cmd):
         ShellCommand.setupEnvironment(self, cmd)
         if self.testpath != None:
@@ -406,6 +439,7 @@
         log.msg("Trial.start: command is", self.command)
         ShellCommand.start(self)
 
+
     def _commandComplete(self, cmd):
         # before doing the summary, etc, fetch _trial_temp/test.log
         # TODO: refactor ShellCommand so I don't have to override such




From warner at users.sourceforge.net  Thu Jun 15 05:47:57 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:57 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.658,1.659
Message-ID: <E1Fqki0-00053o-Tl@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30797

Modified Files:
	ChangeLog 
Log Message:
[project @ add user's manual sections on LogObservers, writing status plugins]

Original author: warner at lothar.com
Date: 2006-06-15 01:30:52

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.658
retrieving revision 1.659
diff -u -d -r1.658 -r1.659
--- ChangeLog	15 Jun 2006 05:47:48 -0000	1.658
+++ ChangeLog	15 Jun 2006 05:47:55 -0000	1.659
@@ -1,5 +1,10 @@
 2006-06-14  Brian Warner  <warner at lothar.com>
 
+	* docs/buildbot.texinfo (Adding LogObservers): add some limited
+	docs on writing new LogObserver classes
+	(Writing New Status Plugins): brief docs on how Status Plugins fit
+	together
+
 	* buildbot/process/step_twisted.py (TrialTestCaseCounter):
 	implement a LogObserver that counts how many unit tests have been
 	run so far




From warner at users.sourceforge.net  Thu Jun 15 05:47:57 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Thu, 15 Jun 2006 05:47:57 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.54,1.55
Message-ID: <E1Fqki2-00055R-8I@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv30797/docs

Modified Files:
	buildbot.texinfo 
Log Message:
[project @ add user's manual sections on LogObservers, writing status plugins]

Original author: warner at lothar.com
Date: 2006-06-15 01:30:52

Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.54
retrieving revision 1.55
diff -u -d -r1.54 -r1.55
--- buildbot.texinfo	12 Jun 2006 08:36:08 -0000	1.54
+++ buildbot.texinfo	15 Jun 2006 05:47:55 -0000	1.55
@@ -142,6 +142,7 @@
 * Source Checkout::             
 * ShellCommand::                
 * Simple ShellCommand Subclasses::  
+* Writing New BuildSteps::      
 
 Source Checkout
 
@@ -158,9 +159,12 @@
 * Configure::                   
 * Compile::                     
 * Test::                        
-* Writing New BuildSteps::      
 * Build Properties::            
 
+Writing New BuildSteps
+
+* Adding LogObservers::         
+
 Build Factories
 
 * BuildStep Objects::           
@@ -184,6 +188,7 @@
 * HTML Waterfall::              
 * IRC Bot::                     
 * PBListener::                  
+* Writing New Status Plugins::  
 
 Command-line tool
 
@@ -2509,7 +2514,7 @@
 @end table
 
 
- at node P4Source, , PBChangeSource, Change Sources
+ at node P4Source,  , PBChangeSource, Change Sources
 @subsection P4Source
 
 The @code{P4Source} periodically polls a @uref{http://www.perforce.com/,
@@ -2623,6 +2628,7 @@
 * Source Checkout::             
 * ShellCommand::                
 * Simple ShellCommand Subclasses::  
+* Writing New BuildSteps::      
 @end menu
 
 @node Common Parameters, Source Checkout, Build Steps, Build Steps
@@ -3181,7 +3187,7 @@
 
 @end table
 
- at node Simple ShellCommand Subclasses,  , ShellCommand, Build Steps
+ at node Simple ShellCommand Subclasses, Writing New BuildSteps, ShellCommand, Build Steps
 @subsection Simple ShellCommand Subclasses
 
 Several subclasses of ShellCommand are provided as starting points for
@@ -3193,7 +3199,6 @@
 * Configure::                   
 * Compile::                     
 * Test::                        
-* Writing New BuildSteps::      
 * Build Properties::            
 @end menu
 
@@ -3214,7 +3219,7 @@
 created with any problems that were seen (TODO: the summary is not yet
 created).
 
- at node Test, Writing New BuildSteps, Compile, Simple ShellCommand Subclasses
+ at node Test, Build Properties, Compile, Simple ShellCommand Subclasses
 @subsubsection Test
 
 This is meant to handle unit tests. The default command is @code{make
@@ -3222,28 +3227,7 @@
 
 
 
-
-
- at node Writing New BuildSteps, Build Properties, Test, Simple ShellCommand Subclasses
- at subsubsection Writing New BuildSteps
-
-While it is a good idea to keep your build process self-contained in
-the source code tree, sometimes it is convenient to put more
-intelligence into your Buildbot configuration. One was to do this is
-to write a custom BuildStep. Once written, this Step can be used in
-the @file{master.cfg} file.
-
-The best reason for writing a custom BuildStep is to better parse the
-results of the command being run. For example, a BuildStep that knows
-about JUnit could look at the logfiles to determine which tests had
-been run, how many passed and how many failed, and then report more
-detailed information than a simple @code{rc==0} -based ``good/bad''
-decision.
-
-TODO: add more description of BuildSteps.
-
-
- at node Build Properties,  , Writing New BuildSteps, Simple ShellCommand Subclasses
+ at node Build Properties,  , Test, Simple ShellCommand Subclasses
 @subsubsection Build Properties
 
 @cindex build properties
@@ -3394,6 +3378,126 @@
 @end table
 
 
+ at node Writing New BuildSteps,  , Simple ShellCommand Subclasses, Build Steps
+ at subsection Writing New BuildSteps
+
+While it is a good idea to keep your build process self-contained in
+the source code tree, sometimes it is convenient to put more
+intelligence into your Buildbot configuration. One was to do this is
+to write a custom BuildStep. Once written, this Step can be used in
+the @file{master.cfg} file.
+
+The best reason for writing a custom BuildStep is to better parse the
+results of the command being run. For example, a BuildStep that knows
+about JUnit could look at the logfiles to determine which tests had
+been run, how many passed and how many failed, and then report more
+detailed information than a simple @code{rc==0} -based ``good/bad''
+decision.
+
+TODO: add more description of BuildSteps.
+
+ at menu
+* Adding LogObservers::         
+ at end menu
+
+ at node Adding LogObservers,  , Writing New BuildSteps, Writing New BuildSteps
+ at subsubsection Adding LogObservers
+
+ at cindex LogObserver
+ at cindex LogLineObserver
+
+Most shell commands emit messages to stdout or stderr as they operate,
+especially if you ask them nicely with a @code{--verbose} flag of some
+sort. They may also write text to a log file while they run. Your
+BuildStep can watch this output as it arrives, to keep track of how
+much progress the command has made. You can get a better measure of
+progress by counting the number of source files compiled or test cases
+run than by merely tracking the number of bytes that have been written
+to stdout. This improves the accuracy and the smoothness of the ETA
+display.
+
+To accomplish this, you will need to attach a @code{LogObserver} to
+one of the log channels, most commonly to the ``stdio'' channel but
+perhaps to another one which tracks a log file. This observer is given
+all text as it is emitted from the command, and has the opportunity to
+parse that output incrementally. Once the observer has decided that
+some event has occurred (like a source file being compiled), it can
+use the @code{setProgress} method to tell the BuildStep about the
+progress that this event represents.
+
+There are a number of pre-built @code{LogObserver} classes that you
+can choose from, and of course you can subclass them to add further
+customization. The @code{LogLineObserver} class handles the grunt work
+of buffering and scanning for end-of-line delimiters, allowing your
+parser to operate on complete stdout/stderr lines.
+
+For example, let's take a look at the @code{TrialTestCaseCounter},
+which is used by the Trial step to count test cases as they are run.
+As Trial executes, it emits lines like the following:
+
+ at example
+buildbot.test.test_config.ConfigTest.testDebugPassword ... [OK]
+buildbot.test.test_config.ConfigTest.testEmpty ... [OK]
+buildbot.test.test_config.ConfigTest.testIRC ... [FAIL]
+buildbot.test.test_config.ConfigTest.testLocks ... [OK]
+ at end example
+
+When the tests are finished, trial emits a long line of ``======'' and
+then some lines which summarize the tests that failed. We want to
+avoid parsing these trailing lines, because their format is less
+well-defined than the ``[OK]'' lines.
+
+The parser class looks like this:
+
+ at example
+class TrialTestCaseCounter(step.LogLineObserver):
+    _line_re = re.compile(r'^([\w\.]+) \.\.\. \[([^\]]+)\]$')
+    numTests = 0
+    finished = False
+
+    def outLineReceived(self, line):
+        if self.finished:
+            return
+        if line.startswith("=" * 40):
+            self.finished = True
+            return
+
+        m = self._line_re.search(line.strip())
+        if m:
+            testname, result = m.groups()
+            self.numTests += 1
+            self.step.setProgress('tests', self.numTests)
+ at end example
+
+This parser only pays attention to stdout, since that's where trial
+writes the progress lines. It has a mode flag named @code{finished} to
+ignore everything after the ``===='' marker, and a scary-looking
+regular expression to match each line while hopefully ignoring other
+messages that might get displayed as the test runs.
+
+Each time it identifies a test has been completed, it increments its
+counter and delivers the new progress value to the step with
+ at code{self.step.setProgress}. This class is specifically measuring
+progress along the ``tests'' metric, in units of test cases (as
+opposed to other kinds of progress like the ``output'' metric, which
+measures in units of bytes). The Progress-tracking code uses each
+progress metric separately to come up with an overall completion
+percentage and an ETA value.
+
+To connect this parser into the @code{Trial} BuildStep,
+ at code{Trial.__init__} ends with the following clause:
+
+ at example
+        # this counter will feed Progress along the 'test cases' metric
+        counter = TrialTestCaseCounter()
+        self.addLogObserver('stdio', counter)
+ at end example
+
+This creates a TrialTestCaseCounter and tells the step to attach it to
+the ``stdio'' log. The observer is automatically given a reference to
+the step in its @code{.step} attribute.
+
+
 @node Interlocks, Build Factories, Build Steps, Build Process
 @section Interlocks
 
@@ -4068,10 +4172,11 @@
 * HTML Waterfall::              
 * IRC Bot::                     
 * PBListener::                  
+* Writing New Status Plugins::  
 @end menu
 
 @node HTML Waterfall, IRC Bot, Status Delivery, Status Delivery
- at subsection HTML Waterfall
+ at section HTML Waterfall
 
 @cindex Waterfall
 
@@ -4152,7 +4257,7 @@
 
 
 @node IRC Bot, PBListener, HTML Waterfall, Status Delivery
- at subsection IRC Bot
+ at section IRC Bot
 
 @cindex IRC
 
@@ -4221,8 +4326,8 @@
 before starting the second (hopefully fixed) build.
 @end table
 
- at node PBListener,  , IRC Bot, Status Delivery
- at subsection PBListener
+ at node PBListener, Writing New Status Plugins, IRC Bot, Status Delivery
+ at section PBListener
 
 @cindex PBListener
 
@@ -4239,6 +4344,34 @@
 status client. The @code{port} argument can also be a strports
 specification string.
 
+ at node Writing New Status Plugins,  , PBListener, Status Delivery
+ at section Writing New Status Plugins
+
+TODO: this needs a lot more examples
+
+Each status plugin is an object which provides the
+ at code{twisted.application.service.IService} interface, which creates a
+tree of Services with the buildmaster at the top [not strictly true].
+The status plugins are all children of an object which implements
+ at code{buildbot.interfaces.IStatus}, the main status object. From this
+object, the plugin can retrieve anything it wants about current and
+past builds. It can also subscribe to hear about new and upcoming
+builds.
+
+Status plugins which only react to human queries (like the Waterfall
+display) never need to subscribe to anything: they are idle until
+someone asks a question, then wake up and extract the information they
+need to answer it, then they go back to sleep. Plugins which need to
+act spontaneously when builds complete (like the Mail plugin) need to
+subscribe to hear about new builds.
+
+If the status plugin needs to run network services (like the HTTP
+server used by the Waterfall plugin), they can be attached as Service
+children of the plugin itself, using the @code{IServiceCollection}
+interface.
+
+
+
 @node Command-line tool, Resources, Status Delivery, Top
 @chapter Command-line tool
 




From warner at users.sourceforge.net  Fri Jun 16 05:27:53 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:27:53 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.659,1.660
Message-ID: <E1Fr6s6-0004je-D2@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29414

Modified Files:
	ChangeLog 
Log Message:
[project @ improve BuildStepStatus creation and setting]

Original author: warner at lothar.com
Date: 2006-06-16 01:01:40

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.659
retrieving revision 1.660
diff -u -d -r1.659 -r1.660
--- ChangeLog	15 Jun 2006 05:47:55 -0000	1.659
+++ ChangeLog	16 Jun 2006 05:27:51 -0000	1.660
@@ -1,3 +1,13 @@
+2006-06-15  Brian Warner  <warner at lothar.com>
+
+	* buildbot/status/builder.py (BuildStatus.addStep): change API to
+	take a name instead of a step, reducing the coupling somewhat.
+	This returns the BuildStepStatus object so it can be passed to the
+	new Step, instead of jamming it directly into the Step.
+	* buildbot/process/step.py (BuildStep.setStepStatus): add a setter
+	method
+	* buildbot/process/base.py (Build.setupBuild): use both methods
+
 2006-06-14  Brian Warner  <warner at lothar.com>
 
 	* docs/buildbot.texinfo (Adding LogObservers): add some limited




From warner at users.sourceforge.net  Fri Jun 16 05:27:53 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:27:53 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process base.py, 1.65,
	1.66 step.py, 1.90, 1.91
Message-ID: <E1Fr6s6-0004jm-H6@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29414/buildbot/process

Modified Files:
	base.py step.py 
Log Message:
[project @ improve BuildStepStatus creation and setting]

Original author: warner at lothar.com
Date: 2006-06-16 01:01:40

Index: base.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/base.py,v
retrieving revision 1.65
retrieving revision 1.66
diff -u -d -r1.65 -r1.66
--- base.py	23 May 2006 16:29:56 -0000	1.65
+++ base.py	16 Jun 2006 05:27:51 -0000	1.66
@@ -372,7 +372,8 @@
 
             # tell the BuildStatus about the step. This will create a
             # BuildStepStatus and bind it to the Step.
-            self.build_status.addStep(step)
+            step_status = self.build_status.addStep(name)
+            step.setStepStatus(step_status)
 
             sp = None
             if self.useProgress:

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.90
retrieving revision 1.91
diff -u -d -r1.90 -r1.91
--- step.py	15 Jun 2006 05:47:35 -0000	1.90
+++ step.py	16 Jun 2006 05:27:51 -0000	1.91
@@ -539,6 +539,9 @@
             raise TypeError(why)
         self._pendingLogObservers = []
 
+    def setStepStatus(self, step_status):
+        self.step_status = step_status
+
     def setupProgress(self):
         if self.useProgress:
             sp = progress.StepProgress(self.name, self.progressMetrics)




From warner at users.sourceforge.net  Fri Jun 16 05:27:54 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:27:54 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.82,1.83
Message-ID: <E1Fr6s6-0004jr-Ii@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29414/buildbot/status

Modified Files:
	builder.py 
Log Message:
[project @ improve BuildStepStatus creation and setting]

Original author: warner at lothar.com
Date: 2006-06-16 01:01:40

Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v
retrieving revision 1.82
retrieving revision 1.83
diff -u -d -r1.82 -r1.83
--- builder.py	15 Jun 2006 05:47:35 -0000	1.82
+++ builder.py	16 Jun 2006 05:27:51 -0000	1.83
@@ -1068,16 +1068,15 @@
 
     # methods for the base.Build to invoke
 
-    def addStep(self, step):
+    def addStep(self, name):
         """The Build is setting up, and has added a new BuildStep to its
-        list. The BuildStep object is ready for static queries (everything
-        except ETA). Give it a BuildStepStatus object to which it can send
-        status updates."""
+        list. Create a BuildStepStatus object to which it can send status
+        updates."""
 
         s = BuildStepStatus(self)
-        s.setName(step.name)
-        step.step_status = s
+        s.setName(name)
         self.steps.append(s)
+        return s
 
     def setProperty(self, propname, value):
         self.properties[propname] = value




From warner at users.sourceforge.net  Fri Jun 16 05:27:54 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:27:54 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_steps.py,1.17,1.18
Message-ID: <E1Fr6s7-0001Ag-Qy@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29414/buildbot/test

Modified Files:
	test_steps.py 
Log Message:
[project @ improve BuildStepStatus creation and setting]

Original author: warner at lothar.com
Date: 2006-06-16 01:01:40

Index: test_steps.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v
retrieving revision 1.17
retrieving revision 1.18
diff -u -d -r1.17 -r1.18
--- test_steps.py	24 Oct 2005 21:49:18 -0000	1.17
+++ test_steps.py	16 Jun 2006 05:27:52 -0000	1.18
@@ -101,7 +101,7 @@
         c = MyShellCommand(workdir=dir, command=cmd, build=self.build,
                            timeout=10)
         self.assertEqual(self.remote.events, expectedEvents)
-        self.build_status.addStep(c)
+        c.step_status = self.build_status.addStep("myshellcommand")
         d = c.startStep(self.remote)
         self.failUnless(c.started)
         rc = c.rc




From warner at users.sourceforge.net  Fri Jun 16 05:28:01 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:01 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.91,1.92
Message-ID: <E1Fr6sF-0004nu-6J@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29475/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ LogObserver: add outReceived and errReceived base methods]

Original author: warner at lothar.com
Date: 2006-06-16 01:06:20

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.91
retrieving revision 1.92
diff -u -d -r1.91 -r1.92
--- step.py	16 Jun 2006 05:27:51 -0000	1.91
+++ step.py	16 Jun 2006 05:27:59 -0000	1.92
@@ -345,6 +345,17 @@
 
     # TODO: add a logEnded method? er, stepFinished?
 
+    def outReceived(self, data):
+        """This will be called with chunks of stdout data. Override this in
+        your observer."""
+        pass
+
+    def errReceived(self, data):
+        """This will be called with chunks of stderr data. Override this in
+        your observer."""
+        pass
+
+
 class LogLineObserver(LogObserver):
     def __init__(self):
         self.stdoutParser = basic.LineOnlyReceiver()




From warner at users.sourceforge.net  Fri Jun 16 05:28:01 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:01 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.660,1.661
Message-ID: <E1Fr6sF-0001F1-6D@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29475

Modified Files:
	ChangeLog 
Log Message:
[project @ LogObserver: add outReceived and errReceived base methods]

Original author: warner at lothar.com
Date: 2006-06-16 01:06:20

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.660
retrieving revision 1.661
diff -u -d -r1.660 -r1.661
--- ChangeLog	16 Jun 2006 05:27:51 -0000	1.660
+++ ChangeLog	16 Jun 2006 05:27:59 -0000	1.661
@@ -1,5 +1,8 @@
 2006-06-15  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step.py (LogObserver): add outReceived and
+	errReceived base methods, to be overridden
+
 	* buildbot/status/builder.py (BuildStatus.addStep): change API to
 	take a name instead of a step, reducing the coupling somewhat.
 	This returns the BuildStepStatus object so it can be passed to the




From warner at users.sourceforge.net  Fri Jun 16 05:28:08 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:08 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.661,1.662
Message-ID: <E1Fr6sL-0001IK-6H@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29826

Modified Files:
	ChangeLog 
Log Message:
[project @ new LogObserver test, new setupBuildStepStatus utility method]

Original author: warner at lothar.com
Date: 2006-06-16 01:06:50

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.661
retrieving revision 1.662
diff -u -d -r1.661 -r1.662
--- ChangeLog	16 Jun 2006 05:27:59 -0000	1.661
+++ ChangeLog	16 Jun 2006 05:28:06 -0000	1.662
@@ -1,5 +1,12 @@
 2006-06-15  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/test_steps.py (LogObserver): new test to verify
+	LogObservers can be created at various times and still get
+	connected up properly
+
+	* buildbot/test/runutils.py (setupBuildStepStatus): utility method
+	to create BuildStepStatus instances that actually work.
+
 	* buildbot/process/step.py (LogObserver): add outReceived and
 	errReceived base methods, to be overridden
 




From warner at users.sourceforge.net  Fri Jun 16 05:28:08 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:08 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py, 1.5,
	1.6 test_steps.py, 1.18, 1.19
Message-ID: <E1Fr6sL-0001Ib-Ga@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29826/buildbot/test

Modified Files:
	runutils.py test_steps.py 
Log Message:
[project @ new LogObserver test, new setupBuildStepStatus utility method]

Original author: warner at lothar.com
Date: 2006-06-16 01:06:50

Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -d -r1.5 -r1.6
--- runutils.py	29 May 2006 00:10:38 -0000	1.5
+++ runutils.py	16 Jun 2006 05:28:06 -0000	1.6
@@ -208,3 +208,17 @@
         # connection, and sees two connections at once.
         raise NotImplementedError
 
+
+def setupBuildStepStatus(basedir):
+    """Return a BuildStep with a suitable BuildStepStatus object, ready to
+    use."""
+    os.mkdir(basedir)
+    botmaster = None
+    s0 = builder.Status(botmaster, basedir)
+    s1 = s0.builderAdded("buildername", "buildername")
+    s2 = builder.BuildStatus(s1, 1)
+    s3 = builder.BuildStepStatus(s2)
+    s3.setName("foostep")
+    s3.started = True
+    s3.stepStarted()
+    return s3

Index: test_steps.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v
retrieving revision 1.18
retrieving revision 1.19
diff -u -d -r1.18 -r1.19
--- test_steps.py	16 Jun 2006 05:27:52 -0000	1.18
+++ test_steps.py	16 Jun 2006 05:28:06 -0000	1.19
@@ -24,7 +24,7 @@
 from buildbot.process import step, base, factory
 from buildbot.process.step import ShellCommand #, ShellCommands
 from buildbot.status import builder
-from buildbot.test.runutils import RunMixin
+from buildbot.test.runutils import RunMixin, setupBuildStep
 from buildbot.twcompat import maybeWait
 from buildbot.slave import commands
 
@@ -234,3 +234,31 @@
         d = self.doBuild("quick")
         return maybeWait(d)
 
+
+class MyObserver(step.LogObserver):
+    out = ""
+    def outReceived(self, data):
+        self.out = self.out + data
+
+class LogObserver(unittest.TestCase):
+    def testAdd(self):
+        bss = setupBuildStepStatus("logobserver")
+        build = None
+        s = step.BuildStep(build)
+        s.setStepStatus(bss)
+        o1,o2,o3 = MyObserver(), MyObserver(), MyObserver()
+
+        # add the log before the observer
+        l1 = s.addLog("one")
+        l1.addStdout("onestuff")
+        s.addLogObserver("one", o1)
+        self.failUnlessEqual(o1.out, "onestuff")
+        l1.addStdout(" morestuff")
+        self.failUnlessEqual(o1.out, "onestuff morestuff")
+
+        # add the observer before the log
+        s.addLogObserver("two", o2)
+        l2 = s.addLog("two")
+        l2.addStdout("twostuff")
+        self.failUnlessEqual(o2.out, "twostuff")
+




From warner at users.sourceforge.net  Fri Jun 16 05:28:15 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:15 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process base.py,1.66,1.67
Message-ID: <E1Fr6sR-0001Jp-OC@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29924/buildbot/process

Modified Files:
	base.py 
Log Message:
[project @ rearrange BuildStatus.addStepWithName a bit]

Original author: warner at lothar.com
Date: 2006-06-16 05:10:59

Index: base.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/base.py,v
retrieving revision 1.66
retrieving revision 1.67
diff -u -d -r1.66 -r1.67
--- base.py	16 Jun 2006 05:27:51 -0000	1.66
+++ base.py	16 Jun 2006 05:28:13 -0000	1.67
@@ -372,7 +372,7 @@
 
             # tell the BuildStatus about the step. This will create a
             # BuildStepStatus and bind it to the Step.
-            step_status = self.build_status.addStep(name)
+            step_status = self.build_status.addStepWithName(name)
             step.setStepStatus(step_status)
 
             sp = None




From warner at users.sourceforge.net  Fri Jun 16 05:28:15 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:15 +0000
Subject: [Buildbot-commits] buildbot/buildbot/status builder.py,1.83,1.84
Message-ID: <E1Fr6sR-0004rj-Vl@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/status
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29924/buildbot/status

Modified Files:
	builder.py 
Log Message:
[project @ rearrange BuildStatus.addStepWithName a bit]

Original author: warner at lothar.com
Date: 2006-06-16 05:10:59

Index: builder.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/status/builder.py,v
retrieving revision 1.83
retrieving revision 1.84
diff -u -d -r1.83 -r1.84
--- builder.py	16 Jun 2006 05:27:51 -0000	1.83
+++ builder.py	16 Jun 2006 05:28:13 -0000	1.84
@@ -1068,7 +1068,7 @@
 
     # methods for the base.Build to invoke
 
-    def addStep(self, name):
+    def addStepWithName(self, name):
         """The Build is setting up, and has added a new BuildStep to its
         list. Create a BuildStepStatus object to which it can send status
         updates."""




From warner at users.sourceforge.net  Fri Jun 16 05:28:15 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:15 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_steps.py, 1.19,
	1.20 test_web.py, 1.30, 1.31
Message-ID: <E1Fr6sS-0004ro-4q@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29924/buildbot/test

Modified Files:
	test_steps.py test_web.py 
Log Message:
[project @ rearrange BuildStatus.addStepWithName a bit]

Original author: warner at lothar.com
Date: 2006-06-16 05:10:59

Index: test_steps.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v
retrieving revision 1.19
retrieving revision 1.20
diff -u -d -r1.19 -r1.20
--- test_steps.py	16 Jun 2006 05:28:06 -0000	1.19
+++ test_steps.py	16 Jun 2006 05:28:13 -0000	1.20
@@ -24,7 +24,7 @@
 from buildbot.process import step, base, factory
 from buildbot.process.step import ShellCommand #, ShellCommands
 from buildbot.status import builder
-from buildbot.test.runutils import RunMixin, setupBuildStep
+from buildbot.test.runutils import RunMixin, setupBuildStepStatus
 from buildbot.twcompat import maybeWait
 from buildbot.slave import commands
 
@@ -101,7 +101,7 @@
         c = MyShellCommand(workdir=dir, command=cmd, build=self.build,
                            timeout=10)
         self.assertEqual(self.remote.events, expectedEvents)
-        c.step_status = self.build_status.addStep("myshellcommand")
+        c.step_status = self.build_status.addStepWithName("myshellcommand")
         d = c.startStep(self.remote)
         self.failUnless(c.started)
         rc = c.rc

Index: test_web.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_web.py,v
retrieving revision 1.30
retrieving revision 1.31
diff -u -d -r1.30 -r1.31
--- test_web.py	2 Jun 2006 06:26:28 -0000	1.30
+++ test_web.py	16 Jun 2006 05:28:13 -0000	1.31
@@ -389,16 +389,16 @@
         self.port = port
         # insert an event
 
-        s = m.status.getBuilder("builder1")
         req = base.BuildRequest("reason", sourcestamp.SourceStamp())
-        bs = s.newBuild()
         build1 = base.Build([req])
-        step1 = step.BuildStep(build=build1)
-        step1.name = "setup"
-        bs.addStep(step1)
-        bs.buildStarted(build1)
-        step1.step_status.stepStarted()
+        bs = m.status.getBuilder("builder1").newBuild()
         bs.setReason("reason")
+        bs.buildStarted(build1)
+
+        step1 = step.BuildStep(build=build1, name="setup")
+        bss = bs.addStepWithName("setup")
+        step1.setStepStatus(bss)
+        bss.stepStarted()
 
         log1 = step1.addLog("output")
         log1.addStdout("some stdout\n")




From warner at users.sourceforge.net  Fri Jun 16 05:28:15 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:28:15 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.662,1.663
Message-ID: <E1Fr6sR-0001Jd-DI@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv29924

Modified Files:
	ChangeLog 
Log Message:
[project @ rearrange BuildStatus.addStepWithName a bit]

Original author: warner at lothar.com
Date: 2006-06-16 05:10:59

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.662
retrieving revision 1.663
diff -u -d -r1.662 -r1.663
--- ChangeLog	16 Jun 2006 05:28:06 -0000	1.662
+++ ChangeLog	16 Jun 2006 05:28:12 -0000	1.663
@@ -10,13 +10,16 @@
 	* buildbot/process/step.py (LogObserver): add outReceived and
 	errReceived base methods, to be overridden
 
-	* buildbot/status/builder.py (BuildStatus.addStep): change API to
-	take a name instead of a step, reducing the coupling somewhat.
-	This returns the BuildStepStatus object so it can be passed to the
-	new Step, instead of jamming it directly into the Step.
+	* buildbot/status/builder.py (BuildStatus.addStepWithName): change
+	API to take a name instead of a step, reducing the coupling
+	somewhat. This returns the BuildStepStatus object so it can be
+	passed to the new Step, instead of jamming it directly into the
+	Step.
 	* buildbot/process/step.py (BuildStep.setStepStatus): add a setter
 	method
 	* buildbot/process/base.py (Build.setupBuild): use both methods
+	* buildbot/test/test_web.py (Logfile.setUp): rearrange the setup
+	process a bit to match
 
 2006-06-14  Brian Warner  <warner at lothar.com>
 




From warner at users.sourceforge.net  Fri Jun 16 05:41:24 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:41:24 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_vc.py,1.61,1.62
Message-ID: <E1Fr75B-00043n-4A@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv6365/buildbot/test

Modified Files:
	test_vc.py 
Log Message:
[project @ test_vc.P4: rename testBranch to testCheckoutBranch to match others]

Original author: warner at lothar.com
Date: 2006-06-16 05:40:20

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.61
retrieving revision 1.62
diff -u -d -r1.61 -r1.62
--- test_vc.py	12 Jun 2006 08:36:44 -0000	1.61
+++ test_vc.py	16 Jun 2006 05:41:22 -0000	1.62
@@ -1516,18 +1516,18 @@
         # changed.
         return maybeWait(d)
 
-    def testPatch(self):
+    def testCheckoutBranch(self):
         self.helper.vcargs = { 'p4port': self.helper.p4port,
                                'p4base': '//depot/',
                                'defaultBranch': 'trunk' }
-        d = self.do_patch()
+        d = self.do_branch()
         return maybeWait(d)
 
-    def testBranch(self):
+    def testPatch(self):
         self.helper.vcargs = { 'p4port': self.helper.p4port,
                                'p4base': '//depot/',
                                'defaultBranch': 'trunk' }
-        d = self.do_branch()
+        d = self.do_patch()
         return maybeWait(d)
 
 VCS.registerVC(P4.vc_name, P4Helper())




From warner at users.sourceforge.net  Fri Jun 16 05:41:24 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Fri, 16 Jun 2006 05:41:24 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.663,1.664
Message-ID: <E1Fr75B-0007sE-38@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv6365

Modified Files:
	ChangeLog 
Log Message:
[project @ test_vc.P4: rename testBranch to testCheckoutBranch to match others]

Original author: warner at lothar.com
Date: 2006-06-16 05:40:20

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.663
retrieving revision 1.664
diff -u -d -r1.663 -r1.664
--- ChangeLog	16 Jun 2006 05:28:12 -0000	1.663
+++ ChangeLog	16 Jun 2006 05:41:22 -0000	1.664
@@ -1,5 +1,9 @@
 2006-06-15  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/test_vc.py (P4.testCheckoutBranch): rename from
+	'testBranch' to match other VC tests and have the tests run in
+	roughly increasing order of dependency
+
 	* buildbot/test/test_steps.py (LogObserver): new test to verify
 	LogObservers can be created at various times and still get
 	connected up properly




From warner at users.sourceforge.net  Tue Jun 20 08:08:13 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:13 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.664,1.665
Message-ID: <E1FsbHS-0001B3-5f@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14443

Modified Files:
	ChangeLog 
Log Message:
[project @ move some test utilities like SignalMixin and FakeSlaveBuilder to a common file]

Original author: warner at lothar.com
Date: 2006-06-16 17:03:56

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.664
retrieving revision 1.665
diff -u -d -r1.664 -r1.665
--- ChangeLog	16 Jun 2006 05:41:22 -0000	1.664
+++ ChangeLog	20 Jun 2006 08:08:10 -0000	1.665
@@ -1,3 +1,10 @@
+2006-06-16  Brian Warner  <warner at lothar.com>
+
+	* buildbot/test/test_slavecommand.py: Move some utilities like
+	SignalMixin and FakeSlaveBuilder from here ..
+	* buildbot/test/runutils.py: .. to here, so they can be used by
+	other test classes too
+
 2006-06-15  Brian Warner  <warner at lothar.com>
 
 	* buildbot/test/test_vc.py (P4.testCheckoutBranch): rename from




From warner at users.sourceforge.net  Tue Jun 20 08:08:13 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:13 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py, 1.6,
	1.7 test_slavecommand.py, 1.20, 1.21
Message-ID: <E1FsbHS-0002Ty-8N@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14443/buildbot/test

Modified Files:
	runutils.py test_slavecommand.py 
Log Message:
[project @ move some test utilities like SignalMixin and FakeSlaveBuilder to a common file]

Original author: warner at lothar.com
Date: 2006-06-16 17:03:56

Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -d -r1.6 -r1.7
--- runutils.py	16 Jun 2006 05:28:06 -0000	1.6
+++ runutils.py	20 Jun 2006 08:08:11 -0000	1.7
@@ -1,7 +1,8 @@
 
+import signal
 import shutil, os, errno
-from twisted.internet import defer
-from twisted.python import log
+from twisted.internet import defer, reactor
+from twisted.python import log, util
 
 from buildbot import master, interfaces
 from buildbot.twcompat import maybeWait
@@ -17,16 +18,19 @@
 class MyBuildSlave(bot.BuildSlave):
     botClass = MyBot
 
+def rmtree(d):
+    try:
+        shutil.rmtree(d, ignore_errors=1)
+    except OSError, e:
+        # stupid 2.2 appears to ignore ignore_errors
+        if e.errno != errno.ENOENT:
+            raise
+
 class RunMixin:
     master = None
 
     def rmtree(self, d):
-        try:
-            shutil.rmtree(d, ignore_errors=1)
-        except OSError, e:
-            # stupid 2.2 appears to ignore ignore_errors
-            if e.errno != errno.ENOENT:
-                raise
+        rmtree(d)
 
     def setUp(self):
         self.slaves = {}
@@ -222,3 +226,38 @@
     s3.started = True
     s3.stepStarted()
     return s3
+
+def findDir():
+    # the same directory that holds this script
+    return util.sibpath(__file__, ".")
+
+class SignalMixin:
+    sigchldHandler = None
+    
+    def setUpClass(self):
+        # make sure SIGCHLD handler is installed, as it should be on
+        # reactor.run(). problem is reactor may not have been run when this
+        # test runs.
+        if hasattr(reactor, "_handleSigchld") and hasattr(signal, "SIGCHLD"):
+            self.sigchldHandler = signal.signal(signal.SIGCHLD,
+                                                reactor._handleSigchld)
+
+    def tearDownClass(self):
+        if self.sigchldHandler:
+            signal.signal(signal.SIGCHLD, self.sigchldHandler)
+
+# these classes are used to test SlaveCommands in isolation
+
+class FakeSlaveBuilder:
+    debug = False
+    def __init__(self, usePTY):
+        self.updates = []
+        self.basedir = findDir()
+        self.usePTY = usePTY
+
+    def sendUpdate(self, data):
+        if self.debug:
+            print "FakeSlaveBuilder.sendUpdate", data
+        self.updates.append(data)
+
+

Index: test_slavecommand.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slavecommand.py,v
retrieving revision 1.20
retrieving revision 1.21
diff -u -d -r1.20 -r1.21
--- test_slavecommand.py	7 May 2006 00:44:28 -0000	1.20
+++ test_slavecommand.py	20 Jun 2006 08:08:11 -0000	1.21
@@ -2,53 +2,19 @@
 
 from twisted.trial import unittest
 from twisted.internet import reactor, interfaces
-from twisted.python import util, runtime, failure
+from twisted.python import runtime, failure
 from buildbot.twcompat import maybeWait
 
-noisy = False
-if noisy:
-    from twisted.python.log import startLogging
-    import sys
-    startLogging(sys.stdout)
-
 import os, re, sys
-import signal
 
 from buildbot.slave import commands
 SlaveShellCommand = commands.SlaveShellCommand
 
+from buildbot.test.runutils import findDir, SignalMixin, FakeSlaveBuilder
+
 # test slavecommand.py by running the various commands with a fake
 # SlaveBuilder object that logs the calls to sendUpdate()
 
-def findDir():
-    # the same directory that holds this script
-    return util.sibpath(__file__, ".")
-        
-class FakeSlaveBuilder:
-    def __init__(self, usePTY):
-        self.updates = []
-        self.basedir = findDir()
-        self.usePTY = usePTY
-
-    def sendUpdate(self, data):
-        if noisy: print "FakeSlaveBuilder.sendUpdate", data
-        self.updates.append(data)
-
-
-class SignalMixin:
-    sigchldHandler = None
-    
-    def setUpClass(self):
-        # make sure SIGCHLD handler is installed, as it should be on
-        # reactor.run(). problem is reactor may not have been run when this
-        # test runs.
-        if hasattr(reactor, "_handleSigchld") and hasattr(signal, "SIGCHLD"):
-            self.sigchldHandler = signal.signal(signal.SIGCHLD,
-                                                reactor._handleSigchld)
-    
-    def tearDownClass(self):
-        if self.sigchldHandler:
-            signal.signal(signal.SIGCHLD, self.sigchldHandler)
 
 
 class ShellBase(SignalMixin):




From warner at users.sourceforge.net  Tue Jun 20 08:08:19 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:19 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_control.py, 1.9,
	1.10 test_vc.py, 1.62, 1.63
Message-ID: <E1FsbHY-0002Vg-40@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14486/buildbot/test

Modified Files:
	test_control.py test_vc.py 
Log Message:
[project @ more SignalMixin refactoring]

Original author: warner at lothar.com
Date: 2006-06-16 17:08:55

Index: test_control.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_control.py,v
retrieving revision 1.9
retrieving revision 1.10
diff -u -d -r1.9 -r1.10
--- test_control.py	14 Oct 2005 19:47:58 -0000	1.9
+++ test_control.py	20 Jun 2006 08:08:17 -0000	1.10
@@ -12,6 +12,7 @@
 from buildbot.status import builder
 from buildbot.status.builder import SUCCESS
 from buildbot.process import base
+from buildbot.test.runutils import SignalMixin, rmtree
 
 config = """
 from buildbot.process import factory, step
@@ -37,30 +38,11 @@
     def getSlaveCommandVersion(self, command, oldversion=None):
         return "1.10"
 
-class SignalMixin:
-    sigchldHandler = None
-    
-    def setUpClass(self):
-        # make sure SIGCHLD handler is installed, as it should be on
-        # reactor.run(). problem is reactor may not have been run when this
-        # test runs.
-        if hasattr(reactor, "_handleSigchld") and hasattr(signal, "SIGCHLD"):
-            self.sigchldHandler = signal.signal(signal.SIGCHLD,
-                                                reactor._handleSigchld)
-    
-    def tearDownClass(self):
-        if self.sigchldHandler:
-            signal.signal(signal.SIGCHLD, self.sigchldHandler)
 
 class Force(unittest.TestCase):
 
     def rmtree(self, d):
-        try:
-            shutil.rmtree(d, ignore_errors=1)
-        except OSError, e:
-            # stupid 2.2 appears to ignore ignore_errors
-            if e.errno != errno.ENOENT:
-                raise
+        rmtree(d)
 
     def setUp(self):
         self.master = None

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.62
retrieving revision 1.63
diff -u -d -r1.62 -r1.63
--- test_vc.py	16 Jun 2006 05:41:22 -0000	1.62
+++ test_vc.py	20 Jun 2006 08:08:17 -0000	1.63
@@ -22,6 +22,7 @@
 from buildbot.sourcestamp import SourceStamp
 from buildbot.twcompat import maybeWait, which
 from buildbot.scripts import tryclient
+from buildbot.test.runutils import SignalMixin
 
 #step.LoggedRemoteCommand.debug = True
 
@@ -259,21 +260,6 @@
 
 VCS = VCS_Helper()
 
-class SignalMixin:
-    sigchldHandler = None
-    
-    def setUpClass(self):
-        # make sure SIGCHLD handler is installed, as it should be on
-        # reactor.run(). problem is reactor may not have been run when this
-        # test runs.
-        if hasattr(reactor, "_handleSigchld") and hasattr(signal, "SIGCHLD"):
-            self.sigchldHandler = signal.signal(signal.SIGCHLD,
-                                                reactor._handleSigchld)
-    
-    def tearDownClass(self):
-        if self.sigchldHandler:
-            signal.signal(signal.SIGCHLD, self.sigchldHandler)
-
 
 # the overall plan here:
 #




From warner at users.sourceforge.net  Tue Jun 20 08:08:19 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:19 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.665,1.666
Message-ID: <E1FsbHX-0002VZ-UL@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14486

Modified Files:
	ChangeLog 
Log Message:
[project @ more SignalMixin refactoring]

Original author: warner at lothar.com
Date: 2006-06-16 17:08:55

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.665
retrieving revision 1.666
diff -u -d -r1.665 -r1.666
--- ChangeLog	20 Jun 2006 08:08:10 -0000	1.665
+++ ChangeLog	20 Jun 2006 08:08:17 -0000	1.666
@@ -4,6 +4,8 @@
 	SignalMixin and FakeSlaveBuilder from here ..
 	* buildbot/test/runutils.py: .. to here, so they can be used by
 	other test classes too
+	* buildbot/test/test_vc.py: more SignalMixin refactoring
+	* buildbot/test/test_control.py: same
 
 2006-06-15  Brian Warner  <warner at lothar.com>
 




From warner at users.sourceforge.net  Tue Jun 20 08:08:25 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:25 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.666,1.667
Message-ID: <E1FsbHe-0001DE-97@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14510

Modified Files:
	ChangeLog 
Log Message:
[project @ test_run.py: factor out rmtree]

Original author: warner at lothar.com
Date: 2006-06-16 17:10:11

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.666
retrieving revision 1.667
diff -u -d -r1.666 -r1.667
--- ChangeLog	20 Jun 2006 08:08:17 -0000	1.666
+++ ChangeLog	20 Jun 2006 08:08:23 -0000	1.667
@@ -6,6 +6,7 @@
 	other test classes too
 	* buildbot/test/test_vc.py: more SignalMixin refactoring
 	* buildbot/test/test_control.py: same
+	* buildbot/test/test_run.py: and some rmtree refactoring
 
 2006-06-15  Brian Warner  <warner at lothar.com>
 




From warner at users.sourceforge.net  Tue Jun 20 08:08:26 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:26 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_run.py,1.37,1.38
Message-ID: <E1FsbHe-0001DP-Ha@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14510/buildbot/test

Modified Files:
	test_run.py 
Log Message:
[project @ test_run.py: factor out rmtree]

Original author: warner at lothar.com
Date: 2006-06-16 17:10:11

Index: test_run.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_run.py,v
retrieving revision 1.37
retrieving revision 1.38
diff -u -d -r1.37 -r1.38
--- test_run.py	3 Jan 2006 09:26:41 -0000	1.37
+++ test_run.py	20 Jun 2006 08:08:23 -0000	1.38
@@ -14,7 +14,7 @@
 from buildbot.process.base import BuildRequest
 from buildbot.twcompat import maybeWait
 
-from buildbot.test.runutils import RunMixin
+from buildbot.test.runutils import RunMixin, rmtree
 
 config_base = """
 from buildbot.process import factory, step
@@ -74,12 +74,7 @@
 
 class Run(unittest.TestCase):
     def rmtree(self, d):
-        try:
-            shutil.rmtree(d, ignore_errors=1)
-        except OSError, e:
-            # stupid 2.2 appears to ignore ignore_errors
-            if e.errno != errno.ENOENT:
-                raise
+        rmtree(d)
 
     def testMaster(self):
         self.rmtree("basedir")




From warner at users.sourceforge.net  Tue Jun 20 08:08:32 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:32 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.667,1.668
Message-ID: <E1FsbHl-0002XJ-Jy@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14527

Modified Files:
	ChangeLog 
Log Message:
[project @ tests: some rmtree refactoring, and create SlaveCommandTestBase]

Original author: warner at lothar.com
Date: 2006-06-16 19:53:52

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.667
retrieving revision 1.668
diff -u -d -r1.667 -r1.668
--- ChangeLog	20 Jun 2006 08:08:23 -0000	1.667
+++ ChangeLog	20 Jun 2006 08:08:29 -0000	1.668
@@ -1,5 +1,10 @@
 2006-06-16  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/test_steps.py (BuildStep.setUp): rmtree refactoring
+
+	* buildbot/test/runutils.py (SlaveCommandTestBase): utility class
+	for tests which exercise SlaveCommands in isolation.
+
 	* buildbot/test/test_slavecommand.py: Move some utilities like
 	SignalMixin and FakeSlaveBuilder from here ..
 	* buildbot/test/runutils.py: .. to here, so they can be used by




From warner at users.sourceforge.net  Tue Jun 20 08:08:34 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:34 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py, 1.7,
	1.8 test_steps.py, 1.20, 1.21
Message-ID: <E1FsbHm-0001F9-Fo@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14527/buildbot/test

Modified Files:
	runutils.py test_steps.py 
Log Message:
[project @ tests: some rmtree refactoring, and create SlaveCommandTestBase]

Original author: warner at lothar.com
Date: 2006-06-16 19:53:52

Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -d -r1.7 -r1.8
--- runutils.py	20 Jun 2006 08:08:11 -0000	1.7
+++ runutils.py	20 Jun 2006 08:08:30 -0000	1.8
@@ -261,3 +261,28 @@
         self.updates.append(data)
 
 
+class SlaveCommandTestBase(SignalMixin):
+    usePTY = False
+    def setUp(self):
+        self.builder = FakeSlaveBuilder(self.usePTY)
+
+    def startCommand(self, cmdclass, args):
+        stepId = 0
+        c = cmdclass(self.builder, stepId, args)
+        c.running = True
+        d = c.start()
+        return d
+
+    def collectUpdates(self, res):
+        logs = {}
+        for u in self.builder.updates:
+            for k in u.keys():
+                if k == "log":
+                    logname,data = u[k]
+                    oldlog = logs.get(("log",logname), "")
+                    logs[("log",logname)] = oldlog + u[k]
+                elif k == "rc":
+                    pass
+                else:
+                    logs[k] = logs.get(k, "") + u[k]
+        return logs

Index: test_steps.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v
retrieving revision 1.20
retrieving revision 1.21
diff -u -d -r1.20 -r1.21
--- test_steps.py	16 Jun 2006 05:28:13 -0000	1.20
+++ test_steps.py	20 Jun 2006 08:08:32 -0000	1.21
@@ -5,7 +5,7 @@
 # the test harness should send statusUpdate() messages in with assorted
 # data, eventually calling remote_complete(). Then we can verify that the
 # Step's rc was correct, and that the status it was supposed to return
-# mathces.
+# matches.
 
 # sometimes, .callRemote should raise an exception because of a stale
 # reference. Sometimes it should errBack with an UnknownCommand failure.
@@ -24,7 +24,7 @@
 from buildbot.process import step, base, factory
 from buildbot.process.step import ShellCommand #, ShellCommands
 from buildbot.status import builder
-from buildbot.test.runutils import RunMixin, setupBuildStepStatus
+from buildbot.test.runutils import RunMixin, rmtree, setupBuildStepStatus
 from buildbot.twcompat import maybeWait
 from buildbot.slave import commands
 
@@ -67,7 +67,9 @@
 
 
 class BuildStep(unittest.TestCase):
+
     def setUp(self):
+        rmtree("test_steps")
         self.builder = FakeBuilder()
         self.builder_status = builder.BuilderStatus("fakebuilder")
         self.builder_status.basedir = "test_steps"
@@ -156,6 +158,7 @@
         self.assertEqual(self.failed, 0)
         self.assertEqual(self.results, 0)
 
+
 class Steps(unittest.TestCase):
     def testMultipleStepInstances(self):
         steps = [




From warner at users.sourceforge.net  Tue Jun 20 08:08:41 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:41 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.55,1.56
Message-ID: <E1FsbHu-0001Gm-47@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14565/docs

Modified Files:
	buildbot.texinfo 
Log Message:
[project @ add test and docs for the (not-yet implemented) watch-multiple-logfiles feature]

Original author: warner at lothar.com
Date: 2006-06-16 19:57:42

Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.55
retrieving revision 1.56
diff -u -d -r1.55 -r1.56
--- buildbot.texinfo	15 Jun 2006 05:47:55 -0000	1.55
+++ buildbot.texinfo	20 Jun 2006 08:08:39 -0000	1.56
@@ -3167,6 +3167,29 @@
 a PTY do not have separate stdout/stderr streams: both are merged into
 stdout.
 
+ at item logfiles
+Sometimes commands will log interesting data to a local file, rather
+than emitting everything to stdout or stderr. For example, Twisted's
+``trial'' command (which runs unit tests) only presents summary
+information to stdout, and puts the rest into a file named
+ at file{_trial_temp/test.log}. It is often useful to watch these files
+as the command runs, rather than using @command{/bin/cat} to dump
+their contents afterwards.
+
+The @code{logfiles=} argument allows you to collect data from these
+secondary logfiles in real-time, as the step is running. It accepts a
+dictionary which maps from a local Log name (which is how the log data
+is presented in the build results) to a remote filename (interpreted
+relative to the build's working directory). Each named file will be
+polled on a regular basis (every couple of seconds) as the build runs,
+and any new text will be sent over to the buildmaster.
+
+ at example
+s(ShellCommand, command=["make", "test"],
+  logfiles=@{"triallog": "_trial_temp/test.log"@})
+ at end example
+
+
 @item timeout
 if the command fails to produce any output for this many seconds, it
 is assumed to be locked up and will be killed.




From warner at users.sourceforge.net  Tue Jun 20 08:08:41 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:41 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test emitlogs.py, NONE,
	1.1 test_shell.py, NONE, 1.1
Message-ID: <E1FsbHu-0001Gc-1K@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14565/buildbot/test

Added Files:
	emitlogs.py test_shell.py 
Log Message:
[project @ add test and docs for the (not-yet implemented) watch-multiple-logfiles feature]

Original author: warner at lothar.com
Date: 2006-06-16 19:57:42

--- NEW FILE: emitlogs.py ---
#! /usr/bin/python

import os, sys, time

log2 = open("log2", "wt")
log3 = open("log3", "wt")

for i in range(3):
    sys.stdout.write("this is stdout %d\n" % i)
    log2.write("this is log2 %d\n" % i)
    log3.write("this is log3 %d\n" % i)
    time.sleep(1)

log2.close()
log3.close()

sys.exit(0)


--- NEW FILE: test_shell.py ---


# test step.ShellCommand and the slave-side commands.ShellCommand

import sys
from twisted.trial import unittest
from buildbot.process.step import ShellCommand
from buildbot.slave.commands import SlaveShellCommand
from buildbot.twcompat import maybeWait
from buildbot.test.runutils import SlaveCommandTestBase

class SlaveSide(SlaveCommandTestBase, unittest.TestCase):
    def testOne(self):
        args = {
            'command': [sys.executable, "emit.py", "0"],
            'workdir': ".",
            }
        d = self.startCommand(SlaveShellCommand, args)
        d.addCallback(self.collectUpdates)
        def _check(logs):
            self.failUnlessEqual(logs['stdout'], "this is stdout\n")
            self.failUnlessEqual(logs['stderr'], "this is stderr\n")
        d.addCallback(_check)
        return maybeWait(d)

    # TODO: move test_slavecommand.Shell and .ShellPTY over here

    def _generateText(self, filename):
        lines = []
        for i in range(3):
            lines.append("this is %s %d\n" % (filename, i))
        return "".join(lines)

    def testLogFiles(self):
        # emitlogs.py writes one line per second to stdout and two logfiles,
        # for 3 seconds total.
        args = {
            'command': [sys.executable, "emitlogs.py"],
            'workdir': ".",
            'logfiles': {"log2": "log2.out",
                         "log3": "log3.out"},
            }
        d = self.startCommand(SlaveShellCommand, args)
        # after two seconds, there should be some data in the secondary
        # logfiles

        # TODO: I want to test that logfiles are being read in a timely
        # fashion. How can I do this and still have the tests be reliable
        # under load?

        d.addCallback(self.collectUpdates)
        def _check(logs):
            self.failUnlessEqual(logs['stdout'], self._generateText("stdout"))
            self.failUnlessEqual(logs[('log','log2')],
                                 self._generateText("log2"))
            self.failUnlessEqual(logs[('log','log3')],
                                 self._generateText("log3"))
        d.addCallback(_check)
        return maybeWait(d)
    testLogFiles.todo = "doesn't work yet"


def OFF_testLogfiles_1(self, res, ss):
    logs = {}
    for l in ss.getLogs():
        logs[l.getName()] = l
    self.failUnlessEqual(logs['stdio'].getText(),
                         self.generateText("stdout"))
    return
    self.failUnlessEqual(logs['log2'].getText(),
                         self.generateText("log2"))
    self.failUnlessEqual(logs['log3'].getText(),
                         self.generateText("log3"))




From warner at users.sourceforge.net  Tue Jun 20 08:08:41 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:41 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.668,1.669
Message-ID: <E1FsbHu-0001Ge-1K@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14565

Modified Files:
	ChangeLog 
Log Message:
[project @ add test and docs for the (not-yet implemented) watch-multiple-logfiles feature]

Original author: warner at lothar.com
Date: 2006-06-16 19:57:42

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.668
retrieving revision 1.669
diff -u -d -r1.668 -r1.669
--- ChangeLog	20 Jun 2006 08:08:29 -0000	1.668
+++ ChangeLog	20 Jun 2006 08:08:39 -0000	1.669
@@ -1,5 +1,12 @@
 2006-06-16  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/test_shell.py: new test file to contain everything
+	relating to ShellCommand
+	(SlaveSide.testLogFiles): (todo) test for the upcoming "watch
+	multiple logfiles in realtime" feature, not yet implemented
+	* buildbot/test/emitlogs.py: support file for testLogFiles
+	* docs/buildbot.texinfo (ShellCommand): document the feature
+
 	* buildbot/test/test_steps.py (BuildStep.setUp): rmtree refactoring
 
 	* buildbot/test/runutils.py (SlaveCommandTestBase): utility class




From warner at users.sourceforge.net  Tue Jun 20 08:08:47 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:47 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave bot.py, 1.17,
	1.18 commands.py, 1.52, 1.53
Message-ID: <E1FsbI0-0002a5-7l@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14596/buildbot/slave

Modified Files:
	bot.py commands.py 
Log Message:
[project @ slave: refactor Command startup/completion a bit]

Original author: warner at lothar.com
Date: 2006-06-20 03:55:56

Index: bot.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/bot.py,v
retrieving revision 1.17
retrieving revision 1.18
diff -u -d -r1.17 -r1.18
--- bot.py	8 Feb 2006 20:50:30 -0000	1.17
+++ bot.py	20 Jun 2006 08:08:45 -0000	1.18
@@ -166,8 +166,7 @@
         log.msg(" startCommand:%s [id %s]" % (command,stepId))
         self.remoteStep = stepref
         self.remoteStep.notifyOnDisconnect(self.lostRemoteStep)
-        self.command.running = True
-        d = defer.maybeDeferred(self.command.start)
+        d = self.command.doStart()
         d.addCallback(lambda res: None)
         d.addBoth(self.commandComplete)
         return None
@@ -181,7 +180,7 @@
             # command that wasn't actually running
             log.msg(" .. but none was running")
             return
-        self.command.interrupt()
+        self.command.doInterrupt()
 
 
     def stopCommand(self):
@@ -192,8 +191,7 @@
         if not self.command:
             return
         log.msg("stopCommand: halting current command %s" % self.command)
-        self.command.running = False # shut up!
-        self.command.interrupt() # die!
+        self.command.doInterrupt() # shut up! and die!
         self.command = None # forget you!
 
     # sendUpdate is invoked by the Commands we spawn

Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.52
retrieving revision 1.53
diff -u -d -r1.52 -r1.53
--- commands.py	12 Jun 2006 08:36:08 -0000	1.52
+++ commands.py	20 Jun 2006 08:08:45 -0000	1.53
@@ -350,6 +350,7 @@
                     log.msg("os module is missing the 'kill' function")
                 else:
                     log.msg("trying os.kill(-pid, %d)" % (sig,))
+                    # TODO: maybe use os.killpg instead of a negative pid?
                     os.kill(-self.process.pid, sig)
                     log.msg(" signal %s sent successfully" % sig)
                     hit = 1
@@ -468,11 +469,17 @@
     def setup(self, args):
         """Override this in a subclass to extract items from the args dict."""
         pass
-        
+
+    def doStart(self):
+        self.running = True
+        d = defer.maybeDeferred(self.start)
+        d.addBoth(self.commandComplete)
+        return d
+
     def start(self):
-        """Start the command. self.running will be set just before this is
-        called. This method should return a Deferred that will fire when the
-        command has completed. The Deferred's argument will be ignored.
+        """Start the command. This method should return a Deferred that will
+        fire when the command has completed. The Deferred's argument will be
+        ignored.
 
         This method should be overridden by subclasses."""
         raise NotImplementedError, "You must implement this in a subclass"
@@ -486,12 +493,20 @@
             return
         self.builder.sendUpdate(status)
 
+    def doInterrupt(self):
+        self.running = False
+        self.interrupt()
+
     def interrupt(self):
         """Override this in a subclass to allow commands to be interrupted.
         May be called multiple times, test and set self.interrupted=True if
         this matters."""
         pass
 
+    def commandComplete(self, res):
+        self.running = False
+        return res
+
     # utility methods, mostly used by SlaveShellCommand and the like
 
     def _abandonOnFailure(self, rc):




From warner at users.sourceforge.net  Tue Jun 20 08:08:47 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:47 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.669,1.670
Message-ID: <E1FsbI0-0002Zz-5m@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14596

Modified Files:
	ChangeLog 
Log Message:
[project @ slave: refactor Command startup/completion a bit]

Original author: warner at lothar.com
Date: 2006-06-20 03:55:56

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.669
retrieving revision 1.670
diff -u -d -r1.669 -r1.670
--- ChangeLog	20 Jun 2006 08:08:39 -0000	1.669
+++ ChangeLog	20 Jun 2006 08:08:45 -0000	1.670
@@ -1,3 +1,19 @@
+2006-06-19  Brian Warner  <warner at lothar.com>
+
+	* buildbot/slave/commands.py (Command.doStart): refactor Command
+	startup/completion a bit: now the SlaveBuilder calls doStart(),
+	which is not meant for overridding by subclasses, and doStart()
+	calls start(), which is. Likewise the SlaveBuilder calls
+	doInterrupt(), and subclasses override interrupt(). This also puts
+	responsibility for maintaining .running in Command rather than in
+	SlaveBuilder.
+	(Command.doInterrupt): same
+	(Command.commandComplete): same, this is called when the deferred
+	returned by start() completes.
+	* buildbot/slave/bot.py (SlaveBuilder.remote_startCommand): same
+	(SlaveBuilder.remote_interruptCommand): same
+	(SlaveBuilder.stopCommand): same
+
 2006-06-16  Brian Warner  <warner at lothar.com>
 
 	* buildbot/test/test_shell.py: new test file to contain everything




From warner at users.sourceforge.net  Tue Jun 20 08:08:47 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:47 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test runutils.py,1.8,1.9
Message-ID: <E1FsbI0-0001I5-6N@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14596/buildbot/test

Modified Files:
	runutils.py 
Log Message:
[project @ slave: refactor Command startup/completion a bit]

Original author: warner at lothar.com
Date: 2006-06-20 03:55:56

Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -d -r1.8 -r1.9
--- runutils.py	20 Jun 2006 08:08:30 -0000	1.8
+++ runutils.py	20 Jun 2006 08:08:45 -0000	1.9
@@ -270,7 +270,7 @@
         stepId = 0
         c = cmdclass(self.builder, stepId, args)
         c.running = True
-        d = c.start()
+        d = c.doStart()
         return d
 
     def collectUpdates(self, res):




From warner at users.sourceforge.net  Tue Jun 20 08:08:53 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:53 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test/subdir emit.py,1.1,1.2
Message-ID: <E1FsbI5-0002bH-VW@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test/subdir
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14620/buildbot/test/subdir

Modified Files:
	emit.py 
Log Message:
[project @ tests: run commands from _trial_temp, not buildbot/test/]

Original author: warner at lothar.com
Date: 2006-06-20 03:57:27

Index: emit.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/subdir/emit.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -d -r1.1 -r1.2
--- emit.py	8 Jan 2004 20:05:24 -0000	1.1
+++ emit.py	20 Jun 2006 08:08:51 -0000	1.2
@@ -6,5 +6,6 @@
 sys.stderr.write("this is stderr\n")
 if os.environ.has_key("EMIT_TEST"):
     sys.stdout.write("EMIT_TEST: %s\n" % os.environ["EMIT_TEST"])
+open("log1.out","wt").write("this is log1\n")
 rc = int(sys.argv[1])
 sys.exit(rc)




From warner at users.sourceforge.net  Tue Jun 20 08:08:53 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:53 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.670,1.671
Message-ID: <E1FsbI5-0002bF-VT@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14620

Modified Files:
	ChangeLog 
Log Message:
[project @ tests: run commands from _trial_temp, not buildbot/test/]

Original author: warner at lothar.com
Date: 2006-06-20 03:57:27

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.670
retrieving revision 1.671
diff -u -d -r1.670 -r1.671
--- ChangeLog	20 Jun 2006 08:08:45 -0000	1.670
+++ ChangeLog	20 Jun 2006 08:08:51 -0000	1.671
@@ -1,5 +1,20 @@
 2006-06-19  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/emit.py: write to a logfile in the current
+	directory. We use this to figure out what was used as a basedir
+	rather than looking to see which copy of emit.py gets run, so that
+	we can run the commands from inside _trial_temp rather than inside
+	buildbot/test
+	* buildbot/test/subdir/emit.py: same
+	* buildbot/test/runutils.py (FakeSlaveBuilder): take a 'basedir'
+	argument rather than running from buildbot/test/
+	(SlaveCommandTestBase.setUpBuilder): explicitly set up the Builder
+	rather than using an implicit setUp()
+	* buildbot/test/test_slavecommand.py (ShellBase.setUp): same
+	(ShellBase.testShell1, etc): use explicit path to emit.py instead
+	of assuming that we're running in buildbot/test/ (and that '.' is
+	on our $PATH)
+
 	* buildbot/slave/commands.py (Command.doStart): refactor Command
 	startup/completion a bit: now the SlaveBuilder calls doStart(),
 	which is not meant for overridding by subclasses, and doStart()




From warner at users.sourceforge.net  Tue Jun 20 08:08:53 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:08:53 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test emit.py, 1.1,
	1.2 runutils.py, 1.9, 1.10 test_slavecommand.py, 1.21, 1.22
Message-ID: <E1FsbI5-0002bJ-W7@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14620/buildbot/test

Modified Files:
	emit.py runutils.py test_slavecommand.py 
Log Message:
[project @ tests: run commands from _trial_temp, not buildbot/test/]

Original author: warner at lothar.com
Date: 2006-06-20 03:57:27

Index: emit.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/emit.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -d -r1.1 -r1.2
--- emit.py	8 Jan 2004 20:05:24 -0000	1.1
+++ emit.py	20 Jun 2006 08:08:51 -0000	1.2
@@ -6,5 +6,7 @@
 sys.stderr.write("this is stderr\n")
 if os.environ.has_key("EMIT_TEST"):
     sys.stdout.write("EMIT_TEST: %s\n" % os.environ["EMIT_TEST"])
+open("log1.out","wt").write("this is log1\n")
+
 rc = int(sys.argv[1])
 sys.exit(rc)

Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.9
retrieving revision 1.10
diff -u -d -r1.9 -r1.10
--- runutils.py	20 Jun 2006 08:08:45 -0000	1.9
+++ runutils.py	20 Jun 2006 08:08:51 -0000	1.10
@@ -250,9 +250,9 @@
 
 class FakeSlaveBuilder:
     debug = False
-    def __init__(self, usePTY):
+    def __init__(self, usePTY, basedir):
         self.updates = []
-        self.basedir = findDir()
+        self.basedir = basedir
         self.usePTY = usePTY
 
     def sendUpdate(self, data):
@@ -263,12 +263,15 @@
 
 class SlaveCommandTestBase(SignalMixin):
     usePTY = False
-    def setUp(self):
-        self.builder = FakeSlaveBuilder(self.usePTY)
+
+    def setUpBuilder(self, basedir):
+        if not os.path.exists(basedir):
+            os.mkdir(basedir)
+        self.builder = FakeSlaveBuilder(self.usePTY, basedir)
 
     def startCommand(self, cmdclass, args):
         stepId = 0
-        c = cmdclass(self.builder, stepId, args)
+        self.cmd = c = cmdclass(self.builder, stepId, args)
         c.running = True
         d = c.doStart()
         return d

Index: test_slavecommand.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_slavecommand.py,v
retrieving revision 1.21
retrieving revision 1.22
diff -u -d -r1.21 -r1.22
--- test_slavecommand.py	20 Jun 2006 08:08:11 -0000	1.21
+++ test_slavecommand.py	20 Jun 2006 08:08:51 -0000	1.22
@@ -2,7 +2,7 @@
 
 from twisted.trial import unittest
 from twisted.internet import reactor, interfaces
-from twisted.python import runtime, failure
+from twisted.python import runtime, failure, util
 from buildbot.twcompat import maybeWait
 
 import os, re, sys
@@ -20,10 +20,21 @@
 class ShellBase(SignalMixin):
 
     def setUp(self):
-        self.builder = FakeSlaveBuilder(self.usePTY)
+        self.basedir = "test_slavecommand"
+        if not os.path.isdir(self.basedir):
+            os.mkdir(self.basedir)
+        self.subdir = os.path.join(self.basedir, "subdir")
+        if not os.path.isdir(self.subdir):
+            os.mkdir(self.subdir)
+        self.builder = FakeSlaveBuilder(self.usePTY, self.basedir)
+        self.emitcmd = util.sibpath(__file__, "emit.py")
+        self.subemitcmd = os.path.join(util.sibpath(__file__, "subdir"),
+                                       "emit.py")
+        self.sleepcmd = util.sibpath(__file__, "sleep.py")
 
     def failUnlessIn(self, substring, string):
-        self.failUnless(string.find(substring) != -1)
+        self.failUnless(string.find(substring) != -1,
+                        "'%s' not in '%s'" % (substring, string))
 
     def getfile(self, which):
         got = ""
@@ -64,13 +75,19 @@
         self.assertEquals(got, expected)
         
     def testShell1(self):
-        cmd = sys.executable + " emit.py 0"
+        targetfile = os.path.join(self.basedir, "log1.out")
+        if os.path.exists(targetfile):
+            os.unlink(targetfile)
+        cmd = "%s %s 0" % (sys.executable, self.emitcmd)
         args = {'command': cmd, 'workdir': '.', 'timeout': 60}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
         expected = [('stdout', "this is stdout\n"),
                     ('stderr', "this is stderr\n")]
         d.addCallback(self._checkPass, expected, 0)
+        def _check_targetfile(res):
+            self.failUnless(os.path.exists(targetfile))
+        d.addCallback(_check_targetfile)
         return maybeWait(d)
 
     def _checkPass(self, res, expected, rc):
@@ -78,7 +95,7 @@
         self.checkrc(rc)
 
     def testShell2(self):
-        cmd = [sys.executable, "emit.py", "0"]
+        cmd = [sys.executable, self.emitcmd, "0"]
         args = {'command': cmd, 'workdir': '.', 'timeout': 60}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
@@ -88,7 +105,7 @@
         return maybeWait(d)
 
     def testShellRC(self):
-        cmd = [sys.executable, "emit.py", "1"]
+        cmd = [sys.executable, self.emitcmd, "1"]
         args = {'command': cmd, 'workdir': '.', 'timeout': 60}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
@@ -98,7 +115,7 @@
         return maybeWait(d)
 
     def testShellEnv(self):
-        cmd = sys.executable + " emit.py 0"
+        cmd = "%s %s 0" % (sys.executable, self.emitcmd)
         args = {'command': cmd, 'workdir': '.',
                 'env': {'EMIT_TEST': "envtest"}, 'timeout': 60}
         c = SlaveShellCommand(self.builder, None, args)
@@ -111,13 +128,19 @@
         return maybeWait(d)
 
     def testShellSubdir(self):
-        cmd = sys.executable + " emit.py 0"
+        targetfile = os.path.join(self.basedir, "subdir", "log1.out")
+        if os.path.exists(targetfile):
+            os.unlink(targetfile)
+        cmd = "%s %s 0" % (sys.executable, self.subemitcmd)
         args = {'command': cmd, 'workdir': "subdir", 'timeout': 60}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
         expected = [('stdout', "this is stdout in subdir\n"),
                     ('stderr', "this is stderr\n")]
         d.addCallback(self._checkPass, expected, 0)
+        def _check_targetfile(res):
+            self.failUnless(os.path.exists(targetfile))
+        d.addCallback(_check_targetfile)
         return maybeWait(d)
 
     def testShellMissingCommand(self):
@@ -137,7 +160,7 @@
         # stopped trying.
 
     def testTimeout(self):
-        args = {'command': [sys.executable, "sleep.py", "10"],
+        args = {'command': [sys.executable, self.sleepcmd, "10"],
                 'workdir': '.', 'timeout': 2}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
@@ -157,7 +180,7 @@
         testTimeout.todo = "timeout doesn't appear to work under windows"
 
     def testInterrupt1(self):
-        args = {'command': [sys.executable, "sleep.py", "10"],
+        args = {'command': [sys.executable, self.sleepcmd, "10"],
                 'workdir': '.', 'timeout': 20}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
@@ -183,7 +206,7 @@
         # test the backup timeout. This doesn't work under a PTY, because the
         # transport.loseConnection we do in the timeout handler actually
         # *does* kill the process.
-        args = {'command': [sys.executable, "sleep.py", "5"],
+        args = {'command': [sys.executable, self.sleepcmd, "5"],
                 'workdir': '.', 'timeout': 20}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()
@@ -208,7 +231,7 @@
         # make sure that a) the old command's output doesn't interfere with
         # the new one, and b) the old command's actual termination doesn't
         # break anything
-        args = {'command': [sys.executable, "sleep.py", "5"],
+        args = {'command': [sys.executable, self.sleepcmd, "5"],
                 'workdir': '.', 'timeout': 20}
         c = SlaveShellCommand(self.builder, None, args)
         d = c.start()




From warner at users.sourceforge.net  Tue Jun 20 08:09:00 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:00 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.671,1.672
Message-ID: <E1FsbID-0001KZ-6o@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14657

Modified Files:
	ChangeLog 
Log Message:
[project @ add support for following multiple LogFiles in a ShellCommand]

Original author: warner at lothar.com
Date: 2006-06-20 04:17:18

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.671
retrieving revision 1.672
diff -u -d -r1.671 -r1.672
--- ChangeLog	20 Jun 2006 08:08:51 -0000	1.671
+++ ChangeLog	20 Jun 2006 08:08:58 -0000	1.672
@@ -1,5 +1,45 @@
 2006-06-19  Brian Warner  <warner at lothar.com>
 
+	* buildbot/test/test_shell.py: new test to validate LogFiles
+	* buildbot/test/runutils.py (SlaveCommandTestBase): updates to
+	test LogFiles
+	* buildbot/test/emitlogs.py: enhance to wait for a line on stdin
+	before printing the last batch of lines, to test that the polling
+	logic is working properly
+
+	* buildbot/process/step.py (LoggedRemoteCommand): improve LogFile
+	handling, making it possible to track multiple logs for a single
+	RemoteCommand. The previous single logfile is now known as the
+	'stdio' log.
+	(LoggedRemoteCommand.remoteUpdate): accept key='log' updates
+	(RemoteShellCommand.__init__): accept logfiles=
+	(LoggingBuildStep.startCommand): stdio_log is now one of many
+	(ShellCommand): added logfiles= argument, as well as a class-level
+	.logfiles attribute, which will be merged together to figure out
+	which logfiles should be tracked. The latter maybe be useful for
+	subclasses of ShellCommand which know they will aways produce
+	secondary logfiles in the same location.
+
+	* buildbot/slave/commands.py (ShellCommandPP): add writeStdin()
+	and closeStdin() methods, preparing to make it possible to write
+	to a ShellCommand's stdin at any time, not just at startup. These
+	writes are buffered if the child process hasn't started yet.
+	(LogFileWatcher): new helper class to watch arbitrary logfiles
+	while a ShellCommand runs. This class polls the file every two
+	seconds, and sends back 10k chunks to the buildmaster.
+	(ShellCommand): rename stdin= to initialStdin=, and add
+	keepStdinOpen= and logfiles= to arguments. Set up LogFileWatchers
+	at startup.
+	(ShellCommand.addLogfile): LogFile text is sent in updates with a
+	key of "log" and a value of (logname, data).
+	(SlaveShellCommand): add 'initial_stdin', 'keep_stdin_open', and
+	'logfiles' to the master-visible args dictionary.
+	(SourceBase.doPatch): match s/stdin/initialStdin/ change
+	(CVS.start): same
+	(P4.doVCFull): same
+	* buildbot/test/test_vc.py (Patch.testPatch): same
+
+
 	* buildbot/test/emit.py: write to a logfile in the current
 	directory. We use this to figure out what was used as a basedir
 	rather than looking to see which copy of emit.py gets run, so that




From warner at users.sourceforge.net  Tue Jun 20 08:09:00 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:00 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.53,1.54
Message-ID: <E1FsbID-0001KX-6Q@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14657/buildbot/slave

Modified Files:
	commands.py 
Log Message:
[project @ add support for following multiple LogFiles in a ShellCommand]

Original author: warner at lothar.com
Date: 2006-06-20 04:17:18

Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.53
retrieving revision 1.54
diff -u -d -r1.53 -r1.54
--- commands.py	20 Jun 2006 08:08:45 -0000	1.53
+++ commands.py	20 Jun 2006 08:08:58 -0000	1.54
@@ -1,9 +1,10 @@
 # -*- test-case-name: buildbot.test.test_slavecommand -*-
 
 import os, os.path, re, signal, shutil, types, time
+from stat import ST_CTIME, ST_MTIME, ST_SIZE
 
 from twisted.internet.protocol import ProcessProtocol
-from twisted.internet import reactor, defer
+from twisted.internet import reactor, defer, task
 from twisted.python import log, failure, runtime
 
 from buildbot.twcompat import implements, which
@@ -69,6 +70,21 @@
 
     def __init__(self, command):
         self.command = command
+        self.pending_stdin = ""
+        self.stdin_finished = False
+
+    def writeStdin(self, data):
+        assert not self.stdin_finished
+        if self.connected:
+            self.transport.write(data)
+        else:
+            self.pending_stdin += data
+
+    def closeStdin(self):
+        if self.connected:
+            if self.debug: log.msg(" closing stdin")
+            self.transport.closeStdin()
+        self.stdin_finished = True
 
     def connectionMade(self):
         if self.debug:
@@ -79,10 +95,6 @@
                         (self.transport,))
             self.command.process = self.transport
 
-        if self.command.stdin:
-            if self.debug: log.msg(" writing to stdin")
-            self.transport.write(self.command.stdin)
-
         # TODO: maybe we shouldn't close stdin when using a PTY. I can't test
         # this yet, recent debian glibc has a bug which causes thread-using
         # test cases to SIGHUP trial, and the workaround is to either run
@@ -94,8 +106,12 @@
         # from being sent.
         #if not self.command.usePTY:
 
-        if self.debug: log.msg(" closing stdin")
-        self.transport.closeStdin()
+        if self.pending_stdin:
+            if self.debug: log.msg(" writing to stdin")
+            self.transport.write(self.pending_stdin)
+        if self.stdin_finished:
+            if self.debug: log.msg(" closing stdin")
+            self.transport.closeStdin()
 
     def outReceived(self, data):
         if self.debug:
@@ -117,6 +133,49 @@
         rc = status_object.value.exitCode
         self.command.finished(sig, rc)
 
+class LogFileWatcher:
+    POLL_INTERVAL = 2
+
+    def __init__(self, command, name, logfile):
+        self.command = command
+        self.name = name
+        self.logfile = logfile
+        # we are created before the ShellCommand starts. If the logfile we're
+        # supposed to be watching already exists, record its size and
+        # ctime/mtime so we can tell when it starts to change.
+        self.old_logfile_stats = self.statFile()
+        self.started = False
+
+        # every 2 seconds we check on the file again
+        self.poller = task.LoopingCall(self.poll)
+
+    def start(self):
+        self.poller.start(self.POLL_INTERVAL)
+
+    def stop(self):
+        self.poll()
+        self.poller.stop()
+
+    def statFile(self):
+        if os.path.exists(self.logfile):
+            s = os.stat(self.logfile)
+            return (s[ST_CTIME], s[ST_MTIME], s[ST_SIZE])
+        return None
+
+    def poll(self):
+        if not self.started:
+            s = self.statFile()
+            if s == self.old_logfile_stats:
+                return # not started yet
+            self.f = open(self.logfile, "rb")
+            self.started = True
+        while True:
+            data = self.f.read(10000)
+            if not data:
+                return
+            self.command.addLogfile(self.name, data)
+
+
 class ShellCommand:
     # This is a helper class, used by SlaveCommands to run programs in a
     # child shell.
@@ -128,13 +187,16 @@
     def __init__(self, builder, command,
                  workdir, environ=None,
                  sendStdout=True, sendStderr=True, sendRC=True,
-                 timeout=None, stdin=None, keepStdout=False):
+                 timeout=None, initialStdin=None, keepStdinOpen=False,
+                 keepStdout=False,
+                 logfiles={}):
         """
 
         @param keepStdout: if True, we keep a copy of all the stdout text
                            that we've seen. This copy is available in
                            self.stdout, which can be read after the command
                            has finished.
+
         """
 
         self.builder = builder
@@ -142,6 +204,7 @@
         self.sendStdout = sendStdout
         self.sendStderr = sendStderr
         self.sendRC = sendRC
+        self.logfiles = logfiles
         self.workdir = workdir
         self.environ = os.environ.copy()
         if environ:
@@ -155,7 +218,8 @@
                                          + self.environ['PYTHONPATH'])
                 # this will proceed to replace the old one
             self.environ.update(environ)
-        self.stdin = stdin
+        self.initialStdin = initialStdin
+        self.keepStdinOpen = keepStdinOpen
         self.timeout = timeout
         self.timer = None
         self.keepStdout = keepStdout
@@ -167,10 +231,16 @@
         self.usePTY = self.builder.usePTY
         if runtime.platformType != "posix":
             self.usePTY = False # PTYs are posix-only
-        if stdin is not None:
+        if initialStdin is not None:
             # for .closeStdin to matter, we must use a pipe, not a PTY
             self.usePTY = False
 
+        self.logFileWatchers = []
+        for name,filename in self.logfiles.items():
+            w = LogFileWatcher(self, name,
+                               os.path.join(self.workdir, filename))
+            self.logFileWatchers.append(w)
+
     def __repr__(self):
         return "<slavecommand.ShellCommand '%s'>" % self.command
 
@@ -243,6 +313,12 @@
         log.msg(" " + msg)
         self.sendStatus({'header': msg+"\n"})
 
+        # this will be buffered until connectionMade is called
+        if self.initialStdin:
+            self.pp.writeStdin(self.initialStdin)
+        if not self.keepStdinOpen:
+            self.pp.closeStdin()
+
         # win32eventreactor's spawnProcess (under twisted <= 2.0.1) returns
         # None, as opposed to all the posixbase-derived reactors (which
         # return the new Process object). This is a nuisance. We can make up
@@ -270,17 +346,34 @@
         if self.timeout:
             self.timer = reactor.callLater(self.timeout, self.doTimeout)
 
+        for w in self.logFileWatchers:
+            w.start()
+
+
     def addStdout(self, data):
-        if self.sendStdout: self.sendStatus({'stdout': data})
-        if self.keepStdout: self.stdout += data
-        if self.timer: self.timer.reset(self.timeout)
+        if self.sendStdout:
+            self.sendStatus({'stdout': data})
+        if self.keepStdout:
+            self.stdout += data
+        if self.timer:
+            self.timer.reset(self.timeout)
 
     def addStderr(self, data):
-        if self.sendStderr: self.sendStatus({'stderr': data})
-        if self.timer: self.timer.reset(self.timeout)
+        if self.sendStderr:
+            self.sendStatus({'stderr': data})
+        if self.timer:
+            self.timer.reset(self.timeout)
+
+    def addLogfile(self, name, data):
+        self.sendStatus({'log': (name, data)})
+        if self.timer:
+            self.timer.reset(self.timeout)
 
     def finished(self, sig, rc):
         log.msg("command finished with signal %s, exit code %s" % (sig,rc))
+        for w in self.logFileWatchers:
+             # this will send the final updates
+            w.stop()
         if sig is not None:
             rc = -1
         if self.sendRC:
@@ -395,6 +488,13 @@
         self.failed(TimeoutError("SIGKILL failed to kill process"))
 
 
+    def writeStdin(self, data):
+        self.pp.writeStdin(data)
+
+    def closeStdin(self):
+        self.pp.closeStdin()
+
+
 class Command:
     if implements:
         implements(ISlaveCommand)
@@ -534,37 +634,52 @@
     the following keys:
 
         - ['command'] (required): a shell command to run. If this is a string,
-          it will be run with /bin/sh (['/bin/sh', '-c', command]). If it is a
-          list (preferred), it will be used directly.
-        - ['workdir'] (required): subdirectory in which the command will be run,
-          relative to the builder dir
-        - ['env']: a dict of environment variables to augment/replace os.environ
+                                  it will be run with /bin/sh (['/bin/sh',
+                                  '-c', command]). If it is a list
+                                  (preferred), it will be used directly.
+        - ['workdir'] (required): subdirectory in which the command will be
+                                  run, relative to the builder dir
+        - ['env']: a dict of environment variables to augment/replace
+                   os.environ
+        - ['initial_stdin']: a string which will be written to the command's
+                             stdin as soon as it starts
+        - ['keep_stdin_open']: unless True, the command's stdin will be
+                               closed as soon as initial_stdin has been
+                               written. Set this to True if you plan to write
+                               to stdin after the command has been started.
         - ['want_stdout']: 0 if stdout should be thrown away
         - ['want_stderr']: 0 if stderr should be thrown away
         - ['not_really']: 1 to skip execution and return rc=0
         - ['timeout']: seconds of silence to tolerate before killing command
+        - ['logfiles']: dict mapping LogFile name to the workdir-relative
+                        filename of a local log file. This local file will be
+                        watched just like 'tail -f', and all changes will be
+                        written to 'log' status updates.
 
     ShellCommand creates the following status messages:
         - {'stdout': data} : when stdout data is available
         - {'stderr': data} : when stderr data is available
         - {'header': data} : when headers (command start/stop) are available
+        - {'log': (logfile_name, data)} : when log files have new contents
         - {'rc': rc} : when the process has terminated
     """
 
     def start(self):
         args = self.args
-        sendStdout = args.get('want_stdout', True)
-        sendStderr = args.get('want_stderr', True)
         # args['workdir'] is relative to Builder directory, and is required.
         assert args['workdir'] is not None
         workdir = os.path.join(self.builder.basedir, args['workdir'])
-        timeout = args.get('timeout', None)
 
         c = ShellCommand(self.builder, args['command'],
                          workdir, environ=args.get('env'),
-                         timeout=timeout,
-                         sendStdout=sendStdout, sendStderr=sendStderr,
-                         sendRC=True)
+                         timeout=args.get('timeout', None),
+                         sendStdout=args.get('want_stdout', True),
+                         sendStderr=args.get('want_stderr', True),
+                         sendRC=True,
+                         initialStdin=args.get('initial_stdin'),
+                         keepStdinOpen=args.get('keep_stdin_open'),
+                         logfiles=args.get('logfiles', {}),
+                         )
         self.command = c
         d = self.command.start()
         return d
@@ -573,6 +688,11 @@
         self.interrupted = True
         self.command.kill("command interrupted")
 
+    def writeStdin(self, data):
+        self.command.writeStdin(data)
+
+    def closeStdin(self):
+        self.command.closeStdin()
 
 registerSlaveCommand("shell", SlaveShellCommand, cvs_ver)
 
@@ -879,7 +999,7 @@
         # now apply the patch
         c = ShellCommand(self.builder, command, dir,
                          sendRC=False, timeout=self.timeout,
-                         stdin=diff)
+                         initialStdin=diff)
         self.command = c
         d = c.start()
         d.addCallback(self._abandonOnFailure)
@@ -925,7 +1045,7 @@
                        + ['login'])
             c = ShellCommand(self.builder, command, d,
                              sendRC=False, timeout=self.timeout,
-                             stdin=self.login+"\n")
+                             initialStdin=self.login+"\n")
             self.command = c
             d = c.start()
             d.addCallback(self._abandonOnFailure)
@@ -1566,7 +1686,7 @@
         log.msg(client_spec)
         c = ShellCommand(self.builder, command, self.builder.basedir,
                          environ=env, sendRC=False, timeout=self.timeout,
-                         stdin=client_spec)
+                         initialStdin=client_spec)
         self.command = c
         d = c.start()
         d.addCallback(self._abandonOnFailure)




From warner at users.sourceforge.net  Tue Jun 20 08:09:00 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:00 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test emitlogs.py, 1.1,
	1.2 runutils.py, 1.10, 1.11 test_shell.py, 1.1, 1.2 test_vc.py,
	1.63, 1.64
Message-ID: <E1FsbID-0002eB-Dk@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14657/buildbot/test

Modified Files:
	emitlogs.py runutils.py test_shell.py test_vc.py 
Log Message:
[project @ add support for following multiple LogFiles in a ShellCommand]

Original author: warner at lothar.com
Date: 2006-06-20 04:17:18

Index: emitlogs.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/emitlogs.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -d -r1.1 -r1.2
--- emitlogs.py	20 Jun 2006 08:08:39 -0000	1.1
+++ emitlogs.py	20 Jun 2006 08:08:58 -0000	1.2
@@ -1,15 +1,23 @@
 #! /usr/bin/python
 
-import os, sys, time
+import sys, time
 
-log2 = open("log2", "wt")
-log3 = open("log3", "wt")
+log2 = open("log2.out", "wt")
+log3 = open("log3.out", "wt")
 
-for i in range(3):
-    sys.stdout.write("this is stdout %d\n" % i)
+def write(i):
     log2.write("this is log2 %d\n" % i)
+    log2.flush()
     log3.write("this is log3 %d\n" % i)
-    time.sleep(1)
+    log3.flush()
+    sys.stdout.write("this is stdout %d\n" % i)
+    sys.stdout.flush()
+
+write(0)
+time.sleep(1)
+write(1)
+sys.stdin.read(1)
+write(2)
 
 log2.close()
 log3.close()

Index: runutils.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/runutils.py,v
retrieving revision 1.10
retrieving revision 1.11
diff -u -d -r1.10 -r1.11
--- runutils.py	20 Jun 2006 08:08:51 -0000	1.10
+++ runutils.py	20 Jun 2006 08:08:58 -0000	1.11
@@ -276,16 +276,28 @@
         d = c.doStart()
         return d
 
-    def collectUpdates(self, res):
+    def collectUpdates(self, res=None):
         logs = {}
         for u in self.builder.updates:
             for k in u.keys():
                 if k == "log":
                     logname,data = u[k]
                     oldlog = logs.get(("log",logname), "")
-                    logs[("log",logname)] = oldlog + u[k]
+                    logs[("log",logname)] = oldlog + data
                 elif k == "rc":
                     pass
                 else:
                     logs[k] = logs.get(k, "") + u[k]
         return logs
+
+    def findRC(self):
+        for u in self.builder.updates:
+            if "rc" in u:
+                return u["rc"]
+        return None
+
+    def printStderr(self):
+        for u in self.builder.updates:
+            if "stderr" in u:
+                print u["stderr"]
+

Index: test_shell.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_shell.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -d -r1.1 -r1.2
--- test_shell.py	20 Jun 2006 08:08:39 -0000	1.1
+++ test_shell.py	20 Jun 2006 08:08:58 -0000	1.2
@@ -2,8 +2,10 @@
 
 # test step.ShellCommand and the slave-side commands.ShellCommand
 
-import sys
+import sys, time, os
 from twisted.trial import unittest
+from twisted.internet import reactor, defer
+from twisted.python import util
 from buildbot.process.step import ShellCommand
 from buildbot.slave.commands import SlaveShellCommand
 from buildbot.twcompat import maybeWait
@@ -11,8 +13,10 @@
 
 class SlaveSide(SlaveCommandTestBase, unittest.TestCase):
     def testOne(self):
+        self.setUpBuilder("test_shell.testOne")
+        emitcmd = util.sibpath(__file__, "emit.py")
         args = {
-            'command': [sys.executable, "emit.py", "0"],
+            'command': [sys.executable, emitcmd, "0"],
             'workdir': ".",
             }
         d = self.startCommand(SlaveShellCommand, args)
@@ -32,22 +36,41 @@
         return "".join(lines)
 
     def testLogFiles(self):
-        # emitlogs.py writes one line per second to stdout and two logfiles,
-        # for 3 seconds total.
+        basedir = "test_shell.testLogFiles"
+        self.setUpBuilder(basedir)
+        # emitlogs.py writes two lines to stdout and two logfiles, one second
+        # apart. Then it waits for us to write something to stdin, then it
+        # writes one more line.
+
+        # we write something to the log file first, to exercise the logic
+        # that distinguishes between the old file and the one as modified by
+        # the ShellCommand. We set the timestamp back 5 seconds so that
+        # timestamps can be used to distinguish old from new.
+        log2file = os.path.join(basedir, "log2.out")
+        f = open(log2file, "w")
+        f.write("dummy text\n")
+        f.close()
+        earlier = time.time() - 5
+        os.utime(log2file, (earlier, earlier))
+
         args = {
-            'command': [sys.executable, "emitlogs.py"],
+            'command': [sys.executable,
+                        util.sibpath(__file__, "emitlogs.py")],
             'workdir': ".",
             'logfiles': {"log2": "log2.out",
                          "log3": "log3.out"},
+            'keep_stdin_open': True,
             }
-        d = self.startCommand(SlaveShellCommand, args)
-        # after two seconds, there should be some data in the secondary
-        # logfiles
-
-        # TODO: I want to test that logfiles are being read in a timely
-        # fashion. How can I do this and still have the tests be reliable
-        # under load?
+        finishd = self.startCommand(SlaveShellCommand, args)
+        # The first batch of lines is written immediately. The second is
+        # written after a pause of one second. We poll once per second until
+        # we see both batches.
 
+        self._check_timeout = 10
+        d = self._check_and_wait()
+        def _wait_for_finish(res, finishd):
+            return finishd
+        d.addCallback(_wait_for_finish, finishd)
         d.addCallback(self.collectUpdates)
         def _check(logs):
             self.failUnlessEqual(logs['stdout'], self._generateText("stdout"))
@@ -56,18 +79,35 @@
             self.failUnlessEqual(logs[('log','log3')],
                                  self._generateText("log3"))
         d.addCallback(_check)
+        d.addBoth(self._maybePrintError)
         return maybeWait(d)
-    testLogFiles.todo = "doesn't work yet"
 
+    def _check_and_wait(self, res=None):
+        self._check_timeout -= 1
+        if self._check_timeout <= 0:
+            raise defer.TimeoutError("gave up on command")
+        logs = self.collectUpdates()
+        if logs.get('stdout') == "this is stdout 0\nthis is stdout 1\n":
+            # the emitlogs.py process is now waiting for something to arrive
+            # on stdin
+            self.cmd.command.pp.transport.write("poke\n")
+            return
+        if not self.cmd.running:
+            self.fail("command finished too early")
+        spin = defer.Deferred()
+        spin.addCallback(self._check_and_wait)
+        reactor.callLater(1, spin.callback, None)
+        return spin
+
+    def _maybePrintError(self, res):
+        rc = self.findRC()
+        if rc != 0:
+            print "Command ended with rc=%s" % rc
+            print "STDERR:"
+            self.printStderr()
+        return res
+
+    # MAYBE TODO: a command which appends to an existing logfile should
+    # result in only the new text being sent up to the master. I need to
+    # think about this more first.
 
-def OFF_testLogfiles_1(self, res, ss):
-    logs = {}
-    for l in ss.getLogs():
-        logs[l.getName()] = l
-    self.failUnlessEqual(logs['stdio'].getText(),
-                         self.generateText("stdout"))
-    return
-    self.failUnlessEqual(logs['log2'].getText(),
-                         self.generateText("log2"))
-    self.failUnlessEqual(logs['log3'].getText(),
-                         self.generateText("log3"))

Index: test_vc.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_vc.py,v
retrieving revision 1.63
retrieving revision 1.64
diff -u -d -r1.63 -r1.64
--- test_vc.py	20 Jun 2006 08:08:17 -0000	1.63
+++ test_vc.py	20 Jun 2006 08:08:58 -0000	1.64
@@ -2354,7 +2354,7 @@
             def sendUpdate(self, status):
                 pass
         c = commands.ShellCommand(FakeBuilder(), command, self.workdir,
-                                  sendRC=False, stdin=p0_diff)
+                                  sendRC=False, initialStdin=p0_diff)
         d = c.start()
         d.addCallback(self._testPatch_1)
         return maybeWait(d)




From warner at users.sourceforge.net  Tue Jun 20 08:09:00 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:00 +0000
Subject: [Buildbot-commits] buildbot/docs buildbot.texinfo,1.56,1.57
Message-ID: <E1FsbID-0002e6-CF@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/docs
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14657/docs

Modified Files:
	buildbot.texinfo 
Log Message:
[project @ add support for following multiple LogFiles in a ShellCommand]

Original author: warner at lothar.com
Date: 2006-06-20 04:17:18

Index: buildbot.texinfo
===================================================================
RCS file: /cvsroot/buildbot/buildbot/docs/buildbot.texinfo,v
retrieving revision 1.56
retrieving revision 1.57
diff -u -d -r1.56 -r1.57
--- buildbot.texinfo	20 Jun 2006 08:08:39 -0000	1.56
+++ buildbot.texinfo	20 Jun 2006 08:08:58 -0000	1.57
@@ -3177,12 +3177,12 @@
 their contents afterwards.
 
 The @code{logfiles=} argument allows you to collect data from these
-secondary logfiles in real-time, as the step is running. It accepts a
-dictionary which maps from a local Log name (which is how the log data
-is presented in the build results) to a remote filename (interpreted
-relative to the build's working directory). Each named file will be
-polled on a regular basis (every couple of seconds) as the build runs,
-and any new text will be sent over to the buildmaster.
+secondary logfiles in near-real-time, as the step is running. It
+accepts a dictionary which maps from a local Log name (which is how
+the log data is presented in the build results) to a remote filename
+(interpreted relative to the build's working directory). Each named
+file will be polled on a regular basis (every couple of seconds) as
+the build runs, and any new text will be sent over to the buildmaster.
 
 @example
 s(ShellCommand, command=["make", "test"],




From warner at users.sourceforge.net  Tue Jun 20 08:09:00 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:00 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.92,1.93
Message-ID: <E1FsbID-0001KV-64@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14657/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ add support for following multiple LogFiles in a ShellCommand]

Original author: warner at lothar.com
Date: 2006-06-20 04:17:18

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.92
retrieving revision 1.93
diff -u -d -r1.92 -r1.93
--- step.py	16 Jun 2006 05:27:59 -0000	1.92
+++ step.py	20 Jun 2006 08:08:58 -0000	1.93
@@ -30,7 +30,7 @@
     of reliably gathering status updates from the slave (acknowledging each),
     and (eventually, in a future release) recovering from interrupted builds.
     This is the master-side object that is known to the slave-side
-    L{buildbot.slave.bot.SlaveBuilder}, to which status update are sent.
+    L{buildbot.slave.bot.SlaveBuilder}, to which status updates are sent.
 
     My command should be started by calling .run(), which returns a
     Deferred that will fire when the command has finished, or will
@@ -234,18 +234,18 @@
     """
 
     I am a L{RemoteCommand} which gathers output from the remote command into
-    one or more local log files. These L{buildbot.status.builder.Logfile}
-    instances live in C{self.logs}. If the slave sends back
-    stdout/stderr/header updates, these will be put into
-    C{self.logs['stdio']}, if present. If the remote command uses other log
-    channels, they will go into other entries in C{self.logs}.
+    one or more local log files. My C{self.logs} dictionary contains
+    references to these L{buildbot.status.builder.LogFile} instances. Any
+    stdout/stderr/header updates from the slave will be put into
+    C{self.logs['stdio']}, if it exists. If the remote command uses other log
+    files, they will go into other entries in C{self.logs}.
 
-    If you want to use stdout, you should create a LogFile named 'stdio' and
-    pass it to my useLog() message. Otherwise stdout/stderr will be ignored,
-    which is probably not what you want.
+    If you want to use stdout or stderr, you should create a LogFile named
+    'stdio' and pass it to my useLog() message. Otherwise stdout/stderr will
+    be ignored, which is probably not what you want.
 
-    Unless you tell me otherwise, I will close all logs when the command is
-    complete.
+    Unless you tell me otherwise, when my command completes I will close all
+    the LogFiles that I know about.
 
     @ivar logs: maps logname to a LogFile instance
     @ivar _closeWhenFinished: maps logname to a boolean. If true, this
@@ -292,21 +292,34 @@
         if 'stdio' in self.logs:
             self.logs['stdio'].addHeader(data)
 
+    def addToLog(self, logname, data):
+        if logname in self.logs:
+            self.logs[logname].addStdout(data)
+        else:
+            log.msg("%s.addToLog: no such log %s" % (self, logname))
+
     def remoteUpdate(self, update):
         if self.debug:
             for k,v in update.items():
                 log.msg("Update[%s]: %s" % (k,v))
         if update.has_key('stdout'):
+            # 'stdout': data
             self.addStdout(update['stdout'])
         if update.has_key('stderr'):
+            # 'stderr': data
             self.addStderr(update['stderr'])
         if update.has_key('header'):
+            # 'header': data
             self.addHeader(update['header'])
+        if update.has_key('log'):
+            # 'log': (logname, data)
+            logname, data = update['log']
+            self.addToLog(logname, data)
         if update.has_key('rc'):
             rc = self.rc = update['rc']
             log.msg("%s rc=%s" % (self, rc))
             self.addHeader("program finished with exit code %d\n" % rc)
-        # TODO: other log channels
+
         for k in update:
             if k not in ('stdout', 'stderr', 'header', 'rc'):
                 if k not in self.updates:
@@ -394,7 +407,7 @@
 
     def __init__(self, workdir, command, env=None, 
                  want_stdout=1, want_stderr=1,
-                 timeout=20*60, **kwargs):
+                 timeout=20*60, logfiles={}, **kwargs):
         """
         @type  workdir: string
         @param workdir: directory where the command ought to run,
@@ -434,6 +447,7 @@
                         the command is hung and should be killed. Use
                         None to disable the timeout.
         """
+
         self.command = command # stash .command, set it later
         if env is not None:
             # avoid mutating the original master.cfg dictionary. Each
@@ -485,7 +499,7 @@
     C{self.step_status}. It can also feed progress data (like how much text
     is output by a shell command) to the
     L{buildbot.status.progress.StepProgress} object that lives in
-    C{self.progress}, by calling C{progress.setProgress(metric, value)} as it
+    C{self.progress}, by calling C{self.setProgress(metric, value)} as it
     runs.
 
     @type build: L{buildbot.process.base.Build}
@@ -834,8 +848,9 @@
 
 
 class LoggingBuildStep(BuildStep):
-    # This is an abstract base class, suitable for inheritance by all
-    # BuildSteps that invoke RemoteCommands which emit stdout/stderr messages
+    """This is an abstract base class, suitable for inheritance by all
+    BuildSteps that invoke RemoteCommands which emit stdout/stderr messages.
+    """
 
     progressMetrics = ('output',)
 
@@ -854,12 +869,12 @@
         self.cmd = cmd # so we can interrupt it
         self.step_status.setColor("yellow")
         self.step_status.setText(self.describe(False))
-        loog = self.addLog("stdio")
-        log.msg("ShellCommand.start using log", loog)
+        stdio_log = self.addLog("stdio")
+        log.msg("ShellCommand.start using log", stdio_log)
         log.msg(" for cmd", cmd)
-        cmd.useLog(loog, True)
+        cmd.useLog(stdio_log, True)
         for em in errorMessages:
-            loog.addHeader(em)
+            stdio_log.addHeader(em)
         d = self.runCommand(cmd)
         d.addCallbacks(self._commandComplete, self.checkDisconnect)
         d.addErrback(self.failed)
@@ -1046,6 +1061,7 @@
     description = None # set this to a list of short strings to override
     descriptionDone = None # alternate description when the step is complete
     command = None # set this to a command, or set in kwargs
+    logfiles = {}
 
     # override this on a specific ShellCommand if you want to let it fail
     # without dooming the entire build to a status of FAILURE
@@ -1053,7 +1069,7 @@
 
     def __init__(self, workdir,
                  description=None, descriptionDone=None,
-                 command=None,
+                 command=None, logfiles={},
                  **kwargs):
         # most of our arguments get passed through to the RemoteShellCommand
         # that we create, but first strip out the ones that we pass to
@@ -1066,6 +1082,10 @@
             self.descriptionDone = descriptionDone
         if command:
             self.command = command
+        # merge a class-level 'logfiles' attribute with one passed in as an
+        # argument
+        self.logfiles = self.logfiles.copy()
+        self.logfiles.update(logfiles)
 
         # pull out the ones that BuildStep wants, then upcall
         buildstep_kwargs = {}
@@ -1142,13 +1162,23 @@
             # note that each RemoteShellCommand gets its own copy of the
             # dictionary, so we shouldn't be affecting anyone but ourselves.
 
+    def setupLogfiles(self, cmd, logfiles):
+        if logfiles:
+            for logname,remotefilename in logfiles.items():
+                # tell the BuildStepStatus to add a LogFile
+                newlog = self.addLog(logname)
+                # and tell the LoggedRemoteCommand to feed it
+                cmd.useLog(newlog, True)
+
     def start(self):
         command = self._interpolateProperties(self.command)
         # create the actual RemoteShellCommand instance now
         kwargs = self.remote_kwargs
         kwargs['command'] = command
+        kwargs['logfiles'] = self.logfiles
         cmd = RemoteShellCommand(**kwargs)
         self.setupEnvironment(cmd)
+        self.setupLogfiles(cmd, self.logfiles)
         self.startCommand(cmd)
 
 




From warner at users.sourceforge.net  Tue Jun 20 08:09:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:06 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.672,1.673
Message-ID: <E1FsbII-0002hv-Ve@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14954

Modified Files:
	ChangeLog 
Log Message:
[project @ LogFileWatcher: oops, make it work from real BuildSteps]

Original author: warner at lothar.com
Date: 2006-06-20 05:09:15

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.672
retrieving revision 1.673
diff -u -d -r1.672 -r1.673
--- ChangeLog	20 Jun 2006 08:08:58 -0000	1.672
+++ ChangeLog	20 Jun 2006 08:09:04 -0000	1.673
@@ -1,5 +1,14 @@
 2006-06-19  Brian Warner  <warner at lothar.com>
 
+	* buildbot/slave/commands.py (LogFileWatcher.__init__): note the
+	creation of LogFileWatchers
+	(ShellCommand._startCommand): and record the files that were
+	watched in the 'headers' section of the ShellCommand output
+
+	* buildbot/process/step.py (RemoteShellCommand.__init__): duh, you
+	need to actually pass it to the slave if you want it to work.
+	(ShellCommand): document it a bit
+
 	* buildbot/test/test_shell.py: new test to validate LogFiles
 	* buildbot/test/runutils.py (SlaveCommandTestBase): updates to
 	test LogFiles




From warner at users.sourceforge.net  Tue Jun 20 08:09:06 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:06 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.93,1.94
Message-ID: <E1FsbIJ-0001PQ-5e@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14954/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ LogFileWatcher: oops, make it work from real BuildSteps]

Original author: warner at lothar.com
Date: 2006-06-20 05:09:15

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.93
retrieving revision 1.94
diff -u -d -r1.93 -r1.94
--- step.py	20 Jun 2006 08:08:58 -0000	1.93
+++ step.py	20 Jun 2006 08:09:04 -0000	1.94
@@ -458,6 +458,7 @@
                 'env': env,
                 'want_stdout': want_stdout,
                 'want_stderr': want_stderr,
+                'logfiles': logfiles,
                 'timeout': timeout,
                 }
         LoggedRemoteCommand.__init__(self, "shell", args)
@@ -1055,6 +1056,11 @@
                    This will be used by start() to create a
                    RemoteShellCommand instance.
 
+    @ivar logfiles: a dict mapping log NAMEs to workdir-relative FILENAMEs
+                    of their corresponding logfiles. The contents of the file
+                    named FILENAME will be put into a LogFile named NAME, in
+                    something approximating real-time.
+
     """
 
     name = "shell"




From warner at users.sourceforge.net  Tue Jun 20 08:09:07 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:07 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.54,1.55
Message-ID: <E1FsbIJ-0002iB-9s@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv14954/buildbot/slave

Modified Files:
	commands.py 
Log Message:
[project @ LogFileWatcher: oops, make it work from real BuildSteps]

Original author: warner at lothar.com
Date: 2006-06-20 05:09:15

Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.54
retrieving revision 1.55
diff -u -d -r1.54 -r1.55
--- commands.py	20 Jun 2006 08:08:58 -0000	1.54
+++ commands.py	20 Jun 2006 08:09:04 -0000	1.55
@@ -140,6 +140,7 @@
         self.command = command
         self.name = name
         self.logfile = logfile
+        log.msg("LogFileWatcher created to watch %s" % logfile)
         # we are created before the ShellCommand starts. If the logfile we're
         # supposed to be watching already exists, record its size and
         # ctime/mtime so we can tell when it starts to change.
@@ -303,6 +304,10 @@
         log.msg(" " + msg)
         self.sendStatus({'header': msg+"\n"})
 
+        msg = " watching logfiles %s" % (self.logfiles,)
+        log.msg(" " + msg)
+        self.sendStatus({'header': msg+"\n"})
+
         # then the argv array for resolving unambiguity
         msg = " argv: %s" % (argv,)
         log.msg(" " + msg)




From warner at users.sourceforge.net  Tue Jun 20 08:09:12 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:12 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.673,1.674
Message-ID: <E1FsbIO-0002jF-K9@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15073

Modified Files:
	ChangeLog 
Log Message:
[project @ step_twisted.Trial: use logfiles= to get test.log instead of 'cat']

Original author: warner at lothar.com
Date: 2006-06-20 05:11:41

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.673
retrieving revision 1.674
diff -u -d -r1.673 -r1.674
--- ChangeLog	20 Jun 2006 08:09:04 -0000	1.673
+++ ChangeLog	20 Jun 2006 08:09:10 -0000	1.674
@@ -1,5 +1,10 @@
 2006-06-19  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step_twisted.py (Trial): use logfiles= to track
+	_trial_temp/test.log, not a separate 'cat' command. TODO: this
+	will fail under windows because of os.sep issues. It might have
+	worked before if 'cat' was doing cygwin path conversion.
+
 	* buildbot/slave/commands.py (LogFileWatcher.__init__): note the
 	creation of LogFileWatchers
 	(ShellCommand._startCommand): and record the files that were




From warner at users.sourceforge.net  Tue Jun 20 08:09:12 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:12 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py, 1.80,
	1.81
Message-ID: <E1FsbIO-0001Qw-PL@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15073/buildbot/process

Modified Files:
	step_twisted.py 
Log Message:
[project @ step_twisted.Trial: use logfiles= to get test.log instead of 'cat']

Original author: warner at lothar.com
Date: 2006-06-20 05:11:41

Index: step_twisted.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v
retrieving revision 1.80
retrieving revision 1.81
diff -u -d -r1.80 -r1.81
--- step_twisted.py	15 Jun 2006 05:47:48 -0000	1.80
+++ step_twisted.py	20 Jun 2006 08:09:10 -0000	1.81
@@ -231,6 +231,11 @@
 
     name = "trial"
     progressMetrics = ('output', 'tests')
+    # note: the slash only works on unix buildslaves, of course, but we have
+    # no way to know what the buildslave uses as a separator. TODO: figure
+    # out something clever.
+    logfiles = {"test.log": "_trial_temp/test.log"}
+    # TODO: the text in test.log should feed progressMetrics too
 
     flunkOnFailure = True
     python = None
@@ -440,25 +445,6 @@
         ShellCommand.start(self)
 
 
-    def _commandComplete(self, cmd):
-        # before doing the summary, etc, fetch _trial_temp/test.log
-        # TODO: refactor ShellCommand so I don't have to override such
-        # an internal method
-        catcmd = ["cat", "_trial_temp/test.log"]
-        c2 = step.RemoteShellCommand(command=catcmd,
-                                     workdir=self.workdir,
-                                     )
-        self.cmd = c2
-        loog = self.addLog("test.log")
-        c2.useLog(loog, True)
-        d = c2.run(self, self.remote)
-        d.addCallback(self._commandComplete2, cmd)
-        return d
-
-    def _commandComplete2(self, c2, cmd):
-        # pass the original RemoteShellCommand to the summarizer
-        return ShellCommand._commandComplete(self, cmd)
-
     def rtext(self, fmt='%s'):
         if self.reactor:
             rtext = fmt % self.reactor




From warner at users.sourceforge.net  Tue Jun 20 08:09:18 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:18 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step.py,1.94,1.95
Message-ID: <E1FsbIU-0002kd-AK@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15107/buildbot/process

Modified Files:
	step.py 
Log Message:
[project @ step.py: generalize OutputProgressObserver]

Original author: warner at lothar.com
Date: 2006-06-20 06:21:11

Index: step.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step.py,v
retrieving revision 1.94
retrieving revision 1.95
diff -u -d -r1.94 -r1.95
--- step.py	20 Jun 2006 08:09:04 -0000	1.94
+++ step.py	20 Jun 2006 08:09:15 -0000	1.95
@@ -840,13 +840,15 @@
         return d
 
 
-class StdioProgressObserver(LogObserver):
+class OutputProgressObserver(LogObserver):
     length = 0
 
+    def __init__(self, name):
+        self.name = name
+
     def logChunk(self, build, step, log, channel, text):
         self.length += len(text)
-        self.step.setProgress("output", self.length)
-
+        self.step.setProgress(self.name, self.length)
 
 class LoggingBuildStep(BuildStep):
     """This is an abstract base class, suitable for inheritance by all
@@ -857,7 +859,7 @@
 
     def __init__(self, *args, **kwargs):
         BuildStep.__init__(self, *args, **kwargs)
-        self.addLogObserver('stdio', StdioProgressObserver())
+        self.addLogObserver('stdio', OutputProgressObserver("output"))
 
     def describe(self, done=False):
         raise NotImplementedError("implement this in a subclass")




From warner at users.sourceforge.net  Tue Jun 20 08:09:17 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:17 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.674,1.675
Message-ID: <E1FsbIU-0001SK-68@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15107

Modified Files:
	ChangeLog 
Log Message:
[project @ step.py: generalize OutputProgressObserver]

Original author: warner at lothar.com
Date: 2006-06-20 06:21:11

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.674
retrieving revision 1.675
diff -u -d -r1.674 -r1.675
--- ChangeLog	20 Jun 2006 08:09:10 -0000	1.674
+++ ChangeLog	20 Jun 2006 08:09:15 -0000	1.675
@@ -1,5 +1,10 @@
 2006-06-19  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step.py (OutputProgressObserver): generalize
+	the earlier StdioProgressObserver into an OutputProgressObserver
+	that can track LogFiles other than stdio.
+	(LoggingBuildStep.__init__): same
+
 	* buildbot/process/step_twisted.py (Trial): use logfiles= to track
 	_trial_temp/test.log, not a separate 'cat' command. TODO: this
 	will fail under windows because of os.sep issues. It might have




From warner at users.sourceforge.net  Tue Jun 20 08:09:23 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:23 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.675,1.676
Message-ID: <E1FsbIZ-0001Td-P4@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15124

Modified Files:
	ChangeLog 
Log Message:
[project @ Trial: track Progress from _trial_temp/test.log too]

Original author: warner at lothar.com
Date: 2006-06-20 06:22:23

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.675
retrieving revision 1.676
diff -u -d -r1.675 -r1.676
--- ChangeLog	20 Jun 2006 08:09:15 -0000	1.675
+++ ChangeLog	20 Jun 2006 08:09:21 -0000	1.676
@@ -1,5 +1,8 @@
 2006-06-19  Brian Warner  <warner at lothar.com>
 
+	* buildbot/process/step_twisted.py (Trial): track Progress from
+	_trial_temp/test.log too
+
 	* buildbot/process/step.py (OutputProgressObserver): generalize
 	the earlier StdioProgressObserver into an OutputProgressObserver
 	that can track LogFiles other than stdio.




From warner at users.sourceforge.net  Tue Jun 20 08:09:23 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 08:09:23 +0000
Subject: [Buildbot-commits] buildbot/buildbot/process step_twisted.py, 1.81,
	1.82
Message-ID: <E1FsbIZ-0001Tf-PZ@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/process
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv15124/buildbot/process

Modified Files:
	step_twisted.py 
Log Message:
[project @ Trial: track Progress from _trial_temp/test.log too]

Original author: warner at lothar.com
Date: 2006-06-20 06:22:23

Index: step_twisted.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/process/step_twisted.py,v
retrieving revision 1.81
retrieving revision 1.82
diff -u -d -r1.81 -r1.82
--- step_twisted.py	20 Jun 2006 08:09:10 -0000	1.81
+++ step_twisted.py	20 Jun 2006 08:09:21 -0000	1.82
@@ -230,12 +230,12 @@
     """
 
     name = "trial"
-    progressMetrics = ('output', 'tests')
+    progressMetrics = ('output', 'tests', 'test.log')
     # note: the slash only works on unix buildslaves, of course, but we have
     # no way to know what the buildslave uses as a separator. TODO: figure
     # out something clever.
     logfiles = {"test.log": "_trial_temp/test.log"}
-    # TODO: the text in test.log should feed progressMetrics too
+    # we use test.log to track Progress at the end of __init__()
 
     flunkOnFailure = True
     python = None
@@ -405,8 +405,10 @@
             self.descriptionDone = ["tests"]
 
         # this counter will feed Progress along the 'test cases' metric
-        counter = TrialTestCaseCounter()
-        self.addLogObserver('stdio', counter)
+        self.addLogObserver('stdio', TrialTestCaseCounter())
+        # this one just measures bytes of output in _trial_temp/test.log
+        self.addLogObserver('test.log',
+                            step.OutputProgressObserver('test.log'))
 
     def setupEnvironment(self, cmd):
         ShellCommand.setupEnvironment(self, cmd)




From warner at users.sourceforge.net  Tue Jun 20 16:08:49 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 16:08:49 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.676,1.677
Message-ID: <E1Fsima-0007WB-AT@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv9200

Modified Files:
	ChangeLog 
Log Message:
[project @ test_steps.py: update to include 'logfiles' argument sent to slave]

Original author: warner at lothar.com
Date: 2006-06-20 16:07:53

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.676
retrieving revision 1.677
diff -u -d -r1.676 -r1.677
--- ChangeLog	20 Jun 2006 08:09:21 -0000	1.676
+++ ChangeLog	20 Jun 2006 16:08:46 -0000	1.677
@@ -1,3 +1,8 @@
+2006-06-20  Brian Warner  <warner at lothar.com>
+
+	* buildbot/test/test_steps.py (BuildStep.testShellCommand1): update
+	test to include new 'logfiles' argument sent from master to slave
+
 2006-06-19  Brian Warner  <warner at lothar.com>
 
 	* buildbot/process/step_twisted.py (Trial): track Progress from




From warner at users.sourceforge.net  Tue Jun 20 16:08:49 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Tue, 20 Jun 2006 16:08:49 +0000
Subject: [Buildbot-commits] buildbot/buildbot/test test_steps.py,1.21,1.22
Message-ID: <E1Fsima-0007WD-I2@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/test
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv9200/buildbot/test

Modified Files:
	test_steps.py 
Log Message:
[project @ test_steps.py: update to include 'logfiles' argument sent to slave]

Original author: warner at lothar.com
Date: 2006-06-20 16:07:53

Index: test_steps.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/test/test_steps.py,v
retrieving revision 1.21
retrieving revision 1.22
diff -u -d -r1.21 -r1.22
--- test_steps.py	20 Jun 2006 08:08:32 -0000	1.21
+++ test_steps.py	20 Jun 2006 16:08:47 -0000	1.22
@@ -120,6 +120,7 @@
                                  'workdir': "murkle",
                                  'want_stdout': 1,
                                  'want_stderr': 1,
+                                 'logfiles': {},
                                  'timeout': 10,
                                  'env': None}) ] )
         self.assertEqual(self.remote.events, expectedEvents)




From warner at users.sourceforge.net  Wed Jun 28 17:42:18 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 28 Jun 2006 17:42:18 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.677,1.678
Message-ID: <E1Fve3R-00020L-QP@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22924

Modified Files:
	ChangeLog 
Log Message:
[project @ scripts.runner: remove obsolete mention of mktap]

Original author: warner at lothar.com
Date: 2006-06-28 17:37:39

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.677
retrieving revision 1.678
diff -u -d -r1.677 -r1.678
--- ChangeLog	20 Jun 2006 16:08:46 -0000	1.677
+++ ChangeLog	28 Jun 2006 17:42:16 -0000	1.678
@@ -1,3 +1,8 @@
+2006-06-28  Brian Warner  <warner at lothar.com>
+
+	* buildbot/scripts/runner.py (SlaveOptions.longdesc): remove
+	obsolete reference to mktap.
+
 2006-06-20  Brian Warner  <warner at lothar.com>
 
 	* buildbot/test/test_steps.py (BuildStep.testShellCommand1): update




From warner at users.sourceforge.net  Wed Jun 28 17:42:18 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 28 Jun 2006 17:42:18 +0000
Subject: [Buildbot-commits] buildbot/buildbot/scripts runner.py,1.42,1.43
Message-ID: <E1Fve3R-00020N-RE@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/scripts
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22924/buildbot/scripts

Modified Files:
	runner.py 
Log Message:
[project @ scripts.runner: remove obsolete mention of mktap]

Original author: warner at lothar.com
Date: 2006-06-28 17:37:39

Index: runner.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/scripts/runner.py,v
retrieving revision 1.42
retrieving revision 1.43
diff -u -d -r1.42 -r1.43
--- runner.py	12 Mar 2006 11:54:00 -0000	1.42
+++ runner.py	28 Jun 2006 17:42:16 -0000	1.43
@@ -221,12 +221,11 @@
     This command creates a buildslave working directory and buildbot.tac
     file. The bot will use the <name> and <passwd> arguments to authenticate
     itself when connecting to the master. All commands are run in a
-    build-specific subdirectory of <basedir>, which defaults to the working
-    directory that mktap was run from. <master> is a string of the form
-    'hostname:port', and specifies where the buildmaster can be reached.
+    build-specific subdirectory of <basedir>. <master> is a string of the
+    form 'hostname:port', and specifies where the buildmaster can be reached.
 
     <name>, <passwd>, and <master> will be provided by the buildmaster
-    administrator for your bot.
+    administrator for your bot. You must choose <basedir> yourself.
     """
 
     def getSynopsis(self):




From warner at users.sourceforge.net  Wed Jun 28 17:42:30 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 28 Jun 2006 17:42:30 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.678,1.679
Message-ID: <E1Fve3b-0000Lq-IP@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22948

Modified Files:
	ChangeLog 
Log Message:
[project @ bot.py: add minor log message]

Original author: warner at lothar.com
Date: 2006-06-28 17:39:57

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.678
retrieving revision 1.679
diff -u -d -r1.678 -r1.679
--- ChangeLog	28 Jun 2006 17:42:16 -0000	1.678
+++ ChangeLog	28 Jun 2006 17:42:27 -0000	1.679
@@ -1,5 +1,8 @@
 2006-06-28  Brian Warner  <warner at lothar.com>
 
+	* buildbot/slave/bot.py (SlaveBuilder.commandComplete): add minor
+	log message if the step was shut down
+
 	* buildbot/scripts/runner.py (SlaveOptions.longdesc): remove
 	obsolete reference to mktap.
 




From warner at users.sourceforge.net  Wed Jun 28 17:42:30 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 28 Jun 2006 17:42:30 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave bot.py,1.18,1.19
Message-ID: <E1Fve3e-00023I-3R@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv22948/buildbot/slave

Modified Files:
	bot.py 
Log Message:
[project @ bot.py: add minor log message]

Original author: warner at lothar.com
Date: 2006-06-28 17:39:57

Index: bot.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/bot.py,v
retrieving revision 1.18
retrieving revision 1.19
diff -u -d -r1.18 -r1.19
--- bot.py	20 Jun 2006 08:08:45 -0000	1.18
+++ bot.py	28 Jun 2006 17:42:28 -0000	1.19
@@ -242,6 +242,7 @@
             log.msg("SlaveBuilder.commandComplete", self.command)
         self.command = None
         if not self.running:
+            log.msg(" but we weren't running, quitting silently")
             return
         if self.remoteStep:
             self.remoteStep.dontNotifyOnDisconnect(self.lostRemoteStep)




From warner at users.sourceforge.net  Wed Jun 28 20:05:48 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 28 Jun 2006 20:05:48 +0000
Subject: [Buildbot-commits] buildbot/buildbot/slave commands.py,1.55,1.56
Message-ID: <E1FvgIK-0002UK-Fl@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot/buildbot/slave
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv21514/buildbot/slave

Modified Files:
	commands.py 
Log Message:
[project @ SVN: add --non-interactive to all spawned commands]

Original author: warner at lothar.com
Date: 2006-06-28 20:03:44

Index: commands.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/slave/commands.py,v
retrieving revision 1.55
retrieving revision 1.56
diff -u -d -r1.55 -r1.56
--- commands.py	20 Jun 2006 08:09:04 -0000	1.55
+++ commands.py	28 Jun 2006 20:05:46 -0000	1.56
@@ -1129,7 +1129,8 @@
         revision = self.args['revision'] or 'HEAD'
         # update: possible for mode in ('copy', 'update')
         d = os.path.join(self.builder.basedir, self.srcdir)
-        command = [self.vcexe, 'update', '--revision', str(revision)]
+        command = [self.vcexe, 'update', '--revision', str(revision),
+                   '--non-interactive']
         c = ShellCommand(self.builder, command, d,
                          sendRC=False, timeout=self.timeout,
                          keepStdout=True)
@@ -1141,10 +1142,12 @@
         d = self.builder.basedir
         if self.mode == "export":
             command = [self.vcexe, 'export', '--revision', str(revision),
+                       '--non-interactive',
                        self.svnurl, self.srcdir]
         else:
             # mode=='clobber', or copy/update on a broken workspace
             command = [self.vcexe, 'checkout', '--revision', str(revision),
+                       '--non-interactive',
                        self.svnurl, self.srcdir]
         c = ShellCommand(self.builder, command, d,
                          sendRC=False, timeout=self.timeout,




From warner at users.sourceforge.net  Wed Jun 28 20:05:48 2006
From: warner at users.sourceforge.net (Brian Warner)
Date: Wed, 28 Jun 2006 20:05:48 +0000
Subject: [Buildbot-commits] buildbot ChangeLog,1.679,1.680
Message-ID: <E1FvgIK-0003na-4Z@mail.sourceforge.net>

Update of /cvsroot/buildbot/buildbot
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv21514

Modified Files:
	ChangeLog 
Log Message:
[project @ SVN: add --non-interactive to all spawned commands]

Original author: warner at lothar.com
Date: 2006-06-28 20:03:44

Index: ChangeLog
===================================================================
RCS file: /cvsroot/buildbot/buildbot/ChangeLog,v
retrieving revision 1.679
retrieving revision 1.680
diff -u -d -r1.679 -r1.680
--- ChangeLog	28 Jun 2006 17:42:27 -0000	1.679
+++ ChangeLog	28 Jun 2006 20:05:45 -0000	1.680
@@ -1,5 +1,9 @@
 2006-06-28  Brian Warner  <warner at lothar.com>
 
+	* buildbot/slave/commands.py (SVN): add --non-interactive to all
+	svn commands, so it will fail immediately instead of hanging while
+	it waits for a username/password to be typed in.
+
 	* buildbot/slave/bot.py (SlaveBuilder.commandComplete): add minor
 	log message if the step was shut down