[Buildbot-devel] FW: Buildbot-devel digest, Vol 1 #334 - 7 msgs
Li Chao
lchao at idengines.com
Fri Mar 31 20:51:09 UTC 2006
Hello, All,
I am new to buildbot. I create buildmaster and buildbot slave on the
same host and write some simple step to do the svn update only. My code
is like:
source=s(step.SVN, mode='update',timeout=5*60, retry=(10,3),
baseURL='http://svn/svn/jupiter/trunk/', defaultBranch='releng')
Then when I start it, it will check out the code in the "$Basedir/build"
directory. I have two questions.
1. My svn mode is update. How come every time it deleted the "build"
directory and check out the new code again instead of updating the
existing code? I don't have a shell command "rm" to delete the "build"
directory.
2. The build failed every time. It successfully checked out the code,
but it showed "Fail to kill the process". You know what the problem is .
The logfile looks like:
starting svn operation
rm -rf
/amd/xserve/Volumes/RAID0/Home/releng/buildbot/jupiter_slave2/updatecomp
ile/build
in dir
/amd/xserve/Volumes/RAID0/Home/releng/buildbot/jupiter_slave2/updatecomp
ile (timeout 300 secs)
argv: ['rm', '-rf',
'/amd/xserve/Volumes/RAID0/Home/releng/buildbot/jupiter_slave2/updatecom
pile/build']
environment: {.........}
svn checkout --revision HEAD http://svn/svn/jupiter/trunk/releng build
in dir
/amd/xserve/Volumes/RAID0/Home/releng/buildbot/jupiter_slave2/updatecomp
ile (timeout 300 secs)
argv: ['svn', 'checkout', '--revision', 'HEAD',
'http://svn/svn/jupiter/trunk/releng', 'build']
environment: {...}
A build/bin
A build/bin/cron
.....
....
..
Checked out revision 5993.
command timed out: 300 seconds without output, killing pid 3230
SIGKILL failed to kill process
remoteFailed: [Failure instance: Traceback from remote host -- Traceback
(most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill
process
]
Thanks a million.
Victor
-----Original Message-----
From: buildbot-devel-admin at lists.sourceforge.net
[mailto:buildbot-devel-admin at lists.sourceforge.net] On Behalf Of
buildbot-devel-request at lists.sourceforge.net
Sent: Thursday, March 30, 2006 8:24 PM
To: buildbot-devel at lists.sourceforge.net
Subject: Buildbot-devel digest, Vol 1 #334 - 7 msgs
Send Buildbot-devel mailing list submissions to
buildbot-devel at lists.sourceforge.net
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/buildbot-devel
or, via email, send a message with subject or body 'help' to
buildbot-devel-request at lists.sourceforge.net
You can reach the person managing the list at
buildbot-devel-admin at lists.sourceforge.net
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Buildbot-devel digest..."
Today's Topics:
1. google pays a visit... (John Pye)
2. svn_watcher.py (Niklaus Giger)
3. Re: google pays a visit... (Stephen Thorne)
4. [PATCH] fix for issue 1109469 : make buildbot.bat on windows use
python install directory (Lieven Govaerts)
5. [PATCH] fix for try-client so it works on Windows (Lieven
Govaerts)
6. Re: Dynamically updating text displayed in build step status (Grig
Gheorghiu)
7. Antw: svn_watcher.py (with attachement) (Niklaus Giger)
--__--__--
Message: 1
Date: Thu, 30 Mar 2006 16:44:12 +1100
From: John Pye <john.pye at student.unsw.edu.au>
To: buildbot-devel at lists.sourceforge.net
Subject: [Buildbot-devel] google pays a visit...
Hi all,
Two questions here:
Google came an visited my Buildbot today. I don't know if that's the
cause but it also seems to have triggered a few builds. Or maybe it was
wayward travellers...
Is there any way that the default setup of BuildBot could be set to
exclude robots from most of the pages of BuildBot? Like with a
'robots.txt' file?
Or does that require that I dig in and get the Apache Proxy
configuration sorted out, then use Apache to control that stuff. There's
really no need for Google, Yahoo etc to pull down all my config & build
log files, for example.
Secondly:
Where do I change the text that appears on the Force Build page. For
example the name of the different buildslaves, and who runs them, etc. I
checked the manual but I didn't see anything about this there.
Cheers
JP
--
John Pye
School of Mechanical and Manufacturing Engineering
The University of New South Wales
Sydney NSW 2052 Australia
t +61 2 9385 5127
f +61 2 9663 1222
mailto:john.pye_AT_student_DOT_unsw.edu.au
http://pye.dyndns.org/
--__--__--
Message: 2
To: buildbot-devel at lists.sourceforge.net
From: "Niklaus Giger" <Niklaus.Giger at netstal.com>
Date: Thu, 30 Mar 2006 11:08:30 +0200
Subject: [Buildbot-devel] svn_watcher.py
Hi
First I would like to thanks to all who contributed to buildbot. I was
able to
setup and use in under GNU/Debian Linux and Windows without problems
and will try to use it regularly.
I took the svn.py script which John Pye put on this list about two weeks
ago and
improved it a little bit:
- It runs now also on Windows
- Can really be used as a watcher for a Subversion repository
- Options for repository and master are passed on the command line, e.g.
invoke it with:
svn_watcher.py https://server/path/to/repo localhost:9989 watch
Is it possible to add this script to the contrib subdirectory?
Best regards
--__--__--
Message: 3
Date: Thu, 30 Mar 2006 16:40:01 +1000
From: "Stephen Thorne" <stephen.thorne at gmail.com>
To: BuildBot <buildbot-devel at lists.sourceforge.net>
Subject: Re: [Buildbot-devel] google pays a visit...
On 3/30/06, John Pye <john.pye at student.unsw.edu.au> wrote:
> Is there any way that the default setup of BuildBot could be set to
> exclude robots from most of the pages of BuildBot? Like with a
> 'robots.txt' file?
>
> Or does that require that I dig in and get the Apache Proxy
> configuration sorted out, then use Apache to control that stuff.
There's
> really no need for Google, Yahoo etc to pull down all my config &
build
> log files, for example.
having a file called 'robots.txt', and then hacking
buildbot.status.html.StatusResource.getChild to have
if path =3D=3D "robots.txt":
return static.File('pathto/robots.txt')
User-agent: *
Disallow: /
Yeah. something like that. icky and evil I know. :) Bob? Got a better
solut=
ion?
> Where do I change the text that appears on the Force Build page. For
> example the name of the different buildslaves, and who runs them, etc.
I
> checked the manual but I didn't see anything about this there.
They're stored in textfiles in the info directory where your buildslaves
ar=
e:
shiny:~/slaves/twisted buildbot$ cat info/admin
Stephen 'Jerub' Thorne <stephen at thorne.id.au>
shiny:~/slaves/twisted buildbot$ cat info/host
OS-X 10.4.<latest>
Python2.4.2 from fink
--
Stephen Thorne
Development Engineer
--__--__--
Message: 4
From: "Lieven Govaerts" <lgo at mobsol.be>
To: <buildbot-devel at lists.sourceforge.net>
Date: Thu, 30 Mar 2006 22:36:09 +0200
Subject: [Buildbot-devel] [PATCH] fix for issue 1109469 : make
buildbot.bat on windows use python install directory
This is a multi-part message in MIME format.
------=_NextPart_000_00C9_01C6544A.5752B8F0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Hi,
I'm trying to get the buildbot client tools working on windows, and
encountered some problems while doing so.
I'll try to send a patch for each of the issues I encounter and am able
resolve. I'm not a Python expert, so feel free to improve these patches
where needed.
Attached patch solves issue 1109449, by replacing the path c:\python23\
with
%PYTHONDIR%\ which I expect is automatically set by the python
installer.
I didn't find a description on how to send in patches, so I used the
default
subversion style (based on my own local svn repository).
regards,
Lieven.
[[[
Fix for issue #1109449: make buildbot.bat on windows use python install
directory
* contrib/windows/buildbot.bat: replace the path c:\python23\ with
%PYTHONDIR%\
]]]
------=_NextPart_000_00C9_01C6544A.5752B8F0
Content-Type: text/plain;
name="issue-1109449.patch.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="issue-1109449.patch.txt"
Index: contrib/windows/buildbot.bat=0A=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A=
--- contrib/windows/buildbot.bat (revision 34)=0A=
+++ contrib/windows/buildbot.bat (working copy)=0A=
@@ -1,2 +1,2 @@=0A=
- at python C:\Python23\Scripts\buildbot %*=0A=
+ at python %PYTHONDIR%\Scripts\buildbot %*=0A=
=0A=
------=_NextPart_000_00C9_01C6544A.5752B8F0--
--__--__--
Message: 5
From: "Lieven Govaerts" <lgo at mobsol.be>
To: <buildbot-devel at lists.sourceforge.net>
Date: Thu, 30 Mar 2006 22:57:30 +0200
Subject: [Buildbot-devel] [PATCH] fix for try-client so it works on
Windows
This is a multi-part message in MIME format.
------=_NextPart_000_00CF_01C6544D.52DF2AD0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Hi,
I had to make some changes in the try client code to make it work on
Windows. Attached patch contain those changes.
Basically two issues had to be solved:
1. Find the location of 'svn' in the PATH and execute it from that
location.
2. Accept --try-topdir and --try-topfile params at the command-line.
With this patch the try client is now running fine, using the PB
approach to
a server (running Linux).
Similar as my previous patch, I created this from my own local svn
repository, so the format may not be 100% correct.
regards,
Lieven.
[[[
Fix to make the try-client work on Windows.
* buildbot/scripts/runner.py: accept --try-topdir and --try-topfile
parameters.
* buildbot/scripts/tryclient.py: if on Windows then search for the
executable in PATH.
]]]
------=_NextPart_000_00CF_01C6544D.52DF2AD0
Content-Type: text/plain;
name="issue-try-windows.patch.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="issue-try-windows.patch.txt"
Index: buildbot/scripts/runner.py=0A=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A=
--- buildbot/scripts/runner.py (revision 34)=0A=
+++ buildbot/scripts/runner.py (working copy)=0A=
@@ -599,6 +599,11 @@=0A=
=0A=
["builder", "b", None,=0A=
"Run the trial build on this Builder. Can be used multiple =
times."],=0A=
+ =0A=
+ ["try-topfile", None, None, "Filename that indicates we found =
the top dir"],=0A=
+=0A=
+ ["try-topdir", None, None, "Dirname that indicates we found the
=
top dir"],=0A=
+ =0A=
]=0A=
=0A=
optFlags =3D [=0A=
Index: buildbot/scripts/tryclient.py=0A=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A=
--- buildbot/scripts/tryclient.py (revision 34)=0A=
+++ buildbot/scripts/tryclient.py (working copy)=0A=
@@ -5,6 +5,7 @@=0A=
from twisted.spread import pb=0A=
from twisted.cred import credentials=0A=
from twisted.python import log=0A=
+from twisted.python.procutils import which=0A=
=0A=
from buildbot.sourcestamp import SourceStamp=0A=
from buildbot.scripts import runner=0A=
@@ -18,7 +19,11 @@=0A=
self.branch =3D branch=0A=
=0A=
def do(self, cmd):=0A=
- return utils.getProcessOutput(cmd[0], cmd[1:],
env=3Dos.environ,=0A=
+ if sys.platform =3D=3D 'win32':=0A=
+ exe =3D which(cmd[0])[0]=0A=
+ else:=0A=
+ exe =3D cmd[0]=0A=
+ return utils.getProcessOutput(exe, cmd[1:],
env=3Dos.environ,=0A=
path=3Dself.treetop)=0A=
def get(self):=0A=
"""Return a Deferred that fires with a SourceStamp
instance."""=0A=
------=_NextPart_000_00CF_01C6544D.52DF2AD0--
--__--__--
Message: 6
Date: Thu, 30 Mar 2006 14:26:12 -0800 (PST)
From: Grig Gheorghiu <grig at gheorghiu.net>
Reply-To: grig at gheorghiu.net
To: buildbot-devel at lists.sourceforge.net
Subject: [Buildbot-devel] Re: Dynamically updating text displayed in
build step status
One more question:
Here's a fragment of code from my master.cfg file:
c['builders'] = []
for client_bot in client_bots:
client_name = client_bot[0]
client_type = client_types[client_name]
client_uninstall_step = s(ClientUninstall, command="python
client_uninstall.py", timeout=3600)
client_install_step = s(ClientInstall, command="python
client_install.py", timeout=3600)
client_smoke_test_step = s(ClientSmokeTests, command="python
run_smoke_tests.py", timeout=3600)
client_test_factory = factory.BuildFactory([
client_uninstall_step,
client_install_step,
client_smoke_test_step,
])
builder = {
'name': client_type,
'slavename': client_name,
'builddir': client_type.lower(),
'factory': client_test_factory,
}
c['builders'].append(builder)
All the different classes specified in the step object definition
(ClientInstall, ClientUninstall, etc.) are derived from ShellCommand.
The ClientInstall class is the one I described in my original message
(see below).
Although I instantiate new step objects and factory objects for each of
the builders, my suspicion is that a single step instance is shared
among all the builders. I say this because if I leave this code in
ClientInstall.getText:
def getText(self, cmd, results):
text = self.describe(True)
if results == WARNINGS:
text += ["warnings"]
if results == FAILURE:
text += ["failed"]
if self.version:
text += ["version=" + self.version]
return text
then the text "version=x.y.z" gets printed multiple times in each of
the ClientInstall status boxes, once for each of the builders. I end up
with something like this in every box:
client install
version=3.5.0.371
version=3.5.0.371
version=3.5.0.371
version=3.5.0.371
version=3.5.0.371
version=3.5.0.367
version=3.5.0.371
version=3.5.0.371
Is there a way I can make sure that each builder has its own instance
of the ClientInstall class, so that they don't trample on each other?
Again, I might be missing something here in terms of my understanding
of how everything works together, so I'd appreciate if anybody could
shed some light.
Grig
--- Grig Gheorghiu <grig at gheorghiu.net> wrote:
> Hi,
>
> I've been trying to find ways to dynamically update the text that is
> displayed in the build step status in the HTML Waterfall page.
>
> For static customization of the text, I just subclassed from
> ShellCommand and I set 'descriptionDone' to my custom text.
>
> However, I also wanted to be able to display a piece of information
> (such as a version number) that was computed by the slave running
> that
> build step. I looked through the mailing list, but I couldn't find
> something that covered my exact situation, so I came up with the
> following:
>
> Assume there's a build step where the slave installs some package and
> identifies its version number. I want to be able to display that
> version number in the status for that build step.
>
> I defined the following subclass of ShellCommand:
>
> class ClientInstall(ShellCommand):
> name = "client install"
> description = ["running %s" % name]
> descriptionDone = [name]
>
> def __init__(self, **kwargs):
> ShellCommand.__init__(self, **kwargs)
> self.version = None
>
> def createSummary(self, log):
> log_text = log.getText()
> s = re.search("--version=(.*)", log_text)
> if s:
> self.version = s.group(1)
>
> def getText(self, cmd, results):
> text = self.describe(True)
> for elem in text:
> if re.search("version", elem):
> return text
> if results == WARNINGS:
> text += ["warnings"]
> if results == FAILURE:
> text += ["failed"]
> if self.version:
> text += ["version=" + self.version]
> return text
>
>
> I'm retrieving the version number via a regular expression match from
> the slave's log file. Then I'm appending this version to the list
> returned by getText.
>
> Note that I had to add this condition inside getText:
>
> for elem in text:
> if re.search("version", elem):
> return text
>
> otherwise the my custom addition would appear several times in the
> status text (which means that getText got called repeatedly).
>
> I'm using my custom ClientInstall class as one of the steps defined
> in
> master.cfg.
>
> Question: am I on the right track here, or am I needlessly
> complicating
> my life?
>
> Thanks,
>
> Grig Gheorghiu
>
--__--__--
Message: 7
To: buildbot-devel at lists.sourceforge.net
From: "Niklaus Giger" <Niklaus.Giger at netstal.com>
Date: Fri, 31 Mar 2006 06:06:07 +0200
Subject: [Buildbot-devel] Antw: svn_watcher.py (with attachement)
Sorry forgot to attach the script
>>> Niklaus Giger<Niklaus.Giger at netstal.com> 30.03.06 11:08:30 >>>
>Hi
>
>First I would like to thanks to all who contributed to buildbot. I was
able to
>setup and use in under GNU/Debian Linux and Windows without problems
>and will try to use it regularly.
>
>I took the svn.py script which John Pye put on this list about two
weeks ago and
>improved it a little bit:
>- It runs now also on Windows
>- Can really be used as a watcher for a Subversion repository
>- Options for repository and master are passed on the command line,
e.g.
>invoke it with:
>svn_watcher.py https://server/path/to/repo localhost:9989 watch
>
>Is it possible to add this script to the contrib subdirectory?
Best regards
Niklaus Giger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20060331/3aac2c57/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: svn_watcher.py
Type: application/octet-stream
Size: 2560 bytes
Desc: svn_watcher.py
URL: <http://buildbot.net/pipermail/devel/attachments/20060331/3aac2c57/attachment.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ATT38053.txt
URL: <http://buildbot.net/pipermail/devel/attachments/20060331/3aac2c57/attachment.txt>
More information about the devel
mailing list