[Buildbot-devel] understanding buildbot & writing custom 'process'

[Jeroen Janssen]

Stephen Davis wrote:

>> *) I want to seperate per product testing, this way I can choose how 
>> to configure the test machines (a different PC per 
>> increment/product/whatever), instead of having 'one' testSuite that 
>> tests all products.
>
>
> That fits in pretty well.  The "change" that triggers your build is a 
> successful completion of a "real" build.  I'm not sure I'm 
> understanding how you differentiate the "increments" but that 
> shouldn't matter for what I say below.

I'll come back to this at the end.

>> *) I will need to write a 'replacement' for cvs/mail 'changes' in 
>> order to parse the build log and provide decent information to 
>> buildbot (but this might be able to wait a little, because I can 
>> manually trigger a slave to start doing it's work if I'm correct)
>
>
> If you can run a Python script to parse incoming mail messages then 
> you're golden because you can just adapt 
> buildbot/contrib/fakechange.py to connect directly to the build master 
> and trigger the build.  Also, look at buildbot/changes/mail.py for 
> some parsing examples.

Ok, I think that should do it.

>> I found that there is a seperate 'process' for twisted in the 
>> buildbot tree, that I'm currently digging into in order to understand 
>> how to extend buildbot for the above purposes.
>
>
> Those are just customizations of the base classes for Twisted.  Since 
> Twisted was the first customer for the buildbot, its customizations 
> got included into the package.  They are handy as examples though.  :-)

Yes, they are.. although I imagine that some part of it could be really 
promoted to 'generic' buildbot classes/behaviour (especially the 
unittestresult parsing is something I'm going to 'reuse' for my project)

>> I would like to know if my current ideas on how to deal with this 
>> sound ok, and maybe some hints on how other people currently deal 
>> with this :)
>
>
> You're not really doing anything different other than skipping the 
> actual "build" step so this doesn't seem that unusual to me.  In fact, 
> the buildbot doesn't care what steps it runs so it doesn't have to do 
> "builds" at all.  That's a pretty important point to keep in mind when 
> doing "unusual" setups -- it doesn't _have_ to follow an "update, 
> build, test" model.
>
> A tip for Windows.  You'll want to download the win32all python 
> package (I forget from where, just google for it) and, when the 
> buildbot docs say "twistd -f buildbot.tap...", you'll want to add "-r 
> win32".  This tells Twisted to use the custom Win32 "reactor" which is 
> a fancy way of saying "event loop".  :-)
>
> Also, the bots don't seem to shut down cleanly on Windows when you hit 
> ctrl-c.  I haven't bothered to try and figure out what the issue is 
> since I hardly shut that bot down anymore now that it's configured 
> correctly.

Again, thanks very much for this feedback.

Now about the 'increments' part, I think that can be best explained like 
this :
I want to use the same buildbot master to control slaves for different 
sources. Imagine to use the same buildbot for both the twisted and the 
glib projects.
Change notifications for twisted should only trigger 'twisted builders' 
(slaves) and the same goes for the glib ones.

Can this currently be done or not with buildbot?
---

        Best regards,
                
                Jeroen Janssen

+++
The Monks of Cool, whose tiny and exclusive monastery is hidden in a really
cool and laid-back valley in the lower Ramtops, have a passing-out test for a
novice. He is taken into a room full of all types of clothing and asked: Yo,
my son, which of these is the most stylish thing to wear? And the correct
answer is: Hey, whatever I select.
        -- (Terry Pratchett, Lords and Ladies)
+++