[Buildbot-devel] Custom build properties - patches, try builds, future?

[Roch Gadsdon]

> Roch Gadsdon <rochg at bakbone.co.uk> writes:
>
>> Hello list,
>>
>> I have a few thoughts / questions on custom build properties.
>
> Hi there.. sorry to be so slow in responding to this.
>

Thanks for replying Brian - really appreciate it. Sorry to take so  
long to get back - it's definitely not disinterest.


I have opened a trac ticket with both Paul's original patch and my  
further patches for the try functionality. I know the code I wrote is  
not yet sufficiently robust, but it would be interesting to see if  
this looks like a valid approach or there are better ways. I am still  
new to Python and BuildBot so please just say if it doesn't look right.

Here's a summary of the current code:

Paul's original patch (custombuildproperties.diff):

- Added a dictionary called custom_props to BuildRequest. The  
properties in this dictionary are added to the existing "build  
properties" with self.setProperty
- Added code to read a description of available build properties from  
the master config file. These descriptions are stored in the  
BuildMaster class
- Added code to the force-build page to retrieve the property  
descriptions, generate form fields for the user to fill out, and  
include the properties in the BuildRequest once the build is started

This worked for the forced builds when custom properties were  
defined, but not for other types of build so I have added:

- Code in html.py to guard against no custom properties being defined  
(html.py.diff)
- Code to BuildSet to optionally accept a custom_props dictionary at  
creation and pass it to the BuildRequest it creates ((buildset.py.diff)

Then to add support for "try" builds I:

- Made TryOptions interpret a single new option called  
"customproperties" that accepted a comma separated list of properties  
e.g. "tests=quick,dependencies=no". The comma separated list gets  
converted into a dictionary (runner.py.diff)
- Modified the Try class so that when it makes the remote "try" call  
it includes the custom properties dictionary (tryclient.py.diff)
- Modified Try_Userpass_Perspective to expect a custom properties  
dictionary and pass it to the BuildSet it creates (scheduler.py.diff)

I have also included shell.py.diff on the ticket, which is an example  
of how we use the the input properties in our BuildBot setup.

> Yeah, I definitely think there's a need for this sort of thing. I'm  
> tempted
> to think that we might want to distinguish between these things  
> you've added
> which are inputs to the build process from the existing "build  
> properties"
> which could be considered as strictly outputs of the build. Maybe  
> the new
> input things should be called "parameters" or "arguments" or  
> something. Or
> maybe it's reasonable to use not bother with this input-vs-output  
> distinction
> and just call them all "properties" and treat them identically.
>

Thinking of this type of custom property as a read-only input sounds  
like a nice idea. I'd be happy to change to that. Discouraging/ 
preventing the modification of the inputs to a method/task/whatever  
is usually a good thing.

Thinking about things now, I guess we would also need to preserve the  
input properties so that a build with input properties could be  
exactly repeated with the "Rebuild" button. I'd have to look some  
more at the code to understand how to do that.

> I've wanted this personally for a while: I've been slowing working  
> on a
> "buildbot force" command that would be like "buildbot try" but  
> without the
> patch, because when I'm bringing up a new buildslave and  
> discovering what
> libraries are missing, etc, it's convenient to be able to retrigger  
> builds
> without making spurious source code changes (or going to the web  
> page's Force
> Build button, or using 'buildbot debugclient'). And I'd really like  
> to be
> able to say "buildbot try buildbot.test.test_web" to just do a  
> subset of the
> tests.
>

It works out really nicely for us. Running a subset of the tests  
while you're developing is really handy.



I will place a copy of this message on the trac ticket as well. Would  
you prefer future discussion to be on this list or on the ticket? I  
would think the list is better as anyone else can read and  
contribute, but I'll fit in with whatever you prefer.

Thanks again,

-- Roch