[Buildbot-devel] Status of Multi-Project Support

[John Carr]

GNOME would love enhanced multi-project support. We have
http://build.gnome.org/, which has a few hundred projects on it.

The current live system actually uses a buildmaster for each project,
and the slave machines then has a slave for each project.

I'm currently working on a rewrite which you can grab from the bzr
branch below. Its the same configuration, but everything runs from one
buildmaster and one a slave per machine (with a *lot* of builders
floating around).

bzr branch http://bzr-playground.gnome.org/~johncarr/jhbuildbot/trunk jhbuildbot

There might be something useful in there for some of you. The
documentation is in my head right now, of course...

The dependency issue is solved for us by the use of jhbuild (GNOMEs
internal build system that speaks every VCS and build system that we
care about). The projects are added to buildbot in the order that
jhbuild dictates, and then we have a serial scheduler to run them in
turn. Of course, when we want to do more clever things, the problem is
no longer solved...

John

On Thu, Aug 7, 2008 at 6:53 PM, Mark Melvin <mark.melvin at gmail.com> wrote:
> Thanks for the replies, everyone.  It sounds like full-fledged
> multi-project support is one of the things that would be on my list
> for a 1.0 release. ;o)
>
> Mark.
>
> On 8/6/08, Steve James <ste at cpan.org> wrote:
>> On Wednesday 06 August 2008, Mark Melvin wrote:
>>  > Hi There,
>>  >
>>  > I'm looking at writing a build system for our internal product builds
>>  > and in addition to being a huge Python fan, I have been looking pretty
>>  > hard at Buildbot and it is almost everything I would sit down and
>>  > think about writing myself.  However - there seems to be a recurring
>>  > issue with people wanting to build more than one "project",
>>  > additionally complicated by the fact that these projects may be
>>  > interdependent (i.e. the output of one project is involved in the
>>  > build of another).
>>  >
>>  > I was wondering what the status of this was?  Does buildbot now
>>  > support this workflow in some undocumented way, have people hacked it
>>  > to suit their needs, or do people just not use it this way at all?
>>
>>
>> What a good question. I'd also like know what the established lore is on this
>>  too. Wouldn't this be an ideal wiki topic? Is it an FAQ even?
>>
>>  FWIW, I recently tried out a Buildbot configuration for our established builds
>>  here, currently based on Test::AutoBuild. The first obstacle is the
>>  assumption that there is only one source repository. Fortunately there's a
>>  patch for multiple Subversion repository support so I continued. The next
>>  obstacle was defining dependencies such that if A or B is changed A or B
>>  re-builds but _also_ that B rebuilds is A rebuilds, and so on. Amazingly (to
>>  me at least) I hacked up a new kind of scheduler that does just what I want.
>>  The experiment continues, but so far it looks like I can get all the benefits
>>  of Buildbot _and_ continue to build interdependant projects with it. Yay!
>>
>>  Steve.
>>
>>
>>
>>  --
>>  You shall be rewarded for a dastardly deed.
>>
>> -------------------------------------------------------------------------
>>  This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>  Build the coolest Linux based applications with Moblin SDK & win great prizes
>>  Grand prize is a trip for two to an Open Source event anywhere in the world
>>  http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>>  Buildbot-devel mailing list
>>  Buildbot-devel at lists.sourceforge.net
>>  https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>>
>>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>