[Buildbot-devel] multi-repo support: build triggers

[Tobias Oberstein]

This one is about the "build trigger" question.

=================================================================================

When multiple repos with multiple branches are involved, there is the question on
what triggers builds.

If you have one repo and one "main" branch, you can filter for that "main" branch
in a BuildBot scheduler, and thus only have stuff on "main" be built. Other stuff
on other branches won't get built.

In multiple repo scenarios, these are the options I currently can think of.
If someone has more ideas, pls comment!

1)
You have a set of predefined repos, where all changes will trigger builds.

Some kind of "buildout" repos.

Just limit hook installations to repos which should get built. Works today.

2)
You have a predefined pattern for branch name (e.g. "*build*"). Stuff committed to
any repo, but to branches that match that pattern will get build.

E.g. checkin to repo1/feature1_build will trigger build.

You can do that today using the ChangeFilter() feature on Scheduler/AnyBranchScheduler.

3)
A variation to 2): filter on a pattern for repo.

e.g. only have repos "*/home/*/repos/buildout/*" be built.

Don't know if that’s possible today, didn't try it.

---
Sidenote: when filtering is done like in 2) or 3), all changes (weather built or not)
still appear in waterfall page. If you envision the majority of changes not to be built,
this is somehow suboptimal IMHO ... visual clutter. But this is a UI question, and there
is more .. 

4)
When it's neither the repo nor the branch that decides if some change is to be built,
whats left in a commit to base that decision on?

For Git, in a commit there is the commit message.

I've implemented an option in git_buildbot hook:

* --filter-commits : control via commit message iff commit is sent as 
change, what category is sent
                     and properties sent

http://github.com/oberstet/buildbot/commit/7ae6b064869fcb0a4134353c3767d39099f1918e

which allows one to control buildout via commit message.

A commit with a "normal" commit message like

"Some change"

won't get send to buildbot and thus not built.

A commit with message

"Some change (buildbot)"

will get send to buildbot and may thus get built.

You can further control the "category" and "properties" sent in the change:

"Some change (buildbot full {'style': 'frantic', 'use_threads': 4 * 4})"

=> category = full
=> properties = {'style': 'frantic', 'use_threads': 16}


5)
In 4), filtering is already done in the hook. You could also have the filtering
completely on BuildBot side.

e.g. only build changes where category = full, quick, whatever.

If a change does not have category, it won't get build.

The point of 4) and 5) is, that the commit message is parsed in hook and somehow
used to control if a commit gets build, and with what category/properties.


6)
You can have builds be controlled not by repo hook at all, but "manually" triggered.

e.g. a buildout on buildbot is triggered from command line by developer on his PC


cd <git repo>
git checkout feature1
buildout --master=...

This would send a "fake" change to buildbot

revision	=>	git rev-parse HEAD
user		=> 	git config --get user.name
repository	=>	git config --get remote.origin.url

There are downsides:

- obviously, rev has to be avail. at server (pushed previously) ... if dev. forgets to push, sourcing fails
- developer needs Python+Twisted on his PC

===================================================

Benoît commented on 4) and didn't like it;) That’s fine, and though I don't agree with all
his objections, I can see valid points. After playing around with it a bit, I also see
that it might not be so cool in practice. But hey, it's just an option.

So currently, I'm leaning towards 6) and writing the trigger script.