[Buildbot-devel] Strategy for highly branched codebase

[Marc-Antoine Ruel]

2013/5/30 Ludovic Chabant <ludovic at chabant.com>

> Hey everyone,
>
> So this may be as much of a buildbot setup question as it is a project
> setup question. Our codebase is shared across several different projects,
> each of which has any number of branches and variants. There is a crazy
> total number of branches.
>
> Now we don't have all those branches on our build server, but we need to
> add/remove any branch at any moment to/from the build server. Basically
> when we need to work with some of our clients, we would add some of their
> branches to the build server to monitor the changes we make there, and also
> have the build server do stuff for us like populate our Perforce proxies
> (customers are all around the world) or warm up our data caches (there is
> data building involved, not just code).
>
> Now the problem lies with the various scripts that our builders execute.
> They're in source control, but different projects and different branches
> have different versions of those scripts -- and sometime they don't have
> them yet because that team is still on an older version of the codebase.
>
> So far, I can think of only 2 solutions:
>
> 1. Write custom build steps with all the logic in them.
> I know this is frowned upon, but this may be an edge case. The problem is
> that there isn't a lot of information about doing this kind of thing, so I
> don't know what can be done, what can't be done, etc. Does anybody have
> experience with this?
>
> 2. Put all the build stuff in a separate codebase.
> Basically put all the scripts in a standalone repository, and have the
> buildslaves not only sync the normal codebase, but also sync the "scripts"
> codebase, so any changes to them would apply to any project, regardless of
> what version of our codebase they have.
> This could be a problem however since, at least with Perforce, each
> builder is supposed to have only one workspace, located inside the "build"
> directory. That workspace could only include the build scripts (through
> some custom viewspec) if they're on the same Perforce server as the normal
> codebase, which is not necessarily the case.
> Has anybody tried something like that?
>

As an example, in chromium we have the build slaves checkout build scripts
that helps building and running tests. It's these scripts that gets called
by the ShellCommands which then call the test executables built in the
checkout.

Our setup is definitely more complicated than what you need but it's just a
data point that yes, some people are doing that. We have the build slave
sync their own checkout and auto reboot so that the build scripts stay
somewhat up to date.
http://www.chromium.org/developers/testing/chromium-build-infrastructure/getting-the-buildbot-source

With perforce specifically, you could get in the case where you would have
a client view inside a client view, which I don't think Perforce would
like. Fixing that is left as an exercise to the reader. :)

M-A


> Is there maybe a 3rd option I haven't thought about?
> --
>    l u d o .
>    . 8 0 17 80
>
>
> ------------------------------------------------------------------------------
> Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
> Get 100% visibility into your production application - at no cost.
> Code-level diagnostics for performance bottlenecks with <2% overhead
> Download for free and get started troubleshooting in minutes.
> http://p.sf.net/sfu/appdyn_d2d_ap1
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20130530/1fcf24f6/attachment.html>