No subject

docs/examples/glib_master.cfg .

>  3) If I don't want my source tree modified, could I just set
> c['sources'] to [] and use a command line step to 'cd /xxx' to the
> right folder, or should it just set it to the absolute value & check
> clobber's false?
>     (Aside - I'm not convinced a default of clobber="true" is a good idea!)

Nope. Hmm, let's take a step back.

1: c['sources']

c['sources'] specifies how the BuildBot should find out about new changes to
the source tree. It is a list of *inputs*. You'll eventually tie this into
some kind of ClearCase notification script.. if you currently have a mailing
list where commits are announced, then whatever CC mechanism drives those
mails will be useful for telling the BuildBot that something has changed. The
basic idea is to run builds at a "reasonable" rate: don't waste CPU time by
running them every hour even though nothing has changed, but still run builds
soon enough after a commit to try and detect problems before other developers
get inconvenienced by them.

If you don't have any sources (if c['sources'] = []), then you won't get
commit-driven builds, that's all. You'll have to push the "Force Build"
button manually, or set up a periodic build (described below). c['sources']
has nothing to do with how any given build is performed.


2: the clobber= argument to the CVS step

One of the many kinds of Steps you can use in a build is named CVS (actually
the full name is buildbot.process.step.CVS). You would use it to perform a
CVS checkout. It is kind of a shorthand for:

 step.ShellCommand(command=['cvs','-d','CVSROOT..','checkout','-d',workdir])

except that it has some local (slave-side) intelligence to try and do an
update instead of a full checkout whenever possible.

The clobber= argument tells the CVS step to delete the tree before performing
a checkout, rather than trying to update it in place.

Now, since you're using ClearCase instead of CVS, you won't use the CVS step
at all. Instead, you'll use a basic ShellCommand step to set up the ClearCase
view (maybe.. see below).

3: 'cd /xxx'

Each Step is performed in a separate shell, just like the commands of a
Makefile. 'cd' is not very useful as a ShellCommand, because it doesn't
affect any of the following steps.

Each Step takes a 'workdir' argument, which specifes the directory where the
associated command should be run. So something like:

 step.ShellCommand(workdir='build', command=['make', 'all'])

is effectively like doing:

 cd build && make all

Actually, it is more like doing:

 make -C build all

because you aren't left in a different directory when you finish.

4: clobber=True

For something like ClearCase (and possibly Perforce), clobber=True would
probably not be a good default. For CVS, Subversion, Darcs, Arch, and other
version control systems that don't use a CC-style "magic filesystem", it
provides the "clean build" that (I believe) most people expect when you tell
them you're doing a nightly or automated build. Once the basic buildbot is up
and running, users can set up an "update" build (by turning off clobber=).
This is useful to discover any broken dependency-checking, etc, and behaves
much more like developers usually do (I do an update and rebuild each morning
to pick up whatever changes my coworkers have commited the previous day, that
keeps me from falling too far behind the rest of the project).

>  4) How do I automate a daily run?

You can set up a periodic build by adding an argument to the Builder
specification dictionary in the config file:

 b1 = {'name': 'full',
       'slavename': 'bot1',
       'builddir': 'full-builddir',
       'factory': ConfigurableBuildFactory(steps)
       'periodicBuildTime': 24*60*60,
      }
 c['builders'].append(b1)

Every 'periodicBuildTime' seconds, a new build will be started.

>  5) Anyone got a config file to do some or most of the above! :-)  

Your biggest challenge is going to be with ClearCase and their magic
filesystem (Versioned File System: "vfs"?). The problem is that the ClearCase
model (in my limited understanding) tends to be one where you do *all* of
your work in their VFS. Doing a "checkout" is really when you modify your
View to map the project branch that you're currently working on into some
subdirectory of the VFS (as opposed to most other VC systems where a checkout
is populating source files in a new subdirectory of some non-VC-controlled
filesystem). In CC, the sources are sort of "above" any build infrastructure.

In BuildBot, each slave gets a directory all to itself. Within that, each
Builder gets a directory of its own. Within *that*, each build causes a
source tree to be checked out and compiled. The idea is that the build
doesn't touch anything outside that source directory. So the BuildBot model
is that the sources are "below" the build infrastructure.

So I can think of a couple possibilities for implementing your source
"checkout" step:

 1: don't do it. Set up your view on the buildslave machine once. Arrange for
    it to be visible from the buildslave's working directory (perhaps a
    symlink from slavedir/builderdir/build to /vfs/somewhere). Let ClearCase
    be responsible for updating the files as they are modified (using the
    'LATEST' setting in the View specification).

    With this approach, the build process is just a 'make all': no source
    checkout required. You could also add some kind of timestamp-setting step
    which freezes the source into place while the build runs.. this starts
    getting into ClearCase magic that I don't have the experience to help
    with.

 2: Create a new View for each build. The sources will still live in /vfs/ or
    wherever the magic filesystem is mounted, but you'll have a ShellCommand
    that creates a new View for each build. The first step will be to set the
    View, the second step would create a symlink from the buildslave's
    workdir to the target sources. You'll need to delete the symlink (just
    blow away the local workdir with rm -rf) at the beginning of each build.

 3: Copy the sources into the slavedir. You can leave a static View set up
    like in approach #1, one that is always at LATEST and thus always has the
    most recent set of source files. The first BuildStep would just be a
    simple 'cp -r /vfs/somewhere workdir' to copy the sources into the slave
    directory for each build. (again, you'd want to rm -rf the workdir
    first). This fits more into the usual buildbot model, and slightly
    reduces the chance that a change in the middle of your build will cause
    mismatched sources to get compiled.


I'd start with #1, then go for #3. Basically you'll do your normal ClearCase
view setup to create the sources that you want to build somewhere, then run
the buildslave from a shell that's been 'ct setview'ed to use that View. The
"checkout" step is then just a symlink or a copy, and the end result is that
some source files are available in the directory that the buildslave expects
them.


Not straightforward, but then ClearCase is really a very different beast than
anything else we have available to us. Even Perforce (which uses View
specifications) is a bit more checkout/sync -oriented, and doesn't use the
magic filesystem cleverness that CC is based around.

> (I'll persevere with this, as part of the problem has been that I've
> been looking in the CVS docs & the mailing list but have been
> confusing myself with the changes since 0.5.0.  By the way, when the
> next release is made, I'd urge that the released .xhtml files are
> added to the website, just for ease of access to casual browsers.)

Good point, I'll make a note to get those docs posted.

thanks, and good luck!
 -Brian