[Buildbot-devel] Buildbot: doing occasional full builds

[Brian Warner]

> > > To wipe out the build occassionally you could (presumably) add a
> > > starting step to the Python 'builder' (in the build master.cfg) to 
> > >     rm -rf $builddir
> > > every, say, Sunday night.

That would work, although to be honest the buildbot is more about repeatable
builds. My first suggestion would be to have two separate Builders, one of
which does incremental builds, the other which does full (from-scratch)
builds. Both modes are useful to test: I've seen build problems that affected
one but not the other (deleted source files that leave .o files in place,
Makefile changes that don't notice modified files correctly, etc).

If you want to minimize the compile load placed on the build slaves, you
could use the Schedulers to reduce the frequency of triggering the "full"
builder. A subclass of the default Scheduler class could enforce a
minimum-time-between-builds timer, and you could crank it up to only allow
one full build per hour, or per day, or whatever.

The reason I'd encourage you to let every build of a given Builder behave the
same way is that I hope to add more problem-tracking code to the Buildbot,
and that code will work better if builds really are reproducible and stable.
If the tests pass in build #9 and fail in build #10, you should be able to
blame the failure on the changes that went into build #10. If instead the
failure is due to the fact that build #10 just happened to be the one where
you do a clean build instead of an incremental one, then it will be harder to
automatically isolate the cause of the failure.

It isn't a big issue, though. Just thought I'd share what my design
philosophy is so you can decide which direction you want to go. The code is
flexible enough to handle it.

> The current problem with this is that I don't know if the build step
> objects have access to a local data store -- so it could, say, count how
> many builds have been done to know to clobber every tenth one. The
> current code just chooses to clobber for any build on Sunday. Not a very
> satisfying solution.
> 
> Brian,
> Is there a way for build steps to maintain some state?

In general, no.. the long-term status is kept in a separate BuildStatus
object, the idea is that BuildStatus gets persisted but Builds do not. You
can always have your Build subclass use the regular status interface to look
up an earlier BuildStatus, and base some decisions upon the data you find
there, although I haven't really provided a way to set arbitrary attributes
on the BuildStatus.

For this case, however, you can have the Build reach into its BuildStatus
object and look at the .number attribute, and then do a simple %10==0 test.
Not especially elegant but it would get the job done. From a BuildStep you
would use 'self.build.build_status.number % 10 == 0'.

Apart from that, your SVNEx subclass seems feasible. I'd be inclined to have
the subclass always implement this update_and_clobber_occassionally behavior
(rather than providing a special mode= switch for it), but that's just a
usage detail.

cheers,
 -Brian