[Buildbot-devel] Automatically Creating builddir?

[Roy S. Rapoport]

On Wed, Sep 20, 2006 at 02:11:16PM +0200, christian unger wrote:
> On 18.09.2006, at 21:11, Roy S. Rapoport wrote:
> >we have a meta VC layer between buildbot and the VC
> 
> what does this mean?

Think of it like a network stack.  We have an environment that we interface
with to deal with code checkouts and building.  It manages things like:

A-head depends on
	B-head
	C-1.2.3.4.5

When you do '<tool> update -a A-head', the -a means 'ancestors' and will
also update B-head; 

Building A-head will see if anything in B-head needs to be built first;

When you freeze A-head, freezing is a recursive operation -- so it'll
freeze B-head into, say, B-5.2.1.1.3, change A-head's dependency from
B-head to B-5.2.1.1.3, and then freeze A-head into, say, A-1.1.1.3.2.  It
will then use the VC (in this case SVN, but the tool doesn't much care --
it hides the particular VC from the user) to tag B and A with their
respective versions.  

We have a significantly large dependency tree in some cases -- our
engineering groups are small, and depend on a very robust shared code-base.
Our meta-VC systems allows us to 

	A) automatically guarantee that every shipping distribution we make is
perfectly repeatable, no matter how many dependencies it contains, without
chance of error (I probably spoke too soon, but in my 751 days at work I've
never yet seen a situation where one could be created); 

	B) It simplifies management of a private build area for each of our
developers where they can update -- or not update -- any given component at
their leisure, either manually or automatically; 

	C) Sitting on top of SCons, it simplifies builds -- all I have to do is
type '<tool> build A-head', and magic will happen.  The right stuff will be
checked out if it is not;

	D) It allows us to factor out OS-specific package building.  To build a
package, I declare a manifest (foo = manifest()), add files to it with
specified locations where they need to install, and things like
after-install actions and dependencies (foo.requires('ourotherproduct'
'>=2.0.4.2').  When building on Solaris, I get a solaris package; when
building on RHEL, I get an RPM.  When building on Debian, HPUX, FBSD, NCR,
etc ... well, you see where this goes.

	E) It manages VC migrations.  For example, originally we used CVS.  We
now use SVN.  It knows not only which products are in SVN and which are in
CVS, but also which versions of which product are in each repository.  It
may choose to update A-head from SVN in the above example, but check
C-1.2.3.4.5 out from CVS.  Adding a new VC is relatively trivial -- it'd
likely be about a two hour job or less.

Does this answer your question adequately?

-roy