[Buildbot-devel] Buildbot 1.0: The Shimmering Vision

[Eveque, Philippe]

Quite similar to Sebastien's need.
But we do not use BB for testing purpose.


> -----Original Message-----
> From: buildbot-devel-bounces at lists.sourceforge.net
> [mailto:buildbot-devel-bounces at lists.sourceforge.net] On
> Behalf Of Sebastien Douche
> Sent: jeudi 7 août 2008 11:47
> To: buildbot
> Subject: Re: [Buildbot-devel] Buildbot 1.0: The Shimmering Vision
>
> On Thu, Aug 7, 2008 at 01:29, Dustin J. Mitchell
> <dustin at zmanda.com> wrote:
> > Well, I posted my thoughts.  What is your vision of what Buildbot
> > could be in a perfect world with an infinite supply of
> developer time
> > and money?
>
> My bad, my question was about the media (mailing list, wiki page).
>
> Like Terry, We have :
> 1/ multiple projects (worse, we have a component approach : a Python
> project have multiples components, ie with interdependencies each
> others)

Ditto.
In one CMS for some projects, over more than one CMS for others.

>
> 2/ multiple software deliverables (ex: Python2.4, Python2.5, Python2.6
> for each components)

Ditto, targetting various OS/OS flavors

>
> 3/ multiple sources per software deliverables (trunk and multiple
> branches, and we set up a new repository with Mercurial : branche per
> functionnality or bug !)
also even if trying to limit this.
the underlying SCM has some influence here.

>
> 4/ multiple build platforms (Sarge, Etch, Lenny, FreeBSD, GCC / ICC).

Ditto.

>
>
> Maybe a big picture here is interesting :  we use BB to set up a
> complete chain building :
>
> 1/ for earch commit on any branches, we want :
> - build the component alone, with or without unit tests on sandbox
> environnement (virtualenv for Python)

in our case the last build stage (optional) is to generate the packages (rpm, deb, ...)
and make them available in the package repository
for later deployment. BB is not used for deployment.

> - copy the result on root distribution server (internal Pypi)
> - a warning on irc
>
> 2/ each hour :
> - build each component separately with dependencies with unit and
> functional tests on sandbox environment (virtualenv + buildout)
> - using lint/pylint and others source analysis
> - a complete results for each components (email)
>
> 3/ nightly build
> - build the complete application on sandbox environment (our stuff +
> Postgresql, Apache...)
> - launch unit tests for each components, functional tests, UI tests
> (like Selenium)
> - a complete result (email)
>
>
> So, my initial thoughts / suggestions :
>
> - multiple projects on single repository (who use a repos per project
> ?) with one master
Would also need to deal with multiple SCM repository on one master (some of our project use svn externals...).
But that might not be the general case.

> - automatic log parsing to know the built file (very interesting with
> fileupload command to copy outside)
> - more steps for C and Python language (Pylint, lint...)
Or rpmlint stage for rpm packaging. This is easy to do.

> - more properties on VCS (ex to know the revision of the file
> we build)
> - simplification of waterfall page (we have 50 columns...)
>
Yeap having a per module display summarising the stae for every OS target (RHEL5, HP-UX 11iv2, RHEL4, Debian, ... )
with RSS/Atom feed for each module (will have for each target OS on all the modules soon).
When you need the details just enter the link (then you will find RHEL5 32, RHEL5 64, 11iv1 for PARIS, 11i v2 for IPF, ...)


I would add:

- nightly builds option to scheduler so it is triggered only if some changes happened.
  (difficult to changes every module every days)

- build slaves are mostly hosting of one or more build environements.
  Build environements can be the hosting OS, a virtual machine, a static chroot, a dynamically instantiated chroot (via mock)
  or some combination of these entities (e.g: a dynamic chroot in a VM running on a host OS)
  Would like to Buildbot knowing something about Build Environment (to track them and their properties, not to generate them)
  Managing buildslave workload can be not trivial here.

Phil.