[Buildbot-devel] 0.7.x community?

[Dmitry Nezhevenko]

On Wed, May 30, 2012 at 08:00:23PM -0600, Tom Prince wrote:
> Dmitry Nezhevenko <dion at dion.org.ua> writes:
> 
> > - module for custom proprietary VCS (AccuRev). Still client-side. I'm
> >   working on general "command"-based master step that I'll try to ask for
> >   merge.
> 
> Do you mean slave-side? It is fairly straightforward, and probably
> easiest, to write new VCS steps as master-side steps. Unless you
> *always* run the same commands, which likely isn't the case (for
> example, initial checkout), there needs to be some logic around getting
> the repository in the right state. If that code lives on the master, it
> is much easier to depoly, than if each slave needs custom code.

Yeah. It was written in times when there were no master-side checkout
steps at all. I need to perform something around 5-6 calls to VCS for
update. For example there is no way to "cleanup" source tree. Instead I
should query for untracked files and then delete them manually. Also I've
non-twisted python bindings to VCS. It's impossible to use such library on
master

> > - "Aggregate" build step that can contain a few child build steps and
> >   execute them one after another. Mostly done to avoid very huge amount of
> >   steps in WebStatus. Now it mostly looks like one step per high-level
> >   target and each substep just produces own logfiles. Very useful if we've
> >   for example 10 targets and each requires something like "configure,
> >   make, run tests, create tarball, upload file". I don't think that it can
> >   be merged, but can try to cleanup and ask for this if somebody needs it.
> 
> There are a couple of other approaches to this.
> 1) You can run multiple commands per build step, something like
> https://github.com/Jc2k/buildbot_travis
> 
> 2) You can run a script that outputs different phases to different log
> files, and then include all those log files (or similarily, have the
> output parseable, and do the parsing on the server side).
> https://github.com/tomprince/buildbot-configs/blob/notmuch/notmuch_test.py
> is a somewhat simple example of the later (but only running a single command).

Yeah. We were using something like this before. But now some substeps
require own "locks". For example we've large step named "Foo" that
contains followed substeps:

1. configure
2. make
3. run tests
4. create tarball
5. FileUpload()

Everything except p3 can be done in parallel way together with other
builders on same slave. But step 3 requires exclusive access to some
shared stuff.

Also "custom script" will not work for FileUpload.
Because it's just impossible to run it from shell script.

Our builders already have ~20 steps, some of them contains 3-5 substeps.
That's why such "AggregateStep" simplifies things for us.

-- 
WBR, Dmitry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://buildbot.net/pipermail/devel/attachments/20120531/7c12a04f/attachment.bin>