[Buildbot-devel] Buildbot try with mercurial bundles

[Greg Ward]

On Thu, Jun 24, 2010 at 2:26 PM, Gunnison, Brian
<brian.gunnison at intel.com> wrote:
> I have a prototype “try” with hg bundles working. I thought I’d illustrate
> who does what to elicit comments. Basically I hijacked the “-p” and “—diff”
> args to indicate to the slave Mercurial class that the diff is a bundle. We
> then clone, apply bundle, then update to the bundle’s revision.
[...]
> i.         converts binary try1.hg to ascii  file try1.hg.uu via uuencode
>
> j.        invokes “buildbot try –p 99 –diff try1.hg.uu –baserev wrRevision

Ewww, ick.  What a hack.  Well, you did say this is a prototype, so I
guess you've proven that it's *possible* to send bundles around like
they were patches.

> b.      Buildstep.py: no longer logs repourl with password in the clear,
> instead logs: “Not logging cmd.args with 'repourl' for security reasons”
>
> c.       Base.py, ShellCommand,_startCommand: as above

What does this have to do with try and bundles?  Sounds like a
general-purpose fix to me.

> e.      Source.py, Source,start: prints “user” and “revision” in waterfall
> along with “patch”. “patch” is uuencoded text which can be decoded to get
> bundle back.

Isn't this something anyone using try would want, whether they are
sending patches or bundles?  I thought there was even a ticket open
for this feature.

                                                            i.
> SourceBase, start: if “-p” == 99, do not call doPatch()
>
>                                                              ii.
> Mercurial, setup: if “-p” == 99, applyBundle = True
>
> 1.       doClobber is True, so in doVCFull, we clone (sans revision)
>
> 2.       Then if applyBundle we call “doBundle()” which does:
>
> a.       Writes .buildbot-patched to working dir
>
> b.      Converts uu encoded diff text back to binary bundle file: “ try1.hg”
>
> c.       Calls “hg unbundle try1.hg”
>
> 3.       We then update to the bundle’s revision and we have replicated wr
> at the slave, securely, reliably, and efficiently.

OK, cool, it works.  Good proof of concept.

Now the tricky question: obviously patches are universal and bundles
are specific to Mercurial.  But do other DVCSes have anything like
bundles, i.e. a binary representation of commits/revisions/changesets
that is more robust and exact than patch files, but not human
readable?  (Mercurial happens to just write its wire protocol to disk,
but that's an implementation detail.)  If so, then it's probably worth
teaching BuildBot's try machinery that there is more to the world than
patch files -- i.e. bundles are one example of this hypothetical set
of file formats that encapsulate one or more revisions in a robust,
exact, non-human-readable way.

But if this is just a Mercurial thing, wiring it into the
general-purpose bits of BuildBot feels wrong.  Hmmm.

Git/Bazaar/etc experts, this is where you step in and tell us what we
don't know about Git/Bazaar/etc.  ;-)

Greg