[Buildbot-devel] Buildbot try with mercurial bundles

[Gunnison, Brian]

Hi,

I have a prototype "try" with hg bundles working. I thought I'd illustrate who does what to elicit comments. Basically I hijacked the "-p" and "-diff" args to indicate to the slave Mercurial class that the diff is a bundle. We then clone, apply bundle, then update to the bundle's revision.


1)      I created a wrapper script "try.py" to do the client side heavy lifting and minimize impact on buildbot src

a.       Invoked in working repository (wr)

b.      Prints "./buildbot/options"

c.       Identifies wr revision

d.      Asks developer to continue if there are uncommitted changes in wr

e.      Asks developer to continue if wr is not "tip".

f.        Authenticates master repository (mr) (accessed via https), asks for password.

g.       If verbosity set, lists changesets (vs. mr)

h.      Creates bundle "hg bundle try1.hg mr"

i.         converts binary try1.hg to ascii  file try1.hg.uu via uuencode

j.        invokes "buildbot try -p 99 -diff try1.hg.uu -baserev wrRevision

k.       TODO - add "-wait" option to above

2)      Modified buildbot (0.8.0)

a.       Tryclient.py: tried to add username to patch tuple, but not showing up at slaves yet...

b.      Buildstep.py: no longer logs repourl with password in the clear, instead logs: "Not logging cmd.args with 'repourl' for security reasons"

c.       Base.py, ShellCommand,_startCommand: as above

d.      Source.py, Source, computeRepositoryURL: can't deal with repository with "@" symbols (from password in https URL)

e.      Source.py, Source,start: prints "user" and "revision" in waterfall along with "patch". "patch" is uuencoded text which can be decoded to get bundle back.

f.        Source.py,Mercurial, __init__: Added "logEnviron=True" to this class so to turn off that damn env noise...

g.       Sourcestamp.py: try to get user name in patch tuple..

h.      Vcs.py: Where the slave work is done as follows:

                                                               i.      SourceBase, start: if "-p" == 99, do not call doPatch()

                                                             ii.      Mercurial, setup: if "-p" == 99, applyBundle = True

1.       doClobber is True, so in doVCFull, we clone (sans revision)

2.       Then if applyBundle we call "doBundle()" which does:

a.       Writes .buildbot-patched to working dir

b.      Converts uu encoded diff text back to binary bundle file: " try1.hg"

c.       Calls "hg unbundle try1.hg"

3.       We then update to the bundle's revision and we have replicated wr at the slave, securely, reliably, and efficiently.

The controversy is how much of my wrapper script should be applied to buildbot, my hijacking of "-p, --diff" and in general can we avoid the binary to ascii conversion without ripping up buildbot. Ideally I'd like a link to the patch binary file in the waterfall.

Comments?

Brian


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20100624/9152c7bb/attachment.html>