[Buildbot-devel] svn mode="copy" broken for multiple svn steps (unnecessary rm -rf/checkout cycles)?

[Natalie Tasman]

Hello all,

I am re-posting a thread I started last week.  Buildbot experts, if  
you could please tell me if this is a known issue (or if not, is  
reproducible) that would be a great start.   And pointing me to  
locations in the code where I might start working on a fix would be  
even better-- I'm willing to help out if I can!

In short, the problem is that the SVN step always wants to check out  
to "source"; if there are multiple svn steps the "source" dir is  
zapped (rm -rf) and then re-checked-out, every time.

The project that I'm working on has a SVN repository structure like  
this:
   .../trunk/projectA, .../trunk/projectB, .../trunk/ 
projectC, ..., .../trunk/projectF
And I only need projectB and projectF, so I really don't want to have  
to checkout the entire trunk (it's very large: 100s of MBs)

In order to build, I need the following local directory strucure:
   <some path>/projectB, <some path>/projectF
because ProjectF has references like "../../../projectB", so it's  
necessary to preserve this structure.

With  0.7.11p1, buildbot.steps.source.SVN set to mode "copy" like so  
(note: another person on this list suggested overriding "workdir";  
that does fix mode "update", but has no effect on mode "copy"):

   BuildFactory.addStep(SVN(
       mode="copy",
       baseURL=svn_base_url,
       defaultBranch="projectB",
       workdir="projectB-src",
       ))

   BuildFactory.addStep(SVN(
       mode="copy",
       baseURL=svn_base_url,
       defaultBranch="projectF",
       workdir="projectF-src",
       ))

   BuildFactory.addStep(ShellCommand(command="pwd"))



Here's what automatically happens:

   rm -rf projectB-src
   rm -rf source
   svn checkout [http...]/projectB source
   svnversion "."
   cp [...]/source [...]/projectB-src

   rm -rf projectF-src
   rm -rf source
   svn checkout [http...]/projectF source
   svnversion "."
   cp [...]/source [...]/projectF-src


pwd: "[...]/build" (empty directory)


The problems here:

1) "copy" is intended to keep a clean working checkout, then copy to a
new build directory.  That doesn't happen here: *Buildbot always zaps  
the
default "source" directory and re-checks-out everything, for both
projects.*  My checkouts are over 100MB, so doing a clean checkout
every time is prohibitively expensive.

Instead, I would like to see buildbot maintain two source directories
so that the next "svn update" step would just pick up deltas, not
require the entire checkout again.


2) "copy" doesn't copy the sources to the build directory.


Thanks again, and again, any advice on how to move forward would be  
appreciated!

Natalie