[Buildbot-devel] Multi-repo and repo-caches on slave-side

[Marcus Lindblom]

Hi,

I just want to throw an idea into the mix now that we're talking about 
multi-repos, source-step modes (shallow clones), DVCS repo-forks etc.

To save bandwidth and improve performance on "clobber" style steps, we 
could add a "repo-cache" folder on each build-slave that always pulls 
all changes locally and then the source-step clones/checks out from that 
(using hard-links on filesystems that allow that).

For DVCS'es this type of workflow makes a lot of sense, but one could 
also imagine a repo-repo-sync for SVN et.al. to reduce bandwidth-load 
for the central server, especially for clobber mode.

The slight downside is for workflows with DVCS:es who build a lot from 
rebased branches, the local cache repo will perhaps grow quite a lot. 
But we could, OTOH, just allow that and run some sort of 
garbage-branch-collect step every week or so (that would just delete the 
entire cache repo as a start, and, further on, prune every non-branch 
HEAD against some master repo.)

I belive this scheme would help reduce our arguments visavis what 
different type of sourcestep modes are most appropriate for everyone.

Cheers
/Marcus