[Buildbot-devel] Multi-repo and repo-caches on slave-side

[Ben Hearsum]

We've been talking about doing something similar, implemented at the 
BuildFactory level, but this sounds like a much better idea.

One problem that will need to be solved in how to determine when a 
checkout needs to be refreshed from scratch. Obviously, if the VCS is 
unable to update it, it will need to be wiped, but under what other 
circumstances is this the case. Perhaps every N time periods?

On 10-04-13 6:44 AM, Marcus Lindblom wrote:
> Hi,
>
> I just want to throw an idea into the mix now that we're talking about
> multi-repos, source-step modes (shallow clones), DVCS repo-forks etc.
>
> To save bandwidth and improve performance on "clobber" style steps, we
> could add a "repo-cache" folder on each build-slave that always pulls
> all changes locally and then the source-step clones/checks out from that
> (using hard-links on filesystems that allow that).
>
> For DVCS'es this type of workflow makes a lot of sense, but one could
> also imagine a repo-repo-sync for SVN et.al. to reduce bandwidth-load
> for the central server, especially for clobber mode.
>
> The slight downside is for workflows with DVCS:es who build a lot from
> rebased branches, the local cache repo will perhaps grow quite a lot.
> But we could, OTOH, just allow that and run some sort of
> garbage-branch-collect step every week or so (that would just delete the
> entire cache repo as a start, and, further on, prune every non-branch
> HEAD against some master repo.)
>
> I belive this scheme would help reduce our arguments visavis what
> different type of sourcestep modes are most appropriate for everyone.
>
> Cheers
> /Marcus
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel