[Buildbot-devel] Scaling buildbot

[Dan Kegel]

On Thu, Jun 18, 2015 at 4:56 PM, Jim Rowan <jmr at computing.com> wrote:
> On a related topic… we’ve noticed that FileUpload performance from buildslaves that are far away (50ms WAN) is horrible, compared to scp or rsync over ssh; about 10X slower.   For us, it’s always that way and doesn’t seem to vary a lot.  The local (LAN) upload performance is fine.   Anyone noticed this and/or have solutions?

That reminds me of a strange scp behavior I noticed; it was terrible
over a local gigabit link, but great over a local 100baseT link.
Turns out the gigabit hub introduced significant delay, and the scp
protocol (based on the old rcp protocol) was so chatty and
delay-sensitive it made a huge difference.

Similarly, if FileUpload is over an rpc-style interface with small
chunks, it's going to be slow if rtt is high.