[Buildbot-devel] lengthening times for slave reconnects...

[Brian Warner]

> Twisted seems to have a lengthening waiting time for reconnecting
> slaves. For example:

Yes.. the slaves use an exponential backoff algorithm, vaguely like the
congestion-control algorithm that TCP uses, to prevent wasting bandwidth and
CPU cycles with connection attempts that are unlikely to succeed.

The parameters that control the backoff rate are class-level attributes set
in twisted.internet.protocol.ReconnectingClientFactory, with some amusing
comments warning you against changing them (because it wouldn't be very
network-friendly if you accidentally dropped the reconnect-rate down to, say,
.01 seconds: i.e. 100 connections per second). However you can probably
accomplish what you want safely by just dropping the 'maxDelay' parameter
down to maybe 60 seconds. It will still do the exponential-backoff, but the
delay will be capped at one minute (plus/minus some random jitter to avoid
the thundering-herd problem).

The easiest place to do that will be on the bot.BotFactory class, at
buildbot/bot.py:278 . Just add a class-level attribute to override maxDelay,
like so:

class BotFactory(ReconnectingPBClientFactory):
    maxDelay = 60 # more aggressive bots
    def __init__(self, keepaliveInterval=0):
        ...

Remember that this has to be done on the slave side: slaves connect to the
master, and not the other way around.

cheers,
 -Brian