[Buildbot-devel] Acceptable dependency versions for patches?

[Jean-Paul Calderone]

On Fri, 14 Sep 2007 17:39:26 -0500, "Dustin J. Mitchell" <dustin at zmanda.com> wrote:
>On 9/14/07, David Bolen <db3l.net at gmail.com> wrote:
> [snip]
>
>> * File uploading, which breaks down for large files due to chaining
>>   all the deferreds for every block of the file, which if the file is
>>   large enough will blow up by exceeding Python's recursion limit as
>>   the chain is processed (not to mention using more memory).  I've
>>   switched this to a generator based approach.  (I haven't patched
>>   downloading, but it has the same potential issue)
>
>Generators should be safe.  You could alternately "break" the call
>chain periodically by scheduling a timer with duration 0.
>
> [snip]

You can also break the call chain like this (untested):

    def lotsOfDeferreds(listOfStuff):
        overallResult = Deferred()
        def oneDeferred(ignored):
            if listOfStuff:
                try:
                    oneResult = processOne(listOfStuff.pop())
                except:
                    overallResult.errback()
                else:
                    oneResult.addCallback(oneDeferred)
                    oneResult.addErrback(overallResult.errback)
            else:
                overallResult.callback(None)
        oneDeferred()
        return overallResult

The advantage of this approach over using callLater is primarily
that it is simpler to test.

Jean-Paul