[Buildbot] #3291: Maximum patch size for try-jobs

[Buildbot trac]

#3291: Maximum patch size for try-jobs
----------------------+-----------------------
Reporter:  david1234  |      Owner:
    Type:  defect     |     Status:  new
Priority:  major      |  Milestone:  undecided
 Version:  0.8.8      |   Keywords:  try
----------------------+-----------------------
 I use try-jobs submitted via ssh and recently ran into the default max.
 size limit for netstrings. After increasing it to 1MB as described in [1],
 I get the following error in twisted.log

 {{{
 2015-06-25 11:37:26+0200 [-] command '['git', 'apply', '--index', '-p',
 '0']' in dir 'build'
 2015-06-25 11:37:26+0200 [-] BuildStep.failed; traceback follows
         Traceback (most recent call last):
           File "/usr/lib/python2.7/site-
 packages/buildbot/steps/source/git.py", line 480, in patch
             initialStdin=patch[1])
           File "/usr/lib/python2.7/site-
 packages/buildbot/steps/source/git.py", line 314, in _dovccmd
             d = self.runCommand(cmd)
           File "/usr/lib/python2.7/site-
 packages/buildbot/process/buildstep.py", line 779, in runCommand
             d = c.run(self, self.remote)
           File "/usr/lib/python2.7/site-
 packages/buildbot/process/buildstep.py", line 79, in run
             d = defer.maybeDeferred(self._start)
         --- <exception caught here> ---
           File "/usr/lib64/python2.7/site-
 packages/twisted/internet/defer.py", line 139, in maybeDeferred
             result = f(*args, **kw)
           File "/usr/lib/python2.7/site-
 packages/buildbot/process/buildstep.py", line 399, in _start
             return RemoteCommand._start(self)
           File "/usr/lib/python2.7/site-
 packages/buildbot/process/buildstep.py", line 116, in _start
             self.remote_command, self.args)
           File "/usr/lib64/python2.7/site-packages/twisted/spread/pb.py",
 line 345, in callRemote
             _name, args, kw)
           File "/usr/lib64/python2.7/site-packages/twisted/spread/pb.py",
 line 873, in _sendMessage
             self.sendCall(prefix+"message", requestID, objectID, message,
 answerRequired, netArgs, netKw)
           File "/usr/lib64/python2.7/site-packages/twisted/spread/pb.py",
 line 586, in sendCall
             self.sendEncoded(exp)
           File "/usr/lib64/python2.7/site-
 packages/twisted/spread/banana.py", line 284, in sendEncoded
             self._encode(obj, io.write)
           File "/usr/lib64/python2.7/site-
 packages/twisted/spread/banana.py", line 296, in _encode
             self._encode(elem, write)
           File "/usr/lib64/python2.7/site-
 packages/twisted/spread/banana.py", line 296, in _encode
             self._encode(elem, write)
           File "/usr/lib64/python2.7/site-
 packages/twisted/spread/banana.py", line 296, in _encode
             self._encode(elem, write)
           File "/usr/lib64/python2.7/site-
 packages/twisted/spread/banana.py", line 296, in _encode
             self._encode(elem, write)
           File "/usr/lib64/python2.7/site-
 packages/twisted/spread/banana.py", line 325, in _encode
             "string is too long to send (%d)" % (len(obj),))
         twisted.spread.banana.BananaError: string is too long to send
 (717271)
 }}}

 The limit here seems to be 640k as set in twisted/spread/banana.py [2].

 I normally don't have such large diffs (> 700k), but in this case, I did
 rename some directories, which clearly results in a lot of files being
 added and removed.
 Is there a way around this limitation?

 Note: I tested this with version 0.8.8, but AFAICT this issues still
 exists in the most recent release.

 [1] http://docs.buildbot.net/0.8.12/manual/cfg-
 schedulers.html?highlight=netstringreceiver#try-schedulers
 [2]
 https://twistedmatrix.com/trac/browser/tags/releases/twisted-13.2.0/twisted/spread/banana.py#L84

--
Ticket URL: <http://trac.buildbot.net/ticket/3291>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation