[Buildbot-devel] gitpoller fails when there are lots of changes: [Errno 24] Too many open files

[Matisse Enzer]

I just got back to working on our buildbot config after a couple of crazy weeks at work, and found that gitpoller, or rather python, blows up when there are lots of unprocessed changes.

This is a stack trace from 0.8.3:
 
2010-12-21 19:25:56-0800 [-] gitpoller: processing 47 changes: ['a5247437da838e6a82b2a4df76db232eba7ba0d4', '2d517e7ff5414b89a778291d837fe89ec58688be', '78d0655d51e8b816398c88806282e139677c8812', '4cea993654898aae0eaec34883ac2252939cf2f6', '4e40fb1e2b81d74908555afc0e4ca49c15221ec5', '7b77c1b2f6ac88229a81fbdcee776bd1d84a3e7d', '36eb81e5c33a8fd8793c34d2c20b77fcda4b6306', '76618cbb36d6250a7d26413faeb6357c23c53595', '1c6fcffb6a07224d60b3b597357159d9118968ef', '654643b184785e0b77ac669ff5f86390c8e614c8', '5212f53034b220aa570797dd51e064cbee0f7d8b', '9ace138d6be87e018f0a8e47a082d8edd39a4f8d', 'a0524302ae1000855c2f82b8d3d2e1de58fc2abf', '1bd74af93f7bedcf80589cb15320b5de9ca245a3', 'e16c50704b31685805b0a17af8eda97d9d39cd2c', 'ed1a3949c372e2be4b70470fda318252f16b2395', 'f44d25958b31bbbcd257438454315e5d131392bd', 'db25329da30ad4454934e292e854de602b1663d3', '7d827507679ca4206c3268396f961681f72790ac', '09286aa8e78bf2ff16136598feaacf7d948f5b6b', 'adbf5cefc577ec8d851796773d890b5c9ccfef27', '9b2e7f81b6b444e6247deb3a756878df652e3c35', 'a34bb797b9540c7614956fbb88034edb1bd65f96', '11722b830904c2bdd205d965b3ea4b0be2adef0d', 'faa5320812c58bae3de700b428f853b7b62658a4', 'a74cffb6559ec3834cf2409fe7287b1503369579', '588b2a72f890f5f8a575fabdcfa752cac6a1b9c8', '44c2d97735220d5bc5d8fbb0fa42e73cf82cc734', '033506700b67836cf27f3cfe35808c0d322f78b2', '03e49570b69710a6b9ab5be1891a307119766903', '0002bc0e8eaf644466b52a56a67d015a8cfd2d3b', 'a8e8ef5dc4da58d541795d742e3ebefacf049e1f', 'dbfa45c25f4232174f174e5980615cb752d9dd0c', '4462c03aa2b9be67d8bd375a2fbbb8b92c7431d9', '95f1fba30ef9f666fc032cb79b6da1d1ef496727', '3ebcdfea4c4434d1cbe30ea3641ddb0b4ee21867', '8bf1cc81550bf884471962ce3baa48738da9cd49', 'c394ceb6a541b400e6f159ad3f2f1205fe2f9564', '06538c03df6c29a571d2f6c1caae10d363dddd2d', 'a3621221ced3ba46999c12cb10452e28a0a6e2d5', '4f27cdc62c068c20d5bdf8b92af3885c10bfa584', '18591492c396a8321187658bc292718deb8b33a3', '35ccf3c993c2eda677583b9b9a362f3d54131039', '5b1f7dfc831ecd4c538818a8b9f59dce93c14290', 'a3f2acbd4f1c24bfee48a1d49710ca65267b7fe0', 'b567dac784febe43e68bc18f2b7637d83b134a4b', '8cf6051277eea786621bdaefde2f7b6de3439105'] in "workdir"
2010-12-21 19:25:57-0800 [-] gitpoller: repo poll failed
2010-12-21 19:25:57-0800 [-] Unhandled Error
	Traceback (most recent call last):
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/_baseprocess.py", line 60, in maybeCallProcessEnded
	    proto.processEnded(Failure(reason))
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/utils.py", line 89, in processEnded
	    self.deferred.callback(self.s.getvalue())
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/defer.py", line 243, in callback
	    self._startRunCallbacks(result)
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/defer.py", line 312, in _startRunCallbacks
	    self._runCallbacks()
	--- <exception caught here> ---
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/defer.py", line 328, in _runCallbacks
	    self.result = callback(self.result, *args, **kw)
	  File "/Library/Python/2.6/site-packages/buildbot-0.8.3-py2.6.egg/buildbot/changes/gitpoller.py", line 182, in _process_changes_in_output
	    self._get_commit_files(rev),
	  File "/Library/Python/2.6/site-packages/buildbot-0.8.3-py2.6.egg/buildbot/changes/gitpoller.py", line 121, in _get_commit_files
	    d = utils.getProcessOutput(self.gitbin, args, path=self.workdir, env={}, errortoo=False )
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/utils.py", line 124, in getProcessOutput
	    reactor)
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/utils.py", line 25, in _callProtocolWithDeferred
	    reactor.spawnProcess(p, executable, (executable,)+tuple(args), env, path)
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/posixbase.py", line 224, in spawnProcess
	    processProtocol, uid, gid, childFDs)
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/process.py", line 524, in __init__
	    readFD, writeFD = pipe()
	  File "/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/process.py", line 513, in pipe
	    r, w = os.pipe()
	exceptions.OSError: [Errno 24] Too many open files