[Buildbot-devel] Python wedges when buildslave connects

[Matthew Morse]

On the chance that someone else has run into this before...

We have a server that runs several buildbot masters for our test fleet. I recently updated the python installation to 2.7.2 and the buildbot software to 0.8.4p1.  I upgraded all the buildmasters, and all except for one behave as expected. The problematic one starts up properly and I can access its web page, but as soon as the buildslave (on another machine) connects with the buildmaster, the Python process starts to churn, using over 50% of the CPU. The webpage becomes unresponsive--although the build on the slave seems to proceed properly.  From the sample, it seems that Python is getting stuck in a call to PyEval_EvalFrameEx. 

Call graph:
    2025 Thread_182242   DispatchQueue_1: com.apple.main-thread  (serial)
      2025 0x100000f14
        2025 Py_Main
          2025 PyRun_SimpleFileExFlags
            2025 PyRun_FileExFlags
              2025 PyEval_EvalCode
                2025 PyEval_EvalCodeEx
                  2025 PyEval_EvalFrameEx
                    2025 PyEval_EvalFrameEx
                      2025 PyEval_EvalCodeEx
                        2025 PyEval_EvalFrameEx
                          2025 builtin_execfile
                            2025 PyRun_FileExFlags
                              2025 PyEval_EvalCode
                                2025 PyEval_EvalCodeEx
                                  2025 PyEval_EvalFrameEx
                                    2025 PyEval_EvalFrameEx
                                      2025 PyEval_EvalFrameEx
                                        2025 PyEval_EvalFrameEx
                                          2025 PyEval_EvalFrameEx
                                            2025 PyEval_EvalFrameEx
                                              2025 PyEval_EvalFrameEx
                                                2025 PyEval_EvalCodeEx
                                                  2025 PyEval_EvalFrameEx
                                                    2025 PyEval_EvalCodeEx
                                                      2025 PyEval_EvalFrameEx
                                                        2025 PyEval_EvalFrameEx
                                                          2025 PyEval_EvalFrameEx
                                                            2025 PyObject_Call
                                                              2025 function_call
                                                                2025 PyEval_EvalCodeEx
                                                                  2025 PyEval_EvalFrameEx
                                                                    2025 PyEval_EvalFrameEx
                                                                      2025 PyEval_EvalFrameEx
                                                                        2025 PyObject_Call
                                                                          2025 function_call
                                                                            2025 PyEval_EvalCodeEx
                                                                              2025 PyEval_EvalFrameEx
                                                                                2025 PyEval_EvalFrameEx
                                                                                  2025 PyEval_EvalFrameEx
                                                                                    2025 PyEval_EvalCodeEx
                                                                                      2025 PyEval_EvalFrameEx
...




I've tried numerous things to try to isolate the problem--recreated the buildmaster from scratch, changed the port number used by the slave, and so on. I've compared its configuration with other buildmasters that are working properly, and I don't see how this particular buildmaster is significantly different from the others. In fact, it is one of the simplest we have. The log shows a normal start-up, and then nothing more.

I'm using Mac OS X Server 10.6.8, Python 2.7.2, and Buildbot 0.8.4p1, and Twistd 11.0.0.

Thanks,
- Matt