[Buildbot-commits] [Buildbot] #2273: Performance items

Buildbot nobody at buildbot.net
Wed Apr 4 00:40:44 UTC 2012


#2273: Performance items
----------------------+-----------------------
Reporter:  szager     |      Owner:
    Type:  undecided  |     Status:  new
Priority:  minor      |  Milestone:  undecided
 Version:  0.8.6p1    |   Keywords:
----------------------+-----------------------
 This is an FYI ticket about some performance-related observations gleaned
 from running a buildbot master under cProfile.

 Here's what I'm looking for in the profile data:
  - Ways to reduce overall CPU utilization by looking for code paths that
 hog CPU cycles.
  - Synchronous code paths that take a long time to execute,
 increasing the latency of the web UI.

 Observations:

 - In console.py, getBuildsForRevision() accounts for about 12% of all
 CPU cycles.  It's called once for each builder, every time the console is
 generated.  For us, collectively those invocations take about 0.8 seconds
 to run, which accounts for about 80% of the time required to build the
 console.  Since the return values change pretty infrequently, it should be
 possible to cache the results and speed up subsequent lookups.

 - We use an SVNPoller with a polling interval of 10 seconds.  In
 svnpoller.py, parse_logs() accounts for 8% of CPU cycles, and takes 0.8
 seconds per invocation (each time the poller runs).  99% of that time is
 spent in xml.dom.minidom.parseString().  It should be possible to speed
 that up by either switching to a faster xml parser; or *not* using the
 --xml option to `svn log` and custom parsing the output.

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2273>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation


More information about the Commits mailing list