[Buildbot-commits] [Buildbot] #2273: Performance items
Buildbot
nobody at buildbot.net
Wed Apr 4 00:40:44 UTC 2012
#2273: Performance items
----------------------+-----------------------
Reporter: szager | Owner:
Type: undecided | Status: new
Priority: minor | Milestone: undecided
Version: 0.8.6p1 | Keywords:
----------------------+-----------------------
This is an FYI ticket about some performance-related observations gleaned
from running a buildbot master under cProfile.
Here's what I'm looking for in the profile data:
- Ways to reduce overall CPU utilization by looking for code paths that
hog CPU cycles.
- Synchronous code paths that take a long time to execute,
increasing the latency of the web UI.
Observations:
- In console.py, getBuildsForRevision() accounts for about 12% of all
CPU cycles. It's called once for each builder, every time the console is
generated. For us, collectively those invocations take about 0.8 seconds
to run, which accounts for about 80% of the time required to build the
console. Since the return values change pretty infrequently, it should be
possible to cache the results and speed up subsequent lookups.
- We use an SVNPoller with a polling interval of 10 seconds. In
svnpoller.py, parse_logs() accounts for 8% of CPU cycles, and takes 0.8
seconds per invocation (each time the poller runs). 99% of that time is
spent in xml.dom.minidom.parseString(). It should be possible to speed
that up by either switching to a faster xml parser; or *not* using the
--xml option to `svn log` and custom parsing the output.
--
Ticket URL: <http://trac.buildbot.net/ticket/2273>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation
More information about the Commits
mailing list