[Buildbot-commits] buildbot/buildbot/changes p4poller.py,1.8,1.9

[Brian Warner]

Update of /cvsroot/buildbot/buildbot/buildbot/changes
In directory sc8-pr-cvs3.sourceforge.net:/tmp/cvs-serv18011/buildbot/changes

Modified Files:
	p4poller.py 
Log Message:
[project @ p4changes]
Use a more optimal form of "p4 changes", as suggested by Stewart Lord of
Perforce technical support. From the p4d release notes:

  Special handling for @changelist - #85130 **

    The syntax '//path/file at change1, at somethingelse' now performs
    better in many common cases.  Previously, the server would
    search based on changelist number or file path according to
    hard-coded rules.  This worked poorly in a few common cases:
    both '//singlefile at 1, at date' and '//... at 5000,5001' started
    with the wrong search.  This left users trying odd syntax
    to second guess the server.  Now the server adaptively tries
    searching by change number, but if that appears to be too
    inefficient (more than 80% of the revisions not matching the
    path), it switches to searching by path.  (Bug #18689).

In 2wire's case, this was the top source of server load, scanning
~2,200,000 rows for one project and ~2,700,000 rows for another every
10 minutes. Now each scans 10-20 rows on average.
(Figures gathered with -Ztrack=1.)

Original author: Scott Lamb <slamb at slamb.org>
Date: 2007-02-24 02:19:46

Index: p4poller.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/changes/p4poller.py,v
retrieving revision 1.8
retrieving revision 1.9
diff -u -d -r1.8 -r1.9
--- p4poller.py	30 Sep 2006 22:35:59 -0000	1.8
+++ p4poller.py	28 Feb 2007 06:27:49 -0000	1.9
@@ -28,7 +28,7 @@
     them to the change master."""
 
     compare_attrs = ["p4port", "p4user", "p4passwd", "p4base",
-                     "p4bin", "pollinterval", "histmax"]
+                     "p4bin", "pollinterval"]
 
     changes_line_re = re.compile(
             r"Change (?P<num>\d+) on \S+ by \S+@\S+ '.+'$")
@@ -45,7 +45,7 @@
     def __init__(self, p4port=None, p4user=None, p4passwd=None,
                  p4base='//', p4bin='p4',
                  split_file=lambda branchfile: (None, branchfile),
-                 pollinterval=60 * 10, histmax=100):
+                 pollinterval=60 * 10, histmax=None):
         """
         @type  p4port:       string
         @param p4port:       p4 port definition (host:portno)
@@ -63,7 +63,8 @@
         @type  pollinterval: int
         @param pollinterval: interval in seconds between polls
         @type  histmax:      int
-        @param histmax:      maximum number of changes to look back through
+        @param histmax:      (obsolete) maximum number of changes to look back through.
+                             ignored; accepted for backwards compatibility.
         """
 
         self.p4port = p4port
@@ -73,7 +74,6 @@
         self.p4bin = p4bin
         self.split_file = split_file
         self.pollinterval = pollinterval
-        self.histmax = histmax
         self.loop = LoopingCall(self.checkp4)
 
     def startService(self):
@@ -121,7 +121,11 @@
             args.extend(['-u', self.p4user])
         if self.p4passwd:
             args.extend(['-P', self.p4passwd])
-        args.extend(['changes', '-m', str(self.histmax), self.p4base + '...'])
+        args.extend(['changes'])
+        if self.last_change is not None:
+            args.extend(['%s...@%d,now' % (self.p4base, self.last_change+1)])
+        else:
+            args.extend(['-m', '1', '%s...' % (self.p4base,)])
         env = {}
         return getProcessOutput(self.p4bin, args, env)
 
@@ -133,13 +137,11 @@
             if not line: continue
             m = self.changes_line_re.match(line)
             assert m, "Unexpected 'p4 changes' output: %r" % result
-            num = m.group('num')
+            num = int(m.group('num'))
             if last_change is None:
-                log.msg('P4Poller: starting at change %s' % num)
+                log.msg('P4Poller: starting at change %d' % num)
                 self.last_change = num
                 return []
-            if last_change == num:
-                break
             changelists.append(num)
         changelists.reverse() # oldest first
 
@@ -158,7 +160,7 @@
             args.extend(['-u', self.p4user])
         if self.p4passwd:
             args.extend(['-P', self.p4passwd])
-        args.extend(['describe', '-s', num])
+        args.extend(['describe', '-s', str(num)])
         env = {}
         d = getProcessOutput(self.p4bin, args, env)
         return d