[Buildbot-devel] patch: support for svn commit-emails.pl change source, for 0.6.2

Justin Mason jm at jmason.org
Mon Dec 13 21:58:40 UTC 2004


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> >>   - I had to hack up svn commit-emails.pl support -- patch in the sf.net
> >>     patches queue
> >
> >Cool, I'll take a look. Is this a notification script that ships with svn or
> >is it a third-party thing?
> 
> It's distributed with svn, as far as I know; that one works with the
> ASF's svn repository, which is what we're using ;)
> 
> However, that patch is slightly incomplete -- while it *does* trigger the
> builds at the right time, parses the rev correctly, and gets blame right, 
> it doesn't seem to be parsing the commit message.   I haven't got
> around to figuring out why though...

I have now -- here's the result.   This patch now works, correctly parsing
the change messages and the list of files.

(Note btw the slight misfeature I spotted with "prefix"; the prefix-compare
code uses f.split(sep) with the directory-separator as the split char.
Unfortunately, this means that prefix cannot be more than one directory, so
e.g. "spamassassin/trunk" cannot be used as a multi-level prefix. all of the
mail-parsing change sources, as far as I can see, have this issue, so it'd
be nice to document it for now at least.)

But apart from that common issue, the patch works nicely ;)


diff -ru buildbot-0.6.2/buildbot/changes/mail.py buildbot-0.6.2jm/buildbot/changes/mail.py
- --- buildbot-0.6.2/buildbot/changes/mail.py     Mon Nov  8 11:43:51 2004
+++ buildbot-0.6.2jm/buildbot/changes/mail.py   Mon Dec 13 13:49:50 2004
@@ -189,6 +189,138 @@

     return change

+# svn "commit-email.pl" handler.  The format is very similar to freshcvs mail;
+# here's a sample:
+
+#  From: username [at] apache.org    [slightly obfuscated to avoid spam here]
+#  To: commits [at] spamassassin.apache.org
+#  Subject: svn commit: r105955 - in spamassassin/trunk: . lib/Mail
+#  ...
+#
+#  Author: username
+#  Date: Sat Nov 20 00:17:49 2004      [note: TZ = local tz on server!]
+#  New Revision: 105955
+#
+#  Modified:   [also Removed: and Added:]
+#    [filename]
+#    ...
+#  Log:
+#  [log message]
+#  ...
+#
+#
+#  Modified: spamassassin/trunk/lib/Mail/SpamAssassin.pm
+#  [unified diff]
+#
+#  [end of mail]
+def parseSvnCommitEmail(self, fd, prefix=None, sep="/"):
+    """Parse messages sent by the svn 'commit-email.pl' trigger.
+    """
+
+    m = Message(fd)
+    # The mail is sent from the person doing the checkin. Assume that the
+    # local username is enough to identify them (this assumes a one-server
+    # cvs-over-rsh environment rather than the server-dirs-shared-over-NFS
+    # model)
+    name, addr = m.getaddr("from")
+    if not addr:
+        return None # no From means this message isn't from FreshCVS
+    at = addr.find("@")
+    if at == -1:
+        who = addr # might still be useful
+    else:
+        who = addr[:at]
+
+    # we take the time of receipt as the time of checkin. Not correct (it
+    # depends upon the email latency), but it avoids the out-of-order-changes
+    # issue. Also syncmail doesn't give us anything better to work with,
+    # unless you count pulling the v1-vs-v2 timestamp out of the diffs, which
+    # would be ugly. TODO: Pulling the 'Date:' header from the mail is a
+    # possibility, and email.Utils.parsedate_tz may be useful. It should be
+    # configurable, however, because there are a lot of broken clocks out
+    # there.
+    when = util.now()
+
+    files = []
+    comments = ""
+    isdir = 0
+    lines = m.fp.readlines()
+    rev = None
+    while lines:
+        line = lines.pop(0)
+
+       # "Author: jmason"
+       match = re.search(r"^Author: (\S+)", line)
+       if match:
+          who = match.group(1)
+
+       # "New Revision: 105955"
+       match = re.search(r"^New Revision: (\d+)", line)
+       if match:
+          rev = match.group(1)
+
+       # possible TODO: use "Date: ..." data here instead of time of
+       # commit message receipt, above.   however, this timestamp is
+       # specified *without* a timezone, in the server's local TZ,
+       # so to be accurate buildbot would need a config setting to
+       # specify the source server's expected TZ setting!  messy.
+
+       # this stanza ends with the "Log:"
+       if (line == "Log:\n"):
+           break
+
+    # commit message is terminated by the file-listing section
+    while lines:
+       line = lines.pop(0)
+       if (line == "Modified:\n" or
+           line == "Added:\n" or
+           line == "Removed:\n"):
+           break
+       comments += line
+    comments = comments.rstrip() + "\n"
+
+
+    while lines:
+       line = lines.pop(0)
+       if line == "\n":
+           break
+       if line.find("Modified:\n") == 0:
+           continue            # ignore this line
+       if line.find("Added:\n") == 0:
+           continue            # ignore this line
+       if line.find("Removed:\n") == 0:
+           continue            # ignore this line
+       line = line.lstrip()
+       line = line.rstrip()
+       # note: it doesn't actually make sense to use portable functions
+       # like os.path.join and os.sep, because these filenames all use
+       # separator conventions established by the remote CVS server (which
+       # is probably running on unix), not the local buildmaster system.
+       thesefiles = line.split(" ")
+       for f in thesefiles:
+           if prefix:
+               # insist that the file start with the prefix: we may get
+               # changes we don't care about too
+              # [TODO: this fails if prefix contains >1 level of path!!
+              # e.g. prefix = "spamassassin/trunk" will never match.]
+               bits = f.split(sep)
+               if bits[0] == prefix:
+                   f = sep.join(bits[1:])
+               else:
+                   print "ignored file from svn commit: prefix "+prefix+" does not match "+bits[0]+" in "+f
+                   break
+
+           # TODO: figure out how new directories are described, set .isdir
+           files.append(f)
+
+    if not files:
+        print "no matching files found, ignoring commit"
+       return None
+
+    change = Change(who, files, comments, isdir, when=when, revision=rev)
+
+    return change
+
 # Bonsai mail parser by Stephen Davis.
 #
 # This handles changes for CVS repositories that are watched by Bonsai
@@ -318,3 +450,9 @@
     __implements__ = (IChangeSource, MaildirSource.__implements__)
     parser = parseBonsaiMail
     name = "Bonsai"
+
+class SvnCommitsMaildirSource(MaildirSource):
+    __implements__ = (IChangeSource, MaildirSource.__implements__)
+    parser = parseSvnCommitEmail
+    name = "SVNCommits"
+
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFBvhCQMJF5cimLx9ARAmxaAKCpeDOYzOrEjl19VSYAPwmCVTUaIQCfQ72P
NCg1BQwEKjkjUzGWhtHR2/4=
=zCDh
-----END PGP SIGNATURE-----





More information about the devel mailing list