[Buildbot-devel] Svnpoller with externals

[GD]

Hello,

In order to solve my externals problem, I wrote a new poller that maintains
a list of externals, for a given svn url. It also provides a method that,
given a file path, checks if it is located in one of the externals.

Then I crated a scheduler that takes a change source object as parameter,
and forces the fileIsImportant to call its change source special method to
check if the change is relevant.

The main drawback is that you have to declare a
poller/scheduler/builder/factory for each location to scan, plus a global
location.

For example,for the following directory layout:
Code/module1.v1/<source files>
Code/module1.v1.3/<source files>
Code/module2.v1/<source files>
Code/module2.v2/<source files>
Int/release1/<externals to the relevant versions of modules, points to
module1.v1.3 module2.v1>
Int/release2/<externals to the relevant versions of modules, points to
module1.v1.3 module2.v2>

I have to declare a classical svnPoller that watches the / of my svn, then a
svnPollerExt/schedulerExt/builder/factory for each one of Int/release1 and
Int/release2
And maybe a scheduler/builder/factory for each directory under code/ if I
have a mean of building those modules apart from each others.

I'll try to write a hook equivalent to the externals polling, if hooks are
available, it might be better than simple polling.


Regards,
Guillaume

###########################################################
###########################################################

class SVNPollerExt(SVNPoller):
    """Instead of polling a svn repo for the file changes through svn logs
	this class polls for externals in a particular directory and
maintains an internal list of externals (self.externals)
	It also provides a method for separate changes in an external or
not.
    """

    def __init__(self, rootpath, **kwargs):
	self.externals = []
	self.rootpath  = rootpath
	SVNPoller.__init__(self, **kwargs)

    # keep the svn loop when polling, there may be some non-externals in the
directory
    def get_externals(self, ignored_prefix=None):
	args = []
	# set the svn command line
	args.extend(["propget", "svn:externals", "--non-interactive", "%s" %
self.svnurl])
        if self.svnuser:
            args.extend(["--username=%s" % self.svnuser])
        if self.svnpasswd:
            args.extend(["--password=%s" % self.svnpasswd])
        d = self.getProcessOutput(args)
        return d
	
    def parse_externals(self, output):
	# parse the command line
	# d should contain a string, in the form name<tab>path\n
	# we only want to keep a list of short pathes, ie. path minus the
svnurl
	table_exts = output.splitlines()
	# empty the current externals list:
	self.externals[:] = []
	for external_def in table_exts:
		ext_name_path = external_def.split()
		if len(ext_name_path) == 2:
			ext_path = ext_name_path[1]
			if ext_path.count(self.rootpath):
	
self.externals.append(ext_path.replace(self.rootpath, "", 1))
			else:
				self.externals.append(ext_path)
	return self.externals

    def checksvn(self):
        if self.working:
            log.msg("SVNPoller(%s) overrun: timer fired but the previous "
                    "poll had not yet finished.")
            self.overrun_counter += 1
            return defer.succeed(None)
        self.working = True

        log.msg("SVNPoller polling")
        if not self._prefix:
            d = self.get_root()
            d.addCallback(self.determine_prefix)
        else:
            d = defer.succeed(self._prefix)

        d.addCallback(self.get_externals)
        d.addCallback(self.parse_externals)
        d.addCallbacks(self.finished_ok, self.finished_failure)
        return d

    def checkFileInExt(self, filepath):
	# as we stored only short pathes from the root of the svn
	# it doesn't matter if the change path is a full url or not
	# Cautious though: sometimes, a change may be a false true
	file = filepath.replace(self.rootpath, "", 1)
	for external in self.externals:
		if file.count(external):
			return true
	return false


class ExtScheduler(Scheduler):
	
	def __init__(self, associatedSource, **kwargs):
		self.associatedSource = associatedSource
		Scheduler.__init__(self, **kwargs)
		self.fileIsImportant = self.checkExt
	
	def checkExt(change):
		result = false
		for file in change.files:
			temp_result =
self.associatedSource.checkFileInExt(file)
			result = result or temp_result
		return result