[Buildbot-devel] How to dynamically monitor trunk plus multiple branches in svn

[David Young]

Classification: Public

We're using Buildbot 0.8.7 tocurrently monitors changes stored in 
subversion on the trunk only. We want to monitor branches as well, and we 
branch *a lot* (hundreds of branches) We devised a solution that "works" 
as far as it goes, in that upon startup the master queries subversion for 
all existing branches and sets up an SVNPoller ChangeSource for each 
branch, along with corresponding Schedulers and Builders for each and 
every branch (and the trunk). 

I don't see this solution scaling and I don't think I'm approaching this 
properly. Is there an example master.cfg that illustrates what I'm trying 
to do? Here's the solution I have that a) won't scale I believe; and b) 
doesn't pick up new branches as they're created.

Thanks very much.

david

stdout, stderr = subprocess.Popen('svn list %s/branches' % svnroot, 
shell=True, stdout=subprocess.PIPE).communicate()
hp_branches = ['/trunk'] + ['/branches/%s' % b.strip('/') for b in 
stdout.split('\n')[:-1]]

def artifact_name(prefix, branch):
    return prefix + '-' + branch.split('/')[-1]

c['change_source'] = []

for branch in hp_branches:
    repourl = svnroot + branch
    poller = SVNPoller(repourl, pollinterval=60, project='Hosting 
Platform', histmax=10)
    c['change_source'].append(poller)

from buildbot.changes.pb import PBChangeSource

c['change_source'].append(PBChangeSource())

####### SCHEDULERS

from buildbot.schedulers.basic import Scheduler
from buildbot.schedulers.basic import SingleBranchScheduler
from buildbot.changes.filter import ChangeFilter

c['schedulers'] = []

for branch in hp_branches:
    repourl = svnroot + branch
    sname = artifact_name('hostingplatform', branch)
    bs = SingleBranchScheduler(name=sname, treeStableTimer=20,
 change_filter=ChangeFilter(repository=repourl),
                               builderNames=[sname])
    c['schedulers'].append(bs)

####### BUILDERS

# The 'builders' list defines the Builders, which tell Buildbot how to 
perform a build:
# what steps, and which slaves can execute them.  Note that any particular 
build will
# only take place on one slave.

from buildbot.process.factory import BuildFactory
#from buildbot.steps.source.svn import SVN
from buildbot.steps import shell, source
from buildbot.config import BuilderConfig

c['builders'] = []

for branch in hp_branches:
    repourl = svnroot + branch
    bname = artifact_name('hostingplatform', branch)
    fact = BuildFactory()

    fact.addStep(source.SVN(svnurl=repourl, mode='clobber', 
workdir='builddir'))
    fact.addStep(shell.ShellCommand(command=['./build.sh'], 
workdir='builddir'))

    # fact.addStep(SVN(repourl=repourl, mode='full'))
    # fact.addStep(shell.ShellCommand(command=['./build.sh']))

    bc = BuilderConfig(name=bname, slavenames=['hostingportal_slave'], 
factory=fact)
    c['builders'].append(bc)

---
This communication may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this communication
in error) please notify the sender immediately and destroy this
communication. Any unauthorized copying, disclosure or distribution of the
material in this communication is strictly forbidden.

Deutsche Bank does not render legal or tax advice, and the information
contained in this communication should not be regarded as such.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20131031/96051118/attachment.html>