[Buildbot-devel] Create collection of build steps

[Roy S. Rapoport]

On Fri, Sep 29, 2006 at 12:30:12AM +0200, Mateusz Loskot wrote:
> Could anyone explain me how to gain collection of re-usable build steps?

Here's how we do it here.  

To start with, I should note that most of our products can be "built" by
performing the following steps:
"kelpie update -a <product>-<branch>"
"kelpie build <product>-<branch>"   
"copy_tarball <product>-<branch>"

Though some products (as you'll see below) require different steps.

So we defined some helper classes.  First, the generic one:

class CompanyProductSteps(object):
    def __init__(self, name, branch):
        self.name = name
        self.branch = branch
        copy_cmd = "%(workdir)s/tare/head/nightly/copy_tarball "
        copy_cmd += "%(workdir)s "
        copy_cmd += "%s %s" % (name, branch)
        self._steps = [
            ('update', 'kelpie update -a %s-%s' % (name,branch)),
            ('build', 'kelpie build --no-cache %s-%s' %
                (name,branch)),
            ('copy', copy_cmd),
            ]

    def steps(self):
        return self._steps

You'll notice you instantantiate an object with the name and branch of the
product, which will create three steps, each in the form of a (name,
command) tuple.  

Now, we override/inherit as necessary:

class AllProductSteps(CompanyProductSteps):

    def __init__(self, name, branch):
        self._steps = [
            ("update", "kelpie force-update"),
            ("tidy", "kelpie tidy -f -y"),
            ("build", "kelpie build --no-cache build_datestamp=1")
        ]

the 'AllProductSteps' object will be used for building the 'all' target; as
such, it doesn't refer to a specific name or branch, and follows slightly
different build steps.

Lastly, we have a product that has a test suite:

class OurProdProductSteps(CompanyProductSteps):
    def __init__(self, name, branch):
        super(OurProdProductSteps, self).__init__(name, branch)
        self._steps.append(['test', '%(workdir)s/tare/head/ourprod/ourprod-rtests ' + '%s' % (branch)])

So as you can see, this last one uses the steps defined in the base class, and adds a test
step.

We have a dictionary:
CompanyBuildSteps = {}
CompanyBuildSteps['all'] = AllProductSteps
CompanyBuildSteps['ourprod-dist'] = OurProdProductSteps

So later, when we're setting up the builder for, say, ourprod-dist, we do:

productBuildSteps = CompanyBuildSteps.get(product, CompanyProductSteps)

Which will get us one of the above productstep objects; then we do:

newFactory = factory.BuildFactory()

and then:

cmds = stepObject.steps()

for cmd in cmds:
    [desc, cmdStringOrList] = cmd
    cmdArgs = cmdStringOrList
    if type(cmdStringOrList) == type(""):
        cmdArgs = cmdStringOrList.split()
    newFactory.addStep(step.ShellCommand,
        name = desc,
        description = desc,
        descriptionDone = desc,
        command = cmdArgs,
        timeout=1800,
        haltOnFailure = 1,
		)




Does this make any sense?

-roy