[Buildbot-devel] BuildBot, twisted reactor and dynamic config

[Vitali Lovich]

The builder is pretty simple.  CommandInjectionStep is a custom build step I have to have the repository generate some sanitized python code that I parse the AST for & convert into build steps.
It’s pretty hacky: a formal JSON protocol might be nicer & safer (if potentially more limiting).
SubmissionBuildMixin is something else that lets me inject locks into a build.  Dustin suggested in an earlier thread a better mechanism that would be appropriate for buildbot.

class DynamicRemoteBuild(Build, SubmissionBuildMixin):
    def startBuild(self, build_status, expectations, slavebuilder):
        SubmissionBuildMixin.configureSubmissionLock(self, build_status)
        return Build.startBuild(self, build_status, expectations, slavebuilder)

    def stepDone(self, result, step):
	# I have a special CommandInjectionStep that I know about.
        # If this were part of buildbot proper, we could just add all of this to the base builder/step (i.e. any step
	# could declare extraNextSteps or something like that.
        if isinstance(step, CommandInjectionStep):
            # Python doesn't have an API to do bulk insert in-place
            # reverse in-place & appending seems like it should be OK
            toInject = step.stepsToInject()
            for injectedStep in toInject:
                step_status = self.build_status.addStepWithName(injectedStep.name)
                injectedStep.setStepStatus(step_status)

            log.msg("Injecting {} steps: [{}]".format(len(toInject), '\n\t'.join([str(s) for s in toInject])))

            # step == step0
            # (remaining) self.steps = [step1, step2, step3, step4]
            # toInject = [step0a, step0b, step0c]
            # we want self.steps to be:
            # [step0a, step0b, step0c, step1, step2, step3, step4]

            # reverse both lists:
            # self.steps = [step4, step3, step2, step1]
            # toInject = [step0c, step0b, step0a]
            toInject.reverse()
            self.steps.reverse()
            # append all the steps to inject
            # [step4, step3, step2, step1, step0c, step0b, step0a]
            self.steps.extend(toInject)
            # reverse again & we get the intended ordering
            self.steps.reverse()
            log.msg("remaining steps after injection = {}".format(self.steps))

        # Build is currently an old-style class for some reason
        if issubclass(self.__class__, object):
            terminate = super(Build, self).stepDone(result, step)
        else:
            terminate = Build.stepDone(self, result, step)
        return terminate                                                                                     

> On Nov 5, 2014, at 1:48 AM, Vasily <vasslitvinov at pisem.net> wrote:
> 
> Hi Vitali,
> 
> Could you please post some snippets of how your idea works?
> Maybe posting it to Buildbot Wiki as well would be great...
> 
> Thanks,
> Vasily
> 
> 2014-11-03 21:37 GMT+03:00 Vitali Lovich <vlovich at gmail.com <mailto:vlovich at gmail.com>>:
> 
> > On Nov 3, 2014, at 10:12 AM, Edward Armes <edward.armes at gmail.com <mailto:edward.armes at gmail.com>> wrote:
> >
> > Hi all,
> >
> > I'm currently using BuildBot in a personal project and I'm looking into doing dynamic config.
> >
> > First of all a disclaimer: I know that the documented way of doing this is to make a change and then tell BuildBot to restart.
> >
> > That being said I'm currently looking at more complex methods involving concurrency and the twisted reactor. However I could do with knowing a few more things or knowing the best place to look.
> >
> > 1. Does the BuildBot configuration object store the items in some way that would prevent a dynamic dictionary of some form?
> I did something similar, but without a dynamic dictionary (I don’t imagine it would work).
> Instead, I just set a custom Build class.  I wanted to allow for a step whereby the project could specify it’s own steps (e.g. build source code, run tests, etc) instead of the typical way of having a single giant
> shell script.  So I overrode stepDone: if the step that was done was my special injection step, I grab the new set of steps and simply extend self.steps (actually self.steps.reverse(), self.steps.extend(newSteps), self.steps.reverse()).  Then I just delegate to the parent’s stepDone.
> 
> This works reasonably well.
> 
> > 2. Can all items that use the configuration object handle twisted defer objects?
> >
> > 3. How difficult is it to customize and modify the web status to add auth (via twisted.cred).
> >
> > 4. Other than the "magic" URL's described in the docs are there any other undocumented gotchas to watch out for?
> > ------------------------------------------------------------------------------
> > _______________________________________________
> > Buildbot-devel mailing list
> > Buildbot-devel at lists.sourceforge.net <mailto:Buildbot-devel at lists.sourceforge.net>
> > https://lists.sourceforge.net/lists/listinfo/buildbot-devel <https://lists.sourceforge.net/lists/listinfo/buildbot-devel>
> 
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net <mailto:Buildbot-devel at lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel <https://lists.sourceforge.net/lists/listinfo/buildbot-devel>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20141105/1bd09741/attachment.html>