[Buildbot-devel] BuildBot, twisted reactor and dynamic config

[Edward Armes]

Hi Vitali,

Thank you for your response. I has coincided doing something similar but
sadly its not quite what I'm trying to achieve. Thanks for your suggestion.

Edward

On Wed, Nov 5, 2014 at 10:19 AM, Vitali Lovich <vlovich at gmail.com> wrote:

> The builder is pretty simple.  CommandInjectionStep is a custom build step
> I have to have the repository generate some sanitized python code that I
> parse the AST for & convert into build steps.
> It’s pretty hacky: a formal JSON protocol might be nicer & safer (if
> potentially more limiting).
> SubmissionBuildMixin is something else that lets me inject locks into a
> build.  Dustin suggested in an earlier thread a better mechanism that would
> be appropriate for buildbot.
>
> class DynamicRemoteBuild(Build, SubmissionBuildMixin):
>     def startBuild(self, build_status, expectations, slavebuilder):
>         SubmissionBuildMixin.configureSubmissionLock(self, build_status)
>         return Build.startBuild(self, build_status, expectations,
> slavebuilder)
>
>     def stepDone(self, result, step):
> # I have a special CommandInjectionStep that I know about.
>         # If this were part of buildbot proper, we could just add all of
> this to the base builder/step (i.e. any step
> # could declare extraNextSteps or something like that.
>         if isinstance(step, CommandInjectionStep):
>             # Python doesn't have an API to do bulk insert in-place
>             # reverse in-place & appending seems like it should be OK
>             toInject = step.stepsToInject()
>             for injectedStep in toInject:
>                 step_status =
> self.build_status.addStepWithName(injectedStep.name)
>                 injectedStep.setStepStatus(step_status)
>
>             log.msg("Injecting {} steps: [{}]".format(len(toInject),
> '\n\t'.join([str(s) for s in toInject])))
>
>             # step == step0
>             # (remaining) self.steps = [step1, step2, step3, step4]
>             # toInject = [step0a, step0b, step0c]
>             # we want self.steps to be:
>             # [step0a, step0b, step0c, step1, step2, step3, step4]
>
>             # reverse both lists:
>             # self.steps = [step4, step3, step2, step1]
>             # toInject = [step0c, step0b, step0a]
>             toInject.reverse()
>             self.steps.reverse()
>             # append all the steps to inject
>             # [step4, step3, step2, step1, step0c, step0b, step0a]
>             self.steps.extend(toInject)
>             # reverse again & we get the intended ordering
>             self.steps.reverse()
>             log.msg("remaining steps after injection =
> {}".format(self.steps))
>
>         # Build is currently an old-style class for some reason
>         if issubclass(self.__class__, object):
>             terminate = super(Build, self).stepDone(result, step)
>         else:
>             terminate = Build.stepDone(self, result, step)
>         return terminate
>
>
>
> On Nov 5, 2014, at 1:48 AM, Vasily <vasslitvinov at pisem.net> wrote:
>
> Hi Vitali,
>
> Could you please post some snippets of how your idea works?
> Maybe posting it to Buildbot Wiki as well would be great...
>
> Thanks,
> Vasily
>
> 2014-11-03 21:37 GMT+03:00 Vitali Lovich <vlovich at gmail.com>:
>
>>
>> > On Nov 3, 2014, at 10:12 AM, Edward Armes <edward.armes at gmail.com>
>> wrote:
>> >
>> > Hi all,
>> >
>> > I'm currently using BuildBot in a personal project and I'm looking into
>> doing dynamic config.
>> >
>> > First of all a disclaimer: I know that the documented way of doing this
>> is to make a change and then tell BuildBot to restart.
>> >
>> > That being said I'm currently looking at more complex methods involving
>> concurrency and the twisted reactor. However I could do with knowing a few
>> more things or knowing the best place to look.
>> >
>> > 1. Does the BuildBot configuration object store the items in some way
>> that would prevent a dynamic dictionary of some form?
>> I did something similar, but without a dynamic dictionary (I don’t
>> imagine it would work).
>> Instead, I just set a custom Build class.  I wanted to allow for a step
>> whereby the project could specify it’s own steps (e.g. build source code,
>> run tests, etc) instead of the typical way of having a single giant
>> shell script.  So I overrode stepDone: if the step that was done was my
>> special injection step, I grab the new set of steps and simply extend
>> self.steps (actually self.steps.reverse(), self.steps.extend(newSteps),
>> self.steps.reverse()).  Then I just delegate to the parent’s stepDone.
>>
>> This works reasonably well.
>>
>> > 2. Can all items that use the configuration object handle twisted defer
>> objects?
>> >
>> > 3. How difficult is it to customize and modify the web status to add
>> auth (via twisted.cred).
>> >
>> > 4. Other than the "magic" URL's described in the docs are there any
>> other undocumented gotchas to watch out for?
>> >
>> ------------------------------------------------------------------------------
>> > _______________________________________________
>> > Buildbot-devel mailing list
>> > Buildbot-devel at lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20141106/8bc1f849/attachment.html>