[Buildbot-devel] copying properties from a triggered build to the triggering build

AMARASINGHAM, Chandra Chandra.Amarasingham at sdspathology.com.au
Tue Jun 24 23:48:38 UTC 2014


Thanks Yasser.



That's what I tried but it does not seem to work for some reason, even though I have confirmed that the parent property gets set.  My thinking is that there is a slight delay in the parent property being set due to the asynchronicity of twisted so that the second step fires before the parent property is set.  I maybe wrong though...



Chandra



Chandra Amarasingham
Analyst Programmer
Specialist Diagnostic Services Pty Ltd
17 Enterprise Grove, Mt Helen, 3350
Federation University Technology Park

T: (03) 5330 1056 ext 212

This email (including any attachments) is intended only for the addressee.  It may contain confidential or legally privileged information. Confidentiality and privilege are not waived or lost if you are not the intended recipient of this email, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. If you receive this email in error, please delete it and any attachments from your system and notify us immediately. It is your responsibility to check this email and any attachments, before opening or using them, for viruses or defects. You assume all liability arising from opening or using this email and any attachment.
________________________________
From: Yasser LEBON [yasser.lebon at gmail.com]
Sent: Tuesday, 24 June 2014 7:38 PM
To: AMARASINGHAM, Chandra
Cc: buildbot-devel at lists.sourceforge.net
Subject: Re: [Buildbot-devel] copying properties from a triggered build to the triggering build

You can do something like this to set the sharre the property to your next build.

 step1 = Trigger(
        set_parent_properties = ['<your_shared_property>']
 )

 step2 = Trigger(
         set_properties = {
            'property1': <your_shared_property>
         }
 )




2014-06-24 3:09 GMT+02:00 AMARASINGHAM, Chandra <Chandra.Amarasingham at sdspathology.com.au<mailto:Chandra.Amarasingham at sdspathology.com.au>>:

the property does get set on the parent...I can see it...but I think there is a slight delay, so the next trigger does not get it.

Chandra



Chandra Amarasingham
Analyst Programmer
Specialist Diagnostic Services Pty Ltd
17 Enterprise Grove, Mt Helen, 3350
Federation University Technology Park

T: (03) 5330 1056 ext 212<tel:%2803%29%205330%201056%20ext%20212>

This email (including any attachments) is intended only for the addressee.  It may contain confidential or legally privileged information. Confidentiality and privilege are not waived or lost if you are not the intended recipient of this email, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. If you receive this email in error, please delete it and any attachments from your system and notify us immediately. It is your responsibility to check this email and any attachments, before opening or using them, for viruses or defects. You assume all liability arising from opening or using this email and any attachment.
________________________________
From: AMARASINGHAM, Chandra
Sent: Tuesday, 24 June 2014 11:00 AM
To: buildbot-devel at lists.sourceforge.net<mailto:buildbot-devel at lists.sourceforge.net>

Subject: Re: [Buildbot-devel] copying properties from a triggered build to the triggering build


Hi Vasily, Yasser.



Yasser's suggestion worked for me except for one hitch...I think due to the asynchronous nature of things.  I copied code from where Yasser suggested and have included the new class I created below.  I think the problem is due to self.createLinksAndProperties firing slightly after the next buildstep (also a trigger) runs.  I want the next buildstep to inherit a property from the previous trigger, thus the reason for this code.  Any comments or ideas?



class CopyPropertyParentTrigger(LoggingBuildStep):
    name = "trigger"

    renderables = [ 'set_properties', 'schedulerNames', 'sourceStamps',
                    'updateSourceStamp', 'alwaysUseLatest' ]

    flunkOnFailure = True

    def __init__(self, schedulerNames=[], sourceStamp = None, sourceStamps = None,
                 updateSourceStamp=None, alwaysUseLatest=False,
                 waitForFinish=False, set_properties={}, set_parent_properties=[],
                 copy_properties=[], **kwargs):
        if not schedulerNames:
            config.error(
                "You must specify a scheduler to trigger")
        if (sourceStamp or sourceStamps) and (updateSourceStamp is not None):
            config.error(
                "You can't specify both sourceStamps and updateSourceStamp")
        if (sourceStamp or sourceStamps) and alwaysUseLatest:
            config.error(
                "You can't specify both sourceStamps and alwaysUseLatest")
        if alwaysUseLatest and (updateSourceStamp is not None):
            config.error(
                "You can't specify both alwaysUseLatest and updateSourceStamp"
            )
        self.schedulerNames = schedulerNames
        self.sourceStamps = sourceStamps or []
        if sourceStamp:
            self.sourceStamps.append(sourceStamp)
        if updateSourceStamp is not None:
            self.updateSourceStamp = updateSourceStamp
        else:
            self.updateSourceStamp = not (alwaysUseLatest or self.sourceStamps)
        self.alwaysUseLatest = alwaysUseLatest
        self.waitForFinish = waitForFinish
        properties = {}
        properties.update(set_properties)
        for i in copy_properties:
            properties[i] = Property(i)
        self.set_properties = properties
        self.set_parent_properties = set_parent_properties
        self.running = False
        self.ended = False
        LoggingBuildStep.__init__(self, **kwargs)

    def interrupt(self, reason):
        if self.running and not self.ended:
            self.step_status.setText(["interrupted"])
            return self.end(EXCEPTION)

    def end(self, result):
        if not self.ended:
            self.ended = True
            return self.finished(result)

    # Create the properties that are used for the trigger
    def createTriggerProperties(self):
        # make a new properties object from a dict rendered by the old
        # properties object
        trigger_properties = Properties()
        trigger_properties.update(self.set_properties, "Trigger")
        return trigger_properties

    # Get all scheduler instances that were configured
    # A tuple of (triggerables, invalidnames) is returned
    def getSchedulers(self):
        all_schedulers = self.build.builder.botmaster.parent.allSchedulers()
        all_schedulers = dict([(sch.name<http://sch.name>, sch) for sch in all_schedulers])
        invalid_schedulers = []
        triggered_schedulers = []
        # don't fire any schedulers if we discover an unknown one
        for scheduler in self.schedulerNames:
            scheduler = scheduler
            if all_schedulers.has_key(scheduler):
                sch = all_schedulers[scheduler]
                if ITriggerableScheduler.providedBy(sch):
                    triggered_schedulers.append(sch)
                else:
                    invalid_schedulers.append(scheduler)
            else:
                invalid_schedulers.append(scheduler)

        return (triggered_schedulers, invalid_schedulers)

    def prepareSourcestampListForTrigger(self):
        if self.sourceStamps:
            ss_for_trigger = {}
            for ss in self.sourceStamps:
                codebase = ss.get('codebase','')
                assert codebase not in ss_for_trigger, "codebase specified multiple times"
                ss_for_trigger[codebase] = ss
            return ss_for_trigger

        if self.alwaysUseLatest:
            return {}

        # start with the sourcestamps from current build
        ss_for_trigger = {}
        objs_from_build = self.build.getAllSourceStamps()
        for ss in objs_from_build:
            ss_for_trigger[ss.codebase] = ss.asDict()

        # overrule revision in sourcestamps with got revision
        if self.updateSourceStamp:
            got = self.build.build_status.getAllGotRevisions()
            for codebase in ss_for_trigger:
                if codebase in got:
                    ss_for_trigger[codebase]['revision'] = got[codebase]

        return ss_for_trigger

    @defer.inlineCallbacks
    def start(self):
        # Get all triggerable schedulers and check if there are invalid schedules
        (triggered_schedulers, invalid_schedulers) = self.getSchedulers()
        if invalid_schedulers:
            self.step_status.setText(['not valid scheduler:'] + invalid_schedulers)
            self.end(FAILURE)
            return

        self.running = True

        props_to_set = self.createTriggerProperties()

        ss_for_trigger = self.prepareSourcestampListForTrigger()

        dl = []
        triggered_names = []
        for sch in triggered_schedulers:
            dl.append(sch.trigger(ss_for_trigger, set_props=props_to_set))
            triggered_names.append(sch.name<http://sch.name>)
        self.step_status.setText(['triggered'] + triggered_names)

        if self.waitForFinish:
            rclist = yield defer.DeferredList(dl, consumeErrors=1)
        else:
            # do something to handle errors
            for d in dl:
                d.addErrback(log.err,
                    '(ignored) while invoking Triggerable schedulers:')
            rclist = None
            self.end(SUCCESS)
            return

        was_exception = was_failure = False
        brids = {}
        for was_cb, results in rclist:
            if isinstance(results, tuple):
                results, some_brids = results
                brids.update(some_brids)

            if not was_cb:
                was_exception = True
                log.err(results)
                continue

            if results==FAILURE:
                was_failure = True

        if was_exception:
            result = EXCEPTION
        elif was_failure:
            result = FAILURE
        else:
            result = SUCCESS

        if brids:
            master = self.build.builder.botmaster.parent
            def add_links(res):
                # reverse the dictionary lookup for brid to builder name
                brid_to_bn = dict((_brid,_bn) for _bn,_brid in brids.iteritems())

                for was_cb, builddicts in res:
                    if was_cb:
                        for build in builddicts:
                            bn = brid_to_bn[build['brid']]
                            num = build['number']

                            url = master.status.getURLForBuild(bn, num)
                            self.step_status.addURL("%s #%d" % (bn,num), url)

                return self.end(result)

            builddicts = [master.db.builds.getBuildsForRequest(br) for br in brids.values()]
            dl = defer.DeferredList(builddicts, consumeErrors=1)
            dl.addCallback(self.createLinksAndProperties, result, brids)
 #           import time
#            while not dl.called:
#            time.sleep(1000)
#                pass
            #log = LogPublisher()
            #log.msg("after adding createLinksAndProperties callback")
            self.step_status.setText("after adding createLinksAndProperties callback")

        self.end(result)
        return

    def createLinksAndProperties(self, res, result, brids):

        master = self.build.builder.botmaster.parent

        # reverse the dictionary lookup for brid to builder name

        brid_to_bn = dict((_brid, _bn) for _bn, _brid in brids.iteritems())



        for was_cb, builddicts in res:

            if was_cb:

                for build in builddicts:

                    bn = brid_to_bn[build['brid']]

                    num = build['number']

                    url = master.status.getURLForBuild(bn, num)

                    self.step_status.addURL("%s #%d" % (bn, num), url)
                    self.step_status.setText("in createLinksAndProperties")


            if self.waitForFinish and self.set_parent_properties:

                build_instance = master.status.getBuilder(bn).getBuildByNumber(build['number'])

                for prop_name in self.set_parent_properties:

                    if build_instance.getProperty(prop_name):

                        prop_value = build_instance.getProperty(prop_name)

                        self.setProperty(prop_name, prop_value, "Trigger")
                        self.step_status.setText("in setting property " + prop_name)
                        #log = LogPublisher()
                        #log.msg("after setting property")




        return self.end(result)



Chandra Amarasingham
Analyst Programmer
Specialist Diagnostic Services Pty Ltd
17 Enterprise Grove, Mt Helen, 3350
Federation University Technology Park

T: (03) 5330 1056 ext 212<tel:%2803%29%205330%201056%20ext%20212>

This email (including any attachments) is intended only for the addressee.  It may contain confidential or legally privileged information. Confidentiality and privilege are not waived or lost if you are not the intended recipient of this email, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. If you receive this email in error, please delete it and any attachments from your system and notify us immediately. It is your responsibility to check this email and any attachments, before opening or using them, for viruses or defects. You assume all liability arising from opening or using this email and any attachment.
________________________________
From: Vasily [vasslitvinov at pisem.net<mailto:vasslitvinov at pisem.net>]
Sent: Wednesday, 18 June 2014 6:10 PM
To: AMARASINGHAM, Chandra
Cc: buildbot-devel at lists.sourceforge.net<mailto:buildbot-devel at lists.sourceforge.net>
Subject: Re: [Buildbot-devel] copying properties from a triggered build to the triggering build


Hi,

That seems to be impossible, at least without tinkering with low-level stuff.

Could you post what you're trying to achieve? Maybe there's an easier way...

Thanks,
Vasily

18.06.2014 6:15 пользователь "AMARASINGHAM, Chandra" <Chandra.Amarasingham at sdspathology.com.au<mailto:Chandra.Amarasingham at sdspathology.com.au>> написал:

Hi All,



Is there an easy way to copy properties from a triggered build to the triggering build?



The approach I am trying to take at the moment, is to pass a property which contains a parent build identifier (I am a little confused as to what I should use...bid, brid, bsid.  And the set the property on the parent build from the triggered build....I am bit confused as to how to achieve this also.



Regards,

Chandra



Chandra Amarasingham
Analyst Programmer
Specialist Diagnostic Services Pty Ltd
17 Enterprise Grove, Mt Helen, 3350
Federation University Technology Park

T: (03) 5330 1056 ext 212<tel:%2803%29%205330%201056%20ext%20212>

This email (including any attachments) is intended only for the addressee.  It may contain confidential or legally privileged information. Confidentiality and privilege are not waived or lost if you are not the intended recipient of this email, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. If you receive this email in error, please delete it and any attachments from your system and notify us immediately. It is your responsibility to check this email and any attachments, before opening or using them, for viruses or defects. You assume all liability arising from opening or using this email and any attachment.

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel at lists.sourceforge.net<mailto:Buildbot-devel at lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/buildbot-devel


------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel at lists.sourceforge.net<mailto:Buildbot-devel at lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/buildbot-devel


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20140624/d6082880/attachment.html>


More information about the devel mailing list