[Buildbot-devel] evaluation of WithProperties under a step containing a doStepIf
Jared Grubb
jared.grubb at gmail.com
Thu Aug 23 01:59:15 UTC 2012
So the issue is that the doStepIf is evaluated in parallel with the renderings:
def _startStep_2(self, res):
...
if isinstance(self.doStepIf, bool):
doStep = defer.succeed(self.doStepIf)
else:
doStep = defer.maybeDeferred(self.doStepIf, self)
...
dl = [ doStep ]
for renderable in renderables:
d = self.build.render(getattr(self, renderable))
d.addCallback(setRenderable, renderable)
dl.append(d)
dl = defer.gatherResults(dl)
dl.addCallback(self._startStep_3)
return dl
And whether the step actaully runs is evaluated by _startStep_3 -- after all the renderings are done.
So, given the implementation, the exception you see below is expected and your workaround is correct.
Would you mind filing a trac about this? I personally think your initial expectation of how it works is better than how ti really does work, and it would be fairly easy to fix, I think. And in the alternative, there may be someone else that can explain why it was done like this and shouldnt be fixed.
Thanks!
Jared
On 22 Aug 2012, at 00:04, Jean-Michel Beuken <jean-michel.beuken at uclouvain.be> wrote:
> Hi Jared,
>> What is the exception?
>>
> ----------------------------------
> Traceback (most recent call last):
> File "/opt/python2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 134, in maybeDeferred
> result = f(*args, **kw)
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 501, in startStep
> d.addCallback(self._startStep_2)
> File "/opt/python2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 301, in addCallback
> callbackKeywords=kw)
> File "/opt/python2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 290, in addCallbacks
> self._runCallbacks()
> --- <exception caught here> ---
> File "/opt/python2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 551, in _runCallbacks
> current.result = callback(current.result, *args, **kw)
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 543, in _startStep_2
> setattr(self, renderable, self.build.render(getattr(self, renderable)))
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/properties.py", line 177, in render
> return props.render(value)
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/properties.py", line 137, in render
> return renderable.getRenderingFor(self)
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/properties.py", line 282, in getRenderingFor
> strings.append(pmap[name])
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/properties.py", line 244, in __getitem__
> rv = properties[key]
> File "/opt/python2.7/lib/python2.7/site-packages/buildbot/process/properties.py", line 71, in __getitem__
> rv = self.properties[name][0]
> exceptions.KeyError: 'packname'
> ----------------------------------
>
>
> as workaround, I "solved" my problem by running the step shell.SetProperty() without the doStepIf()
> so the variable packname is initialized
>
> regards
>
> jmb
>> Jared
>>
>> On 17 Aug 2012, at 02:04, Jean-Michel Beuken <jean-michel.beuken at uclouvain.be> wrote:
>>
>>> Hi,
>>>
>>> I migrate my 0.7.12 installation to version 0.8.6p1
>>> Overall, it works pretty well ...
>>> I just have a problem with a step, which, despite the fact that it should not be run ( with a doStepIf ),
>>> the function WithProperties is even evaluated... and then, there is a exception
>>> the property="packname" is only defined in shell.SetProperty if the "sendpack" property is "On"
>>>
>>> it "works" with the version 0.7.12
>>>
>>> a part of my code :
>>>
>>> ------------------------------------------
>>> def making_pack(step):
>>> try:
>>> a = step.getProperty("sendpack")
>>> except:
>>> a = "off"
>>> return a != "off"
>>>
>>> Step_Create_Pack_Buda_gcc44 = [
>>> shell.ShellCommand(name = "pack",
>>> command='make dist',timeout=12000,
>>> doStepIf = making_pack,
>>> workdir = WithProperties("%s_%s","username","mybranch"),
>>> env={ }, logfiles={},
>>> description=["making dist..."], descriptionDone=["pack done"],haltOnFailure = 1,),
>>>
>>> shell.SetProperty(name="packname",command="ls abinit*tar.gz > /dev/null 2>&1 && ls abinit*tar.gz | sed -e 's/\.tar\.gz//'",
>>> doStepIf = making_pack,
>>> workdir = WithProperties("%s_%s","username","mybranch"),
>>> property="packname",descriptionDone=["packname done"], haltOnFailure = 1,),
>>>
>>> FileUpload( doStepIf = making_pack,
>>> slavesrc = WithProperties("%s_%s/%s.tar.gz","username","mybranch","packname"),
>>> masterdest= WithProperties("/var/www/html//UnOfficialPackages/abinit_%s_r%s_%s_unofficial.tar.gz","mybranch","revno","username"),
>>> workdir="",mode=0666,haltOnFailure = 1, ),
>>>
>>> MasterShellCommand(
>>> doStepIf = making_pack,
>>> command= WithProperties("~buildbot/nightly/SendNotificationPackage.sh abinit_%s_r%s_%s_unofficial.tar.gz %s","mybranch","revno","username","sendpack"),
>>> ),
>>>
>>> ]
>>> ---------------------------------------
>>>
>>> is it a bad coding , a "feature" or a bug ?
>>> how to handle this problem ?
>>>
>>> thanks in advance
>>>
>>> jmb
>>>
>>> --
>>> ----------------------------------------------------
>>> Jean-Michel Beuken
>>> Computer Scientist
>>> Software Engineer for ETSF and ABINIT.ORG
>>>
>>> Université catholique de Louvain
>>> pôle NAPS/IMCN (ex-PCPM), Bat. SC17
>>> 8, Chemin des étoiles, Bt L7.03.01
>>> 1348 Louvain-la-Neuve BELGIUM
>>> ----------------------------------------------------
>>> Tel : (3210) 473570 Fax : (3210) 473102
>>> http://www.uclouvain.be/jean-michel.beuken
>>> ----------------------------------------------------
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
>>> Buildbot-devel mailing list
>>> Buildbot-devel at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20120822/963865d4/attachment.html>
More information about the devel
mailing list