[users at bb.net] Accessing properties within an extract_fn

Tue Jan 3 10:20:32 UTC 2017

I would love a manipulateProperties, so that I don't have to use a
workaround like this.  Something that would allow me to return a dictionary
of new property values (akin to an extract_fn) would be ideal.  Although I
should say, I'm having some troubles with the workaround right now because
I'm not entirely certain how to get util.Interpolate() and setProperty() to
play nicely together.  I have this snippet
<https://github.com/staticfloat/julia-buildbot/blob/57258667835271c27aba559033898d0000f623ad/master/package.py#L45-L75>
right now, but I'm getting errors like this:

twisted.internet.defer.FirstError: FirstError[#14, [Failure instance:
Traceback: <type 'exceptions.TypeError'>: <DeferredList at 0x7f9f3d3d9638
current result: u'tar.gz'> is not JSON serializable
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:1299:_inlineCallbacks
/usr/local/lib/python2.7/site-packages/buildbot/process/buildstep.py:508:startStep
/usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:198:render
/usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:157:render
--- <exception caught here> --- /usr/local/lib/python2.7/site-
packages/twisted/internet/defer.py:150:maybeDeferred
package.py:73:munge_artifact_filename /usr/local/lib/python2.7/site-
packages/buildbot/process/properties.py:142:setProperty
/usr/local/lib/python2.7/json/__init__.py:244:dumps
/usr/local/lib/python2.7/json/encoder.py:207:encode
/usr/local/lib/python2.7/json/encoder.py:270:iterencode
/usr/local/lib/python2.7/json/encoder.py:184:default ]]
twisted.internet.defer.FirstError: FirstError[#14, [Failure instance:
Traceback: <type 'exceptions.TypeError'>: <DeferredList at 0x7f9f3d3d9638
current result: u'tar.gz'> is not JSON serializable
/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:1299:_inlineCallbacks
/usr/local/lib/python2.7/site-packages/buildbot/process/buildstep.py:508:startStep
/usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:198:render
/usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:157:render
--- <exception caught here> --- /usr/local/lib/python2.7/site-
packages/twisted/internet/defer.py:150:maybeDeferred
package.py:73:munge_artifact_filename /usr/local/lib/python2.7/site-
packages/buildbot/process/properties.py:142:setProperty
/usr/local/lib/python2.7/json/__init__.py:244:dumps
/usr/local/lib/python2.7/json/encoder.py:207:encode
/usr/local/lib/python2.7/json/encoder.py:270:iterencode
/usr/local/lib/python2.7/json/encoder.py:184:default ]]

It appears that the manual "rendering" step I'm attempting to perform in
order to coerce my util.Interpolate() results to a string are not working;
I'm getting a DeferredList instead of a string.  What do I need to do to
massage my interpolation results into a form that I can pass into
setProperty?

On Tue, Jan 3, 2017 at 12:47 AM, Pierre Tardy <tardyp at gmail.com> wrote:

> renderer looks good.
> Maybe we need an extra step kind like manipulateProperties, which would
> only take a renderer and run it, so that we dont need the extra shell
> command on the worker.
>
> BTW, I like you master.cfg a lot. nice work!
>
> Would you like to make a blog post explaining it, like the nice folks at
> i3 did?
>
> https://i3wm.org/docs/buildbot.html
>
> We can host it in buildbot's Medium publication:
> https://medium.com/buildbot
>
> Regards
> Pierre
>
>
> Le mar. 3 janv. 2017 à 00:14, Elliot Saba <staticfloat at gmail.com> a
> écrit :
>
>> Thanks Pierre,
>>
>> I have worked around this in a different way, with a dummy
>> setPropertyFromCommand step (my workaround is here
>> <https://github.com/staticfloat/julia-buildbot/blob/level_up/master/package.py#L45-L77>)
>> that just runs `/bin/true`, but within the `command=` callback I manipulate
>> the properties as I desire.
>> -E
>>
>> On Thu, Dec 29, 2016 at 6:15 AM, Pierre Tardy <tardyp at gmail.com> wrote:
>>
>> Hi Elliot,
>>
>> Looking at the source code, it is not possible to get the properties from
>> extract_fn.
>> https://github.com/buildbot/buildbot/blob/master/master/buil
>> dbot/steps/shell.py#L335
>> However, you could easily override commandComplete in order to make your
>> implementation.
>>
>>
>> class mySetPropertyFromCommand(steps.SetPropertyFromCommand):
>>     def commandComplete(self, cmd):
>>         stdout = self.observer.getStdout()
>>         stderr = self.observer.getStderr()
>>         rc = cmd.rc
>>         os = self.getProperty("os")
>>         ...
>>         self.setProperty("prop_name", "value", "mySetPropertyFromCommand")
>>
>>
>> Le mer. 28 déc. 2016 à 20:09, Elliot Saba <staticfloat at gmail.com> a
>> écrit :
>>
>> Hi Pierre,
>>
>> I mean the function that gets called to parse out the results of a
>> setPropertyFromCommand call.
>>
>> On Wed, Dec 28, 2016, 04:14 Pierre Tardy <tardyp at gmail.com> wrote:
>>
>> Hi Elliot
>> What do you mean an extraction function?
>>
>> Le mer. 28 déc. 2016 12:16, Elliot Saba <staticfloat at gmail.com> a écrit :
>>
>> I want to access the properties of my build within a buildstep.  To do
>> so, I need to use something like `step.getProperty('foo')`, but I don't
>> know how to get the current step as a variable into my extraction
>> function.  Is there an easy way to get this information?  The reason I need
>> to do this is that I need to customize my extraction function based upon
>> the operating system of the worker my step is executing on.
>>
>> Thanks in advance,
>> -E
>>
>> _______________________________________________
>> users mailing list
>> users at buildbot.net
>> https://lists.buildbot.net/mailman/listinfo/users
>>
>> --
>>
>> -E
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170103/7c5f5d8a/attachment.html>