[users at bb.net] Accessing properties within an extract_fn

Tue Jan 3 10:42:53 UTC 2017

Hi Elliot,

getRenderingFor is indeed returning a string, but via deferred (even if
your actual usecase does not really need io, the generic renderable case
may).

So if you insist in using Interpolate to build your string, you will need
to make you renderer function an inlineCallbacks, and use

local_filename = yield local_filename.getRenderingFor(props)

That said, I would rather use string.format as you are only doing "prop"
interpolation e.g:

            local_filename =
"contrib/mac/app/Julia-{version}-{shortcommit}.{os_pkg_ext}".format(props.getProperties())

This way you dont need to bother about deferred

I like you proposal of manipulateProperties with full renderable
implementation will give it a try soon.

Pierre

Le mar. 3 janv. 2017 à 11:21, Elliot Saba <staticfloat at gmail.com> a écrit :

> I would love a manipulateProperties, so that I don't have to use a
> workaround like this.  Something that would allow me to return a dictionary
> of new property values (akin to an extract_fn) would be ideal.  Although
> I should say, I'm having some troubles with the workaround right now
> because I'm not entirely certain how to get util.Interpolate() and
> setProperty() to play nicely together.  I have this snippet
> <https://github.com/staticfloat/julia-buildbot/blob/57258667835271c27aba559033898d0000f623ad/master/package.py#L45-L75>
> right now, but I'm getting errors like this:
>
> twisted.internet.defer.FirstError: FirstError[#14, [Failure instance:
> Traceback: <type 'exceptions.TypeError'>: <DeferredList at 0x7f9f3d3d9638
> current result: u'tar.gz'> is not JSON serializable
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:1299:_inlineCallbacks
> /usr/local/lib/python2.7/site-packages/buildbot/process/buildstep.py:508:startStep
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:198:render
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:157:render
> --- <exception caught here> ---
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:150:maybeDeferred
> package.py:73:munge_artifact_filename
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:142:setProperty
> /usr/local/lib/python2.7/json/__init__.py:244:dumps
> /usr/local/lib/python2.7/json/encoder.py:207:encode
> /usr/local/lib/python2.7/json/encoder.py:270:iterencode
> /usr/local/lib/python2.7/json/encoder.py:184:default ]]
> twisted.internet.defer.FirstError: FirstError[#14, [Failure instance:
> Traceback: <type 'exceptions.TypeError'>: <DeferredList at 0x7f9f3d3d9638
> current result: u'tar.gz'> is not JSON serializable
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:1299:_inlineCallbacks
> /usr/local/lib/python2.7/site-packages/buildbot/process/buildstep.py:508:startStep
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:198:render
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:157:render
> --- <exception caught here> ---
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:150:maybeDeferred
> package.py:73:munge_artifact_filename
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:142:setProperty
> /usr/local/lib/python2.7/json/__init__.py:244:dumps
> /usr/local/lib/python2.7/json/encoder.py:207:encode
> /usr/local/lib/python2.7/json/encoder.py:270:iterencode
> /usr/local/lib/python2.7/json/encoder.py:184:default ]]
>
> It appears that the manual "rendering" step I'm attempting to perform in
> order to coerce my util.Interpolate() results to a string are not
> working; I'm getting a DeferredList instead of a string.  What do I need to
> do to massage my interpolation results into a form that I can pass into
> setProperty?
>
> On Tue, Jan 3, 2017 at 12:47 AM, Pierre Tardy <tardyp at gmail.com> wrote:
>
> renderer looks good.
> Maybe we need an extra step kind like manipulateProperties, which would
> only take a renderer and run it, so that we dont need the extra shell
> command on the worker.
>
> BTW, I like you master.cfg a lot. nice work!
>
> Would you like to make a blog post explaining it, like the nice folks at
> i3 did?
>
> https://i3wm.org/docs/buildbot.html
>
> We can host it in buildbot's Medium publication:
> https://medium.com/buildbot
>
> Regards
> Pierre
>
>
> Le mar. 3 janv. 2017 à 00:14, Elliot Saba <staticfloat at gmail.com> a
> écrit :
>
> Thanks Pierre,
>
> I have worked around this in a different way, with a dummy
> setPropertyFromCommand step (my workaround is here
> <https://github.com/staticfloat/julia-buildbot/blob/level_up/master/package.py#L45-L77>)
> that just runs `/bin/true`, but within the `command=` callback I manipulate
> the properties as I desire.
> -E
>
> On Thu, Dec 29, 2016 at 6:15 AM, Pierre Tardy <tardyp at gmail.com> wrote:
>
> Hi Elliot,
>
> Looking at the source code, it is not possible to get the properties from
> extract_fn.
>
> https://github.com/buildbot/buildbot/blob/master/master/buildbot/steps/shell.py#L335
> However, you could easily override commandComplete in order to make your
> implementation.
>
>
> class mySetPropertyFromCommand(steps.SetPropertyFromCommand):
>     def commandComplete(self, cmd):
>         stdout = self.observer.getStdout()
>         stderr = self.observer.getStderr()
>         rc = cmd.rc
>         os = self.getProperty("os")
>         ...
>         self.setProperty("prop_name", "value", "mySetPropertyFromCommand")
>
>
> Le mer. 28 déc. 2016 à 20:09, Elliot Saba <staticfloat at gmail.com> a
> écrit :
>
> Hi Pierre,
>
> I mean the function that gets called to parse out the results of a
> setPropertyFromCommand call.
>
> On Wed, Dec 28, 2016, 04:14 Pierre Tardy <tardyp at gmail.com> wrote:
>
> Hi Elliot
> What do you mean an extraction function?
>
> Le mer. 28 déc. 2016 12:16, Elliot Saba <staticfloat at gmail.com> a écrit :
>
> I want to access the properties of my build within a buildstep.  To do so,
> I need to use something like `step.getProperty('foo')`, but I don't know
> how to get the current step as a variable into my extraction function.  Is
> there an easy way to get this information?  The reason I need to do this is
> that I need to customize my extraction function based upon the operating
> system of the worker my step is executing on.
>
> Thanks in advance,
> -E
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
>
> --
>
> -E
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170103/5f0d834c/attachment.html>