[users at bb.net] Accessing properties within an extract_fn

Pierre Tardy tardyp at gmail.com
Tue Jan 3 13:23:12 UTC 2017


Please see:
https://github.com/buildbot/buildbot/pull/2554/files

I hope you don't mind, I took your artifact name function as an concrete
example.

Regards,
Pierre

Le mar. 3 janv. 2017 à 11:42, Pierre Tardy <tardyp at gmail.com> a écrit :

> Hi Elliot,
>
> getRenderingFor is indeed returning a string, but via deferred (even if
> your actual usecase does not really need io, the generic renderable case
> may).
>
> So if you insist in using Interpolate to build your string, you will need
> to make you renderer function an inlineCallbacks, and use
>
> local_filename = yield local_filename.getRenderingFor(props)
>
>
> That said, I would rather use string.format as you are only doing "prop"
> interpolation e.g:
>
>             local_filename =
> "contrib/mac/app/Julia-{version}-{shortcommit}.{os_pkg_ext}".format(props.getProperties())
>
> This way you dont need to bother about deferred
>
> I like you proposal of manipulateProperties with full renderable
> implementation will give it a try soon.
>
> Pierre
>
> Le mar. 3 janv. 2017 à 11:21, Elliot Saba <staticfloat at gmail.com> a
> écrit :
>
> I would love a manipulateProperties, so that I don't have to use a
> workaround like this.  Something that would allow me to return a dictionary
> of new property values (akin to an extract_fn) would be ideal.  Although
> I should say, I'm having some troubles with the workaround right now
> because I'm not entirely certain how to get util.Interpolate() and
> setProperty() to play nicely together.  I have this snippet
> <https://github.com/staticfloat/julia-buildbot/blob/57258667835271c27aba559033898d0000f623ad/master/package.py#L45-L75>
> right now, but I'm getting errors like this:
>
> twisted.internet.defer.FirstError: FirstError[#14, [Failure instance:
> Traceback: <type 'exceptions.TypeError'>: <DeferredList at 0x7f9f3d3d9638
> current result: u'tar.gz'> is not JSON serializable
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:1299:_inlineCallbacks
> /usr/local/lib/python2.7/site-packages/buildbot/process/buildstep.py:508:startStep
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:198:render
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:157:render
> --- <exception caught here> ---
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:150:maybeDeferred
> package.py:73:munge_artifact_filename
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:142:setProperty
> /usr/local/lib/python2.7/json/__init__.py:244:dumps
> /usr/local/lib/python2.7/json/encoder.py:207:encode
> /usr/local/lib/python2.7/json/encoder.py:270:iterencode
> /usr/local/lib/python2.7/json/encoder.py:184:default ]]
> twisted.internet.defer.FirstError: FirstError[#14, [Failure instance:
> Traceback: <type 'exceptions.TypeError'>: <DeferredList at 0x7f9f3d3d9638
> current result: u'tar.gz'> is not JSON serializable
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:1299:_inlineCallbacks
> /usr/local/lib/python2.7/site-packages/buildbot/process/buildstep.py:508:startStep
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:198:render
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:157:render
> --- <exception caught here> ---
> /usr/local/lib/python2.7/site-packages/twisted/internet/defer.py:150:maybeDeferred
> package.py:73:munge_artifact_filename
> /usr/local/lib/python2.7/site-packages/buildbot/process/properties.py:142:setProperty
> /usr/local/lib/python2.7/json/__init__.py:244:dumps
> /usr/local/lib/python2.7/json/encoder.py:207:encode
> /usr/local/lib/python2.7/json/encoder.py:270:iterencode
> /usr/local/lib/python2.7/json/encoder.py:184:default ]]
>
> It appears that the manual "rendering" step I'm attempting to perform in
> order to coerce my util.Interpolate() results to a string are not
> working; I'm getting a DeferredList instead of a string.  What do I need to
> do to massage my interpolation results into a form that I can pass into
> setProperty?
>
> On Tue, Jan 3, 2017 at 12:47 AM, Pierre Tardy <tardyp at gmail.com> wrote:
>
> renderer looks good.
> Maybe we need an extra step kind like manipulateProperties, which would
> only take a renderer and run it, so that we dont need the extra shell
> command on the worker.
>
> BTW, I like you master.cfg a lot. nice work!
>
> Would you like to make a blog post explaining it, like the nice folks at
> i3 did?
>
> https://i3wm.org/docs/buildbot.html
>
> We can host it in buildbot's Medium publication:
> https://medium.com/buildbot
>
> Regards
> Pierre
>
>
> Le mar. 3 janv. 2017 à 00:14, Elliot Saba <staticfloat at gmail.com> a
> écrit :
>
> Thanks Pierre,
>
> I have worked around this in a different way, with a dummy
> setPropertyFromCommand step (my workaround is here
> <https://github.com/staticfloat/julia-buildbot/blob/level_up/master/package.py#L45-L77>)
> that just runs `/bin/true`, but within the `command=` callback I manipulate
> the properties as I desire.
> -E
>
> On Thu, Dec 29, 2016 at 6:15 AM, Pierre Tardy <tardyp at gmail.com> wrote:
>
> Hi Elliot,
>
> Looking at the source code, it is not possible to get the properties from
> extract_fn.
>
> https://github.com/buildbot/buildbot/blob/master/master/buildbot/steps/shell.py#L335
> However, you could easily override commandComplete in order to make your
> implementation.
>
>
> class mySetPropertyFromCommand(steps.SetPropertyFromCommand):
>     def commandComplete(self, cmd):
>         stdout = self.observer.getStdout()
>         stderr = self.observer.getStderr()
>         rc = cmd.rc
>         os = self.getProperty("os")
>         ...
>         self.setProperty("prop_name", "value", "mySetPropertyFromCommand")
>
>
> Le mer. 28 déc. 2016 à 20:09, Elliot Saba <staticfloat at gmail.com> a
> écrit :
>
> Hi Pierre,
>
> I mean the function that gets called to parse out the results of a
> setPropertyFromCommand call.
>
> On Wed, Dec 28, 2016, 04:14 Pierre Tardy <tardyp at gmail.com> wrote:
>
> Hi Elliot
> What do you mean an extraction function?
>
> Le mer. 28 déc. 2016 12:16, Elliot Saba <staticfloat at gmail.com> a écrit :
>
> I want to access the properties of my build within a buildstep.  To do so,
> I need to use something like `step.getProperty('foo')`, but I don't know
> how to get the current step as a variable into my extraction function.  Is
> there an easy way to get this information?  The reason I need to do this is
> that I need to customize my extraction function based upon the operating
> system of the worker my step is executing on.
>
> Thanks in advance,
> -E
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
>
> --
>
> -E
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170103/a600d7e8/attachment.html>


More information about the users mailing list