[Buildbot-devel] Build Properties through the DB / data API

[Pierre Tardy]

> inspired myself from buildset_properties and change_properties, both
> also do it that way. Then I thought, I *just* needed to wire my table to
> the data API, and all calls from buildbot setting / retrieving the
> properties would automatically be backed-up in the DB, hence available
> to the web interface ...

Sounds like a good start. Gaetan did suggest that we fix the datamodel, so
that value, source have their own column instead of of storing a JSON blob
of the tuple.

Turned out to be quite a naive idea ...
>
 Sometimes naive works well.

>
> Back in eight's time, build properties used to be the properties that
> are related to a specific build. I.e. all values accessible through a
> `WithProperties` call. They include change properties, buildset
> properties, as well as a dozen of other source for properties. For
> instance, the build itself sets some, and some steps also do. Some come
> from the config, other from the schedulers, even the builders can set
> some. All properties regardless from origin were gathered in the build
> object, saved as pickle on disk, and the content of that pickle was
> displayed in the web interface. Back in the time, everything seemed easy
> !
>
Actually it did not change at all in nine. The build properties are still
gathered at the beginning of the build. Code is there:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/process/build.py#L172

You can see here how the properties of a build are gathered together. Back
in the eight time the properties was stored in the status object (the one
that is pickled). In nine this is stored directly in the process.build
object (or it will, I'm not sure that this task is done yet)

Anyway, when the build are started, everything is gathered together in
memory, and I think this is fine that way.
The only thing that needs to be done is to flush this information to the db
from time to time so that the data is available to the web interface.

What I would suggest is to do that at the end of each step. probably at
_stepDone:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/process/build.py#L409

You could flush the whole set of properties to the db, or manage a cache of
what has already been flushed, and only flush the difference.


> Now I'm quite confused.
>
How this help

Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20141118/801a6f15/attachment.html>