[Buildbot-devel] GSoC: Initial thoughts on the Graphs and Data Charts Project
mss at mawhrin.net
Tue Mar 17 05:45:50 UTC 2015
On Tue, Mar 17, 2015 at 02:45:02AM +0530, Prasoon Shukla wrote:
> Disclaimer: This post has no new data - just a clarification of the
> discussion that has happened till now.
> I think that we're not all thinking on the same page and that's leading to
> confusions (as Dustin pointed out) so here's a sort of a quick
> disambiguation post.
> The first is the metrics module (I will refer to it as the
> inbuilt-metrics-module from here on out to remove any confusion) - it
> cannot be used outside of buildbot. I wrote as much in my last reply and
> Dustin said it as well. It can only be used to gather data inside
> It was my impression that I was supposed to use this module
> (inbuilt-metrics-module) to gather data for graphing. Of course, since the
> module could not be used outside buildbot, it wasn't really much use in
> gathering data except being called indirectly to just store data in the
> db. This was also something I mentioned in my last mail.
Can you please provide examples of the data that are not inside Buildbot and
that you consider to provide graphs for?
> Thinking that I had to use the inbuilt-metrics-module, I came up with the
> problem of storage and retrieval for the inbuilt-metrics-module (not for
> gathered statistics in general). So I proposed a solution for the storage
> of data for this inbuilt-metrics-module which was simply this: Use the
> same db as buildbot, create one table for each MetricEvent (as defined in
> the inbuilt-metrics-module), store data as key value pair with a
Storing metrics data is a huge thing and it's not just about storing it in a
database (be it the same as Buildbot uses or other), it's also about providing
a fast access to various aggregations of the data and this is the part that I
am the most concerned if the internal database is to be used.
> Now, I considered the problem of actually gathering (not storing) build
> data from test suite. For clarity, let us call this measurable data as
> To use existing buildbot data gathering to the fullest, I divided the set
> of all test-statistics into two kinds - time test-statistics and non-time
It would be nice if you provided examples for both kinds, then it'd be a bit
easier follow your reasoning.
> Anyway, I'll reply to Mikhail's and Dustin's questions in the my next
To make your points clearer, please try to give several specific examples:
discussing general things are a bit more difficult, and it's easier to come to
a misunderstanding (different people might understand different things under
those general names).
More information about the devel