[Buildbot-devel] GSoC: Initial thoughts on the Graphs and Data Charts Project

[Dustin J. Mitchell]

I fear I may have confused the issue. There's an existing "metrics"
framework within Buildbot, but that's meant mainly to gather
performance characteristics of Buildbot itself, similar to what New
Relic does.  This might help Buildbot users see that, for example,
their builds are taking several seconds each to schedule because they
have too many builders.

That framework is probably *not* useful for this bug, which is about
gathering information about the builds themselves -- how long the
compile step took, how many lint warnings were logged, etc.

I agree completely that the less we do in Buildbot itself, the better.
Where other tools do things better, use those tools.  Displaying
graphs is definitely out of scope -- graphite, pencil, influx, etc. do
much better than we ever will.  Even storing the data is probably out
of scope -- instead, Buildbot should send the data directly to the
external tool as it is generated.

As for what data to generate, I think that a relatively simple set of
built-in metrics will be useful: build time per builder, for example.
After that, it should be easy for users to add information about a
build.  I think the easiest way to do so is to put it in build
properties, and then configure the code sending the data to an
external service to include those properties.

Overall, I see the configuration looking something like

c['services'].append(
  InfluxDbInjector(
    influxUrl="...",
    trackBuildTimes=True,
    trackStepTimes=['compile', 'test'],
    trackProperties=['lint_warnings', 'test_skips']
  ))

Dustin

On Mon, Mar 16, 2015 at 2:41 PM, Mikhail Sobolev <mss at mawhrin.net> wrote:
> Hi Prasoon,
>
> On Mon, Mar 16, 2015 at 07:00:25PM +0530, Prasoon Shukla wrote:
>> On Mon, Mar 16, 2015 at 12:31 PM, Mikhail Sobolev <mss at mawhrin.net> wrote:
>>      Meanwhile you could present your ideas in more details.
>>
>>    Of course.
> [snip]
>
> I'm sorry for not updating the project idea promptly, however I wanted to see
> if somebody would suggest what I've been pondering for a while (maybe not that
> nice of me).
>
> If one searches for the word 'metric', one could easily find a rather huge
> number of products and projects dedicated to the topic.  There are two rather
> big parts in there:
>
> * storage of metrics
> * visualisation of metrics
>
> Each part, as I said, is big by itself.  I am afraid that if a GSoC project
> would include both of them, it will not deliver anything more or less complete
> in the allocated time.
>
> As we aim to have projects that can delivery something complete and useful, I'd
> see the scope of "Graphs and Data Charts Projecet" to include:
>
> * identify what kind of metric "entities" need to be ipmlemented beside those
>   that already exist and implement them
> * implement a way to store metrics in an external metrics storage (my
>   inclination would be InfluxDB)
> * identify what kind of metrics we'd like to produce (entities (e.g. builds),
>   parts of them (e.g. steps), etc); propose a naming scheme and list of
>   "missing" functionality for metric generation (if any)
> * implement metric generation in the agreed order (this will be agreed after
>   the list in the previous step is produced)
> * use an existing metric visualisation tool to see metrics for a test
>   installation
> * deploy the whole thing for nine.buildbot.net
>
> Prasoon, what do you think?
>
> --
> Misha
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel