[Buildbot-devel] Metrics

[Daniel Eggert]

On Jun 17, 2008, at 8:21 PM, Charles Lepple wrote:

> On Jun 17, 2008, at 10:05 PM, Daniel Eggert wrote:
>
>> On Jun 17, 2008, at 6:53 PM, Charles Lepple wrote:
>>
>>> On Tue, Jun 17, 2008 at 6:45 PM, Daniel Eggert <deggert at apple.com>  
>>> wrote:
>>>> Can I use buildbot to track metrics (e.g. size of my binary, test  
>>>> case
>>>> execution time)?
>>>>
>>>> How would I go about doing that?
>>>
>>> I haven't checked to see how different 0.7.7 is in this department,
>>> but buildbot tracks bytes of output and step execution time without
>>> any modification.
>>>
>>> My problem with 0.7.6 was that the storage for these metrics wasn't
>>> persistent, so every time I reconfigured the master, it would lose  
>>> its
>>> execution time estimate.
>>>
>>> However, since the master.cfg is a Python script, and since you can
>>> import your own modules that extend Buildbot steps, it isn't too  
>>> hard
>>> to capture metrics as they are generated, and send them to an  
>>> external
>>> database (which could be as simple as appending to a CSV file).
>>>
>>> -- 
>>> - Charles Lepple
>>
>> I thought buildbot had its own database for metrics. Or is there  
>> just the one used for ETA estimates?
>
>
> It does keep track of metrics, but primarily to feed the ETA  
> estimate. I am pretty sure those metrics still disappear when you  
> reconfigure, although maybe only the ETA estimate is invalidated.  
> (Stefan Seefeld mentioned that the status logs store the progress  
> metrics as well - maybe there is a way to go back and extract old  
> metrics from the log objects.)
>
> There was talk of adding more persistent storage, such as http://buildbot.net/trac/ticket/24 
>  , but if you have to sample values such as binary size, you might  
> find it easier to just write the metrics to another database as part  
> of whatever code is sampling the values. (There is a "tree size"  
> step, but it is probably not fine-grained enough for binary sizes.)
>
> -- 
> Charles Lepple

Yes. Once <http://buildbot.net/trac/ticket/24> is in place, adding  
storage for arbitrary data for each build shouldn't be too difficult.

I'll try to do my own DB stuff in the meantime.

/Daniel