[Buildbot-devel] Metrics

[Charles Lepple]

On Jun 17, 2008, at 10:05 PM, Daniel Eggert wrote:

> On Jun 17, 2008, at 6:53 PM, Charles Lepple wrote:
>
>> On Tue, Jun 17, 2008 at 6:45 PM, Daniel Eggert <deggert at apple.com>  
>> wrote:
>>> Can I use buildbot to track metrics (e.g. size of my binary, test  
>>> case
>>> execution time)?
>>>
>>> How would I go about doing that?
>>
>> I haven't checked to see how different 0.7.7 is in this department,
>> but buildbot tracks bytes of output and step execution time without
>> any modification.
>>
>> My problem with 0.7.6 was that the storage for these metrics wasn't
>> persistent, so every time I reconfigured the master, it would lose  
>> its
>> execution time estimate.
>>
>> However, since the master.cfg is a Python script, and since you can
>> import your own modules that extend Buildbot steps, it isn't too hard
>> to capture metrics as they are generated, and send them to an  
>> external
>> database (which could be as simple as appending to a CSV file).
>>
>> -- 
>> - Charles Lepple
>
> I thought buildbot had its own database for metrics. Or is there  
> just the one used for ETA estimates?


It does keep track of metrics, but primarily to feed the ETA  
estimate. I am pretty sure those metrics still disappear when you  
reconfigure, although maybe only the ETA estimate is invalidated.  
(Stefan Seefeld mentioned that the status logs store the progress  
metrics as well - maybe there is a way to go back and extract old  
metrics from the log objects.)

There was talk of adding more persistent storage, such as http:// 
buildbot.net/trac/ticket/24 , but if you have to sample values such  
as binary size, you might find it easier to just write the metrics to  
another database as part of whatever code is sampling the values.  
(There is a "tree size" step, but it is probably not fine-grained  
enough for binary sizes.)

-- 
Charles Lepple