[Buildbot-devel] New, more efficient web stuff

[Shaun McDonald]

On 5 Jan 2008, at 09:10, Nathaniel Smith wrote:
> [...]
>
> Note also BTW that for 2.3G of data you will surely will have far, far
> fewer than 2.3 billion *rows*; I guess the vast majority of that 2.3G
> is build logs, which would be stored in big blobs in the database and
> sqlite wouldn't even touch those pages when doing anything
> query-related.  The important question is something like, how many
> total builds (probably more like "thousands" than "billions") do you
> want to keep track of, and can sqlite scale to *that*?
>

With the current OpenOffice.org production buildbot, we have around  
6000 builds. I currently have a cron script removing the logs that are  
more than 25 days old. The buildmaster directory has about 66GB in it  
just now. The log files are about 15% lower than normal due to the low  
number of builds over the festive period. Would the log files be in  
database or stored separately?

We also have custom build properties, which need to be stored.  
Examples include, whether an install was requested, which languages to  
build, and custom config flags. Please make the database easily  
expandable to store these.

Shaun