[users at bb.net] 0.9 cleanup logs from DB

[Mikhail Alexeev]

Hello Pierre!


Thank you for reply.

Janitor service is interesting idea.  I'm going to explore your solution 
next week.


This feature is not urgent for me, because I have already done cleanup 
script with delete operations

using sqlalchemy.

Later of course it would be great to have janitor service or cleanup 
feature in cleanupdb script.



On 03/18/2017 05:37 PM, Pierre Tardy wrote:
> Hi Mike,
>
> This is for me the last big feature that is missing in buildbot nine.
> I have a wip branch here 
> https://github.com/tardyp/buildbot/commits/janitor
> which has quite ambitious design, but I didn't manage to find the 
> contiguous time required to finish it properly.
>
> The goal of that branch is to add a a janitor service which would 
> implement the logHorizon and buildHorizon features that eight already had.
> There are challenges to be handled in order to implement that properly 
> including multimaster in the picture.
>
> Maybe I was too ambitious indeed, and a smaller step would be to add a 
> cleanup feature to cleanupdb script.
> This would indeed require some more API in the db api, but nothing 
> particularly hard. The script could be easily written to not require 
> downtime.
>
> build horizon is a bit more annoying as there is the need to take care 
> of foreign keys cleanup ( steps, logs, buildset, buildrequests, 
> sourcestamps )
>
> Please let me know if you want to take a look at that feature
> Regards,
> Pierre
>
> On Fri, Mar 17, 2017 at 9:17 AM Mike Alexeev 
> <mike.42.alexeev at gmail.com <mailto:mike.42.alexeev at gmail.com>> wrote:
>
>     Hello.
>
>     I have a question about deleting old logchunks from DB in buildbot
>     0.9.
>     Logchunks table is the biggest and grows quickly even with
>     compression.
>
>
>     I haven't found methods in data api for delete.
>     So I decided to delete logs and their logchunks using sqlalchemy
>     in cron task.
>     It's not breaks anything as I can see.
>
>     Is there better way?
>     Maybe this feature should be added in cleanupdb and configured in
>     main config?
>     Maybe there should be options for delete other old records in DB
>     (builds, steps, etc)?
>     _______________________________________________
>     users mailing list
>     users at buildbot.net <mailto:users at buildbot.net>
>     https://lists.buildbot.net/mailman/listinfo/users
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170319/93e50819/attachment.html>