[users at bb.net] Convert buildbot 0.8 sqlite database to postgresql

[Pierre Tardy]

Le lun. 17 oct. 2016 à 21:18, Ryan Schmidt <buildbot at ryandesign.com> a
écrit :

>
> > On Oct 17, 2016, at 3:19 AM, Pierre Tardy <tardyp at gmail.com> wrote:
> >
> > Hello.
> > On buildbot eight, the database only store information of the build
> requests before the build is started.
> >
> > Switching to a new database you will only lose the buildrequests and not
> the builds which are stored in pickle files.
>
> So, if I wait until all scheduled builds have been completed and the
> builders are all idle, I could then stop the system, switch the config to
> postgresql, start the system, and not have lost any information about prior
> builds? If so, that would be fine.
>
> Correct.



> We use buildbot to build binaries of ports for MacPorts. When a new
> version of macOS is released and we set up a new buildbot worker and
> builder for it, we need to build all 20,000 ports on it.
>

Indeed, in that case, multimaster can't help.


>
> With our old buildbot setup, we would force a single build that would
> build all 20,000 ports. This build would take several weeks to complete. If
> a problem was encountered that caused the build to abort, we had to start
> over. And when it did complete, the huge log that was generated caused an
> out of memory condition in buildbot, but not before making the web
> interface unresponsive for a few minutes.
>

That makes lot of sense to me. this is a good idea.

Maybe you could batch builds per 100 packages if this makes sense.


> With our new buildbot setup, the single build that gets forced in turn
> triggers individual builds for each port on another builder. We haven't
> tried doing this for all 20,000 ports yet, but I have tried doing around
> 3,000 ports at once, and the process of triggering the individual builds
> took several minutes, during which the web interface would become
> unresponsive. Also, looking at a builder page for a builder that has
> hundreds of scheduled builds can take a long time to display.
>

How do you do that? via a trigger step?

>  Also, looking at a builder page for a builder that has hundreds of
scheduled builds can take a long time to display.
Those hundreds of buildrequests will be stored on the db indeed pg may help.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20161017/4d3524e2/attachment.html>