[users at bb.net] Save build logs in a standalone report

[Pierre Tardy]

Hi all,
Usually, the way to do that is to setup a custom reporter class.
You can then use the data api to fetch as many information on the build as
you need to build your html report, and push it to S3.

The details of what you can get with rest api is documented here:
https://docs.buildbot.net/0.9.5/developer/rest.html

The best is to look at the example of reporters/utils
https://github.com/buildbot/buildbot/blob/master/master/buildbot/reporters/utils.py

If you are not confident enough with twisted and async, you can just put a
MasterShellCommand with a python script in the end of your builds and use
the REST api to fetch the data you need.
In order to get raw logs, you can use the raw log endpoint (no need for
logobservers!):

e.g: https://nine.buildbot.net/api/v2/logs/357614/raw

Note that buildbot store the stdout/stderr information, which is removed
when you fetch it via raw endpoint.

Here is an example on how to process the stream logs that we use for
debugging our integration tests:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/test/util/integration.py#L246

Regards
Pierre


Le lun. 27 mars 2017 à 18:41, Alvaro Erickson <alvaro.erickson at gmail.com> a
écrit :

> Hi Mike,
>
> I too would like to see what your solution for saving a log (into plain
> text, not html) for all steps after a build is finished. Specifically, if
> you could please share the subclassed Build class for log generation. Any
> examples would be very helpful.
>
> Background: On my work project, one of the goals is they want all the
> build logs to be archived.
>
> Regards,
>
> Alvaro
>
> On Mon, Mar 27, 2017 at 8:52 AM, Mike Alexeev <mike.42.alexeev at gmail.com>
> wrote:
>
> Hello, Koutheir
>
>
> I have also similar problem.
> I need to save log from all steps after build finished.
> No html just plain text.
>
> It seems that there is no ready solution.
> For example there is no file storage in buildbot for files (reports, logs
> and other build artifacts).
> So my solution is not universal though it can be helpful.
>
>
> For log generation I subclassed Build class. You can set your own custom
> Build class
> to build factory in buildbot config.
>
> I overrided buildFinished method and added there code to collect log from
> all steps.
> In build instance you have access to Data API and you can read logs from
> DB.
> I can share example if you want.
>
> For file serving I use nginx.
> I added links to log in build page template and substitute path to log
> based on build properties.
>
> So every build has link on its page to log.
>
> I think that saving log can also be implemented as step running on Local
> Worker
> (it runs on master).  But I used custom build class because it looks
> cleaner (no odd steps in build).
>
>
> By the way, can be there default implementation for file storage in
> buildbot?
> For example default solution with twisted file serving
> and possibility to use server like nginx in bigger setups. Just like db
> setup - sqlite has no external
>  processes and easy to setup, but  also you can use postgre which much
> more faster.
>
>
> What about adding post-build hook in configuration? Log-saving method for
> example could be
>  set there
>
>
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
>
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170327/41d04a99/attachment.html>