[users at bb.net] Save build logs in a standalone report

[Mike Alexeev]

Hello Pierre,

Thank you for advice.
I will use reporter class instead of Build class customization.

Alvaro, anyway here is an example of subclassing Build:
https://pastebin.com/gfq1wQLG

Maybe it will be helpful in other cases

2017-03-27 21:49 GMT+03:00 Pierre Tardy <tardyp at gmail.com>:

> Hi all,
> Usually, the way to do that is to setup a custom reporter class.
> You can then use the data api to fetch as many information on the build as
> you need to build your html report, and push it to S3.
>
> The details of what you can get with rest api is documented here:
> https://docs.buildbot.net/0.9.5/developer/rest.html
>
> The best is to look at the example of reporters/utils https://
> github.com/buildbot/buildbot/blob/master/master/buildbot/
> reporters/utils.py
>
> If you are not confident enough with twisted and async, you can just put a
> MasterShellCommand with a python script in the end of your builds and use
> the REST api to fetch the data you need.
> In order to get raw logs, you can use the raw log endpoint (no need for
> logobservers!):
>
> e.g: https://nine.buildbot.net/api/v2/logs/357614/raw
>
> Note that buildbot store the stdout/stderr information, which is removed
> when you fetch it via raw endpoint.
>
> Here is an example on how to process the stream logs that we use for
> debugging our integration tests:
> https://github.com/buildbot/buildbot/blob/master/master/
> buildbot/test/util/integration.py#L246
>
> Regards
> Pierre
>
>
> Le lun. 27 mars 2017 à 18:41, Alvaro Erickson <alvaro.erickson at gmail.com>
> a écrit :
>
>> Hi Mike,
>>
>> I too would like to see what your solution for saving a log (into plain
>> text, not html) for all steps after a build is finished. Specifically, if
>> you could please share the subclassed Build class for log generation. Any
>> examples would be very helpful.
>>
>> Background: On my work project, one of the goals is they want all the
>> build logs to be archived.
>>
>> Regards,
>>
>> Alvaro
>>
>> On Mon, Mar 27, 2017 at 8:52 AM, Mike Alexeev <mike.42.alexeev at gmail.com>
>> wrote:
>>
>> Hello, Koutheir
>>
>>
>> I have also similar problem.
>> I need to save log from all steps after build finished.
>> No html just plain text.
>>
>> It seems that there is no ready solution.
>> For example there is no file storage in buildbot for files (reports, logs
>> and other build artifacts).
>> So my solution is not universal though it can be helpful.
>>
>>
>> For log generation I subclassed Build class. You can set your own custom
>> Build class
>> to build factory in buildbot config.
>>
>> I overrided buildFinished method and added there code to collect log from
>> all steps.
>> In build instance you have access to Data API and you can read logs from
>> DB.
>> I can share example if you want.
>>
>> For file serving I use nginx.
>> I added links to log in build page template and substitute path to log
>> based on build properties.
>>
>> So every build has link on its page to log.
>>
>> I think that saving log can also be implemented as step running on Local
>> Worker
>> (it runs on master).  But I used custom build class because it looks
>> cleaner (no odd steps in build).
>>
>>
>> By the way, can be there default implementation for file storage in
>> buildbot?
>> For example default solution with twisted file serving
>> and possibility to use server like nginx in bigger setups. Just like db
>> setup - sqlite has no external
>>  processes and easy to setup, but  also you can use postgre which much
>> more faster.
>>
>>
>> What about adding post-build hook in configuration? Log-saving method for
>> example could be
>>  set there
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users at buildbot.net
>> https://lists.buildbot.net/mailman/listinfo/users
>>
>>
>> _______________________________________________
>> users mailing list
>> users at buildbot.net
>> https://lists.buildbot.net/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170328/23489793/attachment.html>