[users at bb.net] Save build logs in a standalone report

[Alvaro Erickson]

Hi Mike,

Thank you for sharing your example of subclassing Build. It will be very
helpful.

Regards,

Alvaro

On Tue, Mar 28, 2017 at 3:49 AM, Mike Alexeev <mike.42.alexeev at gmail.com>
wrote:

> Hello Pierre,
>
> Thank you for advice.
> I will use reporter class instead of Build class customization.
>
> Alvaro, anyway here is an example of subclassing Build:
> https://pastebin.com/gfq1wQLG
>
> Maybe it will be helpful in other cases
>
> 2017-03-27 21:49 GMT+03:00 Pierre Tardy <tardyp at gmail.com>:
>
>> Hi all,
>> Usually, the way to do that is to setup a custom reporter class.
>> You can then use the data api to fetch as many information on the build
>> as you need to build your html report, and push it to S3.
>>
>> The details of what you can get with rest api is documented here:
>> https://docs.buildbot.net/0.9.5/developer/rest.html
>>
>> The best is to look at the example of reporters/utils https://github
>> .com/buildbot/buildbot/blob/master/master/buildbot/reporters/utils.py
>>
>> If you are not confident enough with twisted and async, you can just put
>> a MasterShellCommand with a python script in the end of your builds and use
>> the REST api to fetch the data you need.
>> In order to get raw logs, you can use the raw log endpoint (no need for
>> logobservers!):
>>
>> e.g: https://nine.buildbot.net/api/v2/logs/357614/raw
>>
>> Note that buildbot store the stdout/stderr information, which is removed
>> when you fetch it via raw endpoint.
>>
>> Here is an example on how to process the stream logs that we use for
>> debugging our integration tests:
>> https://github.com/buildbot/buildbot/blob/master/master/buil
>> dbot/test/util/integration.py#L246
>>
>> Regards
>> Pierre
>>
>>
>> Le lun. 27 mars 2017 à 18:41, Alvaro Erickson <alvaro.erickson at gmail.com>
>> a écrit :
>>
>>> Hi Mike,
>>>
>>> I too would like to see what your solution for saving a log (into plain
>>> text, not html) for all steps after a build is finished. Specifically, if
>>> you could please share the subclassed Build class for log generation. Any
>>> examples would be very helpful.
>>>
>>> Background: On my work project, one of the goals is they want all the
>>> build logs to be archived.
>>>
>>> Regards,
>>>
>>> Alvaro
>>>
>>> On Mon, Mar 27, 2017 at 8:52 AM, Mike Alexeev <mike.42.alexeev at gmail.com
>>> > wrote:
>>>
>>> Hello, Koutheir
>>>
>>>
>>> I have also similar problem.
>>> I need to save log from all steps after build finished.
>>> No html just plain text.
>>>
>>> It seems that there is no ready solution.
>>> For example there is no file storage in buildbot for files (reports,
>>> logs and other build artifacts).
>>> So my solution is not universal though it can be helpful.
>>>
>>>
>>> For log generation I subclassed Build class. You can set your own custom
>>> Build class
>>> to build factory in buildbot config.
>>>
>>> I overrided buildFinished method and added there code to collect log
>>> from all steps.
>>> In build instance you have access to Data API and you can read logs from
>>> DB.
>>> I can share example if you want.
>>>
>>> For file serving I use nginx.
>>> I added links to log in build page template and substitute path to log
>>> based on build properties.
>>>
>>> So every build has link on its page to log.
>>>
>>> I think that saving log can also be implemented as step running on Local
>>> Worker
>>> (it runs on master).  But I used custom build class because it looks
>>> cleaner (no odd steps in build).
>>>
>>>
>>> By the way, can be there default implementation for file storage in
>>> buildbot?
>>> For example default solution with twisted file serving
>>> and possibility to use server like nginx in bigger setups. Just like db
>>> setup - sqlite has no external
>>>  processes and easy to setup, but  also you can use postgre which much
>>> more faster.
>>>
>>>
>>> What about adding post-build hook in configuration? Log-saving method
>>> for example could be
>>>  set there
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users at buildbot.net
>>> https://lists.buildbot.net/mailman/listinfo/users
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users at buildbot.net
>>> https://lists.buildbot.net/mailman/listinfo/users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170328/615914e0/attachment.html>