<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Pierre,<br>
<br>
Thanks for the advice.<br>
<br>
I see that the key is to go through master.data more than trying to
go through the object instances and their parents/sibling/children.<br>
<br>
Neil Gilmore<br>
grammatech.com<br>
<br>
<div class="moz-cite-prefix">On 2/1/2017 10:22 AM, Pierre Tardy
wrote:<br>
</div>
<blockquote
cite="mid:CAJ+soVea_ugeGvB6ho58LCwjK4HBECoyPbHjyh2ndiCHH8B_UQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div dir="ltr" class="gmail_msg">Hi Neil,
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">My goto code for examples on how to get
stuff from the data API is here:</div>
<div class="gmail_msg"><a moz-do-not-send="true"
href="http://github.com/buildbot/buildbot/blob/master/master/buildbot/reporters/utils.py#L109"
class="gmail_msg" target="_blank">http://github.com/buildbot/buildbot/blob/master/master/buildbot/reporters/utils.py#L109</a><br
class="gmail_msg">
</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">I would suggest you to copy some part
of this code </div>
<div class="gmail_msg">
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">something like this should work:</div>
<div class="gmail_msg">
<div class="gmail_msg"><br>
</div>
<div class="gmail_msg"> buildsteps = yield
defer.gatherResults(</div>
<div class="gmail_msg"> [master.data.get(("builds",
build['buildid'], 'steps'))</div>
<div class="gmail_msg"> for build in builds])</div>
<div class="gmail_msg"> for s in buildsteps:</div>
<div class="gmail_msg"> if step['name'] in
interresting_step_names:</div>
<div class="gmail_msg"> s['logs'] = yield
master.data.get(("steps", s['stepid'], 'logs'))</div>
<div class="gmail_msg"> for l in s['logs']:</div>
<div class="gmail_msg"> first_100_lines =
yield master.data.get(("logs", l['logid'], 'contents',
resultSpec=resultspec.ResultSpec(offset=0, limit=100))</div>
<div class="gmail_msg"><br>
</div>
</div>
</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
</div>
<br class="gmail_msg">
<div class="gmail_quote gmail_msg">
<div dir="ltr" class="gmail_msg">Le mer. 1 févr. 2017 à 17:08,
Neil Gilmore <<a moz-do-not-send="true"
href="mailto:ngilmore@grammatech.com" class="gmail_msg"
target="_blank">ngilmore@grammatech.com</a>> a écrit :<br
class="gmail_msg">
</div>
<blockquote class="gmail_quote gmail_msg" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
everyone.<br class="gmail_msg">
<br class="gmail_msg">
Thanks to everyone on this list, and Pierre's patches in
particular, we<br class="gmail_msg">
now have our masters running 0.9.3, with a couple, um,
enhancements. In<br class="gmail_msg">
particular, we're using Pierre's patches to enable
reconfiguration in<br class="gmail_msg">
the schedulers. My users thank you.<br class="gmail_msg">
<br class="gmail_msg">
We did see a few oddities upon restarting the masters,
though. Quite a<br class="gmail_msg">
few of our builders reported 2 builds running for a builder
at the same<br class="gmail_msg">
time. And one builder even had an intervening build between
the two that<br class="gmail_msg">
were building.<br class="gmail_msg">
<br class="gmail_msg">
And now the question.<br class="gmail_msg">
<br class="gmail_msg">
Back last fall, I inquired as to why our custom build step
was failing<br class="gmail_msg">
to produce a correct log (I haven't had a chance to follow
up on this<br class="gmail_msg">
particular item since then). Here's code:<br
class="gmail_msg">
<br class="gmail_msg">
class MyCustomStep(ShellCommand):<br class="gmail_msg">
name = "errorlog"<br class="gmail_msg">
haltOnFailure = 1<br class="gmail_msg">
description = ["checking for errors"]<br
class="gmail_msg">
descriptionDone = ["done checking for errors"]<br
class="gmail_msg">
<br class="gmail_msg">
OFFprogressMetrics = ('output',)<br class="gmail_msg">
# things to track: number of files compiled, number of
directories<br class="gmail_msg">
# traversed (assuming 'make' is being used)<br
class="gmail_msg">
<br class="gmail_msg">
def createSummary(self, cmd):<br class="gmail_msg">
self.addCompleteLog('stdio_head',cmd.getText()[0:65536])<br
class="gmail_msg">
<br class="gmail_msg">
Pierre's response that synchronous logs don't really work
any more is<br class="gmail_msg">
fine, and he offered this instead:<br class="gmail_msg">
first_100_lines = yield self.master.data.get(("logs",
log['logid'],<br class="gmail_msg">
"contents"), resultSpec=resultspec.ResultSpec(offset=0,
limit=100))<br class="gmail_msg">
<br class="gmail_msg">
Unfortunately, I haven't been too successful at getting a
log id from<br class="gmail_msg">
inside of createSummary(). Is there any easy way to do that?
(if anyone<br class="gmail_msg">
knows off the top of their head, it'll save me some work).<br
class="gmail_msg">
<br class="gmail_msg">
My current fix was to change master.cfg have our mail stuff
use stdio<br class="gmail_msg">
instead of stdio_head, and have the mail code to cut down
the log to 64K<br class="gmail_msg">
there. This didn't quite have the intended effect.<br
class="gmail_msg">
<br class="gmail_msg">
I'd sort of thought that only the logs for the failed steps
would be<br class="gmail_msg">
included. I was wrong. Any log in the build named stdio for
any step<br class="gmail_msg">
would be included. On the bright side, they're all cut down
to 64K or<br class="gmail_msg">
less. On the less-than-bright side, sometimes we get many
logs attached<br class="gmail_msg">
(to the point where some emails were getting caught by an
email list<br class="gmail_msg">
waiting for moderator approval, as they got too large for
the lists' limit).<br class="gmail_msg">
<br class="gmail_msg">
So it looks like I'm back to spelunking the code to find a
log id. My<br class="gmail_msg">
other option (which is really a hack) is to know that
stdio_head is only<br class="gmail_msg">
generated in steps named errorlog, and only process those.<br
class="gmail_msg">
<br class="gmail_msg">
Neil Gilmore<br class="gmail_msg">
<a moz-do-not-send="true" href="http://grammatech.com"
rel="noreferrer" class="gmail_msg" target="_blank">grammatech.com</a><br
class="gmail_msg">
<br class="gmail_msg">
_______________________________________________<br
class="gmail_msg">
users mailing list<br class="gmail_msg">
<a moz-do-not-send="true" href="mailto:users@buildbot.net"
class="gmail_msg" target="_blank">users@buildbot.net</a><br
class="gmail_msg">
<a moz-do-not-send="true"
href="https://lists.buildbot.net/mailman/listinfo/users"
rel="noreferrer" class="gmail_msg" target="_blank">https://lists.buildbot.net/mailman/listinfo/users</a><br
class="gmail_msg">
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>