<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    Hi Pierre,<br>
    <br>
    Thanks for the advice.<br>
    <br>
    I see that the key is to go through master.data more than trying to
    go through the object instances and their parents/sibling/children.<br>
    <br>
    Neil Gilmore<br>
    grammatech.com<br>
    <br>
    <div class="moz-cite-prefix">On 2/1/2017 10:22 AM, Pierre Tardy
      wrote:<br>
    </div>
    <blockquote
cite="mid:CAJ+soVea_ugeGvB6ho58LCwjK4HBECoyPbHjyh2ndiCHH8B_UQ@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div dir="ltr" class="gmail_msg">Hi Neil,
          <div class="gmail_msg"><br class="gmail_msg">
          </div>
          <div class="gmail_msg">My goto code for examples on how to get
            stuff from the data API is here:</div>
          <div class="gmail_msg"><a moz-do-not-send="true"
href="http://github.com/buildbot/buildbot/blob/master/master/buildbot/reporters/utils.py#L109"
              class="gmail_msg" target="_blank">http://github.com/buildbot/buildbot/blob/master/master/buildbot/reporters/utils.py#L109</a><br
              class="gmail_msg">
          </div>
          <div class="gmail_msg"><br class="gmail_msg">
          </div>
          <div class="gmail_msg">I would suggest you to copy some part
            of this code </div>
          <div class="gmail_msg">
            <div class="gmail_msg"><br class="gmail_msg">
            </div>
            <div class="gmail_msg">something like this should work:</div>
            <div class="gmail_msg">
              <div class="gmail_msg"><br>
              </div>
              <div class="gmail_msg">    buildsteps = yield
                defer.gatherResults(</div>
              <div class="gmail_msg">        [master.data.get(("builds",
                build['buildid'], 'steps'))</div>
              <div class="gmail_msg">         for build in builds])</div>
              <div class="gmail_msg">    for s in buildsteps:</div>
              <div class="gmail_msg">        if step['name'] in
                interresting_step_names:</div>
              <div class="gmail_msg">            s['logs'] = yield
                master.data.get(("steps", s['stepid'], 'logs'))</div>
              <div class="gmail_msg">            for l in s['logs']:</div>
              <div class="gmail_msg">                first_100_lines =
                yield master.data.get(("logs", l['logid'], 'contents',
                resultSpec=resultspec.ResultSpec(offset=0, limit=100))</div>
              <div class="gmail_msg"><br>
              </div>
            </div>
          </div>
          <div class="gmail_msg"><br class="gmail_msg">
          </div>
        </div>
        <br class="gmail_msg">
        <div class="gmail_quote gmail_msg">
          <div dir="ltr" class="gmail_msg">Le mer. 1 févr. 2017 à 17:08,
            Neil Gilmore <<a moz-do-not-send="true"
              href="mailto:ngilmore@grammatech.com" class="gmail_msg"
              target="_blank">ngilmore@grammatech.com</a>> a écrit :<br
              class="gmail_msg">
          </div>
          <blockquote class="gmail_quote gmail_msg" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
            everyone.<br class="gmail_msg">
            <br class="gmail_msg">
            Thanks to everyone on this list, and Pierre's patches in
            particular, we<br class="gmail_msg">
            now have our masters running 0.9.3, with a couple, um,
            enhancements. In<br class="gmail_msg">
            particular, we're using Pierre's patches to enable
            reconfiguration in<br class="gmail_msg">
            the schedulers. My users thank you.<br class="gmail_msg">
            <br class="gmail_msg">
            We did see a few oddities upon restarting the masters,
            though. Quite a<br class="gmail_msg">
            few of our builders reported 2 builds running for a builder
            at the same<br class="gmail_msg">
            time. And one builder even had an intervening build between
            the two that<br class="gmail_msg">
            were building.<br class="gmail_msg">
            <br class="gmail_msg">
            And now the question.<br class="gmail_msg">
            <br class="gmail_msg">
            Back last fall, I inquired as to why our custom build step
            was failing<br class="gmail_msg">
            to produce a correct log (I haven't had a chance to follow
            up on this<br class="gmail_msg">
            particular item since then). Here's code:<br
              class="gmail_msg">
            <br class="gmail_msg">
            class MyCustomStep(ShellCommand):<br class="gmail_msg">
                  name = "errorlog"<br class="gmail_msg">
                  haltOnFailure = 1<br class="gmail_msg">
                  description = ["checking for errors"]<br
              class="gmail_msg">
                  descriptionDone = ["done checking for errors"]<br
              class="gmail_msg">
            <br class="gmail_msg">
                  OFFprogressMetrics = ('output',)<br class="gmail_msg">
                  # things to track: number of files compiled, number of
            directories<br class="gmail_msg">
                  # traversed (assuming 'make' is being used)<br
              class="gmail_msg">
            <br class="gmail_msg">
                  def createSummary(self, cmd):<br class="gmail_msg">
                     
            self.addCompleteLog('stdio_head',cmd.getText()[0:65536])<br
              class="gmail_msg">
            <br class="gmail_msg">
            Pierre's response that synchronous logs don't really work
            any more is<br class="gmail_msg">
            fine, and he offered this instead:<br class="gmail_msg">
            first_100_lines = yield self.master.data.get(("logs",
            log['logid'],<br class="gmail_msg">
            "contents"), resultSpec=resultspec.ResultSpec(offset=0,
            limit=100))<br class="gmail_msg">
            <br class="gmail_msg">
            Unfortunately, I haven't been too successful at getting a
            log id from<br class="gmail_msg">
            inside of createSummary(). Is there any easy way to do that?
            (if anyone<br class="gmail_msg">
            knows off the top of their head, it'll save me some work).<br
              class="gmail_msg">
            <br class="gmail_msg">
            My current fix was to change master.cfg have our mail stuff
            use stdio<br class="gmail_msg">
            instead of stdio_head, and have the mail code to cut down
            the log to 64K<br class="gmail_msg">
            there. This didn't quite have the intended effect.<br
              class="gmail_msg">
            <br class="gmail_msg">
            I'd sort of thought that only the logs for the failed steps
            would be<br class="gmail_msg">
            included. I was wrong. Any log in the build named stdio for
            any step<br class="gmail_msg">
            would be included. On the bright side, they're all cut down
            to 64K or<br class="gmail_msg">
            less. On the less-than-bright side, sometimes we get many
            logs attached<br class="gmail_msg">
            (to the point where some emails were getting caught by an
            email list<br class="gmail_msg">
            waiting for moderator approval, as they got too large for
            the lists' limit).<br class="gmail_msg">
            <br class="gmail_msg">
            So it looks like I'm back to spelunking the code to find a
            log id. My<br class="gmail_msg">
            other option (which is really a hack) is to know that
            stdio_head is only<br class="gmail_msg">
            generated in steps named errorlog, and only process those.<br
              class="gmail_msg">
            <br class="gmail_msg">
            Neil Gilmore<br class="gmail_msg">
            <a moz-do-not-send="true" href="http://grammatech.com"
              rel="noreferrer" class="gmail_msg" target="_blank">grammatech.com</a><br
              class="gmail_msg">
            <br class="gmail_msg">
            _______________________________________________<br
              class="gmail_msg">
            users mailing list<br class="gmail_msg">
            <a moz-do-not-send="true" href="mailto:users@buildbot.net"
              class="gmail_msg" target="_blank">users@buildbot.net</a><br
              class="gmail_msg">
            <a moz-do-not-send="true"
              href="https://lists.buildbot.net/mailman/listinfo/users"
              rel="noreferrer" class="gmail_msg" target="_blank">https://lists.buildbot.net/mailman/listinfo/users</a><br
              class="gmail_msg">
          </blockquote>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>