<div dir="ltr">Hi,<br><br>So the changes issue I was referring to doesn't seem to be fixed (I checked the latest Buildbot code) and looked back over my changes. The problem that I identified is here <a href="https://github.com/buildbot/buildbot/blob/9f1cac1d3bb61baa0b6c836cc18812a64cfa9c2b/master/buildbot/data/resultspec.py#L320">https://github.com/buildbot/buildbot/blob/9f1cac1d3bb61baa0b6c836cc18812a64cfa9c2b/master/buildbot/data/resultspec.py#L320</a> because is some cases there would be an unmatched_filter or unmatched_order. <b>So if this is indeed the case, you should see the warning in the buildbot log file:</b><br><br>"Warning: limited data api query is not backed by db because of following filters..." as defined here <a href="https://github.com/buildbot/buildbot/blob/9f1cac1d3bb61baa0b6c836cc18812a64cfa9c2b/master/buildbot/data/resultspec.py#L322">https://github.com/buildbot/buildbot/blob/9f1cac1d3bb61baa0b6c836cc18812a64cfa9c2b/master/buildbot/data/resultspec.py#L322</a><div><br></div><div>There are two reasons why this happens. One, an incomplete definition of the fieldMapping for the data/changes.py and this would be the fix:</div><div><br></div><div>+++ b/master/buildbot/data/changes.py<br>@@ -42,6 +42,19 @@ class FixerMixin:<br> return change<br> fieldMapping = {<br> 'changeid': '<a href="http://changes.id">changes.id</a>',<br>+ 'author': 'changes.author',<br>+ 'committer': 'changes.committer',<br>+ 'comments': 'changes.comments',<br>+ 'branch': 'changes.branch',<br>+ 'revision': 'changes.revision',<br>+ 'revlink': 'changes.revlink',<br>+ 'when_timestamp': 'changes.when_timestamp',<br>+ 'category': 'changes.category',<br>+ 'repository': 'changes.repository',<br>+ 'codebase': 'changes.codebase',<br>+ 'project': 'changes.project',<br>+ 'sourcestampid': 'changes.sourcestampid',<br>+ 'parent_changeids': 'changes.parent_changeids',<br> }</div><div><br></div><div>and two not having all the columns in the select statement in order to be able to check for matched/unmatched filtering or ordering as it is done here <a href="https://github.com/buildbot/buildbot/blob/9f1cac1d3bb61baa0b6c836cc18812a64cfa9c2b/master/buildbot/data/resultspec.py#L269">https://github.com/buildbot/buildbot/blob/9f1cac1d3bb61baa0b6c836cc18812a64cfa9c2b/master/buildbot/data/resultspec.py#L269</a>. My solution was to get all the fields from the changes table from the database (don't know if this is the best approach) as you can see here:</div><div><br>+++ b/master/buildbot/db/changes.py<br>@@ -249,7 +249,7 @@ class ChangesConnectorComponent(base.DBConnectorComponent):<br> def thd(conn):<br> # get the changeids from the 'changes' table<br> changes_tbl = self.db.model.changes<br>- q = sa.select([changes_tbl.c.changeid])</div><div>+ q = changes_tbl.select()<br></div><div><br></div><div>However, since only the changeid would be needed, don't sure if this would be the right approach. Also, I would suspect some missing tests because I would expect this to be a pretty common use case.</div><div><br></div><div>Regarding the profiler, I also had some problems running it for a longer time, so if you find a fix, please share it with me.</div><div><br></div><div>Cheers,</div><div>Vlad</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jan 13, 2021 at 1:33 PM Pierre Tardy <<a href="mailto:tardyp@gmail.com">tardyp@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div>I am not sure why this wouldn't work. I vaguely recall there was an issue there, but can't figure it out staring at the code.<div>As this is quite ancient, I am not sure the JS part will even build anymore :-/</div><div><div><br></div><div>You can change the default values at that line in the python<br><div><a href="https://github.com/tardyp/buildbot_profiler/blob/master/buildbot_profiler/api.py#L193" target="_blank">https://github.com/tardyp/buildbot_profiler/blob/master/buildbot_profiler/api.py#L193</a><br clear="all"><div><div>I think you will be able to force them by editing this file inside your virtualenv..</div><div><br></div><div dir="ltr"><br></div><div>Regards</div><div dir="ltr">Pierre</div></div><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le mer. 13 janv. 2021 à 11:46, Yngve N. Pettersen <<a href="mailto:yngve@vivaldi.com" target="_blank">yngve@vivaldi.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> I insist on the buildbot profiler. What I was saying before is that you<br>
> need to hit the record button before the problem appears, and put a large<br>
> enough record time to be sure to catch a spike.<br>
> Then, you will be able to zoom to the cpu spike and catch the issue<br>
> precisely.<br>
><br>
> If the spike is in the order of minutes like you said, you can configure <br>
> it<br>
> like this and get enough samples to get enough evidence to where the code<br>
> is actually spending time:<br>
><br>
> ProfilerService(frequency=500, gatherperiod=60 * 60, mode='virtual',<br>
> basepath=None, wantBuilds=100<br>
<br>
I tried configuring this with the settings dropdown in the WebGUI plugin, <br>
but AFAICT it is not working, it only gathers info for 30 seconds.<br>
<br>
I guess I must be holding it incorrectly.<br>
<br>
> This will record for one hour, and mitigate the memory used if you worry<br>
> about it.<br>
<br>
-- <br>
Sincerely,<br>
Yngve N. Pettersen<br>
Vivaldi Technologies AS<br>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Vlad</div></div>