<div dir="ltr"><div><div>I'm glad Pierre suggested this. Back in the 0.7.x era, we had a "web bug" on the web UI that included the Buildbot version. It was useful in deciding how much to worry about compatibility, security backporting, and so on.<br><br></div>In 0.9.0, we have a lot of features that may compete for development attention. I suspect we are carrying along a number of features with no users, based on irc and mailing-list traffic, but that's not a great metric. Having an estimate of total audience for those features will help us to prioritize the development of those that are seeing active use, and cut loose those which are seeing no use at all.<br><br></div><div>That said, there are obvious privacy issues here! If you have concerns about the implementation, please bring them up now. Once this is 'live' it will be a bit more difficult to change.<br><br></div><div>Dustin<br></div><div><br></div>Dustin<br></div><div class="gmail_extra"><br><div class="gmail_quote">2016-09-07 5:34 GMT-04:00 Pierre Tardy <span dir="ltr"><<a href="mailto:tardyp@gmail.com" target="_blank">tardyp@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello all,<div><br></div><div>It has always been very difficult for us to understand how buildbot is used, and how much people are upgrading to newer version</div><div><br></div><div>There is of course <a href="http://trac.buildbot.net/wiki/SuccessStories" target="_blank">http://trac.buildbot.<wbr>net/wiki/SuccessStories</a> but beyond that there are tons of small shops using buildbot quietly in a variety of environment.</div><div><br></div><div>Having a good idea of how much it is used will also help people to embrace buildbot nine even more, knowing how much other people already use it without problems.</div><div><br></div><div>This is why we will introduce the buildbotNetUsageData feature in buildbot 0.9.0rc3.</div><div>The idea of this feature is to send usage data to our <a href="http://buildbot.net" target="_blank">buildbot.net</a> infrastructure.</div><div>In the tradition of buildbot, this feature is very configurable, so that you can disable it or filter the data sent as you want.</div><div><br></div><div><a href="https://github.com/buildbot/buildbot/pull/2393" target="_blank">https://github.com/buildbot/<wbr>buildbot/pull/2393</a><br></div><div>We wanted to heads-up the mailing list before this lands</div><div><br></div><div><br></div><div> Following is a copy of the documentation for your reference:</div><div><br></div><div><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">Since buildbot 0.9.0, buildbot has a simple feature which sends usage analysis info to <a href="http://buildbot.net" target="_blank">buildbot.net</a>. This is very important for buildbot developers to understand how the community is using the tools. This allows to better prioritize issues, and understand what plugins are actually being used. This will also be a tool to decide whether to keep support for very old tools. For example buildbot contains support for the venerable CVS, but we have no information whether it actually works beyond the unit tests. We rely on the community to test and report issues with the old features.</p><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">With BuildbotNetUsageData, we can know exactly what combination of plugins are working together, how much people are customizing plugins, what versions of the main dependencies people run.</p><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">We take your privacy very seriously.</p><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">BuildbotNetUsageData will never send information specific to your Code or Intellectual Property. No repository url, shell command values, host names, ip address or custom class names. If it does, then this is a bug, please report.</p><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">We still need to track unique number for installation. This is done via doing a sha1 hash of master's hostname, installation path and fqdn. Using a secure hash means there is no way of knowing hostname, path and fqdn given the hash, but still there is a different hash for each master.</p><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">You can see exactly what is sent in the master's twisted.log. Usage data is sent everytime the master is started.</p><p style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)">BuildbotNetUsageData can be configured with 4 values:</p><ul style="font-family:"open sans",sans-serif;font-size:15px;line-height:21px;background-color:rgb(246,246,246)"><li><p style="margin-top:0px"><code><span>c['buildbotNetUsageData']</span><span> </span><span>=</span><span> </span><span>No<wbr>ne</span></code><span> </span>disables the feature</p></li><li><p style="margin-top:0px"><code><span>c['buildbotNetUsageData']</span><span> </span><span>=</span><span> </span><span>'<wbr>basic'</span></code><span> </span>sends the basic information to buildbot including:</p><blockquote><div><ul><li>versions of buildbot, python and twisted</li><li>platform information (CPU, OS, distribution, python flavor (i.e CPython vs PyPy))</li><li>mq and database type (mysql or sqlite?)</li><li>www plugins usage</li><li>Plugins usages: This counts the number of time each class of buildbot is used in your configuration. This counts workers, builders, steps, schedulers, change sources. If the plugin is subclassed, then it will be prefixed with a<span> </span><cite>></cite></li></ul><p>example of basic report (for the metabuildbot):</p><div><div style="border-width:1px 1px 1px 3px;border-style:solid;border-color:rgb(204,204,204);font-size:1.1em;padding:10px;background-image:initial;background-color:rgb(255,255,255);background-position:initial;background-repeat:initial"><pre style="overflow-x:auto;overflow-y:hidden;margin-top:0px;margin-bottom:0px"><span>{</span>
<span style="color:rgb(187,136,68)">'versions'</span><span style="font-weight:bold">:</span> <span>{</span>
<span style="color:rgb(187,136,68)">'Python'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'2.7.6'</span><span>,</span>
<span style="color:rgb(187,136,68)">'Twisted'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'15.5.0'</span><span>,</span>
<span style="color:rgb(187,136,68)">'Buildbot'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'0.9.0rc2-176-g5fa9dbf'</span>
<span>},</span>
<span style="color:rgb(187,136,68)">'platform'</span><span style="font-weight:bold">:</span> <span>{</span>
<span style="color:rgb(187,136,68)">'machine'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'x86_64'</span><span>,</span>
<span style="color:rgb(187,136,68)">'python_implementation'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'CPython'</span><span>,</span>
<span style="color:rgb(187,136,68)">'version'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'#140-Ubuntu SMP Mon Jul'</span><span>,</span>
<span style="color:rgb(187,136,68)">'processor'</span><span style="font-weight:bold">:</span>
<span style="color:rgb(187,136,68)">'x86_64'</span><span>,</span>
<span style="color:rgb(187,136,68)">'distro:'</span><span style="font-weight:bold">:</span> <span>(</span><span style="color:rgb(187,136,68)">'Ubuntu'</span><span>,</span> <span style="color:rgb(187,136,68)">'14.04'</span><span>,</span> <span style="color:rgb(187,136,68)">'trusty'</span><span>)</span>
<span>},</span>
<span style="color:rgb(187,136,68)">'db'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'sqlite'</span><span>,</span>
<span style="color:rgb(187,136,68)">'mq'</span><span style="font-weight:bold">:</span> <span style="color:rgb(187,136,68)">'simple'</span><span>,</span>
<span style="color:rgb(187,136,68)">'plugins'</span><span style="font-weight:bold">:</span> <span>{</span>
<span style="color:rgb(187,136,68)">'buildbot.schedulers.<wbr>forcesched.ForceScheduler'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">2</span><span>,</span>
<span style="color:rgb(187,136,68)">'buildbot.schedulers.<wbr>triggerable.Triggerable'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">1</span><span>,</span>
<span style="color:rgb(187,136,68)">'buildbot.config.<wbr>BuilderConfig'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">4</span><span>,</span>
<span style="color:rgb(187,136,68)">'buildbot.schedulers.basic.<wbr>AnyBranchScheduler'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">2</span><span>,</span>
<span style="color:rgb(187,136,68)">'buildbot.steps.source.git.<wbr>Git'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">4</span><span>,</span>
<span style="color:rgb(187,136,68)">'>>buildbot.steps.trigger.<wbr>Trigger'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">2</span><span>,</span>
<span style="color:rgb(187,136,68)">'>>>buildbot.worker.base.<wbr>Worker'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">4</span><span>,</span>
<span style="color:rgb(187,136,68)">'buildbot.reporters.irc.IRC'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">1</span><span>,</span>
<span style="color:rgb(187,136,68)">'>>>buildbot.process.<wbr>buildstep.LoggingBuildStep'</span><span style="font-weight:bold">:</span> <span style="color:rgb(0,153,153)">2</span><span>},</span>
<span style="color:rgb(187,136,68)">'www_plugins'</span><span style="font-weight:bold">:</span> <span>[</span><span style="color:rgb(187,136,68)">'buildbot_travis'</span><span>,</span> <span style="color:rgb(187,136,68)">'waterfall_view'</span><span>]</span>
<span>}</span>
</pre></div></div></div></blockquote></li><li><p style="margin-top:0px"><code><span>c['buildbotNetUsageData']</span><span> </span><span>=</span><span> </span><span>'<wbr>full'</span></code><span> </span>sends the basic information plus additional information:</p><blockquote><div><ul><li>configuration of each builders: how the steps are arranged together. for ex:</li></ul><div><div style="border-width:1px 1px 1px 3px;border-style:solid;border-color:rgb(204,204,204);font-size:1.1em;padding:10px;background-image:initial;background-color:rgb(255,255,255);background-position:initial;background-repeat:initial"><pre style="overflow-x:auto;overflow-y:hidden;margin-top:0px;margin-bottom:0px"><span>{</span>
<span style="color:rgb(187,136,68)">'builders'</span><span style="font-weight:bold">:</span> <span>[</span>
<span>[</span><span style="color:rgb(187,136,68)">'buildbot.steps.source.git.<wbr>Git'</span><span>,</span> <span style="color:rgb(187,136,68)">'>>>buildbot.process.<wbr>buildstep.LoggingBuildStep'</span><span>],</span>
<span>[</span><span style="color:rgb(187,136,68)">'buildbot.steps.source.git.<wbr>Git'</span><span>,</span> <span style="color:rgb(187,136,68)">'>>buildbot.steps.trigger.<wbr>Trigger'</span><span>],</span>
<span>[</span><span style="color:rgb(187,136,68)">'buildbot.steps.source.git.<wbr>Git'</span><span>,</span> <span style="color:rgb(187,136,68)">'>>>buildbot.process.<wbr>buildstep.LoggingBuildStep'</span><span>],</span>
<span>[</span><span style="color:rgb(187,136,68)">'buildbot.steps.source.git.<wbr>Git'</span><span>,</span> <span style="color:rgb(187,136,68)">'>>buildbot.steps.trigger.<wbr>Trigger'</span><span>]]</span>
<span>}</span>
</pre></div></div></div></blockquote></li><li><p style="margin-top:0px"><code><span>c['buildbotNetUsageData']</span><span> </span><span>=</span><span> </span><span>my<wbr>CustomFunction</span></code>. You can also specify exactly what to send using a callback.</p><blockquote><div><p>The custom function will take the generated data from full report in the form of a dictionary, and return a customized report as a jsonable dictionary. You can use this to filter any information you dont want to disclose. You can use a custom http_proxy environment variable in order to not send any data while developing your callback.</p></div></blockquote></li></ul></div><div><br></div></div>
<br>______________________________<wbr>_________________<br>
announce mailing list<br>
<a href="mailto:announce@buildbot.net">announce@buildbot.net</a><br>
<a href="https://lists.buildbot.net/mailman/listinfo/announce" rel="noreferrer" target="_blank">https://lists.buildbot.net/<wbr>mailman/listinfo/announce</a><br></blockquote></div><br></div>