[devel at bb.net] BuildbotNetUsageData feature introduction
Dustin J. Mitchell
dustin at v.igoro.us
Sun Sep 11 17:03:39 UTC 2016
I'm glad Pierre suggested this. Back in the 0.7.x era, we had a "web bug"
on the web UI that included the Buildbot version. It was useful in
deciding how much to worry about compatibility, security backporting, and
so on.
In 0.9.0, we have a lot of features that may compete for development
attention. I suspect we are carrying along a number of features with no
users, based on irc and mailing-list traffic, but that's not a great
metric. Having an estimate of total audience for those features will help
us to prioritize the development of those that are seeing active use, and
cut loose those which are seeing no use at all.
That said, there are obvious privacy issues here! If you have concerns
about the implementation, please bring them up now. Once this is 'live' it
will be a bit more difficult to change.
Dustin
Dustin
2016-09-07 5:34 GMT-04:00 Pierre Tardy <tardyp at gmail.com>:
> Hello all,
>
> It has always been very difficult for us to understand how buildbot is
> used, and how much people are upgrading to newer version
>
> There is of course http://trac.buildbot.net/wiki/SuccessStories but
> beyond that there are tons of small shops using buildbot quietly in a
> variety of environment.
>
> Having a good idea of how much it is used will also help people to embrace
> buildbot nine even more, knowing how much other people already use it
> without problems.
>
> This is why we will introduce the buildbotNetUsageData feature in buildbot
> 0.9.0rc3.
> The idea of this feature is to send usage data to our buildbot.net
> infrastructure.
> In the tradition of buildbot, this feature is very configurable, so that
> you can disable it or filter the data sent as you want.
>
> https://github.com/buildbot/buildbot/pull/2393
> We wanted to heads-up the mailing list before this lands
>
>
> Following is a copy of the documentation for your reference:
>
> Since buildbot 0.9.0, buildbot has a simple feature which sends usage
> analysis info to buildbot.net. This is very important for buildbot
> developers to understand how the community is using the tools. This allows
> to better prioritize issues, and understand what plugins are actually being
> used. This will also be a tool to decide whether to keep support for very
> old tools. For example buildbot contains support for the venerable CVS, but
> we have no information whether it actually works beyond the unit tests. We
> rely on the community to test and report issues with the old features.
>
> With BuildbotNetUsageData, we can know exactly what combination of plugins
> are working together, how much people are customizing plugins, what
> versions of the main dependencies people run.
>
> We take your privacy very seriously.
>
> BuildbotNetUsageData will never send information specific to your Code or
> Intellectual Property. No repository url, shell command values, host names,
> ip address or custom class names. If it does, then this is a bug, please
> report.
>
> We still need to track unique number for installation. This is done via
> doing a sha1 hash of master's hostname, installation path and fqdn. Using a
> secure hash means there is no way of knowing hostname, path and fqdn given
> the hash, but still there is a different hash for each master.
>
> You can see exactly what is sent in the master's twisted.log. Usage data
> is sent everytime the master is started.
>
> BuildbotNetUsageData can be configured with 4 values:
>
> -
>
> c['buildbotNetUsageData'] = None disables the feature
> -
>
> c['buildbotNetUsageData'] = 'basic' sends the basic information to
> buildbot including:
>
> - versions of buildbot, python and twisted
> - platform information (CPU, OS, distribution, python flavor (i.e
> CPython vs PyPy))
> - mq and database type (mysql or sqlite?)
> - www plugins usage
> - Plugins usages: This counts the number of time each class of
> buildbot is used in your configuration. This counts workers, builders,
> steps, schedulers, change sources. If the plugin is subclassed, then it
> will be prefixed with a >
>
> example of basic report (for the metabuildbot):
>
> {'versions': {
> 'Python': '2.7.6',
> 'Twisted': '15.5.0',
> 'Buildbot': '0.9.0rc2-176-g5fa9dbf'},'platform': {
> 'machine': 'x86_64',
> 'python_implementation': 'CPython',
> 'version': '#140-Ubuntu SMP Mon Jul',
> 'processor':
> 'x86_64',
> 'distro:': ('Ubuntu', '14.04', 'trusty')
> },'db': 'sqlite','mq': 'simple','plugins': {
> 'buildbot.schedulers.forcesched.ForceScheduler': 2,
> 'buildbot.schedulers.triggerable.Triggerable': 1,
> 'buildbot.config.BuilderConfig': 4,
> 'buildbot.schedulers.basic.AnyBranchScheduler': 2,
> 'buildbot.steps.source.git.Git': 4,
> '>>buildbot.steps.trigger.Trigger': 2,
> '>>>buildbot.worker.base.Worker': 4,
> 'buildbot.reporters.irc.IRC': 1,
> '>>>buildbot.process.buildstep.LoggingBuildStep': 2},'www_plugins': ['buildbot_travis', 'waterfall_view']}
>
> -
>
> c['buildbotNetUsageData'] = 'full' sends the basic information plus
> additional information:
>
> - configuration of each builders: how the steps are arranged together.
> for ex:
>
> {
> 'builders': [
> ['buildbot.steps.source.git.Git', '>>>buildbot.process.buildstep.LoggingBuildStep'],
> ['buildbot.steps.source.git.Git', '>>buildbot.steps.trigger.Trigger'],
> ['buildbot.steps.source.git.Git', '>>>buildbot.process.buildstep.LoggingBuildStep'],
> ['buildbot.steps.source.git.Git', '>>buildbot.steps.trigger.Trigger']]}
>
> -
>
> c['buildbotNetUsageData'] = myCustomFunction. You can also specify
> exactly what to send using a callback.
>
> The custom function will take the generated data from full report in
> the form of a dictionary, and return a customized report as a jsonable
> dictionary. You can use this to filter any information you dont want to
> disclose. You can use a custom http_proxy environment variable in order to
> not send any data while developing your callback.
>
>
>
>
> _______________________________________________
> announce mailing list
> announce at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/announce
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/devel/attachments/20160911/2cce8f44/attachment.html>
More information about the devel
mailing list