[users at bb.net] buildbot-data issue

Greg MacDonald gmacdonald at trionworlds.com
Wed Nov 18 00:38:53 UTC 2015


I was able to have a clean run of make frontend. The culprit was something with bower and https vs git url’s:

https://github.com/Hacklone/private-bower/issues/102

This fixed it:

git config --global url."https://".insteadOf git://



From: users [mailto:users-bounces at buildbot.net] On Behalf Of Greg MacDonald
Sent: Monday, November 16, 2015 11:50 AM
To: Pierre Tardy; users at buildbot.net
Subject: Re: [users at bb.net] buildbot-data issue

Hi Pierre,

Thanks for all your patience and help. I really appreciate it. I’ll try it again from a fresh install. Good to know about sudo, I added because there was a line complaining about it in the output…

Thx,
Greg

From: Pierre Tardy [mailto:tardyp at gmail.com]
Sent: Saturday, November 14, 2015 6:54 AM
To: Greg MacDonald; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net] buildbot-data issue

Hi Greg,

One word makes me worry is the sudo word.
There is no such "sudo make frontend" part in the doc; http://docs.buildbot.net/latest/developer/www.html#hacking-the-buildbot-javascript

I really advise you to follow it thoroughly, and never work as root or with sudo to build any nodejs stuff. It just is not supported, no developer do it, and they don't fix the weird error that you will get.

Please as advised use virtualenv, and build your buildbot using virtualenv with no root rights.

Also a thing that people do bad usually is to use the node.js from the distro.. don't.. just use the one from the nodejs community (e.g chris-lea for ubuntu)

Regards,
Pierre


Le ven. 13 nov. 2015 à 03:12, Greg MacDonald <gmacdonald at trionworlds.com<mailto:gmacdonald at trionworlds.com>> a écrit :
So I’ve got a virtual box running with Ubuntu 14.04. I’m following the “Hacking the Buildbot Javascript” guide. I’ve got an egg_info failed error during “sudo make frontend”. And when I try to start up the server, I get a runtime error about how it couldn’t find the buildbot-www package. Any ideas?

Cleaning up...
  Removing temporary dir /tmp/pip_build_root...
Command python setup.py egg_info failed with error code 1 in /home/ch1build/buildbot/www/base
Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1146, in prepare_files
    req_to_install.run_egg_info()
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 326, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 715, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /home/ch1build/buildbot/www/base


2015-11-12 17:49:41-0800 [-] while starting BuildMaster
                Traceback (most recent call last):
                  File "/home/ch1build/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/inte
rnet/defer.py", line 1184, in gotResult
                    _inlineCallbacks(r, g, deferred)
                  File "/home/ch1build/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/inte
rnet/defer.py", line 1128, in _inlineCallbacks
                    result = g.send(result)
                  File "/home/ch1build/buildbot/master/buildbot/util/service.py", line 46, in reconfig
ServiceWithBuildbotConfig
                    yield svc.reconfigServiceWithBuildbotConfig(new_config)
                  File "/home/ch1build/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/inte
rnet/defer.py", line 1274, in unwindGenerator
                    return _inlineCallbacks(None, gen, Deferred())
                --- <exception caught here> ---
                  File "/home/ch1build/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/inte
rnet/defer.py", line 1128, in _inlineCallbacks
                    result = g.send(result)
                  File "/home/ch1build/buildbot/master/buildbot/www/service.py", line 73, in reconfigS
erviceWithBuildbotConfig
                    self.setupSite(new_config)
                  File "/home/ch1build/buildbot/master/buildbot/www/service.py", line 130, in setupSit
e
                    raise RuntimeError("could not find buildbot-www; is it installed?")
                exceptions.RuntimeError: could not find buildbot-www; is it installed?

2015-11-12 17:49:41-0800 [-] BuildMaster is running
2015-11-12 17:49:41-0800 [-] BuildMaster is stopped
2015-11-12 17:49:41-0800 [-] Main loop terminated.
2015-11-12 17:49:41-0800 [-] Server Shut Down.


From: users [mailto:users-bounces at buildbot.net<mailto:users-bounces at buildbot.net>] On Behalf Of Greg MacDonald
Sent: Wednesday, November 11, 2015 6:59 PM

To: Pierre Tardy; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net<mailto:users at bb.net>] buildbot-data issue

So I started trying to setup my hacking environment for the ui and I’m having an issue with the “make frontend” step because I’m on windows. Do you have any suggestions? VirtualBox it?

Love the functional features in coffee script, btw. My time with Haskell is now vindicated. ☺

-Greg


From: Pierre Tardy [mailto:tardyp at gmail.com]
Sent: Friday, November 06, 2015 12:31 PM
To: Greg MacDonald; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net<mailto:users at bb.net>] buildbot-data issue


Looks like a plan!

Le ven. 6 nov. 2015 21:29, Greg MacDonald <gmacdonald at trionworlds.com<mailto:gmacdonald at trionworlds.com>> a écrit :
Hi Pierre,

Thanks for illuminating things; I’ll definitely take your suggestions. The possible multi-master path to ensure responsiveness is just what I need. So now the plan is to take a step back and learn the webstack and coffeescript and go from there with extending the stock web page. Thx!

-Greg

From: Pierre Tardy [mailto:tardyp at gmail.com<mailto:tardyp at gmail.com>]
Sent: Friday, November 06, 2015 4:24 AM

To: Greg MacDonald; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net<mailto:users at bb.net>] buildbot-data issue

Hi Greg,
With angularJS, the server that serves the javascript files is not very loaded, and contributes slightly in the responsiveness of the UI.
The server that serves the REST api is however important for responsiveness.
If I understand the setup you want to do, you are not addressing the problem of REST api responsiveness.

However, with buildbot nine multi-master design, you can create a master which only serves the UI and the REST api, and has no slave connected, and thus less source of reactor hangs.

Multi master is implemented in b2 via the wamp message queue, and need a crossbar.io<http://crossbar.io> router to synchronise the masters
http://docs.buildbot.net/latest/manual/cfg-global.html#wamp

I would recommend to keep that in mind, and only implement multimaster if it is needed as you scale up.
It is the power of buildbot, you can start small, and know that the system is designed to scale later. You will be able to keep the same configuration, and enable the scalability features.


As for the details of the data_module, it is written so that functions like getBuilds, loadBuilds, etc are automatically generated from the spec:
https://github.com/buildbot/buildbot/blob/master/www/data_module/src/specification.constant.coffee

So you will not find it in the source code.
I would recommend if you want to hack on the UI to spend a few hours learning http://coffeescript.org/ . It helps a lot for writing needed asynchronous code.
The equivalent javascript code is much more verbose, even if in 2015 we would have chosen ES6, which is the next standard for javascript, and has all the important syntax sugar features as coffee script.

Regards,

Le jeu. 5 nov. 2015 à 22:45, Greg MacDonald <gmacdonald at trionworlds.com<mailto:gmacdonald at trionworlds.com>> a écrit :
I wanted to ensure the web UI’s responsiveness by keeping it on another machine. Our home grown solution has responsiveness problems that people have complained about (mostly from poorly written mysql queries I haven’t had time to fix). And the stock buildbot web page hangs on occasion (0.9.0b2), probably because of a bug and not contention but either way eliminating the possibility could save me some time if it happens to come up later. (We could use a team of build engineers, but it’s just me so I’m shooting for max flexibility and stability.) No hard reason though other than a little paranoid insurance. If it’s cheap enough to do I’d still like to try. I’ve already got the CORS issue sorted out for the REST API. Do you remember where those url’s are off the top of your head?

BTW, has the code for data service been minified? I can’t seem to search for the getBuilds method. I’m new to all this (everything but python) and examples are the fastest way for me to catch up. It would’ve been helpful for me if the code examples were in javascript instead of coffee script, which I don’t know about.

Thx again.

-Greg

From: Pierre Tardy [mailto:tardyp at gmail.com<mailto:tardyp at gmail.com>]
Sent: Thursday, November 05, 2015 12:59 PM

To: Greg MacDonald; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net<mailto:users at bb.net>] buildbot-data issue

Greg,

Indeed, it is not designed to be run from another machine. I think you may workaround the autoconfiguration for the ws and api urls, but then you will get CORS issues.


I would be interrested to understand the use cases you are trying to achieve by putting a UI separately from the master

Pierre

Le jeu. 5 nov. 2015 à 21:41, Greg MacDonald <gmacdonald at trionworlds.com<mailto:gmacdonald at trionworlds.com>> a écrit :
I had a typo and now I’m not getting the websocket error. But I still need to know how to point the web page at the master buildbot machine, or if it’s even possible to run them on different machines.

-Greg
From: users [mailto:users-bounces at buildbot.net<mailto:users-bounces at buildbot.net>] On Behalf Of Greg MacDonald
Sent: Thursday, November 05, 2015 12:19 PM
To: Pierre Tardy; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net<mailto:users at bb.net>] buildbot-data issue

That got me a bit further, now I’m getting this:

WebSocket connection to 'ws://localhost:63342/angular_sandbox/app/index.htmlws' failed: Error during WebSocket handshake: Unexpected response code: 404

I’m guessing this is because it doesn’t know where the buildbot master is and it’s on another machine. Where does that get specified?

-Greg

From: Greg MacDonald
Sent: Thursday, November 05, 2015 11:27 AM
To: 'Pierre Tardy'; users at buildbot.net<mailto:users at buildbot.net>
Subject: RE: [users at bb.net<mailto:users at bb.net>] buildbot-data issue

Thanks a lot! I just couldn’t see it for some reason. ☺

From: Pierre Tardy [mailto:tardyp at gmail.com]
Sent: Wednesday, November 04, 2015 11:53 PM
To: Greg MacDonald; users at buildbot.net<mailto:users at buildbot.net>
Subject: Re: [users at bb.net<mailto:users at bb.net>] buildbot-data issue

Hi greg,

as per http://docs.buildbot.net/latest/developer/www.html#dataservice
What you need to inject is the dataService. the module is called bbData (what you need to depend in you angular,module), and the service is dataService.

so you should replace this line
            controller: ['$scope', '$attrs', 'bbData', function($scope, $attrs, dataService) {
by
            controller: ['$scope', '$attrs', 'dataService', function($scope, $attrs, dataService) {
Cheers
Pierre


Le jeu. 5 nov. 2015 à 03:15, Greg MacDonald <gmacdonald at trionworlds.com<mailto:gmacdonald at trionworlds.com>> a écrit :
Hi Everyone,

I’m trying to use the buildbot-data angular service (for 0.9.0b2) for a separate web ui and I’m having problems getting started. Bower installs buildbot-data fine, but I get an error when I try to use it:

Error: [$injector:unpr] Unknown provider: bbDataProvider <- bbData


Here’s the code that tries to use it. It’s not getting past the dependency injection.

angular.module('myApp.buildStatus', ['bbData', 'bbData'])
    .directive("buildStatus", function() {
        return {
            restrict: "E",
            replace: true,
            scope: {
                name: '@',
                builder: '@',
                status: '='
            },
            templateUrl: 'directives/build_status/build_status.html',
            controller: ['$scope', '$attrs', 'bbData', function($scope, $attrs, dataService) {
                $scope.builder = $attrs.builder;
                $scope.name<http://scope.name> = $attrs.name<http://attrs.name>;
                $scope.status = 'unknown';

            }]
        }
    });


App dependency works:

angular.module('myApp', [
    'bbData',
    'ngMaterial',
    'ngRoute',
    'myApp.view1',
    'myApp.view2',
    'myApp.version',
    'myApp.buildStatus'
]).
config(['$routeProvider', '$httpProvider', function($routeProvider, $httpProvider) {
    $routeProvider.otherwise({redirectTo: '/view1'});
    $httpProvider.defaults.useXDomain = true;
    delete $httpProvider.defaults.headers.common['X-Requested-With'];
}]);


And It’s being included like this in index.html

  <script src="bower_components/buildbot-data/dist/scripts.js"></script>

I’m not sure where to go from here. Any help is greatly appreciated. Thx! ☺

-Greg

_______________________________________________
users mailing list
users at buildbot.net<mailto:users at buildbot.net>
https://lists.buildbot.net/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20151118/eb584b5e/attachment.html>


More information about the users mailing list