[devel at bb.net] buildbot worker in c

[Pierre Tardy]

Here is my wamp proof of concept, which is very old and clunky.
https://github.com/buildbot/buildbot/pull/1526

It cannot really work now because it needs wamp routing capability of
autobahn, which was available at that time, but not available anymore.
I did try to make it work over crossbar, but it does not make much sense,
it was to unreliable to detect wether a worker was connected or not.

That poc does at least show what you have to implement to describe a new
protocol backend.

The null protocol (made after that poc) is also a very good start to
understand what api the upper layer are expecting to see.
https://github.com/buildbot/buildbot/blob/master/master/buildbot/worker/protocols/null.py

There are interop integration tests which are designed to be used for
testing protocol implementation, and worker/master compatibility:
https://github.com/buildbot/buildbot/tree/master/master/buildbot/test/integration/interop

Twisted amp looks like a good candidate to implement a new protocol. I
think it would be a good idea to make it available over websocket, rather
than using another port, this would allow to delegate authentication parts
to the http stack, and the encryption part to an https frontend.
amp has implementation for c: https://amp-protocol.net/Implementations/
Problem with amp is that it is not very much used.

Another good option for serialization and RPC would be protobuf, which is a
bit more complex, but benefits from strong experience/support from Google.
https://developers.google.com/protocol-buffers/

both amp and protobuf require separate specification of message, which is
clearly a must have for a new protocol.

I'd second Dustin on the suggestion to use go for implementation. I think
go has good options for cross compilation and static binaries.
Go has very good support for http/https client and protobuf code generation.
If you really want c, for sure that will work too, but probably a little
bit more work to make it secure/reliable.



On Wed, Jan 31, 2018 at 8:48 PM Michał Łyszczek <michal.lyszczek at bofc.pl>
wrote:

> 31 January 2018 19:56:40 CET Dustin J. Mitchell wrote:
> > I would not recommend it!  The current protocol is based on Perspective
> > Broker, which is pretty Python-related as it involves seralization of
> > Python objects.  However, Buildbot has an abstraction over its protocols,
> > and there is room to add a different, more language-agnostic protocol.
> In
> > fact, there was a good bit of work to develop one based on AMP.
> >
> > As for language, I'd recommend Rust or Golang over C for this sort of
> > thing.  Safety is going to be pretty important.
>
> I want to write in C for a reason. It's everywhere, it already has
> virtually
> all dependencies in all (posix) systems, it's very easy to deploy and it
> can
> run *everywhere*. And if python is not available, I'd say rust and goland
> won't be either.
>
> So as I understand, right now only one protocol is implemented -
> Perspective
> Broker which is only for python clients and AMP is work in progress. Is
> someone working on it? Is there some public branch with current work?
>
> --
> Best Regards
> Michal Lyszczek_______________________________________________
> devel mailing list
> devel at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/devel/attachments/20180131/784bce6a/attachment.html>