[Buildbot-devel] proxy support?

[Brian Warner]

> We have a situation where our buildbot master for an Open Source project
> we use lives outside the company, but we would like to run buildbot
> slaves for two target platforms inside the company firewall.

If you don't have direct network connectivity from the buildslave to the
buildmaster, then yes, you have to use some sort of port-forwarding or
firewall. The buildslave-to-buildmaster connection is designed to easily work
around NAT boxes, but not against restrictive filtering policies.

Benoit's suggestion of using an ssh-based port forwarding is one possibility,
if you have a suitable target host that both (A) you are able to log into,
and (B) has a better chance of connecting to the buildmaster than the
original machine.

Many firewalls offer a SOCKS server, which is basically a service that runs
on the firewall/proxy box that offers to make connections to external hosts
for you. You could run the buildslave under the 'tsocks' utility to make it
use the SOCKS server instead of direct connections: that tool uses
shared-library tricks to cause the unsuspecting application's
networking-related system calls to use the SOCKS server. (actually you can
configure a specified subset of their connections to use the proxy, so
in-perimeter targets get direct connections, etc).

If your network policies allow direct connections to certain ports (say, port
80), then you could rig a port forwarder on some external host that listens
on port 80 and relays the data to the buildmaster. Or, you could tell the
buildmaster to listen on port 80 (and patch the kernel to remove that
pointless low-numbered-ports-must-be-opened-as-root misfeature).

In general, if you can get *some* connectivity to the outside work, you can
jury-rig some sort of proxy that will let the program of your choice get to
the target of your choice.

Now that I've proposed a couple of solutions, allow me to complain about the
problem for a moment :-)

> Our sysadmin (rightly) refuses to just open the port the buildbot master is
> listening on (9989),

<rant directed_against="that sysadmin's policies, not you!">

I have to disagree with that "rightly" there. I know it seems to be
fashionable in contemporary IT circles to lock down everything inbound and
outbound, and to refuse to honor requests to open ports, but what purpose
does this serve other than to inhibit their customers (i.e. the employees)
from getting their job done?

I appreciate that network-using applications are too hard to implement safely
and represent too large of an attack surface, such that prohibiting arbitrary
inbound connections represents a reasonable compromise between safety and
utility: the practice is meant to protect internal machines that are too
buggy to protect themselves, at the expense of being unable to use the
network in new and interesting ways.

And to protect certain services on external machines, some environments
require that all connections to, say, port 25 must go through an
accounting/proxy box, prohibiting any direct connection to external hosts at
this port. This allows the proxy box to function as a monitoring tool to
detect when an internal machine is spewing spam, probably because it has been
converted into a spam-zombie by a virus that arrived through some other
vector. This is a compromise between protecting external hosts and reducing
the robustness and efficiency of the network (now all your outbound mail is
dependent upon that one accounting/proxy box, even though your local machine
is perfectly capable of speaking SMTP itself).

But what benefit can there be to prohibiting outbound connections to a
non-well-known port like 9989? There aren't any public services being offered
there (except for the buildmaster you're trying to connect to), so there's
nothing external to protect. Malware that takes over a local machine that
wants to phone home will go through some other port, usually 80, to make
itself look less suspicious, and will probably just take advantage of the
same proxy settings as everything else on your system.

If the goal of the proxy/firewall is monitoring, then the sysadmin would want
the different kinds of traffic to look as different as possible. Forcing both
virus authors and application developers to make their traffic pretend to be
HTTP on port 80 just to get past these filters is only reducing the amount of
information available, and making it more expensive to inspect the packets to
figure out what they're for.

(in a related vein, requiring that applications run as root to listen on port
80 increases the capabilites of the server, which makes any bug or server
compromise much more dangerous, and/or requires code to shed root after
obtaining the port which is easy to get wrong)

It's hard for me to imagine a reason for these sorts of policies that doesn't
stem from either a desire for power/control or from a lack of analysis. If
this were my office, I'd ask the sysadmin to reconsider that policy, and if
they wouldn't, I'd ask the management to reconsider the sysadmin.

In an engineering environment, IT is there to provide safe services to the
engineers. If there is an antagonistic relationship between the two,
something is wrong.

</rant>

Whew! :)

So I'd ask your sysadmin if they would allow just your buildslave to make a
direct connection to the buildmaster, at that one port: a hole in the
firewall that is specific to the 3-tuple of (buildslave_ipaddr,
buildmaster_ipaddr, buildmaster_port). If they refuse, ask them what security
benefit is gained by that prohibition (at least make them think about the
question.. they may benefit in other ways from doing that sort of analysis).
Then rig up some sort of proxy to work around the bug.


cheers,
 -Brian