[Buildbot] #3183: fix network "spottiness" on new infra

[Buildbot trac]

#3183: fix network "spottiness" on new infra
---------------------+--------------------------
Reporter:  dustin    |       Owner:  verm
    Type:  defect    |      Status:  accepted
Priority:  critical  |   Milestone:  sys - other
 Version:            |  Resolution:
Keywords:            |
---------------------+--------------------------

Comment (by dustin):

 OK, so let's reason through this.  All of these hosts are showing up on
 Po33, but there are only two physical ports in Po33.  So there's a bridge
 somewhere.

 The only thing plugged into Po33 is vm1.  Does vm1 have a bridge?  Yup.

 {{{
 bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
 1500
         ether 02:fe:61:83:ab:00
         nd6 options=1<PERFORMNUD>
         id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
         maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
         member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                 ifmaxaddr 0 port 6 priority 128 path cost 20000
         member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                 ifmaxaddr 0 port 9 priority 128 path cost 2000000
         member: vlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                 ifmaxaddr 0 port 7 priority 128 path cost 2000000
 }}}

 I'll wager that putting a lagg port into a bridge allows traffic to bridge
 in one port and out the other.

 I took igb1 down on that host, just to be sure.  From what I can tell it
 was already disabled by the switch.  Even before doing so, I wasn't able
 to replicate the network error, although I've gotten various emails today
 indicating failure.

 I'm not sure how to fix this. I know that LACP is, in part, intended to
 replace STP, so it doesn't make sense to run STP across those two links.
 Maybe this is a FreeBSD bug?

--
Ticket URL: <http://trac.buildbot.net/ticket/3183#comment:36>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation