[Buildbot-commits] [Buildbot] #2672: [nine] re-evaluate how we're doing mq

[Buildbot trac]

#2672: [nine] re-evaluate how we're doing mq
-------------------+-------------------
Reporter:  dustin  |      Owner:
    Type:  task    |     Status:  new
Priority:  major   |  Milestone:  0.9.0
 Version:          |   Keywords:  mq
-------------------+-------------------
 In the original design, mq messages about data objects had topics of the
 form type.attr.attr.attr.event, e.g., `buildrequest.1.4.9.claimed`.  This
 was convenient because a receiver knew exactly what the data inside shoud
 look like based on the first atom in the topic.

 We've recently switched things up to instead send messages with topics
 corresponding to URI paths.  So buildrequests would be sent twice, with
 topics
 {{{
   buildrequest.1.claimed
   builder.19.buildrequest.1.claimed
 }}}

 This has two problems:
  1. Substantially increased message traffic
  2. Type is no longer clear from the topic

 I think that we should re-evaluate our approach to message queueing.
 Ideally, we'd have something already built to model this on, rather than
 forging our own way.

 When I first wrote the MQ stuff, it was with the idea that we'd support a
 lot of backends, and thus shouldn't give too much flexibility -- only one
 exchange, of topic type only, and only simple and persistent queues.  Now
 it looks like we could use Kombu as the only backend (since it has all the
 backends we'd want), which means we can use the full expressivity of AMQP.

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2672>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation