[Buildbot-devel] MQ design plan and code review

[Dustin J. Mitchell]

(sorry for the self-reply)

Just to be clear, the pattern we're implementing is pub-sub, which is
*not* what OpenStack uses.  That may be causing some of the confusion
here.

  http://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern

Dustin

On Wed, Jun 18, 2014 at 8:50 AM, Dustin J. Mitchell <dustin at v.igoro.us> wrote:
> On Wed, Jun 18, 2014 at 5:55 AM, Damon Wang <damon.devops at gmail.com> wrote:
>> I have a question that all master can connect one amqp server, so masters will see all queues?
>
> Yes, well, one cluster of amqp servers.
>
>> Besides, I wanna share my new design plan, set all queues at start is a bad idea indeed, especially if we set lots of queues, I do this for setup consumers easily -- we can give consumer the certain queue, but how to set consumer when we do not set queues at start? My idea is to search the queue list first and establish one if doesn't exist, we can make the queue list a dict, and use routing_key as dict's keys, so we just get queues["builder.*.started"], and catch exception.
>
> When a (non-persistent) consumer starts, declare a queue for that
> consumer with exclusive=True and some automatically generated unique
> name, and then bind that queue to the exchange using the supplied
> routing key patterns.  When that consumer stops, delete the queue.
>
> Your suggestion would have two problems:
>  1. Since queues are never deleted, queues which no longer have an
> attached consumer would fill up and cause the amqp server to fail
>  2. Two startConsuming calls with the same routing key pattern would
> consume from the same queue, so any message in that queue would only
> go to one of the consumers -- not what we want.
>
> Dustin