Broker topology
queuey uses no custom exchanges. Every publish goes to the default exchange with the queue name as the routing key. All of the behaviour lives in queue arguments, which means you can read the whole design off the management UI.
For a logical queue q:
| Queue | Role | Arguments |
|---|---|---|
q |
Main work queue | x-message-ttl when QueueConfig::message_ttl is set, x-max-priority when max_priority is Some |
q.dead |
Dead-letter queue | none |
q.deferred.{ttl_ms} |
Hold queue, one per distinct delay | x-message-ttl = ttl_ms, x-dead-letter-exchange = "", x-dead-letter-routing-key = q, x-expires = 2 * ttl_ms |
So a queue set with #[queues(prefix = "myapp")] and an Emails variant
produces myapp.emails, myapp.emails.dead, and hold queues like
myapp.emails.deferred.30000 as needed.
Hold queues
Section titled “Hold queues”A hold queue exists to expire. A message published into
myapp.emails.deferred.30000 sits there for 30 seconds because that is the
queue’s x-message-ttl, and when the TTL runs out the broker dead-letters it
back onto myapp.emails, because that is the queue’s
x-dead-letter-routing-key. Nothing consumes a hold queue.
x-expires = 2 * ttl_ms makes an idle hold queue delete itself, so a system
that stops retrying does not leave hundreds of empty queues behind.
Both arguments follow from the name alone. That is the reason hold-queue
arguments are not configurable: every process computes the same arguments for
the same name, so no two processes can deadlock each other with
PRECONDITION_FAILED.
Why one queue per delay, rather than per message
Section titled “Why one queue per delay, rather than per message”Per-message TTL would be the obvious alternative and it is the wrong one. RabbitMQ expires messages from the head of a queue, so a message with a 30-second TTL sitting in front of one with a 1-second TTL blocks it for the full 30 seconds. Head-of-line blocking, with the delays fighting each other.
Every message in a hold queue having the same TTL means the queue drains strictly in order, and no delay can block another. That is also why retries use this mechanism rather than a shared wait queue: exponential backoff produces a wide spread of delays, which is exactly the case per-message TTL handles worst.
The cost is queue count. Delays are rounded up to a granularity — 1 second by
default — so Retry-After: 30 and a 29.2 second delay share one queue.
retry_granularity bounds the worst case: with a five-minute cap, 1s allows
up to 300 hold queues on one work queue, 10s up to 30.
Declared on demand, every time
Section titled “Declared on demand, every time”Hold queues are not created by declare(). They are declared immediately
before every held publish, even when the queue already exists. The redeclare is
what resets x-expires, so caching the fact that a queue exists would let the
broker delete one that is still in use.
The declaration runs on its own channel, not the publishing one, so a declaration the broker rejects cannot fail the publishes in flight beside it.
Priority
Section titled “Priority”The main queue is declared with x-max-priority when max_priority is Some,
which it is by default at 10 levels. Every publish carries the AMQP priority
property and an x-deferrals header.
Normal work is published at priority 0. A deferred
job comes back at the queue’s maximum, ahead of the backlog. Hold queues never
get a priority argument, and q.dead is untouched by any of this.
Declaration flags
Section titled “Declaration flags”q, q.dead and hold queues are all declared with passive: false,
exclusive: false, auto_delete: false, nowait: false. Durability comes from
QueueConfig::durable; hold queues inherit the main queue’s durability, and
q.dead is always durable, because a dead letter you lose on a broker restart
is a dead letter you will never know about.
Requirements for delays
Section titled “Requirements for delays”The queue you defer onto, retry on, or enqueue_after on has to have been
declared through the same backend — the one Producer::new or
WorkerBuilder::build gave you. A Producer::new_undeclared cannot defer or
delay: there would be no q for the hold queue to dead-letter into. That case
returns Error::UnknownQueue.
Name length
Section titled “Name length”AMQP short strings cap at 255 bytes, and a hold queue name is the work queue
name plus a suffix plus up to ten digits of TTL. declare() pre-checks every
queue against the longest possible hold-queue name and fails at startup rather
than on the first retry.
Keep prefixes and queue names short and this never comes up. Around 230 bytes of work-queue name is the practical limit.
What you see in the management UI
Section titled “What you see in the management UI”Running the end-to-end example against a local broker, the Queues tab shows:
aq-example.emails ready: 0 unacked: 1aq-example.emails.dead ready: 0aq-example.emails.deferred.500 ready: 1 <- appears, holds, then deletes itselfaq-example.emails.deferred.1000 ready: 0Hold queues coming and going is normal and healthy. A hold queue that is not draining means nothing is consuming the work queue it dead-letters into.
Migrating an existing deployment
Section titled “Migrating an existing deployment”RabbitMQ refuses to change the arguments of a queue that already exists, so adopting priorities on a live queue needs a plan. See Upgrading to 0.2.