Skip to content

Broker topology

queuey uses no custom exchanges. Every publish goes to the default exchange with the queue name as the routing key. All of the behaviour lives in queue arguments, which means you can read the whole design off the management UI.

For a logical queue q:

Queue Role Arguments
q Main work queue x-message-ttl when QueueConfig::message_ttl is set, x-max-priority when max_priority is Some
q.dead Dead-letter queue none
q.deferred.{ttl_ms} Hold queue, one per distinct delay x-message-ttl = ttl_ms, x-dead-letter-exchange = "", x-dead-letter-routing-key = q, x-expires = 2 * ttl_ms

So a queue set with #[queues(prefix = "myapp")] and an Emails variant produces myapp.emails, myapp.emails.dead, and hold queues like myapp.emails.deferred.30000 as needed.

A hold queue exists to expire. A message published into myapp.emails.deferred.30000 sits there for 30 seconds because that is the queue’s x-message-ttl, and when the TTL runs out the broker dead-letters it back onto myapp.emails, because that is the queue’s x-dead-letter-routing-key. Nothing consumes a hold queue.

x-expires = 2 * ttl_ms makes an idle hold queue delete itself, so a system that stops retrying does not leave hundreds of empty queues behind.

Both arguments follow from the name alone. That is the reason hold-queue arguments are not configurable: every process computes the same arguments for the same name, so no two processes can deadlock each other with PRECONDITION_FAILED.

Why one queue per delay, rather than per message

Section titled “Why one queue per delay, rather than per message”

Per-message TTL would be the obvious alternative and it is the wrong one. RabbitMQ expires messages from the head of a queue, so a message with a 30-second TTL sitting in front of one with a 1-second TTL blocks it for the full 30 seconds. Head-of-line blocking, with the delays fighting each other.

Every message in a hold queue having the same TTL means the queue drains strictly in order, and no delay can block another. That is also why retries use this mechanism rather than a shared wait queue: exponential backoff produces a wide spread of delays, which is exactly the case per-message TTL handles worst.

The cost is queue count. Delays are rounded up to a granularity — 1 second by default — so Retry-After: 30 and a 29.2 second delay share one queue. retry_granularity bounds the worst case: with a five-minute cap, 1s allows up to 300 hold queues on one work queue, 10s up to 30.

Hold queues are not created by declare(). They are declared immediately before every held publish, even when the queue already exists. The redeclare is what resets x-expires, so caching the fact that a queue exists would let the broker delete one that is still in use.

The declaration runs on its own channel, not the publishing one, so a declaration the broker rejects cannot fail the publishes in flight beside it.

The main queue is declared with x-max-priority when max_priority is Some, which it is by default at 10 levels. Every publish carries the AMQP priority property and an x-deferrals header.

Normal work is published at priority 0. A deferred job comes back at the queue’s maximum, ahead of the backlog. Hold queues never get a priority argument, and q.dead is untouched by any of this.

q, q.dead and hold queues are all declared with passive: false, exclusive: false, auto_delete: false, nowait: false. Durability comes from QueueConfig::durable; hold queues inherit the main queue’s durability, and q.dead is always durable, because a dead letter you lose on a broker restart is a dead letter you will never know about.

The queue you defer onto, retry on, or enqueue_after on has to have been declared through the same backend — the one Producer::new or WorkerBuilder::build gave you. A Producer::new_undeclared cannot defer or delay: there would be no q for the hold queue to dead-letter into. That case returns Error::UnknownQueue.

AMQP short strings cap at 255 bytes, and a hold queue name is the work queue name plus a suffix plus up to ten digits of TTL. declare() pre-checks every queue against the longest possible hold-queue name and fails at startup rather than on the first retry.

Keep prefixes and queue names short and this never comes up. Around 230 bytes of work-queue name is the practical limit.

Running the end-to-end example against a local broker, the Queues tab shows:

aq-example.emails ready: 0 unacked: 1
aq-example.emails.dead ready: 0
aq-example.emails.deferred.500 ready: 1 <- appears, holds, then deletes itself
aq-example.emails.deferred.1000 ready: 0

Hold queues coming and going is normal and healthy. A hold queue that is not draining means nothing is consuming the work queue it dead-letters into.

RabbitMQ refuses to change the arguments of a queue that already exists, so adopting priorities on a live queue needs a plan. See Upgrading to 0.2.