A B2B integration API hit 2M events per day. Endpoints timed out during partner spikes. Someone added RabbitMQ.

Latency dropped roughly 60%. Then messages piled up nobody watched. Retries doubled charges. On-call learned a new 3am page.

A queue is not a performance trick. It is a contract about failure, order, and time.

Wrong Reasons to Queue

“Make the endpoint faster.”
Sometimes true. Often you moved latency to a worker and lost visibility.

“We might spike later.”
Spikes need backpressure and consumer capacity — not a deeper queue.

Queue when you need to:

  • decouple producers from slow consumers
  • absorb bursts without dropping requests
  • retry work safely when failures are transient

If none of those apply, fix the hot path first.

What Breaks in Production

Poison messages — one bad payload retries forever.

Duplicate processing — at-least-once delivery without idempotency.

Prefetch too high — one slow job blocks prefetched work behind it.

No dead-letter path — failures clog the main queue or vanish.

The broker did not fail. The design around it did.

Technology Stack

Layer Choice Why
Broker RabbitMQ Durable queues, DLX, at-least-once, proven at high volume
App Laravel Queue ShouldQueue jobs, retries, failed-job table
Driver rabbitmq queue connection Native AMQP, not Redis pretending to be a queue
Idempotency Redis or DB processed_events Dedupe by event_id before side effects
Ops RabbitMQ Management + alerts Queue depth, consumer util, DLQ rate
Deploy Horizon (if Redis aux queues) or supervisor Process workers; RabbitMQ workers via queue:work rabbitmq

Do not use the queue as your database. Messages are for delivery, not source of truth.

A Job That Survives

Dispatch from the HTTP boundary:

// app/Jobs/ProcessPartnerEvent.php
final class ProcessPartnerEvent implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 5;
    public array $backoff = [10, 30, 60, 120, 300];

    public function __construct(public string $eventId, public array $payload) {}

    public function handle(PartnerEventProcessor $processor): void
    {
        if (ProcessedEvent::where('event_id', $this->eventId)->exists()) {
            return; // idempotent — safe on redelivery
        }

        $processor->run($this->payload);

        ProcessedEvent::create(['event_id' => $this->eventId]);
    }

    public function failed(Throwable $e): void
    {
        // lands in failed_jobs — route to DLQ review workflow
        report($e);
    }
}

RabbitMQ dead-letter exchange (concept):

main queue: events
  → on reject / TTL exceeded → DLX → events.dlq

Configure x-dead-letter-exchange on the main queue. Inspect DLQ daily. Poison messages belong there, not in infinite retry.

Three rules:

  • Idempotent by event_id
  • Bounded retries with backoff
  • Failed jobs visible and reviewable

Pair with Idempotency-Key at the HTTP boundary. You need both.

What to Watch

Worker CPU is the wrong metric.

Metric Source Alert when
Queue depth RabbitMQ Management Growing for >15 min
Consumer lag Oldest ready message age > SLA threshold
DLQ rate events.dlq publish rate Any sustained spike
Job fail rate Laravel failed_jobs Above baseline
Process p95 App logs per job class Jumps after deploy 
rabbitmq_queue_messages_ready{queue="events"} > 10000
rabbitmq_queue_consumer_utilisation < 0.5  # consumers idle while depth grows — bug

Alert on lag and DLQ. Not on “workers look busy.”

When It Paid Off

Moving synchronous partner calls to RabbitMQ cut end-to-end latency roughly 60% — because:

  • HTTP threads stopped waiting on slow partners
  • Workers scaled independently on AWS
  • Retries no longer blocked ingest
  • Spikes flattened instead of timing out

The win was architecture. RabbitMQ was the tool.

Key Takeaways

Queues decouple and retry. They do not fix slow code by default.

Use RabbitMQ + Laravel jobs with bounded retries and idempotent handlers.

Configure dead-letter routing. Inspect failed jobs.

Watch queue depth and message age — not CPU.

Queue for decouple, burst, and safe retry — not because the diagram looks modern.