Real-world combination
Background Job Worker
Offload slow work onto a durable queue and a pool of workers so requests return instantly — and the work survives even if a process crashes.
✗ The problem
Slow work done inline blocks the request
Resizing an image, sending an email, generating a report — do it inline and the caller waits for all of it. Worse: if the process dies mid-way, the work is simply lost.
app.post('/upload', (req, res) => {
resizeImage(req.file); // 4s... blocks the request
sendEmail(user.email); // crash here? job is LOST
res.send('OK');
});Slow responses, request timeouts, and work that vanishes if the process crashes
mid-way — the request thread and the job's fate are the same thing.
✓ The combination
→
→
Job (Command) → durable Queue → Worker pool
Wrap the task as a Command object, push it onto a durable queue, and let a pool of workers pull and run it asynchronously. The request just enqueues.
class ResizeImageJob {
constructor(id) { this.id = id; }
run() { resize(this.id); }
}
app.post('/upload', (req, res) => {
queue.push(new ResizeImageJob(req.id));
res.status(202).send(); // returns NOW
});Request
POST /upload
Queue
durable
Worker pool
async
Failures retry, then land in a dead-letter queue;
completion can notify interested parties (Observer).
✓ See it live
Enqueue instantly — workers process on their own clock
Submitting a job returns immediately. Click worker tick to make idle workers pull the next job and process it.
Queue
empty
Workers
👷
Worker 1
idle
👷
Worker 2
idle
👷
Worker 3
idle
queue: 0 · done: 0 · retried: 0 · dead-letter: 0
Deterministic: every 3rd job submitted (⚠) is marked to fail — watch it retry once, then dead-letter.
✓ Takeaway
Fast, resilient, and horizontally scalable
- Fast responses: the request only enqueues — heavy work happens off the critical path.
- Absorbs spikes: the queue buffers bursts instead of overloading the app server.
- Durable + retryable: jobs survive crashes; failures retry, then dead-letter for inspection.
- Scales horizontally: add more workers to drain the queue faster — no code changes.
- Real uses: emails, thumbnails, exports, video transcoding — Sidekiq, Celery, BullMQ.
- Caution: make jobs idempotent (retries may re-run them) and set sane visibility timeouts so a crashed worker's job gets picked up again.
Back to all topics →