Backend60-90 min
Background Jobs with BullMQ
Create queues and workers, enqueue jobs from API routes, retry failures, and monitor background processing safely.
Node.jsBullMQRedis
Prerequisites
- A Node.js backend with slow or retryable work.
- A Redis instance.
- A separate worker process or deployment target.
1
Plan the implementation
Start by choosing the exact page, route, API, or deployment surface you want to improve. A narrow target makes the implementation measurable and easier to verify.
- Write down the current behavior and the user-facing problem it creates.
- Pick one measurable success signal such as bundle size, latency, error rate, security coverage, or UI responsiveness.
- Identify the files, routes, providers, and environment variables involved.
- Create a rollback note before changing production-sensitive configuration.
2
Set up the required tools
Install or configure only the tools needed for this implementation. Keep config close to the feature so future developers can find the moving parts quickly.
Implementation snippet
npm install bullmq ioredisChecklist
- Dependencies are added to the correct workspace package.
- Environment variables are documented in `.env.example` when needed.
- Local development still starts without production-only secrets.
- The change is small enough to review in one pull request.
3
Implement the core pattern
- Choose jobs that are slow, retryable, or not required before the HTTP response.
- Create a queue module with shared Redis connection settings.
- Enqueue jobs from API routes after validating request data.
- Run workers as separate processes.
- Add attempts, backoff, logging, and dead-letter handling for failures.
Implementation snippet
export const emailQueue = new Queue("email", { connection });
await emailQueue.add("send-welcome", { userId }, {
attempts: 3,
backoff: { type: "exponential", delay: 5000 },
});
new Worker("email", async (job) => {
await sendWelcomeEmail(job.data.userId);
}, { connection });4
Handle edge cases
Checklist
- HTTP requests return without waiting for slow work.
- Failed jobs retry with a bounded retry policy.
- Workers can be scaled independently from the API.
- Job payloads avoid secrets and unnecessary large data.
5
Verify before production
- Run the app locally and test the normal success path.
- Test one failure path, one empty state, and one slow-network or retry path.
- Run the project build and any related unit or integration tests.
- Check browser console, server logs, and network responses for hidden warnings.
- Document the final behavior, commands used, and any follow-up work.