06 — Background Jobs: Event-Driven and Schedule-Driven Work

"Just cron and queues" was my background-jobs model, and it hid the one property that defines the category. Writing it down gave it a shape: a background job is work moved out of the user's request so the request can return fast, and it is triggered either by an event (something happened) or by a schedule (a timer fired). [1] Everything else — queues, workers, results — is plumbing around those two triggers.

The framing that landed is that background jobs exist to fix a mismatch: the user wants a fast response, but the actual work (sending emails, generating reports, processing uploads, training a model) takes longer than any request should hold open [1]. So the request hands the work off and returns immediately, and a separate process does the work later. The two questions that define any background-job system are _when does the work start_ and _how does the result get back_.

The two triggers: event-driven vs schedule-driven

The roadmap distinguishes the triggers cleanly, and that distinction is the whole point [1][2]:

• Event-driven triggers fire when something happens — the UI or another job drops a message in a queue, updates a value in storage, or calls an endpoint, and the background task reacts to it [1]. This is asynchronous, message-based communication: the request "place order" enqueues a "send confirmation email" job and returns; the worker picks it up and does the work. The trigger is the event itself.
• Schedule-driven triggers fire on a timer — a local timer, an external scheduler, or a one-shot delay [2]. Batch processing (update related-products lists overnight), routine maintenance (rebuild indexes, generate daily reports), data-retention cleanup, and consistency checks all live here. The trigger is the clock.

The diagnostic question I use: does this work need to happen _because something happened_ (event-driven) or _because it's time_ (schedule-driven)? Sending a welcome email is event-driven; generating the daily revenue report is schedule-driven. Mixing them up is how I once shipped an "every time the user signs up, also run the nightly report" bug.

Returning results: fire-and-forget, mostly

The second question — how the result gets back — is where I had to adjust my intuition. A background job runs in a separate process, often a separate machine, from whatever invoked it. Ideally it is fire-and-forget: the caller does not wait, and the job's progress has no impact on the caller [3]. That means the caller cannot automatically detect when the job ends.

So results come back through side channels, not return values:

• The worker writes the result to storage (a database row, an object store) and the caller polls or is notified.
• The worker publishes a completion event on a queue, and an interested consumer reacts.
• For user-facing jobs, the worker updates a status field the UI renders ("Your video is processing…").

The pattern is: the job does not return a value to the caller; it produces a side effect the caller can observe later. That is the whole reason it can run asynchronously.

How I use this

The decision check is whether work belongs on the request path. If a request does something expensive that the user does not need to wait for — sending notifications, resizing an upload, updating a derived dataset, anything that calls a flaky third-party API — it goes into a background job. I then pick the trigger by cause (event vs schedule), and I make the result observable through storage or a status field rather than a return value. The discipline of moving slow work off the request path is, more than any single tool, what keeps response times acceptable as a system grows.

References

[1] Microsoft, "Background jobs — best practices," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/background-jobs

[2] Microsoft, "Background jobs — schedule-driven triggers," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/background-jobs#schedule-driven-triggers

[3] Microsoft, "Background jobs — returning results," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/background-jobs#returning-results

[4] Microsoft, "Background jobs — event-driven triggers," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/background-jobs#event-driven-triggers