Streaming vs Batch databases

To understand the difference between Batch and Streaming databases, it’s best to stop thinking about the "database" as just a storage box and start thinking about how it handles time.

The Simple Analogy: The Laundry vs. The Waterfall

• Batch Processing (The Laundry): You don't wash every sock the second you take it off. You wait until the basket is full (a "batch"), then run a cycle. You get a lot of clean clothes at once, but you have to wait for the cycle to finish to wear any of them.

• Streaming Processing (The Waterfall): Imagine standing under a waterfall with a cup. You aren't waiting for the river to stop; you are interacting with the water as it moves. You get a drink the instant the water hits your cup.

I will use 2 scenarios to explain these 2 concepts;

Scenario 1: The "Dull" Monthly Report (Batch)

The Setup: A retail chain wants to know their total profit for November.

• How it works: At the end of the month, the system gathers millions of rows from various stores, "shuts the door," and runs a massive calculation.

• Why Batch? You don't need to know November's total on November 2nd. You want 100% accuracy and the ability to join complex tables (like "Discounts" + "Returns" + "Sales") without worrying about the data moving while you calculate.

Scenario 2: The "High-Stakes" Fraud Hunter (Streaming)

The Setup: A bank needs to block a credit card if three "suspicious" transactions happen within 30 seconds.

• How it works: If you use a Batch database, you won't find out about the fraud until the "nightly run"—by then, the money is gone. In a Streaming database, the query is "always on." The moment the 3rd transaction hits the stream, the database triggers an alert.

• Advanced Concept: Temporal Windows. Streaming databases are experts at "Sliding Windows" (e.g., "Look at the last 10 minutes, updating every second").

---

Advanced View: "Push" vs. "Pull"

In an advanced architecture, the difference is where the Computation happens:

• Batch is Pull: The data sits still. You "Pull" an answer out of it by running a query.

• Streaming is Push: The query sits still. The data "Pushes" through the query, and the result is updated incrementally.

Example: Think of a Google Map.
A Batch approach would be like downloading a static map image every 5 minutes to see where you are. A Streaming approach is the blue dot moving in real-time. The map doesn't re-render the whole world; it just updates your coordinates.