Lambda to kappa
From Lambda to Kappa — One Engine for History and Now
Lambda ran everything twice; kappa replays history through the stream engine.
LAMBDA — retired
batch layer: reprocess all history
speed layer: fresh events only
serving layer merges both views
two code paths, double bugs
merge logic never quite right
complexity retired it
one engine
instead
KAPPA — unified
one stream engine does both jobs
replay history through it
fresh events flow as usual
needs exactly-once semantics
window by EVENT time
Beam on Flink / Dataflow
Kappa's three conditions:
1. replay history from the same engine that eats fresh events.
2. exactly-once: broken tasks' partial output discarded, like batch.
3. event-time windows — processing time is meaningless on replay.