Slow consumer: drop, buffer, backpressure
Producer Faster Than Consumer — Three Options
Drop it, buffer it, or push back: pick by what loss costs you.
1 · DROP
discard messages when behind
sensor readings: one lost
dot barely matters
but event COUNTS go wrong
counting needs reliability
use when loss is OK
2 · BUFFER
queue in memory, then
spill to disk
log broker = giant ring
buffer (oldest drops)
full disk drops too
monitor the lag!
3 · BACKPRESSURE
stall the producer
Unix pipes and TCP:
small fixed buffer;
sender blocks till read
nothing lost, nothing free
lossless, but slower
Second question: crash or offline — is anything lost?
durability = disk writes + replication, paid in throughput and latency.
sampling a sensor? dropping is fine. counting money? never drop.
durable enough for the job — no more, no less.