Timeout tuning tradeoff Choosing a Timeout — Fast Detection vs False Alarms A timeout is a guess, never a proof of failure. TOO SHORT good: fast detection failed node leaves rotation in seconds, not minutes risk: false positives cascade live node is called dead, its load hits the rest, rest looks dead too: outage verdict: only with spare capacity TOO LONG good: rarely cries wolf almost no false alarms stable membership view risk: users pay the wait requests hang till timeout, failover takes forever, SLO budget burns away verdict: safe but sluggish ADAPTIVE good: tracks real RTT Phi Accrual follows the measured delay distribution risk: still never certain harder to implement, needs live measurements, a guess with better math verdict: best default choice No correct value exists: measure first, then balance. Networks have unbounded delay, so a formula like 2d + r is fiction. Record the RTT distribution and pick the tradeoff explicitly.