Figure 13-1: The Write Path and the Read Path Meet at the Derived Dataset

Figure 13-1: The Write Path and the Read Path Meet at the Derived Dataset A data-flow diagram generated by Archify. 01 / Input 02 / Write Path 03 / Derived Dataset 04 / Read Path 05 / Output Document Updates · input events · 01 / Input Document Updates input events Write Path (Eager) · transform & index · 02 / Write Path · batch / stream Write Path (Eager) transform & index batch / stream Derived Dataset · search index: where paths meet · 03 / Derived Dataset · caches / views shift this boundary Derived Dataset search index: where paths meet caches / views shift this boundary Read Path (Lazy) · query processing · 04 / Read Path · boolean AND / OR Read Path (Lazy) query processing boolean AND / OR User Response · final answer · 05 / Output User Response final answer No Index · cheap writes · 03 / Derived Dataset · expensive scans No Index cheap writes expensive scans Precompute All · every query · 03 / Derived Dataset · impossible Precompute All every query impossible apply changes eager evaluation update index write-time work keyword query read-time work ranked results lazy evaluation append as-is no index to update scan every doc grep over all docs precompute answers infinitely many lookup only cheap reads Legend primary data async batch data store