Change Data Capture — One Leader, Many Derived Followers (Figure 12-5)

Change Data Capture — One Leader, Many Derived Followers (Figure 12-5) A data-flow diagram generated by Archify. 01 / System of Record 02 / Replication Log 03 / CDC Capture 04 / Change Stream 05 / Derived Data Initial Snapshot · at known log offset · 01 / System of Record Initial Snapshot at known log offset Outbox Table · stable CDC schema · 02 / Replication Log · written in one txn Outbox Table stable CDC schema written in one txn Search Index · derived data system · 05 / Derived Data Search Index derived data system Cache · derived data system · 05 / Derived Data Cache derived data system OLTP Database · system of record · 01 / System of Record · decides write order OLTP Database system of record decides write order Replication Log · WAL / logical log · 02 / Replication Log · in commit order Replication Log WAL / logical log in commit order CDC Connector · Debezium-style · 03 / CDC Capture · change events CDC Connector Debezium-style change events Log-Based Broker · ordered by key · 04 / Change Stream · log-compacted Log-Based Broker ordered by key log-compacted Data Warehouse · derived data system · 05 / Derived Data Data Warehouse derived data system Stream Processor · derived data system · 05 / Derived Data Stream Processor derived data system commit in one order the database decides the order read the log standard event schema change events keyed by primary key index cache load feed consistent copy then apply changes from offset same transaction reads outbox schema not the internal schema Legend primary data async batch data store