Choosing a Data Encoding Format

Choosing a Data Encoding Format A workflow diagram generated by Archify. 01 / Decision EX / Language-Native 03 / Textual Formats 04 / Binary Schema-Driven Scope Choose the format Defined evolution rules Record to Encode · object to bytes · Decision › Scope Record to Encode object to bytes Transient + One Language? · in-process handoff only · Decision › Scope · decide Transient + One Language? in-process handoff only decide Human-Readable? · interchange and debugging · Decision › Choose the format · decide Human-Readable? interchange and debugging decide Schema-Driven Binary? · defined evolution rules · Decision › Choose the format · decide Schema-Driven Binary? defined evolution rules decide Language-Native · Serializable / pickle · Language-Native › Scope · compat risk Language-Native Serializable / pickle compat risk JSON / XML / CSV · textual, schema optional · Textual Formats › Choose the format · JSON Schema for validation JSON / XML / CSV textual, schema optional JSON Schema for validation Protocol Buffers · field tags in the data · Binary Schema-Driven › Defined evolution rules › Choose the format · Thrift similar Protocol Buffers field tags in the data Thrift similar Avro · writer and reader schemas · Binary Schema-Driven › Defined evolution rules › Choose the format · no tag numbers Avro writer and reader schemas no tag numbers yes no yes no name-based tag-based encode once Legend Agent logic External system