Batch Task Attempt Lifecycle — Queued, Running, Failed, Preempted, Committed

Batch Task Attempt Lifecycle — Queued, Running, Failed, Preempted, Committed A lifecycle diagram generated by Archify. Task attempt Preemption — not a fault Faults and retries Terminal outcomes Queued · waiting in the job queue · Task attempt · no resources yet 01 Queued waiting in the job queue no resources yet Assigned · executor picked on a node · Task attempt · locality preferred 02 Assigned executor picked on a node locality preferred Running · intermediate output written · Task attempt · heartbeat flowing 03 Running intermediate output written heartbeat flowing Succeeded · task output written · Task attempt · all records processed 04 Succeeded task output written all records processed Preempted · not a fault; requeued · Preemption — not a fault · no penalty Preempted not a fault; requeued no penalty Failed · node crash or lost heartbeat · Faults and retries · partial output deleted Failed node crash or lost heartbeat partial output deleted Cancelled · job cancelled by the client · Terminal outcomes · terminal Cancelled job cancelled by the client terminal Failed Permanently · retry budget spent · Terminal outcomes · or job aborted Failed Permanently retry budget spent or job aborted Committed · job output committed atomically · Terminal outcomes · all-or-nothing output Committed job output committed atomically all-or-nothing output scheduler assigns task fetch code, start process all records processed spot reclaim or priority job crash or lost heartbeat requeue on spare capacity retry: attempts left attempts exhausted all tasks done: commit job cancelled by client Legend initial state active waiting success failure / exit final state