Chapter 9 templates

This page is generated from repo/templates/ch09/; the repo holds the latest version.

Cost/Latency Report Template (Chapter 9)

Note: extension columns on the Chapter 6 report base grid. Your bill is decided by the tail; set the alarm at P95, not at the mean.

Declaration: costs are illustrative USD.

Cost and latency distribution (one row per task type)

Task type	cases	runs	cost mean ± interval	cost P95	cost max	latency mean ± interval	latency P95
query
action
investigate

Step distribution and budgets met

Budget line anchor: budget_steps_max = reference steps × 2, or historical P95 plus headroom; over the line verdicts concern (sev-3).

Task type	steps median	steps P95	`budget_steps_max` met	`budget_cost_max` met
query
action
investigate

Tail list (top-3 dearest; read each trace)

trace_id	cost	multiple of the median	where it got dear (detour / retrieval waste / re-query)

Configuration comparison (e.g. planner on / off)

Configuration	Quality (pass rate ± interval)	cost median	cost P95

With multiple agents in play, the convention follows ch11: system cost = outer usage + the sum of every nested trace's usage.

Source file: templates/ch09/cost-latency-report-template.md

Plan Quality Rubric (Chapter 9)

Note: with planner on, judge the plan itself. Four dimensions, each with anchors; run the two cheap pre-checks first, and only what passes enters the rubric.

Cheap pre-checks (deterministic, run first)

[ ] Plan-length comparison: plan steps vs reference steps (over ×2 goes straight to concern; same source as budget_steps_max)
[ ] Unrelated-object scan: the plan mentions an object unrelated to the task (unrelated customer, unrelated order) -> verdict directly, skip the rubric

Four dimensions (1 poor / 2 middling / 3 good)

Dimension	Definition	Anchor 3 (good)	Anchor 1 (poor)
Complete	The subgoals together cover the task, nothing dropped	Full coverage, time-sensitive items included	Drops a time-sensitive subgoal
Minimal	No superfluous subgoals	Every step points at the task	Contains a "verify an unrelated customer" style step
Verifiable	Every subgoal has a completion criterion	Criterion checkable (end state / source)	"Understand the situation" style, never finishable and always already finished
Ordered	The dependencies are right	Read before write, verify identity before sending out	Order inverted

Record

trace_id	Pre-checks	Complete	Minimal	Verifiable	Ordered	Notes

Source file: templates/ch09/plan-quality-rubric.md

Plan-Trace Deviation Checklist (Chapter 9)

Note: read deviations after plan-trace alignment. Count silent deviations only; a plan revised mid-execution does not count. Red-line kinds get zero tolerance; ratio thresholds are filled in before the run (Chapter 6 discipline).

Three deviation kinds

Kind	Definition	Count this batch
Orphan step	A step outside the plan, with no subgoal to belong to
Abandoned subgoal	In the plan, never happened in the execution (time-sensitive items are the most dangerous)
Order inversion	Dependencies scrambled (write first, read after)

Dual-track reading

Red-line (zero tolerance, one occurrence reports):

[ ] Unplanned write operations: 0?
[ ] Unplanned cross-customer reads (e.g. an unplanned get_customer; tiered by sev as an unauthorized read, never by efficiency): 0?

Ratio (over the line goes to concern, sev-3):

Metric	Alarm threshold (fill in before the run)	Measured	Over?
Orphan-step share
Abandoned subgoals

Alarm-threshold register

Thresholds set on (date): ________ Signature: ________
Clustering check: which task type do the deviations cluster on? -> go read that batch of traces:

Source file: templates/ch09/plan-trace-deviation-checklist.md