Lab ch03: Error analysis, code blind, then compare
Discipline first: code before you look at the key. Do not open reference.md
until you have coded 20 traces, a borrowed taxonomy tempts you to jam traces into
ready-made slots.
Per the Chapter 3 lab steps:
python viewer/trace_viewer.py traces/pregen-60.jsonl: 60 pre-generated traces, three task families mixed. Mini is Lv.0 read-only, so execution-class requests appear here as replies and handoffs, language can still go wrong.- Blind-code 20 traces with
templates/ch03/trace-review-form.md, covering all three task families, do not cherry-pick the short ones. Fill all four fields on each; whenfirst_bad_stepis unclear, ask: "given the information available at this step, is this action reasonable?" (Coding protocol intemplates/ch03/qualitative-coding-protocol.md.) - Cluster into your failure mode atlas v1 (
templates/ch03/failure-mode-atlas-starter.md): behavioral names, all six columns filled, a question mark where the suspected component is unclear. - Compare against
labs/ch03/reference.md. A different mode name is not a disagreement; comparefirst_bad_stepcase by case, and where it differs by more than one step go reread that trace, that is symptom-step versus cause-step. - Saturation check: still producing new modes in the last few? keep coding; no more new modes, the atlas v1 is final, the direct raw material for Chapter 4.
Fully offline, no model API needed (the traces are pre-generated, which is exactly the point: error analysis reads failures that have already happened).
Files in this lab
Full directory: repo/labs/ch03/