Complete series
Making-of do Decisum
Four parts, one thread: how a 322M encoder learned to make PT-BR e-commerce support decisions — from the tokenization bug that nearly buried the first training run, through an evaluation ruler that lied, to the v10+swa ensemble running in production today.
- 1Part 1 of 4
A "System 1" Decision Model for PT-BR Customer Support — Part 1: Context, Data, and the Bug That Nearly Buried the Project
The beginning of the Decisum project: why deciding is different from conversing, the System 1 + System 2 architecture, the 447 thousand examples from three sources, the AMD ROCm infrastructure, and the tokenization bug that nearly buried the first training run.
11 min readRead this part - 2Part 2 of 4
A "System 1" Decision Model for PT-BR Customer Support — Part 2: Results, Calibration, and the Verdict on RL
The results of decisor v2 and v3: gold set accuracy, P/R/F1 of the 17 categories, calibration via temperature scaling, three RLCD rounds with a negative verdict, and the audit that revealed the evaluation ruler was wrong.
24 min readRead this part - 3Part 3 of 4
A "System 1" Decision Model for PT-BR Customer Support — Part 3: The Ruler Was Wrong (Benchmark, Gold, and the v3→v4→v5 Trilogy)
The ruler was wrong: a paired benchmark against the zero-shot backbone, blind adjudication of 175 disagreements, a gold set of 1.000 tickets, and the v3 → v4 → v5 trilogy up to the root-cause fix — re-labeling the entire train set with a strong judge.
27 min readRead this part - 4Part 4 of 4
A "System 1" Decision Model for PT-BR Customer Support — Part 4: The Judge Bake-off and the Hybrid v6
The judge bake-off against the human gold, the hybrid distillation of v6, the v7 to v12 experiments, the nine-challenger campaign, and the champion v10+swa ensemble published on Hugging Face under Apache 2.0.
40 min readRead this part