Engagement, Decomposed — Implementation

Decisions only Sarah can make

Eight questions, each one sentence, with the default this plan proceeds under if she says

nothing and what unblocks when she answers. Nothing below Phase 1 should start before these are

answered or explicitly deferred.

• Sarah answers or defers each of the eight, in one pass

Section 0: What was verified before this plan was written

The handoff warns that the room built a full round on field shapes nobody had opened, and that

the Samples packets are not the evidence base. That warning was applied to the handoff itself.

Every load-bearing count below was re-derived from entry JSON in the corpus repo, and every

code reference was read at the named line.

• Correct the M-CONCEPT line wherever it is restated: the hole is a trap class that exists in the analysis with con_id: null and no playable representation, not twelve unscored items
• Record that the existing runner scopes to 439 (Wager + Lineup) and Pass A needs 441; the two Red String items are already named in rubric.json's scope block as out of scope, and Phase 1 changes that to a reported completeness finding

Success criteria: every number this plan restates traces to an entry file or a read line,

and the one that did not is marked rather than repeated.

Phase 0: The decision list and the plan document

Dependencies: none. Blocks: Phases 3, 4, 6.

• Write docs/reviews/2026-09-20-engagement-decomposition.html with a Markdown source beside it
• Open its first paragraph with the scope result: a copy-evaluation exercise produced a serving-order finding, a 213-item authoring backfill and a 379-passage theming job
• Put the eight-decision list at the front, each question in one sentence with its default and what it unblocks — the Experience Curator's structural requirement
• Record the dimension table in the Game Feel Expert's four-part frame: name, owner, unit, fields read, both failure directions, a corpus line anchoring each direction, and a ships / flag-only / held marker
• Mark as ships: defensibility (gates everything under it), connection to theme, the open-question test, earned wrongness, trap recognition
• Mark as held, each with its named unblock condition: contested-ness (its own gate result), resonance of voice (the register classification), heckle fairness (the register test), novelty and the Brief hook (the target student), fairness (Pass A returning a failing anchor), con recurrence (dead for run one; unblocks when the 213 backfill lands)
• Record the cuts with their reasons: pressure-pull load, stop-ability, verb distinctness, word-count reading tax, duplicated-option-text, case-level consistency, Brief body — each killed by the expert who proposed it, on the inert-criterion law or on corpus evidence
• Record the narrowed inert-criterion law verbatim: a ceiling always gets one promoted line, and whether a template or a validator forced the uniformity decides only how the line is worded. A ceiling is not automatically a defect.

Success criteria: Sarah can accept or overturn all eight decisions in one pass without

reading anything else, and every cut dimension has a reason attached that outlives the meeting.

Phase 1: Pass A — defensibility, against the corpus as it stands

Dependencies: Section 0. Blocks: fairness, the synthetic anchor, Phase 6's exemplars.

Pass A runs now. It does not wait for the backfill.

• Extend the runner's scope from 439 to 441 so the two Red String items are read rather than filtered, and report them separately
• Implement the decision rule verbatim: reading only what the student sees before tapping — stem, passage or account where one exists, and the four option texts — exactly one option is forced; it is a hit when the keyed option is not forced by that copy, or when a second option is equally forced by it
• Emit per hit: file, option key, which of the two verdicts fired, the specific sentence of copy that would have had to exist for the key to be forced, and the minimal repair
• Pass the 44 math passage-less items through on stem and options alone — a math item standing on its stem is the normal shape of a math item, so neither a hit nor an exclusion
• Report the 2 Red String writing items as a completeness finding, two lines with their ids, not a defensibility hit
• Write the "reading item with a missing passage" heading into the spec anyway, since the next season can produce one
• Commit Pass A's output to a file in the corpus repo carrying a run date and the corpus commit hash, diffable run over run
• If Pass A returns near-empty, return the near-misses ranked by how thin the margin is
• Leave l-math-translate-1 and l-reading-herring-1 unrepaired through run one as real fail anchors for earned wrongness, with an explicit worklist line saying they are known and deliberately unfixed

Success criteria: a committed, diffable hit list where every hit names the missing sentence

rather than asserting a verdict, and the two deliberately unfixed items are visible as such.

Phase 2: The three other queue-head jobs

Dependencies: Section 0. Blocks: Phase 3's prompt. Nothing here blocks anything else here.

The con → trap fan-out join

• Emit one row per con_id/trap_code pair with an item count
• Emit a per-con rollup carrying distinct codes spanned and total items
• Flag one-to-one pairs; sort by pair count so the five-appearance floor reads off the top
• Group by con where a con is one-to-one with a code; group by the pair and apply the floor to the pair where it spans several
• Build this before any con-grouped dimension is specified — a con spanning several codes answers "same move" yes while teaching two different moves behind one face

The gloss render check

• One look at where gloss renders in the app
• If gloss is authoring-only metadata no student reads, pull trap recognition rather than reframing it. Scoring internal labels as if they taught somebody is the failure mode the whole meeting avoided.
• Fold in the export-side read of the 45 Wager items: what _vigilance writes at export-machine.py:918 and :965 is not what the Wager view renders
• Give the errand a named repo, a named owner and all three questions together — it is one blocker wearing three rows' clothes

The pre-registered register classification

• Run all 150 beat-the-shakedown lines, filename stripped, eleven labels offered, scored for exact match plus a confusion matrix
• Apply the binding merge rule, fixed before the run: any register pair Jev swaps in both directions more than a quarter of the time gets merged
• Ship resonance with whatever band set survives merging
• If fewer than three bands survive, take resonance out of the prompt entirely rather than shipping it as a two-way dry-or-not flag

Success criteria: the fan-out exists as a file, the render check has an owner and an answer,

and the register run's merge rule was written down before any result was seen.

Phase 3: The coverage grid and the prompt

Dependencies: Phases 1 and 2. Blocks: Phase 4.

• Build the family-by-dimension grid as a committed file beside Pass A's output — Wager, Lineup, Prime Suspect and flavor as columns, dimensions as rows, empty cells visible
• Fill the Wager column as settled: defensibility on 258, connection to theme on 258, the open-question test on 258 (open half only — the stakes clause is suspended, a Wager item has no case for anything to turn on), contested-ness on 258 held at the gate, earned wrongness and fairness on 45, con recurrence out
• Send the grid to Sarah before the prompt is written, because it shows the majority family carrying three live dimensions and a fourth held on its owner's own failure prediction
• Draft the scoring prompt after the four queue-head jobs land and before the gate, with their anchors folded in — the queue as handed over has a cycle, since the gate tests a prompt nobody was assigned to write
• Mark any dimension whose anchors are still owed as held in the draft rather than giving it an invented anchor
• Name the prompt's author

The evaluability gate — one bar, no exemptions

• Forty units, scored twice in independent runs, option order shuffled and file id stripped
• A yes-or-no dimension passes at 90% self-agreement
• A banded dimension passes when 90% land within one band and nothing lands two bands apart
• Between that and 70% → flag-only: surfaces as a second list under the ranked worklist carrying its agreement number, holds no rank position, does not feed the per-dimension open count, is re-gated every run, and comes out of the prompt after two consecutive flag-only runs
• Under 70% → out of the prompt
• Clear the token scan through both the ground-truth agreement check and the repeat test: agreement with a scan is an average over 157 passages and is compatible with Jev flipping individual passages between runs, and the worklist is per-item
• Watch contested-ness specifically. Its owner predicts it fails, worst on the 133 Lineup items carrying no trap label. Report that outcome explicitly rather than letting it disappear into the flag-only list — if it fails, nothing per-item asks whether the wrong answers were ever live, which is close to the question Sarah asked.

Success criteria: every dimension in the prompt passed the same bar, and the grid reached

Sarah before the prompt existed.

Phase 4: The worklist

Dependencies: Phase 3.

• Build the worklist as an HTML document in docs/reviews/ with a dated filename
• Specify its shape before generating it. It currently carries six structures: rank, the flag-only list, promoted lines with expandable members, the per-dimension open count, the known-and-unfixed line, and carried findings outside the rubric. Name the format, what sits on the first screen, and a length ceiling.
• Implement the promotion rule as a display rule over a complete per-unit result set — Jev still scores every unit individually; the worklist collapses a dimension firing across most of a family into one line with the count attached and members expandable underneath
• Verify run-over-run movement works without recomputation: when forty passages get themed, the promoted line's count drops from 149 to 109
• Give carried findings their own section. The massed-practice finding sits at rank one and the rubric can never close it; a worklist whose first entry never moves teaches the reader to start at entry two.
• Add a good-news section with counts attached: 71 Prime Suspect closing lines that all generalize, nine Briefs whose frame carries real exhibit lists, 92 of 94 breaks landing mid-chain
• State what a clean run looks like. As designed the instrument can only ever get shorter, and nobody could say what Jev returns when nothing is wrong.
• Define what a retired dimension leaves behind: connection to theme retires when its count reaches zero, leaving a dated verified-clean line carrying its final count, re-checked whenever its family's content changes. Clean does not mean silent.
• Offer the score as a number per family per dimension, with the plain statement that one figure per corpus is available only by inventing weights nobody in the room can justify. This is Sarah's call to overturn, not the room's ruling.

Success criteria: the first screen is readable without scrolling, rank one moves between

runs, and a dimension at zero leaves a dated line rather than silence.

Phase 5: Export-side code changes

Dependencies: none on the rubric. Blocks: nothing here.

The case authoring lock

• Add _check_case_authoring_locks to tools/export-machine.py beside _check_season_writing_rules at line 2035, following the same pattern
• Collect every failure rather than the first, reporting under the existing BLOCKED — the season's authoring locks path at line 2258
• Clause one, a blocking verdict: every Prime Suspect case Brief states what the incident left behind as a measurable list of exhibits; a Brief omitting it blocks the export. Mechanically checkable.
• Clause two, a blocking report rather than a verdict: a trap_code appearing in threads across more than one case resolves the same way in each. Export can compare codes but not resolutions, so the run prints every cross-case trap_code with its thread ids, and case ten cannot export until an author signs the list off.
• Document clause two as weaker than clause one, in the code, where the next reader is

The interleave

• Implement inside _vigilance at line 885, not at export-machine.py:2311
• Record why in the code: 2311 is the shared sort feeding a dozen projections, and interleaving there would silently reorder rallies, rotations, strategy, knowledge-rules and undercover — a knowledge re-grain comment two branches down says it "invalidates every spaced return computed before it"
• Interleave lineup and wager separately, since they are distinct lists in one file and a con's spacing only means anything within the bank a student is actually playing
• Interleave by con_id

The standing re-check rule

• Write it beside the authoring lock, with two clauses
• (a) A re-themed passage re-opens its defensibility verdict, because a Wager passage is the copy the key is forced by and re-theming is a rewrite of load-bearing text
• (b) A re-themed passage re-opens its difficulty assignment and its difficulty_basis note, because difficulty is a property of the reading and those notes were written against August copy that would no longer exist
• Any re-themed passage goes back through Pass A before it exports

Success criteria: a Brief with no exhibit list blocks the export; the interleave changes the

Wager and Lineup lists and provably nothing else.

Phase 6: Content work — larger than the rubric that found it

Dependencies: decisions 1 and 5.

• Author the themed exemplars before commissioning anything. One Lineup passage and one Wager passage, themed, stems and keys intact.
• Run both through Pass A and the difficulty re-check, so the true per-item cost is measured on two rather than estimated across 379
• Author two versions of each, one per target-student reading — this removes the target-student blocker from the largest content job without anyone inventing a student
• Schedule the 213-item Wager backfill as content authoring, not rubric work: the whole authored layer — mechanism, error_mode, drill_rx, option-level trap_codes, plus item-level skill_code, difficulty, question_type — all of which cover the identical set of 45 items today
• Record the unblock condition: transfer becomes measurable at all only after the backfill lands, and con recurrence returns the moment it does
• Treat theming and reveal-layer authoring as one pass per item where they overlap, with reveal layer first — a beautifully themed item with no explanation behind it produces a student who enjoys missing the same con eighteen times, and the reveal layer is additive while theming rewrites the copy the key is forced by

Success criteria: two themed exemplars exist and have been through Pass A, so the 379-item

estimate rests on measurement.

Phase 7: The two tester sessions

Dependencies: Phase 1 for Session A; Run One for Session B.

Session A — before the backfill, against the unrepaired corpus

• One purpose: specify what a reveal has to do to get read, so the output can be handed to whoever authors the 213
• Observation sheet, four items per miss: seconds between the reveal appearing and the next tap; whether attention returns to the stem or goes straight to the button; whether the same con re-encountered later is answered faster or missed the same way
• One closing question with nothing in front of them: name the trick that got you, in your own words — the only item on the sheet separating a reveal that got read from one that taught
• Draw at the student's own level with no seeded above-level items. Seeding was cut twice: a student pitched above their level meets 25 savage and 11 menace heckle lines on every miss, which breaks the anti-dark-pattern law by research method, and an above-level miss has no trick to name so the closing question has no answer there.
• Record the cost: three or four usable misses instead of a dozen, so Session A needs two students or a longer sitting, and its output is a sketch rather than a spec
• Draw from The Lineup, which has 113 of 181 items carrying difficulty with a basis and reveals on 94 con rounds. On the Wager, difficulty sits on the same 45 items that carry the reveal payload, so a level-matched draw can only pull items that already have reveals.
• Mark the transfer limit on the sheet: Session A's output is explicitly Lineup-only as specification, because a break_idx reveal is positional and a drill_rx reveal is propositional
• Record the one thing that does transfer, and is the reason to run it: if a student shown exactly where their reasoning turned still cannot name the trick, no mechanism sentence will do it either, and the backfill needs something stronger than a paragraph
• Add the fair-or-cheap prompt before the reveal, not after. One word at the moment of the miss. Placement is load-bearing: a good explanation makes a trap feel fair in retrospect, so asking afterwards measures the reveal and reports it as the item's fairness.
• Treat any "cheap" verdict as the failing anchor fairness has been blocked on since round six
• Record that "fair" every time is also worth knowing — it would mean the dimension has nothing to find and should come out rather than ship uncalibrated
• Suppress heckles entirely for Session A. With three or four misses, logging which register fired labels the confound rather than removing it.
• Move heckle fairness ground truth to Session B, which runs across a full sitting
• Write the synthetic fairness fail anchor regardless, behind the class the exporter explicitly refuses, with the exporter guard written in the same change
• Set its trigger: Session A returning fewer than one "cheap" verdict per family, not zero overall — a single Lineup anchor does not calibrate the 45 Wager items

Session B — after Run One

• Write Jev's predictions down first; a rank is not a prediction
• State explicitly what counts as confirming or disconfirming a rank position
• State plainly that one session against eleven dimensions can falsify a few loud claims and cannot validate the rubric
• Instrument both sessions to record what the app served and in what order

Success criteria: Session A yields a fair-or-cheap verdict per family or triggers the

synthetic anchor, and neither session's sheet can be read as validating the rubric.

Phase 8: Findings that belong to someone other than Jev

Dependencies: none.

• Record the ordering finding as one finding with all three counts, at the top of the worklist's carried-findings section: concept on 2 of 441; difficulty on 160 of 441; the bank sorted by number at export-machine.py:2311 shipping in con-blocks with 128 adjacent same-con appearances
• State what they mean together: the complete absence of any axis an adaptive server could order on. You cannot space by concept, you cannot ramp by difficulty, and the default order is the worst one available.
• Record why it matters beyond ordering: massed practice reliably produces good in-session performance, poor retention, and a feeling of fluency, so a tester session under corpus order could come back looking good and still be the worst version of the game
• Note both repairs with their evidence bases visible: the export-side interleave (Python, evidence in this repo) and app-side interleaved serving by con_id (Swift, evidence nobody here can see)
• Name an owner and a review trigger for anti-dark-pattern enforcement, stated plainly as out of Jev's scope. Pressure-pull load, stop-ability and verb distinctness were all cut and heckle fairness is held, so nothing currently checks whether Peregrine drifts toward coercion — and the guide changed on 12 September to allow score and points in UI copy, which is the direction that drift comes from.
• Set the trigger: any change to player-facing UI copy or to the guide's register rules
• Record that three separate cuts leave this looking handled when it is not
• Record endurance as explicitly nowhere until Sarah names a surface, with her reasoning on the record: immersive passages teach the reading moves and will not train the sustained- attention demand of a real section. A student can do well across twenty interleaved rounds and stall forty minutes into a real Reading section, and both tester sessions as designed would report the first and never see the second.
• Record the rubric's standing limit: it measures the bank and never the playthrough. Whether a student learns a con depends on the spacing of its appearances, which lives in selection logic this room cut from Jev correctly. No amount of corpus reading closes this.
• Record the two coverage holes: M-CONCEPT, corrected per Section 0 — a trap class with con_id: null and no playable representation, its twelve instances counted in the calibration corpus rather than in the bank — and the 133 Lineup items carrying no trap label, where contested-ness has nothing to check itself against

Success criteria: each finding has an owner who is not Jev, and the ordering finding reads

as one finding rather than three counts.

Section 9: How this reconciles with the Jev evaluation-loop plan

The other plan is built through Phase 2 and its machinery is what this one runs on. Four points

of contact, and each needs a deliberate resolution rather than a merge.

• engagement in rubric.json is superseded. Its three levels were written as a placeholder pending this meeting, and its note says so. Replace it with the dimensions Phase 0 marks as shipping, and keep its note as a record of why the single dimension failed.
• Finding F5 is the meeting's own evidence. The existing rubric's engagement scored reading 1.91, writing 1.36, math 0.64 — it split by whether an item has a passage rather than by quality. Carry that number into the Phase 0 document as corroboration for the decomposition, not as a separate finding.
• The evaluability gate replaces the old plan's Phase 3 calibration for these dimensions. Forty units scored twice at a 90% self-agreement bar is a stricter and better-specified instrument than the forty hand labels, and it needs no ground truth. The hand-label calibration still applies to skill_code and difficulty, which do have corpus values.
• The old plan's Phase 4 write path stays gated. Nothing here changes the rule that no verdict is written onto an entry until thresholds exist.
• Record in both plans that they are siblings, so neither reads as the whole picture

Constraints that must not be quietly relaxed

• Do not build a composite engagement score. Nine dimensions span four units and two labelling schemes, and defensibility invalidates every other score on an item when it fails.
• Do not calibrate any dimension on one side only. Fairness stays out of the first prompt until a real failing anchor exists.
• skill_code cannot anchor cross-case matching — an eight-value enum on Lineup entries only, with skill_decomposition free-text and the nine Prime Suspect entries carrying none.
• con_id and trap_code are not the same label. A trap code is a class; a con is a character. The fan-out must exist before any con-grouped dimension is specified.
• Do not quietly reinstate con recurrence before the backfill. The five-appearance floor was applied against its own author and was not waived for thin cons.
• Cite entry files, not packets. The packets render Clock as deadline_note prose and print Break Index: 3 with no chain.
• The Wager's reveal surface is 45 items, not 258, and it is the same 45 on every axis. Score the 45 and carry the 213 beside it every run rather than averaging over the sixth that has payload.
• Do not run the gate or the 480 calibration reads against a corpus about to change underneath them.
• Record that two of Sarah's three named parameters do not ship in run one — novelty held on the target student, resonance held on the register test with a kill condition. Connection to theme is the only one of her three that ships. Fairness is held on Pass A; consistency moved off the rubric into an authoring lock, so nothing in run one catches a trap behaving one way in ps-case004 and another in ps-case009.

Verification (Sarah)

Before anything starts

• Answer or explicitly defer each of the eight decisions
• Confirm the M-CONCEPT correction in Section 0 matches your understanding of the calibration corpus

After Phase 0

• Read the decision list and confirm each default is one you can live with if you never answer
• Check that every cut dimension's reason still reads as a reason a year from now

After Phase 1

• Read ten Pass A hits and confirm each names the missing sentence rather than asserting a verdict
• Confirm the two deliberately unfixed items are visible as deliberate

After Phase 2

• Answer the gloss render question, or name who can
• Confirm the register merge rule was recorded before the run, not after

After Phase 3

• Read the coverage grid before the prompt is written, and confirm the Wager column is acceptable
• Read the contested-ness gate result specifically, whichever way it goes

After Phase 4

• Confirm the worklist's first screen is readable without scrolling
• Decide whether a per-family-per-dimension number is acceptable in place of one score

After Phase 5

• Verify a Brief with no exhibit list blocks the export
• Verify the interleave changed the Wager and Lineup lists and nothing else

After Phase 7

• Confirm Session A's sheet cannot be read as validating the rubric

Final reviewconfidence 71 / 100

What this gets right

The meeting killed every dimension that could not survive its own tests, and each cut was made

by the expert who proposed it, on corpus evidence. That is the hardest part of rubric design and

it is already done. The evaluability gate is a stricter instrument than the hand-label

calibration in the sibling plan, and it needs no ground truth, which is exactly the problem

engagement had. The promotion rule as a display rule over a complete result set is the right

shape: it keeps per-item scoring intact while making run-over-run movement readable.

Pros

• Every load-bearing number verifies. Fourteen of fifteen checked claims were exact, including

all six code line references. That is unusual and it means the plan can be built on.

• The decision list is genuinely blocking-aware. Each question names what it unblocks, so

Sarah can answer the cheap ones first.

• The flag-only tier solves a real problem. A dimension that is useful but not reliable had no

home before; now it has one with an exit condition.

• The findings outside the rubric have owners. Three cuts left anti-dark-pattern enforcement

looking handled, and the plan says so in those words.

Cons and what is genuinely uncertain

• This plan is large enough that its own phases will not all happen. Eight phases, two tester

sessions, a 213-item backfill and a 379-passage theming job is a quarter of work, not a sprint.

The ordering is stated but the scoping is not, and nothing here says what to drop.

• Run One returns two promoted lines Sarah already knows about if it runs before the backfill.

Decision 4 is therefore not really optional; the split is the only version that produces new

information.

• The gate's 480 reads are specified but not costed here. At the sibling plan's measured rates

this is pennies, but forty units scored twice across eleven dimensions is a different shape from

anything run so far, and the estimate should be made before it is scheduled.

• Three dimensions hang on one unanswered question. Decision 7 — which repo holds the gameplay

views — gates the render check, which gates trap recognition and two more rows. If that answer

is "nobody knows", the ships list shrinks.

• Session A's output is a sketch by its own admission. Three or four usable misses, explicitly

Lineup-only as specification, feeding a 213-item authoring job. That is thin evidence for a

large commission, and the plan says so without resolving it.

Where this plan could be inadequate

• If the answer to decision 7 is that no gameplay view is reachable from any repo here, trap

recognition and the gloss-dependent rows are unbuildable and the rubric ships with five

dimensions rather than eight.

• The re-check rule in Phase 5 creates a loop nobody has costed: theming re-opens defensibility

and difficulty on every passage it touches, across 379 passages.

• The worklist's carried-findings section is a good fix for rank one never moving, and it does not

solve the underlying problem that the largest findings are not the rubric's to close.

Adjustments worth considering

• Answer decisions 1, 3 and 4 first. They are the cheapest and they unblock the most.
• Consider running Phase 5's interleave immediately, independently of everything else. It has

evidence in this repo, it is a contained change at a named line, and the massed-practice finding

is the largest thing the meeting produced.

• Consider scoping run one to the five shipping dimensions and treating the held six as a

second release, rather than holding the whole rubric on the target student.

Confidence: 71 / 100

The dimension work is sound and unusually well evidenced — the verification pass found one

imprecise referent in fifteen claims. What holds this down is scope and sequencing rather than

correctness. Eight phases with two tester sessions and two large content jobs is a quarter of

work presented as a plan, three shipping dimensions hang on a single unanswered question about

where the gameplay views live, and the largest outputs belong to authors rather than to the

instrument that found them. The number moves up when decisions 1, 4 and 7 are answered, because

those three determine whether run one produces new information or restates what Sarah already

knows.