Hopper — moving the paint inventory into the corpus

Repos: PressRaven-Corpus (guides/hopper, guides/acrylics-field-guide), Hopper (build pipeline)

Filed here because the quick task that produced it came through Peregrine's Code Raven. Nothing in this plan touches Peregrine.

STATUS — what actually ran

Verified, not asserted

What the plan got wrong

This also splits step 6 in two: buildRecipes has no dependency on the description work and could be retired now, while attachGuide must wait for step 3.

Decisions taken overnight

• Recipe ids preserved verbatim. db.js layers ratio annotations and mixture photos onto built-in recipes by id, and those sync to a second device. Renaming combo-0 would orphan someone else's work.
• Paint ids are the app's own slugs, read from the built inventory so every canonical-name rename comes across rather than being reimplemented.
• Stale references resolved only within the same brand. 27 name titan-golden-hb, which no longer exists; 13 more resolve through the app's own alias table. Where a stale slug's brand has no match the reference is left unresolved — zinc-white-holbein-hb has no Holbein match and the nearest candidate is a different manufacturer's tube. 15 recipes carry one and name it in their confidence note.
• Descriptions seeded only for single-product colours. The 53 that share a colour stay empty; a shared paragraph asserts the formulations are interchangeable.

Waiting for a person

1. 125 descriptions, 53 of which need one paragraph pulled apart into two or three.
2. Grey V5 — recipes call it Charvin, the only swatch on disk says Holbein, and it is in no inventory. Three recipes cite it.
3. 18 unresolved recipe chips across 15 recipes: Cadmium Yellow Deep (10), Grey V5 (3), Transparent Orange (2), Zinc White, Prussian Blue.
4. Ten paints with no consistency — the field guide names a brand but never says which body.
5. Three near-duplicates to confirm: Cobalt Blue Deep against Cobalt Blue, Hansa Yellow Deep and Medium against Hansa Yellow.
6. PressRaven/tools/export-machine.py is uncommitted and holds two independent sets of changes: ~306 lines of Peregrine projections that were already in the working tree, and my ~133 Hopper lines. I did not commit it rather than attribute your earlier work to a Hopper change. My half is archived at PressRaven-Corpus/tools/export-machine-hopper.py with instructions. The live file works — python3 tools/export-machine.py hopper runs today.

Why this exists

A new PressRaven project guide for Hopper exported a game-specific session packet. The cause was structural: kind is a property of the category, Game was the only kind: project category, and NewGuideSheet hides the category picker when a kind offers exactly one. "+ New Project" silently wrote category: game.

That is fixed. An app category now exists, Hopper is recategorised, and its schema is written. This plan covers only what remains: the content migration.

Already done (not part of this plan)

• categories/app/ — category.json (kind: project), base schemas/entry.json, brief.json, design/tokens.json
• guides/hopper/guide.json — category: app, four surfaces in group_labels / group_order
• guides/hopper/schema-extensions.json — per-surface fields for colors, paints, recipes, primers
• Press Raven — Guide.kind resolved at load; four project affordances retargeted from categoryID == "game" to kind == "project"; the stale comment corrected. Builds clean.

The decision this plan implements

Alizarin Crimson's description exists once, in guides/acrylics-field-guide, at colour grain. Hopper's inventory is at product grain — colour × brand × consistency — so Alizarin Crimson is three rows.

That collapses everything a colour entry would have carried. versions was the cross-product comparison, which per-product descriptions subsume. formulations and combinations are derivable from the products and the recipes. subfamily and color_family are two-word shelf labels that cost nothing to repeat. similar_to moves to the product, because the archival substitute for Golden Open Alizarin is not automatically the substitute for the Liquitex. Three surfaces, not four: paints, recipes, primers.

How the content reaches the app today

Hopper/scripts/build-data.mjs → attachGuide() finds the rendered public/reference/acrylics-guide.html in the catalog, parses it with regex (parseGuidePaintEntries), matches paint names through a fuzzy guideMatchKey, and patches p.guide = {description, similarTo, versionsText} onto each inventory record. Misses are covered by a hand-maintained GUIDE_NAME_ALIASES table and logged as coverage warnings.

So corpus prose reaches the app by being printed to HTML and scraped back out. Killing that round-trip is the point of the migration, not a side effect.

Measured joins

Run against the live data on 2026-08-27.

The three problems the numbers expose

1. Every product needs its own description, and most do not have one.

The worst shared colours are quinacridone magenta (5 products), titanium white (4), burnt umber (4), then green gold, burnt sienna and paynes gray at 3 each. Those 53 are where the correction bites: one paragraph has to become three or five that actually differ, and only a person who has used the tubes can write the difference.

Nine of the shared colours also disagree on pigment across their products — Golden's Alizarin Crimson is PR 83, Charvin's is PR112/PR146/PBK7; cobalt green is PG 19 vs PG 26 — so those seeds are not merely undifferentiated, they are wrong for at least one sibling. The app already shows Charvin's Alizarin wearing Golden's prose, so per-product descriptions fix a live inaccuracy rather than merely avoiding a new one.

2. Fourteen field-guide descriptions have no product to land on. Of the 18 colours with no row in inventory.source.json, only 4 exist anywhere in the built 447 (Hansa Yellow Medium, Olive Green, both Phthalo Blues, all Charvin, all unowned). The other 14 include Nickel Azo Gold and Ultramarine — the same two names the recipes reference 13 and 17 times. Under one grain these need creating as unowned paint entries rather than being dropped, which resolves the dangling recipe references in the same pass. Two are not paints at all and should be resolved rather than preserved: Cadmium Yellow Medium or Light is a compound title, Ultramarine is an alias for Ultramarine Blue.

3. Seventy-one recipe references that match no paint. Concentrated in eight names, and mostly one alias each: Titan ×27 (= Titan Buff), Ultramarine ×17 (= Ultramarine Blue), Nickel Azo Gold ×13, Titan / Unbleached Titanium ×4, Grey V5 ×3, Phthalo Blue (Green Shade) ×2. The app resolves these through GUIDE_NAME_ALIASES in build-data.mjs. The migration should resolve them once, into real references, and let the alias table die with the scrape.

Steps

1. Branch. hopper-corpus-migration, pushed with -u on the first push. This is a wide migration writing ~297 files at once and touching a frozen guide, which is the case the branching rule exists for. Merge back as soon as step 5 verifies; delete after.
2. Products — ~222 entries — mechanical pass. From Hopper/public/data/inventory.source.json: name→title, notes→summary, bodyLabel→paint_body, plus brand, owns, family→color_family (singularise Browns), pigment, nature, bias, opacity, intensity, uses{}, visuals from the swatch. Then create the ~14 colours that have no product as unowned entries, resolving the two that are aliases rather than paints. Every entry lands at status: seed with body empty. Write no descriptions in this step — a generated paragraph that looks authored is worse than a visibly empty field.
3. Descriptions — 208 entries — authoring pass. Seed from the field guide where a colour matches, then differentiate. Order the work by how wrong the seed is: the 53 sharing a colour first (they are the ones a shared paragraph actively misdescribes, and the 9 pigment disagreements sit inside them), then the 52 with no seed, then the 103 that are 1:1 and mostly need a formulation sentence added. This is the step that cannot be scripted and should not be rushed to a deadline set by the other five.
4. Recipes — 71 — and primers — 4. Independent of steps 2–3 and can start immediately. Recipes: kind→recipe_kind, category→recipe_category, subject, tier, title, intro→summary, body, paintsText→paints_text, and paints resolved to real entry titles with the 71 aliases applied — which step 2 has already made resolvable by creating Nickel Azo Gold and Ultramarine Blue. Primers from catalog.json: volume, subject, audience, palette, file, class: document.
5. Machine target, then verify by diffing. Declare machine_target on guides/hopper/guide.json with outputs for inventory.source.json, recipes.json and the primer metadata. Export, then diff the generated files against the committed ones. A clean diff is the acceptance test — it proves the corpus reproduces the app's current content before anything downstream changes. Reconcile every difference; do not accept a diff by explaining it.
6. Retire the scrape. Only after step 5 is clean. Delete attachGuide() and GUIDE_NAME_ALIASES from build-data.mjs: description and similarTo now arrive as real per-product fields, and versionsText has no successor — the app should render sibling products' summaries instead, which it can compute from group. Then retire guides/acrylics-field-guide, keeping its snapshot. It cannot become a view over Hopper, because Hopper no longer holds colours.

Sequencing note

Steps 2 and 4 are independent and can run in parallel; step 3 depends on step 2 and is the long pole by a wide margin. Step 5 is a hard barrier — the diff cannot be read until 2 and 4 have landed, though it does NOT wait on step 3, since an empty body diffs cleanly against an app that has no per-product description today. Step 6 must not begin until step 5 is clean, because deleting the scrape while the export is still wrong removes the only working path content has into the app.

That split matters: steps 2, 4, 5 and 6 are a few days of mechanical work that can land on their own, and step 3 is an open-ended writing project that can run for as long as it needs behind them. Treating them as one deliverable is how this stalls.

Final review

What this buys. One home for every fact, at one grain. A description now describes a tube a person can actually buy, instead of a colour abstraction that three different formulations wear badly. The regex round-trip through a rendered document dies, and with it the alias table and the coverage warnings. Recipe references become checkable. Press Raven gets a real editing surface over content that is currently hand-edited JSON.

What it costs. A frozen, finished guide loses its status as source of truth, and acrylics-field-guide is the guide with the most careful editorial history in the corpus — its vessel_notes document a recategorisation and a schema audit done specifically to make it safe to unfreeze. Roughly 297 files appear at once. The export in step 5 has to be written before any of it is provably correct.

Edge cases it does not cover. The 447-record built inventory includes 239 manufacturer full-line paints that exist only in the xlsx files and are not in inventory.source.json; this plan migrates the 208 source records and leaves the full-line merge exactly as it is. Swatch images stay in the Hopper repo and are referenced, not moved. The intensity vocabulary has nine values doing the work of about four, and locking it into the corpus unexamined would make it permanent.

Worth considering instead. Steps 3 and 4 alone — products and recipes, no colours — deliver a working project guide and leave the field guide frozen and the scrape alive. That is roughly a quarter of the work for most of the day-to-day benefit, and it keeps the hardest question (what happens to a finished guide) unanswered rather than answered under time pressure. Sarah chose the full move with the duplication cost stated, so this plan implements that; the smaller version stays available if step 2 proves as messy as the versions problem suggests.

Confidence: 80 on the schema and the mechanical steps — settled, verified against real data, joins measured rather than assumed, and the one-grain model removes the cross-reference machinery that was the previous version's weakest part. Confidence: 45 that this plan describes how long the whole thing takes, because step 3 is a writing project wearing a migration's clothes.