# Plan: SEE / SEE-IR / HELP — Introspection Trio

Status: implemented 2026-08-05 (all phases; HELP covers every word in a fresh VM, enforced by test)
Scope: `SEE` (source-level decompile), `SEE-IR` (optimized-IR dump), `HELP` (per-word docs), shared lookup infrastructure.
Each phase is self-contained and executable in a fresh context. Execute in order; every phase leaves the tree green (`cargo test --workspace` passes).

---

## Phase 0 — Consolidated Findings (read this first, do not re-derive)

All references verified at commit `e31407a` (branch `usability`).

### The template to copy: WORDS

`WORDS` is a **host primitive whose body runs Rust-side via the `pending_define` mechanism**. This is the exact pattern for SEE/SEE-IR/HELP because it gives a real dictionary entry (→ findable by `'`, listed by `WORDS`, tab-completable in the CLI via `crates/cli/src/main.rs:452-455`, exposed to web palette via `crates/web/src/lib.rs:61`) while the implementation can still call `next_token()` and write `self.output`.

- Registration: `register_words()` at `crates/core/src/outer.rs:6301-6310` — host fn pushes code `40` into `pending_define`, called from `register_primitives()` at `outer.rs:2991` under the `// -- Programming-Tools word set --` header.
- Dispatch: `handle_pending_define()` arm at `outer.rs:5328`: `40 => self.do_words(),`.
- Body: `do_words()` at `outer.rs:6045-6074`. Note `outer.rs:6050-6052`: it reads an optional same-line argument with `self.next_token()` **gated on `self.state == 0`** — SEE must copy this gate.
- **Used `pending_define` codes: 1–12, 20, 21, 25, 33, 40.** Free: **41 (SEE), 42 (SEE-IR), 43 (HELP)**. Legend comment at `outer.rs:232-233` must be extended.

### Allowed APIs (verified signatures)

| API | Location | Notes |
|---|---|---|
| `Dictionary::find(&self, name: &str) -> Option<(u32, WordId, bool)>` | `dictionary.rs:182` | `(word_addr, WordId, is_immediate)`, case-insensitive |
| `Dictionary::word_name(word_addr)` / `code_field(word_addr)` / `read_link` / `latest()` | `dictionary.rs:375/392/287/282` | manual entry walk |
| `flags::IMMEDIATE = 0x80`, `HIDDEN = 0x40`, `INTERNAL = 0x20` | `dictionary.rs:16-27` | raw flags byte = `dict.memory()[(word_addr+4) as usize]` — no getter exists |
| `ir_bodies: HashMap<WordId, Vec<IrOp>>` | `outer.rs:256` | **post-optimization** IR; populated for colon words AND all defining-word products AND IR primitives (see kind table below) |
| `host_word_names: HashMap<WordId, String>` | `outer.rs:214` | only populated by `register_host_primitive` (`outer.rs:2702-2703`) |
| `does_definitions: HashMap<WordId, DoesDefinition>` | `outer.rs:223` | DOES>-words |
| `output: Arc<Mutex<String>>` | `outer.rs:208` | ALL text output goes here; `HostAccess` has **no** emit method (`runtime.rs:17-60`) |
| `next_token()` | `outer.rs:690-704` | whitespace-delimited, advances `input_pos` |
| `register_host_primitive(name, immediate, func) -> anyhow::Result<WordId>` | `outer.rs:2686-2691` | public |
| `IrOp` enum, `#[derive(Debug, Clone, PartialEq)]` | `ir.rs:9-218` | **no `Display` impl exists anywhere in core** — formatter is net-new |
| `eval_output(input) -> String` test helper | `outer.rs:7566` | fresh VM per call; multi-eval tests build VM inline like `outer.rs:9077-9080` |

### Word-kind classification (SEE must distinguish these)

| Kind | Detectable via | SEE output strategy |
|---|---|---|
| Colon word / `:NONAME` | in `ir_bodies`, has captured source (Phase 3) | source (Phase 3) or IR (Phase 2) |
| IR primitive (`DUP`…) | in `ir_bodies`, no source | IR body + "primitive" tag |
| Host primitive (`.S`, `WORDS`…) | in `host_word_names` | `<built-in host word>` stub |
| CONSTANT / VARIABLE / VALUE / CREATE / DEFER / SYNONYM / BUFFER: / 2\*/F\* | in `ir_bodies` with recognizable shape (e.g. CONSTANT = `[PushI32(v)]`, insert sites: `outer.rs:3194/3226/3263/3309/3351/3388/3440/6517/6547/6590/7344`) | synthesized definition, e.g. `42 CONSTANT ANSWER` (Phase 3) |
| DOES>-defined word | key in `does_definitions` | show CREATE part + DOES> body IR |
| Interpreter special token (`:`, `;`, `VARIABLE`, `'`, `CHAR`…) | hardcoded matches `outer.rs:755-820`, `927-992`; several have **no dictionary entry at all** | `<compiler word, handled by the outer interpreter>` stub |

### Hard constraints

1. **Feature-free.** `outer.rs`, `ir.rs`, `dictionary.rs` compile without the `native` feature (`lib.rs:17-42`); web consumes core with `default-features = false` (`crates/web/Cargo.toml:15`). No `#[cfg(feature = "native")]` in any SEE code. Unit tests live in the existing `#[cfg(all(test, feature = "native"))]` module (`outer.rs:7553`) — that is fine and matches practice.
2. **`ir_bodies` stores post-optimization IR** (`finish_colon_def`: optimize at `outer.rs:2297`, insert at `outer.rs:2298`; inlining threshold 8 at `optimizer.rs:56`). `: FOO SQ SQ ;` shows `SQ`'s body inlined. This is a *feature* for SEE-IR (shows what the optimizer did) and the *reason* SEE needs separate source capture (Phase 3).
3. **Multi-line definitions**: compile state persists across `evaluate()` calls (`outer.rs:512-514` resets only `input_buffer`/`input_pos`); the driver (CLI `main.rs:411-416`) feeds lines. Source capture must accumulate across calls. On error, `evaluate()` wipes compile state (`outer.rs:523-535`) — capture state must be wiped there too.
4. **Error house style** (`outer.rs:1294/3400/4057` precedents): `anyhow::bail!("SEE: unknown word: {name}")`, `anyhow::bail!("SEE: expected word name")`.
5. **Compliance suite gives SEE zero coverage** — `toolstest.fth:38-39` explicitly excludes it. All coverage is hand-written unit tests. Adding SEE cannot break `compliance_tools`.
6. **MARKER correctness**: any new per-word map (source text, docs) must be snapshotted/restored in `MarkerState` (`outer.rs:166-176`, snapshot `outer.rs:3462-3480`, restore `outer.rs:3484-3506`), mirroring how `ir_bodies` is handled there.

### Anti-patterns (verified NOT to exist — do not invent)

- `HostAccess::emit(...)` / any output method on `HostAccess` — does not exist; capture `Arc::clone(&self.output)` instead.
- `Display for IrOp` — does not exist; write the formatter.
- A dictionary "entry struct" or kind tag — does not exist; classify via the VM-side maps above.
- `Dictionary::flags(addr)` getter — does not exist; read the raw byte.
- Refill-on-demand for a missing SEE argument — `REFILL`/`ACCEPT` are hardcoded to fail (`outer.rs:5824-5852`); `SEE` at end of line is an error, same as `'` (`outer.rs:4051-4053`).

---

## Phase 1 — IR pretty-printer (pure function, no VM changes)

**Goal:** a feature-free formatter turning `&[IrOp]` into readable, indented text. Foundation for SEE-IR and the SEE fallback path.

**What to implement:**

1. New module `crates/core/src/see.rs`, registered unconditionally in `lib.rs` next to `pub mod outer;` (`lib.rs:30`). Public API:
   ```rust
   /// Format an IR body as indented, one-op-per-line text.
   pub fn format_ir(ops: &[IrOp]) -> String
   ```
2. Exhaustive `match` over every `IrOp` variant (full list at `ir.rs:10-218`) — **no wildcard arm**, so adding a variant later forces a formatter update at compile time.
3. Simple ops print as their Forth-ish name plus payload: `PushI32(7)` → `push 7`, `Call(WordId(12))` → `call #12`, `TailCall` → `tail-call #12`. Resolve `#12` to a word name at a higher level (Phase 2) — `format_ir` itself stays name-agnostic, but takes an optional resolver to keep it pure:
   ```rust
   pub fn format_ir_with(ops: &[IrOp], resolve: &dyn Fn(WordId) -> Option<String>) -> String
   ```
   (`format_ir` delegates with a `|_| None` resolver.)
4. The six nested variants (`If`, `DoLoop`, `BeginUntil`, `BeginAgain`, `BeginWhileRepeat` at `ir.rs:78-99`, `BeginDoubleWhileRepeat` at `ir.rs:105-111`) print as Forth control words with 2-space indented bodies:
   ```
   if
     dup
     mul
   else
     drop
   then
   ```
5. Flat branch ops (`Block`/`BranchIfFalse`/`EndBlock`, `ir.rs:118-124`) print literally (`block L3` etc.) — they have no clean Forth surface syntax; do not attempt reconstruction.

**Verification checklist:**
- [ ] Unit tests in `see.rs` (plain `#[cfg(test)]`, NOT feature-gated — the module has no runtime dependency): nested `If` inside `DoLoop` indents correctly; every-variant smoke test via a `Vec` containing one of each simple op.
- [ ] `cargo check -p wafer-core --no-default-features` passes (proves feature-freedom).
- [ ] `cargo test --workspace` green; `cargo fmt --all` + `cargo clippy --workspace` clean.

**Anti-pattern guards:** no `impl Display for IrOp` (keep the formatter in `see.rs`, IrOp is data); no wildcard match arm; no `#[cfg(feature = "native")]`.

---## Phase 2 — SEE-IR word

**Goal:** `SEE-IR name` prints the stored post-optimization IR for any word — the optimizer-debugging view. Ship this before source-SEE: it is nearly free and immediately useful.

**What to implement:**

1. Copy the WORDS registration pattern verbatim (`outer.rs:6301-6310`): `register_see_ir()` pushes pending code **42**; register in `register_primitives()` next to `self.register_words()?` (`outer.rs:2991`). Extend the legend comment at `outer.rs:232-233`.
2. Dispatch arm in `handle_pending_define()` next to `outer.rs:5328`: `42 => self.do_see_ir(),`.
3. `do_see_ir()` (place near `do_words()`, `outer.rs:6045`):
   - Parse name: `let Some(name) = self.next_token() else { bail!("SEE-IR: expected word name") }` — **no** interpret-mode gate here (unlike WORDS' optional filter, the argument is mandatory; compile-mode `SEE-IR` may simply also parse — matches `'`).
   - Lookup: `self.dictionary.find(&name)` → else `bail!("SEE-IR: unknown word: {name}")`.
   - Classify per the Phase 0 kind table, in this order: `ir_bodies` hit → header line + `see::format_ir_with(...)` with a resolver that maps `WordId` → name (build once from a dictionary walk: `latest()`/`read_link`/`word_name`/`code_field`, `dictionary.rs:282/287/375/392`); `host_word_names` hit → `SEE-IR: <name> is a built-in host word`; neither → `SEE-IR: <name> has no IR body`.
   - Header line format: `\ <NAME> — <n> ops (optimized IR)`, plus ` immediate` when the find() flag is set, plus `does>` info when `does_definitions` has the id.
   - Write everything into `self.output.lock().unwrap()`; end with `\n` (multi-line output convention from commit `2910884`).
4. Special-token names (`:`, `VARIABLE`, `'`, …): after dictionary miss, check a small const list of known interpreter tokens (source: match arms at `outer.rs:755-820`, `927-992`) and print `SEE-IR: <name> is handled directly by the outer interpreter` instead of erroring.

**Documentation references:** WORDS pattern `outer.rs:6301-6310`, `5328`, `6045-6074`; error style `outer.rs:4057`; output convention `outer.rs:6056-6073`.

**Verification checklist:**
- [ ] Tests (in `outer.rs` test module, `eval_output` style, cf. `outer.rs:9035-9041`):
  - `: SQ DUP * ; SEE-IR SQ` output contains `dup` and `mul`;
  - `: FOO SQ SQ ; SEE-IR FOO` shows the **inlined** body (contains two `mul`, no `call`) — locks in the "optimized view" semantics;
  - `SEE-IR DUP` works (IR primitive); `SEE-IR WORDS` prints host-word stub; `SEE-IR NOSUCHWORD` errors with `SEE-IR: unknown word: NOSUCHWORD`; bare `SEE-IR` errors with `expected word name`;
  - `SEE-IR :` prints the interpreter-token message.
- [ ] `IF`/`ELSE`/`THEN` and `DO LOOP` bodies render indented (one structured-word test).
- [ ] `cargo test --workspace` green; fmt + clippy clean; `cargo check -p wafer-core --no-default-features` passes.

**Anti-pattern guards:** do not print via a nonexistent `HostAccess` emit; do not gate name parsing on `state == 0` (mandatory arg, not optional filter); do not `THROW -13` (plain `bail!` matches TO/SYNONYM precedent).

---

## Phase 3 — Source capture + SEE

**Goal:** `SEE name` prints the original source text `: name … ;` for colon words, synthesized definitions for data words, graceful stubs otherwise. This is the user-facing SEE.

**What to implement:**

1. **Capture fields** on `ForthVM` (near `compiling_ir`, `outer.rs:204`):
   ```rust
   compiling_source: String,      // accumulated raw text of the definition in progress
   source_capture_from: Option<usize>,  // input_pos where capture started in the CURRENT buffer
   word_sources: HashMap<WordId, String>,
   ```
2. **Capture protocol** (verbatim source, including comments and string literals — token-level reassembly would lose them):
   - `start_colon_def()` (`outer.rs:2097`): set `source_capture_from` to the position where `:` began. `interpret_token()` receives the token already consumed, so record the position **before** dispatch: in the `evaluate()` loop (`outer.rs:518-521`), remember `pos_before = self.input_pos` minus token — simplest correct form: capture `token_start` inside `next_token()` (`outer.rs:690-704`) into a new field `last_token_start: usize` as it skips whitespace; `start_colon_def` then does `self.source_capture_from = Some(self.last_token_start)`.
   - End of `evaluate()` (after the loop, `outer.rs:~536`): if still compiling and capture active, flush `input_buffer[from..]` + `'\n'` into `compiling_source`, reset `source_capture_from = Some(0)` so the next buffer continues capture from its start.
   - `finish_colon_def()` (`outer.rs:2266`): flush `input_buffer[from..=pos of ';']`, store `word_sources.insert(word_id, normalized)`, clear capture state. Normalize only trailing whitespace; keep interior verbatim.
   - Error path `outer.rs:523-535`: clear both capture fields alongside the existing compile-state wipe.
   - `:NONAME` and quotations (`outer.rs:759-761, 773-778`): skip capture (no name to SEE) — guard on `compiling_name.is_some()`.
3. **MARKER integration**: add `word_sources` to `MarkerState` (`outer.rs:166-176`), snapshot (`outer.rs:3462-3480`) and restore (`outer.rs:3484-3506`) exactly as `ir_bodies` is handled there.
4. **SEE word**: pending code **41**, same registration/dispatch shape as Phase 2. `do_see()` resolution order:
   1. `word_sources` hit → print stored source verbatim, append ` immediate` on its own line if flagged (cf. `set_immediate`, `dictionary.rs:404`).
   2. Recognizable data-word IR shape (Phase 0 kind table) → synthesized one-liner. CONSTANT `[PushI32(v)]` → `<v> CONSTANT <NAME>`; VARIABLE → `VARIABLE <NAME>  ( addr=<v> )`; VALUE `[PushI32(a), Fetch]` → `<cur> VALUE <NAME>` reading current value via `self.rt` memory read if cheap, else `VALUE <NAME>`; SYNONYM `[Call(id)]` → `SYNONYM <NAME> <OLD>`; DEFER → `DEFER <NAME>` plus current target name via `does`/pfa lookup when resolvable.
   3. `ir_bodies` hit (primitive or pre-capture colon word) → `\ <NAME> is a primitive; IR:` + `format_ir_with` output (reuse Phase 1/2 machinery — SEE never dead-ends).
   4. `host_word_names` hit → `<NAME> is a built-in host word`.
   5. Interpreter-token list → `<NAME> is handled by the outer interpreter (compiler word)`.
   6. Else → `bail!("SEE: unknown word: {name}")`.
5. **Boot words get sources for free**: `boot.fth` definitions flow through the same `evaluate()`/`finish_colon_def` path, so `SEE NIP` etc. shows real boot source. Verify, don't assume — one test below.

**Documentation references:** compile-state lifecycle `outer.rs:512-535`, `2097-2121`, `2266-2329`; multi-line REPL driver `main.rs:411-416`; MarkerState `outer.rs:166-176, 3462-3506`.

**Verification checklist:**
- [ ] `: SQ DUP * ; SEE SQ` prints `: SQ DUP * ;` (verbatim, one line).
- [ ] Multi-line: inline-VM test (pattern `outer.rs:9077-9080`): `evaluate(": TRI\")` then `evaluate(\"  DUP DUP ;")`, then `SEE TRI` shows both lines.
- [ ] Comment survives: `: C ( n -- n ) 1+ ; SEE C` output contains `( n -- n )`.
- [ ] `42 CONSTANT A SEE A` → `42 CONSTANT A`; `VARIABLE V SEE V` → contains `VARIABLE V`.
- [ ] `SEE NIP` (boot word) prints a colon definition, not an IR dump.
- [ ] `SEE DUP` prints the primitive-IR fallback; `SEE WORDS` prints host stub; `SEE '` prints interpreter-token message; unknown word errors in house style.
- [ ] MARKER round-trip: define word, set marker, redefine, execute marker, `SEE` shows the original — plus existing marker tests still green.
- [ ] Immediate flag: `: I2 ; IMMEDIATE SEE I2` output contains `immediate`.
- [ ] Error path: force `unknown word` mid-definition, then define a fresh word — its captured source must not contain debris from the aborted definition.
- [ ] Full suite + fmt + clippy + `--no-default-features` check.

**Anti-pattern guards:** do not reconstruct source from tokens (loses comments/strings/spacing); do not capture into `word_sources` for `:NONAME`; do not forget the error-path wipe (`outer.rs:523-535`) — stale capture corrupts the next definition's source; `evaluate()` resets `input_pos` per call (`outer.rs:512-514`) so `source_capture_from` is per-buffer, never carried across calls uncleared.

---

## Phase 4 — HELP word + doc table

**Goal:** `HELP name` prints stack effect + one-line description; `HELP` alone prints usage. Shares lookup/classification with SEE.

**What to implement:**

1. New feature-free module `crates/core/src/wordhelp.rs`: a static table
   ```rust
   /// (NAME, stack effect, one-line description)
   pub const WORD_DOCS: &[(&str, &str, &str)] = &[
       ("DUP", "( x -- x x )", "Duplicate the top of the data stack."),
       ...
   ];
   pub fn lookup(name: &str) -> Option<(&'static str, &'static str)>  // case-insensitive
   ```
   Seed from the Forth 2012 glossary (stack effects are standardized). Cover, in priority order: core + core-ext words WAFER implements, then tools/double/float sets. Incomplete coverage is acceptable and expected — `HELP` says `no help for <name> (word exists)` when the word is defined but undocumented, which doubles as the TODO list.
2. `HELP` word: pending code **43**, same registration/dispatch shape as Phase 2. Resolution: parse optional name (bare `HELP` → usage line `HELP <word>  — also try: WORDS, SEE <word>, SEE-IR <word>`); table hit → print `NAME  ( stack effect )  description`; miss but dictionary hit → `no help for <name>` + hint `try SEE <name>`; miss both → house-style unknown-word error.
3. Cross-wiring (the "as useful as possible" part):
   - `SEE`/`SEE-IR` prepend the HELP line as a `\ ...` comment when the table has one.
   - `HELP` appends ` immediate` / `built-in` / `defined in boot.fth or user code` classification reusing the Phase 2/3 classifier — factor that classifier into a shared `fn classify_word(&self, name) -> WordClass` when Phase 4 lands (do NOT pre-build it in Phase 2; extract once there are two users, per smallest-change rule).
4. User-defined words: optional docstring convention — if the captured source's first parenthesized comment looks like a stack effect (`( ... -- ... )`), `HELP` echoes it for user words. No new syntax, zero cost, rewards idiomatic Forth style.

**Verification checklist:**
- [ ] `HELP DUP` prints stack effect + description; `HELP dup` (lowercase) same.
- [ ] `HELP` alone prints usage; `HELP NOSUCH` errors house-style; `HELP MYWORD` for undocumented-but-defined word prints the `no help` + `SEE` hint.
- [ ] `: SQ ( n -- n^2 ) DUP * ; HELP SQ` echoes `( n -- n^2 )`.
- [ ] Table lint test: iterate `WORD_DOCS`, assert every documented name resolves in a booted VM's dictionary (catches typos/renames mechanically).
- [ ] Full suite + fmt + clippy + `--no-default-features`.

**Anti-pattern guards:** no doc strings threaded through `register_primitive` call sites (200+ call-site churn, bloats outer.rs — the side table is deliberate); no partial-coverage panic — missing docs degrade gracefully.

---

## Phase 5 — Final verification + docs

1. **Full gate:** `cargo fmt --all` && `cargo clippy --workspace` (zero warnings) && `cargo test --workspace` (expect baseline 431 unit + new SEE/SEE-IR/HELP tests, 1 benchmark, 11 compliance, 9 comparison — all green).
2. **Feature-freedom proof:** `cargo check -p wafer-core --no-default-features` and web build `cd crates/web && wasm-pack build --target web --dev --out-dir www/pkg`.
3. **Manual REPL pass** (CLI): `SEE SQ`, `SEE-IR FOO` with inlining, `HELP DUP`, multi-line definition then SEE, tab-complete `SE<tab>` — confirm multi-line output renders per commit `2910884` conventions (block output, ` ok` on own line).
4. **Web REPL smoke:** serve `crates/web/www`, run the same commands — output flows through `take_output()` (`web/src/lib.rs:38-43`), no web-side changes expected.
5. **Anti-pattern grep:** `grep -n "cfg(feature" crates/core/src/see.rs crates/core/src/wordhelp.rs` → empty; `grep -n "impl Display for IrOp" -r crates/core` → empty; `grep -rn "emit" crates/core/src/see.rs` → empty.
6. **Docs:** `docs/FORTH.md:95` already lists SEE under Programming-Tools — verify claim now true; add SEE/SEE-IR/HELP to README feature list if words are enumerated there; extend CLAUDE.md test-count line.
7. **Compliance untouched:** `cargo test -p wafer-core --test compliance` — must stay 11/11 (suite excludes SEE by design, `toolstest.fth:38-39`).

---

## Deliberate scope cuts (revisit later, not now)

- **`SEE-WASM`** (disassemble compiled module via `wasmprinter`): compiled bytes are likely dropped after instantiation; `codegen.rs` unexamined. Separate plan if wanted.
- **IR→Forth source reconstruction** for optimized bodies: lossy and misleading post-inlining; the source-capture path makes it unnecessary.
- **`LOCATE` / editor integration**: needs file/line provenance in the dictionary; out of scope.
- **Forth-side doc syntax (`:doc`)**: revisit after self-hosting work starts; the `( n -- n^2 )` echo in Phase 4 covers the 80% case with zero syntax.