# WAFER Codebase Reading Order

Optimal sequence for learning the entire system. Each step builds on the previous.

---

## Phase 1: Mental Model Foundation

### 1. `crates/core/src/memory.rs` (148 lines)
**Read first.** Defines the physical memory map — every address, every region. You'll reference these constants everywhere else.
- Key insight: stacks grow DOWN, dictionary grows UP
- Memorize: DATA_STACK_TOP=0x1600, DICTIONARY_BASE=0x2D40
- System variables at offset 0: STATE, BASE, >IN, HERE, LATEST, SOURCE-ID, #TIB, HLD, LEAVE-FLAG
- Notice how regions are laid out to never overlap (verified by compile-time assertions)

### 2. `crates/core/src/ir.rs` (259 lines)
**The central data structure.** Every Forth word compiles to `Vec<IrOp>`. This is the language the optimizer speaks and the codegen consumes.
- ~70 variants across 10 categories
- Pay attention to control-flow variants: `If`, `DoLoop`, `BeginUntil`, `BeginWhileRepeat`, `BeginDoubleWhileRepeat` — they contain nested `Vec<IrOp>` bodies (tree structure, not flat)
- `Call(WordId)` and `TailCall(WordId)` — how words reference each other
- Float ops are separate from integer ops (separate stack)
- `IrWord` struct: name + body + is_immediate

### 3. `crates/core/src/error.rs` (84 lines)
**Quick read.** 15 error variants. Note `Throw(i32)` for the Exception word set and `Abort(String)` for ABORT".

### 4. `crates/core/src/config.rs` (61 lines)
**Quick read.** 7 optimization flags in two tiers: OptConfig (IR-level) and CodegenOpts (codegen-level). Default = all enabled.

---

## Phase 2: Data Structures

### 5. `crates/core/src/dictionary.rs` (906 lines)
**How words live in memory.** The dictionary is a linked list stored in a `Vec<u8>` that simulates WASM linear memory.
- Entry format: link(4) + flags(1) + name(N) + padding + code_field(4)
- Flags byte: IMMEDIATE=0x80, HIDDEN=0x40, LENGTH_MASK=0x1F
- `create()` writes the entry, starts HIDDEN; `reveal()` removes HIDDEN flag
- `find()`: fast path via HashMap index, fallback via linked-list walk
- Wordlist support: `current_wid`, `search_order`, `find_in_wid()`
- `DictionaryState` for MARKER save/restore
- Read every test — they document exact behavior

---

## Phase 3: The Pipeline

### 6. `crates/core/src/optimizer.rs` (1013 lines)
**IR transformations.** Read the `optimize()` function first to see the pass ordering, then each pass.
- `peephole()`: pattern-match adjacent ops. ~15 patterns. Runs to fixpoint. Study each match arm.
- `constant_fold()`: evaluate PushI32+PushI32+BinaryOp at compile time. Also unary and float.
- `strength_reduce()`: multiply by power-of-2 → shift. 0 compare → ZeroEq/ZeroLt.
- `dce()`: eliminate dead branches (constant condition), truncate after Exit.
- `inline()`: replace Call(id) with body if ≤8 ops, non-recursive, no Exit, no ForthLocals. `detailcall()` converts TailCall back to Call.
- `tail_call_detect()`: last Call → TailCall if return stack balanced. Recurses into If branches.
- Key: `apply_to_bodies()` — every pass recurses into control-flow nested bodies.

### 7. `crates/core/src/codegen.rs` (4205 lines) — **The Big One**
**IR → WASM translation.** Read in order:
1. **Constants** (lines 1-80): import indices, type indices, DSP/RSP/FSP globals, memory alignment
2. **Helper functions** (lines 80-210): `dsp_dec/inc`, `push_via_local`, `pop`, `peek`, `dsp_writeback/reload`, `rpush_via_local`, `rpop`
3. **Float helpers** (lines 225-330): `fsp_dec/inc`, `fpush_via_local`, `fpop`, `fpeek`, `emit_float_binary/unary/cmp`
4. **`emit_op()`** (line 344+): the giant match — each IrOp variant → WASM instructions. This is the heart.
5. **`compile_word()`**: builds the WASM module structure (imports, types, functions, element section)
6. **`compile_consolidated_module()`**: multi-function module for CONSOLIDATE/export
7. **Stack-to-local promotion**: analysis pass that replaces memory stack operations with WASM locals

Key patterns to understand:
- DSP cached in local 0: read from global at function entry, write back before calls and at exit
- Scratch locals at SCRATCH_BASE(1): used as temporaries for stack manipulation
- `EmitCtx`: carries f64 locals, Forth local base, loop local base, self_word_id for recursion
- DO/LOOP: index+limit in WASM locals when possible (fast path), fallback to return stack

---

## Phase 4: The Runtime Abstraction

### 8. `crates/core/src/runtime.rs` (152 lines)
**NEW: Read this before outer.rs.** Defines two traits:
- `Runtime` — abstraction over WASM execution backend (memory, globals, table, module instantiation, host function registration)
- `HostAccess` — memory/global ops available to host function callbacks
- `HostFn = Box<dyn Fn(&mut dyn HostAccess) -> Result<()>>` — runtime-agnostic host function type
- Key insight: ForthVM is now `ForthVM<R: Runtime>`, completely decoupled from wasmtime

### 8b. `crates/core/src/runtime_native.rs` (328 lines)
**NativeRuntime**: wasmtime implementation of Runtime trait.
- `CallerHostAccess` wraps wasmtime `Caller` to implement `HostAccess`
- `NativeRuntime` owns Engine, Store, Memory, Table, Globals
- `register_host_func`: creates a wasmtime `Func` that bridges `HostFn` → wasmtime callback
- Study how `instantiate_and_install` provides the 6 imports

### 9. `crates/core/src/outer.rs` — ForthVM struct (lines 1-240)
**Read the struct definition carefully.** ~35 fields. Group them mentally:
- Runtime: `rt: R` (generic over Runtime trait — no more direct wasmtime fields)
- Compilation state: state, compiling_name, compiling_ir, control_stack, compiling_word_id, compiling_locals
- Output: output (Arc<Mutex<String>>)
- Dictionary bridge: dictionary, user_here, here_cell, base_cell
- Word metadata: ir_bodies, host_word_names, word_pfa_map, does_definitions
- Shared state for host functions: pending_define, pending_actions, pending_does_patch, throw_code, word_lookup
- Configuration: config, batch_mode, deferred_ir
- Export support: toplevel_ir, recording_toplevel
- Advanced: marker_states, conditional_skip_depth, next_block_label, substitutions, search_order, next_wid

### 10. `crates/core/src/outer.rs` — new() and primitive registration
**How the VM boots.** Read:
- `new_with_config()`: creates `R::new()` runtime, then calls `register_primitives()` and loads boot.fth
- `register_primitive()`: creates dictionary entry → optimizes IR → compiles to WASM → `rt.instantiate_and_install()`
- `register_host_primitive()`: creates dictionary entry → `rt.register_host_func()` with HostFn closure
- `register_primitives()`: ~130 words registered in batch_mode, then `compile_batch()`
- Each host function: study 5-10 representative ones to understand the pattern

### 11. `crates/core/src/outer.rs` — Outer interpreter loop
**The main loop.** Read:
- `evaluate()`: sets up input buffer, calls `interpret_token()` in a loop
- `interpret_token()`: conditional compilation, `:` handling, `]` handling, dispatch to compile/interpret mode
- `interpret_token_immediate()`: string literals, dictionary lookup, execute found word, parse number
- `compile_token()`: POSTPONE, string literals, control-flow words (IF/ELSE/THEN/DO/LOOP/BEGIN/WHILE/REPEAT/AGAIN/UNTIL/CASE/OF/ENDOF/ENDCASE), dictionary lookup, compile Call(id), parse number → PushI32
- `finish_colon_def()`: optimize → codegen → install

### 12. `crates/core/src/outer.rs` — Control flow compilation
**Most complex part.** 13 `ControlEntry` variants. Understand:
- `ControlEntry::If { then_body }` → pushed when IF seen, then_body accumulates until ELSE or THEN
- `ControlEntry::Do { body }` → pushed by DO, body accumulates until LOOP/+LOOP
- `ControlEntry::Begin { body }` → pushed by BEGIN, resolved by UNTIL/AGAIN/WHILE
- `ControlEntry::BeginWhile { test, body }` → WHILE splits Begin into test + body
- `ControlEntry::Case/Of` → CASE/OF/ENDOF/ENDCASE pattern
- `ControlEntry::QDo` → ?DO (conditional entry)
- `ControlEntry::Ahead` → AHEAD (unconditional forward branch)
- CS-PICK and CS-ROLL: advanced control-flow manipulation for tools word set

---

## Phase 5: Self-Hosting

### 13. `crates/core/boot.fth` (307 lines)
**Forth replaces Rust.** 7 phases of definitions that replace host functions with compiled Forth.
- Phase 1: Stack/memory (DEPTH, PICK, 2OVER, FILL, MOVE, /STRING, -TRAILING)
- Phase 2: Double-cell arithmetic (D+, DNEGATE, D-, DABS, D0=, D0<, D=, D<, DU<)
- Phase 3: Mixed arithmetic (SM/REM, FM/MOD, */, */MOD) — built on M* and UM/MOD host primitives
- Phase 4: HERE, ALLOT, comma, C-comma, ALIGN — magic numbers for sysvar offsets
- Phase 5: I/O and pictured numeric output (TYPE, SPACES, <# HOLD # #S #> . U. .R U.R D. D.R)
- Phase 6: DEFER support (DEFER!, DEFER@)
- Phase 7: String operations, SOURCE, FALIGNED, etc.
- Key insight: why Forth not Rust? Self-hosting goal + compiled Forth with direct calls beats host function dispatch

---

## Phase 6: Production Features

### 14. `crates/core/src/consolidate.rs` (169 lines)
**Quick read.** Mostly tests. Real logic is in `codegen::compile_consolidated_module()` and `outer::ForthVM::consolidate()`. Understand the concept: merge all JIT modules into one, replacing call_indirect with direct call.

### 15. `crates/core/src/export.rs` (409 lines)
**wafer build pipeline.** Entry point resolution (--entry > MAIN > top-level), IR collection, memory snapshot, metadata embedding in custom section.

### 16. `crates/core/src/runner.rs` (402 lines)
**Standalone execution.** Creates the 6 imports from scratch, registers host function stubs for known words (., TYPE, SPACES, .S, M*, UM*, UM/MOD, DEPTH). Shows the minimal set needed to run exported modules.

### 17. `crates/cli/src/main.rs` (354 lines)
**CLI ties it together.** Three modes: REPL (rustyline), file evaluation, subcommands (build, run). Native executable trick: append AOT payload + "WAFEREXE" trailer to binary.

### 18. `crates/web/src/lib.rs` (56 lines)
**Browser entry point.** `WaferRepl` struct with `#[wasm_bindgen]`:
- `new()` → `ForthVM::<WebRuntime>::new()`
- `evaluate(input)` → returns output string
- `data_stack()`, `is_compiling()`, `reset()`

### 19. `crates/web/src/runtime_web.rs` (542 lines)
**WebRuntime**: browser implementation of Runtime trait.
- Uses `js_sys::WebAssembly` for module instantiation
- `WebHostAccess`: implements HostAccess via `js_sys` typed arrays
- Memory access through `Int32Array`/`Uint8Array` views on `WebAssembly.Memory.buffer`
- Closures kept alive via `_closures: Vec<JsValue>` to prevent GC

### 20. `crates/web/www/` (727 lines)
**Frontend**: app.js (terminal emulation, stack display), index.html, style.css.

---

## Phase 7: Testing

### 21. Unit tests (embedded in each source file)
Re-read each file's `#[cfg(test)] mod tests`. They document edge cases and expected behavior.

### 22. `crates/core/tests/compliance.rs`
Forth 2012 compliance infrastructure: boot_with_prerequisites, run_suite, 11 word set tests.

### 23. `crates/core/tests/comparison.rs`
Cross-engine benchmarks vs gforth. Performance validation.