From 38b956c7a367bc895078f12a510766f460eda063 Mon Sep 17 00:00:00 2001 From: Oleksandr Kozachuk Date: Mon, 13 Apr 2026 18:21:25 +0200 Subject: [PATCH] Fix markdown formatting to pass dprint CI check --- tools/reading_order.md | 41 +++++++++++++++++++++++++++++++++++++++- tools/trace_exercises.md | 25 ++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/tools/reading_order.md b/tools/reading_order.md index fcfb57e..95928b7 100644 --- a/tools/reading_order.md +++ b/tools/reading_order.md @@ -7,14 +7,18 @@ Optimal sequence for learning the entire system. Each step builds on the previou ## Phase 1: Mental Model Foundation ### 1. `crates/core/src/memory.rs` (148 lines) + **Read first.** Defines the physical memory map — every address, every region. You'll reference these constants everywhere else. + - Key insight: stacks grow DOWN, dictionary grows UP - Memorize: DATA_STACK_TOP=0x1600, DICTIONARY_BASE=0x2D40 - System variables at offset 0: STATE, BASE, >IN, HERE, LATEST, SOURCE-ID, #TIB, HLD, LEAVE-FLAG - Notice how regions are laid out to never overlap (verified by compile-time assertions) ### 2. `crates/core/src/ir.rs` (259 lines) + **The central data structure.** Every Forth word compiles to `Vec`. This is the language the optimizer speaks and the codegen consumes. + - ~70 variants across 10 categories - Pay attention to control-flow variants: `If`, `DoLoop`, `BeginUntil`, `BeginWhileRepeat`, `BeginDoubleWhileRepeat` — they contain nested `Vec` bodies (tree structure, not flat) - `Call(WordId)` and `TailCall(WordId)` — how words reference each other @@ -22,9 +26,11 @@ Optimal sequence for learning the entire system. Each step builds on the previou - `IrWord` struct: name + body + is_immediate ### 3. `crates/core/src/error.rs` (84 lines) + **Quick read.** 15 error variants. Note `Throw(i32)` for the Exception word set and `Abort(String)` for ABORT". ### 4. `crates/core/src/config.rs` (61 lines) + **Quick read.** 7 optimization flags in two tiers: OptConfig (IR-level) and CodegenOpts (codegen-level). Default = all enabled. --- @@ -32,7 +38,9 @@ Optimal sequence for learning the entire system. Each step builds on the previou ## Phase 2: Data Structures ### 5. `crates/core/src/dictionary.rs` (906 lines) + **How words live in memory.** The dictionary is a linked list stored in a `Vec` that simulates WASM linear memory. + - Entry format: link(4) + flags(1) + name(N) + padding + code_field(4) - Flags byte: IMMEDIATE=0x80, HIDDEN=0x40, LENGTH_MASK=0x1F - `create()` writes the entry, starts HIDDEN; `reveal()` removes HIDDEN flag @@ -46,7 +54,9 @@ Optimal sequence for learning the entire system. Each step builds on the previou ## Phase 3: The Pipeline ### 6. `crates/core/src/optimizer.rs` (1013 lines) + **IR transformations.** Read the `optimize()` function first to see the pass ordering, then each pass. + - `peephole()`: pattern-match adjacent ops. ~15 patterns. Runs to fixpoint. Study each match arm. - `constant_fold()`: evaluate PushI32+PushI32+BinaryOp at compile time. Also unary and float. - `strength_reduce()`: multiply by power-of-2 → shift. 0 compare → ZeroEq/ZeroLt. @@ -56,7 +66,9 @@ Optimal sequence for learning the entire system. Each step builds on the previou - Key: `apply_to_bodies()` — every pass recurses into control-flow nested bodies. ### 7. `crates/core/src/codegen.rs` (4205 lines) — **The Big One** + **IR → WASM translation.** Read in order: + 1. **Constants** (lines 1-80): import indices, type indices, DSP/RSP/FSP globals, memory alignment 2. **Helper functions** (lines 80-210): `dsp_dec/inc`, `push_via_local`, `pop`, `peek`, `dsp_writeback/reload`, `rpush_via_local`, `rpop` 3. **Float helpers** (lines 225-330): `fsp_dec/inc`, `fpush_via_local`, `fpop`, `fpeek`, `emit_float_binary/unary/cmp` @@ -66,6 +78,7 @@ Optimal sequence for learning the entire system. Each step builds on the previou 7. **Stack-to-local promotion**: analysis pass that replaces memory stack operations with WASM locals Key patterns to understand: + - DSP cached in local 0: read from global at function entry, write back before calls and at exit - Scratch locals at SCRATCH_BASE(1): used as temporaries for stack manipulation - `EmitCtx`: carries f64 locals, Forth local base, loop local base, self_word_id for recursion @@ -76,21 +89,27 @@ Key patterns to understand: ## Phase 4: The Runtime Abstraction ### 8. `crates/core/src/runtime.rs` (152 lines) + **NEW: Read this before outer.rs.** Defines two traits: + - `Runtime` — abstraction over WASM execution backend (memory, globals, table, module instantiation, host function registration) - `HostAccess` — memory/global ops available to host function callbacks - `HostFn = Box Result<()>>` — runtime-agnostic host function type - Key insight: ForthVM is now `ForthVM`, completely decoupled from wasmtime ### 8b. `crates/core/src/runtime_native.rs` (328 lines) + **NativeRuntime**: wasmtime implementation of Runtime trait. + - `CallerHostAccess` wraps wasmtime `Caller` to implement `HostAccess` - `NativeRuntime` owns Engine, Store, Memory, Table, Globals - `register_host_func`: creates a wasmtime `Func` that bridges `HostFn` → wasmtime callback - Study how `instantiate_and_install` provides the 6 imports ### 9. `crates/core/src/outer.rs` — ForthVM struct (lines 1-240) + **Read the struct definition carefully.** ~35 fields. Group them mentally: + - Runtime: `rt: R` (generic over Runtime trait — no more direct wasmtime fields) - Compilation state: state, compiling_name, compiling_ir, control_stack, compiling_word_id, compiling_locals - Output: output (Arc>) @@ -102,7 +121,9 @@ Key patterns to understand: - Advanced: marker_states, conditional_skip_depth, next_block_label, substitutions, search_order, next_wid ### 10. `crates/core/src/outer.rs` — new() and primitive registration + **How the VM boots.** Read: + - `new_with_config()`: creates `R::new()` runtime, then calls `register_primitives()` and loads boot.fth - `register_primitive()`: creates dictionary entry → optimizes IR → compiles to WASM → `rt.instantiate_and_install()` - `register_host_primitive()`: creates dictionary entry → `rt.register_host_func()` with HostFn closure @@ -110,7 +131,9 @@ Key patterns to understand: - Each host function: study 5-10 representative ones to understand the pattern ### 11. `crates/core/src/outer.rs` — Outer interpreter loop + **The main loop.** Read: + - `evaluate()`: sets up input buffer, calls `interpret_token()` in a loop - `interpret_token()`: conditional compilation, `:` handling, `]` handling, dispatch to compile/interpret mode - `interpret_token_immediate()`: string literals, dictionary lookup, execute found word, parse number @@ -118,7 +141,9 @@ Key patterns to understand: - `finish_colon_def()`: optimize → codegen → install ### 12. `crates/core/src/outer.rs` — Control flow compilation + **Most complex part.** 13 `ControlEntry` variants. Understand: + - `ControlEntry::If { then_body }` → pushed when IF seen, then_body accumulates until ELSE or THEN - `ControlEntry::Do { body }` → pushed by DO, body accumulates until LOOP/+LOOP - `ControlEntry::Begin { body }` → pushed by BEGIN, resolved by UNTIL/AGAIN/WHILE @@ -133,10 +158,12 @@ Key patterns to understand: ## Phase 5: Self-Hosting ### 13. `crates/core/boot.fth` (307 lines) + **Forth replaces Rust.** 7 phases of definitions that replace host functions with compiled Forth. + - Phase 1: Stack/memory (DEPTH, PICK, 2OVER, FILL, MOVE, /STRING, -TRAILING) - Phase 2: Double-cell arithmetic (D+, DNEGATE, D-, DABS, D0=, D0<, D=, D<, DU<) -- Phase 3: Mixed arithmetic (SM/REM, FM/MOD, */, */MOD) — built on M* and UM/MOD host primitives +- Phase 3: Mixed arithmetic (SM/REM, FM/MOD, */, _/MOD) — built on M_ and UM/MOD host primitives - Phase 4: HERE, ALLOT, comma, C-comma, ALIGN — magic numbers for sysvar offsets - Phase 5: I/O and pictured numeric output (TYPE, SPACES, <# HOLD # #S #> . U. .R U.R D. D.R) - Phase 6: DEFER support (DEFER!, DEFER@) @@ -148,31 +175,40 @@ Key patterns to understand: ## Phase 6: Production Features ### 14. `crates/core/src/consolidate.rs` (169 lines) + **Quick read.** Mostly tests. Real logic is in `codegen::compile_consolidated_module()` and `outer::ForthVM::consolidate()`. Understand the concept: merge all JIT modules into one, replacing call_indirect with direct call. ### 15. `crates/core/src/export.rs` (409 lines) + **wafer build pipeline.** Entry point resolution (--entry > MAIN > top-level), IR collection, memory snapshot, metadata embedding in custom section. ### 16. `crates/core/src/runner.rs` (402 lines) + **Standalone execution.** Creates the 6 imports from scratch, registers host function stubs for known words (., TYPE, SPACES, .S, M*, UM*, UM/MOD, DEPTH). Shows the minimal set needed to run exported modules. ### 17. `crates/cli/src/main.rs` (354 lines) + **CLI ties it together.** Three modes: REPL (rustyline), file evaluation, subcommands (build, run). Native executable trick: append AOT payload + "WAFEREXE" trailer to binary. ### 18. `crates/web/src/lib.rs` (56 lines) + **Browser entry point.** `WaferRepl` struct with `#[wasm_bindgen]`: + - `new()` → `ForthVM::::new()` - `evaluate(input)` → returns output string - `data_stack()`, `is_compiling()`, `reset()` ### 19. `crates/web/src/runtime_web.rs` (542 lines) + **WebRuntime**: browser implementation of Runtime trait. + - Uses `js_sys::WebAssembly` for module instantiation - `WebHostAccess`: implements HostAccess via `js_sys` typed arrays - Memory access through `Int32Array`/`Uint8Array` views on `WebAssembly.Memory.buffer` - Closures kept alive via `_closures: Vec` to prevent GC ### 20. `crates/web/www/` (727 lines) + **Frontend**: app.js (terminal emulation, stack display), index.html, style.css. --- @@ -180,10 +216,13 @@ Key patterns to understand: ## Phase 7: Testing ### 21. Unit tests (embedded in each source file) + Re-read each file's `#[cfg(test)] mod tests`. They document edge cases and expected behavior. ### 22. `crates/core/tests/compliance.rs` + Forth 2012 compliance infrastructure: boot_with_prerequisites, run_suite, 11 word set tests. ### 23. `crates/core/tests/comparison.rs` + Cross-engine benchmarks vs gforth. Performance validation. diff --git a/tools/trace_exercises.md b/tools/trace_exercises.md index 4f21e04..5940f81 100644 --- a/tools/trace_exercises.md +++ b/tools/trace_exercises.md @@ -1,6 +1,7 @@ # WAFER Trace-the-Compilation Exercises For each exercise, manually trace the Forth code through the full pipeline: + 1. **Outer interpreter** — tokenization, dictionary lookup, compile/interpret dispatch 2. **IR generation** — what Vec is produced 3. **Optimization** — which passes fire, what changes @@ -12,6 +13,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 1: Simple Arithmetic + ```forth : SQUARE DUP * ; ``` @@ -39,6 +41,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 2: Constant Folding + ```forth : TEN 5 5 + ; ``` @@ -62,6 +65,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 3: Peephole Elimination + ```forth : NOOP DUP DROP ; ``` @@ -80,6 +84,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 4: Strength Reduction + ```forth : DOUBLE 8 * ; ``` @@ -99,9 +104,11 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 5: Tail Call Detection + ```forth : FOO 1 + BAR ; ``` + (Assume BAR is a previously defined word)
@@ -119,6 +126,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 6: Control Flow — IF/THEN + ```forth : ABS DUP 0< IF NEGATE THEN ; ``` @@ -142,6 +150,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 7: DO LOOP + ```forth : STARS 0 DO 42 EMIT LOOP ; ``` @@ -165,6 +174,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 8: BEGIN UNTIL + ```forth : COUNTDOWN BEGIN DUP . 1 - DUP 0= UNTIL DROP ; ``` @@ -185,6 +195,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 9: Dead Code Elimination + ```forth : ALWAYS-TRUE TRUE IF 42 ELSE 99 THEN ; ``` @@ -203,6 +214,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 10: Swap Peephole Patterns + ```forth : TEST SWAP SWAP DROP DROP ; ``` @@ -221,6 +233,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 11: Nested Control Flow + ```forth : CLASSIFY DUP 0< IF DROP -1 ELSE 0> IF 1 ELSE 0 THEN THEN ; ``` @@ -229,6 +242,7 @@ Answers are below each exercise (scroll down or cover with paper). Answer 1. IR structure (after inlining): + ``` [Dup, ZeroLt, If { then: [Drop, PushI32(-1)], @@ -238,6 +252,7 @@ Answers are below each exercise (scroll down or cover with paper). }]) }] ``` + 2. Optimizer recurses into both If bodies. No constant conditions → no DCE. 3. Codegen: nested WASM `if/else/end` blocks. @@ -246,6 +261,7 @@ Answers are below each exercise (scroll down or cover with paper). --- ## Exercise 12: DOES> Defining Word + ```forth : CONSTANT CREATE , DOES> @ ; 5 CONSTANT FIVE @@ -277,6 +293,7 @@ FIVE . --- ## Exercise 13: Consolidation + ```forth : A 1 ; : B 2 ; @@ -301,6 +318,7 @@ CONSOLIDATE --- ## Exercise 14: Host Function Execution + ```forth 5 3 M* ``` @@ -326,6 +344,7 @@ CONSOLIDATE --- ## Exercise 15: Float Operations + ```forth : HYPOTENUSE FDUP F* FSWAP FDUP F* F+ FSQRT ; ``` @@ -346,6 +365,7 @@ CONSOLIDATE --- ## Exercise 16: BEGIN WHILE REPEAT + ```forth : COUNTDOWN BEGIN DUP WHILE DUP . 1 - REPEAT DROP ; ``` @@ -365,6 +385,7 @@ CONSOLIDATE --- ## Exercise 17: Batch Mode Compilation + ```forth ( During ForthVM::new() ) ``` @@ -389,10 +410,12 @@ CONSOLIDATE --- ## Exercise 18: wafer build Pipeline + ```forth ( file: hello.fth ) : MAIN ." Hello, World!" CR ; ``` + ```bash wafer build hello.fth -o hello.wasm ``` @@ -417,6 +440,7 @@ wafer build hello.fth -o hello.wasm --- ## Exercise 19: Stack-to-Local Promotion + ```forth : ADD3 + + ; ``` @@ -437,6 +461,7 @@ wafer build hello.fth -o hello.wasm --- ## Exercise 20: MARKER and State Restore + ```forth MARKER CLEAN : FOO 1 ;