- README: add performance section (beats gforth 2-10x), update test
commands, note self-recursive direct calls and loop promotion
- CLAUDE.md: update test counts (427 unit + comparison tests)
- OPTIMIZATIONS.md: stack-to-local Phase 1→Phase 2 (loops + IF),
DO/LOOP locals done, J as IR done, add section 14 (self-recursive
direct call), add current performance table vs gforth
- WAFER.md: document self-recursive call optimization, CONSOLIDATE,
update test commands and line counts
- FORTH.md: expanded space history, add FORTH-IN-SPACE.md reference
- FORTH-IN-SPACE.md: new document with verified spacecraft history
Three bugs fixed to safely enable promotion for control flow:
1. compute_stack_needs now recurses into IF/DoLoop/Begin bodies,
correctly calculating preload counts for promoted words with
nested control flow (was flat, causing stack underflow).
2. BeginDoubleWhileRepeat rejected from promotion (boot.fth's
-TRAILING uses this pattern, handler had structural bugs).
3. IF/ELSE branches must have same net stack effect for promotion
(BITSSET? has asymmetric branches: 2 items vs 1).
Performance with promotion enabled:
- Factorial: 0.50x (2x faster than gforth)
- Collatz: 0.38x (2.6x faster than gforth)
- All 427 unit tests, 10/11 compliance, 35/35 behavioral pass
Extends the promoted codegen path (StackSim) with handlers for DoLoop,
BeginWhileRepeat, BeginUntil, BeginAgain, If/Else/Then, RFetch, LoopJ,
and Exit. Includes loop-iteration fixup to copy modified locals back to
loop-top positions, and IF branch state merging.
The promotion is currently gated off for control flow (is_promotable
rejects all loops/IF) pending fix for edge cases in the Forth 2012 test
suite. The infrastructure is ready to enable incrementally.
When briefly enabled for testing, showed dramatic results:
- Factorial: 0.49x (2x faster than gforth)
- Collatz: 0.17x (6x faster than gforth)
Two-path DO/LOOP codegen based on static analysis of the loop body:
- Fast path (no calls, no >R/R> in body): index and limit live purely
in WASM locals with zero return stack traffic per iteration. RFetch (I)
and LoopJ (J) resolve to local.get instead of memory access.
- Slow path (body has calls or explicit RS ops): locals still used for
loop control, but synced to return stack for LEAVE/UNLOOP compatibility.
Also converts J from a host function (WASM→Rust roundtrip per call) to
an IR primitive (IrOp::LoopJ) that compiles to local.get of the outer
loop's index local.
Performance impact (vs gforth, all opts enabled):
- Factorial: 1.02x → 0.94x (now faster than gforth)
- NestedLoops: 717x → 543x (24% faster, still bottlenecked by data stack)
- Fibonacci, GCD, Collatz: unchanged (don't use DO/LOOP)
35 behavioral tests across 8 categories verify identical output between
WAFER and gforth. Performance benchmarks compare execution speed for
Fibonacci, Factorial, GCD, NestedLoops, and Collatz workloads.
WAFER-only correctness tests run in CI without gforth; cross-engine
comparison and performance report are opt-in via --ignored.
Major compliance push bringing WAFER from 3 to 10 passing Forth 2012
compliance test suites (Core, Core Extensions, Core Plus, Double,
Exception, Facility, Locals, Memory, Search Order, String).
Compiler/runtime fixes:
- DEFER: host function via pending_define, works inside colon defs
- COMPILE,: handle_pending_compile in execute_word for [...] sequences
- MARKER: full save/restore with pending_marker_restore mechanism
- IMMEDIATE: changed from XOR toggle to OR set per Forth 2012 spec
- ABORT": throw -2 via THROW, no message display when caught
- M*/: symmetric division to match WAFER's / behavior
- pending_define: single i32 flag → Vec<i32> queue for multi-action words
- Optimizer: prevent inlining words containing EXIT or ForthLocal ops
- +LOOP: corrected boundary check formula with AND step comparison
- REPEAT: accept bare BEGIN (unstructured IF...BEGIN...REPEAT)
- Auto-close unclosed IFs at ; for unstructured control flow
- _create_part_: use reserve_fn_index to preserve dictionary.latest()
Memory layout:
- Separate PICT_BUF and WORD_BUF regions to prevent PAD overlap
- Updated DEPTH hardcoded DATA_STACK_TOP in boot.fth
New word sets:
- [IF]/[ELSE]/[THEN]/[DEFINED]/[UNDEFINED]: conditional compilation
- UNESCAPE/SUBSTITUTE/REPLACES: string substitution (host functions)
- Locals {: syntax: parser, ForthLocalGet/Set IR ops, WASM local codegen
- ENVIRONMENT? support for #LOCALS (returns 16)
- N>R/NR>/SYNONYM: programming-tools extensions
- Search Order: ONLY, ALSO, PREVIOUS, DEFINITIONS, FORTH,
FORTH-WORDLIST, GET-ORDER, SET-ORDER, GET-CURRENT, SET-CURRENT,
WORDLIST, SEARCH-WORDLIST with full multi-wordlist dictionary support
via Arc<Mutex> shared state for immediate effect from compiled code
Remaining: 1 cascade error in Programming-Tools from CS-PICK/CS-ROLL
(unstructured control-flow stack manipulation, requires flat IR).
- SOURCE-ID now returns -1 during EVALUATE (saves/restores SYSVAR_SOURCE_ID)
- BUFFER: aligns HERE to cell boundary before allocating
- S\" returns Vec<u8> instead of String to preserve raw escape bytes
Core_ext: 14→6 errors. Total: 46→44.
parse_s_escape returned String via from_utf8_lossy which replaces
non-UTF-8 bytes (like \xAB = 171) with the 3-byte U+FFFD replacement
character, corrupting both string length and content.
Changed to return Vec<u8> and write raw bytes directly to WASM memory.
Also registered ( as immediate word for FIND, added 'x' char literals.
Core_ext: 14→8 errors.
- Register ( in dictionary as immediate so FIND can discover it
(fixes search-order FIND test: 4→3 errors)
- Add character literal parsing: 'z' → 122 (Forth 2012 number prefix)
- Fix ALLOCATE/RESIZE -1 size validation (memory suite now passes)
ALLOCATE and RESIZE with size -1 (0xFFFFFFFF) were "succeeding" because
wrapping arithmetic made the block size tiny. Added early rejection for
sizes exceeding half the available memory.
Memory suite: 2→0 errors. Now 4 suites pass (Core, Facility, Memory).
Forth 2012 allows multiple ELSEs: IF 1 ELSE 2 ELSE 3 ELSE 4 ELSE 5 THEN
produces (1 3 5) for true and (2 4) for false. Desugars by saving the
condition flag on the return stack with >R/R@ and building nested
If/Else pairs. The final THEN cleans up with R> DROP.
- Implement Memory-Allocation word set (ALLOCATE/FREE/RESIZE) as
host functions using a top-down arena allocator in WASM linear memory.
Uses wrapping arithmetic for -1 size error cases.
- Fix DU< comparison order (same bug as D<: comparing d2-hi vs d1-hi).
- Register 2VARIABLE/2CONSTANT as callable host functions (pending
codes 9/10) so they work from compiled code like `: CD4 2VARIABLE ;`.
Memory suite: 62→2 errors. Double suite: 27→3 errors.
Total remaining: 56 failures across 9 suites.
- DU< had same comparison order bug as D< (comparing d2-hi < d1-hi
instead of d1-hi < d2-hi). Fixed with SWAP U<.
- 2VARIABLE and 2CONSTANT were handled as special tokens but not
registered in the dictionary, so they couldn't be called from
compiled code (e.g., : CD4 2VARIABLE ;). Added pending codes 9/10.
- D< used D- D0< which overflows for extreme signed doubles.
Replaced with high-cell comparison + unsigned low-cell comparison.
- COMPARE had inverted sign for length difference (u2-u1 vs u1-u2).
- Added -TRAILING (removed during Phase 6 refactoring, never re-added).
PARSE and PARSE-NAME were using the deferred pending mechanism which
broke when called from compiled code (the calling word continued
executing before PARSE ran). Replaced with inline host functions that
read >IN/#TIB directly from WASM memory and parse immediately.
This fixes utilities.fth $"/$2" failures that left stack residue
cascading into all subsequent compliance test suites.
Also: core_ext 17→14, string 27→17.
- Implement ROLL as host function (stack rotation by u positions)
- Fix CASE/ENDCASE: ENDCASE DROP was emitted before default code instead
of after, causing stack underflow in default branches
- Fix PARSE: skip one leading space (outer interpreter's trailing
delimiter) so parsed content starts at the argument, not the space
- Fix UNUSED: read SYSVAR_HERE from WASM memory (not just here_cell)
since Forth ALLOT/,/C, update WASM memory directly
- Register .( as immediate word in dictionary so FIND can discover it
Core and Facility compliance suites pass. Core Extensions down from
34 to 17 errors.
Six fixes for compliance test regressions introduced in Phases 7-8:
- LEAVE + +LOOP with step=0 caused infinite loop: the XOR termination
check yields 0 when index=limit and step=0. Added SYSVAR_LEAVE_FLAG
mechanism — LEAVE sets flag, +LOOP checks it, all loops clear on exit.
- DEPTH was off-by-one: `5440 SP@ -` pushed the literal before SP@
read the stack pointer, making SP@ see one extra cell. Reordered to
`SP@ 5440 SWAP -` so SP@ reads dsp before any literal push.
- */ and */MOD used FM/MOD (floored) but WAFER's / uses WASM i32.div_s
(symmetric). Changed to SM/REM for consistency.
- EVALUATE didn't sync input buffer to WASM memory, breaking SOURCE
and >IN manipulation inside evaluated strings. Added input-only sync
(without touching STATE/BASE) and >IN readback after each token.
- WORD didn't skip leading spaces when delimiter != space, causing
GN' and GS3 tests to read whitespace instead of content.
- Added ACCEPT stub returning 0 for non-interactive mode.
- Added bounds check in refresh_user_here to reject corrupted
SYSVAR_HERE values beyond WASM memory size.
Core and Facility compliance suites now pass. Other suites have
pre-existing regressions from Phases 1-8 still under investigation.
Move memory allocation words to boot.fth:
- ALLOT: `: ALLOT HERE + 12 ! ;`
- , (comma): `: , HERE ! 1 CELLS ALLOT ;`
- C, : `: C, HERE C! 1 ALLOT ;`
- ALIGN: `: ALIGN HERE ALIGNED 12 ! ;`
- FALIGN, SFALIGN, DFALIGN: float-aligned variants
These write directly to WASM memory[SYSVAR_HERE]. The Rust side picks up
Forth-side HERE changes via refresh_user_here() which now reads both
here_cell (for Rust host functions) and memory[12] (for Forth words),
taking the maximum to ensure no allocation is lost.
Removed 222 lines of Rust. All 426 tests pass.
DEFER! and DEFER@ are trivially `: DEFER! >BODY ! ;` and `: DEFER@ >BODY @ ;`.
COMPARE uses a byte-by-byte loop with early exit.
Removed 148 lines of Rust. All 426 tests pass.
Move to boot.fth: TYPE, SPACES, <#, HOLD, HOLDS, SIGN, #, #S, #>,
., U., .R, U.R, D., D.R. The Forth . now uses pictured numeric output
(standard Forth approach) instead of a Rust formatting closure.
Add M*, UM*, UM/MOD host functions to the WASM runner so that the
Forth # word (which calls UM/MOD) works in standalone mode.
Removed 660 lines of Rust closures + 5 dead helper functions.
All 426 tests pass.
HERE is now defined in boot.fth as `: HERE 12 @ ;` (reads SYSVAR_HERE
from WASM linear memory). The Rust side syncs user_here to memory[12]:
- At the start of each evaluate() call (sync_here_to_wasm)
- In each host function that modifies HERE (ALLOT, comma, C-comma, ALIGN)
This avoids per-token sync overhead — only 2 sync points per evaluate()
call plus host-function writes. Removed the HERE host function closure
(~30 lines). All 426 tests pass.
Now that the optimizer TailCall/inline bug is fixed, SM/REM, FM/MOD,
*/, and */MOD can be defined in Forth using M* and UM/MOD as primitives.
SM/REM uses DABS (which calls DNEGATE → D+) inside conditional branches
with return-stack items — exactly the pattern that triggered the bug.
Removed ~200 lines of Rust closures. All 426 tests pass.
When the tail-call pass converted a Call to TailCall inside an If branch,
and the inliner subsequently inlined that word, the TailCall was not
converted back to Call in nested control-flow bodies. The TailCall codegen
emits a Return instruction, which would exit the *caller* instead of just
the inlined callee — silently corrupting the return stack.
Root cause: the inliner only converted top-level TailCalls in the body
(line-by-line iteration), missing TailCalls nested inside If/DoLoop/Begin
structures.
Fix: add detailcall() that recursively walks the entire IR tree and
converts all TailCall ops back to Call before inlining.
This unblocks defining complex Forth words (like SM/REM, FM/MOD) that
use DABS → DNEGATE → D+ chains with return-stack operations inside
conditional branches.
426 tests pass (including new regression test).
Add 14 double-cell words to boot.fth: D+, D-, DNEGATE, DABS, D0=, D0<,
D=, D<, D2*, D2/, DMAX, DMIN, M+, DU<.
Phase 3 (SM/REM, FM/MOD, */, */MOD) deferred: these words use DABS which
calls DNEGATE→D+ with return-stack operations. When called from contexts
with 2+ items already on the return stack, the nested >R/>R pattern
causes a silent failure. Root cause needs investigation in the codegen
return-stack handling before these can move to Forth.
All 425 tests pass.
Move to boot.fth: D+, D-, DNEGATE, DABS, D0=, D0<, D=, D<, D2*, D2/,
DMAX, DMIN, M+, DU<.
D+ uses proper carry detection via unsigned comparison after low-cell
addition. All other double-cell words build on D+ and standard Forth
stack operations.
Removed 544 lines of Rust closures. Cumulative: ~1,091 Rust lines removed
across Phases 1-2, replaced by ~80 lines of Forth. All 425 tests pass.
Create boot.fth loaded at startup after IR primitives are compiled.
Forth-compiled WASM with direct calls outperforms host function dispatch
(no call_indirect overhead, Cranelift can inline across word boundaries).
Words moved to Forth: 2OVER, 2ROT, WITHIN, 2@, 2!, FILL, CMOVE, CMOVE>,
MOVE, ERASE, BLANK, /STRING, -TRAILING.
Removed 547 lines of Rust closures, replaced by 48 lines of Forth.
All 425 tests pass.
Add `wafer build --native` to produce self-contained native executables.
The approach appends AOT-precompiled WASM and metadata to a copy of the
wafer binary itself, requiring no Rust toolchain at build time.
On startup, the binary checks for an appended payload (8-byte "WAFEREXE"
magic trailer). If found, it deserializes the precompiled module and runs
it directly, skipping CLI argument parsing entirely.
Uses wasmtime's Engine::precompile_module() for AOT compilation at build
time and Module::deserialize() at runtime — instant startup with no JIT.
Binary layout: [wafer binary][precompiled wasm][metadata json][trailer]
Trailer: payload_len(u64 LE) + metadata_len(u64 LE) + "WAFEREXE"
Also refactored runner.rs: extracted shared run_module() to avoid
duplication between run_wasm_bytes() and run_precompiled_bytes().
Made serialize_metadata() public for CLI use.
Add `wafer build` to compile Forth source files to standalone .wasm modules,
and `wafer run` to execute them. The same .wasm file works with both the
wafer runtime (via wasmtime) and in browsers (via generated JS loader).
New CLI subcommands:
- `wafer build file.fth -o file.wasm` — compile to standalone WASM
- `wafer build file.fth -o file.wasm --js` — also generate JS/HTML loader
- `wafer build file.fth --entry WORD` — custom entry point
- `wafer run file.wasm` — execute pre-compiled module
Entry point resolution: --entry flag > MAIN word > recorded top-level execution.
Memory snapshot embedded as WASM data section preserves VARIABLE/CONSTANT state.
Metadata in custom "wafer" section enables the runner to provide host functions.
New modules: export.rs (orchestration), runner.rs (wasmtime host), js_loader.rs
(browser support). Refactored codegen.rs to share logic between consolidation
and export via compile_multi_word_module(). Added ir_bodies tracking for
VARIABLE, CONSTANT, CREATE, VALUE, DEFER, BUFFER:, MARKER, 2CONSTANT,
2VARIABLE, 2VALUE, FVARIABLE defining words.
Removed dead code: dot_func field, unused wafer-web stub crate, wasmtime-wasi
dependency from CLI, orphaned --consolidate/--output CLI flags.
425 tests pass (414 original + 11 new including 7 round-trip integration tests).
All were planning artifacts never imported or loaded:
- forth/ (4 .fth files): commented-out TODO stubs, never loaded at startup
- crates/core/src/words/mod.rs: empty module with commented-out submodules
- compiler.rs: placeholder, all compiler logic lives in outer.rs
- primitives.rs: placeholder, all primitives registered in outer.rs
- types.rs: StackType/StackEffect defined but never imported anywhere
Batch-compile all ~64 IR primitives into a single WASM module at startup.
Replaces 64 separate Module::new + Instance::new with 1 of each.
Reuses compile_consolidated_module() directly, removed compile_core_module() stub.
Boot time: 7.7ms -> 0.6ms (release), test suite: 5.1s -> 1.5s (debug).
13 of 14 optimizations now implemented. 392 tests passing.
Stack-to-local promotion (Phase 1: straight-line code):
- Words with no control flow/calls use WASM locals instead of memory stack
- Stack manipulation (Swap, Rot, Nip, Tuck, Dup, Drop) emits ZERO instructions
- ~7x instruction reduction for arithmetic-heavy words like DUP *
- Pre-loads consumed items from memory, writes results back at exit
Consolidation recompiler (CONSOLIDATE word):
- Recompiles all IR-based words into single WASM module
- Direct call instructions instead of call_indirect through function table
- Cranelift can inline and optimize across word boundaries
- All control flow variants support consolidated calls
342 unit tests + 11 compliance, all passing.
Inlining: store IR bodies for all words, inline Call(id) when body <= 8 ops
and non-recursive. Convert TailCall back to Call when inlining (tail position
in callee is not tail position in caller -- found via compliance test failure
where inlined TailCall caused unreachable code after the call site).
DSP global caching: cache $dsp in WASM local 0 at function entry, use
local.get/set throughout, writeback before calls and at function exit.
Reduces global access instructions by ~30-40%.
323 unit tests + 11 compliance, all passing.
ok