WAFER Project Conventions

What is WAFER?

WAFER (WebAssembly Forth Engine in Rust) is an optimizing Forth 2012 compiler targeting WebAssembly. Currently a working Forth system with 200+ words, JIT compilation, 12 word sets at 100% compliance, and a full optimization pipeline (peephole, constant folding, inlining, strength reduction, DCE, tail calls, per-region stack-to-local promotion with DO/BEGIN loop and IF support, self-recursive direct calls, a typed calling convention for words with a known stack effect, self-guard expansion for recursive words, consolidation). Beats gforth on every benchmark, and SwiftForth sf64 on five of six (measured native-vs-native on x86-64; the macOS sf64 build is x86-64 under Rosetta and flatters WAFER). Includes a browser-based REPL via wasm-pack.

Architecture

Each Forth word compiles to its own WASM module via wasm-encoder
Modules share memory, globals (dsp/rsp/fsp), and a function table via runtime imports
IR-based compilation: Forth -> Vec<IrOp> -> WASM codegen -> runtime instantiation
Dictionary: linked-list in a Vec<u8> buffer simulating WASM linear memory
Primitives: either IR-based (compiled to WASM) or host functions (closures via HostFn)
Runtime trait: ForthVM<R: Runtime> is generic over the execution backend
- NativeRuntime (wasmtime) -- CLI, tests, AOT compilation
- WebRuntime (js-sys) -- browser REPL via wasm-pack

Crate Structure

crates/core -- compiler, optimizer, codegen, dictionary, runtime traits, outer interpreter
crates/cli -- CLI REPL with rustyline, wafer build/wafer run commands
crates/web -- browser REPL (wasm-bindgen entry point, WebRuntime, HTML/CSS/JS frontend)

Key Files

crates/core/src/outer.rs -- ForthVM<R: Runtime>: outer interpreter, compiler, all primitives
crates/core/src/runtime.rs -- Runtime + HostAccess traits (execution backend abstraction)
crates/core/src/runtime_native.rs -- NativeRuntime: wasmtime implementation (behind native feature)
crates/core/src/codegen.rs -- IR-to-WASM translation, module generation
crates/core/src/dictionary.rs -- Dictionary data structure with create/find/reveal
crates/core/src/ir.rs -- IrOp enum (the intermediate representation)
crates/core/src/memory.rs -- Memory layout constants (stack regions, dictionary base, etc.)
crates/core/src/optimizer.rs -- IR optimization passes (peephole, fold, inline, DCE, etc.)
crates/core/src/config.rs -- WaferConfig: unified optimization configuration
crates/core/src/consolidate.rs -- Consolidation recompiler (single-module direct calls)
crates/core/boot.fth -- Bootstrap Forth definitions loaded at startup
crates/cli/src/main.rs -- CLI REPL with rustyline
crates/web/src/lib.rs -- WaferRepl wasm-bindgen entry point
crates/web/src/runtime_web.rs -- WebRuntime: browser WebAssembly API via js-sys
crates/web/www/ -- Frontend (index.html, style.css, app.js)

Feature Flags (wafer-core)

default = ["native"] -- includes wasmtime, NativeRuntime, runner, export, etc.
native -- enables dep:wasmtime and all native-only modules
No features -- pure Rust only (dictionary, IR, optimizer, codegen, outer interpreter). Used by wafer-web.

Adding a New Word

IR primitive (simple stack/arithmetic/logic -- preferred when possible):

self.register_primitive("WORD_NAME", false, vec![IrOp::Dup, IrOp::Mul])?;

Host function (needs Rust logic -- I/O, dictionary manipulation, complex stack access):

let shared_state = Arc::clone(&self.some_field);
let func: HostFn = Box::new(move |ctx: &mut dyn HostAccess| {
    let sp = ctx.get_dsp();
    let val = ctx.mem_read_i32(sp);
    // ... logic using ctx for memory/global access ...
    ctx.set_dsp(sp + CELL_SIZE);
    Ok(())
});
self.register_host_primitive("WORD_NAME", false, func)?;

Special interpreter token (defining words like VARIABLE, CONSTANT, CREATE): Handle in interpret_token_immediate() or compile_token() as a special case.

Code Style

cargo fmt --all and cargo clippy --workspace must pass with no warnings
Every public function needs a doc comment
Use thiserror for error types in core crate, anyhow for CLI
Prefer returning Result over panicking

Testing

Run cargo test --workspace before committing (currently 611 unit + 1 benchmark + 12 compliance + 9 comparison + 5 crypto)
Forth 2012 compliance: cargo test -p wafer-core --test compliance
Cross-engine comparison (vs gforth): cargo test -p wafer-core --test comparison
Performance benchmarks (release mode): cargo test -p wafer-core --test comparison -- --nocapture --ignored
Test helper in outer.rs: eval_output("forth code") returns printed output as String
Test helper: eval_stack("forth code") returns data stack as Vec

Web REPL

Build: cd crates/web && wasm-pack build --target web --out-dir www/pkg
Serve: python3 -m http.server -d crates/web/www 8080
Open: http://localhost:8080/
Dev build (faster, unoptimized): wasm-pack build --target web --dev --out-dir www/pkg

Key Principles

Correctness first, performance second
Maximize Forth, minimize Rust (self-hosting goal -- not yet started)
Test-driven: if it's not tested, it doesn't work
Every word set at 100% compliance before moving to the next
Never break existing tests

5.3 KiB Raw Blame History