26 Commits

Author SHA1 Message Date
Oleksandr Kozachuk 20b8754e27 chore(release): v0.2.1 — changelog + version bump
CI / check (push) Has been cancelled
2026-08-06 16:05:02 +02:00
Oleksandr Kozachuk 17852ed459 fix(core): search order is authoritative; host words validate stack args
- Dictionary::find no longer falls back to the newest entry across all
  wordlists when the search order has no match (Forth 2012 16.3.3;
  gforth and SwiftForth agree). Cross-engine corpus program guards it.
- ~40 argument-taking host words (RND-SEED, ACCEPT, RESIZE, ALLOCATE,
  SEARCH, SUBSTITUTE, ROLL, M*, UM/MOD, SF@/SF!/DF@/DF!, F./FE./FS./F~,
  2R@, ...) popped or read stack cells with no underflow check; on an
  empty stack the pointer silently drifted past its base. New host_need/
  host_fneed/host_fpop checked helpers; class-wide regression test
  drives every word on an empty stack.
2026-08-06 16:04:58 +02:00
Oleksandr Kozachuk e6eabb098d Merge pull request #1 from ok2/usability
CI / check (push) Has been cancelled
v0.2.0 — usability release: guards, SEE/HELP, INCLUDE, error overhaul
2026-08-06 12:48:00 +02:00
Oleksandr Kozachuk f8da87187f chore(release): v0.2.0 — changelog, version bump, dependency upgrades 2026-08-06 12:44:37 +02:00
Oleksandr Kozachuk 0645734d94 chore(tools): sync bat syntax with current word set
Alternations diffed against live WORDS output (304 words) plus
outer-interpreter tokens. Adds float transcendentals, double-cell
ops, pictured numeric, conditional compilation, string ops,
SEE/SEE-IR/DUMP/BYE/HELP, RP@/RDEPTH/.RS, REMEMBER/EMPTY/GILD,
DECIMAL/HEX, INCLUDE/INCLUDED, WITHIN, DEFER!/DEFER@, C,.
2026-08-06 12:44:36 +02:00
Oleksandr Kozachuk f83c8f25e4 build: add just install recipe
Installs bat Forth syntax, then cargo install --locked the CLI.
CARGO_PROFILE_RELEASE_STRIP=none because Cargo's release default
(strip = "debuginfo") emits dylibs macOS 27 dyld rejects with
"mis-aligned LINKEDIT string pool"; proc macros then fail to load
during the build.
2026-08-06 12:44:34 +02:00
Oleksandr Kozachuk 9b1cc0cace feat(core): INCLUDE, error overhaul, sf64 lane, WORDS ALL, .RS
WS-012 -- INCLUDE/INCLUDED:
- Injected source loader (core stays IO-free: CLI installs a
  filesystem reader, web leaves it unset -> defined error). Recursive
  include_file feeds files line-by-line through evaluate, so compile
  state and SEE capture span lines for free. Cycle detection, depth
  cap 16, paths relative to the including file, SOURCE-ID per nesting
  level, parent input restored on success/error/BYE.
- CLI file mode now runs through the include machinery: `wafer x.fth`
  gets file:line error context and a base dir for nested INCLUDEs.
- Unlocks the REMEMBER+INCLUDE reload loop.

WS-008 -- error reporting remainder:
- Errors inside included files carry `file.fth:12:` context
  (anyhow context chain; CLI prints {e:#}).
- describe_uncaught now returns typed WaferError::UncaughtThrow
  { code, message } -- display text unchanged, THROW code reachable
  via downcast for CLI/web consumers.
- compile_word emits a WASM name section; wasmtime trap backtraces
  name the faulting word and runtime_native prefixes "in <WORD>:".
  Batch/consolidated modules stay unnamed (no name plumbing there;
  boot primitives rarely trap).

WS-003 -- SwiftForth correctness lane:
- compare_all_programs_sf64 runs the program corpus with sf64 as
  oracle; whitespace-token comparison (sf64 prints numbers
  space-prefixed and echoes piped lines). 34/35 parity; dot-quote
  skipped (interpret-mode ." is a SwiftForth no-op). #[ignore]d like
  the gforth lane; `just compare-correctness` runs both.

WS-011 leftovers:
- WORDS ALL: grouped full view -- one section per wordlist (search
  order first), then internal words, each with counts. Backed by
  Dictionary::visible_entries (name, wid, internal); visible_words
  now derives from it.
- .RS / RDEPTH: return-stack introspection in boot.fth over a new
  RP@ primitive (IrOp::RpFetch); BEGIN/WHILE walk so the walk never
  touches the stack it prints. SPACES clamped per 6.1.2230.

549 unit + 11 compliance + 9(+2) comparison + 5 crypto + 1 bench
green; fmt/clippy clean; --no-default-features and wasm32 web checks
pass.
2026-08-06 12:44:33 +02:00
Oleksandr Kozachuk dc6e0d45e1 feat(core): SEE, SEE-IR, HELP introspection trio (WS-010)
Implements plans/01-see-introspection.md, all phases.

- see.rs: feature-free IR pretty-printer (format_ir/format_ir_with),
  exhaustive over IrOp -- a new variant fails the build, not the output.
- SEE-IR <name>: post-optimization IR view with resolved callee names,
  immediate/does> annotations; host-word and interpreter-token stubs.
- SEE <name>: verbatim source capture for colon words (multi-line,
  comments preserved, EVALUATE-nesting safe, error-path wiped, MARKER/
  REMEMBER/EMPTY roll word sources back too). Data definers (VARIABLE/
  CONSTANT/CREATE/BUFFER:/2*/F*/SYNONYM) record synthesized one-liners
  at definition time; VALUE/2VALUE/FVALUE/DEFER synthesize at SEE time
  so current values and IS targets show. Fallback chain ends at IR dump
  or host-word stub -- SEE never dead-ends on a defined word.
- HELP [<name>]: wordhelp.rs doc table with stack effect + one-line
  description for EVERY word in a fresh VM (300+ dictionary words plus
  all outer-interpreter tokens); a coverage test fails the build if a
  word is ever added undocumented. User words echo their leading
  ( ... -- ... ) comment. SEE/SEE-IR prepend the HELP line as a
  \ comment. Bare HELP prints usage.
- boot.fth colon definitions get real sources for free (they flow
  through evaluate); INTERPRETER_TOKENS gained the missing ?DO.

524 unit + 11 compliance + 9 comparison + 5 crypto + 1 bench green;
fmt/clippy clean; core still builds --no-default-features; web
wasm-pack build unchanged.
2026-08-06 12:44:32 +02:00
Oleksandr Kozachuk cda296aab5 feat(codegen): stack under/overflow guards in compiled words (WS-007)
Compiled code could silently move dsp/rsp/fsp out of their stack
regions (e.g. DROP on an empty stack), corrupting later pushes with
no diagnostic -- the addresses stay inside valid linear memory, so
nothing could trap. Host-side checks cannot catch it.

- Guards are emitted at the sp-adjustment choke points (dsp_inc/
  dsp_dec, fsp_inc/fsp_dec, rpush/rpop/rpeek, peek, TwoDup/TwoDrop,
  promoted prologue/epilogue -- DROP never loads its value, so
  guarding pop() alone is not enough). On fault: write the code to
  SYSVAR_FAULT_CODE, call _STACK_FAULT_, which THROWs it -- so
  guards are CATCHable and print standard messages (-3/-4/-5/-6/
  -44/-45).
- The batch/consolidated compile path (all boot primitives) and the
  export path are wired too; a thread-local carries the fault index
  into the shared emission helpers.
- Config: codegen.stack_guards, default ON. `wafer build` output
  defaults OFF (production artifact); WAFER_STACK_GUARDS=0|1
  overrides either. Perf comparison lanes run unguarded.
- Measured overhead in release loops: within noise (never-taken
  branches).
- toolstest.fth baseline 37 -> 38: line 368's bare interpreted `R>`
  used to underflow silently and count as passing; the guard now
  correctly reports -6.
2026-08-05 17:00:22 +02:00
Oleksandr Kozachuk e31407ab58 feat(core): REMEMBER + EMPTY/GILD; marker rollback covers namespace state
- REMEMBER <name>: SwiftForth-style re-runnable marker -- restores
  to just AFTER its own definition and survives execution. The
  edit-reload-test loop: REMEMBER fresh ... fresh ... fresh.
- EMPTY rolls back to the boot dictionary; GILD re-baselines EMPTY
  to the current state. Baseline captured at VM construction.
- MarkerState now also snapshots search order, wid allocation,
  compilation wordlist, REPLACES table, and ABORT" texts; restore
  discards marker entries newer than the snapshot (was: newer than
  the executing marker id only).
- Shared snapshot_marker_state/apply_marker_state used by MARKER,
  REMEMBER, EMPTY, and GILD.

Known ambiguity (standard-conformant): a DEFER defined before a
marker but retargeted at a word defined after it dangles after
rollback; stale function-table slots are name-unreachable and get
overwritten by later definitions.
2026-08-05 16:22:47 +02:00
Oleksandr Kozachuk 2910884b83 fix(repl): multi-line output starts on its own line, inline ok only for single-line 2026-08-05 16:07:10 +02:00
Oleksandr Kozachuk f584066a0a feat(repl): usability batch - errors, WORDS, .S, BYE, DUMP, history
Core:
- Uncaught THROW prints its standard message ("Stack underflow
  (throw -4)"; unknown codes as "Catch = <n>") instead of the
  "forth-throw" sentinel. ABORT" text is carried as a structured
  payload and shown only when the -2 throw goes uncaught -- CATCH
  stays silent and the payload cannot go stale. ABORT throws -1
  through the same path.
- WORDS: optional same-line substring filter (WORDS FDEPTH), skips
  internal words (new INTERNAL header flag, set at create for
  underscore-prefixed names), wraps at 78 columns, prints a count.
  ORDER names wids (FORTH / wid#N) instead of Rust debug output.
- .S honors BASE. New: F.S (float stack), DUMP (hex+ASCII,
  bounds-checked, 4K cap), ? (fetch-and-print, boot.fth). BYE is a
  real word now: sets a VM flag the driver honors (exits REPL,
  stops rest of line/file).

CLI:
- Persistent history (~/.local/state/wafer/history, 0600 perms,
  $WAFER_HISTORY override), Tab completion over the live dictionary
  (snapshot refreshed after each line), Up/Down do prefix history
  search, Ctrl-C clears the line instead of exiting.

Web:
- History survives reloads (localStorage, cap 200, dedup, init-code
  runs excluded), User Words palette populated via new words()
  export, stack bar annotates non-decimal BASE, base() reads the
  real BASE sysvar instead of returning a hardcoded 10.
2026-08-05 14:57:12 +02:00
Oleksandr Kozachuk 4980648982 feat(bench): SwiftForth sf64 lane in cross-engine performance report
CI / check (push) Has been cancelled
sf64 discovery + stdin runner (no -e flag; input lines truncate at
~256 chars, so one statement per line), ucounter-based µs timing —
same wrapper shape as gforth utime. New sf64 + WAFER/sf columns,
informational only (no regression limit). Justfile: bench-compare
target; CARGO_PROFILE_RELEASE_STRIP=none for Darwin 27 dlopen bug.
2026-08-04 17:07:47 +02:00
Oleksandr Kozachuk 31dc6c6397 fix(core): no SystemTime on wasm32 — fixed boot seed + UTIME Forth error
CI / check (push) Has been cancelled
2026-07-29 16:56:31 +02:00
Oleksandr Kozachuk 35b78193fd feat(boot): add -ROT <= >= gforth extensions
CI / check (push) Has been cancelled
Non-standard but ubiquitous words; absence aborted otherwise-valid
gforth programs with unknown-word errors.
2026-07-18 15:40:39 +02:00
ok2 d5acdc0e7b fix: Rust 1.95 clippy — match guards + map_or
CI / check (push) Has been cancelled
Rust 1.95 promoted collapsible_match and map_unwrap_or; CI runs
-D warnings so they break the build. Collapse nested `if`s into
match guards across codegen/optimizer/export, and swap
map().unwrap_or(..) for map_or / is_ok_and.
2026-04-21 17:00:21 +02:00
ok2 a66435c93c bat syntax: sync with (LOCAL), quotations, structures, hashes
CI / check (push) Has been cancelled
The syntax file landed in bcccdfb, one commit before `(LOCAL)` and
while several other recently-added words were already in the tree but
unhighlighted. Extend it to cover everything currently registered.

Added contexts:
- `locals` — `{:` `:}` `{F:` `TO` `LOCALS|` `END-LOCALS` `(LOCAL)`.
- `structures` — `BEGIN-STRUCTURE` (captures the following name),
  `END-STRUCTURE`, `+FIELD`, `FIELD:`, `CFIELD:`, `FFIELD:`,
  `SFFIELD:`, `DFFIELD:`.
- `hashing` — `SHA1`, `SHA256`, `SHA512`. Comment notes the list
  mirrors `crypto::ALGOS`.

Extended:
- `definitions` — quotations `[:` / `;]` (Core-ext 6.2.0455).
- `parsing` — state-smart `S` (the string parser from d1a7d55).
- `wafer_extras` — `READ-PASSWORD` (web-side prompter from 9150696).

Context order in `main:` keeps `definitions` ahead of `locals`, so
`: foo` still wins over `{:` / `:}`, and `strings` / `arithmetic`
stay ahead of `parsing` so `S"` and `S>D` keep their existing
highlighting despite the new bare-`S` rule.
2026-04-20 12:40:31 +02:00
ok2 bb217714ac Add (LOCAL) per Forth 2012 §13.6.1.0086
Implement `(LOCAL)` as a host primitive that defers its effect to the
outer-interpreter compile state via two new `PendingAction` variants:

  - `DeclareLocal(name)` — a non-sentinel `(LOCAL)` call with `u > 0`
    appends the name to `compiling_locals` as an int local.
  - `DeclareLocalEnd` — the `0 0 (LOCAL)` sentinel emits reverse-order
    `ForthLocalSet` IR for the batch declared since the last sentinel,
    reusing the same IR shape as the `{: ... :}` locals flow.

`local_batch_base` tracks where the current batch started; it is
saved/restored across nested compile frames and cleared on
`finish_colon_def`. Int-only, per spec — float locals remain `{F: :}`.

Also fix `\` per §6.2.2535: parse-and-discard must stop at the next
`\n`, not at `#TIB`. Under line-wrapped `evaluate` calls (common in
test files) the old behaviour consumed the trailing `;` of a multi-line
`:` definition, silently leaving state in compile mode.

Tighten `compliance.rs`: `load_file` now returns a line-failure count,
every prerequisite is asserted against `expected_load_failures(path)`,
and a new `load_file_whole` handles multi-line definitions (`DOES>`
split across lines in `errorreport.fth`) that the per-line loader
cannot stitch. Baselines document known gaps for `core.fr` (nested
`:`, SOURCE/>IN via EVALUATE), `coreexttest.fth` (SAVE-INPUT, `.(`
inside `[...]`), `exceptiontest.fth` (one garbled parse after
CATCH/THROW source stacking), and `toolstest.fth` (37 `\?`-guarded
lines where `SOURCE >IN ! DROP` fails to skip under per-line
`evaluate`). Each entry is a tech-debt ledger item, not an allowlist.

Regression tests: LT32 (the localstest case that silently skipped
before `(LOCAL)` existed), the `0 0 (LOCAL)` sentinel-only no-op, a
multi-line `:` followed by `VARIABLE` after a `\` comment, and a
direct `\` stops-at-newline case.

Incidental: clear two `implicit_clone` clippy lints in the RANDOM
determinism test (`.to_vec()` → `.clone()`).
2026-04-18 17:12:02 +02:00
ok2 67448caa9c chore: clear pre-existing clippy + fmt in crypto tests
Fix rustfmt drift and two clippy lints (`doc_markdown` missing
backticks around `NativeRuntime`) that surfaced after the Rust 1.94
toolchain update. No functional change.
2026-04-18 17:11:28 +02:00
ok2 bcccdfb49d Add bat syntax for WAFER / Forth 2012
Ship tools/editor-support/bat/WAFER.sublime-syntax so any bat user
(including oked, which probes bat first) renders .fth files with
proper keyword colouring, including the WAFER extras CONSOLIDATE,
RANDOM, RND-SEED, and UTIME.

Keyword list derives from register_primitive/register_host_primitive
calls in crates/core/src/outer.rs plus the boot.fth definitions.
Internal underscore-prefixed words are deliberately omitted.

Install with `just install-syntax`.
2026-04-17 11:22:14 +02:00
ok2 be5dff243f fix: locals beat hardcoded tokens in compile_token
compile_token matched hardcoded tokens (S, ." etc) before
checking compiling_locals. Local named `s` got hijacked by
the `S` string shortcut. Forth 2012 §13.3.3.2 — locals
supersede dict names in scope. Move locals check to top of
compile_token for uniform precedence.

Tests: S-hijack repro, get+set round-trip, int-uninit pipe
syntax coverage (`{: | name :}`).
2026-04-17 10:40:19 +02:00
ok2 49582f7e86 docs: rewrite architecture.txt + fix mem offsets
architecture.txt drifted from code: missing HASH_SCRATCH region,
runtime-trait box, wordlists/search-order, codegen locals layout,
F: locals, quotations, crypto. Rewrite from current source.

memory.rs `// 0x...` annotations were the drift source — RETURN
/ FLOAT / HASH / DICT bases printed values disagreeing with the
const arithmetic. Recompute and correct.
2026-04-16 20:51:12 +02:00
ok2 1a8f27b5bd Add F: float locals (gforth/SwiftForth-style)
`{: F: x F: y :}` now declares float-typed locals that live on the float
stack. `x x F* y y F* F+ FSQRT` writes real float code without manual
FSTACK juggling — previously WAFER had a 100%-compliant float wordset
but no way to name intermediate float values.

New IR ops `ForthFLocalGet(n)` / `ForthFLocalSet(n)` alongside the
existing int-local ops. Each kind has its own index namespace so mixed
declarations like `{: n F: f :}` compose cleanly. Codegen allocates f64
WASM locals after the existing f64 scratch pair; the fsp-bridge logic
mirrors the existing FDup/FSwap path.

Outer interpreter tracks a parallel `compiling_local_kinds` alongside
`compiling_locals` (keeps the 18 existing touch-points unchanged) and
extends `{:` to recognize `F:` as a per-next-name type marker. `TO` and
name resolution branch on kind to pick Int vs Float get/set ops.

Four tests: classic hypot, TO round-trip, mixed int/float args, and
uninitialized float via `|`. Inline-inhibit for the new ops added to
optimizer and is_promotable so they don't sneak into contexts that
would collide with the caller's WASM locals.
2026-04-15 21:29:01 +02:00
ok2 6771f5d46b Add quotations [: ... ;] (Forth 2012 Core-ext 6.2.0455)
State-smart anonymous xt builder. Interpret mode leaves the xt on the
data stack; compile mode emits a literal push into the enclosing word,
so `: APPLY EXECUTE ;  [: 1 2 + ;] APPLY` prints 3.

Supported nested inside colon definitions via a new compile-frame stack
(`Vec<CompileFrame>`). Each frame snapshots `compiling_name`,
`compiling_word_id`, `compiling_word_addr`, `compiling_ir`,
`control_stack`, `saw_create_in_def`, `compiling_locals`, and `state`.
The inner [: ... ;] compiles its body as an anonymous word; on ;] the
outer frame pops back and the xt is either pushed to the data stack
(interpret mode) or compiled as a literal (compile mode).

Also fixes a latent bug: `finish_colon_def` used to reveal `latest`,
which breaks when intermediate dict entries (now including quotations)
move `latest`. Each definition now tracks its own `compiling_word_addr`
and uses `reveal_at`, matching the existing DOES> pattern.

Five tests cover interpret, compile, inside-a-colon-def, two-level
nesting, and the control-stack-travels-with-frame regression (outer
IF/ELSE/THEN must still match around an inner [: ;]).
2026-04-15 21:18:02 +02:00
ok2 64f4b1e857 boot: add structure words (Facility-ext 10.6.2.0935)
BEGIN-STRUCTURE, END-STRUCTURE, +FIELD, FIELD:, CFIELD:, FFIELD:,
SFFIELD:, DFFIELD: — the Forth 2012 structure-definition family plus
the float-typed variants for symmetry with WAFER's float wordset.

Each defining word carries its own inline CREATE .. DOES> — factoring
through a shared +FIELD helper doesn't work in WAFER, because DOES>-
defining words only dispatch at the outer interpreter, not from compiled
IR. So FIELD: can't call +FIELD and have the DOES> action fire; each
FIELD:/CFIELD:/... repeats the pattern directly.

Three tests cover size computation, field offsets, and mixed cell + char
fields with alignment.
2026-04-15 20:50:29 +02:00
ok2 f1752ededa Add RANDOM / RND-SEED — xorshift64 PRNG
Non-standard but ubiquitous in gforth/SwiftForth/VFX. Adds a shared
rng_state on ForthVM, seeded from nanosecond wall-clock at boot.
`RANDOM ( -- u )` returns a 32-bit pseudo-random cell; `RND-SEED ( u -- )`
reseeds, with 0 forced to a nonzero constant to avoid xorshift's fixed
point.

Three tests cover determinism after seeding, distinct-value spread
across 1000 pulls, and the zero-seed safeguard.
2026-04-15 20:31:48 +02:00
34 changed files with 6190 additions and 1343 deletions
+3
View File
@@ -3,3 +3,6 @@
*.swp
.DS_Store
*.bk
# Local planning notes — never tracked
/plans/
+109
View File
@@ -0,0 +1,109 @@
# Changelog
All notable changes to WAFER are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.2.1] - 2026-08-06
### Fixed
- **The search order is now authoritative** (Forth 2012 §16.3.3): a word
whose wordlist is not in the search order is no longer findable.
Previously lookup fell back to the newest entry across all wordlists,
making word hiding impossible. Verified against gforth and SwiftForth,
and guarded by a cross-engine corpus program.
- **Host words validate their stack arguments.** Around 40 host-implemented
words (`RND-SEED`, `ACCEPT`, `RESIZE`, `ALLOCATE`, `FREE`, `SEARCH`,
`SUBSTITUTE`, `ROLL`, `M*`, `UM/MOD`, `SF@ SF! DF@ DF!`, `F. FE. FS. F~`,
`2R@`, and friends) performed raw stack-pointer arithmetic with no
underflow check — calling them on an empty stack silently corrupted the
stack pointer (the compiled-code guards from 0.2.0 do not cover host
words). All argument-taking host words now fail with a clean, CATCHable
underflow error, enforced by a class-wide regression test.
## [0.2.0] - 2026-08-06
The usability release: introspection, source files, honest errors, and a
safety net under every compiled word.
### Added
- **Stack guards in compiled code**: under/overflow checks at the
stack-pointer choke points of generated WASM. Faults THROW standard codes
(`-3`..`-6`, `-44`, `-45`), are CATCHable, and print standard messages
instead of silently corrupting memory. Default on; `wafer build` output
stays unguarded; `WAFER_STACK_GUARDS=0|1` overrides.
- **`SEE`**: source-level decompiler. Colon words (including everything in
`boot.fth`) show their captured verbatim source; data words show
synthesized definitions with current values (`9 VALUE X`,
`DEFER D ( IS DUP )`); primitives fall back to a readable IR dump —
`SEE` never dead-ends on a defined word.
- **`SEE-IR`**: post-optimization IR view with resolved callee names and
indented control flow — shows what the optimizer actually did.
- **`HELP`**: stack effect + one-line description for **every** word in a
fresh VM (dictionary words and outer-interpreter tokens alike); coverage
is enforced by a unit test, so an undocumented new word fails the build.
User words echo their leading `( n -- n )` comment.
- **`INCLUDE` / `INCLUDED`**: nestable source-file loading with cycle
detection, depth bound, paths relative to the including file, and
per-level `SOURCE-ID`. The loader is injected (CLI: filesystem; web:
defined error), so the core stays IO-free. `wafer prog.fth` now runs
through the same machinery.
- **`MARKER` extensions**: `REMEMBER` (re-runnable marker), `EMPTY` and
`GILD` (boot-state rollback and re-baselining). Marker rollback now also
restores search order, wordlists, `REPLACES` substitutions, `ABORT"`
texts, and captured word sources — enabling the `REMEMBER` + `INCLUDE`
edit-reload loop.
- **`WORDS`**: optional substring filter (`WORDS FLOAT`), word count, and
`WORDS ALL` — a grouped full view by wordlist plus internal words.
- **Return-stack introspection**: `.RS`, `RDEPTH`, `RP@`.
- **Tools**: `.S` honors `BASE`, `F.S`, `?`, bounds-checked `DUMP`, real
`BYE`, named `ORDER` output.
- **CLI REPL**: persistent history (XDG state dir, `0600`), dictionary-backed
tab completion, prefix history search on Up/Down, Ctrl-C clears the line.
- **Web REPL**: history persisted to localStorage, User Words palette,
`BASE` indicator in the stack bar.
- **Error reporting**: uncaught `THROW` codes map to standard messages;
`ABORT"` text prints only when uncaught; errors inside included files
carry `file.fth:line:` context; uncaught throws are typed
(`WaferError::UncaughtThrow`) for embedding consumers; compiled words
carry WASM name sections, so genuine traps name the faulting word
(`in CRASHER: wasm trap: out of bounds memory access`).
- **SwiftForth correctness lane**: the cross-engine program corpus can run
against sf64 as an oracle (`just compare-correctness`), alongside the
existing gforth lane and the sf64 performance lane.
### Fixed
- Multi-line command output in the CLI REPL starts on its own line
(inline ` ok` echo only for single-line output).
- `.S` printed in decimal regardless of `BASE`.
- A bare interpreted `R>` underflowed silently (exposed by the new stack
guards; compliance baseline updated).
- `SPACES` with a negative count now outputs nothing, per Forth 2012
6.1.2230.
### Changed
- `wafer prog.fth` reports errors with `file:line` context and resolves
nested `INCLUDE`s relative to the file.
- Internal words (`_`-prefixed) are flagged in the dictionary and hidden
from `WORDS` and completion (`WORDS ALL` shows them).
- Dependencies upgraded across the board: wasmtime 43 → 47,
wasm-encoder/wasmparser 0.246 → 0.255, plus all semver-compatible
updates.
## [0.1.0] - 2026-08-04
Initial development line (untagged): Forth 2012 core with IR optimizer and
WASM codegen via wasm-encoder/wasmtime, ~300 words across Core, Double,
Float, String, Search-Order, Exception, and Tools word sets, Forth 2012
compliance suite, `CONSOLIDATE` whole-program recompilation, `wafer build`
AOT export (WASM / native / JS loader), browser REPL, SHA-1/256/512 words,
and cross-engine benchmark lanes against gforth and SwiftForth.
[0.2.1]: https://github.com/ok2/wafer/compare/v0.2.0...v0.2.1
[0.2.0]: https://github.com/ok2/wafer/compare/v0.1.0...v0.2.0
[0.1.0]: https://github.com/ok2/wafer/releases/tag/v0.1.0
+1 -1
View File
@@ -79,7 +79,7 @@ Handle in `interpret_token_immediate()` or `compile_token()` as a special case.
## Testing
- Run `cargo test --workspace` before committing (currently 431 unit + 1 benchmark + 11 compliance + 9 comparison)
- Run `cargo test --workspace` before committing (currently 542 unit + 1 benchmark + 11 compliance + 9 comparison + 5 crypto)
- Forth 2012 compliance: `cargo test -p wafer-core --test compliance`
- Cross-engine comparison (vs gforth): `cargo test -p wafer-core --test comparison`
- Performance benchmarks (release mode): `cargo test -p wafer-core --test comparison -- --nocapture --ignored`
Generated
+309 -568
View File
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -3,7 +3,7 @@ members = ["crates/*"]
resolver = "2"
[workspace.package]
version = "0.1.0"
version = "0.2.1"
edition = "2024"
license = "MIT OR Apache-2.0"
repository = "https://github.com/ok2/wafer"
@@ -41,9 +41,9 @@ needless_collect = "warn"
or_fun_call = "warn"
[workspace.dependencies]
wasm-encoder = "0.246"
wasmparser = "0.246"
wasmtime = "43"
wasm-encoder = "0.255"
wasmparser = "0.255"
wasmtime = "47"
anyhow = "1"
thiserror = "2"
proptest = "1"
+21
View File
@@ -43,6 +43,14 @@ bench:
bench-opts:
cargo test -p wafer-core --test benchmark_report -- --nocapture --ignored
# Cross-engine performance report: WAFER vs gforth vs SwiftForth (sf64)
bench-compare:
CARGO_PROFILE_RELEASE_STRIP=none cargo test -p wafer-core --release --test comparison -- --nocapture --ignored performance_report
# Cross-engine correctness lanes: program corpus vs gforth + sf64 oracles
compare-correctness:
cargo test -p wafer-core --test comparison -- --nocapture --ignored compare_all_programs
# Check dependency licenses and advisories
deny:
cargo deny check
@@ -57,3 +65,16 @@ ci: fmt clippy deny test
# Check compilation without running
check:
cargo check --workspace
# Install the wafer CLI (release build) and bat syntax highlighting.
# STRIP=none: Cargo's release default (strip = "debuginfo") emits dylibs that
# macOS 27's dyld rejects ("mis-aligned LINKEDIT string pool"), so proc macros
# fail to load during the build itself.
install: install-syntax
CARGO_PROFILE_RELEASE_STRIP=none cargo install --path crates/cli --locked
# Install bat syntax highlighting for WAFER / Forth
install-syntax:
mkdir -p ~/.config/bat/syntaxes
cp tools/editor-support/bat/WAFER.sublime-syntax ~/.config/bat/syntaxes/
bat cache --build
+2 -1
View File
@@ -95,7 +95,7 @@ Times in microseconds. WAFER/gf < 1.0 means WAFER is faster. CONSOL = after `CON
## Testing
```bash
# All tests (~450 currently passing)
# All tests (~570 currently passing)
cargo test --workspace
# Forth 2012 compliance suite
@@ -185,6 +185,7 @@ Over 200 words are implemented across the following categories:
| Strings | `COMPARE SEARCH SLITERAL REPLACES SUBSTITUTE UNESCAPE` |
| Floating-Pt | `F+ F- F* F/ FABS FNEGATE FSQRT FSIN FCOS FTAN FEXP FLOG FMIN FMAX` and 55+ more |
| Case | `CASE OF ENDOF ENDCASE` |
| Tools | `WORDS SEE SEE-IR HELP INCLUDE INCLUDED .S F.S ? DUMP MARKER REMEMBER EMPTY GILD BYE` |
## Web REPL
+1 -1
View File
@@ -9,7 +9,7 @@ license.workspace = true
workspace = true
[dependencies]
wafer-core = { path = "../core", version = "0.1.0" }
wafer-core = { path = "../core", version = "0.2.1" }
wasmtime = { workspace = true }
anyhow = { workspace = true }
clap = { version = "4", features = ["derive"] }
+190 -58
View File
@@ -137,7 +137,9 @@ fn cmd_build(
) -> anyhow::Result<()> {
let source = std::fs::read_to_string(file)?;
let mut vm = ForthVM::<NativeRuntime>::new()?;
// Exported modules are production artifacts: no stack guards by default
let mut vm = ForthVM::<NativeRuntime>::new_with_config(vm_config(false))?;
vm.set_source_loader(fs_loader());
vm.set_recording(true);
vm.evaluate(&source)?;
@@ -260,18 +262,38 @@ fn cmd_run(file: &str) -> anyhow::Result<()> {
Ok(())
}
/// `WaferConfig` for CLI-created VMs. `WAFER_STACK_GUARDS=0|1` overrides
/// the per-command default (REPL/file execution on, build off).
fn vm_config(default_guards: bool) -> wafer_core::config::WaferConfig {
let mut cfg = wafer_core::config::WaferConfig::all();
cfg.codegen.stack_guards = match std::env::var("WAFER_STACK_GUARDS").ok().as_deref() {
Some("0") => false,
Some(_) => true,
None => default_guards,
};
cfg
}
/// Filesystem source loader for INCLUDE/INCLUDED.
fn fs_loader() -> Box<dyn Fn(&str) -> anyhow::Result<String> + Send + Sync> {
Box::new(|path| Ok(std::fs::read_to_string(path)?))
}
/// `wafer` (REPL) or `wafer program.fth` (evaluate and exit)
fn cmd_eval_or_repl(file: Option<&str>) -> anyhow::Result<()> {
let mut vm = ForthVM::<NativeRuntime>::new()?;
let mut vm = ForthVM::<NativeRuntime>::new_with_config(vm_config(true))?;
vm.set_source_loader(fs_loader());
match file {
Some(file) => {
let source = std::fs::read_to_string(file)?;
vm.evaluate(&source)?;
// Through the include machinery: file:line error context and a
// base directory for nested INCLUDEs.
let result = vm.include(file);
let output = vm.take_output();
if !output.is_empty() {
print!("{output}");
}
result?;
}
None => {
if !stdin_is_tty() {
@@ -285,66 +307,17 @@ fn cmd_eval_or_repl(file: Option<&str>) -> anyhow::Result<()> {
if !output.is_empty() {
print!("{output}");
}
if vm.bye_requested() {
break;
}
}
Err(e) => {
eprintln!("Error: {e}");
eprintln!("Error: {e:#}");
}
}
}
} else {
// Interactive REPL
println!(
"WAFER v{} - WebAssembly Forth Engine in Rust",
env!("CARGO_PKG_VERSION")
);
println!("Type BYE to exit.");
let mut rl = rustyline::DefaultEditor::new()?;
loop {
let prompt = if vm.is_compiling() { " ] " } else { "> " };
match rl.readline(prompt) {
Ok(line) => {
let trimmed = line.trim();
if trimmed.eq_ignore_ascii_case("BYE") {
break;
}
let _ = rl.add_history_entry(&line);
match vm.evaluate(&line) {
Ok(()) => {
let output = vm.take_output();
// PAGE (form feed) clears the terminal
if output.contains('\x0C') {
print!("\x1b[2J\x1b[H");
}
let output = output.replace('\x0C', "");
if !vm.is_compiling() {
// Move cursor back up to end of input line so
// output appears inline, like traditional Forth:
// > 2 2 + . 4 ok
let col = prompt.len() + line.len() + 1;
print!("\x1b[A\x1b[{col}G {output} ok");
println!();
} else if !output.is_empty() {
print!("{output}");
}
}
Err(e) => {
eprintln!("Error: {e}");
}
}
}
Err(
rustyline::error::ReadlineError::Interrupted
| rustyline::error::ReadlineError::Eof,
) => {
break;
}
Err(e) => {
eprintln!("Readline error: {e}");
break;
}
}
}
run_repl(&mut vm)?;
}
}
}
@@ -357,3 +330,162 @@ fn stdin_is_tty() -> bool {
use std::io::IsTerminal;
std::io::stdin().is_terminal()
}
/// Completes the token under the cursor against the live dictionary.
struct WaferHelper {
words: Vec<String>,
}
impl rustyline::completion::Completer for WaferHelper {
type Candidate = String;
fn complete(
&self,
line: &str,
pos: usize,
_ctx: &rustyline::Context<'_>,
) -> rustyline::Result<(usize, Vec<String>)> {
let start = line[..pos]
.rfind(|c: char| c.is_whitespace())
.map_or(0, |i| i + 1);
let prefix = line[start..pos].to_ascii_uppercase();
let mut matches: Vec<String> = self
.words
.iter()
.filter(|w| w.to_ascii_uppercase().starts_with(&prefix))
.cloned()
.collect();
matches.sort();
matches.dedup();
Ok((start, matches))
}
}
impl rustyline::hint::Hinter for WaferHelper {
type Hint = String;
}
impl rustyline::highlight::Highlighter for WaferHelper {}
impl rustyline::validate::Validator for WaferHelper {}
impl rustyline::Helper for WaferHelper {}
/// History file: `$WAFER_HISTORY`, else `$XDG_STATE_HOME/wafer/history`,
/// else `~/.local/state/wafer/history`.
fn history_path() -> Option<std::path::PathBuf> {
if let Some(p) = std::env::var_os("WAFER_HISTORY") {
return Some(p.into());
}
let base = std::env::var_os("XDG_STATE_HOME")
.map(std::path::PathBuf::from)
.or_else(|| {
std::env::var_os("HOME").map(|h| std::path::PathBuf::from(h).join(".local/state"))
})?;
Some(base.join("wafer/history"))
}
/// Interactive REPL: line editing, persistent history with prefix search
/// on Up/Down, and Tab completion over the live dictionary.
fn run_repl(vm: &mut ForthVM<NativeRuntime>) -> anyhow::Result<()> {
use rustyline::{Cmd, Editor, EventHandler, KeyCode, KeyEvent, Modifiers};
println!(
"WAFER v{} - WebAssembly Forth Engine in Rust",
env!("CARGO_PKG_VERSION")
);
println!("Type BYE to exit.");
let config = rustyline::Config::builder()
.completion_type(rustyline::CompletionType::List)
.history_ignore_dups(true)?
.build();
let mut rl: Editor<WaferHelper, rustyline::history::DefaultHistory> =
Editor::with_config(config)?;
rl.set_helper(Some(WaferHelper {
words: vm.word_names(),
}));
// Up/Down recall only entries starting with the typed prefix
rl.bind_sequence(
KeyEvent(KeyCode::Up, Modifiers::NONE),
EventHandler::Simple(Cmd::HistorySearchBackward),
);
rl.bind_sequence(
KeyEvent(KeyCode::Down, Modifiers::NONE),
EventHandler::Simple(Cmd::HistorySearchForward),
);
let history = history_path();
if let Some(path) = &history {
if let Some(dir) = path.parent() {
let _ = std::fs::create_dir_all(dir);
}
let _ = rl.load_history(path);
}
let save_history = |rl: &mut Editor<WaferHelper, rustyline::history::DefaultHistory>| {
if let Some(path) = &history {
let _ = rl.save_history(path);
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600));
}
}
};
loop {
let prompt = if vm.is_compiling() { " ] " } else { "> " };
match rl.readline(prompt) {
Ok(line) => {
let _ = rl.add_history_entry(&line);
match vm.evaluate(&line) {
Ok(()) => {
let output = vm.take_output();
if vm.bye_requested() {
break;
}
// PAGE (form feed) clears the terminal
if output.contains('\x0C') {
print!("\x1b[2J\x1b[H");
}
let output = output.replace('\x0C', "");
if !vm.is_compiling() {
if output.contains('\n') {
// Multi-line output (DUMP, WORDS, ...):
// print as a block, then ok on its own line
print!("{output}");
if !output.ends_with('\n') {
println!();
}
println!(" ok");
} else {
// Move cursor back up to end of input line so
// output appears inline, like traditional Forth:
// > 2 2 + . 4 ok
let col = prompt.len() + line.len() + 1;
print!("\x1b[A\x1b[{col}G {output} ok");
println!();
}
} else if !output.is_empty() {
print!("{output}");
}
}
Err(e) => {
eprintln!("Error: {e:#}");
}
}
// New definitions may have appeared: refresh completion
if let Some(h) = rl.helper_mut() {
h.words = vm.word_names();
}
save_history(&mut rl);
}
// Ctrl-C abandons the current line, Ctrl-D exits
Err(rustyline::error::ReadlineError::Interrupted) => {}
Err(rustyline::error::ReadlineError::Eof) => break,
Err(e) => {
eprintln!("Readline error: {e}");
break;
}
}
}
save_history(&mut rl);
Ok(())
}
+69 -2
View File
@@ -72,6 +72,19 @@
1-
REPEAT ;
\ ---------------------------------------------------------------
\ Common extensions (not in Forth 2012, gforth-compatible)
\ ---------------------------------------------------------------
\ -ROT ( x1 x2 x3 -- x3 x1 x2 ) rotate top item to third place
: -ROT ROT ROT ;
\ <= ( n1 n2 -- flag ) true if n1 <= n2 (signed)
: <= > 0= ;
\ >= ( n1 n2 -- flag ) true if n1 >= n2 (signed)
: >= < 0= ;
\ ---------------------------------------------------------------
\ Phase 2: Double-cell arithmetic
\ ---------------------------------------------------------------
@@ -184,8 +197,8 @@
\ TYPE ( c-addr u -- ) output u characters
: TYPE 0 ?DO DUP C@ EMIT 1+ LOOP DROP ;
\ SPACES ( n -- ) output n spaces
: SPACES 0 ?DO SPACE LOOP ;
\ SPACES ( n -- ) output n spaces (nothing for n <= 0, per 6.1.2230)
: SPACES 0 MAX 0 ?DO SPACE LOOP ;
\ Pictured numeric output constants
\ PICT_BUF_TOP = 0x05C0 = 1472, SYSVAR_HLD = 28
@@ -230,6 +243,9 @@
\ U. ( u -- ) print unsigned number and space
: U. 0 <# #S #> TYPE SPACE ;
\ ? ( a-addr -- ) fetch and print
: ? @ . ;
\ .R ( n width -- ) print right-justified signed number
: .R >R DUP ABS 0 <# #S ROT SIGN #> R> OVER - SPACES TYPE ;
@@ -242,6 +258,21 @@
\ D.R ( d width -- ) print right-justified signed double
: D.R >R SWAP OVER DABS <# #S ROT SIGN #> R> OVER - SPACES TYPE ;
\ ---------------------------------------------------------------
\ Return-stack introspection (debug aids)
\ ---------------------------------------------------------------
\ RDEPTH ( -- n ) number of cells on the return stack
\ RETURN_STACK_TOP = 9728 (0x2600). Only >R temps and loop params
\ live there; return addresses are on the WASM call stack.
: RDEPTH 9728 RP@ - 2 RSHIFT ;
\ .RS ( -- ) print the return stack bottom-to-top, like .S
\ Walks with BEGIN/WHILE (not DO) so the walk itself never pushes
\ onto the return stack it is printing.
: .RS ." R:<" RDEPTH 0 .R ." > "
9728 BEGIN DUP RP@ > WHILE 4 - DUP @ . REPEAT DROP ;
\ ---------------------------------------------------------------
\ Phase 6: DEFER support
\ ---------------------------------------------------------------
@@ -310,3 +341,39 @@
\ State-smart string literal for the next whitespace-delimited token.
\ Handled in Rust (outer.rs interpret_token_immediate / compile_token)
\ so the string survives REFILL in interpret mode.
\ ---------------------------------------------------------------
\ Structures (Forth 2012 Facility-ext 10.6.2.0935 family)
\ ---------------------------------------------------------------
\ Usage:
\ BEGIN-STRUCTURE POINT FIELD: P.X FIELD: P.Y END-STRUCTURE
\ CREATE ORIGIN POINT ALLOT
\ 1 ORIGIN P.X ! 2 ORIGIN P.Y !
\ Each defining word factored inline (CREATE .. DOES>). WAFER dispatches
\ DOES>-defining words only at the outer interpreter, so they can't be
\ factored through other compiled words (FIELD: -> +FIELD would no-op).
: BEGIN-STRUCTURE ( "name" -- struct-sys 0 )
CREATE HERE 0 0 , DOES> @ ;
: END-STRUCTURE ( struct-sys +n -- )
SWAP ! ;
: +FIELD ( n1 "name" n2 -- n3 )
CREATE OVER , + DOES> @ + ;
: FIELD: ( n1 "name" -- n2 )
CREATE ALIGNED DUP , 1 CELLS + DOES> @ + ;
: CFIELD: ( n1 "name" -- n2 )
CREATE DUP , 1 CHARS + DOES> @ + ;
: FFIELD: ( n1 "name" -- n2 )
CREATE FALIGNED DUP , 1 FLOATS + DOES> @ + ;
: SFFIELD: ( n1 "name" -- n2 )
CREATE SFALIGNED DUP , 1 SFLOATS + DOES> @ + ;
: DFFIELD: ( n1 "name" -- n2 )
CREATE DFALIGNED DUP , 1 DFLOATS + DOES> @ + ;
+230 -26
View File
@@ -19,7 +19,10 @@ use wasm_encoder::{
use crate::dictionary::WordId;
use crate::error::{WaferError, WaferResult};
use crate::ir::IrOp;
use crate::memory::{CELL_SIZE, SYSVAR_LEAVE_FLAG};
use crate::memory::{
CELL_SIZE, DATA_STACK_BASE, DATA_STACK_TOP, FLOAT_STACK_BASE, FLOAT_STACK_TOP,
RETURN_STACK_BASE, RETURN_STACK_TOP, SYSVAR_FAULT_CODE, SYSVAR_LEAVE_FLAG,
};
// ---------------------------------------------------------------------------
// Import indices (order matters: imports numbered sequentially by kind)
@@ -94,6 +97,9 @@ pub struct CodegenConfig {
pub table_size: u32,
/// Enable stack-to-local promotion for straight-line words.
pub stack_to_local_promotion: bool,
/// Table index of the `_STACK_FAULT_` host word; `Some` enables
/// stack under/overflow guards in the emitted code.
pub stack_guards: Option<u32>,
}
/// Result of compiling a word to WASM.
@@ -109,16 +115,73 @@ pub struct CompiledModule {
// Instruction-level helpers (free functions that take &mut Function)
// ---------------------------------------------------------------------------
/// Decrement the cached `$dsp` local by `CELL_SIZE`.
// Stack-guard emission. The fault word's table index is stashed in a
// thread-local by `compile_word` (None = guards off) so the low-level
// push/pop helpers can stay plain `&mut Function` free functions
// without threading config through every emitter.
thread_local! {
static GUARD_FAULT: std::cell::Cell<Option<u32>> = const { std::cell::Cell::new(None) };
}
/// With guards on: emit `if <cond> { mem[SYSVAR_FAULT_CODE] = code;
/// call _STACK_FAULT_ }`. `cond` must leave an i32 boolean on the
/// operand stack. The fault host word throws, so the `if` never falls
/// through on the failure path.
fn emit_guard(f: &mut Function, code: i32, cond: impl FnOnce(&mut Function)) {
let Some(fault_idx) = GUARD_FAULT.get() else {
return;
};
cond(f);
f.instruction(&Instruction::If(BlockType::Empty))
.instruction(&Instruction::I32Const(SYSVAR_FAULT_CODE as i32))
.instruction(&Instruction::I32Const(code))
.instruction(&Instruction::I32Store(MEM4))
.instruction(&Instruction::I32Const(fault_idx as i32))
.instruction(&Instruction::CallIndirect {
type_index: TYPE_VOID,
table_index: TABLE,
})
.instruction(&Instruction::End);
}
/// Guard: data stack has at least `n` cells (else throw -4).
fn guard_dsp_underflow(f: &mut Function, n: u32) {
emit_guard(f, -4, |f| {
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Const((n * CELL_SIZE) as i32))
.instruction(&Instruction::I32Add)
.instruction(&Instruction::I32Const(DATA_STACK_TOP as i32))
.instruction(&Instruction::I32GtU);
});
}
/// Guard: data stack has room for `n` more cells (else throw -3).
fn guard_dsp_overflow(f: &mut Function, n: u32) {
emit_guard(f, -3, |f| {
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Const(
(DATA_STACK_BASE + n * CELL_SIZE) as i32,
))
.instruction(&Instruction::I32LtU);
});
}
/// Decrement the cached `$dsp` local by `CELL_SIZE` (allocate one cell).
/// This is the single choke point for data-stack pushes, so the
/// overflow guard lives here.
fn dsp_dec(f: &mut Function) {
guard_dsp_overflow(f, 1);
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Const(CELL_SIZE as i32))
.instruction(&Instruction::I32Sub)
.instruction(&Instruction::LocalSet(CACHED_DSP_LOCAL));
}
/// Increment the cached `$dsp` local by `CELL_SIZE`.
/// Increment the cached `$dsp` local by `CELL_SIZE` (free one cell).
/// Single choke point for data-stack pops (`DROP` never loads the
/// value, so the underflow guard must sit here, not in `pop`).
fn dsp_inc(f: &mut Function) {
guard_dsp_underflow(f, 1);
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Const(CELL_SIZE as i32))
.instruction(&Instruction::I32Add)
@@ -160,6 +223,7 @@ fn pop_to(f: &mut Function, local: u32) {
/// Read the top of the data stack without popping (value on operand stack).
fn peek(f: &mut Function) {
guard_dsp_underflow(f, 1);
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Load(MEM4));
}
@@ -182,6 +246,13 @@ fn dsp_reload(f: &mut Function) {
/// Push a value from the WASM operand stack onto the return stack via `tmp`.
fn rpush_via_local(f: &mut Function, tmp: u32) {
emit_guard(f, -5, |f| {
f.instruction(&Instruction::GlobalGet(RSP))
.instruction(&Instruction::I32Const(
(RETURN_STACK_BASE + CELL_SIZE) as i32,
))
.instruction(&Instruction::I32LtU);
});
f.instruction(&Instruction::LocalSet(tmp));
// rsp -= CELL_SIZE
f.instruction(&Instruction::GlobalGet(RSP))
@@ -194,8 +265,18 @@ fn rpush_via_local(f: &mut Function, tmp: u32) {
.instruction(&Instruction::I32Store(MEM4));
}
/// Guard: return stack is non-empty (else throw -6).
fn guard_rsp_underflow(f: &mut Function) {
emit_guard(f, -6, |f| {
f.instruction(&Instruction::GlobalGet(RSP))
.instruction(&Instruction::I32Const(RETURN_STACK_TOP as i32))
.instruction(&Instruction::I32GeU);
});
}
/// Pop the return stack onto the WASM operand stack.
fn rpop(f: &mut Function) {
guard_rsp_underflow(f);
f.instruction(&Instruction::GlobalGet(RSP))
.instruction(&Instruction::I32Load(MEM4));
// rsp += CELL_SIZE
@@ -207,6 +288,7 @@ fn rpop(f: &mut Function) {
/// Peek at the top of the return stack (no pop).
fn rpeek(f: &mut Function) {
guard_rsp_underflow(f);
f.instruction(&Instruction::GlobalGet(RSP))
.instruction(&Instruction::I32Load(MEM4));
}
@@ -229,6 +311,9 @@ fn bool_to_forth_flag(f: &mut Function, tmp: u32) {
struct EmitCtx {
f64_local_0: u32,
f64_local_1: u32,
/// Base WASM local index for float-typed Forth locals (`F:` in `{: ... :}`).
/// Float local N maps to WASM local `forth_f_local_base + N` (f64 type).
forth_f_local_base: u32,
/// Base WASM local index for Forth locals ({: ... :}).
/// Forth local N maps to WASM local `forth_local_base + N`.
forth_local_base: u32,
@@ -250,7 +335,9 @@ struct EmitCtx {
}
/// Decrement the FSP global by 8 (allocate space for one f64).
/// Single choke point for float pushes: overflow guard lives here.
fn fsp_dec(f: &mut Function) {
guard_fsp_overflow(f);
f.instruction(&Instruction::GlobalGet(FSP))
.instruction(&Instruction::I32Const(8))
.instruction(&Instruction::I32Sub)
@@ -258,7 +345,10 @@ fn fsp_dec(f: &mut Function) {
}
/// Increment the FSP global by 8 (free space for one f64).
/// Single choke point for float pops (`FDROP` never loads the value):
/// underflow guard lives here.
fn fsp_inc(f: &mut Function) {
guard_fsp_underflow(f);
f.instruction(&Instruction::GlobalGet(FSP))
.instruction(&Instruction::I32Const(8))
.instruction(&Instruction::I32Add)
@@ -275,6 +365,24 @@ fn fpush_via_local(f: &mut Function, tmp: u32) {
.instruction(&Instruction::F64Store(MEM8));
}
/// Guard: float stack has room for one more f64 (else throw -44).
fn guard_fsp_overflow(f: &mut Function) {
emit_guard(f, -44, |f| {
f.instruction(&Instruction::GlobalGet(FSP))
.instruction(&Instruction::I32Const((FLOAT_STACK_BASE + 8) as i32))
.instruction(&Instruction::I32LtU);
});
}
/// Guard: float stack is non-empty (else throw -45).
fn guard_fsp_underflow(f: &mut Function) {
emit_guard(f, -45, |f| {
f.instruction(&Instruction::GlobalGet(FSP))
.instruction(&Instruction::I32Const(FLOAT_STACK_TOP as i32))
.instruction(&Instruction::I32GeU);
});
}
/// Decrement FSP, then store the f64 from local `src` at [FSP].
fn fpush_from_local(f: &mut Function, src: u32) {
fsp_dec(f);
@@ -292,6 +400,7 @@ fn fpop(f: &mut Function) {
/// Load f64 from [FSP] onto the WASM operand stack without popping.
fn fpeek(f: &mut Function) {
guard_fsp_underflow(f);
f.instruction(&Instruction::GlobalGet(FSP))
.instruction(&Instruction::F64Load(MEM8));
}
@@ -691,6 +800,14 @@ fn emit_op(f: &mut Function, op: &IrOp, ctx: &mut EmitCtx) {
IrOp::ForthLocalSet(n) => {
pop_to(f, ctx.forth_local_base + n);
}
IrOp::ForthFLocalGet(n) => {
f.instruction(&Instruction::LocalGet(ctx.forth_f_local_base + n));
fpush_via_local(f, ctx.f64_local_0);
}
IrOp::ForthFLocalSet(n) => {
fpop(f);
f.instruction(&Instruction::LocalSet(ctx.forth_f_local_base + n));
}
// -- Return stack ---------------------------------------------------
IrOp::ToR => {
@@ -782,9 +899,20 @@ fn emit_op(f: &mut Function, op: &IrOp, ctx: &mut EmitCtx) {
.instruction(&Instruction::I32Store(MEM4));
}
IrOp::RpFetch => {
// Push the current return-stack pointer onto the data stack.
// `$rsp` lives in a global (not cached), so no writeback needed.
dsp_dec(f);
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::GlobalGet(RSP))
.instruction(&Instruction::I32Store(MEM4));
}
// -- Compound operations -----------------------------------------------
IrOp::TwoDup => {
// ( a b -- a b a b )
guard_dsp_underflow(f, 2);
guard_dsp_overflow(f, 2);
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Load(MEM4)); // b
f.instruction(&Instruction::LocalSet(SCRATCH_BASE));
@@ -811,6 +939,7 @@ fn emit_op(f: &mut Function, op: &IrOp, ctx: &mut EmitCtx) {
IrOp::TwoDrop => {
// ( a b -- )
guard_dsp_underflow(f, 2);
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL))
.instruction(&Instruction::I32Const((CELL_SIZE * 2) as i32))
.instruction(&Instruction::I32Add)
@@ -1122,9 +1251,12 @@ fn is_promotable(ops: &[IrOp]) -> bool {
fn is_promotable_body(ops: &[IrOp]) -> bool {
for op in ops {
match op {
IrOp::Call(_) | IrOp::TailCall(_) | IrOp::Execute | IrOp::SpFetch => return false,
IrOp::Call(_) | IrOp::TailCall(_) | IrOp::Execute | IrOp::SpFetch | IrOp::RpFetch => {
return false;
}
IrOp::ToR | IrOp::FromR | IrOp::Exit => return false,
IrOp::ForthLocalGet(_) | IrOp::ForthLocalSet(_) => return false,
IrOp::ForthFLocalGet(_) | IrOp::ForthFLocalSet(_) => return false,
IrOp::Emit | IrOp::Dot | IrOp::Cr | IrOp::Type => return false,
IrOp::PushI64(_) | IrOp::PushF64(_) => return false,
IrOp::FDup
@@ -1533,6 +1665,11 @@ impl StackSim {
/// Emit the promoted prologue: load `preload` items from the memory stack
/// into WASM locals.
fn emit_promoted_prologue(f: &mut Function, preload: u32, sim: &mut StackSim) {
// One entry check covers the whole promoted word: the caller must
// have at least `preload` cells on the data stack.
if preload > 0 {
guard_dsp_underflow(f, preload);
}
// Load items: mem[dsp] = top of stack, mem[dsp+4] = second, etc.
// We load them top-first, then reverse the sim stack so that
// sim.stack[0] = deepest loaded, sim.stack[last] = top.
@@ -1563,6 +1700,7 @@ fn emit_promoted_prologue(f: &mut Function, preload: u32, sim: &mut StackSim) {
fn emit_promoted_epilogue(f: &mut Function, sim: &mut StackSim) {
let remaining = sim.stack.len() as u32;
if remaining > 0 {
guard_dsp_overflow(f, remaining);
// Decrement cached DSP for the items we're pushing back
f.instruction(&Instruction::LocalGet(CACHED_DSP_LOCAL));
f.instruction(&Instruction::I32Const((remaining * CELL_SIZE) as i32));
@@ -2000,15 +2138,13 @@ fn emit_promoted_op(f: &mut Function, op: &IrOp, sim: &mut StackSim) {
// Outside loops, RFetch shouldn't appear in promoted code
}
IrOp::LoopJ => {
if sim.loop_index_stack.len() >= 2 {
IrOp::LoopJ if sim.loop_index_stack.len() >= 2 => {
let (outer_index, _) = sim.loop_index_stack[sim.loop_index_stack.len() - 2];
let result = sim.alloc();
f.instruction(&Instruction::LocalGet(outer_index));
f.instruction(&Instruction::LocalSet(result));
sim.push(result);
}
}
IrOp::Exit => {
// Write remaining promoted locals back to memory stack, then return
@@ -2135,16 +2271,16 @@ fn needs_f64_locals(ops: &[IrOp]) -> bool {
return true;
}
}
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body } => {
if needs_f64_locals(body) {
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body }
if needs_f64_locals(body) =>
{
return true;
}
}
IrOp::BeginWhileRepeat { test, body } => {
if needs_f64_locals(test) || needs_f64_locals(body) {
IrOp::BeginWhileRepeat { test, body }
if needs_f64_locals(test) || needs_f64_locals(body) =>
{
return true;
}
}
IrOp::BeginDoubleWhileRepeat {
outer_test,
inner_test,
@@ -2181,6 +2317,9 @@ fn body_needs_return_stack(ops: &[IrOp]) -> bool {
match op {
IrOp::Call(_) | IrOp::TailCall(_) | IrOp::Execute => return true,
IrOp::ToR | IrOp::FromR => return true,
// RP@ observes the return stack, so loop params must be there
// (otherwise inlined RDEPTH/.RS would report an empty stack).
IrOp::RpFetch => return true,
// RFetch (I) is handled by loop locals in the fast path — not a problem.
// LoopJ is also handled by loop locals.
// Only explicit >R / R> / calls force the slow path.
@@ -2197,16 +2336,16 @@ fn body_needs_return_stack(ops: &[IrOp]) -> bool {
return true;
}
}
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body } => {
if body_needs_return_stack(body) {
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body }
if body_needs_return_stack(body) =>
{
return true;
}
}
IrOp::BeginWhileRepeat { test, body } => {
if body_needs_return_stack(test) || body_needs_return_stack(body) {
IrOp::BeginWhileRepeat { test, body }
if body_needs_return_stack(test) || body_needs_return_stack(body) =>
{
return true;
}
}
IrOp::BeginDoubleWhileRepeat {
outer_test,
inner_test,
@@ -2360,15 +2499,46 @@ fn count_forth_locals(ops: &[IrOp]) -> u32 {
max
}
fn count_forth_f_locals(ops: &[IrOp]) -> u32 {
let mut max: u32 = 0;
for op in ops {
match op {
IrOp::ForthFLocalGet(n) | IrOp::ForthFLocalSet(n) => max = max.max(*n + 1),
IrOp::If {
then_body,
else_body,
} => {
max = max.max(count_forth_f_locals(then_body));
if let Some(eb) = else_body {
max = max.max(count_forth_f_locals(eb));
}
}
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body } => {
max = max.max(count_forth_f_locals(body));
}
IrOp::BeginWhileRepeat { test, body } => {
max = max
.max(count_forth_f_locals(test))
.max(count_forth_f_locals(body));
}
_ => {}
}
}
max
}
/// Generate a complete WASM module for a single compiled word.
///
/// This is the JIT path: each word gets its own module that imports
/// shared memory, globals, and function table from the host.
pub fn compile_word(
_name: &str,
name: &str,
body: &[IrOp],
config: &CodegenConfig,
) -> WaferResult<CompiledModule> {
// Arm (or disarm) stack-guard emission for this compilation.
GUARD_FAULT.set(config.stack_guards);
let mut module = Module::new();
// -- Type section --
@@ -2467,8 +2637,14 @@ pub fn compile_word(
} else {
1 + scratch_count + forth_local_count + loop_local_count
};
let has_floats = needs_f64_locals(body);
let num_f64: u32 = if has_floats { 2 } else { 0 };
let forth_f_local_count = count_forth_f_locals(body);
// F: locals need f64 storage, which also implies the f64 scratch pair.
let has_floats = needs_f64_locals(body) || forth_f_local_count > 0;
let num_f64: u32 = if has_floats {
2 + forth_f_local_count
} else {
0
};
let mut locals_decl = vec![(num_locals, ValType::I32)];
if num_f64 > 0 {
locals_decl.push((num_f64, ValType::F64));
@@ -2482,9 +2658,12 @@ pub fn compile_word(
1 + scratch_count
};
let loop_local_base = forth_local_base + forth_local_count;
// f64 scratch pair first (indices num_locals, num_locals+1), then F: locals.
let forth_f_local_base = num_locals + 2;
let mut ctx = EmitCtx {
f64_local_0: num_locals,
f64_local_1: num_locals + 1,
forth_f_local_base,
forth_local_base,
loop_local_base,
loop_locals: Vec::new(),
@@ -2520,6 +2699,16 @@ pub fn compile_word(
code.function(&func);
module.section(&code);
// -- Name section: carries the Forth word name into wasmtime trap
// backtraces (best-effort symbolication, WS-008).
let mut names = wasm_encoder::NameSection::new();
names.module(name);
let mut fn_names = wasm_encoder::NameMap::new();
fn_names.append(0, "emit");
fn_names.append(WORD_FUNC, name);
names.functions(&fn_names);
module.section(&names);
let bytes = module.finish();
// Validate
@@ -2824,8 +3013,9 @@ pub fn compile_consolidated_module(
words: &[(WordId, Vec<IrOp>)],
local_fn_map: &HashMap<WordId, u32>,
table_size: u32,
stack_guards: Option<u32>,
) -> WaferResult<Vec<u8>> {
compile_multi_word_module(words, local_fn_map, table_size, None)
compile_multi_word_module(words, local_fn_map, table_size, None, stack_guards)
}
/// Compile an exportable WASM module with embedded memory and metadata.
@@ -2838,8 +3028,9 @@ pub fn compile_exportable_module(
local_fn_map: &HashMap<WordId, u32>,
table_size: u32,
export: &ExportSections<'_>,
stack_guards: Option<u32>,
) -> WaferResult<Vec<u8>> {
compile_multi_word_module(words, local_fn_map, table_size, Some(export))
compile_multi_word_module(words, local_fn_map, table_size, Some(export), stack_guards)
}
/// Internal: build a multi-word WASM module. When `export` is `Some`, adds
@@ -2849,7 +3040,11 @@ fn compile_multi_word_module(
local_fn_map: &HashMap<WordId, u32>,
table_size: u32,
export: Option<&ExportSections<'_>>,
stack_guards: Option<u32>,
) -> WaferResult<Vec<u8>> {
// Arm (or disarm) stack-guard emission for this module.
GUARD_FAULT.set(stack_guards);
let has_data = export.is_some_and(|e| !e.memory_snapshot.is_empty());
let mut module = Module::new();
@@ -2969,8 +3164,13 @@ fn compile_multi_word_module(
} else {
1 + scratch_count + forth_local_count + loop_local_count
};
let has_floats = needs_f64_locals(body);
let num_f64: u32 = if has_floats { 2 } else { 0 };
let forth_f_local_count = count_forth_f_locals(body);
let has_floats = needs_f64_locals(body) || forth_f_local_count > 0;
let num_f64: u32 = if has_floats {
2 + forth_f_local_count
} else {
0
};
let mut locals_decl = vec![(num_locals, ValType::I32)];
if num_f64 > 0 {
locals_decl.push((num_f64, ValType::F64));
@@ -2984,9 +3184,11 @@ fn compile_multi_word_module(
1 + scratch_count
};
let loop_local_base = forth_local_base + forth_local_count;
let forth_f_local_base = num_locals + 2;
let mut ctx = EmitCtx {
f64_local_0: num_locals,
f64_local_1: num_locals + 1,
forth_f_local_base,
forth_local_base,
loop_local_base,
loop_locals: Vec::new(),
@@ -3071,6 +3273,7 @@ mod tests {
base_fn_index: 0,
table_size: 16,
stack_to_local_promotion: true,
stack_guards: None,
}
}
@@ -3295,6 +3498,7 @@ mod tests {
base_fn_index: 7,
table_size: 16,
stack_to_local_promotion: true,
stack_guards: None,
};
let m = compile_word("t", &[IrOp::PushI32(1)], &cfg).unwrap();
assert_eq!(m.fn_index, 7);
+7
View File
@@ -7,6 +7,11 @@ use crate::optimizer::OptConfig;
pub struct CodegenOpts {
/// Enable stack-to-local promotion for straight-line words.
pub stack_to_local_promotion: bool,
/// Emit stack under/overflow guards in compiled words. Faults throw
/// standard codes (-3/-4/-5/-6/-44/-45) instead of silently
/// corrupting stack pointers. On by default; benchmarks and
/// exported production modules turn it off.
pub stack_guards: bool,
}
/// Master configuration for all WAFER optimizations.
@@ -32,6 +37,7 @@ impl WaferConfig {
},
codegen: CodegenOpts {
stack_to_local_promotion: true,
stack_guards: true,
},
}
}
@@ -49,6 +55,7 @@ impl WaferConfig {
},
codegen: CodegenOpts {
stack_to_local_promotion: false,
stack_guards: false,
},
}
}
+9 -9
View File
@@ -21,7 +21,7 @@ mod tests {
// Empty word list should produce nothing (but we guard against this at call site)
let words = vec![];
let map = HashMap::new();
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
// Empty is valid -- should produce a valid module with no functions
assert!(result.is_ok());
}
@@ -31,7 +31,7 @@ mod tests {
let words = vec![(WordId(1), vec![IrOp::PushI32(42)])];
let mut map = HashMap::new();
map.insert(WordId(1), 1u32); // function index 1 (after emit import)
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
@@ -49,7 +49,7 @@ mod tests {
map.insert(WordId(1), 1u32);
map.insert(WordId(2), 2u32);
map.insert(WordId(3), 3u32);
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
@@ -59,7 +59,7 @@ mod tests {
let words = vec![(WordId(3), vec![IrOp::Call(WordId(99))])];
let mut map = HashMap::new();
map.insert(WordId(3), 1u32);
let result = compile_consolidated_module(&words, &map, 256);
let result = compile_consolidated_module(&words, &map, 256, None);
assert!(result.is_ok());
}
@@ -72,7 +72,7 @@ mod tests {
let mut map = HashMap::new();
map.insert(WordId(1), 1u32);
map.insert(WordId(2), 2u32);
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
@@ -95,7 +95,7 @@ mod tests {
let mut map = HashMap::new();
map.insert(WordId(1), 1u32);
map.insert(WordId(2), 2u32);
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
@@ -120,7 +120,7 @@ mod tests {
let mut map = HashMap::new();
map.insert(WordId(1), 1u32);
map.insert(WordId(2), 2u32);
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
@@ -141,7 +141,7 @@ mod tests {
let mut map = HashMap::new();
map.insert(WordId(1), 1u32);
map.insert(WordId(2), 2u32);
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
@@ -163,7 +163,7 @@ mod tests {
let mut map = HashMap::new();
map.insert(WordId(1), 1u32);
map.insert(WordId(2), 2u32);
let result = compile_consolidated_module(&words, &map, 16);
let result = compile_consolidated_module(&words, &map, 16, None);
assert!(result.is_ok());
}
}
+4 -1
View File
@@ -80,7 +80,10 @@ mod tests {
#[test]
fn sha1_rfc3174_abc() {
assert_eq!(hex(&sha1_hash(b"abc")), "a9993e364706816aba3e25717850c26c9cd0d89d");
assert_eq!(
hex(&sha1_hash(b"abc")),
"a9993e364706816aba3e25717850c26c9cd0d89d"
);
}
#[test]
+34 -9
View File
@@ -18,6 +18,8 @@ pub mod flags {
pub const IMMEDIATE: u8 = 0x80;
/// Word is hidden (being compiled, not yet findable).
pub const HIDDEN: u8 = 0x40;
/// Word is an implementation detail: findable, but skipped by WORDS.
pub const INTERNAL: u8 = 0x20;
/// Mask for the name length (lower 5 bits).
pub const LENGTH_MASK: u8 = 0x1F;
/// Maximum word name length.
@@ -95,11 +97,17 @@ impl Dictionary {
// Write link field (points to previous LATEST)
self.write_u32_unchecked(entry_start, self.latest);
// Write flags byte: HIDDEN | length, optionally IMMEDIATE
// Write flags byte: HIDDEN | length, optionally IMMEDIATE.
// Underscore-prefixed names are implementation details by repo
// convention (see tools/editor-support): flag them INTERNAL so
// WORDS and completion skip them while FIND still works.
let mut flag_byte = flags::HIDDEN | (name_len as u8 & flags::LENGTH_MASK);
if immediate {
flag_byte |= flags::IMMEDIATE;
}
if name_bytes.first() == Some(&b'_') {
flag_byte |= flags::INTERNAL;
}
self.memory[(entry_start + 4) as usize] = flag_byte;
// Write name bytes
@@ -184,10 +192,9 @@ impl Dictionary {
}
}
}
// Fallback: return newest entry across all wordlists
if let Some(&(_wid, word_addr, fn_index, is_immediate)) = entries.last() {
return Some((word_addr, WordId(fn_index), is_immediate));
}
// In no wordlist of the search order: not findable
// (Forth 2012 §16.3.3 — the order is authoritative).
return None;
}
// Fallback: linked-list walk (for words not yet in the index)
@@ -410,8 +417,21 @@ impl Dictionary {
}
/// Return names of all visible (non-hidden) words, newest first.
pub fn visible_words(&self) -> Vec<String> {
let mut names = Vec::new();
/// With `include_internal` false, words flagged INTERNAL are skipped.
pub fn visible_words(&self, include_internal: bool) -> Vec<String> {
self.visible_entries()
.into_iter()
.filter(|(_, _, internal)| include_internal || !internal)
.map(|(name, _, _)| name)
.collect()
}
/// All visible (non-hidden) entries, newest first:
/// (name, wordlist id, INTERNAL flag). The wid comes from the hash
/// index (entries themselves store no wid); words missing from the
/// index default to wid 1 (FORTH).
pub fn visible_entries(&self) -> Vec<(String, u32, bool)> {
let mut entries = Vec::new();
let mut addr = self.latest;
while addr != 0 {
let flags_byte = self.memory[(addr + 4) as usize];
@@ -420,7 +440,12 @@ impl Dictionary {
let name_start = (addr + 5) as usize;
let name = String::from_utf8_lossy(&self.memory[name_start..name_start + name_len])
.to_string();
names.push(name);
let wid = self
.index
.get(&name)
.and_then(|es| es.iter().find(|e| e.1 == addr))
.map_or(1, |e| e.0);
entries.push((name, wid, flags_byte & flags::INTERNAL != 0));
}
let link = self.read_u32_unchecked(addr);
if link == addr {
@@ -428,7 +453,7 @@ impl Dictionary {
}
addr = link;
}
names
entries
}
/// Get a reference to the raw memory buffer.
+7
View File
@@ -61,6 +61,13 @@ pub enum WaferError {
#[error("{0}")]
Abort(String),
/// An uncaught Forth THROW as reported to the user. `message` is the
/// full display text (standard message or ABORT" payload); `code`
/// carries the THROW code for typed consumers (CLI exit paths, web
/// REPL styling) via `Error::downcast_ref`.
#[error("{message}")]
UncaughtThrow { code: i32, message: String },
}
/// Result type alias for WAFER operations.
+8 -4
View File
@@ -120,7 +120,13 @@ pub fn export_module(
metadata_json: metadata_json.as_bytes(),
};
let wasm_bytes = compile_exportable_module(&words, &local_fn_map, table_size, &export_sections)
let wasm_bytes = compile_exportable_module(
&words,
&local_fn_map,
table_size,
&export_sections,
vm.stack_guard_param(),
)
.map_err(|e| anyhow::anyhow!("export codegen error: {e}"))?;
Ok((wasm_bytes, metadata))
@@ -131,11 +137,9 @@ pub fn export_module(
fn collect_external_calls(ops: &[IrOp], ir_ids: &HashSet<WordId>, host_ids: &mut HashSet<WordId>) {
for op in ops {
match op {
IrOp::Call(id) | IrOp::TailCall(id) => {
if !ir_ids.contains(id) {
IrOp::Call(id) | IrOp::TailCall(id) if !ir_ids.contains(id) => {
host_ids.insert(*id);
}
}
IrOp::If {
then_body,
else_body,
+6
View File
@@ -139,6 +139,10 @@ pub enum IrOp {
ForthLocalGet(u32),
/// Set Forth local variable N: ( x -- )
ForthLocalSet(u32),
/// Push float-typed Forth local N: ( F: -- r )
ForthFLocalGet(u32),
/// Set float-typed Forth local N: ( F: r -- )
ForthFLocalSet(u32),
// -- I/O --
/// Output character: ( char -- )
@@ -155,6 +159,8 @@ pub enum IrOp {
Execute,
/// Push the current data-stack pointer: ( -- addr )
SpFetch,
/// Push the current return-stack pointer: ( -- addr )
RpFetch,
// -- Float stack manipulation --
/// Float duplicate: ( F: r -- r r )
+2
View File
@@ -24,6 +24,8 @@ pub mod ir;
pub mod memory;
pub mod optimizer;
pub mod runtime;
pub mod see;
pub mod wordhelp;
// Outer interpreter: runtime-agnostic, works with any Runtime impl
#[allow(trivial_numeric_casts, clippy::unnecessary_cast)]
+6 -4
View File
@@ -50,23 +50,23 @@ pub const DATA_STACK_BASE: u32 = WORD_BUF_BASE + WORD_BUF_SIZE; // 0x0600
pub const DATA_STACK_SIZE: u32 = 4096; // 1024 cells
/// Return stack region. Grows downward.
pub const RETURN_STACK_BASE: u32 = DATA_STACK_BASE + DATA_STACK_SIZE; // 0x1540
pub const RETURN_STACK_BASE: u32 = DATA_STACK_BASE + DATA_STACK_SIZE; // 0x1600
/// Size of return stack region.
pub const RETURN_STACK_SIZE: u32 = 4096;
/// Floating-point stack region (fallback). Grows downward.
pub const FLOAT_STACK_BASE: u32 = RETURN_STACK_BASE + RETURN_STACK_SIZE; // 0x2540
pub const FLOAT_STACK_BASE: u32 = RETURN_STACK_BASE + RETURN_STACK_SIZE; // 0x2600
/// Size of float stack region.
pub const FLOAT_STACK_SIZE: u32 = 2048; // 256 doubles
/// Hash scratch region — output buffer for `SHA1`/`SHA256`/`SHA512` and
/// other hash host words. Sized for the largest supported digest (SHA512 = 64 B).
pub const HASH_SCRATCH_BASE: u32 = FLOAT_STACK_BASE + FLOAT_STACK_SIZE; // 0x2D40
pub const HASH_SCRATCH_BASE: u32 = FLOAT_STACK_BASE + FLOAT_STACK_SIZE; // 0x2E00
/// Size of hash scratch region.
pub const HASH_SCRATCH_SIZE: u32 = 128;
/// Dictionary region start. Grows upward.
pub const DICTIONARY_BASE: u32 = HASH_SCRATCH_BASE + HASH_SCRATCH_SIZE; // 0x2DC0
pub const DICTIONARY_BASE: u32 = HASH_SCRATCH_BASE + HASH_SCRATCH_SIZE; // 0x2E80
/// Initial top of data stack (grows down from here).
pub const DATA_STACK_TOP: u32 = DATA_STACK_BASE + DATA_STACK_SIZE;
@@ -106,6 +106,8 @@ pub const SYSVAR_NUM_TIB: u32 = SYSVAR_BASE + 24;
pub const SYSVAR_HLD: u32 = SYSVAR_BASE + 28;
/// LEAVE flag: nonzero when LEAVE has been called inside a DO loop.
pub const SYSVAR_LEAVE_FLAG: u32 = SYSVAR_BASE + 32;
/// Throw code left by a compiled stack-guard fault for `_STACK_FAULT_`.
pub const SYSVAR_FAULT_CODE: u32 = SYSVAR_BASE + 36;
#[cfg(test)]
mod tests {
+15 -13
View File
@@ -591,16 +591,16 @@ fn contains_call_to(ops: &[IrOp], target: WordId) -> bool {
return true;
}
}
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body } => {
if contains_call_to(body, target) {
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body }
if contains_call_to(body, target) =>
{
return true;
}
}
IrOp::BeginWhileRepeat { test, body } => {
if contains_call_to(test, target) || contains_call_to(body, target) {
IrOp::BeginWhileRepeat { test, body }
if contains_call_to(test, target) || contains_call_to(body, target) =>
{
return true;
}
}
IrOp::BeginDoubleWhileRepeat {
outer_test,
inner_test,
@@ -633,7 +633,11 @@ fn contains_call_to(ops: &[IrOp], target: WordId) -> bool {
fn contains_exit(ops: &[IrOp]) -> bool {
for op in ops {
match op {
IrOp::Exit | IrOp::ForthLocalGet(_) | IrOp::ForthLocalSet(_) => return true,
IrOp::Exit
| IrOp::ForthLocalGet(_)
| IrOp::ForthLocalSet(_)
| IrOp::ForthFLocalGet(_)
| IrOp::ForthFLocalSet(_) => return true,
IrOp::If {
then_body,
else_body,
@@ -647,16 +651,14 @@ fn contains_exit(ops: &[IrOp]) -> bool {
return true;
}
}
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body } => {
if contains_exit(body) {
IrOp::DoLoop { body, .. } | IrOp::BeginUntil { body } | IrOp::BeginAgain { body }
if contains_exit(body) =>
{
return true;
}
}
IrOp::BeginWhileRepeat { test, body } => {
if contains_exit(test) || contains_exit(body) {
IrOp::BeginWhileRepeat { test, body } if contains_exit(test) || contains_exit(body) => {
return true;
}
}
_ => {}
}
}
+2540 -326
View File
File diff suppressed because it is too large Load Diff
+21 -2
View File
@@ -98,11 +98,29 @@ impl HostAccess for CallerHostAccess<'_, '_> {
let func = *func_ref
.unwrap_func()
.ok_or_else(|| anyhow::anyhow!("call_func: null funcref {fn_index}"))?;
func.call(&mut *self.caller, &[], &mut [])?;
func.call(&mut *self.caller, &[], &mut [])
.map_err(name_trap_frame)?;
Ok(())
}
}
/// Prefix a wasmtime trap error with the innermost named WASM frame.
/// Compiled words carry their Forth name in the module name section, so a
/// genuine trap reads "in <WORD>: wasm trap: ...". THROW-driven unwinds
/// also pass through here, but CATCH and `describe_uncaught` key on the
/// shared `throw_code` cell, never on the message, so the wrap is inert
/// for them.
fn name_trap_frame(e: wasmtime::Error) -> wasmtime::Error {
let name = e
.downcast_ref::<wasmtime::WasmBacktrace>()
.and_then(|bt| bt.frames().iter().find_map(|f| f.func_name()))
.map(str::to_string);
match name {
Some(n) => e.context(format!("in {n}")),
None => e,
}
}
/// Wasmtime-based native runtime.
pub struct NativeRuntime {
engine: Engine,
@@ -293,7 +311,8 @@ impl Runtime for NativeRuntime {
let func = *r
.unwrap_func()
.ok_or_else(|| anyhow::anyhow!("word {fn_index} is null funcref"))?;
func.call(&mut self.store, &[], &mut [])?;
func.call(&mut self.store, &[], &mut [])
.map_err(name_trap_frame)?;
Ok(())
}
+376
View File
@@ -0,0 +1,376 @@
//! IR pretty-printer for `SEE-IR` and the `SEE` fallback path.
//!
//! Renders a post-optimization IR body as indented, one-op-per-line text.
//! Simple ops print as short lowercase mnemonics (Forth glyphs where they
//! are universally recognizable: `@`, `!`, `0=`, `>r`, ...); structured ops
//! print as Forth control words with 2-space indented bodies. Calls resolve
//! `WordId`s to names through an optional resolver so the formatter itself
//! stays independent of the VM.
use crate::dictionary::WordId;
use crate::ir::IrOp;
/// Format an IR body as indented, one-op-per-line text.
pub fn format_ir(ops: &[IrOp]) -> String {
format_ir_with(ops, &|_| None)
}
/// Like [`format_ir`], resolving `Call`/`TailCall`/`Execute` targets to word
/// names via `resolve`; unresolved ids print as `#N`.
pub fn format_ir_with(ops: &[IrOp], resolve: &dyn Fn(WordId) -> Option<String>) -> String {
let mut out = String::new();
write_ops(&mut out, ops, 0, resolve);
out
}
fn line(out: &mut String, depth: usize, text: &str) {
for _ in 0..depth {
out.push_str(" ");
}
out.push_str(text);
out.push('\n');
}
fn callee(id: WordId, resolve: &dyn Fn(WordId) -> Option<String>) -> String {
resolve(id).unwrap_or_else(|| format!("#{}", id.0))
}
fn write_ops(
out: &mut String,
ops: &[IrOp],
depth: usize,
resolve: &dyn Fn(WordId) -> Option<String>,
) {
for op in ops {
write_op(out, op, depth, resolve);
}
}
fn write_op(out: &mut String, op: &IrOp, depth: usize, resolve: &dyn Fn(WordId) -> Option<String>) {
// Exhaustive on purpose: a new IrOp variant must show up here at
// compile time, not silently render wrong.
let simple: String = match op {
// -- Literals --
IrOp::PushI32(v) => format!("push {v}"),
IrOp::PushI64(v) => format!("push64 {v}"),
IrOp::PushF64(v) => format!("fpush {v}"),
// -- Stack manipulation --
IrOp::Drop => "drop".into(),
IrOp::Dup => "dup".into(),
IrOp::Swap => "swap".into(),
IrOp::Over => "over".into(),
IrOp::Rot => "rot".into(),
IrOp::Nip => "nip".into(),
IrOp::Tuck => "tuck".into(),
IrOp::TwoDup => "2dup".into(),
IrOp::TwoDrop => "2drop".into(),
// -- Arithmetic --
IrOp::Add => "add".into(),
IrOp::Sub => "sub".into(),
IrOp::Mul => "mul".into(),
IrOp::DivMod => "divmod".into(),
IrOp::Negate => "negate".into(),
IrOp::Abs => "abs".into(),
// -- Comparison --
IrOp::Eq => "eq".into(),
IrOp::NotEq => "ne".into(),
IrOp::Lt => "lt".into(),
IrOp::Gt => "gt".into(),
IrOp::LtUnsigned => "u<".into(),
IrOp::ZeroEq => "0=".into(),
IrOp::ZeroLt => "0<".into(),
// -- Logic --
IrOp::And => "and".into(),
IrOp::Or => "or".into(),
IrOp::Xor => "xor".into(),
IrOp::Invert => "invert".into(),
IrOp::Lshift => "lshift".into(),
IrOp::Rshift => "rshift".into(),
IrOp::ArithRshift => "arshift".into(),
// -- Memory --
IrOp::Fetch => "@".into(),
IrOp::Store => "!".into(),
IrOp::CFetch => "c@".into(),
IrOp::CStore => "c!".into(),
IrOp::PlusStore => "+!".into(),
// -- Calls --
IrOp::Call(id) => format!("call {}", callee(*id, resolve)),
IrOp::TailCall(id) => format!("tail-call {}", callee(*id, resolve)),
// -- Structured control flow (multi-line) --
IrOp::If {
then_body,
else_body,
} => {
line(out, depth, "if");
write_ops(out, then_body, depth + 1, resolve);
if let Some(eb) = else_body {
line(out, depth, "else");
write_ops(out, eb, depth + 1, resolve);
}
line(out, depth, "then");
return;
}
IrOp::DoLoop { body, is_plus_loop } => {
line(out, depth, "do");
write_ops(out, body, depth + 1, resolve);
line(out, depth, if *is_plus_loop { "+loop" } else { "loop" });
return;
}
IrOp::BeginUntil { body } => {
line(out, depth, "begin");
write_ops(out, body, depth + 1, resolve);
line(out, depth, "until");
return;
}
IrOp::BeginAgain { body } => {
line(out, depth, "begin");
write_ops(out, body, depth + 1, resolve);
line(out, depth, "again");
return;
}
IrOp::BeginWhileRepeat { test, body } => {
line(out, depth, "begin");
write_ops(out, test, depth + 1, resolve);
line(out, depth, "while");
write_ops(out, body, depth + 1, resolve);
line(out, depth, "repeat");
return;
}
IrOp::BeginDoubleWhileRepeat {
outer_test,
inner_test,
body,
after_repeat,
else_body,
} => {
line(out, depth, "begin");
write_ops(out, outer_test, depth + 1, resolve);
line(out, depth, "while");
write_ops(out, inner_test, depth + 1, resolve);
line(out, depth, "while");
write_ops(out, body, depth + 1, resolve);
line(out, depth, "repeat");
write_ops(out, after_repeat, depth + 1, resolve);
if let Some(eb) = else_body {
line(out, depth, "else");
write_ops(out, eb, depth + 1, resolve);
}
line(out, depth, "then");
return;
}
IrOp::Exit => "exit".into(),
IrOp::LoopRestartIfFalse => "loop-restart-if-false".into(),
// -- Flat forward branches --
IrOp::Block(l) => format!("block L{l}"),
IrOp::BranchIfFalse(l) => format!("branch-if-false L{l}"),
IrOp::EndBlock(l) => format!("end-block L{l}"),
// -- Return stack --
IrOp::ToR => ">r".into(),
IrOp::FromR => "r>".into(),
IrOp::RFetch => "r@".into(),
IrOp::LoopJ => "j".into(),
// -- Forth locals --
IrOp::ForthLocalGet(n) => format!("local@ {n}"),
IrOp::ForthLocalSet(n) => format!("local! {n}"),
IrOp::ForthFLocalGet(n) => format!("flocal@ {n}"),
IrOp::ForthFLocalSet(n) => format!("flocal! {n}"),
// -- I/O --
IrOp::Emit => "emit".into(),
IrOp::Dot => ".".into(),
IrOp::Cr => "cr".into(),
IrOp::Type => "type".into(),
// -- System --
IrOp::Execute => "execute".into(),
IrOp::SpFetch => "sp@".into(),
IrOp::RpFetch => "rp@".into(),
// -- Float stack --
IrOp::FDup => "fdup".into(),
IrOp::FDrop => "fdrop".into(),
IrOp::FSwap => "fswap".into(),
IrOp::FOver => "fover".into(),
// -- Float arithmetic --
IrOp::FAdd => "fadd".into(),
IrOp::FSub => "fsub".into(),
IrOp::FMul => "fmul".into(),
IrOp::FDiv => "fdiv".into(),
IrOp::FNegate => "fnegate".into(),
IrOp::FAbs => "fabs".into(),
IrOp::FSqrt => "fsqrt".into(),
IrOp::FMin => "fmin".into(),
IrOp::FMax => "fmax".into(),
IrOp::FFloor => "ffloor".into(),
IrOp::FRound => "fround".into(),
// -- Float comparisons --
IrOp::FZeroEq => "f0=".into(),
IrOp::FZeroLt => "f0<".into(),
IrOp::FEq => "f=".into(),
IrOp::FLt => "f<".into(),
// -- Float memory --
IrOp::FetchFloat => "f@".into(),
IrOp::StoreFloat => "f!".into(),
// -- Conversions --
IrOp::StoF => "s>f".into(),
IrOp::FtoS => "f>s".into(),
};
line(out, depth, &simple);
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn simple_ops_one_per_line() {
let out = format_ir(&[IrOp::Dup, IrOp::Mul, IrOp::PushI32(7)]);
assert_eq!(out, "dup\nmul\npush 7\n");
}
#[test]
fn call_resolves_via_resolver() {
let ops = [IrOp::Call(WordId(12)), IrOp::TailCall(WordId(13))];
assert_eq!(format_ir(&ops), "call #12\ntail-call #13\n");
let named = format_ir_with(&ops, &|id| (id.0 == 12).then(|| "SQ".to_string()));
assert_eq!(named, "call SQ\ntail-call #13\n");
}
#[test]
fn nested_if_inside_do_loop_indents() {
let ops = [IrOp::DoLoop {
body: vec![
IrOp::Dup,
IrOp::If {
then_body: vec![IrOp::Dup, IrOp::Mul],
else_body: Some(vec![IrOp::Drop]),
},
],
is_plus_loop: false,
}];
let expected = "do\n dup\n if\n dup\n mul\n else\n drop\n then\nloop\n";
assert_eq!(format_ir(&ops), expected);
}
#[test]
fn while_loops_and_flat_branches() {
let ops = [
IrOp::BeginWhileRepeat {
test: vec![IrOp::Dup],
body: vec![IrOp::PushI32(1), IrOp::Sub],
},
IrOp::Block(3),
IrOp::BranchIfFalse(3),
IrOp::EndBlock(3),
];
let expected = "begin\n dup\nwhile\n push 1\n sub\nrepeat\nblock L3\nbranch-if-false L3\nend-block L3\n";
assert_eq!(format_ir(&ops), expected);
}
#[test]
fn every_simple_variant_renders() {
// One of each non-structured op; count of output lines must match.
let ops = vec![
IrOp::PushI32(1),
IrOp::PushI64(2),
IrOp::PushF64(1.5),
IrOp::Drop,
IrOp::Dup,
IrOp::Swap,
IrOp::Over,
IrOp::Rot,
IrOp::Nip,
IrOp::Tuck,
IrOp::TwoDup,
IrOp::TwoDrop,
IrOp::Add,
IrOp::Sub,
IrOp::Mul,
IrOp::DivMod,
IrOp::Negate,
IrOp::Abs,
IrOp::Eq,
IrOp::NotEq,
IrOp::Lt,
IrOp::Gt,
IrOp::LtUnsigned,
IrOp::ZeroEq,
IrOp::ZeroLt,
IrOp::And,
IrOp::Or,
IrOp::Xor,
IrOp::Invert,
IrOp::Lshift,
IrOp::Rshift,
IrOp::ArithRshift,
IrOp::Fetch,
IrOp::Store,
IrOp::CFetch,
IrOp::CStore,
IrOp::PlusStore,
IrOp::Call(WordId(1)),
IrOp::TailCall(WordId(2)),
IrOp::Exit,
IrOp::LoopRestartIfFalse,
IrOp::Block(1),
IrOp::BranchIfFalse(1),
IrOp::EndBlock(1),
IrOp::ToR,
IrOp::FromR,
IrOp::RFetch,
IrOp::LoopJ,
IrOp::ForthLocalGet(0),
IrOp::ForthLocalSet(0),
IrOp::ForthFLocalGet(0),
IrOp::ForthFLocalSet(0),
IrOp::Emit,
IrOp::Dot,
IrOp::Cr,
IrOp::Type,
IrOp::Execute,
IrOp::SpFetch,
IrOp::RpFetch,
IrOp::FDup,
IrOp::FDrop,
IrOp::FSwap,
IrOp::FOver,
IrOp::FAdd,
IrOp::FSub,
IrOp::FMul,
IrOp::FDiv,
IrOp::FNegate,
IrOp::FAbs,
IrOp::FSqrt,
IrOp::FMin,
IrOp::FMax,
IrOp::FFloor,
IrOp::FRound,
IrOp::FZeroEq,
IrOp::FZeroLt,
IrOp::FEq,
IrOp::FLt,
IrOp::FetchFloat,
IrOp::StoreFloat,
IrOp::StoF,
IrOp::FtoS,
];
let out = format_ir(&ops);
assert_eq!(out.lines().count(), ops.len());
// Every line non-empty, no accidental blank rendering.
assert!(out.lines().all(|l| !l.trim().is_empty()));
}
}
File diff suppressed because it is too large Load Diff
+198 -57
View File
@@ -1,8 +1,10 @@
#![allow(dead_code)]
//! Cross-engine comparison tests: WAFER vs gforth.
//! Cross-engine comparison tests: WAFER vs gforth (and `SwiftForth` for perf).
//!
//! Validates that WAFER produces identical output to gforth for standard
//! Forth programs, and benchmarks performance of both engines.
//! Forth programs, and benchmarks performance of the engines. `SwiftForth`
//! (`sf64`, native-code commercial compiler) joins the performance report
//! as an upper-bound reference when installed.
//!
//! WAFER-only correctness: `cargo test -p wafer-core --test comparison`
//! Full comparison + perf: `cargo test -p wafer-core --test comparison -- --nocapture --ignored`
@@ -26,8 +28,7 @@ fn probe_gforth(candidate: &str) -> bool {
.arg("-e")
.arg("bye")
.output()
.map(|o| o.status.success())
.unwrap_or(false)
.is_ok_and(|o| o.status.success())
}
fn find_gforth() -> Option<&'static str> {
@@ -64,6 +65,48 @@ fn find_gforth_fast() -> Option<&'static str> {
.as_deref()
}
// -----------------------------------------------------------------------
// SwiftForth (sf64) discovery (cached)
// -----------------------------------------------------------------------
static SF64_PATH: OnceLock<Option<String>> = OnceLock::new();
/// Probe sf64 by piping `bye` via stdin — sf64 has no `-e` flag; it takes
/// Forth source from stdin or as bare command-line arguments.
fn probe_sf64(candidate: &str) -> bool {
run_via_stdin(candidate, "bye\n").is_some_and(|o| o.status.success())
}
fn find_sf64() -> Option<&'static str> {
SF64_PATH
.get_or_init(|| {
for candidate in &["/Applications/ForthInc/SwiftForth/bin/macos/sf64", "sf64"] {
if probe_sf64(candidate) {
return Some(candidate.to_string());
}
}
None
})
.as_deref()
}
/// Spawn `binary`, write `input` to its stdin, and collect the output.
fn run_via_stdin(binary: &str, input: &str) -> Option<std::process::Output> {
Command::new(binary)
// Perf lanes measure unguarded code (only the wafer binary reads this)
.env("WAFER_STACK_GUARDS", "0")
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.and_then(|mut child| {
use std::io::Write;
child.stdin.take().unwrap().write_all(input.as_bytes())?;
child.wait_with_output()
})
.ok()
}
// -----------------------------------------------------------------------
// Engine runners
// -----------------------------------------------------------------------
@@ -410,6 +453,21 @@ fn programs() -> Vec<Program> {
expected: "99 \n",
category: Category::Definitions,
},
Program {
name: "search-order-hides",
code: "WORDLIST CONSTANT MY-WL\n\
MY-WL SET-CURRENT\n\
: SECRET 42 ;\n\
FORTH-WORDLIST SET-CURRENT\n\
[UNDEFINED] SECRET . CR\n\
GET-ORDER MY-WL SWAP 1+ SET-ORDER\n\
[DEFINED] SECRET . CR\n\
SECRET . CR\n\
-1 SET-ORDER\n\
[UNDEFINED] SECRET . CR",
expected: "-1 \n-1 \n42 \n-1 \n",
category: Category::Definitions,
},
// -- Strings --
Program {
name: "s-quote-type",
@@ -581,6 +639,81 @@ fn compare_all_programs() {
);
}
// -----------------------------------------------------------------------
// Cross-engine behavioral comparison (requires SwiftForth sf64) -- WS-003
// -----------------------------------------------------------------------
/// Run Forth code through `SwiftForth`. Piped sf64 is quiet (no banner, no
/// `ok` echo), truncates input lines at ~256 chars, and exits 243 after an
/// error, so statements are fed one per line with a final `bye`.
fn run_sf64_code(sf64: &str, code: &str) -> Option<EngineResult> {
let mut input = String::new();
for line in code.lines() {
let t = line.trim();
if !t.is_empty() {
input.push_str(t);
input.push('\n');
}
}
input.push_str("bye\n");
let out = run_via_stdin(sf64, &input)?;
Some(EngineResult {
output: String::from_utf8_lossy(&out.stdout).to_string(),
success: out.status.success(),
})
}
/// Correctness lane against `SwiftForth`: the same program corpus as the
/// gforth comparison, sf64 as the oracle. Skips gracefully when sf64 is
/// not installed (CI/linux). Programs listed in `SF64_SKIP` use words or
/// output conventions `SwiftForth` does not share.
#[test]
#[ignore = "requires SwiftForth sf64 (run with -- --ignored)"]
fn compare_all_programs_sf64() {
// dot-quote: `."` outside a definition is a no-op in SwiftForth
// (compile-only); WAFER supports the interpret-mode extension.
const SF64_SKIP: &[&str] = &["dot-quote"];
let Some(sf64) = find_sf64() else {
eprintln!("SKIP: sf64 not found");
return;
};
let progs = programs();
let mut passed = 0;
let mut skipped = 0;
for prog in &progs {
if SF64_SKIP.contains(&prog.name) {
skipped += 1;
continue;
}
let wafer = run_wafer(prog.code);
assert!(wafer.success, "{}: WAFER execution failed", prog.name);
let Some(sf) = run_sf64_code(sf64, prog.code) else {
skipped += 1;
continue;
};
if !sf.success {
eprintln!(" WARN {}: sf64 execution failed, skipping", prog.name);
skipped += 1;
continue;
}
// SwiftForth prints numbers space-prefixed and echoes piped input
// lines, so byte-exact comparison is meaningless; compare the
// whitespace-token stream (the printed values and strings).
let wafer_tokens: Vec<&str> = wafer.output.split_whitespace().collect();
let sf_tokens: Vec<&str> = sf.output.split_whitespace().collect();
assert_eq!(
wafer_tokens, sf_tokens,
"{}: output differs\n WAFER: {:?}\n sf64: {:?}",
prog.name, wafer.output, sf.output
);
passed += 1;
}
eprintln!(
"\nsf64 behavioral comparison: {passed} passed, {skipped} skipped (of {})",
progs.len()
);
}
// -----------------------------------------------------------------------
// Performance comparison (requires gforth)
// -----------------------------------------------------------------------
@@ -699,31 +832,11 @@ fn measure_wafer_release(wafer: &str, bench: &PerfBenchmark) -> Option<u64> {
define = bench.define,
run = bench.run_code,
);
let output = Command::new(wafer)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.and_then(|mut child| {
use std::io::Write;
child.stdin.take().unwrap().write_all(code.as_bytes())?;
child.wait_with_output()
})
.ok()?;
let output = run_via_stdin(wafer, &code)?;
if !output.status.success() {
return None;
}
let stdout = String::from_utf8_lossy(&output.stdout);
let mut times: Vec<u64> = stdout
.trim()
.lines()
.filter_map(|l| l.trim().parse::<u64>().ok())
.collect();
times.sort();
if times.is_empty() {
return None;
}
Some(times[times.len() / 2])
median_printed_time(&output.stdout)
}
/// Measure WAFER execution time after CONSOLIDATE (direct calls between all words).
@@ -735,21 +848,17 @@ fn measure_wafer_consolidated(wafer: &str, bench: &PerfBenchmark) -> Option<u64>
define = bench.define,
run = bench.run_code,
);
let output = Command::new(wafer)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.and_then(|mut child| {
use std::io::Write;
child.stdin.take().unwrap().write_all(code.as_bytes())?;
child.wait_with_output()
})
.ok()?;
let output = run_via_stdin(wafer, &code)?;
if !output.status.success() {
return None;
}
let stdout = String::from_utf8_lossy(&output.stdout);
median_printed_time(&output.stdout)
}
/// Parse the microsecond values printed by TIMED-BENCH (one per line) and
/// return the median.
fn median_printed_time(stdout: &[u8]) -> Option<u64> {
let stdout = String::from_utf8_lossy(stdout);
let mut times: Vec<u64> = stdout
.trim()
.lines()
@@ -779,18 +888,28 @@ fn measure_gforth(gforth: &str, bench: &PerfBenchmark) -> Option<u64> {
if !output.status.success() {
return None;
}
let stdout = String::from_utf8_lossy(&output.stdout);
// Parse the 3 timing values and take the median
let mut times: Vec<u64> = stdout
.trim()
.lines()
.filter_map(|l| l.trim().parse::<u64>().ok())
.collect();
times.sort();
if times.is_empty() {
median_printed_time(&output.stdout)
}
/// Measure `SwiftForth` (`sf64`) execution time using Forth-level `ucounter`
/// (double-cell microsecond counter; `2swap d- drop` yields elapsed us —
/// the same wrapper shape as gforth's `utime`). Timing excludes startup.
/// sf64 has no `-e` flag, so the program is piped via stdin — one statement
/// per line, because sf64 truncates input lines at ~256 chars.
/// Returns microseconds, or None if sf64 is unavailable or fails.
fn measure_sf64(sf64: &str, bench: &PerfBenchmark) -> Option<u64> {
let code = format!(
"{define}\n{run}\n\
: TIMED-BENCH ucounter {run} ucounter 2swap d- drop . cr ;\n\
TIMED-BENCH\nTIMED-BENCH\nTIMED-BENCH\nbye\n",
define = bench.define,
run = bench.run_code,
);
let output = run_via_stdin(sf64, &code)?;
if !output.status.success() {
return None;
}
Some(times[times.len() / 2])
median_printed_time(&output.stdout)
}
#[test]
@@ -831,18 +950,31 @@ fn performance_report() {
);
}
let sep = "=".repeat(80);
let thin = "-".repeat(80);
let sf64 = find_sf64();
if sf64.is_none() {
eprintln!("NOTE: sf64 (SwiftForth) not found — column skipped");
}
let sep = "=".repeat(100);
let thin = "-".repeat(100);
println!("\n{sep}");
println!(" WAFER vs Gforth Performance Comparison (release mode)");
println!(" WAFER vs Gforth vs SwiftForth Performance Comparison (release mode)");
println!("{sep}\n");
println!(
"{:<22} {:>10} {:>10} {:>10} {:>10} {:>10} {:>10}",
"Benchmark", "WAFER", "CONSOL", "gforth", "gf-fast", "WAFER/gf", "limit"
"{:<22} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9}",
"Benchmark",
"WAFER",
"CONSOL",
"gforth",
"gf-fast",
"sf64",
"WAFER/gf",
"WAFER/sf",
"limit"
);
println!(
"{:<22} {:>10} {:>10} {:>10} {:>10} {:>10} {:>10}",
"", "(us)", "(us)", "(us)", "(us)", "", ""
"{:<22} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9}",
"", "(us)", "(us)", "(us)", "(us)", "(us)", "", "", ""
);
println!("{thin}");
@@ -857,9 +989,11 @@ fn performance_report() {
.unwrap_or(0);
let gf = gforth.and_then(|g| measure_gforth(g, bench));
let gf_fast = gforth_fast.and_then(|g| measure_gforth(g, bench));
let sf = sf64.and_then(|s| measure_sf64(s, bench));
let gf_str = gf.map_or_else(|| "-".to_string(), |v| format!("{v}"));
let gf_fast_str = gf_fast.map_or_else(|| "-".to_string(), |v| format!("{v}"));
let sf_str = sf.map_or_else(|| "-".to_string(), |v| format!("{v}"));
let best_wafer = if consol > 0 && consol < wafer {
consol
} else {
@@ -873,11 +1007,15 @@ fn performance_report() {
}
});
let ratio = ratio_val.map_or_else(|| "-".to_string(), |r| format!("{r:.2}x"));
let sf_ratio = sf.filter(|&s| s > 0).map_or_else(
|| "-".to_string(),
|s| format!("{:.2}x", best_wafer as f64 / s as f64),
);
let limit_str = format!("{:.2}x", bench.max_ratio);
println!(
"{:<22} {:>10} {:>10} {:>10} {:>10} {:>10} {:>10}",
bench.name, wafer, consol, gf_str, gf_fast_str, ratio, limit_str
"{:<22} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9} {:>9}",
bench.name, wafer, consol, gf_str, gf_fast_str, sf_str, ratio, sf_ratio, limit_str
);
// Check regression limits
@@ -900,6 +1038,9 @@ fn performance_report() {
println!("{thin}");
println!(" WAFER = all optimizations, CONSOL = after CONSOLIDATE");
println!(" WAFER/gf = best(WAFER,CONSOL) vs gforth, < 1.0 means WAFER faster");
println!(
" WAFER/sf = best(WAFER,CONSOL) vs SwiftForth sf64 (native code; informational, no limit)"
);
println!("{sep}\n");
if !regressions.is_empty() {
+185 -24
View File
@@ -13,41 +13,170 @@ const SUITE_DIR: &str = concat!(
"/../../tests/forth2012-test-suite/src"
);
/// Load a file and evaluate it line by line, ignoring errors on individual lines.
fn load_file(vm: &mut ForthVM<NativeRuntime>, path: &str) {
/// Load a file line-by-line, returning the number of lines that raised an
/// `evaluate` error. Each failing line is printed (visible under
/// `cargo test -- --nocapture`) so failures can be triaged without a
/// debugger.
///
/// Historically this helper discarded errors silently, which caused tests
/// like LT32 in `localstest.fth` (compile errors from unknown words such
/// as `(LOCAL)` before it was implemented) to vanish — the T{ }T error
/// counter was never incremented because the `:` definition never ran.
/// Returning the count surfaces silent skips as real failures.
///
/// **Note on multi-line definitions.** WAFER's DOES> handler collects
/// the does-body to `;` via `next_token()` within a *single* `evaluate`
/// call and treats end-of-input as end-of-body. Files with a `DOES>`
/// split across lines (e.g. `errorreport.fth`) therefore cannot be
/// loaded line-by-line; use [`load_file_whole`] for those.
fn load_file(vm: &mut ForthVM<NativeRuntime>, path: &str) -> u32 {
let source = std::fs::read_to_string(path).unwrap_or_else(|_| panic!("Failed to read {path}"));
for line in source.lines() {
let _ = vm.evaluate(line);
let mut fails = 0u32;
for (lineno, line) in source.lines().enumerate() {
if let Err(e) = vm.evaluate(line) {
fails += 1;
eprintln!("{path}:{}: {e}\n line: {line}", lineno + 1);
}
}
vm.take_output(); // discard output
fails
}
/// Load a file as a single `evaluate` call (not line-by-line). Required
/// for files with multi-line definitions that WAFER's per-line handlers
/// can't stitch across calls (notably `: X ... DOES> ... ;` spanning
/// lines — see [`load_file`] note).
///
/// Returns `1` on any failure, `0` on success, so the caller can apply
/// baselines the same way as [`load_file`].
fn load_file_whole(vm: &mut ForthVM<NativeRuntime>, path: &str) -> u32 {
let source = std::fs::read_to_string(path).unwrap_or_else(|_| panic!("Failed to read {path}"));
let fails = match vm.evaluate(&source) {
Ok(()) => 0,
Err(e) => {
eprintln!("{path}: {e}");
1
}
};
vm.take_output();
fails
}
/// Baseline of *known* line-level failures per prerequisite file. The runner
/// asserts `load_fails == expected_load_failures(path)`, so any regression
/// above (or silently-fixed case below) the baseline is caught.
///
/// Baselines are not an allowlist to paper over bugs — they are an explicit
/// tech-debt ledger. Each non-zero entry here is a bug that should be fixed
/// and the baseline lowered to zero. See the in-tree follow-up tasks.
fn expected_load_failures(path: &str) -> u32 {
// core.fr exercises two constructs WAFER does not yet support:
// 1. Nested colon definitions (`: NOP : POSTPONE ; ;` at line 751,
// defining NOP, NOP1, NOP2 — four silent lines).
// 2. `SOURCE`/`>IN` round-trip through `EVALUATE` at line 797
// (GS1 definition) — one line.
// Total: 5. Fix these and drop the baseline to 0.
if path.ends_with("/core.fr") {
return 5;
}
// coreexttest.fth uses two Core-Extension features WAFER lacks:
// 1. SAVE-INPUT / RESTORE-INPUT at line 548 — not implemented.
// 2. `.(` inside `[ ... ]` brackets at line 559 — `.(` isn't
// handled by `compile_token`'s `[ ... ]` interpret-mode path,
// so `First message via .(` tokens leak to the compiler as
// undefined words.
// Total: 2. Fix these and drop the baseline to 0.
if path.ends_with("/coreexttest.fth") {
return 2;
}
// exceptiontest.fth line 95 fails with a garbled parse ("unknown word"
// over non-ASCII bytes): WAFER's parser reads past a prior test's
// scratch region after the preceding `C6` / `T9` frame exercises
// CATCH/THROW source stacking. Root cause not yet diagnosed; baseline
// until fixed.
if path.ends_with("/exceptiontest.fth") {
return 1;
}
// toolstest.fth uses the `\?` conditional-skip idiom defined in
// utilities.fth:37 as `: \? (\?) @ IF EXIT THEN SOURCE >IN ! DROP ;
// IMMEDIATE`. Under WAFER's per-line `evaluate` loader, the
// `SOURCE >IN ! DROP` path does not consume the remainder of the
// current line correctly, so 37 `\?`-guarded lines inside the
// TRAVERSE-WORDLIST / NAME>COMPILE / NAME>INTERPRET blocks leak as
// unknown-word errors. Fix the SOURCE/`>IN` interaction with
// line-mode input and drop this to 0.
//
// The 38th: line 368 `R> DROP TRUE` runs interpreted (its enclosing
// definition aborted on the missing NAME?), and the bare `R>` used
// to underflow the return stack silently; stack guards now report
// it as "Return stack underflow (throw -6)".
if path.ends_with("/toolstest.fth") {
return 38;
}
0
}
/// Assert a file loaded with exactly its baseline number of line-level
/// failures. Used for prerequisites; keeps the runner tight without
/// blocking the whole suite on known gaps.
fn assert_load_fails_within_baseline(path: &str, fails: u32) {
let expected = expected_load_failures(path);
assert_eq!(
fails, expected,
"{path} had {fails} line-level failures (expected baseline: {expected})"
);
}
/// Boot a WAFER VM with full prerequisites loaded.
///
/// Every prerequisite file must load with zero line-level errors. Any
/// regression here points to a missing primitive or a parser bug and must
/// be fixed, not silently tolerated.
fn boot_with_prerequisites() -> ForthVM<NativeRuntime> {
let mut vm = ForthVM::<NativeRuntime>::new().expect("Failed to create ForthVM");
// Load test framework
load_file(&mut vm, &format!("{SUITE_DIR}/tester.fr"));
let tester_path = format!("{SUITE_DIR}/tester.fr");
let f1 = load_file(&mut vm, &tester_path);
assert_load_fails_within_baseline(&tester_path, f1);
// Load core tests (prerequisite)
load_file(&mut vm, &format!("{SUITE_DIR}/core.fr"));
let core_path = format!("{SUITE_DIR}/core.fr");
let f2 = load_file(&mut vm, &core_path);
assert_load_fails_within_baseline(&core_path, f2);
// Switch to decimal and load utilities
let _ = vm.evaluate("DECIMAL");
vm.take_output();
load_file(&mut vm, &format!("{SUITE_DIR}/utilities.fth"));
let util_path = format!("{SUITE_DIR}/utilities.fth");
let f3 = load_file(&mut vm, &util_path);
assert_load_fails_within_baseline(&util_path, f3);
// errorreport.fth defines SET-ERROR-COUNT and the per-wordset counter
// accessors (CORE-ERRORS, STRING-ERRORS, LOCALS-ERRORS, ...). Every
// suite's final `X-ERRORS SET-ERROR-COUNT` line depends on this file,
// and silently errored before the runner was tightened.
let errorreport_path = format!("{SUITE_DIR}/errorreport.fth");
let f_err = load_file_whole(&mut vm, &errorreport_path);
assert_load_fails_within_baseline(&errorreport_path, f_err);
// Load core extensions
load_file(&mut vm, &format!("{SUITE_DIR}/coreexttest.fth"));
let ext_path = format!("{SUITE_DIR}/coreexttest.fth");
let f4 = load_file(&mut vm, &ext_path);
assert_load_fails_within_baseline(&ext_path, f4);
vm
}
/// Run a test suite file and return the #ERRORS count.
/// Run a test suite file and return the *total* error count:
/// `#ERRORS` from the Forth test framework plus any lines where
/// `vm.evaluate` itself failed (e.g. unknown word in a `:` definition
/// outside `T{ }T`, which the framework cannot catch).
fn run_suite(vm: &mut ForthVM<NativeRuntime>, test_file: &str) -> u32 {
// Reset error counter
let _ = vm.evaluate("DECIMAL 0 #ERRORS !");
vm.take_output();
// Load the test file
load_file(vm, &format!("{SUITE_DIR}/{test_file}"));
let file_path = format!("{SUITE_DIR}/{test_file}");
let load_fails = load_file(vm, &file_path);
assert_load_fails_within_baseline(&file_path, load_fails);
// Read error count -- try multiple approaches to be robust
let _ = vm.evaluate("DECIMAL");
@@ -76,8 +205,12 @@ fn run_suite(vm: &mut ForthVM<NativeRuntime>, test_file: &str) -> u32 {
#[test]
fn compliance_core() {
let mut vm = ForthVM::<NativeRuntime>::new().expect("Failed to create ForthVM");
load_file(&mut vm, &format!("{SUITE_DIR}/tester.fr"));
load_file(&mut vm, &format!("{SUITE_DIR}/core.fr"));
let tester_path = format!("{SUITE_DIR}/tester.fr");
let f1 = load_file(&mut vm, &tester_path);
assert_load_fails_within_baseline(&tester_path, f1);
let core_path = format!("{SUITE_DIR}/core.fr");
let f2 = load_file(&mut vm, &core_path);
assert_load_fails_within_baseline(&core_path, f2);
let _ = vm.evaluate("DECIMAL #ERRORS @");
let errors = vm.data_stack().first().copied().unwrap_or(-1);
@@ -96,17 +229,31 @@ fn compliance_core_ext() {
// Core Extensions are loaded as part of prerequisites.
// Run from scratch to get a clean error count.
let mut vm = ForthVM::<NativeRuntime>::new().expect("Failed to create ForthVM");
load_file(&mut vm, &format!("{SUITE_DIR}/tester.fr"));
load_file(&mut vm, &format!("{SUITE_DIR}/core.fr"));
let tester_path = format!("{SUITE_DIR}/tester.fr");
let f1 = load_file(&mut vm, &tester_path);
assert_load_fails_within_baseline(&tester_path, f1);
let core_path = format!("{SUITE_DIR}/core.fr");
let f2 = load_file(&mut vm, &core_path);
assert_load_fails_within_baseline(&core_path, f2);
let _ = vm.evaluate("DECIMAL");
vm.take_output();
load_file(&mut vm, &format!("{SUITE_DIR}/utilities.fth"));
let util_path = format!("{SUITE_DIR}/utilities.fth");
let f3 = load_file(&mut vm, &util_path);
assert_load_fails_within_baseline(&util_path, f3);
let errorreport_path = format!("{SUITE_DIR}/errorreport.fth");
let f_err = load_file_whole(&mut vm, &errorreport_path);
assert_load_fails_within_baseline(&errorreport_path, f_err);
let _ = vm.evaluate("DECIMAL 0 #ERRORS !");
vm.take_output();
load_file(&mut vm, &format!("{SUITE_DIR}/coreexttest.fth"));
let ext_path = format!("{SUITE_DIR}/coreexttest.fth");
let load_fails = load_file(&mut vm, &ext_path);
assert_load_fails_within_baseline(&ext_path, load_fails);
let _ = vm.evaluate("DECIMAL #ERRORS @");
let errors = vm.data_stack().first().copied().unwrap_or(-1) as u32;
assert_eq!(errors, 0, "Core Extensions: {errors} test failures");
let framework_errors = vm.data_stack().first().copied().unwrap_or(-1) as u32;
assert_eq!(
framework_errors, 0,
"Core Extensions: {framework_errors} framework test failures"
);
}
#[test]
@@ -164,17 +311,31 @@ fn compliance_string() {
// Run from scratch -- the stringtest includes CoreExt tests that
// cascade failures when run on top of an already-loaded CoreExt suite.
let mut vm = ForthVM::<NativeRuntime>::new().expect("Failed to create ForthVM");
load_file(&mut vm, &format!("{SUITE_DIR}/tester.fr"));
load_file(&mut vm, &format!("{SUITE_DIR}/core.fr"));
let tester_path = format!("{SUITE_DIR}/tester.fr");
let f1 = load_file(&mut vm, &tester_path);
assert_load_fails_within_baseline(&tester_path, f1);
let core_path = format!("{SUITE_DIR}/core.fr");
let f2 = load_file(&mut vm, &core_path);
assert_load_fails_within_baseline(&core_path, f2);
let _ = vm.evaluate("DECIMAL");
vm.take_output();
load_file(&mut vm, &format!("{SUITE_DIR}/utilities.fth"));
let util_path = format!("{SUITE_DIR}/utilities.fth");
let f3 = load_file(&mut vm, &util_path);
assert_load_fails_within_baseline(&util_path, f3);
let errorreport_path = format!("{SUITE_DIR}/errorreport.fth");
let f_err = load_file_whole(&mut vm, &errorreport_path);
assert_load_fails_within_baseline(&errorreport_path, f_err);
let _ = vm.evaluate("DECIMAL 0 #ERRORS !");
vm.take_output();
load_file(&mut vm, &format!("{SUITE_DIR}/stringtest.fth"));
let str_path = format!("{SUITE_DIR}/stringtest.fth");
let load_fails = load_file(&mut vm, &str_path);
assert_load_fails_within_baseline(&str_path, load_fails);
let _ = vm.evaluate("DECIMAL #ERRORS @");
let errors = vm.data_stack().first().copied().unwrap_or(-1) as u32;
assert_eq!(errors, 0, "String: {errors} test failures");
let framework_errors = vm.data_stack().first().copied().unwrap_or(-1) as u32;
assert_eq!(
framework_errors, 0,
"String: {framework_errors} framework test failures"
);
}
#[test]
+9 -3
View File
@@ -1,6 +1,6 @@
//! End-to-end tests for the `SHA1` / `SHA256` / `SHA512` Forth host words.
//!
//! These run inside a real WAFER VM (NativeRuntime). The Forth program writes
//! These run inside a real WAFER VM (`NativeRuntime`). The Forth program writes
//! a counted string into `PAD`, calls the hash word, then the test reads the
//! digest out of WAFER linear memory and compares it to the RFC-3174 / FIPS-180
//! reference vectors.
@@ -26,10 +26,16 @@ fn hash_via_forth(word: &str, input: &[u8]) -> Vec<u8> {
// Stack now: ( c-addr2 u2 ). Read u2 then c-addr2 from data stack.
let stack = vm.data_stack();
assert!(stack.len() >= 2, "expected (addr len) on stack, got {stack:?}");
assert!(
stack.len() >= 2,
"expected (addr len) on stack, got {stack:?}"
);
let u2 = stack[0] as usize;
let addr2 = stack[1] as u32;
assert_eq!(addr2, HASH_SCRATCH_BASE, "digest should land in HASH_SCRATCH");
assert_eq!(
addr2, HASH_SCRATCH_BASE,
"digest should land in HASH_SCRATCH"
);
// Read the digest out of WAFER linear memory.
let mut bytes = Vec::with_capacity(u2);
+1 -1
View File
@@ -12,7 +12,7 @@ workspace = true
crate-type = ["cdylib", "rlib"]
[dependencies]
wafer-core = { path = "../core", version = "0.1.0", default-features = false, features = ["crypto"] }
wafer-core = { path = "../core", version = "0.2.1", default-features = false, features = ["crypto"] }
wasm-bindgen = "0.2"
js-sys = "0.3"
send_wrapper = { workspace = true }
+8 -4
View File
@@ -6,8 +6,9 @@ use send_wrapper::SendWrapper;
use wasm_bindgen::prelude::*;
use wafer_core::config::WaferConfig;
use wafer_core::memory::{CELL_SIZE, PAD_BASE, PAD_SIZE};
use wafer_core::memory::{CELL_SIZE, PAD_BASE, PAD_SIZE, SYSVAR_BASE_VAR};
use wafer_core::outer::ForthVM;
use wafer_core::runtime::Runtime;
use wafer_core::runtime::{HostAccess, HostFn};
use crate::runtime_web::WebRuntime;
@@ -53,9 +54,12 @@ impl WaferRepl {
/// Get the current number base (10 = decimal, 16 = hex).
pub fn base(&mut self) -> u32 {
// BASE is stored at SYSVAR_BASE_VAR in WASM memory
self.vm.take_output(); // no-op side effect; just return base
10 // TODO: read from memory once we have a getter
self.vm.runtime_mut().mem_read_i32(SYSVAR_BASE_VAR) as u32
}
/// Names of all user-facing words (visible, non-internal), newest first.
pub fn words(&self) -> Vec<String> {
self.vm.word_names()
}
/// Reset the VM to initial state.
+41 -22
View File
@@ -1,8 +1,11 @@
import init, { WaferRepl } from './pkg/wafer_web.js';
let repl = null;
const history = [];
let historyIdx = -1;
const HISTORY_KEY = 'wafer-history';
const HISTORY_MAX = 200;
const history = JSON.parse(localStorage.getItem(HISTORY_KEY) || '[]');
let historyIdx = history.length;
let builtinWords = null;
const WORD_CATEGORIES = {
'Stack': 'DUP DROP SWAP OVER ROT NIP TUCK 2DUP 2DROP 2SWAP 2OVER PICK ROLL DEPTH .S'.split(' '),
@@ -39,10 +42,12 @@ function updateStack() {
if (!repl) return;
try {
const stack = repl.data_stack();
const base = repl.base();
const suffix = base !== 10 ? ` [base ${base}]` : '';
if (stack.length === 0) {
stackBar.textContent = 'Stack: (empty)';
stackBar.textContent = `Stack: (empty)${suffix}`;
} else {
stackBar.textContent = `Stack <${stack.length}> ${stack.join(' ')}`;
stackBar.textContent = `Stack <${stack.length}> ${stack.join(' ')}${suffix}`;
}
} catch {
stackBar.textContent = 'Stack: (error)';
@@ -50,19 +55,25 @@ function updateStack() {
}
function updateUserWords() {
const cat = document.getElementById('cat-user');
if (!cat) return;
// We'll track user words by checking what the REPL evaluates
// For now, just show the category
const list = document.getElementById('user-word-list');
if (!list || !repl || !builtinWords) return;
list.innerHTML = '';
for (const w of repl.words()) {
if (!builtinWords.has(w)) list.appendChild(wordChip(w));
}
}
function evaluate(line) {
function evaluate(line, record = true) {
if (!repl) return;
const trimmed = line.trim();
if (!trimmed) return;
// Add to history
// Add to history (user-typed lines only; skip consecutive duplicates)
if (record && history[history.length - 1] !== trimmed) {
history.push(trimmed);
if (history.length > HISTORY_MAX) history.splice(0, history.length - HISTORY_MAX);
localStorage.setItem(HISTORY_KEY, JSON.stringify(history));
}
historyIdx = history.length;
try {
@@ -83,6 +94,7 @@ function evaluate(line) {
updatePrompt();
updateStack();
updateUserWords();
}
// Input handling
@@ -116,6 +128,18 @@ document.getElementById('btn-toggle-words').addEventListener('click', () => {
document.getElementById('word-panel').classList.toggle('collapsed');
});
function wordChip(w) {
const chip = document.createElement('span');
chip.className = 'word-chip';
chip.textContent = w;
chip.title = w;
chip.addEventListener('click', () => {
input.value += (input.value.length > 0 ? ' ' : '') + w;
input.focus();
});
return chip;
}
function buildWordPanel() {
const container = document.getElementById('word-categories');
container.innerHTML = '';
@@ -129,15 +153,7 @@ function buildWordPanel() {
const list = document.createElement('div');
list.className = 'word-list';
for (const w of words) {
const chip = document.createElement('span');
chip.className = 'word-chip';
chip.textContent = w;
chip.title = w;
chip.addEventListener('click', () => {
input.value += (input.value.length > 0 ? ' ' : '') + w;
input.focus();
});
list.appendChild(chip);
list.appendChild(wordChip(w));
}
cat.appendChild(list);
container.appendChild(cat);
@@ -179,7 +195,7 @@ document.getElementById('btn-run-init').addEventListener('click', () => {
if (code.trim()) {
// Run each line separately
for (const line of code.split('\n')) {
if (line.trim()) evaluate(line);
if (line.trim()) evaluate(line, false);
}
}
localStorage.setItem('wafer-init-code', code);
@@ -214,6 +230,7 @@ document.getElementById('btn-reset').addEventListener('click', () => {
appendLine('WAFER reset.', 'line-ok');
updatePrompt();
updateStack();
updateUserWords();
} catch (e) {
appendLine(`Reset error: ${e.message}`, 'line-error');
}
@@ -225,6 +242,8 @@ async function boot() {
try {
await init();
repl = new WaferRepl();
// Everything defined at boot is "builtin"; later definitions are user words
builtinWords = new Set(repl.words());
output.innerHTML = '';
appendLine('WAFER — WebAssembly Forth Engine in Rust', 'line-output');
appendLine(`Type Forth at the > prompt. Press ? for help.`, 'line-output');
@@ -241,7 +260,7 @@ async function boot() {
const initCode = document.getElementById('init-code').value;
if (initCode.trim()) {
for (const line of initCode.split('\n')) {
if (line.trim()) evaluate(line);
if (line.trim()) evaluate(line, false);
}
localStorage.setItem('wafer-init-code', initCode);
}
@@ -252,7 +271,7 @@ async function boot() {
const code = atob(location.hash.slice(1));
document.getElementById('init-code').value = code;
for (const line of code.split('\n')) {
if (line.trim()) evaluate(line);
if (line.trim()) evaluate(line, false);
}
} catch { /* ignore bad hash */ }
}
+329 -157
View File
@@ -1,6 +1,11 @@
WAFER Architecture Reference (updated 2026-04-13)
WAFER Architecture Reference (updated 2026-04-16)
===================================================
WAFER = WebAssembly Forth Engine in Rust. Optimizing Forth-2012 compiler that
emits WASM at run time. Each colon definition becomes its own WASM module that
shares memory, globals, and a function table with every other word.
1. COMPILATION PIPELINE
-----------------------
@@ -11,96 +16,134 @@ WAFER Architecture Reference (updated 2026-04-13)
+--------------------------------------------+
| Tokenizer: whitespace-delimited words |
| For each token: |
| 1. Dictionary lookup (find) |
| 2. If found + interpret mode: EXECUTE |
| 3. If found + compile mode: |
| - Immediate? Execute now |
| 1. Dictionary lookup (HashMap + wordlist |
| search order) |
| 2. Found + interpret mode: EXECUTE |
| 3. Found + compile mode: |
| - IMMEDIATE? Execute now |
| - Normal? Append Call(WordId) to IR |
| 4. Not found: try parse as number |
| - Interpret: push to data stack |
| - Compile: append PushI32(n) to IR |
| - Compile: append PushI32/64/F64 |
| 5. Neither: error "unknown word" |
| Special cases handled here, not via IR: |
| defining words (CREATE, VARIABLE, :), |
| DOES> dispatch, S" / ." string parsing, |
| {: ... :} locals, [: ... ;] quotations. |
+--------------------------------------------+
| On `;` (end of colon definition):
v
Optimizer (optimizer.rs)
Optimizer (optimizer.rs) — IR -> IR
+--------------------------------------------+
| Phase 1: Simplify |
| Peephole -> Constant Fold -> |
| Strength Reduce -> Peephole |
| Phase 2: Inline then re-simplify |
| Inline(max=8) -> Peephole -> |
| Constant Fold -> Strength Reduce -> |
| Peephole |
| Phase 3: Eliminate dead code |
| DCE -> Peephole |
| Phase 4: Tail calls (must be last) |
| Tail Call Detect |
| Phase 1 simplify: |
| peephole -> fold -> strength -> peephole |
| Phase 2 inline (max 8 ops) then re-simpl.: |
| inline -> peephole -> fold -> strength |
| -> peephole |
| Phase 3 dead code: dce -> peephole |
| Phase 4 tail calls (must be last) |
| Total peephole passes: 5 |
+--------------------------------------------+
|
v
Codegen (codegen.rs)
Codegen (codegen.rs) — IR -> WASM bytes
+--------------------------------------------+
| IR -> WASM bytecode via wasm-encoder |
| Each word = one WASM module with: |
| Imports: emit, memory, dsp, rsp, fsp, |
| table |
| Types: void () -> (), i32 (i32) -> () |
| One defined function (the word body) |
| DSP cached in local 0, writeback before |
| calls, reload after calls |
| Scratch locals start at index 1 |
| wasm-encoder builds one module per word. |
| Function locals (laid out in order): |
| 0 cached DSP (i32) |
| 1..s scratch i32 (or promoted |
| stack-to-local slots) |
| s..f Forth locals from {: ... :} |
| (i32 then f64) |
| f..l loop locals: 2 per nested |
| DO/?DO (index, limit) |
| DSP write-back before every Call, |
| reload after — keeps host functions and |
| call_indirect targets coherent. |
| Stack-to-local promotion (codegen flag): |
| straight-line + simple control flow |
| words skip the linear-memory data stack |
| entirely; values stay in WASM locals. |
+--------------------------------------------+
|
v
Runtime trait (runtime.rs)
Runtime trait (runtime.rs) — execution backend
+--------------------------------------------+
| ForthVM<R: Runtime> generic over backend |
| Runtime provides: |
| - Memory r/w (mem_read_i32, etc.) |
| - Globals (get/set_dsp, rsp, fsp) |
| - Table (ensure_table_size) |
| - instantiate_and_install(wasm_bytes) |
| - call_func(fn_index) |
| - register_host_func(fn_index, HostFn) |
| ForthVM<R: Runtime> generic over backend. |
| Runtime owns: |
| - shared linear memory (16 pages init) |
| - shared funcref table (grows on demand) |
| - 3 mutable i32 globals (dsp/rsp/fsp) |
| - emit() import bound to output buffer |
| Runtime methods: |
| mem_read/write_{i32,u8,slice} |
| get/set_{dsp,rsp,fsp} |
| ensure_table_size(n) |
| instantiate_and_install(wasm, fn_index) |
| call_func(fn_index) |
| register_host_func(fn_index, HostFn) |
| |
| HostAccess trait — memory/global ops for |
| host function callbacks |
| HostFn = Box<dyn Fn(&mut dyn HostAccess)> |
| HostAccess trait — same memory/global ops |
| exposed to host-fn callbacks; lets one |
| HostFn closure run on either runtime. |
| HostFn = Box<dyn Fn(&mut dyn HostAccess) |
| -> Result<()> + Send + Sync> |
+--------------------------------------------+
| |
v v
NativeRuntime WebRuntime
(runtime_native.rs) (crates/web/runtime_web.rs)
(runtime_native.rs, (crates/web/src/
feature = "native") runtime_web.rs)
+------------------+ +------------------+
| wasmtime Engine | | js_sys::WebAsm |
| Store, Memory | | Memory, Table |
| Table, Globals | | Global objects |
| Func closures | | JS Closures |
| wasmtime Engine, | | js_sys WebAsm |
| Store, Memory, | | Memory, Table, |
| Table, Globals, | | Global, JS |
| Func closures | | Closures |
+------------------+ +------------------+
2. MEMORY LAYOUT (Linear Memory)
--------------------------------
2. MEMORY LAYOUT (linear memory, single shared instance)
--------------------------------------------------------
Address Region Size Notes
-------- ------------------ ------- -------------------------
-------- ------------------ ------- --------------------------
0x0000 System Variables 64 B STATE, BASE, >IN, HERE,
LATEST, SOURCE-ID, #TIB,
HLD, LEAVE-FLAG
0x0040 Input Buffer 1024 B Source parsing
0x0440 PAD 256 B Scratch area
0x0540 Pictured Output 128 B <# ... #> (grows down)
0x0040 Input Buffer (TIB) 1024 B Source line being parsed
0x0440 PAD 256 B Scratch for string ops
0x0540 Pictured Output 128 B <# ... #> (HLD grows down)
0x05C0 WORD Buffer 64 B Transient counted string
0x0600 Data Stack 4096 B 1024 cells, grows DOWN
0x1600 (Data Stack Top) DSP starts here
0x1540 Return Stack 4096 B Grows DOWN
0x2540 Float Stack 2048 B 256 doubles, grows DOWN
0x2D40 Dictionary grows UP Linked list of word entries
^ DSP starts at top = 0x1600
0x1600 Return Stack 4096 B Grows DOWN
^ RSP starts at top = 0x2600
0x2600 Float Stack 2048 B 256 doubles, grows DOWN
^ FSP starts at top = 0x2E00
0x2E00 Hash Scratch 128 B SHA1/256/512 output
0x2E80 Dictionary grows UP Linked list of entries
Total initial memory: 16 pages = 1 MiB (max 256 pages = 16 MiB)
Cell size: 4 bytes (i32)
Float size: 8 bytes (f64)
Constants from crates/core/src/memory.rs (authoritative):
SYSVAR_BASE 0x0000 size 64
INPUT_BUFFER_BASE 0x0040 size 1024
PAD_BASE 0x0440 size 256
PICT_BUF_BASE 0x0540 size 128
WORD_BUF_BASE 0x05C0 size 64
DATA_STACK_BASE 0x0600 size 4096 (DATA_STACK_TOP = 0x1600)
RETURN_STACK_BASE 0x1600 size 4096 (RETURN_STACK_TOP = 0x2600)
FLOAT_STACK_BASE 0x2600 size 2048 (FLOAT_STACK_TOP = 0x2E00)
HASH_SCRATCH_BASE 0x2E00 size 128
DICTIONARY_BASE 0x2E80 grows up to memory.len()
(Some inline `// 0x...` comments in memory.rs are stale — the
computed values above are correct; the consts are derived.)
Total initial memory: 16 pages = 1 MiB (max 256 pages = 16 MiB).
Cell size: 4 bytes (i32). Float size: 8 bytes (f64).
Stack layout note: linear-memory data and float stacks are the
fallback used whenever the optimizer can't keep values in WASM
locals. After stack-to-local promotion, many words touch DSP
only on entry/exit.
3. SYSTEM VARIABLES (offsets from 0x0000)
@@ -113,60 +156,86 @@ WAFER Architecture Reference (updated 2026-04-13)
8 >IN Parse offset into input buffer
12 HERE Next free dictionary address
16 LATEST Most recent dictionary entry addr
20 SOURCE-ID 0=user input, -1=string
20 SOURCE-ID 0=user input, -1=string, fileid>0
24 #TIB Length of current input
28 HLD Pictured numeric output pointer
32 LEAVE-FLAG Nonzero when LEAVE called in loop
4. DICTIONARY ENTRY FORMAT
--------------------------
4. DICTIONARY (dictionary.rs)
-----------------------------
+--------+-------+----------+---------+-----------+
| Link | Flags | Name | Padding | Code |
| 4 bytes| 1 byte| N bytes | 0-3 B | 4 bytes |
+--------+-------+----------+---------+-----------+
Entry layout in linear memory:
+--------+-------+----------+---------+-----------+----------+
| Link | Flags | Name | Padding | Code | Param |
| 4 B | 1 B | N B | 0-3 B | 4 B | optional |
+--------+-------+----------+---------+-----------+----------+
^ ^
entry_addr code field (fn table index)
entry_addr code field (fn-table idx)
Flags byte:
Bit 7 (0x80): IMMEDIATE
Bit 6 (0x40): HIDDEN (during compilation)
Bits 0-4 (0x1F): name length (max 31)
Bits 0-4 : name length (max 31)
Link points to previous entry (0 = end of list).
Name stored uppercase, padded to 4-byte alignment.
Code field: index into WASM function table.
Parameter field (if any) follows immediately after code field.
Code field: index into shared WASM function table.
Parameter field follows the code field for CREATE'd /
DOES> / VARIABLE / CONSTANT bodies.
Lookup is NOT linear: dictionary.rs maintains a HashMap
index from name -> Vec<(wid, addr, fn_index, immediate)>.
Each entry is tagged with its wordlist id; resolution
walks the current search order.
Wordlists / Search-Order:
wordlist ids are u32; the FORTH wordlist is id 1.
`current_wid` selects where new definitions land;
`search_order` is the lookup chain (top first).
Implements the Forth-2012 Search-Order word set.
5. THREE TYPES OF WORDS
-----------------------
5. WORD CATEGORIES
------------------
a) IR Primitives (compiled to WASM)
register_primitive("DUP", false, vec![IrOp::Dup])
a) IR Primitives — register_primitive("DUP", false, vec![IrOp::Dup])
- Body stored as Vec<IrOp>
- Optimized, then compiled to WASM module
- Optimized, then compiled to WASM
- Inlineable by optimizer
- FAST: no function call overhead when inlined
- Batched at boot: ~110 primitive registrations compiled
into a single WASM module to amortize instantiation cost
b) Host Functions (HostFn closures)
register_host_primitive(".", false, func)
- HostFn = Box<dyn Fn(&mut dyn HostAccess) -> Result<()>>
- Access memory/globals via HostAccess trait (runtime-agnostic)
b) Host Functions — register_host_primitive(".", false, func)
- HostFn = Box<dyn Fn(&mut dyn HostAccess)
-> Result<()> + Send + Sync>
- Access memory/globals via HostAccess trait
- NOT inlineable
- Used for: I/O, dictionary manipulation, complex logic
- Same closure works on NativeRuntime and WebRuntime
- Used for I/O, dictionary manipulation, complex stack ops
- Same closure runs on NativeRuntime and WebRuntime
c) Forth-defined words
: SQUARE DUP * ;
- Compiled by outer interpreter
- Goes through full optimize -> codegen pipeline
- Stored in ir_bodies for future inlining
c) Forth-defined words — `: SQUARE DUP * ;`
- Compiled by the outer interpreter
- Goes through the full optimize -> codegen pipeline
- Stored in `ir_bodies` for future inlining
d) Special interpreter tokens (immediate, with custom parsing)
- Defining words: CREATE, VARIABLE, CONSTANT, :, ;, DOES>
- String literals: S", ."
- Control structures: IF/ELSE/THEN, BEGIN/UNTIL/WHILE/REPEAT,
DO/?DO/LOOP/+LOOP, [: ... ;] quotations, {: ... :} locals
- CONSOLIDATE
Their body-collection / dictionary-side-effect logic lives
directly in compile_token / interpret_token_immediate.
They still emit IR ops (e.g. IrOp::If, IrOp::DoLoop,
IrOp::ForthLocalGet) — the difference is that they are NOT
registered via register_primitive; the outer interpreter
handles them as special syntax.
6. WASM MODULE STRUCTURE (per word)
-----------------------------------
6. WASM MODULE STRUCTURE (per JIT-compiled word)
------------------------------------------------
Imports (6) — provided by Runtime impl:
0. emit (func: i32 -> void) Character output callback
@@ -176,25 +245,59 @@ WAFER Architecture Reference (updated 2026-04-13)
4. fsp (global: mut i32) Float stack pointer
5. table (table: funcref) Shared function table
Types (2):
0. void: () -> ()
1. i32: (i32) -> ()
Types: () -> () for word bodies; (i32) -> () for emit.
Functions (1):
The compiled word body
The compiled word body, typed () -> ().
Element section:
table[base_fn_index] = function 1
Runtime::instantiate_and_install(wasm_bytes, fn_index):
- NativeRuntime: Module::new + Instance::new with 6 wasmtime imports
- WebRuntime: WebAssembly.instantiate with JS import objects
- NativeRuntime: wasmtime Module::new + Instance::new
with the 6 imports above
- WebRuntime: WebAssembly.instantiate with JS import
objects pulled from the shared WaferRepl state
7. OPTIMIZATION PASSES (detail)
7. IR OPS (ir.rs — IrOp enum)
-----------------------------
Stack: Drop, Dup, Swap, Over, Rot, Nip, Tuck,
TwoDup, TwoDrop
Literals: PushI32, PushI64, PushF64
Arithmetic: Add, Sub, Mul, DivMod, Negate, Abs
Compare: Eq, NotEq, Lt, Gt, LtUnsigned,
ZeroEq, ZeroLt
Logic: And, Or, Xor, Invert,
Lshift, Rshift, ArithRshift
Memory: Fetch, Store, CFetch, CStore, PlusStore
Control: Call, TailCall, Exit,
If{then, else?},
DoLoop{body, is_plus_loop},
BeginUntil, BeginAgain,
BeginWhileRepeat,
BeginDoubleWhileRepeat,
LoopRestartIfFalse,
Block(label), BranchIfFalse(label),
EndBlock(label) -- for CS-ROLL'd patterns
Return stack: ToR, FromR, RFetch, LoopJ
Forth locals: ForthLocalGet/Set,
ForthFLocalGet/Set
I/O: Emit, Dot, Cr, Type
System: Execute, SpFetch
Float stack: FDup, FDrop, FSwap, FOver
Float math: FAdd, FSub, FMul, FDiv, FNegate, FAbs,
FSqrt, FMin, FMax, FFloor, FRound
Float compare:FZeroEq, FZeroLt, FEq, FLt
Float memory: FetchFloat, StoreFloat
Conversion: StoF, FtoS
8. OPTIMIZATION PASSES (detail)
-------------------------------
PEEPHOLE (runs 5x across full pipeline):
PEEPHOLE (5x across pipeline):
PushI32(n), Drop -> (removed) Unused literal
Dup, Drop -> (removed) Redundant copy
Swap, Swap -> (removed) Self-inverse
@@ -205,16 +308,17 @@ WAFER Architecture Reference (updated 2026-04-13)
PushI32(1), Mul -> (removed) Identity
Over, Over -> TwoDup Combine
Drop, Drop -> TwoDrop Combine
(+ float variants: PushF64/FDrop, FDup/FDrop, FSwap/FSwap, FNegate/FNegate)
Float variants:
PushF64(_), FDrop / FDup, FDrop /
FSwap, FSwap / FNegate, FNegate
CONSTANT FOLD:
Binary: PushI32(a), PushI32(b), <op> -> PushI32(result)
Supports: Add, Sub, Mul, And, Or, Xor, Lshift, Rshift, ArithRshift,
Binary i32: PushI32(a), PushI32(b), <op> -> PushI32(r)
Add, Sub, Mul, And, Or, Xor,
Lshift, Rshift, ArithRshift,
Eq, NotEq, Lt, Gt, LtUnsigned
Unary: PushI32(n), <op> -> PushI32(result)
Supports: Negate, Abs, Invert, ZeroEq, ZeroLt
Float binary: PushF64(a), PushF64(b), <op> -> PushF64(result)
Float unary: PushF64(n), <op> -> PushF64(result)
Unary i32: Negate, Abs, Invert, ZeroEq, ZeroLt
Float binary/unary equivalents on PushF64.
STRENGTH REDUCE:
PushI32(2^n), Mul -> PushI32(n), Lshift
@@ -226,81 +330,149 @@ WAFER Architecture Reference (updated 2026-04-13)
PushI32(0), If{then,else} -> else_body only
Everything after Exit -> removed
INLINE (max_size=8, single pass):
Call(id) -> inline body if:
- Body length <= 8 ops
- No self-recursion
- No Exit (would return from caller)
- No ForthLocalGet/Set (would collide with caller's locals)
INLINE (max 8 ops, single pass):
Call(id) -> body if all of:
- body length <= 8 ops
- no self-recursion
- no Exit (would return from caller)
- no ForthLocalGet/Set (would collide with caller locals)
TailCall -> Call when inlined (no longer tail position)
TAIL CALL (last pass):
Last Call(id) -> TailCall(id) if:
- Return stack balanced (equal ToR and FromR)
Recurses into If branches for conditional tail calls
TAIL CALL (last pass, must be last):
trailing Call(id) -> TailCall(id) if return stack balanced
(equal ToR / FromR pairs).
Recurses into If branches for conditional tail calls.
STACK-TO-LOCAL PROMOTION (codegen pass, not optimizer):
Words whose effects on the data stack can be statically
tracked are compiled to use WASM locals 1..s instead of
DSP loads/stores. Triggered by `is_promotable(body)`.
DSP is still written back before any Call so callees and
host functions see a consistent stack.
8. CONSOLIDATION
----------------
9. CONSOLIDATION (consolidate.rs + codegen.rs)
----------------------------------------------
CONSOLIDATE word recompiles all JIT-compiled words into a
single WASM module:
- All call_indirect -> direct call (for words in module)
- External calls (host functions) remain call_indirect
- Maximum performance for final program
CONSOLIDATE recompiles every JIT-compiled word into ONE WASM
module:
- All call_indirect to consolidated words become direct
`call` (single-module direct calls)
- External calls (host functions) stay call_indirect
- Removes per-word instantiation overhead and lets the
WASM engine inline / specialize across word boundaries
Two-part implementation:
codegen::compile_consolidated_module() - builds multi-function module
outer::ForthVM::consolidate() - orchestrates collection + table update
Two parts:
codegen::compile_consolidated_module()
Builds the multi-function module.
outer::ForthVM::consolidate()
Collects ir_bodies, computes table layout, compiles,
instantiates, and patches the shared function table.
9. EXPORT PIPELINE (wafer build)
--------------------------------
10. EXPORT PIPELINE (`wafer build`)
----------------------------------
1. Evaluate source file with recording_toplevel=true
2. Collect all IR words + top-level IR
3. Determine entry: --entry flag > MAIN word > top-level execution
4. Build consolidated module with data section (memory snapshot)
5. Embed metadata in "wafer" custom section (JSON)
6. Optional: --js generates JS loader + HTML page
7. Optional: --native AOT-compiles and appends to wafer binary
Format: [wafer binary][precompiled WASM][metadata][trailer]
Trailer: payload_len(8) + metadata_len(8) + "WAFEREXE"(8)
export.rs::export_module() steps:
1. Evaluate the source file with recording_toplevel = true
2. Collect every IR word + recorded top-level IR
3. Resolve entry point (priority):
--entry <name> > MAIN > synthetic _start from the
recorded top-level
4. Snapshot WASM linear memory (system vars + dictionary +
any user data)
5. Walk the IR, find every Call/TailCall to a host word
not in the consolidated set: those become required
imports of the exported module
6. Build metadata (JSON, custom "wafer" section):
version, entry_table_index, host_functions,
memory_size, dsp/rsp/fsp_init
7. compile_exportable_module() emits the final WASM with
a passive data section seeded from the memory snapshot
8. Optional --js: also emit a JS loader + minimal HTML
9. Optional --native: AOT-compile and append to the wafer
binary itself, in this layout:
[wafer ELF/Mach-O][precompiled WASM][metadata]
[trailer: payload_len(8) | metadata_len(8) | "WAFEREXE"]
The CLI detects the trailer at startup and runs the
embedded payload directly (single-file distribution).
10. CRATE STRUCTURE
11. CRATE STRUCTURE
-------------------
crates/
core/ wafer-core: compiler, optimizer, codegen, dictionary, Runtime trait
Feature flags: default=["native"], "native" enables wasmtime
Without features: pure Rust (dictionary, IR, optimizer, codegen, outer)
cli/ wafer: CLI REPL (rustyline), wafer build/run commands
web/ wafer-web: browser REPL (wasm-bindgen + WebRuntime + HTML/CSS/JS)
core/ wafer-core: compiler, optimizer, codegen,
dictionary, runtime trait, outer interpreter.
Largest file: codegen.rs (~4.3k LOC).
Feature flags:
default = ["native"]
"native" pulls in wasmtime + NativeRuntime +
runner.rs (CLI executor) + export.rs
"crypto" enables SHA1/256/512 host words
No features: pure-Rust core for wafer-web
(dictionary, IR, optimizer, codegen,
outer interpreter only)
cli/ wafer: rustyline REPL + `wafer build` / `wafer run`
web/ wafer-web: browser REPL.
Key web files:
crates/web/src/lib.rs WaferRepl wasm-bindgen entry point
crates/web/src/runtime_web.rs WebRuntime: js_sys WebAssembly API
crates/web/www/app.js Frontend JS (terminal emulation)
crates/web/src/lib.rs WaferRepl wasm-bindgen entry
crates/web/src/runtime_web.rs WebRuntime: js_sys WebAssembly
crates/web/www/app.js Frontend (terminal emulation)
crates/web/www/index.html HTML shell
crates/web/www/style.css Styling
crates/web/www/pkg/ wasm-pack output (gitignored)
11. BOOT SEQUENCE
12. BOOT SEQUENCE
-----------------
ForthVM::<R>::new() ->
1. R::new() — create runtime (wasmtime or browser WASM)
2. register_primitives() in batch_mode:
- ~40 IR primitives (DUP, +, @, etc.)
- ~60 host functions (., .S, M*, ACCEPT, etc.)
- ~30 special words (IF, DO, :, VARIABLE, etc.)
3. compile_batch() - single WASM module for all IR primitives
4. Load boot.fth - Forth replaces Rust host functions:
Phase 1: Stack/memory (DEPTH, PICK, 2OVER, FILL, MOVE)
Phase 2: Double-cell arithmetic (D+, DNEGATE, D<)
Phase 3: Mixed arithmetic (SM/REM, FM/MOD, */, */MOD)
Phase 4: HERE, ALLOT, comma, ALIGN
Phase 5: I/O, pictured numeric output (., U., TYPE, <# # #>)
Phase 6: DEFER support
Phase 7: String operations (COMPARE, SOURCE, FALIGNED)
2. register_primitives() in batch_mode = true:
- ~110 IR primitive registrations (DUP, +, @, ...)
- ~87 host primitive registrations (., .S, M*, ACCEPT, ...)
- special interpreter tokens (IF, DO, :, VARIABLE, S",
{: :}, [: ;], CONSOLIDATE, ...) handled directly in
interpret_token_immediate / compile_token, no IR op
3. Word-set registrations:
core, double, exception, facility, file (subset),
floating-point, locals, memory, search-order,
programming-tools, string, optional crypto
4. batch_compile_deferred() — single WASM module for all
deferred IR primitives
5. Load boot.fth (include_str!), evaluated line by line so
`\` comments terminate at end-of-line:
Phase 1: stack/memory (DEPTH, PICK, 2OVER, FILL, MOVE,
CMOVE, /STRING, -TRAILING)
Phase 2: double-cell arithmetic (D+, DNEGATE, D<, D=)
Phase 3: mixed arithmetic (SM/REM, FM/MOD, */, */MOD)
Phase 4: HERE, ALLOT, comma, ALIGN, ALIGNED
Phase 5: I/O + pictured output (., U., TYPE, <# # #>,
SIGN, HOLD)
Phase 6: DEFER support (DEFER, IS, ACTION-OF)
Phase 7: more replacements (COMPARE, SOURCE, FALIGNED,
DFALIGN, structures, S" hint, ...)
13. RUNTIME-VS-EXPORT NOTE
--------------------------
Two separate codegen entry points produce multi-function
WASM modules from the same IR:
compile_consolidated_module() used by CONSOLIDATE
- Targets the live runtime
- Re-uses the shared globals/table/memory imports
- External calls remain call_indirect
compile_exportable_module() used by `wafer build`
- Targets a standalone module
- Carries its own memory (passive data section seeded
from the snapshot) and embeds metadata
- Required host functions become imports the runner
(or AOT loader) must satisfy
Both share the same per-IrOp lowering helpers; the
difference is in module-level wiring.
+47
View File
@@ -0,0 +1,47 @@
# Editor support for WAFER
Syntax highlighting assets for editors and pagers.
## bat (and other Sublime-Text-compatible tools)
`bat/WAFER.sublime-syntax` is a Sublime Text grammar covering Forth 2012 plus
WAFER-specific words (`CONSOLIDATE`, `RANDOM`, `RND-SEED`, `UTIME`).
### Install
```
just install-syntax
```
or manually:
```
mkdir -p ~/.config/bat/syntaxes
cp tools/editor-support/bat/WAFER.sublime-syntax ~/.config/bat/syntaxes/
bat cache --build
```
### Verify
```
bat --list-languages | grep -i forth # should list Forth
bat --language forth crates/core/boot.fth # should render with colour
```
### Use with `oked`
`oked` auto-detects `.fth` / `.4th` / `.forth` files and invokes `bat` with
`--language forth`. After the install step above, opening any WAFER source in
`oked` and toggling highlight (`H` command, or `oked -S forth`) will use this
syntax.
### Updating the keyword list
Primitives live in `crates/core/src/outer.rs` (`register_primitive` and
`register_host_primitive` calls). When a new **user-facing, non-standard** word
is added, append it to the `wafer_extras` context in
`bat/WAFER.sublime-syntax`. Standard Forth 2012 words are already covered by
the main contexts.
Internal symbols (names that start with `_`) should not be added — they are
implementation details that user code never types.
@@ -0,0 +1,211 @@
%YAML 1.2
---
# WAFER / Forth 2012 syntax for `bat` (and any Sublime Text compatible highlighter).
#
# Keyword list is derived from the primitives registered in
# crates/core/src/outer.rs plus the Forth 2012 core-ext wordset and the boot.fth
# definitions in crates/core/boot.fth. WAFER-specific additions are tagged below.
#
# Install: see tools/editor-support/README.md.
name: Forth
file_extensions:
- fth
- 4th
- forth
scope: source.forth
variables:
ident_break: '(?=\s|$)'
contexts:
main:
- include: comments
- include: strings
- include: numbers
- include: definitions
- include: locals
- include: structures
- include: control
- include: stack_ops
- include: return_stack
- include: arithmetic
- include: logic
- include: compare
- include: memory
- include: io
- include: pictured
- include: string_ops
- include: float
- include: tools
- include: dictionary
- include: exception
- include: parsing
- include: literals
- include: hashing
- include: wafer_extras
comments:
# Line comment: backslash to end of line, must be followed by whitespace or EOL.
- match: '(?i)(?:^|(?<=\s))\\(?=\s|$).*$'
scope: comment.line.backslash.forth
# Stack-effect / block comment: ( ... ) — the `(` must be followed by whitespace.
- match: '(?i)(?:^|(?<=\s))\((?=\s|$)'
scope: punctuation.definition.comment.forth
push:
- meta_scope: comment.block.paren.forth
- match: '\)'
scope: punctuation.definition.comment.forth
pop: true
# Immediate print comment: .( ... )
- match: '(?i)(?:^|(?<=\s))\.\((?=\s|$)'
scope: punctuation.definition.comment.forth
push:
- meta_scope: comment.block.dot-paren.forth
- match: '\)'
scope: punctuation.definition.comment.forth
pop: true
strings:
# Standard Forth strings: leading word followed by space then body, closed with ".
- match: '(?i)(?:^|(?<=\s))(S\\"|S"|C"|\."|ABORT")(\s)'
captures:
1: keyword.other.string-prefix.forth
push:
- meta_scope: string.quoted.double.forth
- match: '"'
pop: true
numbers:
# Hex / binary / decimal / char literals / negatives; all whitespace-delimited.
- match: '(?i)(?:^|(?<=\s))\$[0-9A-F]+{{ident_break}}'
scope: constant.numeric.hex.forth
- match: '(?i)(?:^|(?<=\s))#-?[0-9]+{{ident_break}}'
scope: constant.numeric.decimal.forth
- match: '(?i)(?:^|(?<=\s))%[01]+{{ident_break}}'
scope: constant.numeric.binary.forth
- match: "(?i)(?:^|(?<=\\s))'.'{{ident_break}}"
scope: constant.character.forth
- match: '(?i)(?:^|(?<=\s))-?[0-9]+(?:\.[0-9]*)?(?:[eE]-?[0-9]+)?{{ident_break}}'
scope: constant.numeric.forth
definitions:
- match: '(?i)(?:^|(?<=\s))(:|:NONAME)(\s+)(\S+)?'
captures:
1: keyword.other.definition.forth
3: entity.name.function.forth
- match: '(?i)(?:^|(?<=\s));{{ident_break}}'
scope: keyword.other.definition.forth
# Quotations (Core-Ext 6.2.0455): [: ... ;] compiles an anonymous word.
- match: '(?i)(?:^|(?<=\s))(\[:|;\]){{ident_break}}'
scope: keyword.other.definition.forth
- match: '(?i)(?:^|(?<=\s))(VARIABLE|2VARIABLE|CONSTANT|2CONSTANT|VALUE|CREATE|DEFER|MARKER|REMEMBER|BUFFER:|FCONSTANT|FVARIABLE)(\s+)(\S+)?'
captures:
1: keyword.other.defining.forth
3: entity.name.constant.forth
- match: '(?i)(?:^|(?<=\s))(DOES>|IMMEDIATE|RECURSE|POSTPONE|COMPILE,|LITERAL|2LITERAL|FLITERAL|SLITERAL|DEFER!|DEFER@){{ident_break}}'
scope: keyword.other.defining.forth
control:
- match: '(?i)(?:^|(?<=\s))(IF|THEN|ELSE|BEGIN|UNTIL|WHILE|REPEAT|AGAIN|DO|\?DO|LOOP|\+LOOP|LEAVE|UNLOOP|EXIT|CASE|OF|ENDOF|ENDCASE|QUIT){{ident_break}}'
scope: keyword.control.forth
# Conditional compilation (Tools-ext 15.6.2).
- match: '(?i)(?:^|(?<=\s))(\[IF\]|\[ELSE\]|\[THEN\]|\[DEFINED\]|\[UNDEFINED\]){{ident_break}}'
scope: keyword.control.conditional-compilation.forth
stack_ops:
- match: '(?i)(?:^|(?<=\s))(DUP|\?DUP|DROP|SWAP|OVER|ROT|-ROT|NIP|TUCK|PICK|ROLL|2DUP|2DROP|2SWAP|2OVER|2ROT|DEPTH|SP@){{ident_break}}'
scope: support.function.stack.forth
return_stack:
# RP@ / RDEPTH are WAFER extensions (gforth-style return-stack access).
- match: '(?i)(?:^|(?<=\s))(>R|R>|R@|2>R|2R>|2R@|N>R|NR>|I|J|CS-PICK|CS-ROLL|RP@|RDEPTH){{ident_break}}'
scope: support.function.return-stack.forth
arithmetic:
- match: '(?i)(?:^|(?<=\s))(\+|-|\*|/|MOD|/MOD|\*/|\*/MOD|NEGATE|ABS|MIN|MAX|1\+|1-|2\*|2/|M\*|M\+|M\*/|UM\*|UM/MOD|FM/MOD|SM/REM|S>D|D>S|D\+|D-|DNEGATE|DABS|DMAX|DMIN|D2\*|D2/){{ident_break}}'
scope: keyword.operator.arithmetic.forth
logic:
- match: '(?i)(?:^|(?<=\s))(AND|OR|XOR|INVERT|LSHIFT|RSHIFT){{ident_break}}'
scope: keyword.operator.logical.forth
compare:
- match: '(?i)(?:^|(?<=\s))(=|<>|<|>|<=|>=|U<|U>|0=|0<>|0<|0>|D<|D=|D0<|D0=|DU<|WITHIN){{ident_break}}'
scope: keyword.operator.comparison.forth
memory:
- match: '(?i)(?:^|(?<=\s))(@|!|C@|C!|\+!|2@|2!|C,|ALLOT|HERE|ALIGN|ALIGNED|CELL\+|CELLS|CHAR\+|CHARS|UNUSED|MOVE|CMOVE|CMOVE>|FILL|ERASE|BLANK|ALLOCATE|FREE|RESIZE|PAD){{ident_break}}'
scope: support.function.memory.forth
io:
- match: '(?i)(?:^|(?<=\s))(EMIT|CR|SPACE|SPACES|TYPE|\.|U\.|\.R|U\.R|D\.|D\.R|\?|KEY|KEY\?|PAGE|AT-XY|ACCEPT|EXPECT|\.S|F\.S|\.RS){{ident_break}}'
scope: support.function.io.forth
# Pictured numeric output (6.1: <# # #S #> HOLD SIGN; HOLDS is Core-Ext).
pictured:
- match: '(?i)(?:^|(?<=\s))(<#|#>|#S|#|HOLD|HOLDS|SIGN){{ident_break}}'
scope: support.function.pictured.forth
# String word set (17.6).
string_ops:
- match: '(?i)(?:^|(?<=\s))(COUNT|COMPARE|-TRAILING|/STRING){{ident_break}}'
scope: support.function.string.forth
float:
- match: '(?i)(?:^|(?<=\s))(F\+|F-|F\*\*|F\*|F/|FNEGATE|FABS|FMAX|FMIN|FSQRT|FFLOOR|FROUND|FLOOR|FSINCOS|FSINH|FSIN|FCOSH|FCOS|FTANH|FTAN|FASINH|FASIN|FACOSH|FACOS|FATANH|FATAN2|FATAN|FEXPM1|FEXP|FLNP1|FLN|FLOG|FALOG|F=|F<|F0=|F0<|F~|FDUP|FDROP|FSWAP|FOVER|FROT|FNIP|FTUCK|FDEPTH|F@|F!|FE\.|FS\.|F\.|F>D|D>F|F>S|S>F|>FLOAT|REPRESENT|PRECISION|SET-PRECISION|FALIGN|FALIGNED|DFALIGN|DFALIGNED|SFALIGN|SFALIGNED|FLOAT\+|FLOATS|DFLOAT\+|DFLOATS|SFLOAT\+|SFLOATS|DF@|DF!|SF@|SF!){{ident_break}}'
scope: support.function.float.forth
# Interactive/debug tools (Tools word set + WAFER REPL additions).
tools:
- match: '(?i)(?:^|(?<=\s))(SEE-IR|SEE|DUMP|BYE|HELP){{ident_break}}'
scope: support.function.tools.forth
dictionary:
- match: "(?i)(?:^|(?<=\\s))('|\\[']|,|>BODY|FIND|WORDS|ONLY|ALSO|PREVIOUS|DEFINITIONS|FORTH|GET-ORDER|SET-ORDER|GET-CURRENT|SET-CURRENT|WORDLIST|SEARCH-WORDLIST|FORTH-WORDLIST|ENVIRONMENT\\?|EXECUTE){{ident_break}}"
scope: support.function.dictionary.forth
exception:
- match: '(?i)(?:^|(?<=\s))(CATCH|THROW|ABORT){{ident_break}}'
scope: keyword.control.exception.forth
parsing:
- match: '(?i)(?:^|(?<=\s))(PARSE|PARSE-NAME|WORD|REFILL|EVALUATE|INCLUDE|INCLUDED|SOURCE|SOURCE-ID|>IN|BASE|DECIMAL|HEX|STATE|>NUMBER|SEARCH|SUBSTITUTE|UNESCAPE|REPLACES|S){{ident_break}}'
scope: support.function.parsing.forth
literals:
- match: '(?i)(?:^|(?<=\s))(TRUE|FALSE|BL|CHAR|\[CHAR\]|\[COMPILE\]){{ident_break}}'
scope: constant.language.forth
# Forth 2012 §13 Locals. `{: ... :}` is the user-facing form; `{F:` is the
# float-locals variant (gforth/SwiftForth-style). `(LOCAL)` is the low-level
# primitive from §13.6.1.0086; user code typically builds `LOCAL` /
# `END-LOCALS` on top of it. `TO` rebinds a VALUE or local; `LOCALS|` is the
# §13 legacy (Forth-94) form.
locals:
- match: '(?i)(?:^|(?<=\s))(\{:|:\}|\{F:|LOCALS\|){{ident_break}}'
scope: keyword.other.locals.forth
- match: '(?i)(?:^|(?<=\s))(TO|END-LOCALS){{ident_break}}'
scope: keyword.other.locals.forth
- match: '(?i)(?:^|(?<=\s))\(LOCAL\){{ident_break}}'
scope: support.function.locals.forth
# Structure words — Facility-ext 10.6.2.0935 (defined in boot.fth).
structures:
- match: '(?i)(?:^|(?<=\s))(BEGIN-STRUCTURE)(\s+)(\S+)?'
captures:
1: keyword.other.struct.forth
3: entity.name.struct.forth
- match: '(?i)(?:^|(?<=\s))(END-STRUCTURE|\+FIELD|FIELD:|CFIELD:|FFIELD:|SFFIELD:|DFFIELD:){{ident_break}}'
scope: keyword.other.struct.forth
# Hash primitives — mirrors the registry in crates/core/src/crypto.rs. When
# new algorithms are added to `crypto::ALGOS`, extend this alternation.
hashing:
- match: '(?i)(?:^|(?<=\s))(SHA1|SHA256|SHA512){{ident_break}}'
scope: support.function.hash.forth
wafer_extras:
# WAFER-specific extensions beyond the Forth 2012 standard.
# When the language grows new user-facing non-standard words, add them here.
- match: '(?i)(?:^|(?<=\s))(CONSOLIDATE|RANDOM|RND-SEED|UTIME|READ-PASSWORD|EMPTY|GILD){{ident_break}}'
scope: support.function.wafer-extra.forth