feat(core): SEE, SEE-IR, HELP introspection trio (WS-010)
Implements plans/01-see-introspection.md, all phases. - see.rs: feature-free IR pretty-printer (format_ir/format_ir_with), exhaustive over IrOp -- a new variant fails the build, not the output. - SEE-IR <name>: post-optimization IR view with resolved callee names, immediate/does> annotations; host-word and interpreter-token stubs. - SEE <name>: verbatim source capture for colon words (multi-line, comments preserved, EVALUATE-nesting safe, error-path wiped, MARKER/ REMEMBER/EMPTY roll word sources back too). Data definers (VARIABLE/ CONSTANT/CREATE/BUFFER:/2*/F*/SYNONYM) record synthesized one-liners at definition time; VALUE/2VALUE/FVALUE/DEFER synthesize at SEE time so current values and IS targets show. Fallback chain ends at IR dump or host-word stub -- SEE never dead-ends on a defined word. - HELP [<name>]: wordhelp.rs doc table with stack effect + one-line description for EVERY word in a fresh VM (300+ dictionary words plus all outer-interpreter tokens); a coverage test fails the build if a word is ever added undocumented. User words echo their leading ( ... -- ... ) comment. SEE/SEE-IR prepend the HELP line as a \ comment. Bare HELP prints usage. - boot.fth colon definitions get real sources for free (they flow through evaluate); INTERPRETER_TOKENS gained the missing ?DO. 524 unit + 11 compliance + 9 comparison + 5 crypto + 1 bench green; fmt/clippy clean; core still builds --no-default-features; web wasm-pack build unchanged.
This commit is contained in:
Oleksandr Kozachuk
@@ -79,7 +79,7 @@ Handle in `interpret_token_immediate()` or `compile_token()` as a special case.
|
||||
|
||||
## Testing
|
||||
|
||||
- Run `cargo test --workspace` before committing (currently 431 unit + 1 benchmark + 11 compliance + 9 comparison)
|
||||
- Run `cargo test --workspace` before committing (currently 524 unit + 1 benchmark + 11 compliance + 9 comparison + 5 crypto)
|
||||
- Forth 2012 compliance: `cargo test -p wafer-core --test compliance`
|
||||
- Cross-engine comparison (vs gforth): `cargo test -p wafer-core --test comparison`
|
||||
- Performance benchmarks (release mode): `cargo test -p wafer-core --test comparison -- --nocapture --ignored`
|
||||
|
||||
@@ -95,7 +95,7 @@ Times in microseconds. WAFER/gf < 1.0 means WAFER is faster. CONSOL = after `CON
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# All tests (~450 currently passing)
|
||||
# All tests (~550 currently passing)
|
||||
cargo test --workspace
|
||||
|
||||
# Forth 2012 compliance suite
|
||||
@@ -185,6 +185,7 @@ Over 200 words are implemented across the following categories:
|
||||
| Strings | `COMPARE SEARCH SLITERAL REPLACES SUBSTITUTE UNESCAPE` |
|
||||
| Floating-Pt | `F+ F- F* F/ FABS FNEGATE FSQRT FSIN FCOS FTAN FEXP FLOG FMIN FMAX` and 55+ more |
|
||||
| Case | `CASE OF ENDOF ENDCASE` |
|
||||
| Tools | `WORDS SEE SEE-IR HELP .S F.S ? DUMP MARKER REMEMBER EMPTY GILD BYE` |
|
||||
|
||||
## Web REPL
|
||||
|
||||
|
||||
@@ -24,6 +24,8 @@ pub mod ir;
|
||||
pub mod memory;
|
||||
pub mod optimizer;
|
||||
pub mod runtime;
|
||||
pub mod see;
|
||||
pub mod wordhelp;
|
||||
|
||||
// Outer interpreter: runtime-agnostic, works with any Runtime impl
|
||||
#[allow(trivial_numeric_casts, clippy::unnecessary_cast)]
|
||||
|
||||
+635
-9
@@ -161,6 +161,96 @@ struct DoesDefinition {
|
||||
has_create: bool,
|
||||
}
|
||||
|
||||
/// Tokens handled directly by the outer interpreter (`interpret_token`,
|
||||
/// `interpret_token_immediate`, `compile_token` hardcoded match arms).
|
||||
/// Several have no dictionary entry at all; SEE/SEE-IR/HELP explain them
|
||||
/// instead of erroring. Keep in sync with those match arms.
|
||||
pub(crate) const INTERPRETER_TOKENS: &[&str] = &[
|
||||
// Definition structure
|
||||
":",
|
||||
":NONAME",
|
||||
";",
|
||||
"[:",
|
||||
";]",
|
||||
"[",
|
||||
"]",
|
||||
"{:",
|
||||
// Conditional compilation
|
||||
"[IF]",
|
||||
"[ELSE]",
|
||||
"[THEN]",
|
||||
"[DEFINED]",
|
||||
"[UNDEFINED]",
|
||||
// Strings + comments
|
||||
".\"",
|
||||
".(",
|
||||
"S\"",
|
||||
"S\\\"",
|
||||
"C\"",
|
||||
"S",
|
||||
"(",
|
||||
"\\",
|
||||
"ABORT\"",
|
||||
// Defining words
|
||||
"VARIABLE",
|
||||
"CONSTANT",
|
||||
"CREATE",
|
||||
"VALUE",
|
||||
"DOES>",
|
||||
"2CONSTANT",
|
||||
"2VARIABLE",
|
||||
"2VALUE",
|
||||
"FVARIABLE",
|
||||
"FCONSTANT",
|
||||
"FVALUE",
|
||||
"BUFFER:",
|
||||
"MARKER",
|
||||
"REMEMBER",
|
||||
"GILD",
|
||||
"EMPTY",
|
||||
"SYNONYM",
|
||||
"CONSOLIDATE",
|
||||
// Parsing words
|
||||
"'",
|
||||
"[']",
|
||||
"CHAR",
|
||||
"[CHAR]",
|
||||
"EVALUATE",
|
||||
"WORD",
|
||||
"TO",
|
||||
"IS",
|
||||
"ACTION-OF",
|
||||
"PARSE",
|
||||
"PARSE-NAME",
|
||||
"REFILL",
|
||||
"ORDER",
|
||||
// Compile-mode control flow
|
||||
"IF",
|
||||
"ELSE",
|
||||
"THEN",
|
||||
"DO",
|
||||
"?DO",
|
||||
"LOOP",
|
||||
"+LOOP",
|
||||
"BEGIN",
|
||||
"UNTIL",
|
||||
"AGAIN",
|
||||
"WHILE",
|
||||
"REPEAT",
|
||||
"AHEAD",
|
||||
"CASE",
|
||||
"OF",
|
||||
"ENDOF",
|
||||
"ENDCASE",
|
||||
"RECURSE",
|
||||
"EXIT",
|
||||
"LITERAL",
|
||||
"2LITERAL",
|
||||
"FLITERAL",
|
||||
"SLITERAL",
|
||||
"POSTPONE",
|
||||
];
|
||||
|
||||
/// Saved VM state for a MARKER word.
|
||||
#[derive(Clone)]
|
||||
struct MarkerState {
|
||||
@@ -171,6 +261,7 @@ struct MarkerState {
|
||||
ir_bodies: HashMap<WordId, Vec<IrOp>>,
|
||||
does_definitions: HashMap<WordId, DoesDefinition>,
|
||||
host_word_names: HashMap<WordId, String>,
|
||||
word_sources: HashMap<WordId, String>,
|
||||
two_value_words: std::collections::HashSet<u32>,
|
||||
fvalue_words: std::collections::HashSet<u32>,
|
||||
// Namespace + text state: search order, wordlist allocation,
|
||||
@@ -204,6 +295,14 @@ pub struct ForthVM<R: Runtime> {
|
||||
compiling_ir: Vec<IrOp>,
|
||||
control_stack: Vec<ControlEntry>,
|
||||
compiling_word_id: Option<WordId>,
|
||||
// SEE source capture: verbatim text of the colon definition in progress
|
||||
// (accumulated across evaluate() calls), the position in the CURRENT
|
||||
// input buffer where capture (re)starts, the byte offset of the most
|
||||
// recently read token, and completed sources by word id.
|
||||
compiling_source: String,
|
||||
source_capture_from: Option<usize>,
|
||||
last_token_start: usize,
|
||||
word_sources: HashMap<WordId, String>,
|
||||
// Output buffer
|
||||
output: Arc<Mutex<String>>,
|
||||
// Next table index (mirrors dictionary.next_fn_index conceptually,
|
||||
@@ -230,7 +329,11 @@ pub struct ForthVM<R: Runtime> {
|
||||
// True when CREATE appeared in the current colon definition before DOES>
|
||||
saw_create_in_def: bool,
|
||||
// Pending action from compiled defining/parsing words
|
||||
// 0 = none, 1 = CONSTANT, 2 = VARIABLE, 3 = CREATE, 4 = EVALUATE
|
||||
// 0 = none, 1 = CONSTANT, 2 = VARIABLE, 3 = CREATE, 4 = EVALUATE,
|
||||
// 5 = WORD, 6 = FIND, 7 = PARSE, 8 = PARSE-NAME, 9 = 2CONSTANT,
|
||||
// 10 = 2VARIABLE, 11 = DEFER, 12 = IMMEDIATE, 20 = GET-CURRENT,
|
||||
// 21 = SET-CURRENT, 25 = SEARCH-WORDLIST, 33 = DEFINITIONS,
|
||||
// 40 = WORDS, 41 = SEE, 42 = SEE-IR, 43 = HELP
|
||||
pending_define: Arc<Mutex<Vec<i32>>>,
|
||||
/// Pending actions from host functions (COMPILE,, CS-PICK, CS-ROLL, POSTPONE of control words).
|
||||
pending_actions: Arc<Mutex<Vec<PendingAction>>>,
|
||||
@@ -440,6 +543,10 @@ impl<R: Runtime> ForthVM<R> {
|
||||
compiling_ir: Vec::new(),
|
||||
control_stack: Vec::new(),
|
||||
compiling_word_id: None,
|
||||
compiling_source: String::new(),
|
||||
source_capture_from: None,
|
||||
last_token_start: 0,
|
||||
word_sources: HashMap::new(),
|
||||
output,
|
||||
next_table_index: 0,
|
||||
host_word_names: HashMap::new(),
|
||||
@@ -535,6 +642,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
self.compiling_local_kinds.clear();
|
||||
self.local_batch_base = None;
|
||||
self.compile_frames.clear();
|
||||
self.compiling_source.clear();
|
||||
self.source_capture_from = None;
|
||||
return Err(self.describe_uncaught(e));
|
||||
}
|
||||
}
|
||||
@@ -554,6 +663,17 @@ impl<R: Runtime> ForthVM<R> {
|
||||
}
|
||||
}
|
||||
|
||||
// Multi-line definition: bank this buffer's tail into the capture
|
||||
// and continue from the start of the next buffer.
|
||||
if self.state != 0
|
||||
&& let Some(from) = self.source_capture_from
|
||||
{
|
||||
let from = from.min(self.input_buffer.len());
|
||||
self.compiling_source.push_str(&self.input_buffer[from..]);
|
||||
self.compiling_source.push('\n');
|
||||
self.source_capture_from = Some(0);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -701,6 +821,7 @@ impl<R: Runtime> ForthVM<R> {
|
||||
return None;
|
||||
}
|
||||
let start = self.input_pos;
|
||||
self.last_token_start = start;
|
||||
while self.input_pos < bytes.len() && !bytes[self.input_pos].is_ascii_whitespace() {
|
||||
self.input_pos += 1;
|
||||
}
|
||||
@@ -2102,6 +2223,10 @@ impl<R: Runtime> ForthVM<R> {
|
||||
if self.state != 0 {
|
||||
anyhow::bail!("nested colon definitions not allowed");
|
||||
}
|
||||
// SEE source capture starts at the `:` token itself (its position
|
||||
// was recorded by next_token before dispatch reached us).
|
||||
self.compiling_source.clear();
|
||||
self.source_capture_from = Some(self.last_token_start);
|
||||
let name = self
|
||||
.next_token()
|
||||
.ok_or_else(|| anyhow::anyhow!("expected word name after :"))?;
|
||||
@@ -2296,6 +2421,16 @@ impl<R: Runtime> ForthVM<R> {
|
||||
.compiling_word_id
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("no word being compiled"))?;
|
||||
// SEE: bank the tail of the current buffer through the `;` token.
|
||||
// Capture is only armed by `:` — :NONAME and quotations never store.
|
||||
if let Some(from) = self.source_capture_from.take() {
|
||||
let end = self.input_pos.min(self.input_buffer.len());
|
||||
let mut src = std::mem::take(&mut self.compiling_source);
|
||||
src.push_str(&self.input_buffer[from.min(end)..end]);
|
||||
self.word_sources
|
||||
.insert(word_id, src.trim_end().to_string());
|
||||
}
|
||||
|
||||
let ir = std::mem::take(&mut self.compiling_ir);
|
||||
let bodies = self.ir_bodies.clone();
|
||||
let ir = self.optimize_ir(ir, &bodies);
|
||||
@@ -3229,6 +3364,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
// Compile a tiny word that pushes the variable's address
|
||||
let ir_body = vec![IrOp::PushI32(var_addr as i32)];
|
||||
self.ir_bodies.insert(word_id, ir_body.clone());
|
||||
self.word_sources
|
||||
.insert(word_id, format!("VARIABLE {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir_body, &config)
|
||||
.map_err(|e| anyhow::anyhow!("codegen error for VARIABLE {name}: {e}"))?;
|
||||
@@ -3257,6 +3394,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
// Compile a word that pushes the constant value
|
||||
let ir_body = vec![IrOp::PushI32(value)];
|
||||
self.ir_bodies.insert(word_id, ir_body.clone());
|
||||
self.word_sources
|
||||
.insert(word_id, format!("{value} CONSTANT {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir_body, &config)
|
||||
.map_err(|e| anyhow::anyhow!("codegen error for CONSTANT {name}: {e}"))?;
|
||||
@@ -3290,6 +3429,7 @@ impl<R: Runtime> ForthVM<R> {
|
||||
// Compile a word that pushes the pfa
|
||||
let ir_body = vec![IrOp::PushI32(pfa as i32)];
|
||||
self.ir_bodies.insert(word_id, ir_body.clone());
|
||||
self.word_sources.insert(word_id, format!("CREATE {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir_body, &config)
|
||||
.map_err(|e| anyhow::anyhow!("codegen error for CREATE {name}: {e}"))?;
|
||||
@@ -3403,6 +3543,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
|
||||
let ir_body = vec![IrOp::Call(word_id)];
|
||||
self.ir_bodies.insert(new_word_id, ir_body.clone());
|
||||
self.word_sources
|
||||
.insert(new_word_id, format!("SYNONYM {new_name} {old_name}"));
|
||||
let config = self.codegen_config(new_word_id.0);
|
||||
let compiled = compile_word(&new_name, &ir_body, &config)
|
||||
.map_err(|e| anyhow::anyhow!("codegen error for SYNONYM: {e}"))?;
|
||||
@@ -3451,6 +3593,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
// Compile a word that pushes the buffer address
|
||||
let ir_body = vec![IrOp::PushI32(buf_addr as i32)];
|
||||
self.ir_bodies.insert(word_id, ir_body.clone());
|
||||
self.word_sources
|
||||
.insert(word_id, format!("{size} BUFFER: {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir_body, &config)
|
||||
.map_err(|e| anyhow::anyhow!("codegen error for BUFFER: {name}: {e}"))?;
|
||||
@@ -3477,6 +3621,7 @@ impl<R: Runtime> ForthVM<R> {
|
||||
ir_bodies: self.ir_bodies.clone(),
|
||||
does_definitions: self.does_definitions.clone(),
|
||||
host_word_names: self.host_word_names.clone(),
|
||||
word_sources: self.word_sources.clone(),
|
||||
two_value_words: self.two_value_words.clone(),
|
||||
fvalue_words: self.fvalue_words.clone(),
|
||||
search_order: self.search_order.lock().unwrap().clone(),
|
||||
@@ -3499,6 +3644,7 @@ impl<R: Runtime> ForthVM<R> {
|
||||
self.ir_bodies = state.ir_bodies;
|
||||
self.does_definitions = state.does_definitions;
|
||||
self.host_word_names = state.host_word_names;
|
||||
self.word_sources = state.word_sources;
|
||||
self.two_value_words = state.two_value_words;
|
||||
self.fvalue_words = state.fvalue_words;
|
||||
*self.search_order.lock().unwrap() = state.search_order;
|
||||
@@ -4316,9 +4462,21 @@ impl<R: Runtime> ForthVM<R> {
|
||||
}
|
||||
}
|
||||
|
||||
// A definition left open by the EVALUATEd string: bank its tail and
|
||||
// re-anchor capture at the resume point of the restored buffer.
|
||||
let capture_open = self.state != 0 && self.source_capture_from.is_some();
|
||||
if let Some(from) = self.source_capture_from.filter(|_| self.state != 0) {
|
||||
let from = from.min(self.input_buffer.len());
|
||||
self.compiling_source.push_str(&self.input_buffer[from..]);
|
||||
self.compiling_source.push('\n');
|
||||
}
|
||||
|
||||
// Restore input state, SOURCE-ID, and sync back to WASM
|
||||
self.input_buffer = saved_buffer;
|
||||
self.input_pos = saved_pos;
|
||||
if capture_open {
|
||||
self.source_capture_from = Some(self.input_pos);
|
||||
}
|
||||
{
|
||||
let bytes = self.input_buffer.as_bytes();
|
||||
let len = bytes.len().min(INPUT_BUFFER_SIZE as usize);
|
||||
@@ -5299,6 +5457,9 @@ impl<R: Runtime> ForthVM<R> {
|
||||
}
|
||||
}
|
||||
40 => self.do_words(),
|
||||
41 => self.do_see()?,
|
||||
42 => self.do_see_ir()?,
|
||||
43 => self.do_help()?,
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
@@ -6042,6 +6203,184 @@ impl<R: Runtime> ForthVM<R> {
|
||||
out.push_str(&format!("\n{shown} words\n"));
|
||||
}
|
||||
|
||||
/// Map function-table index -> word name via a dictionary walk.
|
||||
/// Newest-first, so redefinitions resolve to the visible name.
|
||||
fn word_id_names(&self) -> HashMap<u32, String> {
|
||||
let mut map = HashMap::new();
|
||||
let mut addr = self.dictionary.latest();
|
||||
while addr != 0 {
|
||||
if let (Ok(name), Ok(code)) = (
|
||||
self.dictionary.word_name(addr),
|
||||
self.dictionary.code_field(addr),
|
||||
) {
|
||||
map.entry(code).or_insert(name);
|
||||
}
|
||||
let link = self.dictionary.read_link(addr);
|
||||
if link == addr {
|
||||
break;
|
||||
}
|
||||
addr = link;
|
||||
}
|
||||
map
|
||||
}
|
||||
|
||||
/// Parse the mandatory word-name argument of SEE/SEE-IR/HELP.
|
||||
fn parse_name_arg(&mut self, who: &str) -> anyhow::Result<String> {
|
||||
self.next_token()
|
||||
.ok_or_else(|| anyhow::anyhow!("{who}: expected word name"))
|
||||
}
|
||||
|
||||
/// `HELP [name]` — stack effect + description from the doc table; user
|
||||
/// words echo their leading `( ... -- ... )` comment from the captured
|
||||
/// source. Bare HELP prints usage.
|
||||
fn do_help(&mut self) -> anyhow::Result<()> {
|
||||
// Like WORDS' filter, the name is read from the same line (optional).
|
||||
let name = if self.state == 0 {
|
||||
self.next_token()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let Some(name) = name else {
|
||||
self.output.lock().unwrap().push_str(
|
||||
"HELP <word> -- stack effect + description. \
|
||||
Also try: WORDS, SEE <word>, SEE-IR <word>\n",
|
||||
);
|
||||
return Ok(());
|
||||
};
|
||||
let upper = name.to_ascii_uppercase();
|
||||
let found = self.dictionary.find(&upper);
|
||||
let mut line = if let Some((effect, desc)) = crate::wordhelp::lookup(&upper) {
|
||||
format!("{upper} {effect} {desc}")
|
||||
} else if let Some((_, word_id, _)) = found {
|
||||
match self
|
||||
.word_sources
|
||||
.get(&word_id)
|
||||
.and_then(|s| crate::wordhelp::stack_comment(s))
|
||||
{
|
||||
Some(effect) => {
|
||||
format!("{upper} {effect} user word; SEE {upper} shows the source")
|
||||
}
|
||||
None => format!("no help for {upper}; try SEE {upper}"),
|
||||
}
|
||||
} else if INTERPRETER_TOKENS.contains(&upper.as_str()) {
|
||||
format!("{upper} is handled by the outer interpreter")
|
||||
} else {
|
||||
anyhow::bail!("HELP: unknown word: {name}");
|
||||
};
|
||||
if found.is_some_and(|(_, _, imm)| imm) {
|
||||
line.push_str(" immediate");
|
||||
}
|
||||
line.push('\n');
|
||||
self.output.lock().unwrap().push_str(&line);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `SEE name` — print captured source, a synthesized definition for
|
||||
/// data words, or an IR/stub fallback. Never dead-ends on a defined word.
|
||||
fn do_see(&mut self) -> anyhow::Result<()> {
|
||||
let name = self.parse_name_arg("SEE")?;
|
||||
let upper = name.to_ascii_uppercase();
|
||||
let Some((addr, word_id, is_immediate)) = self.dictionary.find(&upper) else {
|
||||
if INTERPRETER_TOKENS.contains(&upper.as_str()) {
|
||||
self.output.lock().unwrap().push_str(&format!(
|
||||
"SEE: {upper} is handled by the outer interpreter (compiler word)\n"
|
||||
));
|
||||
return Ok(());
|
||||
}
|
||||
anyhow::bail!("SEE: unknown word: {name}");
|
||||
};
|
||||
let stored_name = self.dictionary.word_name(addr).unwrap_or(upper);
|
||||
// Built-in words carry their HELP line as a leading comment.
|
||||
let help = crate::wordhelp::lookup(&stored_name)
|
||||
.map(|(effect, desc)| format!("\\ {stored_name} {effect} {desc}\n"))
|
||||
.unwrap_or_default();
|
||||
let mut text = if let Some(src) = self.word_sources.get(&word_id) {
|
||||
src.clone()
|
||||
} else if let Some(synth) = self.synthesize_data_word(&stored_name, word_id) {
|
||||
synth
|
||||
} else if let Some(body) = self.ir_bodies.get(&word_id) {
|
||||
let names = self.word_id_names();
|
||||
let ir = crate::see::format_ir_with(body, &|id| names.get(&id.0).cloned());
|
||||
format!("\\ {stored_name} is a primitive; IR:\n{}", ir.trim_end())
|
||||
} else if self.host_word_names.contains_key(&word_id) {
|
||||
format!("\\ {stored_name} is a built-in host word")
|
||||
} else {
|
||||
format!("\\ {stored_name}: no source available")
|
||||
};
|
||||
if is_immediate {
|
||||
text.push_str("\nimmediate");
|
||||
}
|
||||
text.push('\n');
|
||||
self.output.lock().unwrap().push_str(&(help + &text));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Synthesize `SEE` output for mutable data words (VALUE family, DEFER)
|
||||
/// whose current value lives in WASM memory. Gated on `word_pfa_map` so
|
||||
/// address-pushing primitives (BASE, ...) never masquerade as data
|
||||
/// words. A DOES>-product whose body happens to match a VALUE shape
|
||||
/// prints as one — behaviorally equivalent, provenance lost.
|
||||
fn synthesize_data_word(&mut self, name: &str, word_id: WordId) -> Option<String> {
|
||||
let &pfa = self.word_pfa_map.get(&word_id.0)?;
|
||||
if self.two_value_words.contains(&word_id.0) {
|
||||
let lo = self.rt.mem_read_i32(pfa);
|
||||
let hi = self.rt.mem_read_i32(pfa + CELL_SIZE);
|
||||
return Some(format!("{lo} {hi} 2VALUE {name}"));
|
||||
}
|
||||
if self.fvalue_words.contains(&word_id.0) {
|
||||
let bytes: [u8; 8] = self.rt.mem_read_slice(pfa, 8).try_into().ok()?;
|
||||
let r = f64::from_le_bytes(bytes);
|
||||
return Some(format!("{r:e} FVALUE {name}"));
|
||||
}
|
||||
match self.ir_bodies.get(&word_id)?.as_slice() {
|
||||
[IrOp::PushI32(addr), IrOp::Fetch] => {
|
||||
let cur = self.rt.mem_read_i32(*addr as u32);
|
||||
Some(format!("{cur} VALUE {name}"))
|
||||
}
|
||||
[IrOp::PushI32(addr), IrOp::Fetch, IrOp::Execute] => {
|
||||
let xt = self.rt.mem_read_i32(*addr as u32) as u32;
|
||||
Some(match self.word_id_names().get(&xt) {
|
||||
Some(t) => format!("DEFER {name} ( IS {t} )"),
|
||||
None => format!("DEFER {name}"),
|
||||
})
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// `SEE-IR name` — print the stored post-optimization IR of a word.
|
||||
fn do_see_ir(&mut self) -> anyhow::Result<()> {
|
||||
let name = self.parse_name_arg("SEE-IR")?;
|
||||
let upper = name.to_ascii_uppercase();
|
||||
let help = crate::wordhelp::lookup(&upper)
|
||||
.map(|(effect, desc)| format!("\\ {upper} {effect} {desc}\n"))
|
||||
.unwrap_or_default();
|
||||
let text = if let Some((_addr, word_id, is_immediate)) = self.dictionary.find(&upper) {
|
||||
if let Some(body) = self.ir_bodies.get(&word_id) {
|
||||
let mut header = format!("\\ {upper} -- {} ops (optimized IR)", body.len());
|
||||
if is_immediate {
|
||||
header.push_str(" immediate");
|
||||
}
|
||||
if self.does_definitions.contains_key(&word_id) {
|
||||
header.push_str(" does>");
|
||||
}
|
||||
header.push('\n');
|
||||
let names = self.word_id_names();
|
||||
header + &crate::see::format_ir_with(body, &|id| names.get(&id.0).cloned())
|
||||
} else if self.host_word_names.contains_key(&word_id) {
|
||||
format!("SEE-IR: {upper} is a built-in host word\n")
|
||||
} else {
|
||||
format!("SEE-IR: {upper} has no IR body\n")
|
||||
}
|
||||
} else if INTERPRETER_TOKENS.contains(&upper.as_str()) {
|
||||
format!("SEE-IR: {upper} is handled directly by the outer interpreter\n")
|
||||
} else {
|
||||
anyhow::bail!("SEE-IR: unknown word: {name}");
|
||||
};
|
||||
self.output.lock().unwrap().push_str(&(help + &text));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Register Search-Order word set words.
|
||||
fn register_search_order(&mut self) -> anyhow::Result<()> {
|
||||
// FORTH-WORDLIST ( -- wid )
|
||||
@@ -6267,14 +6606,18 @@ impl<R: Runtime> ForthVM<R> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Register WORDS for the Programming-Tools word set.
|
||||
/// Register WORDS / SEE-IR for the Programming-Tools word set.
|
||||
/// Each runs Rust-side via `pending_define` so it can parse arguments
|
||||
/// with `next_token()` and write to `self.output`.
|
||||
fn register_words(&mut self) -> anyhow::Result<()> {
|
||||
let pending = Arc::clone(&self.pending_define);
|
||||
let func: HostFn = Box::new(move |_ctx: &mut dyn HostAccess| {
|
||||
pending.lock().unwrap().push(40); // WORDS action
|
||||
Ok(())
|
||||
});
|
||||
self.register_host_primitive("WORDS", false, func)?;
|
||||
for (name, code) in [("WORDS", 40), ("SEE", 41), ("SEE-IR", 42), ("HELP", 43)] {
|
||||
let pending = Arc::clone(&self.pending_define);
|
||||
let func: HostFn = Box::new(move |_ctx: &mut dyn HostAccess| {
|
||||
pending.lock().unwrap().push(code);
|
||||
Ok(())
|
||||
});
|
||||
self.register_host_primitive(name, false, func)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -6484,6 +6827,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
|
||||
let ir = vec![IrOp::PushI32(lo), IrOp::PushI32(hi)];
|
||||
self.ir_bodies.insert(word_id, ir.clone());
|
||||
self.word_sources
|
||||
.insert(word_id, format!("{lo} {hi} 2CONSTANT {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir, &config)
|
||||
.map_err(|e| anyhow::anyhow!("2CONSTANT codegen: {e}"))?;
|
||||
@@ -6510,6 +6855,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
|
||||
let ir = vec![IrOp::PushI32(addr as i32)];
|
||||
self.ir_bodies.insert(word_id, ir.clone());
|
||||
self.word_sources
|
||||
.insert(word_id, format!("2VARIABLE {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir, &config)
|
||||
.map_err(|e| anyhow::anyhow!("2VARIABLE codegen: {e}"))?;
|
||||
@@ -7299,6 +7646,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
// Compile a word that pushes the address onto the DATA stack
|
||||
let ir_body = vec![IrOp::PushI32(addr as i32)];
|
||||
self.ir_bodies.insert(word_id, ir_body.clone());
|
||||
self.word_sources
|
||||
.insert(word_id, format!("FVARIABLE {name}"));
|
||||
let config = self.codegen_config(word_id.0);
|
||||
let compiled = compile_word(&name, &ir_body, &config)
|
||||
.map_err(|e| anyhow::anyhow!("codegen error for FVARIABLE {name}: {e}"))?;
|
||||
@@ -7339,6 +7688,8 @@ impl<R: Runtime> ForthVM<R> {
|
||||
self.rt.ensure_table_size(word_id.0)?;
|
||||
self.rt.register_host_func(word_id.0, func)?;
|
||||
self.dictionary.reveal();
|
||||
self.word_sources
|
||||
.insert(word_id, format!("{val:e} FCONSTANT {name}"));
|
||||
self.sync_word_lookup(&name, word_id, false);
|
||||
self.next_table_index = self.next_table_index.max(word_id.0 + 1);
|
||||
|
||||
@@ -9000,6 +9351,281 @@ mod tests {
|
||||
assert!(!output.contains("__CTRL__"));
|
||||
}
|
||||
|
||||
// -- HELP --
|
||||
|
||||
#[test]
|
||||
fn test_help_documented_word() {
|
||||
let output = eval_output("HELP DUP");
|
||||
assert_eq!(
|
||||
output,
|
||||
"DUP ( x -- x x ) Duplicate the top of the data stack.\n"
|
||||
);
|
||||
// Case-insensitive.
|
||||
assert_eq!(eval_output("HELP dup"), output);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_help_bare_prints_usage() {
|
||||
let output = eval_output("HELP");
|
||||
assert!(output.contains("HELP <word>"), "{output}");
|
||||
assert!(output.contains("SEE <word>"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_help_user_word_echoes_stack_comment() {
|
||||
let output = eval_output(": SQ ( n -- n^2 ) DUP * ; HELP SQ");
|
||||
assert!(output.contains("SQ ( n -- n^2 )"), "{output}");
|
||||
assert!(output.contains("SEE SQ"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_help_undocumented_user_word_hints_see() {
|
||||
let output = eval_output(": MYW 1 ; HELP MYW");
|
||||
assert_eq!(output, "no help for MYW; try SEE MYW\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_help_immediate_marker() {
|
||||
let output = eval_output(": IMH 1 ; IMMEDIATE HELP IMH");
|
||||
assert!(output.contains("immediate"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_help_unknown_word_errors() {
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
let err = vm.evaluate("HELP NOSUCHWORD").unwrap_err();
|
||||
assert!(err.to_string().contains("HELP: unknown word: NOSUCHWORD"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_help_covers_every_word_in_fresh_vm() {
|
||||
// Total-coverage gate: every visible dictionary word and every
|
||||
// outer-interpreter token must have a WORD_DOCS entry, and every
|
||||
// entry must resolve back to a real word or token.
|
||||
let vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
let mut missing: Vec<String> = Vec::new();
|
||||
let mut names = vm.word_names();
|
||||
names.extend(INTERPRETER_TOKENS.iter().map(ToString::to_string));
|
||||
for name in &names {
|
||||
if crate::wordhelp::lookup(name).is_none() {
|
||||
missing.push(name.clone());
|
||||
}
|
||||
}
|
||||
missing.sort();
|
||||
missing.dedup();
|
||||
assert!(missing.is_empty(), "words without HELP docs: {missing:?}");
|
||||
|
||||
for (name, effect, desc) in crate::wordhelp::WORD_DOCS {
|
||||
let known = vm.dictionary.find(&name.to_ascii_uppercase()).is_some()
|
||||
|| INTERPRETER_TOKENS
|
||||
.iter()
|
||||
.any(|t| t.eq_ignore_ascii_case(name));
|
||||
// SHA words vanish without the crypto feature; keep their docs.
|
||||
let feature_gated = !cfg!(feature = "crypto") && name.starts_with("SHA");
|
||||
assert!(
|
||||
known || feature_gated,
|
||||
"WORD_DOCS entry for nonexistent word: {name}"
|
||||
);
|
||||
assert!(!desc.is_empty(), "empty description for {name}");
|
||||
let e = *effect;
|
||||
assert!(
|
||||
e.starts_with('(') && e.ends_with(')'),
|
||||
"malformed stack effect for {name}: {e:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// -- SEE --
|
||||
|
||||
#[test]
|
||||
fn test_see_colon_word_verbatim() {
|
||||
let output = eval_output(": SQ DUP * ; SEE SQ");
|
||||
assert_eq!(output, ": SQ DUP * ;\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_multiline_definition() {
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
vm.evaluate(": TRI").unwrap();
|
||||
vm.evaluate(" DUP DUP ;").unwrap();
|
||||
vm.evaluate("SEE TRI").unwrap();
|
||||
assert_eq!(vm.take_output(), ": TRI\n DUP DUP ;\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_comment_survives() {
|
||||
let output = eval_output(": C ( n -- n ) 1+ ; SEE C");
|
||||
assert!(output.contains("( n -- n )"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_data_words() {
|
||||
assert_eq!(eval_output("42 CONSTANT A SEE A"), "42 CONSTANT A\n");
|
||||
assert_eq!(eval_output("VARIABLE V SEE V"), "VARIABLE V\n");
|
||||
assert_eq!(eval_output("CREATE CR8 SEE CR8"), "CREATE CR8\n");
|
||||
assert_eq!(eval_output("16 BUFFER: B SEE B"), "16 BUFFER: B\n");
|
||||
assert_eq!(eval_output("1 2 2CONSTANT D2 SEE D2"), "1 2 2CONSTANT D2\n");
|
||||
assert_eq!(
|
||||
eval_output("SYNONYM NEWDUP DUP SEE NEWDUP"),
|
||||
"SYNONYM NEWDUP DUP\n"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_value_shows_current() {
|
||||
assert_eq!(eval_output("5 VALUE X SEE X"), "5 VALUE X\n");
|
||||
assert_eq!(eval_output("5 VALUE X 9 TO X SEE X"), "9 VALUE X\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_defer_shows_target() {
|
||||
let output = eval_output("DEFER D ' DUP IS D SEE D");
|
||||
assert_eq!(output, "DEFER D ( IS DUP )\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_boot_word_shows_source() {
|
||||
// WITHIN is defined in boot.fth as a colon word; SEE must show
|
||||
// real source (with its HELP header line), not an IR dump.
|
||||
let output = eval_output("SEE WITHIN");
|
||||
assert!(output.starts_with("\\ WITHIN ("), "{output}");
|
||||
assert!(
|
||||
output.ends_with(": WITHIN OVER - >R - R> U< ;\n"),
|
||||
"{output}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_primitive_ir_fallback() {
|
||||
let output = eval_output("SEE DUP");
|
||||
assert!(output.contains("DUP is a primitive; IR:"), "{output}");
|
||||
assert!(output.contains("dup"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_host_word_and_interpreter_token() {
|
||||
let output = eval_output("SEE WORDS");
|
||||
assert!(output.contains("WORDS is a built-in host word"), "{output}");
|
||||
// `:` has no dictionary entry — outer-interpreter stub.
|
||||
let output = eval_output("SEE :");
|
||||
assert!(
|
||||
output.contains(": is handled by the outer interpreter"),
|
||||
"{output}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_immediate_flag() {
|
||||
let output = eval_output(": I2 ; IMMEDIATE SEE I2");
|
||||
assert!(output.contains(": I2 ;\nimmediate"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_unknown_word_errors() {
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
let err = vm.evaluate("SEE NOSUCHWORD").unwrap_err();
|
||||
assert!(err.to_string().contains("SEE: unknown word: NOSUCHWORD"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_error_path_no_capture_debris() {
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
// Force an unknown-word error mid-definition, then define fresh.
|
||||
assert!(vm.evaluate(": BAD NOSUCHWORD ;").is_err());
|
||||
vm.evaluate(": GOOD 1 ; SEE GOOD").unwrap();
|
||||
assert_eq!(vm.take_output(), ": GOOD 1 ;\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_marker_roundtrip_restores_source() {
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
vm.evaluate(": W 1 ; MARKER MK : W 2 ;").unwrap();
|
||||
vm.evaluate("SEE W").unwrap();
|
||||
assert_eq!(vm.take_output(), ": W 2 ;\n");
|
||||
vm.evaluate("MK SEE W").unwrap();
|
||||
assert_eq!(vm.take_output(), ": W 1 ;\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_redefinition_shows_newest() {
|
||||
let output = eval_output(": R 1 ; : R 2 ; SEE R");
|
||||
assert_eq!(output, ": R 2 ;\n");
|
||||
}
|
||||
|
||||
// -- SEE-IR --
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_colon_word() {
|
||||
let output = eval_output(": SQ DUP * ; SEE-IR SQ");
|
||||
assert!(output.contains("\\ SQ -- 2 ops (optimized IR)"), "{output}");
|
||||
assert!(output.contains("dup"));
|
||||
assert!(output.contains("mul"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_shows_inlined_body() {
|
||||
let output = eval_output(": SQ DUP * ; : FOO SQ SQ ; SEE-IR FOO");
|
||||
// Inlining threshold covers SQ: FOO's stored IR has both muls inlined.
|
||||
assert_eq!(output.matches("mul").count(), 2, "{output}");
|
||||
assert!(!output.contains("call"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_resolves_callee_names() {
|
||||
// A body over the inlining threshold keeps its calls.
|
||||
let output = eval_output(
|
||||
": BIG DUP DUP DUP DUP DUP DUP DUP DUP DUP * * * * * * * * * ; \
|
||||
: USER BIG BIG ; SEE-IR USER",
|
||||
);
|
||||
assert!(
|
||||
output.contains("call BIG") || output.contains("tail-call BIG"),
|
||||
"{output}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_primitive_and_host_word() {
|
||||
let output = eval_output("SEE-IR DUP");
|
||||
assert!(output.contains("(optimized IR)"), "{output}");
|
||||
assert!(output.contains("dup"));
|
||||
let output = eval_output("SEE-IR WORDS");
|
||||
assert!(output.contains("WORDS is a built-in host word"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_control_flow_indented() {
|
||||
let output = eval_output(": T IF 1 ELSE 2 THEN ; SEE-IR T");
|
||||
assert!(
|
||||
output.contains("if\n push 1\nelse\n push 2\nthen\n"),
|
||||
"{output}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_immediate_flag() {
|
||||
let output = eval_output(": IMM 1 ; IMMEDIATE SEE-IR IMM");
|
||||
assert!(output.contains("immediate"), "{output}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_interpreter_token() {
|
||||
let output = eval_output("SEE-IR :");
|
||||
assert!(
|
||||
output.contains(": is handled directly by the outer interpreter"),
|
||||
"{output}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_see_ir_errors() {
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
let err = vm.evaluate("SEE-IR NOSUCHWORD").unwrap_err();
|
||||
assert!(err.to_string().contains("SEE-IR: unknown word: NOSUCHWORD"));
|
||||
let mut vm = ForthVM::<NativeRuntime>::new().unwrap();
|
||||
let err = vm.evaluate("SEE-IR").unwrap_err();
|
||||
assert!(err.to_string().contains("SEE-IR: expected word name"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_dot_s_honors_base() {
|
||||
assert_eq!(eval_output("HEX FF .S"), "<1> FF ");
|
||||
@@ -9096,7 +9722,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn test_stack_guards_off_config() {
|
||||
let mut cfg = crate::config::WaferConfig::all();
|
||||
let mut cfg = WaferConfig::all();
|
||||
cfg.codegen.stack_guards = false;
|
||||
let mut vm = ForthVM::<NativeRuntime>::new_with_config(cfg).unwrap();
|
||||
// Compiled DROP underflows silently (documented unguarded mode)
|
||||
|
||||
@@ -0,0 +1,374 @@
|
||||
//! IR pretty-printer for `SEE-IR` and the `SEE` fallback path.
|
||||
//!
|
||||
//! Renders a post-optimization IR body as indented, one-op-per-line text.
|
||||
//! Simple ops print as short lowercase mnemonics (Forth glyphs where they
|
||||
//! are universally recognizable: `@`, `!`, `0=`, `>r`, ...); structured ops
|
||||
//! print as Forth control words with 2-space indented bodies. Calls resolve
|
||||
//! `WordId`s to names through an optional resolver so the formatter itself
|
||||
//! stays independent of the VM.
|
||||
|
||||
use crate::dictionary::WordId;
|
||||
use crate::ir::IrOp;
|
||||
|
||||
/// Format an IR body as indented, one-op-per-line text.
|
||||
pub fn format_ir(ops: &[IrOp]) -> String {
|
||||
format_ir_with(ops, &|_| None)
|
||||
}
|
||||
|
||||
/// Like [`format_ir`], resolving `Call`/`TailCall`/`Execute` targets to word
|
||||
/// names via `resolve`; unresolved ids print as `#N`.
|
||||
pub fn format_ir_with(ops: &[IrOp], resolve: &dyn Fn(WordId) -> Option<String>) -> String {
|
||||
let mut out = String::new();
|
||||
write_ops(&mut out, ops, 0, resolve);
|
||||
out
|
||||
}
|
||||
|
||||
fn line(out: &mut String, depth: usize, text: &str) {
|
||||
for _ in 0..depth {
|
||||
out.push_str(" ");
|
||||
}
|
||||
out.push_str(text);
|
||||
out.push('\n');
|
||||
}
|
||||
|
||||
fn callee(id: WordId, resolve: &dyn Fn(WordId) -> Option<String>) -> String {
|
||||
resolve(id).unwrap_or_else(|| format!("#{}", id.0))
|
||||
}
|
||||
|
||||
fn write_ops(
|
||||
out: &mut String,
|
||||
ops: &[IrOp],
|
||||
depth: usize,
|
||||
resolve: &dyn Fn(WordId) -> Option<String>,
|
||||
) {
|
||||
for op in ops {
|
||||
write_op(out, op, depth, resolve);
|
||||
}
|
||||
}
|
||||
|
||||
fn write_op(out: &mut String, op: &IrOp, depth: usize, resolve: &dyn Fn(WordId) -> Option<String>) {
|
||||
// Exhaustive on purpose: a new IrOp variant must show up here at
|
||||
// compile time, not silently render wrong.
|
||||
let simple: String = match op {
|
||||
// -- Literals --
|
||||
IrOp::PushI32(v) => format!("push {v}"),
|
||||
IrOp::PushI64(v) => format!("push64 {v}"),
|
||||
IrOp::PushF64(v) => format!("fpush {v}"),
|
||||
|
||||
// -- Stack manipulation --
|
||||
IrOp::Drop => "drop".into(),
|
||||
IrOp::Dup => "dup".into(),
|
||||
IrOp::Swap => "swap".into(),
|
||||
IrOp::Over => "over".into(),
|
||||
IrOp::Rot => "rot".into(),
|
||||
IrOp::Nip => "nip".into(),
|
||||
IrOp::Tuck => "tuck".into(),
|
||||
IrOp::TwoDup => "2dup".into(),
|
||||
IrOp::TwoDrop => "2drop".into(),
|
||||
|
||||
// -- Arithmetic --
|
||||
IrOp::Add => "add".into(),
|
||||
IrOp::Sub => "sub".into(),
|
||||
IrOp::Mul => "mul".into(),
|
||||
IrOp::DivMod => "divmod".into(),
|
||||
IrOp::Negate => "negate".into(),
|
||||
IrOp::Abs => "abs".into(),
|
||||
|
||||
// -- Comparison --
|
||||
IrOp::Eq => "eq".into(),
|
||||
IrOp::NotEq => "ne".into(),
|
||||
IrOp::Lt => "lt".into(),
|
||||
IrOp::Gt => "gt".into(),
|
||||
IrOp::LtUnsigned => "u<".into(),
|
||||
IrOp::ZeroEq => "0=".into(),
|
||||
IrOp::ZeroLt => "0<".into(),
|
||||
|
||||
// -- Logic --
|
||||
IrOp::And => "and".into(),
|
||||
IrOp::Or => "or".into(),
|
||||
IrOp::Xor => "xor".into(),
|
||||
IrOp::Invert => "invert".into(),
|
||||
IrOp::Lshift => "lshift".into(),
|
||||
IrOp::Rshift => "rshift".into(),
|
||||
IrOp::ArithRshift => "arshift".into(),
|
||||
|
||||
// -- Memory --
|
||||
IrOp::Fetch => "@".into(),
|
||||
IrOp::Store => "!".into(),
|
||||
IrOp::CFetch => "c@".into(),
|
||||
IrOp::CStore => "c!".into(),
|
||||
IrOp::PlusStore => "+!".into(),
|
||||
|
||||
// -- Calls --
|
||||
IrOp::Call(id) => format!("call {}", callee(*id, resolve)),
|
||||
IrOp::TailCall(id) => format!("tail-call {}", callee(*id, resolve)),
|
||||
|
||||
// -- Structured control flow (multi-line) --
|
||||
IrOp::If {
|
||||
then_body,
|
||||
else_body,
|
||||
} => {
|
||||
line(out, depth, "if");
|
||||
write_ops(out, then_body, depth + 1, resolve);
|
||||
if let Some(eb) = else_body {
|
||||
line(out, depth, "else");
|
||||
write_ops(out, eb, depth + 1, resolve);
|
||||
}
|
||||
line(out, depth, "then");
|
||||
return;
|
||||
}
|
||||
IrOp::DoLoop { body, is_plus_loop } => {
|
||||
line(out, depth, "do");
|
||||
write_ops(out, body, depth + 1, resolve);
|
||||
line(out, depth, if *is_plus_loop { "+loop" } else { "loop" });
|
||||
return;
|
||||
}
|
||||
IrOp::BeginUntil { body } => {
|
||||
line(out, depth, "begin");
|
||||
write_ops(out, body, depth + 1, resolve);
|
||||
line(out, depth, "until");
|
||||
return;
|
||||
}
|
||||
IrOp::BeginAgain { body } => {
|
||||
line(out, depth, "begin");
|
||||
write_ops(out, body, depth + 1, resolve);
|
||||
line(out, depth, "again");
|
||||
return;
|
||||
}
|
||||
IrOp::BeginWhileRepeat { test, body } => {
|
||||
line(out, depth, "begin");
|
||||
write_ops(out, test, depth + 1, resolve);
|
||||
line(out, depth, "while");
|
||||
write_ops(out, body, depth + 1, resolve);
|
||||
line(out, depth, "repeat");
|
||||
return;
|
||||
}
|
||||
IrOp::BeginDoubleWhileRepeat {
|
||||
outer_test,
|
||||
inner_test,
|
||||
body,
|
||||
after_repeat,
|
||||
else_body,
|
||||
} => {
|
||||
line(out, depth, "begin");
|
||||
write_ops(out, outer_test, depth + 1, resolve);
|
||||
line(out, depth, "while");
|
||||
write_ops(out, inner_test, depth + 1, resolve);
|
||||
line(out, depth, "while");
|
||||
write_ops(out, body, depth + 1, resolve);
|
||||
line(out, depth, "repeat");
|
||||
write_ops(out, after_repeat, depth + 1, resolve);
|
||||
if let Some(eb) = else_body {
|
||||
line(out, depth, "else");
|
||||
write_ops(out, eb, depth + 1, resolve);
|
||||
}
|
||||
line(out, depth, "then");
|
||||
return;
|
||||
}
|
||||
IrOp::Exit => "exit".into(),
|
||||
IrOp::LoopRestartIfFalse => "loop-restart-if-false".into(),
|
||||
|
||||
// -- Flat forward branches --
|
||||
IrOp::Block(l) => format!("block L{l}"),
|
||||
IrOp::BranchIfFalse(l) => format!("branch-if-false L{l}"),
|
||||
IrOp::EndBlock(l) => format!("end-block L{l}"),
|
||||
|
||||
// -- Return stack --
|
||||
IrOp::ToR => ">r".into(),
|
||||
IrOp::FromR => "r>".into(),
|
||||
IrOp::RFetch => "r@".into(),
|
||||
IrOp::LoopJ => "j".into(),
|
||||
|
||||
// -- Forth locals --
|
||||
IrOp::ForthLocalGet(n) => format!("local@ {n}"),
|
||||
IrOp::ForthLocalSet(n) => format!("local! {n}"),
|
||||
IrOp::ForthFLocalGet(n) => format!("flocal@ {n}"),
|
||||
IrOp::ForthFLocalSet(n) => format!("flocal! {n}"),
|
||||
|
||||
// -- I/O --
|
||||
IrOp::Emit => "emit".into(),
|
||||
IrOp::Dot => ".".into(),
|
||||
IrOp::Cr => "cr".into(),
|
||||
IrOp::Type => "type".into(),
|
||||
|
||||
// -- System --
|
||||
IrOp::Execute => "execute".into(),
|
||||
IrOp::SpFetch => "sp@".into(),
|
||||
|
||||
// -- Float stack --
|
||||
IrOp::FDup => "fdup".into(),
|
||||
IrOp::FDrop => "fdrop".into(),
|
||||
IrOp::FSwap => "fswap".into(),
|
||||
IrOp::FOver => "fover".into(),
|
||||
|
||||
// -- Float arithmetic --
|
||||
IrOp::FAdd => "fadd".into(),
|
||||
IrOp::FSub => "fsub".into(),
|
||||
IrOp::FMul => "fmul".into(),
|
||||
IrOp::FDiv => "fdiv".into(),
|
||||
IrOp::FNegate => "fnegate".into(),
|
||||
IrOp::FAbs => "fabs".into(),
|
||||
IrOp::FSqrt => "fsqrt".into(),
|
||||
IrOp::FMin => "fmin".into(),
|
||||
IrOp::FMax => "fmax".into(),
|
||||
IrOp::FFloor => "ffloor".into(),
|
||||
IrOp::FRound => "fround".into(),
|
||||
|
||||
// -- Float comparisons --
|
||||
IrOp::FZeroEq => "f0=".into(),
|
||||
IrOp::FZeroLt => "f0<".into(),
|
||||
IrOp::FEq => "f=".into(),
|
||||
IrOp::FLt => "f<".into(),
|
||||
|
||||
// -- Float memory --
|
||||
IrOp::FetchFloat => "f@".into(),
|
||||
IrOp::StoreFloat => "f!".into(),
|
||||
|
||||
// -- Conversions --
|
||||
IrOp::StoF => "s>f".into(),
|
||||
IrOp::FtoS => "f>s".into(),
|
||||
};
|
||||
line(out, depth, &simple);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn simple_ops_one_per_line() {
|
||||
let out = format_ir(&[IrOp::Dup, IrOp::Mul, IrOp::PushI32(7)]);
|
||||
assert_eq!(out, "dup\nmul\npush 7\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn call_resolves_via_resolver() {
|
||||
let ops = [IrOp::Call(WordId(12)), IrOp::TailCall(WordId(13))];
|
||||
assert_eq!(format_ir(&ops), "call #12\ntail-call #13\n");
|
||||
let named = format_ir_with(&ops, &|id| (id.0 == 12).then(|| "SQ".to_string()));
|
||||
assert_eq!(named, "call SQ\ntail-call #13\n");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nested_if_inside_do_loop_indents() {
|
||||
let ops = [IrOp::DoLoop {
|
||||
body: vec![
|
||||
IrOp::Dup,
|
||||
IrOp::If {
|
||||
then_body: vec![IrOp::Dup, IrOp::Mul],
|
||||
else_body: Some(vec![IrOp::Drop]),
|
||||
},
|
||||
],
|
||||
is_plus_loop: false,
|
||||
}];
|
||||
let expected = "do\n dup\n if\n dup\n mul\n else\n drop\n then\nloop\n";
|
||||
assert_eq!(format_ir(&ops), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn while_loops_and_flat_branches() {
|
||||
let ops = [
|
||||
IrOp::BeginWhileRepeat {
|
||||
test: vec![IrOp::Dup],
|
||||
body: vec![IrOp::PushI32(1), IrOp::Sub],
|
||||
},
|
||||
IrOp::Block(3),
|
||||
IrOp::BranchIfFalse(3),
|
||||
IrOp::EndBlock(3),
|
||||
];
|
||||
let expected = "begin\n dup\nwhile\n push 1\n sub\nrepeat\nblock L3\nbranch-if-false L3\nend-block L3\n";
|
||||
assert_eq!(format_ir(&ops), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_simple_variant_renders() {
|
||||
// One of each non-structured op; count of output lines must match.
|
||||
let ops = vec![
|
||||
IrOp::PushI32(1),
|
||||
IrOp::PushI64(2),
|
||||
IrOp::PushF64(1.5),
|
||||
IrOp::Drop,
|
||||
IrOp::Dup,
|
||||
IrOp::Swap,
|
||||
IrOp::Over,
|
||||
IrOp::Rot,
|
||||
IrOp::Nip,
|
||||
IrOp::Tuck,
|
||||
IrOp::TwoDup,
|
||||
IrOp::TwoDrop,
|
||||
IrOp::Add,
|
||||
IrOp::Sub,
|
||||
IrOp::Mul,
|
||||
IrOp::DivMod,
|
||||
IrOp::Negate,
|
||||
IrOp::Abs,
|
||||
IrOp::Eq,
|
||||
IrOp::NotEq,
|
||||
IrOp::Lt,
|
||||
IrOp::Gt,
|
||||
IrOp::LtUnsigned,
|
||||
IrOp::ZeroEq,
|
||||
IrOp::ZeroLt,
|
||||
IrOp::And,
|
||||
IrOp::Or,
|
||||
IrOp::Xor,
|
||||
IrOp::Invert,
|
||||
IrOp::Lshift,
|
||||
IrOp::Rshift,
|
||||
IrOp::ArithRshift,
|
||||
IrOp::Fetch,
|
||||
IrOp::Store,
|
||||
IrOp::CFetch,
|
||||
IrOp::CStore,
|
||||
IrOp::PlusStore,
|
||||
IrOp::Call(WordId(1)),
|
||||
IrOp::TailCall(WordId(2)),
|
||||
IrOp::Exit,
|
||||
IrOp::LoopRestartIfFalse,
|
||||
IrOp::Block(1),
|
||||
IrOp::BranchIfFalse(1),
|
||||
IrOp::EndBlock(1),
|
||||
IrOp::ToR,
|
||||
IrOp::FromR,
|
||||
IrOp::RFetch,
|
||||
IrOp::LoopJ,
|
||||
IrOp::ForthLocalGet(0),
|
||||
IrOp::ForthLocalSet(0),
|
||||
IrOp::ForthFLocalGet(0),
|
||||
IrOp::ForthFLocalSet(0),
|
||||
IrOp::Emit,
|
||||
IrOp::Dot,
|
||||
IrOp::Cr,
|
||||
IrOp::Type,
|
||||
IrOp::Execute,
|
||||
IrOp::SpFetch,
|
||||
IrOp::FDup,
|
||||
IrOp::FDrop,
|
||||
IrOp::FSwap,
|
||||
IrOp::FOver,
|
||||
IrOp::FAdd,
|
||||
IrOp::FSub,
|
||||
IrOp::FMul,
|
||||
IrOp::FDiv,
|
||||
IrOp::FNegate,
|
||||
IrOp::FAbs,
|
||||
IrOp::FSqrt,
|
||||
IrOp::FMin,
|
||||
IrOp::FMax,
|
||||
IrOp::FFloor,
|
||||
IrOp::FRound,
|
||||
IrOp::FZeroEq,
|
||||
IrOp::FZeroLt,
|
||||
IrOp::FEq,
|
||||
IrOp::FLt,
|
||||
IrOp::FetchFloat,
|
||||
IrOp::StoreFloat,
|
||||
IrOp::StoF,
|
||||
IrOp::FtoS,
|
||||
];
|
||||
let out = format_ir(&ops);
|
||||
assert_eq!(out.lines().count(), ops.len());
|
||||
// Every line non-empty, no accidental blank rendering.
|
||||
assert!(out.lines().all(|l| !l.trim().is_empty()));
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,230 @@
|
||||
# Plan: SEE / SEE-IR / HELP — Introspection Trio
|
||||
|
||||
Status: implemented 2026-08-05 (all phases; HELP covers every word in a fresh VM, enforced by test)
|
||||
Scope: `SEE` (source-level decompile), `SEE-IR` (optimized-IR dump), `HELP` (per-word docs), shared lookup infrastructure.
|
||||
Each phase is self-contained and executable in a fresh context. Execute in order; every phase leaves the tree green (`cargo test --workspace` passes).
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Consolidated Findings (read this first, do not re-derive)
|
||||
|
||||
All references verified at commit `e31407a` (branch `usability`).
|
||||
|
||||
### The template to copy: WORDS
|
||||
|
||||
`WORDS` is a **host primitive whose body runs Rust-side via the `pending_define` mechanism**. This is the exact pattern for SEE/SEE-IR/HELP because it gives a real dictionary entry (→ findable by `'`, listed by `WORDS`, tab-completable in the CLI via `crates/cli/src/main.rs:452-455`, exposed to web palette via `crates/web/src/lib.rs:61`) while the implementation can still call `next_token()` and write `self.output`.
|
||||
|
||||
- Registration: `register_words()` at `crates/core/src/outer.rs:6301-6310` — host fn pushes code `40` into `pending_define`, called from `register_primitives()` at `outer.rs:2991` under the `// -- Programming-Tools word set --` header.
|
||||
- Dispatch: `handle_pending_define()` arm at `outer.rs:5328`: `40 => self.do_words(),`.
|
||||
- Body: `do_words()` at `outer.rs:6045-6074`. Note `outer.rs:6050-6052`: it reads an optional same-line argument with `self.next_token()` **gated on `self.state == 0`** — SEE must copy this gate.
|
||||
- **Used `pending_define` codes: 1–12, 20, 21, 25, 33, 40.** Free: **41 (SEE), 42 (SEE-IR), 43 (HELP)**. Legend comment at `outer.rs:232-233` must be extended.
|
||||
|
||||
### Allowed APIs (verified signatures)
|
||||
|
||||
| API | Location | Notes |
|
||||
|---|---|---|
|
||||
| `Dictionary::find(&self, name: &str) -> Option<(u32, WordId, bool)>` | `dictionary.rs:182` | `(word_addr, WordId, is_immediate)`, case-insensitive |
|
||||
| `Dictionary::word_name(word_addr)` / `code_field(word_addr)` / `read_link` / `latest()` | `dictionary.rs:375/392/287/282` | manual entry walk |
|
||||
| `flags::IMMEDIATE = 0x80`, `HIDDEN = 0x40`, `INTERNAL = 0x20` | `dictionary.rs:16-27` | raw flags byte = `dict.memory()[(word_addr+4) as usize]` — no getter exists |
|
||||
| `ir_bodies: HashMap<WordId, Vec<IrOp>>` | `outer.rs:256` | **post-optimization** IR; populated for colon words AND all defining-word products AND IR primitives (see kind table below) |
|
||||
| `host_word_names: HashMap<WordId, String>` | `outer.rs:214` | only populated by `register_host_primitive` (`outer.rs:2702-2703`) |
|
||||
| `does_definitions: HashMap<WordId, DoesDefinition>` | `outer.rs:223` | DOES>-words |
|
||||
| `output: Arc<Mutex<String>>` | `outer.rs:208` | ALL text output goes here; `HostAccess` has **no** emit method (`runtime.rs:17-60`) |
|
||||
| `next_token()` | `outer.rs:690-704` | whitespace-delimited, advances `input_pos` |
|
||||
| `register_host_primitive(name, immediate, func) -> anyhow::Result<WordId>` | `outer.rs:2686-2691` | public |
|
||||
| `IrOp` enum, `#[derive(Debug, Clone, PartialEq)]` | `ir.rs:9-218` | **no `Display` impl exists anywhere in core** — formatter is net-new |
|
||||
| `eval_output(input) -> String` test helper | `outer.rs:7566` | fresh VM per call; multi-eval tests build VM inline like `outer.rs:9077-9080` |
|
||||
|
||||
### Word-kind classification (SEE must distinguish these)
|
||||
|
||||
| Kind | Detectable via | SEE output strategy |
|
||||
|---|---|---|
|
||||
| Colon word / `:NONAME` | in `ir_bodies`, has captured source (Phase 3) | source (Phase 3) or IR (Phase 2) |
|
||||
| IR primitive (`DUP`…) | in `ir_bodies`, no source | IR body + "primitive" tag |
|
||||
| Host primitive (`.S`, `WORDS`…) | in `host_word_names` | `<built-in host word>` stub |
|
||||
| CONSTANT / VARIABLE / VALUE / CREATE / DEFER / SYNONYM / BUFFER: / 2\*/F\* | in `ir_bodies` with recognizable shape (e.g. CONSTANT = `[PushI32(v)]`, insert sites: `outer.rs:3194/3226/3263/3309/3351/3388/3440/6517/6547/6590/7344`) | synthesized definition, e.g. `42 CONSTANT ANSWER` (Phase 3) |
|
||||
| DOES>-defined word | key in `does_definitions` | show CREATE part + DOES> body IR |
|
||||
| Interpreter special token (`:`, `;`, `VARIABLE`, `'`, `CHAR`…) | hardcoded matches `outer.rs:755-820`, `927-992`; several have **no dictionary entry at all** | `<compiler word, handled by the outer interpreter>` stub |
|
||||
|
||||
### Hard constraints
|
||||
|
||||
1. **Feature-free.** `outer.rs`, `ir.rs`, `dictionary.rs` compile without the `native` feature (`lib.rs:17-42`); web consumes core with `default-features = false` (`crates/web/Cargo.toml:15`). No `#[cfg(feature = "native")]` in any SEE code. Unit tests live in the existing `#[cfg(all(test, feature = "native"))]` module (`outer.rs:7553`) — that is fine and matches practice.
|
||||
2. **`ir_bodies` stores post-optimization IR** (`finish_colon_def`: optimize at `outer.rs:2297`, insert at `outer.rs:2298`; inlining threshold 8 at `optimizer.rs:56`). `: FOO SQ SQ ;` shows `SQ`'s body inlined. This is a *feature* for SEE-IR (shows what the optimizer did) and the *reason* SEE needs separate source capture (Phase 3).
|
||||
3. **Multi-line definitions**: compile state persists across `evaluate()` calls (`outer.rs:512-514` resets only `input_buffer`/`input_pos`); the driver (CLI `main.rs:411-416`) feeds lines. Source capture must accumulate across calls. On error, `evaluate()` wipes compile state (`outer.rs:523-535`) — capture state must be wiped there too.
|
||||
4. **Error house style** (`outer.rs:1294/3400/4057` precedents): `anyhow::bail!("SEE: unknown word: {name}")`, `anyhow::bail!("SEE: expected word name")`.
|
||||
5. **Compliance suite gives SEE zero coverage** — `toolstest.fth:38-39` explicitly excludes it. All coverage is hand-written unit tests. Adding SEE cannot break `compliance_tools`.
|
||||
6. **MARKER correctness**: any new per-word map (source text, docs) must be snapshotted/restored in `MarkerState` (`outer.rs:166-176`, snapshot `outer.rs:3462-3480`, restore `outer.rs:3484-3506`), mirroring how `ir_bodies` is handled there.
|
||||
|
||||
### Anti-patterns (verified NOT to exist — do not invent)
|
||||
|
||||
- `HostAccess::emit(...)` / any output method on `HostAccess` — does not exist; capture `Arc::clone(&self.output)` instead.
|
||||
- `Display for IrOp` — does not exist; write the formatter.
|
||||
- A dictionary "entry struct" or kind tag — does not exist; classify via the VM-side maps above.
|
||||
- `Dictionary::flags(addr)` getter — does not exist; read the raw byte.
|
||||
- Refill-on-demand for a missing SEE argument — `REFILL`/`ACCEPT` are hardcoded to fail (`outer.rs:5824-5852`); `SEE` at end of line is an error, same as `'` (`outer.rs:4051-4053`).
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — IR pretty-printer (pure function, no VM changes)
|
||||
|
||||
**Goal:** a feature-free formatter turning `&[IrOp]` into readable, indented text. Foundation for SEE-IR and the SEE fallback path.
|
||||
|
||||
**What to implement:**
|
||||
|
||||
1. New module `crates/core/src/see.rs`, registered unconditionally in `lib.rs` next to `pub mod outer;` (`lib.rs:30`). Public API:
|
||||
```rust
|
||||
/// Format an IR body as indented, one-op-per-line text.
|
||||
pub fn format_ir(ops: &[IrOp]) -> String
|
||||
```
|
||||
2. Exhaustive `match` over every `IrOp` variant (full list at `ir.rs:10-218`) — **no wildcard arm**, so adding a variant later forces a formatter update at compile time.
|
||||
3. Simple ops print as their Forth-ish name plus payload: `PushI32(7)` → `push 7`, `Call(WordId(12))` → `call #12`, `TailCall` → `tail-call #12`. Resolve `#12` to a word name at a higher level (Phase 2) — `format_ir` itself stays name-agnostic, but takes an optional resolver to keep it pure:
|
||||
```rust
|
||||
pub fn format_ir_with(ops: &[IrOp], resolve: &dyn Fn(WordId) -> Option<String>) -> String
|
||||
```
|
||||
(`format_ir` delegates with a `|_| None` resolver.)
|
||||
4. The six nested variants (`If`, `DoLoop`, `BeginUntil`, `BeginAgain`, `BeginWhileRepeat` at `ir.rs:78-99`, `BeginDoubleWhileRepeat` at `ir.rs:105-111`) print as Forth control words with 2-space indented bodies:
|
||||
```
|
||||
if
|
||||
dup
|
||||
mul
|
||||
else
|
||||
drop
|
||||
then
|
||||
```
|
||||
5. Flat branch ops (`Block`/`BranchIfFalse`/`EndBlock`, `ir.rs:118-124`) print literally (`block L3` etc.) — they have no clean Forth surface syntax; do not attempt reconstruction.
|
||||
|
||||
**Verification checklist:**
|
||||
- [ ] Unit tests in `see.rs` (plain `#[cfg(test)]`, NOT feature-gated — the module has no runtime dependency): nested `If` inside `DoLoop` indents correctly; every-variant smoke test via a `Vec` containing one of each simple op.
|
||||
- [ ] `cargo check -p wafer-core --no-default-features` passes (proves feature-freedom).
|
||||
- [ ] `cargo test --workspace` green; `cargo fmt --all` + `cargo clippy --workspace` clean.
|
||||
|
||||
**Anti-pattern guards:** no `impl Display for IrOp` (keep the formatter in `see.rs`, IrOp is data); no wildcard match arm; no `#[cfg(feature = "native")]`.
|
||||
|
||||
---## Phase 2 — SEE-IR word
|
||||
|
||||
**Goal:** `SEE-IR name` prints the stored post-optimization IR for any word — the optimizer-debugging view. Ship this before source-SEE: it is nearly free and immediately useful.
|
||||
|
||||
**What to implement:**
|
||||
|
||||
1. Copy the WORDS registration pattern verbatim (`outer.rs:6301-6310`): `register_see_ir()` pushes pending code **42**; register in `register_primitives()` next to `self.register_words()?` (`outer.rs:2991`). Extend the legend comment at `outer.rs:232-233`.
|
||||
2. Dispatch arm in `handle_pending_define()` next to `outer.rs:5328`: `42 => self.do_see_ir(),`.
|
||||
3. `do_see_ir()` (place near `do_words()`, `outer.rs:6045`):
|
||||
- Parse name: `let Some(name) = self.next_token() else { bail!("SEE-IR: expected word name") }` — **no** interpret-mode gate here (unlike WORDS' optional filter, the argument is mandatory; compile-mode `SEE-IR` may simply also parse — matches `'`).
|
||||
- Lookup: `self.dictionary.find(&name)` → else `bail!("SEE-IR: unknown word: {name}")`.
|
||||
- Classify per the Phase 0 kind table, in this order: `ir_bodies` hit → header line + `see::format_ir_with(...)` with a resolver that maps `WordId` → name (build once from a dictionary walk: `latest()`/`read_link`/`word_name`/`code_field`, `dictionary.rs:282/287/375/392`); `host_word_names` hit → `SEE-IR: <name> is a built-in host word`; neither → `SEE-IR: <name> has no IR body`.
|
||||
- Header line format: `\ <NAME> — <n> ops (optimized IR)`, plus ` immediate` when the find() flag is set, plus `does>` info when `does_definitions` has the id.
|
||||
- Write everything into `self.output.lock().unwrap()`; end with `\n` (multi-line output convention from commit `2910884`).
|
||||
4. Special-token names (`:`, `VARIABLE`, `'`, …): after dictionary miss, check a small const list of known interpreter tokens (source: match arms at `outer.rs:755-820`, `927-992`) and print `SEE-IR: <name> is handled directly by the outer interpreter` instead of erroring.
|
||||
|
||||
**Documentation references:** WORDS pattern `outer.rs:6301-6310`, `5328`, `6045-6074`; error style `outer.rs:4057`; output convention `outer.rs:6056-6073`.
|
||||
|
||||
**Verification checklist:**
|
||||
- [ ] Tests (in `outer.rs` test module, `eval_output` style, cf. `outer.rs:9035-9041`):
|
||||
- `: SQ DUP * ; SEE-IR SQ` output contains `dup` and `mul`;
|
||||
- `: FOO SQ SQ ; SEE-IR FOO` shows the **inlined** body (contains two `mul`, no `call`) — locks in the "optimized view" semantics;
|
||||
- `SEE-IR DUP` works (IR primitive); `SEE-IR WORDS` prints host-word stub; `SEE-IR NOSUCHWORD` errors with `SEE-IR: unknown word: NOSUCHWORD`; bare `SEE-IR` errors with `expected word name`;
|
||||
- `SEE-IR :` prints the interpreter-token message.
|
||||
- [ ] `IF`/`ELSE`/`THEN` and `DO LOOP` bodies render indented (one structured-word test).
|
||||
- [ ] `cargo test --workspace` green; fmt + clippy clean; `cargo check -p wafer-core --no-default-features` passes.
|
||||
|
||||
**Anti-pattern guards:** do not print via a nonexistent `HostAccess` emit; do not gate name parsing on `state == 0` (mandatory arg, not optional filter); do not `THROW -13` (plain `bail!` matches TO/SYNONYM precedent).
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Source capture + SEE
|
||||
|
||||
**Goal:** `SEE name` prints the original source text `: name … ;` for colon words, synthesized definitions for data words, graceful stubs otherwise. This is the user-facing SEE.
|
||||
|
||||
**What to implement:**
|
||||
|
||||
1. **Capture fields** on `ForthVM` (near `compiling_ir`, `outer.rs:204`):
|
||||
```rust
|
||||
compiling_source: String, // accumulated raw text of the definition in progress
|
||||
source_capture_from: Option<usize>, // input_pos where capture started in the CURRENT buffer
|
||||
word_sources: HashMap<WordId, String>,
|
||||
```
|
||||
2. **Capture protocol** (verbatim source, including comments and string literals — token-level reassembly would lose them):
|
||||
- `start_colon_def()` (`outer.rs:2097`): set `source_capture_from` to the position where `:` began. `interpret_token()` receives the token already consumed, so record the position **before** dispatch: in the `evaluate()` loop (`outer.rs:518-521`), remember `pos_before = self.input_pos` minus token — simplest correct form: capture `token_start` inside `next_token()` (`outer.rs:690-704`) into a new field `last_token_start: usize` as it skips whitespace; `start_colon_def` then does `self.source_capture_from = Some(self.last_token_start)`.
|
||||
- End of `evaluate()` (after the loop, `outer.rs:~536`): if still compiling and capture active, flush `input_buffer[from..]` + `'\n'` into `compiling_source`, reset `source_capture_from = Some(0)` so the next buffer continues capture from its start.
|
||||
- `finish_colon_def()` (`outer.rs:2266`): flush `input_buffer[from..=pos of ';']`, store `word_sources.insert(word_id, normalized)`, clear capture state. Normalize only trailing whitespace; keep interior verbatim.
|
||||
- Error path `outer.rs:523-535`: clear both capture fields alongside the existing compile-state wipe.
|
||||
- `:NONAME` and quotations (`outer.rs:759-761, 773-778`): skip capture (no name to SEE) — guard on `compiling_name.is_some()`.
|
||||
3. **MARKER integration**: add `word_sources` to `MarkerState` (`outer.rs:166-176`), snapshot (`outer.rs:3462-3480`) and restore (`outer.rs:3484-3506`) exactly as `ir_bodies` is handled there.
|
||||
4. **SEE word**: pending code **41**, same registration/dispatch shape as Phase 2. `do_see()` resolution order:
|
||||
1. `word_sources` hit → print stored source verbatim, append ` immediate` on its own line if flagged (cf. `set_immediate`, `dictionary.rs:404`).
|
||||
2. Recognizable data-word IR shape (Phase 0 kind table) → synthesized one-liner. CONSTANT `[PushI32(v)]` → `<v> CONSTANT <NAME>`; VARIABLE → `VARIABLE <NAME> ( addr=<v> )`; VALUE `[PushI32(a), Fetch]` → `<cur> VALUE <NAME>` reading current value via `self.rt` memory read if cheap, else `VALUE <NAME>`; SYNONYM `[Call(id)]` → `SYNONYM <NAME> <OLD>`; DEFER → `DEFER <NAME>` plus current target name via `does`/pfa lookup when resolvable.
|
||||
3. `ir_bodies` hit (primitive or pre-capture colon word) → `\ <NAME> is a primitive; IR:` + `format_ir_with` output (reuse Phase 1/2 machinery — SEE never dead-ends).
|
||||
4. `host_word_names` hit → `<NAME> is a built-in host word`.
|
||||
5. Interpreter-token list → `<NAME> is handled by the outer interpreter (compiler word)`.
|
||||
6. Else → `bail!("SEE: unknown word: {name}")`.
|
||||
5. **Boot words get sources for free**: `boot.fth` definitions flow through the same `evaluate()`/`finish_colon_def` path, so `SEE NIP` etc. shows real boot source. Verify, don't assume — one test below.
|
||||
|
||||
**Documentation references:** compile-state lifecycle `outer.rs:512-535`, `2097-2121`, `2266-2329`; multi-line REPL driver `main.rs:411-416`; MarkerState `outer.rs:166-176, 3462-3506`.
|
||||
|
||||
**Verification checklist:**
|
||||
- [ ] `: SQ DUP * ; SEE SQ` prints `: SQ DUP * ;` (verbatim, one line).
|
||||
- [ ] Multi-line: inline-VM test (pattern `outer.rs:9077-9080`): `evaluate(": TRI\")` then `evaluate(\" DUP DUP ;")`, then `SEE TRI` shows both lines.
|
||||
- [ ] Comment survives: `: C ( n -- n ) 1+ ; SEE C` output contains `( n -- n )`.
|
||||
- [ ] `42 CONSTANT A SEE A` → `42 CONSTANT A`; `VARIABLE V SEE V` → contains `VARIABLE V`.
|
||||
- [ ] `SEE NIP` (boot word) prints a colon definition, not an IR dump.
|
||||
- [ ] `SEE DUP` prints the primitive-IR fallback; `SEE WORDS` prints host stub; `SEE '` prints interpreter-token message; unknown word errors in house style.
|
||||
- [ ] MARKER round-trip: define word, set marker, redefine, execute marker, `SEE` shows the original — plus existing marker tests still green.
|
||||
- [ ] Immediate flag: `: I2 ; IMMEDIATE SEE I2` output contains `immediate`.
|
||||
- [ ] Error path: force `unknown word` mid-definition, then define a fresh word — its captured source must not contain debris from the aborted definition.
|
||||
- [ ] Full suite + fmt + clippy + `--no-default-features` check.
|
||||
|
||||
**Anti-pattern guards:** do not reconstruct source from tokens (loses comments/strings/spacing); do not capture into `word_sources` for `:NONAME`; do not forget the error-path wipe (`outer.rs:523-535`) — stale capture corrupts the next definition's source; `evaluate()` resets `input_pos` per call (`outer.rs:512-514`) so `source_capture_from` is per-buffer, never carried across calls uncleared.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — HELP word + doc table
|
||||
|
||||
**Goal:** `HELP name` prints stack effect + one-line description; `HELP` alone prints usage. Shares lookup/classification with SEE.
|
||||
|
||||
**What to implement:**
|
||||
|
||||
1. New feature-free module `crates/core/src/wordhelp.rs`: a static table
|
||||
```rust
|
||||
/// (NAME, stack effect, one-line description)
|
||||
pub const WORD_DOCS: &[(&str, &str, &str)] = &[
|
||||
("DUP", "( x -- x x )", "Duplicate the top of the data stack."),
|
||||
...
|
||||
];
|
||||
pub fn lookup(name: &str) -> Option<(&'static str, &'static str)> // case-insensitive
|
||||
```
|
||||
Seed from the Forth 2012 glossary (stack effects are standardized). Cover, in priority order: core + core-ext words WAFER implements, then tools/double/float sets. Incomplete coverage is acceptable and expected — `HELP` says `no help for <name> (word exists)` when the word is defined but undocumented, which doubles as the TODO list.
|
||||
2. `HELP` word: pending code **43**, same registration/dispatch shape as Phase 2. Resolution: parse optional name (bare `HELP` → usage line `HELP <word> — also try: WORDS, SEE <word>, SEE-IR <word>`); table hit → print `NAME ( stack effect ) description`; miss but dictionary hit → `no help for <name>` + hint `try SEE <name>`; miss both → house-style unknown-word error.
|
||||
3. Cross-wiring (the "as useful as possible" part):
|
||||
- `SEE`/`SEE-IR` prepend the HELP line as a `\ ...` comment when the table has one.
|
||||
- `HELP` appends ` immediate` / `built-in` / `defined in boot.fth or user code` classification reusing the Phase 2/3 classifier — factor that classifier into a shared `fn classify_word(&self, name) -> WordClass` when Phase 4 lands (do NOT pre-build it in Phase 2; extract once there are two users, per smallest-change rule).
|
||||
4. User-defined words: optional docstring convention — if the captured source's first parenthesized comment looks like a stack effect (`( ... -- ... )`), `HELP` echoes it for user words. No new syntax, zero cost, rewards idiomatic Forth style.
|
||||
|
||||
**Verification checklist:**
|
||||
- [ ] `HELP DUP` prints stack effect + description; `HELP dup` (lowercase) same.
|
||||
- [ ] `HELP` alone prints usage; `HELP NOSUCH` errors house-style; `HELP MYWORD` for undocumented-but-defined word prints the `no help` + `SEE` hint.
|
||||
- [ ] `: SQ ( n -- n^2 ) DUP * ; HELP SQ` echoes `( n -- n^2 )`.
|
||||
- [ ] Table lint test: iterate `WORD_DOCS`, assert every documented name resolves in a booted VM's dictionary (catches typos/renames mechanically).
|
||||
- [ ] Full suite + fmt + clippy + `--no-default-features`.
|
||||
|
||||
**Anti-pattern guards:** no doc strings threaded through `register_primitive` call sites (200+ call-site churn, bloats outer.rs — the side table is deliberate); no partial-coverage panic — missing docs degrade gracefully.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Final verification + docs
|
||||
|
||||
1. **Full gate:** `cargo fmt --all` && `cargo clippy --workspace` (zero warnings) && `cargo test --workspace` (expect baseline 431 unit + new SEE/SEE-IR/HELP tests, 1 benchmark, 11 compliance, 9 comparison — all green).
|
||||
2. **Feature-freedom proof:** `cargo check -p wafer-core --no-default-features` and web build `cd crates/web && wasm-pack build --target web --dev --out-dir www/pkg`.
|
||||
3. **Manual REPL pass** (CLI): `SEE SQ`, `SEE-IR FOO` with inlining, `HELP DUP`, multi-line definition then SEE, tab-complete `SE<tab>` — confirm multi-line output renders per commit `2910884` conventions (block output, ` ok` on own line).
|
||||
4. **Web REPL smoke:** serve `crates/web/www`, run the same commands — output flows through `take_output()` (`web/src/lib.rs:38-43`), no web-side changes expected.
|
||||
5. **Anti-pattern grep:** `grep -n "cfg(feature" crates/core/src/see.rs crates/core/src/wordhelp.rs` → empty; `grep -n "impl Display for IrOp" -r crates/core` → empty; `grep -rn "emit" crates/core/src/see.rs` → empty.
|
||||
6. **Docs:** `docs/FORTH.md:95` already lists SEE under Programming-Tools — verify claim now true; add SEE/SEE-IR/HELP to README feature list if words are enumerated there; extend CLAUDE.md test-count line.
|
||||
7. **Compliance untouched:** `cargo test -p wafer-core --test compliance` — must stay 11/11 (suite excludes SEE by design, `toolstest.fth:38-39`).
|
||||
|
||||
---
|
||||
|
||||
## Deliberate scope cuts (revisit later, not now)
|
||||
|
||||
- **`SEE-WASM`** (disassemble compiled module via `wasmprinter`): compiled bytes are likely dropped after instantiation; `codegen.rs` unexamined. Separate plan if wanted.
|
||||
- **IR→Forth source reconstruction** for optimized bodies: lossy and misleading post-inlining; the source-capture path makes it unnecessary.
|
||||
- **`LOCATE` / editor integration**: needs file/line provenance in the dictionary; out of scope.
|
||||
- **Forth-side doc syntax (`:doc`)**: revisit after self-hosting work starts; the `( n -- n^2 )` echo in Phase 4 covers the 80% case with zero syntax.
|
||||
Reference in New Issue
Block a user