diff --git a/README.md b/README.md index f87b095..cd5bc55 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,13 @@ elements, and dependency edges show calls, imports, and inheritance. The same model is emitted as versioned JSON for other tools, including a future Fold integration. -The initial extractor recognizes Rust, Dart, Python, C/C++, -JavaScript/TypeScript, and Swift source extensions. Symbol extraction is intentionally conservative and -lexical in v0.1; every relationship records its backend, confidence, and source -location so consumers can distinguish it from imported symbolic evidence. +The initial extractor provides first-class lexical profiles for Rust, Dart, +C/C++, and JavaScript/TypeScript, with explicitly secondary Python and Swift +coverage. Symbol extraction is intentionally conservative and lexical in v0.1; +every relationship records its backend, confidence, and all contributing source +locations so consumers can distinguish it from imported symbolic evidence. +See [language support and parity](docs/architecture/language-support.md) for the +construct-level coverage matrix and first-class promotion contract. ## Quick start @@ -54,6 +57,8 @@ default; an internal or vendored URL can be supplied with --cytoscape-url. confidence. - A focused Rust test suite, quality gate, bounded auto-loop, and artifact-based benchmark runner. +- A labeled language-parity corpus with independent gates for every advertised + language; Python and Swift remain characterization-only secondary profiles. ## Important limits diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 78fea22..02a2151 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -4,7 +4,9 @@ - Versioned graph JSON. - Fast lexical multi-language preview. -- Dart classes, functions/methods, imports, inheritance, and local calls. +- First-class lexical profiles for Dart, Rust, C/C++, and + JavaScript/TypeScript, with labeled parity fixtures and per-language gates. +- Characterized secondary Python and Swift coverage. - Fold SARIF/native finding overlay. - Interactive Cytoscape building. - Transparent structural indicators and deterministic tests. diff --git a/docs/architecture/graph-model.md b/docs/architecture/graph-model.md index fec8732..bc64b6a 100644 --- a/docs/architecture/graph-model.md +++ b/docs/architecture/graph-model.md @@ -26,7 +26,10 @@ Every edge has: - lane: syntactic, symbolic, imported, or heuristic; - backend: producer and mode; - confidence: producer-calibrated value; -- provenance: source location or report artifact. +- provenance: the first source location or report artifact, retained for v0.1 + consumers; +- locations: every distinct source location contributing to a deduplicated + edge. ## Health indicator @@ -50,4 +53,3 @@ should map Fold function/block/operation/value fact IDs directly to Brick identities and retain source/binary lanes. Source proved, latent, heuristic, and concrete-witness confidence must remain distinguishable. Binary evidence must not be promoted to symbolic evidence without corroboration. - diff --git a/docs/architecture/language-support.md b/docs/architecture/language-support.md new file mode 100644 index 0000000..4b3b102 --- /dev/null +++ b/docs/architecture/language-support.md @@ -0,0 +1,84 @@ +# Language support and parity + +Brick 0.1 recognizes source-file extensions for Rust, Dart, Python, C/C++, +JavaScript/TypeScript, and Swift. Recognition does not imply compiler-level +coverage: the current backend is deliberately lexical, line-oriented, and +evidence-labeled. + +The first-class lexical profiles are Dart, Rust, C/C++, and +JavaScript/TypeScript. Here, parity means each profile has dedicated construct +patterns, relationship evidence, positive and clean-negative fixtures, stable +regression assertions, and an independent benchmark gate. It does not mean the +languages have identical grammar features or compiler-grade resolution. Python +and Swift remain secondary until their extraction contracts and fixtures reach +the same standard. + +Plain `.h` files remain classified as C++ because their language cannot be +determined from the extension alone. C projects that require header symbol +resolution need compilation-database or include-context evidence in a future +frontend; Brick does not guess that ownership. + +## Current coverage + +| Language | Useful current coverage | Important gaps | Tier | +| --- | --- | --- | --- | +| Dart | Classes, mixins, enums, extensions, methods, named constructors, accessors, import/export/part directives, `extends`/`with`/`implements`/`on`, and ordinary calls | operators, generic relationship targets, conditional directives, and multiline declarations | first-class lexical | +| Rust | Structs, enums, traits, ordinary/async/const/unsafe/extern functions, simple inherent/trait impl ownership, ordinary/turbofish/macro calls, braced `use` resolution, and raw-string masking | module ownership, unions/type aliases, nested generic impl headers, glob imports, and macro definitions | first-class lexical | +| C | Distinct `.c` identity, includes, structs/unions/enums, narrow function forms, and ordinary calls | anonymous typedef aggregates, function pointers, C23 attributes, macros, and multiline declarations | first-class lexical | +| C++ | Classes/structs/unions/enums, `final`, constructors/destructors/operators, override-qualified methods, multiple inheritance, ordinary calls, and local include resolution | namespaces, overload disambiguation, lambdas, concepts/requires, complex template bases, and multiline declarations | first-class lexical | +| JavaScript | Default/exported classes, inheritance, owned methods/constructors, named functions, arrows, ES imports/re-exports, CommonJS `require`, dynamic imports, ordinary calls, and multiline template masking | function expressions, computed methods, decorators, JSX structure, package exports, and type-aware member calls | first-class lexical | +| TypeScript | JavaScript forms plus interfaces/enums/namespaces, generic and abstract classes/functions/methods, `implements`, type imports/re-exports, and typed arrows | type aliases, overload declarations, decorators, declaration merging, TSX structure, and path mappings | first-class lexical | +| Python | Characterized fixture coverage for basic classes, functions/async functions, imports, calls, inheritance, and indentation scopes | multiple bases, qualified ownership, decorators, multiline strings, relative/dynamic imports, and complete multi-import handling | secondary | +| Swift | Characterized extension recognition plus incidental simple `class`, `func`, inheritance, ownership, and call matching | no dedicated profile; imports, protocols, actors, enums, extensions, async/generic forms, and Swift-specific signatures are incomplete | secondary | + +## Cross-language invariants + +All extractors must satisfy these rules before first-class parity is claimed: + +1. Bare symbol names never resolve across language boundaries. Cross-language + edges require explicit import, binding, or FFI evidence. +2. Recognized methods are owned by their enclosing type rather than only by the + source file. +3. Unsupported declarations do not silently become call edges. +4. Strings, comments, and language-specific literal forms do not create symbols + or calls. +5. Duplicate names use stable qualified identities and do not resolve + ambiguously. +6. Every edge retains backend, confidence, its first provenance value, and all + distinct source locations contributing to a deduplicated relationship. + +## Parity test contract + +Each first-class language needs deterministic positive and clean-negative +fixtures for: + +- files, types, functions, methods, and containment; +- imports and inheritance/implementation relationships; +- local, qualified, member, ambiguous, and unresolved calls; +- duplicate symbols, nested scopes, strings, comments, and unsupported syntax; +- stable graph identities and evidence under repeated scans. + +The positive baseline corpus lives in `tests/fixtures/languages`, with its +shared contract in `tests/language_parity.rs`. Gap slices should extend these +fixtures with a positive construct and its nearest clean negative before +changing extraction behavior. + +Real-repository benchmarks supplement these fixtures but do not replace them. +Compare graph counts and health metrics only after confirming that analyzed +files, constructs, and evidence coverage remain stable. A lower health score can +be more accurate when a false resolved edge becomes an honest unresolved edge. + +Python and Swift fixtures intentionally describe their current secondary tier in +the same corpus. Their assertions are characterization tests, not evidence of +first-class parity. In particular, the Swift fixture preserves the visible gap +between file/type recognition and Swift-specific relationship extraction. + +## Delivered foundation order + +1. Shared resolver and scope foundation. +2. A golden fixture harness and mixed-language collision corpus. +3. Dart and Rust parity slices. +4. Separate C and C++ extraction profiles and parity slices. +5. Separate JavaScript and TypeScript extraction profiles and parity slices. +6. Explicit Python and Swift characterization, followed by a separate promotion + decision if warranted. diff --git a/src/fold.rs b/src/fold.rs index a6afd96..80452f9 100644 --- a/src/fold.rs +++ b/src/fold.rs @@ -57,6 +57,7 @@ pub fn import_findings(building: &mut Building, report: &Path) -> Result lines_of_code: 0, attributes, }); + let provenance = format!("{}:{}", report.display(), finding.line); building.edges.push(Edge { id: stable_id("edge", &format!("{node_id}:flags:{file_id}")), source: node_id, @@ -70,7 +71,8 @@ pub fn import_findings(building: &mut Building, report: &Path) -> Result }, backend: format!("fold:discover:{}", finding.backend), confidence: confidence_value(&finding.confidence), - provenance: format!("{}:{}", report.display(), finding.line), + locations: vec![provenance.clone()], + provenance, }, }); imported += 1; diff --git a/src/model.rs b/src/model.rs index 026fac9..db32c7c 100644 --- a/src/model.rs +++ b/src/model.rs @@ -72,6 +72,8 @@ pub struct Evidence { pub backend: String, pub confidence: f64, pub provenance: String, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub locations: Vec, } #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] @@ -142,10 +144,12 @@ pub fn stable_id(kind: &str, identity: &str) -> String { } pub fn evidence(backend: &str, confidence: f64, provenance: impl Into) -> Evidence { + let provenance = provenance.into(); Evidence { lane: EvidenceLane::Syntactic, backend: backend.into(), confidence, - provenance: provenance.into(), + locations: vec![provenance.clone()], + provenance, } } diff --git a/src/scan.rs b/src/scan.rs index 3b6ca53..21c70fc 100644 --- a/src/scan.rs +++ b/src/scan.rs @@ -94,7 +94,7 @@ struct ScanState { nodes: Vec, edges: Vec, edge_keys: HashSet<(String, String, EdgeKind)>, - symbols_by_name: HashMap>, + symbols_by_name: HashMap<(String, String), Vec>, files_by_key: HashMap>, pending: Vec, diagnostics: Vec, @@ -242,6 +242,8 @@ impl ScanState { let mut scopes: Vec = Vec::new(); let mut brace_depth = 0_i32; let mut rust_raw_end: Option = None; + let mut javascript_template_open = false; + let mut block_comment_depth = 0_usize; for (index, raw_line) in content.lines().enumerate() { let line_number = index + 1; @@ -251,7 +253,12 @@ impl ScanState { } continue; } - let code_line = if language == "rust" { + let template_masked; + let code_line = if matches!(language, "javascript" | "typescript") { + template_masked = + mask_javascript_templates(raw_line, &mut javascript_template_open); + template_masked.as_str() + } else if language == "rust" { if let Some((start, content_start, delimiter)) = rust_raw_string_start(raw_line) { if !raw_line[content_start..].contains(&delimiter) { rust_raw_end = Some(delimiter); @@ -263,8 +270,9 @@ impl ScanState { } else { raw_line }; - let line = strip_line_comment(code_line, language); - let call_line = mask_quoted_strings(line); + let block_masked = mask_block_comments(code_line, &mut block_comment_depth); + let line = strip_line_comment(&block_masked, language); + let call_line = mask_quoted_strings(&line); let indent = raw_line.len() - raw_line.trim_start().len(); if language == "python" && !line.trim().is_empty() { while scopes.last().is_some_and(|scope| scope.indent >= indent) { @@ -279,9 +287,28 @@ impl ScanState { } } - if let Some(captures) = patterns.class.captures(line) + if let Some(pattern) = patterns.enclosing_scope.as_ref() + && let Some(name) = pattern + .captures(&line) + .and_then(|captures| captures.name("name")) + && let Some(id) = self + .symbols_by_name + .get(&(language.into(), name.as_str().into())) + .and_then(|ids| (ids.len() == 1).then(|| ids[0].clone())) + && self.node_kind(&id) == Some(NodeKind::Class) + { + scopes.push(Scope { + id, + indent, + brace_depth: brace_depth + 1, + }); + } + + let mut type_definition_name = None; + if let Some(captures) = patterns.class.captures(&line) && let Some(name) = captures.name("name") { + type_definition_name = Some(name.as_str().to_owned()); let id = self.add_symbol( file_id, path, @@ -290,10 +317,26 @@ impl ScanState { NodeKind::Class, line_number, ); - if let Some(base) = captures.name("base") { + let mut bases: Vec = patterns + .relationships + .as_ref() + .into_iter() + .flat_map(|pattern| pattern.captures_iter(&line)) + .filter_map(|captures| captures.name("names")) + .flat_map(|names| names.as_str().split(',')) + .map(str::trim) + .filter(|name| !name.is_empty()) + .map(|name| clean_relationship_target(name, language)) + .collect(); + if bases.is_empty() + && let Some(base) = captures.name("base") + { + bases.push(clean_target(base.as_str())); + } + for base in bases { self.pending.push(PendingRelationship { source: id.clone(), - target_name: clean_target(base.as_str()), + target_name: base, kind: EdgeKind::Inherits, language: language.into(), member_call: false, @@ -310,16 +353,24 @@ impl ScanState { } let mut definition_name = None; - if let Some(captures) = patterns.function.captures(line) - && let Some(name) = captures.name("name") - && !is_keyword(name.as_str()) + if let Some(name) = patterns.function.iter().find_map(|pattern| { + pattern + .captures(&line) + .and_then(|captures| captures.name("name").map(|name| name.as_str().to_owned())) + }) && !is_keyword(&name) { - definition_name = Some(name.as_str().to_owned()); + definition_name = Some(name.clone()); + let parent_id = scopes + .iter() + .rev() + .find(|scope| self.node_kind(&scope.id) == Some(NodeKind::Class)) + .map(|scope| scope.id.as_str()) + .unwrap_or(file_id); let id = self.add_symbol( - file_id, + parent_id, path, language, - name.as_str(), + &name, NodeKind::Function, line_number, ); @@ -337,11 +388,15 @@ impl ScanState { .map(|scope| scope.id.clone()) .unwrap_or_else(|| file_id.into()); - for captures in patterns.import.captures_iter(line) { + for captures in patterns + .import + .iter() + .flat_map(|pattern| pattern.captures_iter(&line)) + { if let Some(target) = captures.name("name").or_else(|| captures.name("name2")) { self.pending.push(PendingRelationship { source: file_id.into(), - target_name: clean_target(target.as_str()), + target_name: clean_import_target(target.as_str(), language), kind: EdgeKind::Imports, language: language.into(), member_call: false, @@ -352,14 +407,25 @@ impl ScanState { } } - for captures in patterns.call.captures_iter(&call_line) { + for captures in patterns + .call + .iter() + .flat_map(|pattern| pattern.captures_iter(&call_line)) + { let Some(target_match) = captures.name("name") else { continue; }; let target = target_match.as_str(); let prefix = call_line[..target_match.start()].trim_end(); let member_call = prefix.ends_with('.') || prefix.ends_with("->"); - if definition_name.as_deref() != Some(target) && !is_keyword(target) { + let definition_call_name = definition_name + .as_deref() + .and_then(|name| name.rsplit('.').next()) + .map(|name| name.trim_start_matches('~')); + if definition_call_name != Some(target) + && type_definition_name.as_deref() != Some(target) + && !is_keyword(target) + { self.pending.push(PendingRelationship { source: source.clone(), target_name: clean_target(target), @@ -378,7 +444,7 @@ impl ScanState { fn add_symbol( &mut self, - file_id: &str, + parent_id: &str, path: &str, language: &str, name: &str, @@ -397,18 +463,18 @@ impl ScanState { kind, name: name.into(), path: Some(path.into()), - parent: Some(file_id.into()), + parent: Some(parent_id.into()), language: Some(language.into()), line: Some(line), lines_of_code: 0, attributes: BTreeMap::new(), }); self.symbols_by_name - .entry(name.into()) + .entry((language.into(), name.into())) .or_default() .push(id.clone()); self.add_edge( - file_id.into(), + parent_id.into(), id.clone(), EdgeKind::Defines, "brick-lexical", @@ -452,10 +518,13 @@ impl ScanState { None } else { self.symbols_by_name - .get(name) + .get(&(language.into(), name.into())) .or_else(|| { (!name.contains("::")) - .then(|| self.symbols_by_name.get(leaf_name)) + .then(|| { + self.symbols_by_name + .get(&(language.into(), leaf_name.into())) + }) .flatten() }) .and_then(|ids| (ids.len() == 1).then(|| ids[0].clone())) @@ -522,8 +591,17 @@ impl ScanState { confidence: f64, provenance: impl Into, ) { + let provenance = provenance.into(); let key = (source.clone(), target.clone(), kind); if !self.edge_keys.insert(key) { + if let Some(edge) = self + .edges + .iter_mut() + .find(|edge| edge.source == source && edge.target == target && edge.kind == kind) + && !edge.evidence.locations.contains(&provenance) + { + edge.evidence.locations.push(provenance); + } return; } let identity = format!("{source}:{}:{target}", edge_kind_name(kind)); @@ -561,55 +639,140 @@ struct Scope { struct Patterns { class: Regex, - function: Regex, - import: Regex, - call: Regex, + function: Vec, + import: Vec, + call: Vec, + relationships: Option, + enclosing_scope: Option, } impl Patterns { fn new(language: &str) -> Self { - let (class, function, import) = match language { + let (class, functions, imports, relationships) = match language { "python" => ( r"^\s*class\s+(?P[A-Za-z_]\w*)(?:\((?P[A-Za-z_][\w.]*)\))?", - r"^\s*(?:async\s+)?def\s+(?P[A-Za-z_]\w*)\s*\(", - r"^\s*(?:from\s+(?P[A-Za-z_][\w.]*)\s+import|import\s+(?P[A-Za-z_][\w.]*))", + vec![r"^\s*(?:async\s+)?def\s+(?P[A-Za-z_]\w*)\s*\("], + vec![ + r"^\s*(?:from\s+(?P[A-Za-z_][\w.]*)\s+import|import\s+(?P[A-Za-z_][\w.]*))", + ], + None, ), "rust" => ( r"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:struct|enum|trait)\s+(?P[A-Za-z_]\w*)", - r"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:async\s+)?fn\s+(?P[A-Za-z_]\w*)\s*(?:<[^>]*>)?\s*\(", - r"^\s*use\s+(?P[A-Za-z_][\w:]*)", + vec![ + r#"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:(?:const|async|unsafe)\s+)*(?:extern(?:\s+"[^"]+")?\s+)?fn\s+(?P[A-Za-z_]\w*)\s*(?:<[^>]*>)?\s*\("#, + ], + vec![r"^\s*use\s+(?P[A-Za-z_][\w:]*)"], + None, ), - "javascript" | "typescript" => ( - r"^\s*(?:export\s+)?class\s+(?P[A-Za-z_$][\w$]*)(?:\s+extends\s+(?P[A-Za-z_$][\w$]*))?", - r"^\s*(?:export\s+)?(?:async\s+)?function\s+(?P[A-Za-z_$][\w$]*)\s*\(", - r#"^\s*import(?:[^'\"]*from\s*)?['\"](?P[^'\"]+)['\"]"#, + "c" => ( + r"^\s*(?:typedef\s+)?(?:struct|union|enum)\s+(?P[A-Za-z_]\w*)", + vec![r"^\s*(?:[\w*&]+\s+)+(?P[A-Za-z_]\w*)\s*\([^;{}]*\)\s*\{?\s*$"], + vec![r#"^\s*#\s*include\s*[<\"](?P[^>\"]+)[>\"]"#], + None, + ), + "cpp" => ( + r"^\s*(?:class|struct|union|enum(?:\s+class)?)\s+(?P[A-Za-z_]\w*)(?:\s+final)?", + vec![ + r"^\s*(?:[\w:<>,*&]+\s+)+(?P[A-Za-z_]\w*(?:::\w+)*)\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?(?:(?:override|final)\s*)*(?:\{|=\s*(?:default|delete)\s*;)?\s*$", + r"^\s*(?P~?[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?::[^{}]*)?\s*(?:noexcept\s*)?(?:\{\s*\}|\{|=\s*(?:default|delete)\s*;)\s*$", + r"^\s*(?:[\w:<>,*&]+\s+)+(?Poperator\s*(?:[^\s(]+|\(\)))\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?(?:\{|=\s*(?:default|delete)\s*;)\s*$", + ], + vec![r#"^\s*#\s*include\s*[<\"](?P[^>\"]+)[>\"]"#], + Some(r":\s*(?P[^\{]+)"), + ), + "javascript" => ( + r"^\s*(?:export\s+)?(?:default\s+)?class\s+(?P[A-Za-z_$][\w$]*)(?:\s+extends\s+(?P[A-Za-z_$][\w$]*))?", + vec![ + r"^\s*(?:export\s+)?(?:default\s+)?(?:async\s+)?function\s+(?P[A-Za-z_$][\w$]*)\s*\(", + r"^\s*(?:(?:static|async|get|set)\s+)*(?Pconstructor|[A-Za-z_$][\w$]*)\s*\([^)]*\)\s*\{", + r"^\s*(?:export\s+)?(?:const|let|var)\s+(?P[A-Za-z_$][\w$]*)\s*=\s*(?:async\s*)?(?:\([^)]*\)|[A-Za-z_$][\w$]*)\s*=>", + ], + javascript_import_patterns(), + Some(r"\bextends\s+(?P[A-Za-z_$][\w$]*)"), + ), + "typescript" => ( + r"^\s*(?:export\s+)?(?:default\s+)?(?:(?:abstract|declare)\s+)?(?:class|interface|enum|namespace)\s+(?P[A-Za-z_$][\w$]*)(?:\s*<[^>{}]+>)?", + vec![ + r"^\s*(?:export\s+)?(?:default\s+)?(?:async\s+)?function\s+(?P[A-Za-z_$][\w$]*)\s*(?:<[^>{}]+>)?\s*\(", + r"^\s*(?:(?:public|protected|private|static|abstract|async|get|set|readonly|override)\s+)*(?Pconstructor|[A-Za-z_$][\w$]*)\s*(?:<[^>{}]+>)?\s*\([^)]*\)\s*(?::[^;{]+)?\s*\{", + r"^\s*(?:export\s+)?(?:const|let|var)\s+(?P[A-Za-z_$][\w$]*)\s*(?::[^=]+)?=\s*(?:async\s*)?(?:<[^>{}]+>\s*)?(?:\([^)]*\)|[A-Za-z_$][\w$]*)\s*(?::[^=]+)?=>", + ], + javascript_import_patterns(), + Some( + r"\b(?:extends|implements)\s+(?P[A-Za-z_$][\w$]*(?:<[^>{}]+>)?(?:\s*,\s*[A-Za-z_$][\w$]*(?:<[^>{}]+>)?)*)", + ), ), "dart" => ( r"^\s*(?:(?:abstract|base|final|interface|sealed)\s+)?(?:class|mixin|enum|extension(?:\s+type)?)\s+(?P[A-Za-z_]\w*)(?:\s+extends\s+(?P[A-Za-z_]\w*))?", - r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*(?:(?:[A-Za-z_]\w*(?:<[^>{};()]+>)?[?!]?)\s+)?(?P[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:(?:async|sync)\*?\s*)?(?:\{|=>)", - r#"^\s*import\s+['\"](?P[^'\"]+)['\"]"#, + vec![ + r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*(?:(?:[A-Za-z_]\w*(?:<[^>{};()]+>)?[?!]?)\s+)?(?P[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:(?:async|sync)\*?\s*)?(?:\{|=>)", + r"^\s*(?P[A-Za-z_]\w*\.[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:(?:async|sync)\*?\s*)?(?:\{|=>|;)", + r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*(?:(?:[A-Za-z_]\w*(?:<[^>{};()]+>)?[?!]?)\s+)?get\s+(?P[A-Za-z_]\w*)\s*(?:=>|\{)", + r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*set\s+(?P[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:=>|\{|;)", + ], + vec![r#"^\s*(?:import|export|part)\s+['\"](?P[^'\"]+)['\"]"#], + Some( + r"\b(?:extends|with|implements|on)\s+(?P[A-Za-z_]\w*(?:\s*,\s*[A-Za-z_]\w*)*)", + ), ), _ => ( r"^\s*(?:class|struct)\s+(?P[A-Za-z_]\w*)(?:\s*:\s*(?:public\s+|protected\s+|private\s+)?(?P[A-Za-z_:]\w*(?:::\w+)*))?", - r"^\s*(?:[\w:<>,*&]+\s+)+(?P[A-Za-z_]\w*(?:::\w+)*)\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?\{?\s*$", - r#"^\s*#\s*include\s*[<\"](?P[^>\"]+)[>\"]"#, + vec![ + r"^\s*(?:[\w:<>,*&]+\s+)+(?P[A-Za-z_]\w*(?:::\w+)*)\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?\{?\s*$", + ], + vec![r#"^\s*#\s*include\s*[<\"](?P[^>\"]+)[>\"]"#], + None, ), }; + let mut calls = vec![r"(?P[A-Za-z_]\w*(?:::\w+)*)\s*\("]; + if language == "rust" { + calls.extend([ + r"(?P[A-Za-z_]\w*(?:::\w+)*)\s*::\s*<[^>]+>\s*\(", + r"(?P[A-Za-z_]\w*(?:::\w+)*)!\s*\(", + ]); + } Self { class: Regex::new(class).expect("class regex is static"), - function: Regex::new(function).expect("function regex is static"), - import: Regex::new(import).expect("import regex is static"), - call: Regex::new(r"(?P[A-Za-z_]\w*(?:::\w+)*)\s*\(") - .expect("call regex is static"), + function: functions + .into_iter() + .map(|pattern| Regex::new(pattern).expect("function regex is static")) + .collect(), + import: imports + .into_iter() + .map(|pattern| Regex::new(pattern).expect("import regex is static")) + .collect(), + call: calls + .into_iter() + .map(|pattern| Regex::new(pattern).expect("call regex is static")) + .collect(), + relationships: relationships + .map(|pattern| Regex::new(pattern).expect("relationship regex is static")), + enclosing_scope: (language == "rust").then(|| { + Regex::new( + r"^\s*impl(?:<[^>]*>)?\s+(?:(?:[A-Za-z_]\w*(?:::\w+)*)\s+for\s+)?(?P[A-Za-z_]\w*)", + ) + .expect("scope regex is static") + }), } } } +fn javascript_import_patterns() -> Vec<&'static str> { + vec![ + r#"^\s*import(?:\s+type)?(?:[^'\"]*from\s*)?['\"](?P[^'\"]+)['\"]"#, + r#"^\s*export(?:\s+type)?\s+[^'\"]*from\s*['\"](?P[^'\"]+)['\"]"#, + r#"\brequire\s*\(\s*['\"](?P[^'\"]+)['\"]\s*\)"#, + r#"\bimport\s*\(\s*['\"](?P[^'\"]+)['\"]\s*\)"#, + ] +} + fn language_for(path: &Path) -> Option<&'static str> { match path.extension()?.to_str()? { "rs" => Some("rust"), "py" => Some("python"), - "c" | "h" | "cc" | "cpp" | "cxx" | "hh" | "hpp" | "hxx" => Some("cpp"), + "c" => Some("c"), + "h" | "cc" | "cpp" | "cxx" | "hh" | "hpp" | "hxx" => Some("cpp"), "js" | "jsx" | "mjs" | "cjs" => Some("javascript"), "ts" | "tsx" | "mts" | "cts" => Some("typescript"), "swift" => Some("swift"), @@ -618,13 +781,93 @@ fn language_for(path: &Path) -> Option<&'static str> { } } -fn strip_line_comment<'a>(line: &'a str, language: &str) -> &'a str { - let marker = if language == "python" { '#' } else { '/' }; - if marker == '/' { - line.split_once("//").map_or(line, |(before, _)| before) - } else { - line.split_once('#').map_or(line, |(before, _)| before) +fn strip_line_comment(line: &str, language: &str) -> String { + let chars: Vec<_> = line.chars().collect(); + let mut quote = None; + let mut escaped = false; + let mut index = 0; + while index < chars.len() { + let ch = chars[index]; + if let Some(active_quote) = quote { + if escaped { + escaped = false; + } else if ch == '\\' { + escaped = true; + } else if ch == active_quote { + quote = None; + } + } else if matches!(ch, '\'' | '"') && has_closing_quote(&chars, index + 1, ch) { + quote = Some(ch); + } else if language == "python" && ch == '#' + || language != "python" && ch == '/' && chars.get(index + 1) == Some(&'/') + { + return chars[..index].iter().collect(); + } + index += 1; } + line.into() +} + +fn mask_block_comments(line: &str, depth: &mut usize) -> String { + let chars: Vec<_> = line.chars().collect(); + let mut result = String::with_capacity(line.len()); + let mut quote = None; + let mut escaped = false; + let mut index = 0; + while index < chars.len() { + let ch = chars[index]; + let next = chars.get(index + 1).copied(); + if *depth > 0 { + if ch == '/' && next == Some('*') { + *depth += 1; + result.push_str(" "); + index += 2; + } else if ch == '*' && next == Some('/') { + *depth -= 1; + result.push_str(" "); + index += 2; + } else { + result.push(' '); + index += 1; + } + } else if let Some(active_quote) = quote { + result.push(ch); + if escaped { + escaped = false; + } else if ch == '\\' { + escaped = true; + } else if ch == active_quote { + quote = None; + } + index += 1; + } else if matches!(ch, '\'' | '"') && has_closing_quote(&chars, index + 1, ch) { + quote = Some(ch); + result.push(ch); + index += 1; + } else if ch == '/' && next == Some('*') { + *depth = 1; + result.push_str(" "); + index += 2; + } else { + result.push(ch); + index += 1; + } + } + result +} + +fn has_closing_quote(chars: &[char], start: usize, quote: char) -> bool { + let mut escaped = false; + for ch in &chars[start..] { + if escaped { + escaped = false; + } else if *ch == '\\' { + escaped = true; + } else if *ch == quote { + return true; + } + } + false } fn brace_delta(line: &str) -> i32 { @@ -659,6 +902,29 @@ fn mask_quoted_strings(line: &str) -> String { result } +fn mask_javascript_templates(line: &str, open: &mut bool) -> String { + let mut result = String::with_capacity(line.len()); + let mut escaped = false; + for ch in line.chars() { + if *open { + if escaped { + escaped = false; + } else if ch == '\\' { + escaped = true; + } else if ch == '`' { + *open = false; + } + result.push(' '); + } else if ch == '`' { + *open = true; + result.push(' '); + } else { + result.push(ch); + } + } + result +} + fn rust_raw_string_start(line: &str) -> Option<(usize, usize, String)> { let bytes = line.as_bytes(); for start in 0..bytes.len() { @@ -687,6 +953,30 @@ fn clean_target(value: &str) -> String { .into() } +fn clean_import_target(value: &str, language: &str) -> String { + let target = clean_target(value); + if language == "rust" { + target.trim_end_matches("::").into() + } else { + target + } +} + +fn clean_relationship_target(value: &str, language: &str) -> String { + let target = clean_target(value); + if language == "cpp" { + target + .split_whitespace() + .filter(|part| !matches!(*part, "public" | "protected" | "private" | "virtual")) + .collect::>() + .join(" ") + } else if matches!(language, "javascript" | "typescript") { + target.split('<').next().unwrap_or(&target).trim().into() + } else { + target + } +} + fn import_keys(name: &str, language: &str) -> Vec { let mut keys = vec![name.to_owned()]; match language { @@ -747,15 +1037,39 @@ fn is_keyword(name: &str) -> bool { | "assert" | "assert_eq" | "println" + | "import" + | "require" ) } #[cfg(test)] mod tests { - use super::{ScanOptions, scan, scan_with_options}; + use super::{ScanOptions, mask_block_comments, scan, scan_with_options, strip_line_comment}; use crate::model::{EdgeKind, NodeKind}; use std::fs; + #[test] + fn masks_nested_block_comments_without_truncating_quoted_urls() { + let mut depth = 0; + assert_eq!( + strip_line_comment( + r#"import value from "https://example.invalid/value.js"; // trailing"#, + "javascript" + ), + r#"import value from "https://example.invalid/value.js"; "# + ); + assert_eq!( + mask_block_comments("code /* outer", &mut depth), + "code " + ); + assert_eq!(depth, 1); + let nested = "/* inner */ end */"; + let masked = mask_block_comments(nested, &mut depth); + assert_eq!(masked.len(), nested.len()); + assert!(masked.trim().is_empty()); + assert_eq!(depth, 0); + } + #[test] fn scans_functions_calls_and_cycles() { let dir = tempfile::tempdir().unwrap(); @@ -953,6 +1267,137 @@ mod tests { ); } + #[test] + fn does_not_resolve_calls_across_language_boundaries() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("library.rs"), "fn shared() {}\n").unwrap(); + fs::write( + dir.path().join("application.js"), + "function local() {}\nfunction caller() { local(); shared(); }\n", + ) + .unwrap(); + + let building = scan(dir.path()).unwrap(); + let caller = building + .nodes + .iter() + .find(|node| node.language.as_deref() == Some("javascript") && node.name == "caller") + .unwrap(); + let javascript_local = building + .nodes + .iter() + .find(|node| node.language.as_deref() == Some("javascript") && node.name == "local") + .unwrap(); + let rust_shared = building + .nodes + .iter() + .find(|node| node.language.as_deref() == Some("rust") && node.name == "shared") + .unwrap(); + let caller_edges: Vec<_> = building + .edges + .iter() + .filter(|edge| edge.kind == EdgeKind::Calls && edge.source == caller.id) + .collect(); + + assert!( + caller_edges + .iter() + .any(|edge| edge.target == javascript_local.id) + ); + assert!( + !caller_edges + .iter() + .any(|edge| edge.target == rust_shared.id) + ); + assert!(caller_edges.iter().any(|edge| { + building.nodes.iter().any(|node| { + node.id == edge.target && node.kind == NodeKind::External && node.name == "shared" + }) + })); + } + + #[test] + fn parents_recognized_methods_to_their_enclosing_type() { + let dir = tempfile::tempdir().unwrap(); + fs::write( + dir.path().join("worker.dart"), + "class Worker {\n void execute() {}\n}\n", + ) + .unwrap(); + + let building = scan(dir.path()).unwrap(); + let worker = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Class && node.name == "Worker") + .unwrap(); + let execute = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Function && node.name == "execute") + .unwrap(); + + assert_eq!(execute.parent.as_deref(), Some(worker.id.as_str())); + assert!(building.edges.iter().any(|edge| { + edge.kind == EdgeKind::Defines && edge.source == worker.id && edge.target == execute.id + })); + } + + #[test] + fn does_not_promote_python_type_declarations_to_calls() { + let dir = tempfile::tempdir().unwrap(); + fs::write( + dir.path().join("worker.py"), + "class Base:\n pass\nclass Worker(Base):\n pass\n", + ) + .unwrap(); + + let building = scan(dir.path()).unwrap(); + assert!(building.edges.iter().all(|edge| { + edge.kind != EdgeKind::Calls + || !building + .nodes + .iter() + .any(|node| node.id == edge.target && node.name == "Worker") + })); + } + + #[test] + fn retains_every_location_for_a_deduplicated_dependency_edge() { + let dir = tempfile::tempdir().unwrap(); + fs::write( + dir.path().join("calls.rs"), + "fn helper() {}\nfn caller() {\n helper();\n helper();\n}\n", + ) + .unwrap(); + + let building = scan(dir.path()).unwrap(); + let caller = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Function && node.name == "caller") + .unwrap(); + let helper = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Function && node.name == "helper") + .unwrap(); + let call = building + .edges + .iter() + .find(|edge| { + edge.kind == EdgeKind::Calls && edge.source == caller.id && edge.target == helper.id + }) + .unwrap(); + let serialized = serde_json::to_value(&call.evidence).unwrap(); + + assert_eq!(call.evidence.provenance, "calls.rs:3"); + assert_eq!( + serialized["locations"], + serde_json::json!(["calls.rs:3", "calls.rs:4"]) + ); + } + #[test] fn scans_dart_structure_without_promoting_control_flow_or_strings() { let dir = tempfile::tempdir().unwrap(); diff --git a/tests/benchmarks/README.md b/tests/benchmarks/README.md index fa4087f..bf6db65 100644 --- a/tests/benchmarks/README.md +++ b/tests/benchmarks/README.md @@ -5,6 +5,7 @@ xplan and Fold's optimization harness. python3 tests/benchmarks/runner.py self-test python3 tests/benchmarks/runner.py run lexical_smoke + python3 tests/benchmarks/runner.py run language_parity python3 tests/benchmarks/runner.py run yams_architecture --corpus /path/to/yams Each run writes a resolved plan, per-repeat graph/HTML artifacts, summary JSON, @@ -13,6 +14,11 @@ across repeats. The seed corpus has explicit positive cycle and relationship expectations; it is not representative enough for performance or language-wide precision claims. +The `language_parity` plan repeats the labeled language corpus three times, +requires stable graph identities, and gates minimum file and symbol evidence for +each advertised language independently. Python and Swift counts characterize +their secondary tier; they do not promote those profiles. + The `yams_architecture` plan is a real-corpus integration benchmark. It records three stable-identity scans and excludes YAMS's checked-in `site` output with Brick's explicit `--exclude` option. Validate each resulting graph with YAMS's diff --git a/tests/benchmarks/plans/language_parity.json b/tests/benchmarks/plans/language_parity.json new file mode 100644 index 0000000..c586f3b --- /dev/null +++ b/tests/benchmarks/plans/language_parity.json @@ -0,0 +1,31 @@ +{ + "schema_version": 1, + "name": "language_parity", + "corpus": "tests/fixtures/languages", + "repeats": 3, + "expect": { + "min_symbols": 35, + "min_dependency_edges": 35, + "max_cycles": 2, + "language_min_symbols": { + "dart": 7, + "rust": 8, + "c": 5, + "cpp": 8, + "javascript": 8, + "typescript": 9, + "python": 4, + "swift": 2 + }, + "language_min_files": { + "dart": 1, + "rust": 2, + "c": 1, + "cpp": 1, + "javascript": 5, + "typescript": 3, + "python": 2, + "swift": 1 + } + } +} diff --git a/tests/benchmarks/runner.py b/tests/benchmarks/runner.py index 44f6ba9..edaa050 100755 --- a/tests/benchmarks/runner.py +++ b/tests/benchmarks/runner.py @@ -131,7 +131,9 @@ def command_for( ] -def fixture_assertions(metrics: dict, expected: dict) -> dict[str, bool]: +def fixture_assertions( + metrics: dict, expected: dict, model: dict | None = None +) -> dict[str, bool]: assertions = { "symbols": metrics["symbol_count"] >= expected["min_symbols"], "dependencies": metrics["dependency_edge_count"] @@ -141,6 +143,24 @@ def fixture_assertions(metrics: dict, expected: dict) -> dict[str, bool]: assertions["cycles"] = metrics["cycle_count"] == expected["cycles"] if "max_cycles" in expected: assertions["max_cycles"] = metrics["cycle_count"] <= expected["max_cycles"] + nodes = [] if model is None else model.get("nodes", []) + for language, minimum in expected.get("language_min_symbols", {}).items(): + assertions[f"{language}_symbols"] = ( + sum( + node.get("language") == language + and node.get("kind") in {"class", "function"} + for node in nodes + ) + >= minimum + ) + for language, minimum in expected.get("language_min_files", {}).items(): + assertions[f"{language}_files"] = ( + sum( + node.get("language") == language and node.get("kind") == "file" + for node in nodes + ) + >= minimum + ) return assertions @@ -215,7 +235,7 @@ def run_plan( metrics = samples[0]["metrics"] expected = plan["expect"] - assertions = fixture_assertions(metrics, expected) + assertions = fixture_assertions(metrics, expected, model) if not all(assertions.values()): raise AssertionError(f"fixture expectations failed: {assertions}; metrics={metrics}") diff --git a/tests/benchmarks/test_runner.py b/tests/benchmarks/test_runner.py index 843d5a4..77017c4 100644 --- a/tests/benchmarks/test_runner.py +++ b/tests/benchmarks/test_runner.py @@ -77,6 +77,32 @@ class RunnerTests(unittest.TestCase): {"symbols": True, "dependencies": True, "max_cycles": True}, ) + def test_language_expectations_cannot_be_satisfied_by_another_language(self) -> None: + metrics = {"symbol_count": 3, "dependency_edge_count": 2, "cycle_count": 0} + expected = { + "min_symbols": 2, + "min_dependency_edges": 1, + "cycles": 0, + "language_min_symbols": {"dart": 2, "rust": 2}, + "language_min_files": {"dart": 1, "rust": 1}, + } + model = { + "nodes": [ + {"kind": "file", "language": "dart"}, + {"kind": "class", "language": "dart"}, + {"kind": "function", "language": "dart"}, + {"kind": "file", "language": "rust"}, + {"kind": "function", "language": "rust"}, + ] + } + + assertions = runner.fixture_assertions(metrics, expected, model) + + self.assertTrue(assertions["dart_symbols"]) + self.assertTrue(assertions["dart_files"]) + self.assertFalse(assertions["rust_symbols"]) + self.assertTrue(assertions["rust_files"]) + if __name__ == "__main__": unittest.main() diff --git a/tests/fixtures/languages/c/basic.c b/tests/fixtures/languages/c/basic.c new file mode 100644 index 0000000..097d077 --- /dev/null +++ b/tests/fixtures/languages/c/basic.c @@ -0,0 +1,27 @@ +#include "support.h" + +struct Worker { + int state; +}; + +union Payload { + int integer; + double decimal; +}; + +enum State { + ready +}; + +int start() { + return helper(); +} + +int helper() { + return 0; +} + +/* +struct CommentGhost {}; +int comment_call() { return phantom_call(); } +*/ diff --git a/tests/fixtures/languages/cpp/basic.cpp b/tests/fixtures/languages/cpp/basic.cpp new file mode 100644 index 0000000..683d0cc --- /dev/null +++ b/tests/fixtures/languages/cpp/basic.cpp @@ -0,0 +1,26 @@ +#include "support.hpp" + +class Base {}; +class Runnable {}; + +template +class Worker final : public Base, private Runnable { +public: + Worker() {} + ~Worker() {} + + void start() override { + helper(); + } + + Worker& operator=(const Worker&) = default; +}; + +int helper() { + return 0; +} + +/* +class CommentGhost {}; +int comment_call() { return phantom_call(); } +*/ diff --git a/tests/fixtures/languages/dart/basic.dart b/tests/fixtures/languages/dart/basic.dart new file mode 100644 index 0000000..36a9695 --- /dev/null +++ b/tests/fixtures/languages/dart/basic.dart @@ -0,0 +1,24 @@ +import 'package:example/base.dart'; +export 'package:example/public.dart'; +part 'generated.dart'; + +mixin Logging {} + +abstract class Controller extends Base with Logging implements Runnable { + Controller.named(); + + String get label => 'controller'; + + set label(String value) {} + + void start() { + helper(); + } + + void helper() {} +} + +/* +class CommentGhost {} +void commentCall() { phantomCall(); } +*/ diff --git a/tests/fixtures/languages/javascript/api.js b/tests/fixtures/languages/javascript/api.js new file mode 100644 index 0000000..e9c0441 --- /dev/null +++ b/tests/fixtures/languages/javascript/api.js @@ -0,0 +1 @@ +export function api() {} diff --git a/tests/fixtures/languages/javascript/base.js b/tests/fixtures/languages/javascript/base.js new file mode 100644 index 0000000..421591a --- /dev/null +++ b/tests/fixtures/languages/javascript/base.js @@ -0,0 +1 @@ +export class Base {} diff --git a/tests/fixtures/languages/javascript/basic.js b/tests/fixtures/languages/javascript/basic.js new file mode 100644 index 0000000..b02cf45 --- /dev/null +++ b/tests/fixtures/languages/javascript/basic.js @@ -0,0 +1,27 @@ +import { Base } from './base.js'; +export { api } from './api.js'; + +const legacy = require('./legacy.cjs'); +const lazy = import('./lazy.js'); + +export default class Worker extends Base { + constructor() {} + + start() { + helper(); + } +} + +const arrow = () => helper(); + +function helper() {} + +const ignored = ` +class Phantom extends Ghost {} +function fake() { phantomCall(); } +`; + +/* +class CommentGhost {} +function commentCall() { phantomCall(); } +*/ diff --git a/tests/fixtures/languages/javascript/lazy.js b/tests/fixtures/languages/javascript/lazy.js new file mode 100644 index 0000000..603a6d6 --- /dev/null +++ b/tests/fixtures/languages/javascript/lazy.js @@ -0,0 +1 @@ +export function lazy() {} diff --git a/tests/fixtures/languages/javascript/legacy.cjs b/tests/fixtures/languages/javascript/legacy.cjs new file mode 100644 index 0000000..f053ebf --- /dev/null +++ b/tests/fixtures/languages/javascript/legacy.cjs @@ -0,0 +1 @@ +module.exports = {}; diff --git a/tests/fixtures/languages/python/base.py b/tests/fixtures/languages/python/base.py new file mode 100644 index 0000000..764e53e --- /dev/null +++ b/tests/fixtures/languages/python/base.py @@ -0,0 +1,2 @@ +class Base: + pass diff --git a/tests/fixtures/languages/python/basic.py b/tests/fixtures/languages/python/basic.py new file mode 100644 index 0000000..d83d0ac --- /dev/null +++ b/tests/fixtures/languages/python/basic.py @@ -0,0 +1,10 @@ +from base import Base + + +class Worker(Base): + async def start(self): + helper() + + +def helper(): + pass diff --git a/tests/fixtures/languages/rust/basic.rs b/tests/fixtures/languages/rust/basic.rs new file mode 100644 index 0000000..f460f8e --- /dev/null +++ b/tests/fixtures/languages/rust/basic.rs @@ -0,0 +1,28 @@ +mod support; + +use crate::support::{helper, Dependency}; + +pub struct Worker; + +impl Worker { + pub fn start() { + helper(); + generic::(); + trace!(); + } +} + +fn generic() {} + +pub const fn constant() -> usize { + 1 +} + +pub unsafe fn unsafe_op() {} + +pub extern "C" fn ffi_entry() {} + +/* +struct CommentGhost; +fn comment_call() { phantom_call(); } +*/ diff --git a/tests/fixtures/languages/rust/support.rs b/tests/fixtures/languages/rust/support.rs new file mode 100644 index 0000000..b91c795 --- /dev/null +++ b/tests/fixtures/languages/rust/support.rs @@ -0,0 +1,3 @@ +pub struct Dependency; + +pub fn helper() {} diff --git a/tests/fixtures/languages/swift/basic.swift b/tests/fixtures/languages/swift/basic.swift new file mode 100644 index 0000000..6180272 --- /dev/null +++ b/tests/fixtures/languages/swift/basic.swift @@ -0,0 +1,9 @@ +import Foundation + +protocol Runnable {} + +class Worker: Base { + func start() { + helper() + } +} diff --git a/tests/fixtures/languages/typescript/base.ts b/tests/fixtures/languages/typescript/base.ts new file mode 100644 index 0000000..338e7ef --- /dev/null +++ b/tests/fixtures/languages/typescript/base.ts @@ -0,0 +1 @@ +export class Base {} diff --git a/tests/fixtures/languages/typescript/basic.ts b/tests/fixtures/languages/typescript/basic.ts new file mode 100644 index 0000000..4fd6ce1 --- /dev/null +++ b/tests/fixtures/languages/typescript/basic.ts @@ -0,0 +1,32 @@ +import type { Base } from './base.js'; +export type { Contract } from './contract.js'; + +interface Runnable { + run(value: T): void; +} + +export abstract class Worker extends Base implements Runnable { + constructor() {} + + run(value: U): void { + helper(); + } +} + +const arrow = (value: T): T => identity(value); + +function identity(value: T): T { + return value; +} + +function helper(): void {} + +const ignored = ` +interface Phantom {} +function fake(): void { phantomCall(); } +`; + +/* +interface CommentGhost {} +function commentCall(): void { phantomCall(); } +*/ diff --git a/tests/fixtures/languages/typescript/contract.ts b/tests/fixtures/languages/typescript/contract.ts new file mode 100644 index 0000000..3a68456 --- /dev/null +++ b/tests/fixtures/languages/typescript/contract.ts @@ -0,0 +1 @@ +export interface Contract {} diff --git a/tests/language_parity.rs b/tests/language_parity.rs new file mode 100644 index 0000000..ca0d5bd --- /dev/null +++ b/tests/language_parity.rs @@ -0,0 +1,291 @@ +use brick::model::{EdgeKind, NodeKind}; +use brick::scan::scan; +use std::path::PathBuf; + +struct FixtureCase { + directory: &'static str, + language: &'static str, + classes: &'static [&'static str], + functions: &'static [&'static str], + imports: usize, + inherits: usize, + calls: usize, +} + +#[test] +fn first_class_language_fixtures_have_stable_baseline_evidence() { + let cases = [ + FixtureCase { + directory: "dart", + language: "dart", + classes: &["Controller", "Logging"], + functions: &["Controller.named", "helper", "label", "label", "start"], + imports: 3, + inherits: 3, + calls: 1, + }, + FixtureCase { + directory: "rust", + language: "rust", + classes: &["Dependency", "Worker"], + functions: &[ + "constant", + "ffi_entry", + "generic", + "helper", + "start", + "unsafe_op", + ], + imports: 1, + inherits: 0, + calls: 3, + }, + FixtureCase { + directory: "c", + language: "c", + classes: &["Payload", "State", "Worker"], + functions: &["helper", "start"], + imports: 1, + inherits: 0, + calls: 1, + }, + FixtureCase { + directory: "cpp", + language: "cpp", + classes: &["Base", "Runnable", "Worker"], + functions: &["Worker", "helper", "operator=", "start", "~Worker"], + imports: 1, + inherits: 2, + calls: 1, + }, + FixtureCase { + directory: "javascript", + language: "javascript", + classes: &["Base", "Worker"], + functions: &["api", "arrow", "constructor", "helper", "lazy", "start"], + imports: 4, + inherits: 1, + calls: 2, + }, + FixtureCase { + directory: "typescript", + language: "typescript", + classes: &["Base", "Contract", "Runnable", "Worker"], + functions: &["arrow", "constructor", "helper", "identity", "run"], + imports: 2, + inherits: 2, + calls: 2, + }, + ]; + + for case in cases { + assert_fixture(case); + } +} + +#[test] +fn rust_fixture_models_impl_ownership_and_local_imports() { + let root = fixture_root("rust"); + let building = scan(&root).unwrap(); + let worker = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Class && node.name == "Worker") + .unwrap(); + let start = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Function && node.name == "start") + .unwrap(); + let support = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::File && node.path.as_deref() == Some("support.rs")) + .unwrap(); + + assert_eq!(start.parent.as_deref(), Some(worker.id.as_str())); + assert!( + building + .edges + .iter() + .any(|edge| { edge.kind == EdgeKind::Imports && edge.target == support.id }) + ); +} + +#[test] +fn cpp_fixture_models_member_ownership_without_declaration_calls() { + let root = fixture_root("cpp"); + let building = scan(&root).unwrap(); + let worker = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Class && node.name == "Worker") + .unwrap(); + let members: Vec<_> = building + .nodes + .iter() + .filter(|node| node.kind == NodeKind::Function && node.parent.as_ref() == Some(&worker.id)) + .map(|node| node.name.as_str()) + .collect(); + + assert_eq!(members.len(), 4); + assert!(building.edges.iter().all(|edge| { + edge.kind != EdgeKind::Calls + || !building.nodes.iter().any(|node| { + node.id == edge.target + && matches!( + node.name.as_str(), + "Worker" | "~Worker" | "operator=" | "start" + ) + }) + })); +} + +#[test] +fn javascript_and_typescript_model_members_without_template_phantoms() { + for directory in ["javascript", "typescript"] { + let building = scan(&fixture_root(directory)).unwrap(); + let worker = building + .nodes + .iter() + .find(|node| node.kind == NodeKind::Class && node.name == "Worker") + .unwrap(); + let member_names: Vec<_> = building + .nodes + .iter() + .filter(|node| { + node.kind == NodeKind::Function && node.parent.as_ref() == Some(&worker.id) + }) + .map(|node| node.name.as_str()) + .collect(); + + assert_eq!(member_names.len(), 2, "{directory} member ownership"); + assert!(member_names.contains(&"constructor")); + assert!( + !building + .nodes + .iter() + .any(|node| { matches!(node.name.as_str(), "Phantom" | "fake" | "phantomCall") }) + ); + assert!( + building + .edges + .iter() + .filter(|edge| edge.kind == EdgeKind::Imports) + .all(|edge| building + .nodes + .iter() + .any(|node| { node.id == edge.target && node.kind == NodeKind::File })) + ); + } +} + +#[test] +fn secondary_languages_have_explicit_non_parity_characterization() { + let python = scan(&fixture_root("python")).unwrap(); + let python_classes: Vec<_> = python + .nodes + .iter() + .filter(|node| node.kind == NodeKind::Class) + .map(|node| node.name.as_str()) + .collect(); + let python_functions: Vec<_> = python + .nodes + .iter() + .filter(|node| node.kind == NodeKind::Function) + .map(|node| node.name.as_str()) + .collect(); + assert_eq!(python_classes.len(), 2); + assert!(python_classes.contains(&"Base")); + assert!(python_classes.contains(&"Worker")); + assert_eq!(python_functions.len(), 2); + assert!(python_functions.contains(&"helper")); + assert!(python_functions.contains(&"start")); + assert_eq!(edge_count(&python, EdgeKind::Imports), 1); + assert_eq!(edge_count(&python, EdgeKind::Inherits), 1); + assert_eq!(edge_count(&python, EdgeKind::Calls), 1); + + let swift = scan(&fixture_root("swift")).unwrap(); + let swift_symbols: Vec<_> = swift + .nodes + .iter() + .filter(|node| matches!(node.kind, NodeKind::Class | NodeKind::Function)) + .map(|node| node.name.as_str()) + .collect(); + assert_eq!(swift_symbols.len(), 2); + assert!(swift_symbols.contains(&"Worker")); + assert!(swift_symbols.contains(&"start")); + assert_eq!(edge_count(&swift, EdgeKind::Imports), 0); + assert_eq!(edge_count(&swift, EdgeKind::Inherits), 1); + assert_eq!(edge_count(&swift, EdgeKind::Calls), 1); +} + +fn assert_fixture(case: FixtureCase) { + let root = fixture_root(case.directory); + let building = scan(&root).unwrap(); + let mut classes: Vec<_> = building + .nodes + .iter() + .filter(|node| { + node.language.as_deref() == Some(case.language) && node.kind == NodeKind::Class + }) + .map(|node| node.name.as_str()) + .collect(); + let mut functions: Vec<_> = building + .nodes + .iter() + .filter(|node| { + node.language.as_deref() == Some(case.language) && node.kind == NodeKind::Function + }) + .map(|node| node.name.as_str()) + .collect(); + classes.sort_unstable(); + functions.sort_unstable(); + + assert_eq!(classes, case.classes, "{} class baseline", case.directory); + assert_eq!( + functions, case.functions, + "{} function baseline", + case.directory + ); + assert_eq!( + edge_count(&building, EdgeKind::Imports), + case.imports, + "{} import baseline", + case.directory + ); + assert_eq!( + edge_count(&building, EdgeKind::Inherits), + case.inherits, + "{} inheritance baseline", + case.directory + ); + assert_eq!( + edge_count(&building, EdgeKind::Calls), + case.calls, + "{} call baseline", + case.directory + ); + assert!( + !building.nodes.iter().any(|node| matches!( + node.name.as_str(), + "CommentGhost" | "commentCall" | "comment_call" | "phantom_call" + )), + "{} block-comment clean negative", + case.directory + ); +} + +fn fixture_root(directory: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/languages") + .join(directory) +} + +fn edge_count(building: &brick::Building, kind: EdgeKind) -> usize { + building + .edges + .iter() + .filter(|edge| edge.kind == kind) + .count() +}