feat: establish first-class lexical language parity #2
28 changed files with 1174 additions and 62 deletions
13
README.md
13
README.md
|
|
@ -6,10 +6,13 @@ elements, and dependency edges show calls, imports, and inheritance. The same
|
||||||
model is emitted as versioned JSON for other tools, including a future Fold
|
model is emitted as versioned JSON for other tools, including a future Fold
|
||||||
integration.
|
integration.
|
||||||
|
|
||||||
The initial extractor recognizes Rust, Dart, Python, C/C++,
|
The initial extractor provides first-class lexical profiles for Rust, Dart,
|
||||||
JavaScript/TypeScript, and Swift source extensions. Symbol extraction is intentionally conservative and
|
C/C++, and JavaScript/TypeScript, with explicitly secondary Python and Swift
|
||||||
lexical in v0.1; every relationship records its backend, confidence, and source
|
coverage. Symbol extraction is intentionally conservative and lexical in v0.1;
|
||||||
location so consumers can distinguish it from imported symbolic evidence.
|
every relationship records its backend, confidence, and all contributing source
|
||||||
|
locations so consumers can distinguish it from imported symbolic evidence.
|
||||||
|
See [language support and parity](docs/architecture/language-support.md) for the
|
||||||
|
construct-level coverage matrix and first-class promotion contract.
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
|
|
@ -54,6 +57,8 @@ default; an internal or vendored URL can be supplied with --cytoscape-url.
|
||||||
confidence.
|
confidence.
|
||||||
- A focused Rust test suite, quality gate, bounded auto-loop, and artifact-based
|
- A focused Rust test suite, quality gate, bounded auto-loop, and artifact-based
|
||||||
benchmark runner.
|
benchmark runner.
|
||||||
|
- A labeled language-parity corpus with independent gates for every advertised
|
||||||
|
language; Python and Swift remain characterization-only secondary profiles.
|
||||||
|
|
||||||
## Important limits
|
## Important limits
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,9 @@
|
||||||
|
|
||||||
- Versioned graph JSON.
|
- Versioned graph JSON.
|
||||||
- Fast lexical multi-language preview.
|
- Fast lexical multi-language preview.
|
||||||
- Dart classes, functions/methods, imports, inheritance, and local calls.
|
- First-class lexical profiles for Dart, Rust, C/C++, and
|
||||||
|
JavaScript/TypeScript, with labeled parity fixtures and per-language gates.
|
||||||
|
- Characterized secondary Python and Swift coverage.
|
||||||
- Fold SARIF/native finding overlay.
|
- Fold SARIF/native finding overlay.
|
||||||
- Interactive Cytoscape building.
|
- Interactive Cytoscape building.
|
||||||
- Transparent structural indicators and deterministic tests.
|
- Transparent structural indicators and deterministic tests.
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,10 @@ Every edge has:
|
||||||
- lane: syntactic, symbolic, imported, or heuristic;
|
- lane: syntactic, symbolic, imported, or heuristic;
|
||||||
- backend: producer and mode;
|
- backend: producer and mode;
|
||||||
- confidence: producer-calibrated value;
|
- confidence: producer-calibrated value;
|
||||||
- provenance: source location or report artifact.
|
- provenance: the first source location or report artifact, retained for v0.1
|
||||||
|
consumers;
|
||||||
|
- locations: every distinct source location contributing to a deduplicated
|
||||||
|
edge.
|
||||||
|
|
||||||
## Health indicator
|
## Health indicator
|
||||||
|
|
||||||
|
|
@ -50,4 +53,3 @@ should map Fold function/block/operation/value fact IDs directly to Brick
|
||||||
identities and retain source/binary lanes. Source proved, latent, heuristic, and
|
identities and retain source/binary lanes. Source proved, latent, heuristic, and
|
||||||
concrete-witness confidence must remain distinguishable. Binary evidence must
|
concrete-witness confidence must remain distinguishable. Binary evidence must
|
||||||
not be promoted to symbolic evidence without corroboration.
|
not be promoted to symbolic evidence without corroboration.
|
||||||
|
|
||||||
|
|
|
||||||
84
docs/architecture/language-support.md
Normal file
84
docs/architecture/language-support.md
Normal file
|
|
@ -0,0 +1,84 @@
|
||||||
|
# Language support and parity
|
||||||
|
|
||||||
|
Brick 0.1 recognizes source-file extensions for Rust, Dart, Python, C/C++,
|
||||||
|
JavaScript/TypeScript, and Swift. Recognition does not imply compiler-level
|
||||||
|
coverage: the current backend is deliberately lexical, line-oriented, and
|
||||||
|
evidence-labeled.
|
||||||
|
|
||||||
|
The first-class lexical profiles are Dart, Rust, C/C++, and
|
||||||
|
JavaScript/TypeScript. Here, parity means each profile has dedicated construct
|
||||||
|
patterns, relationship evidence, positive and clean-negative fixtures, stable
|
||||||
|
regression assertions, and an independent benchmark gate. It does not mean the
|
||||||
|
languages have identical grammar features or compiler-grade resolution. Python
|
||||||
|
and Swift remain secondary until their extraction contracts and fixtures reach
|
||||||
|
the same standard.
|
||||||
|
|
||||||
|
Plain `.h` files remain classified as C++ because their language cannot be
|
||||||
|
determined from the extension alone. C projects that require header symbol
|
||||||
|
resolution need compilation-database or include-context evidence in a future
|
||||||
|
frontend; Brick does not guess that ownership.
|
||||||
|
|
||||||
|
## Current coverage
|
||||||
|
|
||||||
|
| Language | Useful current coverage | Important gaps | Tier |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Dart | Classes, mixins, enums, extensions, methods, named constructors, accessors, import/export/part directives, `extends`/`with`/`implements`/`on`, and ordinary calls | operators, generic relationship targets, conditional directives, and multiline declarations | first-class lexical |
|
||||||
|
| Rust | Structs, enums, traits, ordinary/async/const/unsafe/extern functions, simple inherent/trait impl ownership, ordinary/turbofish/macro calls, braced `use` resolution, and raw-string masking | module ownership, unions/type aliases, nested generic impl headers, glob imports, and macro definitions | first-class lexical |
|
||||||
|
| C | Distinct `.c` identity, includes, structs/unions/enums, narrow function forms, and ordinary calls | anonymous typedef aggregates, function pointers, C23 attributes, macros, and multiline declarations | first-class lexical |
|
||||||
|
| C++ | Classes/structs/unions/enums, `final`, constructors/destructors/operators, override-qualified methods, multiple inheritance, ordinary calls, and local include resolution | namespaces, overload disambiguation, lambdas, concepts/requires, complex template bases, and multiline declarations | first-class lexical |
|
||||||
|
| JavaScript | Default/exported classes, inheritance, owned methods/constructors, named functions, arrows, ES imports/re-exports, CommonJS `require`, dynamic imports, ordinary calls, and multiline template masking | function expressions, computed methods, decorators, JSX structure, package exports, and type-aware member calls | first-class lexical |
|
||||||
|
| TypeScript | JavaScript forms plus interfaces/enums/namespaces, generic and abstract classes/functions/methods, `implements`, type imports/re-exports, and typed arrows | type aliases, overload declarations, decorators, declaration merging, TSX structure, and path mappings | first-class lexical |
|
||||||
|
| Python | Characterized fixture coverage for basic classes, functions/async functions, imports, calls, inheritance, and indentation scopes | multiple bases, qualified ownership, decorators, multiline strings, relative/dynamic imports, and complete multi-import handling | secondary |
|
||||||
|
| Swift | Characterized extension recognition plus incidental simple `class`, `func`, inheritance, ownership, and call matching | no dedicated profile; imports, protocols, actors, enums, extensions, async/generic forms, and Swift-specific signatures are incomplete | secondary |
|
||||||
|
|
||||||
|
## Cross-language invariants
|
||||||
|
|
||||||
|
All extractors must satisfy these rules before first-class parity is claimed:
|
||||||
|
|
||||||
|
1. Bare symbol names never resolve across language boundaries. Cross-language
|
||||||
|
edges require explicit import, binding, or FFI evidence.
|
||||||
|
2. Recognized methods are owned by their enclosing type rather than only by the
|
||||||
|
source file.
|
||||||
|
3. Unsupported declarations do not silently become call edges.
|
||||||
|
4. Strings, comments, and language-specific literal forms do not create symbols
|
||||||
|
or calls.
|
||||||
|
5. Duplicate names use stable qualified identities and do not resolve
|
||||||
|
ambiguously.
|
||||||
|
6. Every edge retains backend, confidence, its first provenance value, and all
|
||||||
|
distinct source locations contributing to a deduplicated relationship.
|
||||||
|
|
||||||
|
## Parity test contract
|
||||||
|
|
||||||
|
Each first-class language needs deterministic positive and clean-negative
|
||||||
|
fixtures for:
|
||||||
|
|
||||||
|
- files, types, functions, methods, and containment;
|
||||||
|
- imports and inheritance/implementation relationships;
|
||||||
|
- local, qualified, member, ambiguous, and unresolved calls;
|
||||||
|
- duplicate symbols, nested scopes, strings, comments, and unsupported syntax;
|
||||||
|
- stable graph identities and evidence under repeated scans.
|
||||||
|
|
||||||
|
The positive baseline corpus lives in `tests/fixtures/languages`, with its
|
||||||
|
shared contract in `tests/language_parity.rs`. Gap slices should extend these
|
||||||
|
fixtures with a positive construct and its nearest clean negative before
|
||||||
|
changing extraction behavior.
|
||||||
|
|
||||||
|
Real-repository benchmarks supplement these fixtures but do not replace them.
|
||||||
|
Compare graph counts and health metrics only after confirming that analyzed
|
||||||
|
files, constructs, and evidence coverage remain stable. A lower health score can
|
||||||
|
be more accurate when a false resolved edge becomes an honest unresolved edge.
|
||||||
|
|
||||||
|
Python and Swift fixtures intentionally describe their current secondary tier in
|
||||||
|
the same corpus. Their assertions are characterization tests, not evidence of
|
||||||
|
first-class parity. In particular, the Swift fixture preserves the visible gap
|
||||||
|
between file/type recognition and Swift-specific relationship extraction.
|
||||||
|
|
||||||
|
## Delivered foundation order
|
||||||
|
|
||||||
|
1. Shared resolver and scope foundation.
|
||||||
|
2. A golden fixture harness and mixed-language collision corpus.
|
||||||
|
3. Dart and Rust parity slices.
|
||||||
|
4. Separate C and C++ extraction profiles and parity slices.
|
||||||
|
5. Separate JavaScript and TypeScript extraction profiles and parity slices.
|
||||||
|
6. Explicit Python and Swift characterization, followed by a separate promotion
|
||||||
|
decision if warranted.
|
||||||
|
|
@ -57,6 +57,7 @@ pub fn import_findings(building: &mut Building, report: &Path) -> Result<usize>
|
||||||
lines_of_code: 0,
|
lines_of_code: 0,
|
||||||
attributes,
|
attributes,
|
||||||
});
|
});
|
||||||
|
let provenance = format!("{}:{}", report.display(), finding.line);
|
||||||
building.edges.push(Edge {
|
building.edges.push(Edge {
|
||||||
id: stable_id("edge", &format!("{node_id}:flags:{file_id}")),
|
id: stable_id("edge", &format!("{node_id}:flags:{file_id}")),
|
||||||
source: node_id,
|
source: node_id,
|
||||||
|
|
@ -70,7 +71,8 @@ pub fn import_findings(building: &mut Building, report: &Path) -> Result<usize>
|
||||||
},
|
},
|
||||||
backend: format!("fold:discover:{}", finding.backend),
|
backend: format!("fold:discover:{}", finding.backend),
|
||||||
confidence: confidence_value(&finding.confidence),
|
confidence: confidence_value(&finding.confidence),
|
||||||
provenance: format!("{}:{}", report.display(), finding.line),
|
locations: vec![provenance.clone()],
|
||||||
|
provenance,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
imported += 1;
|
imported += 1;
|
||||||
|
|
|
||||||
|
|
@ -72,6 +72,8 @@ pub struct Evidence {
|
||||||
pub backend: String,
|
pub backend: String,
|
||||||
pub confidence: f64,
|
pub confidence: f64,
|
||||||
pub provenance: String,
|
pub provenance: String,
|
||||||
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
|
pub locations: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
|
@ -142,10 +144,12 @@ pub fn stable_id(kind: &str, identity: &str) -> String {
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn evidence(backend: &str, confidence: f64, provenance: impl Into<String>) -> Evidence {
|
pub fn evidence(backend: &str, confidence: f64, provenance: impl Into<String>) -> Evidence {
|
||||||
|
let provenance = provenance.into();
|
||||||
Evidence {
|
Evidence {
|
||||||
lane: EvidenceLane::Syntactic,
|
lane: EvidenceLane::Syntactic,
|
||||||
backend: backend.into(),
|
backend: backend.into(),
|
||||||
confidence,
|
confidence,
|
||||||
provenance: provenance.into(),
|
locations: vec![provenance.clone()],
|
||||||
|
provenance,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
541
src/scan.rs
541
src/scan.rs
|
|
@ -94,7 +94,7 @@ struct ScanState {
|
||||||
nodes: Vec<Node>,
|
nodes: Vec<Node>,
|
||||||
edges: Vec<Edge>,
|
edges: Vec<Edge>,
|
||||||
edge_keys: HashSet<(String, String, EdgeKind)>,
|
edge_keys: HashSet<(String, String, EdgeKind)>,
|
||||||
symbols_by_name: HashMap<String, Vec<String>>,
|
symbols_by_name: HashMap<(String, String), Vec<String>>,
|
||||||
files_by_key: HashMap<String, Vec<String>>,
|
files_by_key: HashMap<String, Vec<String>>,
|
||||||
pending: Vec<PendingRelationship>,
|
pending: Vec<PendingRelationship>,
|
||||||
diagnostics: Vec<Diagnostic>,
|
diagnostics: Vec<Diagnostic>,
|
||||||
|
|
@ -242,6 +242,8 @@ impl ScanState {
|
||||||
let mut scopes: Vec<Scope> = Vec::new();
|
let mut scopes: Vec<Scope> = Vec::new();
|
||||||
let mut brace_depth = 0_i32;
|
let mut brace_depth = 0_i32;
|
||||||
let mut rust_raw_end: Option<String> = None;
|
let mut rust_raw_end: Option<String> = None;
|
||||||
|
let mut javascript_template_open = false;
|
||||||
|
let mut block_comment_depth = 0_usize;
|
||||||
|
|
||||||
for (index, raw_line) in content.lines().enumerate() {
|
for (index, raw_line) in content.lines().enumerate() {
|
||||||
let line_number = index + 1;
|
let line_number = index + 1;
|
||||||
|
|
@ -251,7 +253,12 @@ impl ScanState {
|
||||||
}
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
let code_line = if language == "rust" {
|
let template_masked;
|
||||||
|
let code_line = if matches!(language, "javascript" | "typescript") {
|
||||||
|
template_masked =
|
||||||
|
mask_javascript_templates(raw_line, &mut javascript_template_open);
|
||||||
|
template_masked.as_str()
|
||||||
|
} else if language == "rust" {
|
||||||
if let Some((start, content_start, delimiter)) = rust_raw_string_start(raw_line) {
|
if let Some((start, content_start, delimiter)) = rust_raw_string_start(raw_line) {
|
||||||
if !raw_line[content_start..].contains(&delimiter) {
|
if !raw_line[content_start..].contains(&delimiter) {
|
||||||
rust_raw_end = Some(delimiter);
|
rust_raw_end = Some(delimiter);
|
||||||
|
|
@ -263,8 +270,9 @@ impl ScanState {
|
||||||
} else {
|
} else {
|
||||||
raw_line
|
raw_line
|
||||||
};
|
};
|
||||||
let line = strip_line_comment(code_line, language);
|
let block_masked = mask_block_comments(code_line, &mut block_comment_depth);
|
||||||
let call_line = mask_quoted_strings(line);
|
let line = strip_line_comment(&block_masked, language);
|
||||||
|
let call_line = mask_quoted_strings(&line);
|
||||||
let indent = raw_line.len() - raw_line.trim_start().len();
|
let indent = raw_line.len() - raw_line.trim_start().len();
|
||||||
if language == "python" && !line.trim().is_empty() {
|
if language == "python" && !line.trim().is_empty() {
|
||||||
while scopes.last().is_some_and(|scope| scope.indent >= indent) {
|
while scopes.last().is_some_and(|scope| scope.indent >= indent) {
|
||||||
|
|
@ -279,9 +287,28 @@ impl ScanState {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if let Some(captures) = patterns.class.captures(line)
|
if let Some(pattern) = patterns.enclosing_scope.as_ref()
|
||||||
|
&& let Some(name) = pattern
|
||||||
|
.captures(&line)
|
||||||
|
.and_then(|captures| captures.name("name"))
|
||||||
|
&& let Some(id) = self
|
||||||
|
.symbols_by_name
|
||||||
|
.get(&(language.into(), name.as_str().into()))
|
||||||
|
.and_then(|ids| (ids.len() == 1).then(|| ids[0].clone()))
|
||||||
|
&& self.node_kind(&id) == Some(NodeKind::Class)
|
||||||
|
{
|
||||||
|
scopes.push(Scope {
|
||||||
|
id,
|
||||||
|
indent,
|
||||||
|
brace_depth: brace_depth + 1,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut type_definition_name = None;
|
||||||
|
if let Some(captures) = patterns.class.captures(&line)
|
||||||
&& let Some(name) = captures.name("name")
|
&& let Some(name) = captures.name("name")
|
||||||
{
|
{
|
||||||
|
type_definition_name = Some(name.as_str().to_owned());
|
||||||
let id = self.add_symbol(
|
let id = self.add_symbol(
|
||||||
file_id,
|
file_id,
|
||||||
path,
|
path,
|
||||||
|
|
@ -290,10 +317,26 @@ impl ScanState {
|
||||||
NodeKind::Class,
|
NodeKind::Class,
|
||||||
line_number,
|
line_number,
|
||||||
);
|
);
|
||||||
if let Some(base) = captures.name("base") {
|
let mut bases: Vec<String> = patterns
|
||||||
|
.relationships
|
||||||
|
.as_ref()
|
||||||
|
.into_iter()
|
||||||
|
.flat_map(|pattern| pattern.captures_iter(&line))
|
||||||
|
.filter_map(|captures| captures.name("names"))
|
||||||
|
.flat_map(|names| names.as_str().split(','))
|
||||||
|
.map(str::trim)
|
||||||
|
.filter(|name| !name.is_empty())
|
||||||
|
.map(|name| clean_relationship_target(name, language))
|
||||||
|
.collect();
|
||||||
|
if bases.is_empty()
|
||||||
|
&& let Some(base) = captures.name("base")
|
||||||
|
{
|
||||||
|
bases.push(clean_target(base.as_str()));
|
||||||
|
}
|
||||||
|
for base in bases {
|
||||||
self.pending.push(PendingRelationship {
|
self.pending.push(PendingRelationship {
|
||||||
source: id.clone(),
|
source: id.clone(),
|
||||||
target_name: clean_target(base.as_str()),
|
target_name: base,
|
||||||
kind: EdgeKind::Inherits,
|
kind: EdgeKind::Inherits,
|
||||||
language: language.into(),
|
language: language.into(),
|
||||||
member_call: false,
|
member_call: false,
|
||||||
|
|
@ -310,16 +353,24 @@ impl ScanState {
|
||||||
}
|
}
|
||||||
|
|
||||||
let mut definition_name = None;
|
let mut definition_name = None;
|
||||||
if let Some(captures) = patterns.function.captures(line)
|
if let Some(name) = patterns.function.iter().find_map(|pattern| {
|
||||||
&& let Some(name) = captures.name("name")
|
pattern
|
||||||
&& !is_keyword(name.as_str())
|
.captures(&line)
|
||||||
|
.and_then(|captures| captures.name("name").map(|name| name.as_str().to_owned()))
|
||||||
|
}) && !is_keyword(&name)
|
||||||
{
|
{
|
||||||
definition_name = Some(name.as_str().to_owned());
|
definition_name = Some(name.clone());
|
||||||
|
let parent_id = scopes
|
||||||
|
.iter()
|
||||||
|
.rev()
|
||||||
|
.find(|scope| self.node_kind(&scope.id) == Some(NodeKind::Class))
|
||||||
|
.map(|scope| scope.id.as_str())
|
||||||
|
.unwrap_or(file_id);
|
||||||
let id = self.add_symbol(
|
let id = self.add_symbol(
|
||||||
file_id,
|
parent_id,
|
||||||
path,
|
path,
|
||||||
language,
|
language,
|
||||||
name.as_str(),
|
&name,
|
||||||
NodeKind::Function,
|
NodeKind::Function,
|
||||||
line_number,
|
line_number,
|
||||||
);
|
);
|
||||||
|
|
@ -337,11 +388,15 @@ impl ScanState {
|
||||||
.map(|scope| scope.id.clone())
|
.map(|scope| scope.id.clone())
|
||||||
.unwrap_or_else(|| file_id.into());
|
.unwrap_or_else(|| file_id.into());
|
||||||
|
|
||||||
for captures in patterns.import.captures_iter(line) {
|
for captures in patterns
|
||||||
|
.import
|
||||||
|
.iter()
|
||||||
|
.flat_map(|pattern| pattern.captures_iter(&line))
|
||||||
|
{
|
||||||
if let Some(target) = captures.name("name").or_else(|| captures.name("name2")) {
|
if let Some(target) = captures.name("name").or_else(|| captures.name("name2")) {
|
||||||
self.pending.push(PendingRelationship {
|
self.pending.push(PendingRelationship {
|
||||||
source: file_id.into(),
|
source: file_id.into(),
|
||||||
target_name: clean_target(target.as_str()),
|
target_name: clean_import_target(target.as_str(), language),
|
||||||
kind: EdgeKind::Imports,
|
kind: EdgeKind::Imports,
|
||||||
language: language.into(),
|
language: language.into(),
|
||||||
member_call: false,
|
member_call: false,
|
||||||
|
|
@ -352,14 +407,25 @@ impl ScanState {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
for captures in patterns.call.captures_iter(&call_line) {
|
for captures in patterns
|
||||||
|
.call
|
||||||
|
.iter()
|
||||||
|
.flat_map(|pattern| pattern.captures_iter(&call_line))
|
||||||
|
{
|
||||||
let Some(target_match) = captures.name("name") else {
|
let Some(target_match) = captures.name("name") else {
|
||||||
continue;
|
continue;
|
||||||
};
|
};
|
||||||
let target = target_match.as_str();
|
let target = target_match.as_str();
|
||||||
let prefix = call_line[..target_match.start()].trim_end();
|
let prefix = call_line[..target_match.start()].trim_end();
|
||||||
let member_call = prefix.ends_with('.') || prefix.ends_with("->");
|
let member_call = prefix.ends_with('.') || prefix.ends_with("->");
|
||||||
if definition_name.as_deref() != Some(target) && !is_keyword(target) {
|
let definition_call_name = definition_name
|
||||||
|
.as_deref()
|
||||||
|
.and_then(|name| name.rsplit('.').next())
|
||||||
|
.map(|name| name.trim_start_matches('~'));
|
||||||
|
if definition_call_name != Some(target)
|
||||||
|
&& type_definition_name.as_deref() != Some(target)
|
||||||
|
&& !is_keyword(target)
|
||||||
|
{
|
||||||
self.pending.push(PendingRelationship {
|
self.pending.push(PendingRelationship {
|
||||||
source: source.clone(),
|
source: source.clone(),
|
||||||
target_name: clean_target(target),
|
target_name: clean_target(target),
|
||||||
|
|
@ -378,7 +444,7 @@ impl ScanState {
|
||||||
|
|
||||||
fn add_symbol(
|
fn add_symbol(
|
||||||
&mut self,
|
&mut self,
|
||||||
file_id: &str,
|
parent_id: &str,
|
||||||
path: &str,
|
path: &str,
|
||||||
language: &str,
|
language: &str,
|
||||||
name: &str,
|
name: &str,
|
||||||
|
|
@ -397,18 +463,18 @@ impl ScanState {
|
||||||
kind,
|
kind,
|
||||||
name: name.into(),
|
name: name.into(),
|
||||||
path: Some(path.into()),
|
path: Some(path.into()),
|
||||||
parent: Some(file_id.into()),
|
parent: Some(parent_id.into()),
|
||||||
language: Some(language.into()),
|
language: Some(language.into()),
|
||||||
line: Some(line),
|
line: Some(line),
|
||||||
lines_of_code: 0,
|
lines_of_code: 0,
|
||||||
attributes: BTreeMap::new(),
|
attributes: BTreeMap::new(),
|
||||||
});
|
});
|
||||||
self.symbols_by_name
|
self.symbols_by_name
|
||||||
.entry(name.into())
|
.entry((language.into(), name.into()))
|
||||||
.or_default()
|
.or_default()
|
||||||
.push(id.clone());
|
.push(id.clone());
|
||||||
self.add_edge(
|
self.add_edge(
|
||||||
file_id.into(),
|
parent_id.into(),
|
||||||
id.clone(),
|
id.clone(),
|
||||||
EdgeKind::Defines,
|
EdgeKind::Defines,
|
||||||
"brick-lexical",
|
"brick-lexical",
|
||||||
|
|
@ -452,10 +518,13 @@ impl ScanState {
|
||||||
None
|
None
|
||||||
} else {
|
} else {
|
||||||
self.symbols_by_name
|
self.symbols_by_name
|
||||||
.get(name)
|
.get(&(language.into(), name.into()))
|
||||||
.or_else(|| {
|
.or_else(|| {
|
||||||
(!name.contains("::"))
|
(!name.contains("::"))
|
||||||
.then(|| self.symbols_by_name.get(leaf_name))
|
.then(|| {
|
||||||
|
self.symbols_by_name
|
||||||
|
.get(&(language.into(), leaf_name.into()))
|
||||||
|
})
|
||||||
.flatten()
|
.flatten()
|
||||||
})
|
})
|
||||||
.and_then(|ids| (ids.len() == 1).then(|| ids[0].clone()))
|
.and_then(|ids| (ids.len() == 1).then(|| ids[0].clone()))
|
||||||
|
|
@ -522,8 +591,17 @@ impl ScanState {
|
||||||
confidence: f64,
|
confidence: f64,
|
||||||
provenance: impl Into<String>,
|
provenance: impl Into<String>,
|
||||||
) {
|
) {
|
||||||
|
let provenance = provenance.into();
|
||||||
let key = (source.clone(), target.clone(), kind);
|
let key = (source.clone(), target.clone(), kind);
|
||||||
if !self.edge_keys.insert(key) {
|
if !self.edge_keys.insert(key) {
|
||||||
|
if let Some(edge) = self
|
||||||
|
.edges
|
||||||
|
.iter_mut()
|
||||||
|
.find(|edge| edge.source == source && edge.target == target && edge.kind == kind)
|
||||||
|
&& !edge.evidence.locations.contains(&provenance)
|
||||||
|
{
|
||||||
|
edge.evidence.locations.push(provenance);
|
||||||
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let identity = format!("{source}:{}:{target}", edge_kind_name(kind));
|
let identity = format!("{source}:{}:{target}", edge_kind_name(kind));
|
||||||
|
|
@ -561,55 +639,140 @@ struct Scope {
|
||||||
|
|
||||||
struct Patterns {
|
struct Patterns {
|
||||||
class: Regex,
|
class: Regex,
|
||||||
function: Regex,
|
function: Vec<Regex>,
|
||||||
import: Regex,
|
import: Vec<Regex>,
|
||||||
call: Regex,
|
call: Vec<Regex>,
|
||||||
|
relationships: Option<Regex>,
|
||||||
|
enclosing_scope: Option<Regex>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Patterns {
|
impl Patterns {
|
||||||
fn new(language: &str) -> Self {
|
fn new(language: &str) -> Self {
|
||||||
let (class, function, import) = match language {
|
let (class, functions, imports, relationships) = match language {
|
||||||
"python" => (
|
"python" => (
|
||||||
r"^\s*class\s+(?P<name>[A-Za-z_]\w*)(?:\((?P<base>[A-Za-z_][\w.]*)\))?",
|
r"^\s*class\s+(?P<name>[A-Za-z_]\w*)(?:\((?P<base>[A-Za-z_][\w.]*)\))?",
|
||||||
r"^\s*(?:async\s+)?def\s+(?P<name>[A-Za-z_]\w*)\s*\(",
|
vec![r"^\s*(?:async\s+)?def\s+(?P<name>[A-Za-z_]\w*)\s*\("],
|
||||||
|
vec![
|
||||||
r"^\s*(?:from\s+(?P<name>[A-Za-z_][\w.]*)\s+import|import\s+(?P<name2>[A-Za-z_][\w.]*))",
|
r"^\s*(?:from\s+(?P<name>[A-Za-z_][\w.]*)\s+import|import\s+(?P<name2>[A-Za-z_][\w.]*))",
|
||||||
|
],
|
||||||
|
None,
|
||||||
),
|
),
|
||||||
"rust" => (
|
"rust" => (
|
||||||
r"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:struct|enum|trait)\s+(?P<name>[A-Za-z_]\w*)",
|
r"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:struct|enum|trait)\s+(?P<name>[A-Za-z_]\w*)",
|
||||||
r"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:async\s+)?fn\s+(?P<name>[A-Za-z_]\w*)\s*(?:<[^>]*>)?\s*\(",
|
vec![
|
||||||
r"^\s*use\s+(?P<name>[A-Za-z_][\w:]*)",
|
r#"^\s*(?:pub(?:\([^)]*\))?\s+)?(?:(?:const|async|unsafe)\s+)*(?:extern(?:\s+"[^"]+")?\s+)?fn\s+(?P<name>[A-Za-z_]\w*)\s*(?:<[^>]*>)?\s*\("#,
|
||||||
|
],
|
||||||
|
vec![r"^\s*use\s+(?P<name>[A-Za-z_][\w:]*)"],
|
||||||
|
None,
|
||||||
|
),
|
||||||
|
"c" => (
|
||||||
|
r"^\s*(?:typedef\s+)?(?:struct|union|enum)\s+(?P<name>[A-Za-z_]\w*)",
|
||||||
|
vec![r"^\s*(?:[\w*&]+\s+)+(?P<name>[A-Za-z_]\w*)\s*\([^;{}]*\)\s*\{?\s*$"],
|
||||||
|
vec[>\"]"#],
|
||||||
|
None,
|
||||||
|
),
|
||||||
|
"cpp" => (
|
||||||
|
r"^\s*(?:class|struct|union|enum(?:\s+class)?)\s+(?P<name>[A-Za-z_]\w*)(?:\s+final)?",
|
||||||
|
vec![
|
||||||
|
r"^\s*(?:[\w:<>,*&]+\s+)+(?P<name>[A-Za-z_]\w*(?:::\w+)*)\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?(?:(?:override|final)\s*)*(?:\{|=\s*(?:default|delete)\s*;)?\s*$",
|
||||||
|
r"^\s*(?P<name>~?[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?::[^{}]*)?\s*(?:noexcept\s*)?(?:\{\s*\}|\{|=\s*(?:default|delete)\s*;)\s*$",
|
||||||
|
r"^\s*(?:[\w:<>,*&]+\s+)+(?P<name>operator\s*(?:[^\s(]+|\(\)))\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?(?:\{|=\s*(?:default|delete)\s*;)\s*$",
|
||||||
|
],
|
||||||
|
vec[>\"]"#],
|
||||||
|
Some(r":\s*(?P<names>[^\{]+)"),
|
||||||
|
),
|
||||||
|
"javascript" => (
|
||||||
|
r"^\s*(?:export\s+)?(?:default\s+)?class\s+(?P<name>[A-Za-z_$][\w$]*)(?:\s+extends\s+(?P<base>[A-Za-z_$][\w$]*))?",
|
||||||
|
vec![
|
||||||
|
r"^\s*(?:export\s+)?(?:default\s+)?(?:async\s+)?function\s+(?P<name>[A-Za-z_$][\w$]*)\s*\(",
|
||||||
|
r"^\s*(?:(?:static|async|get|set)\s+)*(?P<name>constructor|[A-Za-z_$][\w$]*)\s*\([^)]*\)\s*\{",
|
||||||
|
r"^\s*(?:export\s+)?(?:const|let|var)\s+(?P<name>[A-Za-z_$][\w$]*)\s*=\s*(?:async\s*)?(?:\([^)]*\)|[A-Za-z_$][\w$]*)\s*=>",
|
||||||
|
],
|
||||||
|
javascript_import_patterns(),
|
||||||
|
Some(r"\bextends\s+(?P<names>[A-Za-z_$][\w$]*)"),
|
||||||
|
),
|
||||||
|
"typescript" => (
|
||||||
|
r"^\s*(?:export\s+)?(?:default\s+)?(?:(?:abstract|declare)\s+)?(?:class|interface|enum|namespace)\s+(?P<name>[A-Za-z_$][\w$]*)(?:\s*<[^>{}]+>)?",
|
||||||
|
vec![
|
||||||
|
r"^\s*(?:export\s+)?(?:default\s+)?(?:async\s+)?function\s+(?P<name>[A-Za-z_$][\w$]*)\s*(?:<[^>{}]+>)?\s*\(",
|
||||||
|
r"^\s*(?:(?:public|protected|private|static|abstract|async|get|set|readonly|override)\s+)*(?P<name>constructor|[A-Za-z_$][\w$]*)\s*(?:<[^>{}]+>)?\s*\([^)]*\)\s*(?::[^;{]+)?\s*\{",
|
||||||
|
r"^\s*(?:export\s+)?(?:const|let|var)\s+(?P<name>[A-Za-z_$][\w$]*)\s*(?::[^=]+)?=\s*(?:async\s*)?(?:<[^>{}]+>\s*)?(?:\([^)]*\)|[A-Za-z_$][\w$]*)\s*(?::[^=]+)?=>",
|
||||||
|
],
|
||||||
|
javascript_import_patterns(),
|
||||||
|
Some(
|
||||||
|
r"\b(?:extends|implements)\s+(?P<names>[A-Za-z_$][\w$]*(?:<[^>{}]+>)?(?:\s*,\s*[A-Za-z_$][\w$]*(?:<[^>{}]+>)?)*)",
|
||||||
),
|
),
|
||||||
"javascript" | "typescript" => (
|
|
||||||
r"^\s*(?:export\s+)?class\s+(?P<name>[A-Za-z_$][\w$]*)(?:\s+extends\s+(?P<base>[A-Za-z_$][\w$]*))?",
|
|
||||||
r"^\s*(?:export\s+)?(?:async\s+)?function\s+(?P<name>[A-Za-z_$][\w$]*)\s*\(",
|
|
||||||
r#"^\s*import(?:[^'\"]*from\s*)?['\"](?P<name>[^'\"]+)['\"]"#,
|
|
||||||
),
|
),
|
||||||
"dart" => (
|
"dart" => (
|
||||||
r"^\s*(?:(?:abstract|base|final|interface|sealed)\s+)?(?:class|mixin|enum|extension(?:\s+type)?)\s+(?P<name>[A-Za-z_]\w*)(?:\s+extends\s+(?P<base>[A-Za-z_]\w*))?",
|
r"^\s*(?:(?:abstract|base|final|interface|sealed)\s+)?(?:class|mixin|enum|extension(?:\s+type)?)\s+(?P<name>[A-Za-z_]\w*)(?:\s+extends\s+(?P<base>[A-Za-z_]\w*))?",
|
||||||
|
vec![
|
||||||
r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*(?:(?:[A-Za-z_]\w*(?:<[^>{};()]+>)?[?!]?)\s+)?(?P<name>[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:(?:async|sync)\*?\s*)?(?:\{|=>)",
|
r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*(?:(?:[A-Za-z_]\w*(?:<[^>{};()]+>)?[?!]?)\s+)?(?P<name>[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:(?:async|sync)\*?\s*)?(?:\{|=>)",
|
||||||
r#"^\s*import\s+['\"](?P<name>[^'\"]+)['\"]"#,
|
r"^\s*(?P<name>[A-Za-z_]\w*\.[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:(?:async|sync)\*?\s*)?(?:\{|=>|;)",
|
||||||
|
r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*(?:(?:[A-Za-z_]\w*(?:<[^>{};()]+>)?[?!]?)\s+)?get\s+(?P<name>[A-Za-z_]\w*)\s*(?:=>|\{)",
|
||||||
|
r"^\s*(?:(?:external|static|abstract|covariant|late)\s+)*set\s+(?P<name>[A-Za-z_]\w*)\s*\([^;{}]*\)\s*(?:=>|\{|;)",
|
||||||
|
],
|
||||||
|
vec['\"]"#],
|
||||||
|
Some(
|
||||||
|
r"\b(?:extends|with|implements|on)\s+(?P<names>[A-Za-z_]\w*(?:\s*,\s*[A-Za-z_]\w*)*)",
|
||||||
|
),
|
||||||
),
|
),
|
||||||
_ => (
|
_ => (
|
||||||
r"^\s*(?:class|struct)\s+(?P<name>[A-Za-z_]\w*)(?:\s*:\s*(?:public\s+|protected\s+|private\s+)?(?P<base>[A-Za-z_:]\w*(?:::\w+)*))?",
|
r"^\s*(?:class|struct)\s+(?P<name>[A-Za-z_]\w*)(?:\s*:\s*(?:public\s+|protected\s+|private\s+)?(?P<base>[A-Za-z_:]\w*(?:::\w+)*))?",
|
||||||
|
vec![
|
||||||
r"^\s*(?:[\w:<>,*&]+\s+)+(?P<name>[A-Za-z_]\w*(?:::\w+)*)\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?\{?\s*$",
|
r"^\s*(?:[\w:<>,*&]+\s+)+(?P<name>[A-Za-z_]\w*(?:::\w+)*)\s*\([^;{}]*\)\s*(?:const\s*)?(?:noexcept\s*)?\{?\s*$",
|
||||||
r#"^\s*#\s*include\s*[<\"](?P<name>[^>\"]+)[>\"]"#,
|
],
|
||||||
|
vec[>\"]"#],
|
||||||
|
None,
|
||||||
),
|
),
|
||||||
};
|
};
|
||||||
|
let mut calls = vec![r"(?P<name>[A-Za-z_]\w*(?:::\w+)*)\s*\("];
|
||||||
|
if language == "rust" {
|
||||||
|
calls.extend([
|
||||||
|
r"(?P<name>[A-Za-z_]\w*(?:::\w+)*)\s*::\s*<[^>]+>\s*\(",
|
||||||
|
r"(?P<name>[A-Za-z_]\w*(?:::\w+)*)!\s*\(",
|
||||||
|
]);
|
||||||
|
}
|
||||||
Self {
|
Self {
|
||||||
class: Regex::new(class).expect("class regex is static"),
|
class: Regex::new(class).expect("class regex is static"),
|
||||||
function: Regex::new(function).expect("function regex is static"),
|
function: functions
|
||||||
import: Regex::new(import).expect("import regex is static"),
|
.into_iter()
|
||||||
call: Regex::new(r"(?P<name>[A-Za-z_]\w*(?:::\w+)*)\s*\(")
|
.map(|pattern| Regex::new(pattern).expect("function regex is static"))
|
||||||
.expect("call regex is static"),
|
.collect(),
|
||||||
|
import: imports
|
||||||
|
.into_iter()
|
||||||
|
.map(|pattern| Regex::new(pattern).expect("import regex is static"))
|
||||||
|
.collect(),
|
||||||
|
call: calls
|
||||||
|
.into_iter()
|
||||||
|
.map(|pattern| Regex::new(pattern).expect("call regex is static"))
|
||||||
|
.collect(),
|
||||||
|
relationships: relationships
|
||||||
|
.map(|pattern| Regex::new(pattern).expect("relationship regex is static")),
|
||||||
|
enclosing_scope: (language == "rust").then(|| {
|
||||||
|
Regex::new(
|
||||||
|
r"^\s*impl(?:<[^>]*>)?\s+(?:(?:[A-Za-z_]\w*(?:::\w+)*)\s+for\s+)?(?P<name>[A-Za-z_]\w*)",
|
||||||
|
)
|
||||||
|
.expect("scope regex is static")
|
||||||
|
}),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn javascript_import_patterns() -> Vec<&'static str> {
|
||||||
|
vec![
|
||||||
|
r#"^\s*import(?:\s+type)?(?:[^'\"]*from\s*)?['\"](?P<name>[^'\"]+)['\"]"#,
|
||||||
|
r#"^\s*export(?:\s+type)?\s+[^'\"]*from\s*['\"](?P<name>[^'\"]+)['\"]"#,
|
||||||
|
r#"\brequire\s*\(\s*['\"](?P<name>[^'\"]+)['\"]\s*\)"#,
|
||||||
|
r#"\bimport\s*\(\s*['\"](?P<name>[^'\"]+)['\"]\s*\)"#,
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
fn language_for(path: &Path) -> Option<&'static str> {
|
fn language_for(path: &Path) -> Option<&'static str> {
|
||||||
match path.extension()?.to_str()? {
|
match path.extension()?.to_str()? {
|
||||||
"rs" => Some("rust"),
|
"rs" => Some("rust"),
|
||||||
"py" => Some("python"),
|
"py" => Some("python"),
|
||||||
"c" | "h" | "cc" | "cpp" | "cxx" | "hh" | "hpp" | "hxx" => Some("cpp"),
|
"c" => Some("c"),
|
||||||
|
"h" | "cc" | "cpp" | "cxx" | "hh" | "hpp" | "hxx" => Some("cpp"),
|
||||||
"js" | "jsx" | "mjs" | "cjs" => Some("javascript"),
|
"js" | "jsx" | "mjs" | "cjs" => Some("javascript"),
|
||||||
"ts" | "tsx" | "mts" | "cts" => Some("typescript"),
|
"ts" | "tsx" | "mts" | "cts" => Some("typescript"),
|
||||||
"swift" => Some("swift"),
|
"swift" => Some("swift"),
|
||||||
|
|
@ -618,13 +781,93 @@ fn language_for(path: &Path) -> Option<&'static str> {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn strip_line_comment<'a>(line: &'a str, language: &str) -> &'a str {
|
fn strip_line_comment(line: &str, language: &str) -> String {
|
||||||
let marker = if language == "python" { '#' } else { '/' };
|
let chars: Vec<_> = line.chars().collect();
|
||||||
if marker == '/' {
|
let mut quote = None;
|
||||||
line.split_once("//").map_or(line, |(before, _)| before)
|
let mut escaped = false;
|
||||||
} else {
|
let mut index = 0;
|
||||||
line.split_once('#').map_or(line, |(before, _)| before)
|
while index < chars.len() {
|
||||||
|
let ch = chars[index];
|
||||||
|
if let Some(active_quote) = quote {
|
||||||
|
if escaped {
|
||||||
|
escaped = false;
|
||||||
|
} else if ch == '\\' {
|
||||||
|
escaped = true;
|
||||||
|
} else if ch == active_quote {
|
||||||
|
quote = None;
|
||||||
}
|
}
|
||||||
|
} else if matches!(ch, '\'' | '"') && has_closing_quote(&chars, index + 1, ch) {
|
||||||
|
quote = Some(ch);
|
||||||
|
} else if language == "python" && ch == '#'
|
||||||
|
|| language != "python" && ch == '/' && chars.get(index + 1) == Some(&'/')
|
||||||
|
{
|
||||||
|
return chars[..index].iter().collect();
|
||||||
|
}
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
line.into()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mask_block_comments(line: &str, depth: &mut usize) -> String {
|
||||||
|
let chars: Vec<_> = line.chars().collect();
|
||||||
|
let mut result = String::with_capacity(line.len());
|
||||||
|
let mut quote = None;
|
||||||
|
let mut escaped = false;
|
||||||
|
let mut index = 0;
|
||||||
|
while index < chars.len() {
|
||||||
|
let ch = chars[index];
|
||||||
|
let next = chars.get(index + 1).copied();
|
||||||
|
if *depth > 0 {
|
||||||
|
if ch == '/' && next == Some('*') {
|
||||||
|
*depth += 1;
|
||||||
|
result.push_str(" ");
|
||||||
|
index += 2;
|
||||||
|
} else if ch == '*' && next == Some('/') {
|
||||||
|
*depth -= 1;
|
||||||
|
result.push_str(" ");
|
||||||
|
index += 2;
|
||||||
|
} else {
|
||||||
|
result.push(' ');
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
} else if let Some(active_quote) = quote {
|
||||||
|
result.push(ch);
|
||||||
|
if escaped {
|
||||||
|
escaped = false;
|
||||||
|
} else if ch == '\\' {
|
||||||
|
escaped = true;
|
||||||
|
} else if ch == active_quote {
|
||||||
|
quote = None;
|
||||||
|
}
|
||||||
|
index += 1;
|
||||||
|
} else if matches!(ch, '\'' | '"') && has_closing_quote(&chars, index + 1, ch) {
|
||||||
|
quote = Some(ch);
|
||||||
|
result.push(ch);
|
||||||
|
index += 1;
|
||||||
|
} else if ch == '/' && next == Some('*') {
|
||||||
|
*depth = 1;
|
||||||
|
result.push_str(" ");
|
||||||
|
index += 2;
|
||||||
|
} else {
|
||||||
|
result.push(ch);
|
||||||
|
index += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result
|
||||||
|
}
|
||||||
|
|
||||||
|
fn has_closing_quote(chars: &[char], start: usize, quote: char) -> bool {
|
||||||
|
let mut escaped = false;
|
||||||
|
for ch in &chars[start..] {
|
||||||
|
if escaped {
|
||||||
|
escaped = false;
|
||||||
|
} else if *ch == '\\' {
|
||||||
|
escaped = true;
|
||||||
|
} else if *ch == quote {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
false
|
||||||
}
|
}
|
||||||
|
|
||||||
fn brace_delta(line: &str) -> i32 {
|
fn brace_delta(line: &str) -> i32 {
|
||||||
|
|
@ -659,6 +902,29 @@ fn mask_quoted_strings(line: &str) -> String {
|
||||||
result
|
result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn mask_javascript_templates(line: &str, open: &mut bool) -> String {
|
||||||
|
let mut result = String::with_capacity(line.len());
|
||||||
|
let mut escaped = false;
|
||||||
|
for ch in line.chars() {
|
||||||
|
if *open {
|
||||||
|
if escaped {
|
||||||
|
escaped = false;
|
||||||
|
} else if ch == '\\' {
|
||||||
|
escaped = true;
|
||||||
|
} else if ch == '`' {
|
||||||
|
*open = false;
|
||||||
|
}
|
||||||
|
result.push(' ');
|
||||||
|
} else if ch == '`' {
|
||||||
|
*open = true;
|
||||||
|
result.push(' ');
|
||||||
|
} else {
|
||||||
|
result.push(ch);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result
|
||||||
|
}
|
||||||
|
|
||||||
fn rust_raw_string_start(line: &str) -> Option<(usize, usize, String)> {
|
fn rust_raw_string_start(line: &str) -> Option<(usize, usize, String)> {
|
||||||
let bytes = line.as_bytes();
|
let bytes = line.as_bytes();
|
||||||
for start in 0..bytes.len() {
|
for start in 0..bytes.len() {
|
||||||
|
|
@ -687,6 +953,30 @@ fn clean_target(value: &str) -> String {
|
||||||
.into()
|
.into()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn clean_import_target(value: &str, language: &str) -> String {
|
||||||
|
let target = clean_target(value);
|
||||||
|
if language == "rust" {
|
||||||
|
target.trim_end_matches("::").into()
|
||||||
|
} else {
|
||||||
|
target
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn clean_relationship_target(value: &str, language: &str) -> String {
|
||||||
|
let target = clean_target(value);
|
||||||
|
if language == "cpp" {
|
||||||
|
target
|
||||||
|
.split_whitespace()
|
||||||
|
.filter(|part| !matches!(*part, "public" | "protected" | "private" | "virtual"))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(" ")
|
||||||
|
} else if matches!(language, "javascript" | "typescript") {
|
||||||
|
target.split('<').next().unwrap_or(&target).trim().into()
|
||||||
|
} else {
|
||||||
|
target
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn import_keys(name: &str, language: &str) -> Vec<String> {
|
fn import_keys(name: &str, language: &str) -> Vec<String> {
|
||||||
let mut keys = vec![name.to_owned()];
|
let mut keys = vec![name.to_owned()];
|
||||||
match language {
|
match language {
|
||||||
|
|
@ -747,15 +1037,39 @@ fn is_keyword(name: &str) -> bool {
|
||||||
| "assert"
|
| "assert"
|
||||||
| "assert_eq"
|
| "assert_eq"
|
||||||
| "println"
|
| "println"
|
||||||
|
| "import"
|
||||||
|
| "require"
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{ScanOptions, scan, scan_with_options};
|
use super::{ScanOptions, mask_block_comments, scan, scan_with_options, strip_line_comment};
|
||||||
use crate::model::{EdgeKind, NodeKind};
|
use crate::model::{EdgeKind, NodeKind};
|
||||||
use std::fs;
|
use std::fs;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn masks_nested_block_comments_without_truncating_quoted_urls() {
|
||||||
|
let mut depth = 0;
|
||||||
|
assert_eq!(
|
||||||
|
strip_line_comment(
|
||||||
|
r#"import value from "https://example.invalid/value.js"; // trailing"#,
|
||||||
|
"javascript"
|
||||||
|
),
|
||||||
|
r#"import value from "https://example.invalid/value.js"; "#
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
mask_block_comments("code /* outer", &mut depth),
|
||||||
|
"code "
|
||||||
|
);
|
||||||
|
assert_eq!(depth, 1);
|
||||||
|
let nested = "/* inner */ end */";
|
||||||
|
let masked = mask_block_comments(nested, &mut depth);
|
||||||
|
assert_eq!(masked.len(), nested.len());
|
||||||
|
assert!(masked.trim().is_empty());
|
||||||
|
assert_eq!(depth, 0);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn scans_functions_calls_and_cycles() {
|
fn scans_functions_calls_and_cycles() {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
|
@ -953,6 +1267,137 @@ mod tests {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn does_not_resolve_calls_across_language_boundaries() {
|
||||||
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
fs::write(dir.path().join("library.rs"), "fn shared() {}\n").unwrap();
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("application.js"),
|
||||||
|
"function local() {}\nfunction caller() { local(); shared(); }\n",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let building = scan(dir.path()).unwrap();
|
||||||
|
let caller = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.language.as_deref() == Some("javascript") && node.name == "caller")
|
||||||
|
.unwrap();
|
||||||
|
let javascript_local = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.language.as_deref() == Some("javascript") && node.name == "local")
|
||||||
|
.unwrap();
|
||||||
|
let rust_shared = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.language.as_deref() == Some("rust") && node.name == "shared")
|
||||||
|
.unwrap();
|
||||||
|
let caller_edges: Vec<_> = building
|
||||||
|
.edges
|
||||||
|
.iter()
|
||||||
|
.filter(|edge| edge.kind == EdgeKind::Calls && edge.source == caller.id)
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
caller_edges
|
||||||
|
.iter()
|
||||||
|
.any(|edge| edge.target == javascript_local.id)
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!caller_edges
|
||||||
|
.iter()
|
||||||
|
.any(|edge| edge.target == rust_shared.id)
|
||||||
|
);
|
||||||
|
assert!(caller_edges.iter().any(|edge| {
|
||||||
|
building.nodes.iter().any(|node| {
|
||||||
|
node.id == edge.target && node.kind == NodeKind::External && node.name == "shared"
|
||||||
|
})
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn parents_recognized_methods_to_their_enclosing_type() {
|
||||||
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("worker.dart"),
|
||||||
|
"class Worker {\n void execute() {}\n}\n",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let building = scan(dir.path()).unwrap();
|
||||||
|
let worker = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Class && node.name == "Worker")
|
||||||
|
.unwrap();
|
||||||
|
let execute = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Function && node.name == "execute")
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(execute.parent.as_deref(), Some(worker.id.as_str()));
|
||||||
|
assert!(building.edges.iter().any(|edge| {
|
||||||
|
edge.kind == EdgeKind::Defines && edge.source == worker.id && edge.target == execute.id
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn does_not_promote_python_type_declarations_to_calls() {
|
||||||
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("worker.py"),
|
||||||
|
"class Base:\n pass\nclass Worker(Base):\n pass\n",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let building = scan(dir.path()).unwrap();
|
||||||
|
assert!(building.edges.iter().all(|edge| {
|
||||||
|
edge.kind != EdgeKind::Calls
|
||||||
|
|| !building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.any(|node| node.id == edge.target && node.name == "Worker")
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn retains_every_location_for_a_deduplicated_dependency_edge() {
|
||||||
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
fs::write(
|
||||||
|
dir.path().join("calls.rs"),
|
||||||
|
"fn helper() {}\nfn caller() {\n helper();\n helper();\n}\n",
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let building = scan(dir.path()).unwrap();
|
||||||
|
let caller = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Function && node.name == "caller")
|
||||||
|
.unwrap();
|
||||||
|
let helper = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Function && node.name == "helper")
|
||||||
|
.unwrap();
|
||||||
|
let call = building
|
||||||
|
.edges
|
||||||
|
.iter()
|
||||||
|
.find(|edge| {
|
||||||
|
edge.kind == EdgeKind::Calls && edge.source == caller.id && edge.target == helper.id
|
||||||
|
})
|
||||||
|
.unwrap();
|
||||||
|
let serialized = serde_json::to_value(&call.evidence).unwrap();
|
||||||
|
|
||||||
|
assert_eq!(call.evidence.provenance, "calls.rs:3");
|
||||||
|
assert_eq!(
|
||||||
|
serialized["locations"],
|
||||||
|
serde_json::json!(["calls.rs:3", "calls.rs:4"])
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn scans_dart_structure_without_promoting_control_flow_or_strings() {
|
fn scans_dart_structure_without_promoting_control_flow_or_strings() {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,7 @@ xplan and Fold's optimization harness.
|
||||||
|
|
||||||
python3 tests/benchmarks/runner.py self-test
|
python3 tests/benchmarks/runner.py self-test
|
||||||
python3 tests/benchmarks/runner.py run lexical_smoke
|
python3 tests/benchmarks/runner.py run lexical_smoke
|
||||||
|
python3 tests/benchmarks/runner.py run language_parity
|
||||||
python3 tests/benchmarks/runner.py run yams_architecture --corpus /path/to/yams
|
python3 tests/benchmarks/runner.py run yams_architecture --corpus /path/to/yams
|
||||||
|
|
||||||
Each run writes a resolved plan, per-repeat graph/HTML artifacts, summary JSON,
|
Each run writes a resolved plan, per-repeat graph/HTML artifacts, summary JSON,
|
||||||
|
|
@ -13,6 +14,11 @@ across repeats. The seed corpus has explicit positive cycle and relationship
|
||||||
expectations; it is not representative enough for performance or language-wide
|
expectations; it is not representative enough for performance or language-wide
|
||||||
precision claims.
|
precision claims.
|
||||||
|
|
||||||
|
The `language_parity` plan repeats the labeled language corpus three times,
|
||||||
|
requires stable graph identities, and gates minimum file and symbol evidence for
|
||||||
|
each advertised language independently. Python and Swift counts characterize
|
||||||
|
their secondary tier; they do not promote those profiles.
|
||||||
|
|
||||||
The `yams_architecture` plan is a real-corpus integration benchmark. It records
|
The `yams_architecture` plan is a real-corpus integration benchmark. It records
|
||||||
three stable-identity scans and excludes YAMS's checked-in `site` output with
|
three stable-identity scans and excludes YAMS's checked-in `site` output with
|
||||||
Brick's explicit `--exclude` option. Validate each resulting graph with YAMS's
|
Brick's explicit `--exclude` option. Validate each resulting graph with YAMS's
|
||||||
|
|
|
||||||
31
tests/benchmarks/plans/language_parity.json
Normal file
31
tests/benchmarks/plans/language_parity.json
Normal file
|
|
@ -0,0 +1,31 @@
|
||||||
|
{
|
||||||
|
"schema_version": 1,
|
||||||
|
"name": "language_parity",
|
||||||
|
"corpus": "tests/fixtures/languages",
|
||||||
|
"repeats": 3,
|
||||||
|
"expect": {
|
||||||
|
"min_symbols": 35,
|
||||||
|
"min_dependency_edges": 35,
|
||||||
|
"max_cycles": 2,
|
||||||
|
"language_min_symbols": {
|
||||||
|
"dart": 7,
|
||||||
|
"rust": 8,
|
||||||
|
"c": 5,
|
||||||
|
"cpp": 8,
|
||||||
|
"javascript": 8,
|
||||||
|
"typescript": 9,
|
||||||
|
"python": 4,
|
||||||
|
"swift": 2
|
||||||
|
},
|
||||||
|
"language_min_files": {
|
||||||
|
"dart": 1,
|
||||||
|
"rust": 2,
|
||||||
|
"c": 1,
|
||||||
|
"cpp": 1,
|
||||||
|
"javascript": 5,
|
||||||
|
"typescript": 3,
|
||||||
|
"python": 2,
|
||||||
|
"swift": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -131,7 +131,9 @@ def command_for(
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def fixture_assertions(metrics: dict, expected: dict) -> dict[str, bool]:
|
def fixture_assertions(
|
||||||
|
metrics: dict, expected: dict, model: dict | None = None
|
||||||
|
) -> dict[str, bool]:
|
||||||
assertions = {
|
assertions = {
|
||||||
"symbols": metrics["symbol_count"] >= expected["min_symbols"],
|
"symbols": metrics["symbol_count"] >= expected["min_symbols"],
|
||||||
"dependencies": metrics["dependency_edge_count"]
|
"dependencies": metrics["dependency_edge_count"]
|
||||||
|
|
@ -141,6 +143,24 @@ def fixture_assertions(metrics: dict, expected: dict) -> dict[str, bool]:
|
||||||
assertions["cycles"] = metrics["cycle_count"] == expected["cycles"]
|
assertions["cycles"] = metrics["cycle_count"] == expected["cycles"]
|
||||||
if "max_cycles" in expected:
|
if "max_cycles" in expected:
|
||||||
assertions["max_cycles"] = metrics["cycle_count"] <= expected["max_cycles"]
|
assertions["max_cycles"] = metrics["cycle_count"] <= expected["max_cycles"]
|
||||||
|
nodes = [] if model is None else model.get("nodes", [])
|
||||||
|
for language, minimum in expected.get("language_min_symbols", {}).items():
|
||||||
|
assertions[f"{language}_symbols"] = (
|
||||||
|
sum(
|
||||||
|
node.get("language") == language
|
||||||
|
and node.get("kind") in {"class", "function"}
|
||||||
|
for node in nodes
|
||||||
|
)
|
||||||
|
>= minimum
|
||||||
|
)
|
||||||
|
for language, minimum in expected.get("language_min_files", {}).items():
|
||||||
|
assertions[f"{language}_files"] = (
|
||||||
|
sum(
|
||||||
|
node.get("language") == language and node.get("kind") == "file"
|
||||||
|
for node in nodes
|
||||||
|
)
|
||||||
|
>= minimum
|
||||||
|
)
|
||||||
return assertions
|
return assertions
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -215,7 +235,7 @@ def run_plan(
|
||||||
|
|
||||||
metrics = samples[0]["metrics"]
|
metrics = samples[0]["metrics"]
|
||||||
expected = plan["expect"]
|
expected = plan["expect"]
|
||||||
assertions = fixture_assertions(metrics, expected)
|
assertions = fixture_assertions(metrics, expected, model)
|
||||||
if not all(assertions.values()):
|
if not all(assertions.values()):
|
||||||
raise AssertionError(f"fixture expectations failed: {assertions}; metrics={metrics}")
|
raise AssertionError(f"fixture expectations failed: {assertions}; metrics={metrics}")
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -77,6 +77,32 @@ class RunnerTests(unittest.TestCase):
|
||||||
{"symbols": True, "dependencies": True, "max_cycles": True},
|
{"symbols": True, "dependencies": True, "max_cycles": True},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_language_expectations_cannot_be_satisfied_by_another_language(self) -> None:
|
||||||
|
metrics = {"symbol_count": 3, "dependency_edge_count": 2, "cycle_count": 0}
|
||||||
|
expected = {
|
||||||
|
"min_symbols": 2,
|
||||||
|
"min_dependency_edges": 1,
|
||||||
|
"cycles": 0,
|
||||||
|
"language_min_symbols": {"dart": 2, "rust": 2},
|
||||||
|
"language_min_files": {"dart": 1, "rust": 1},
|
||||||
|
}
|
||||||
|
model = {
|
||||||
|
"nodes": [
|
||||||
|
{"kind": "file", "language": "dart"},
|
||||||
|
{"kind": "class", "language": "dart"},
|
||||||
|
{"kind": "function", "language": "dart"},
|
||||||
|
{"kind": "file", "language": "rust"},
|
||||||
|
{"kind": "function", "language": "rust"},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
assertions = runner.fixture_assertions(metrics, expected, model)
|
||||||
|
|
||||||
|
self.assertTrue(assertions["dart_symbols"])
|
||||||
|
self.assertTrue(assertions["dart_files"])
|
||||||
|
self.assertFalse(assertions["rust_symbols"])
|
||||||
|
self.assertTrue(assertions["rust_files"])
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
27
tests/fixtures/languages/c/basic.c
vendored
Normal file
27
tests/fixtures/languages/c/basic.c
vendored
Normal file
|
|
@ -0,0 +1,27 @@
|
||||||
|
#include "support.h"
|
||||||
|
|
||||||
|
struct Worker {
|
||||||
|
int state;
|
||||||
|
};
|
||||||
|
|
||||||
|
union Payload {
|
||||||
|
int integer;
|
||||||
|
double decimal;
|
||||||
|
};
|
||||||
|
|
||||||
|
enum State {
|
||||||
|
ready
|
||||||
|
};
|
||||||
|
|
||||||
|
int start() {
|
||||||
|
return helper();
|
||||||
|
}
|
||||||
|
|
||||||
|
int helper() {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
struct CommentGhost {};
|
||||||
|
int comment_call() { return phantom_call(); }
|
||||||
|
*/
|
||||||
26
tests/fixtures/languages/cpp/basic.cpp
vendored
Normal file
26
tests/fixtures/languages/cpp/basic.cpp
vendored
Normal file
|
|
@ -0,0 +1,26 @@
|
||||||
|
#include "support.hpp"
|
||||||
|
|
||||||
|
class Base {};
|
||||||
|
class Runnable {};
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
class Worker final : public Base, private Runnable {
|
||||||
|
public:
|
||||||
|
Worker() {}
|
||||||
|
~Worker() {}
|
||||||
|
|
||||||
|
void start() override {
|
||||||
|
helper();
|
||||||
|
}
|
||||||
|
|
||||||
|
Worker& operator=(const Worker&) = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
int helper() {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
class CommentGhost {};
|
||||||
|
int comment_call() { return phantom_call(); }
|
||||||
|
*/
|
||||||
24
tests/fixtures/languages/dart/basic.dart
vendored
Normal file
24
tests/fixtures/languages/dart/basic.dart
vendored
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
import 'package:example/base.dart';
|
||||||
|
export 'package:example/public.dart';
|
||||||
|
part 'generated.dart';
|
||||||
|
|
||||||
|
mixin Logging {}
|
||||||
|
|
||||||
|
abstract class Controller extends Base with Logging implements Runnable {
|
||||||
|
Controller.named();
|
||||||
|
|
||||||
|
String get label => 'controller';
|
||||||
|
|
||||||
|
set label(String value) {}
|
||||||
|
|
||||||
|
void start() {
|
||||||
|
helper();
|
||||||
|
}
|
||||||
|
|
||||||
|
void helper() {}
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
class CommentGhost {}
|
||||||
|
void commentCall() { phantomCall(); }
|
||||||
|
*/
|
||||||
1
tests/fixtures/languages/javascript/api.js
vendored
Normal file
1
tests/fixtures/languages/javascript/api.js
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
export function api() {}
|
||||||
1
tests/fixtures/languages/javascript/base.js
vendored
Normal file
1
tests/fixtures/languages/javascript/base.js
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
export class Base {}
|
||||||
27
tests/fixtures/languages/javascript/basic.js
vendored
Normal file
27
tests/fixtures/languages/javascript/basic.js
vendored
Normal file
|
|
@ -0,0 +1,27 @@
|
||||||
|
import { Base } from './base.js';
|
||||||
|
export { api } from './api.js';
|
||||||
|
|
||||||
|
const legacy = require('./legacy.cjs');
|
||||||
|
const lazy = import('./lazy.js');
|
||||||
|
|
||||||
|
export default class Worker extends Base {
|
||||||
|
constructor() {}
|
||||||
|
|
||||||
|
start() {
|
||||||
|
helper();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const arrow = () => helper();
|
||||||
|
|
||||||
|
function helper() {}
|
||||||
|
|
||||||
|
const ignored = `
|
||||||
|
class Phantom extends Ghost {}
|
||||||
|
function fake() { phantomCall(); }
|
||||||
|
`;
|
||||||
|
|
||||||
|
/*
|
||||||
|
class CommentGhost {}
|
||||||
|
function commentCall() { phantomCall(); }
|
||||||
|
*/
|
||||||
1
tests/fixtures/languages/javascript/lazy.js
vendored
Normal file
1
tests/fixtures/languages/javascript/lazy.js
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
export function lazy() {}
|
||||||
1
tests/fixtures/languages/javascript/legacy.cjs
vendored
Normal file
1
tests/fixtures/languages/javascript/legacy.cjs
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
module.exports = {};
|
||||||
2
tests/fixtures/languages/python/base.py
vendored
Normal file
2
tests/fixtures/languages/python/base.py
vendored
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
class Base:
|
||||||
|
pass
|
||||||
10
tests/fixtures/languages/python/basic.py
vendored
Normal file
10
tests/fixtures/languages/python/basic.py
vendored
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
from base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class Worker(Base):
|
||||||
|
async def start(self):
|
||||||
|
helper()
|
||||||
|
|
||||||
|
|
||||||
|
def helper():
|
||||||
|
pass
|
||||||
28
tests/fixtures/languages/rust/basic.rs
vendored
Normal file
28
tests/fixtures/languages/rust/basic.rs
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
mod support;
|
||||||
|
|
||||||
|
use crate::support::{helper, Dependency};
|
||||||
|
|
||||||
|
pub struct Worker;
|
||||||
|
|
||||||
|
impl Worker {
|
||||||
|
pub fn start() {
|
||||||
|
helper();
|
||||||
|
generic::<usize>();
|
||||||
|
trace!();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn generic<T>() {}
|
||||||
|
|
||||||
|
pub const fn constant() -> usize {
|
||||||
|
1
|
||||||
|
}
|
||||||
|
|
||||||
|
pub unsafe fn unsafe_op() {}
|
||||||
|
|
||||||
|
pub extern "C" fn ffi_entry() {}
|
||||||
|
|
||||||
|
/*
|
||||||
|
struct CommentGhost;
|
||||||
|
fn comment_call() { phantom_call(); }
|
||||||
|
*/
|
||||||
3
tests/fixtures/languages/rust/support.rs
vendored
Normal file
3
tests/fixtures/languages/rust/support.rs
vendored
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
pub struct Dependency;
|
||||||
|
|
||||||
|
pub fn helper() {}
|
||||||
9
tests/fixtures/languages/swift/basic.swift
vendored
Normal file
9
tests/fixtures/languages/swift/basic.swift
vendored
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
import Foundation
|
||||||
|
|
||||||
|
protocol Runnable {}
|
||||||
|
|
||||||
|
class Worker: Base {
|
||||||
|
func start() {
|
||||||
|
helper()
|
||||||
|
}
|
||||||
|
}
|
||||||
1
tests/fixtures/languages/typescript/base.ts
vendored
Normal file
1
tests/fixtures/languages/typescript/base.ts
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
export class Base<T> {}
|
||||||
32
tests/fixtures/languages/typescript/basic.ts
vendored
Normal file
32
tests/fixtures/languages/typescript/basic.ts
vendored
Normal file
|
|
@ -0,0 +1,32 @@
|
||||||
|
import type { Base } from './base.js';
|
||||||
|
export type { Contract } from './contract.js';
|
||||||
|
|
||||||
|
interface Runnable {
|
||||||
|
run<T>(value: T): void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export abstract class Worker<T> extends Base<T> implements Runnable {
|
||||||
|
constructor() {}
|
||||||
|
|
||||||
|
run<U>(value: U): void {
|
||||||
|
helper();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const arrow = <T>(value: T): T => identity(value);
|
||||||
|
|
||||||
|
function identity<T>(value: T): T {
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function helper(): void {}
|
||||||
|
|
||||||
|
const ignored = `
|
||||||
|
interface Phantom {}
|
||||||
|
function fake(): void { phantomCall(); }
|
||||||
|
`;
|
||||||
|
|
||||||
|
/*
|
||||||
|
interface CommentGhost {}
|
||||||
|
function commentCall(): void { phantomCall(); }
|
||||||
|
*/
|
||||||
1
tests/fixtures/languages/typescript/contract.ts
vendored
Normal file
1
tests/fixtures/languages/typescript/contract.ts
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
export interface Contract {}
|
||||||
291
tests/language_parity.rs
Normal file
291
tests/language_parity.rs
Normal file
|
|
@ -0,0 +1,291 @@
|
||||||
|
use brick::model::{EdgeKind, NodeKind};
|
||||||
|
use brick::scan::scan;
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
struct FixtureCase {
|
||||||
|
directory: &'static str,
|
||||||
|
language: &'static str,
|
||||||
|
classes: &'static [&'static str],
|
||||||
|
functions: &'static [&'static str],
|
||||||
|
imports: usize,
|
||||||
|
inherits: usize,
|
||||||
|
calls: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn first_class_language_fixtures_have_stable_baseline_evidence() {
|
||||||
|
let cases = [
|
||||||
|
FixtureCase {
|
||||||
|
directory: "dart",
|
||||||
|
language: "dart",
|
||||||
|
classes: &["Controller", "Logging"],
|
||||||
|
functions: &["Controller.named", "helper", "label", "label", "start"],
|
||||||
|
imports: 3,
|
||||||
|
inherits: 3,
|
||||||
|
calls: 1,
|
||||||
|
},
|
||||||
|
FixtureCase {
|
||||||
|
directory: "rust",
|
||||||
|
language: "rust",
|
||||||
|
classes: &["Dependency", "Worker"],
|
||||||
|
functions: &[
|
||||||
|
"constant",
|
||||||
|
"ffi_entry",
|
||||||
|
"generic",
|
||||||
|
"helper",
|
||||||
|
"start",
|
||||||
|
"unsafe_op",
|
||||||
|
],
|
||||||
|
imports: 1,
|
||||||
|
inherits: 0,
|
||||||
|
calls: 3,
|
||||||
|
},
|
||||||
|
FixtureCase {
|
||||||
|
directory: "c",
|
||||||
|
language: "c",
|
||||||
|
classes: &["Payload", "State", "Worker"],
|
||||||
|
functions: &["helper", "start"],
|
||||||
|
imports: 1,
|
||||||
|
inherits: 0,
|
||||||
|
calls: 1,
|
||||||
|
},
|
||||||
|
FixtureCase {
|
||||||
|
directory: "cpp",
|
||||||
|
language: "cpp",
|
||||||
|
classes: &["Base", "Runnable", "Worker"],
|
||||||
|
functions: &["Worker", "helper", "operator=", "start", "~Worker"],
|
||||||
|
imports: 1,
|
||||||
|
inherits: 2,
|
||||||
|
calls: 1,
|
||||||
|
},
|
||||||
|
FixtureCase {
|
||||||
|
directory: "javascript",
|
||||||
|
language: "javascript",
|
||||||
|
classes: &["Base", "Worker"],
|
||||||
|
functions: &["api", "arrow", "constructor", "helper", "lazy", "start"],
|
||||||
|
imports: 4,
|
||||||
|
inherits: 1,
|
||||||
|
calls: 2,
|
||||||
|
},
|
||||||
|
FixtureCase {
|
||||||
|
directory: "typescript",
|
||||||
|
language: "typescript",
|
||||||
|
classes: &["Base", "Contract", "Runnable", "Worker"],
|
||||||
|
functions: &["arrow", "constructor", "helper", "identity", "run"],
|
||||||
|
imports: 2,
|
||||||
|
inherits: 2,
|
||||||
|
calls: 2,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
for case in cases {
|
||||||
|
assert_fixture(case);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn rust_fixture_models_impl_ownership_and_local_imports() {
|
||||||
|
let root = fixture_root("rust");
|
||||||
|
let building = scan(&root).unwrap();
|
||||||
|
let worker = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Class && node.name == "Worker")
|
||||||
|
.unwrap();
|
||||||
|
let start = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Function && node.name == "start")
|
||||||
|
.unwrap();
|
||||||
|
let support = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::File && node.path.as_deref() == Some("support.rs"))
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert_eq!(start.parent.as_deref(), Some(worker.id.as_str()));
|
||||||
|
assert!(
|
||||||
|
building
|
||||||
|
.edges
|
||||||
|
.iter()
|
||||||
|
.any(|edge| { edge.kind == EdgeKind::Imports && edge.target == support.id })
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn cpp_fixture_models_member_ownership_without_declaration_calls() {
|
||||||
|
let root = fixture_root("cpp");
|
||||||
|
let building = scan(&root).unwrap();
|
||||||
|
let worker = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Class && node.name == "Worker")
|
||||||
|
.unwrap();
|
||||||
|
let members: Vec<_> = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| node.kind == NodeKind::Function && node.parent.as_ref() == Some(&worker.id))
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
assert_eq!(members.len(), 4);
|
||||||
|
assert!(building.edges.iter().all(|edge| {
|
||||||
|
edge.kind != EdgeKind::Calls
|
||||||
|
|| !building.nodes.iter().any(|node| {
|
||||||
|
node.id == edge.target
|
||||||
|
&& matches!(
|
||||||
|
node.name.as_str(),
|
||||||
|
"Worker" | "~Worker" | "operator=" | "start"
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn javascript_and_typescript_model_members_without_template_phantoms() {
|
||||||
|
for directory in ["javascript", "typescript"] {
|
||||||
|
let building = scan(&fixture_root(directory)).unwrap();
|
||||||
|
let worker = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.find(|node| node.kind == NodeKind::Class && node.name == "Worker")
|
||||||
|
.unwrap();
|
||||||
|
let member_names: Vec<_> = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| {
|
||||||
|
node.kind == NodeKind::Function && node.parent.as_ref() == Some(&worker.id)
|
||||||
|
})
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
assert_eq!(member_names.len(), 2, "{directory} member ownership");
|
||||||
|
assert!(member_names.contains(&"constructor"));
|
||||||
|
assert!(
|
||||||
|
!building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.any(|node| { matches!(node.name.as_str(), "Phantom" | "fake" | "phantomCall") })
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
building
|
||||||
|
.edges
|
||||||
|
.iter()
|
||||||
|
.filter(|edge| edge.kind == EdgeKind::Imports)
|
||||||
|
.all(|edge| building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.any(|node| { node.id == edge.target && node.kind == NodeKind::File }))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn secondary_languages_have_explicit_non_parity_characterization() {
|
||||||
|
let python = scan(&fixture_root("python")).unwrap();
|
||||||
|
let python_classes: Vec<_> = python
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| node.kind == NodeKind::Class)
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
let python_functions: Vec<_> = python
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| node.kind == NodeKind::Function)
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
assert_eq!(python_classes.len(), 2);
|
||||||
|
assert!(python_classes.contains(&"Base"));
|
||||||
|
assert!(python_classes.contains(&"Worker"));
|
||||||
|
assert_eq!(python_functions.len(), 2);
|
||||||
|
assert!(python_functions.contains(&"helper"));
|
||||||
|
assert!(python_functions.contains(&"start"));
|
||||||
|
assert_eq!(edge_count(&python, EdgeKind::Imports), 1);
|
||||||
|
assert_eq!(edge_count(&python, EdgeKind::Inherits), 1);
|
||||||
|
assert_eq!(edge_count(&python, EdgeKind::Calls), 1);
|
||||||
|
|
||||||
|
let swift = scan(&fixture_root("swift")).unwrap();
|
||||||
|
let swift_symbols: Vec<_> = swift
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| matches!(node.kind, NodeKind::Class | NodeKind::Function))
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
assert_eq!(swift_symbols.len(), 2);
|
||||||
|
assert!(swift_symbols.contains(&"Worker"));
|
||||||
|
assert!(swift_symbols.contains(&"start"));
|
||||||
|
assert_eq!(edge_count(&swift, EdgeKind::Imports), 0);
|
||||||
|
assert_eq!(edge_count(&swift, EdgeKind::Inherits), 1);
|
||||||
|
assert_eq!(edge_count(&swift, EdgeKind::Calls), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn assert_fixture(case: FixtureCase) {
|
||||||
|
let root = fixture_root(case.directory);
|
||||||
|
let building = scan(&root).unwrap();
|
||||||
|
let mut classes: Vec<_> = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| {
|
||||||
|
node.language.as_deref() == Some(case.language) && node.kind == NodeKind::Class
|
||||||
|
})
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
let mut functions: Vec<_> = building
|
||||||
|
.nodes
|
||||||
|
.iter()
|
||||||
|
.filter(|node| {
|
||||||
|
node.language.as_deref() == Some(case.language) && node.kind == NodeKind::Function
|
||||||
|
})
|
||||||
|
.map(|node| node.name.as_str())
|
||||||
|
.collect();
|
||||||
|
classes.sort_unstable();
|
||||||
|
functions.sort_unstable();
|
||||||
|
|
||||||
|
assert_eq!(classes, case.classes, "{} class baseline", case.directory);
|
||||||
|
assert_eq!(
|
||||||
|
functions, case.functions,
|
||||||
|
"{} function baseline",
|
||||||
|
case.directory
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
edge_count(&building, EdgeKind::Imports),
|
||||||
|
case.imports,
|
||||||
|
"{} import baseline",
|
||||||
|
case.directory
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
edge_count(&building, EdgeKind::Inherits),
|
||||||
|
case.inherits,
|
||||||
|
"{} inheritance baseline",
|
||||||
|
case.directory
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
edge_count(&building, EdgeKind::Calls),
|
||||||
|
case.calls,
|
||||||
|
"{} call baseline",
|
||||||
|
case.directory
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!building.nodes.iter().any(|node| matches!(
|
||||||
|
node.name.as_str(),
|
||||||
|
"CommentGhost" | "commentCall" | "comment_call" | "phantom_call"
|
||||||
|
)),
|
||||||
|
"{} block-comment clean negative",
|
||||||
|
case.directory
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn fixture_root(directory: &str) -> PathBuf {
|
||||||
|
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||||
|
.join("tests/fixtures/languages")
|
||||||
|
.join(directory)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn edge_count(building: &brick::Building, kind: EdgeKind) -> usize {
|
||||||
|
building
|
||||||
|
.edges
|
||||||
|
.iter()
|
||||||
|
.filter(|edge| edge.kind == kind)
|
||||||
|
.count()
|
||||||
|
}
|
||||||
Loading…
Add table
Add a link
Reference in a new issue