fold/bench/juliet/expansion/README.md

Juliet Expansion — Full CWE Coverage for fold Detectors

Expands fold's Juliet test suite integration beyond the 14 initially-covered CWEs to a comprehensive catalog of all 118 Juliet C/C++ 1.3 CWE categories.

Contents

File	Purpose
cwe_catalog.json	Full CWE→detector mapping for all 118 CWEs
_generate_gold.py	Generate gold-standard files for all fold-relevant CWEs
run_expanded.py	Run fold against all mapped Juliet categories
score_expanded.py	Per-detector precision/recall aggregator

Quick Start

# 1. Fetch and filter the Juliet corpus (one-time)
bench/corpora/juliet/fetch.sh
python3 bench/corpora/juliet/filter_cwes.py --catalog bench/juliet/expansion/cwe_catalog.json

# 2. Generate gold standards for expanded CWE set
python3 bench/juliet/expansion/_generate_gold.py

# 3. Run fold against all mapped CWEs
python3 bench/juliet/expansion/run_expanded.py --detectors null-deref,div-by-zero,int-overflow,int-truncation,sign-conversion,oob --processes 4

# 4. Score results
python3 bench/juliet/expansion/score_expanded.py --run-dir bench/results/<latest>

Detector Coverage Summary

The catalog maps all 118 Juliet CWEs to fold's detectors. The target six detectors cover these CWE categories:

Detector	Primary CWEs	Additional CWEs	Total testcases
null-deref	476	690, 252, 588, 562	~2,900
div-by-zero	369	—	~1,480
int-overflow	190, 191	680, 789	~13,800
int-truncation	197	681	~1,464
sign-conversion	194, 195, 196	467	~3,828
oob-access	121, 122, 124, 126, 127	123, 468, 785	~19,100

Beyond the six target detectors, fold has other detectors that map to additional CWEs (cataloged but not the focus of this expansion):

Detector	CWEs
uninit-local-read	457, 665
use-after-free	401, 415, 416, 562, 590, 672, 675, 761, 762
deprecated-api	242, 676
taint-track	78, 114, 134, 426

CWE Catalog Schema

cwe_catalog.json is the authoritative mapping. Each CWE entry:

{
  "15": {
    "title": "External Control of System or Configuration Setting",
    "fold_detector": null,
    "fold_confidence": "none",
    "notes": "No fold detector covers configuration injection",
    "testcase_count": 79,
    "already_filtered": false
  }
}

Fields:

• fold_detector — fold detector name (or null if unmapped)
• fold_confidence — "primary" (direct CWE match), "plausible" (may trigger detector), "none"
• notes — rationale for mapping (or why unmapped)
• testcase_count — number of .c/.cpp files in Juliet source
• already_filtered — true if currently in the filtered corpus