Smart Contract Security Assistant
Deterministic vulnerability screening and SQLite Evidence Graph with constrained LLM remediation.
01 · Problem Space & Threat Model
Generative AI tools are frequently deployed to analyze smart contracts, but LLMs suffer from high hallucination rates: inventing non-existent reentrancy vectors, hallucinating CVE references, or completely missing subtle AST-level state mutation flaws. Conversely, raw static analyzers like Slither output voluminous, noisy diagnostic logs that overwhelm developers.
SCSA was built to bridge this divide. It establishes a deterministic truth foundation: static analyzers own vulnerability facts, normalized findings are stored in an immutable SQLite Evidence Graph, and LLMs are constrained strictly to explaining and proposing diffs for verified evidence nodes.
02 · System Constraints & Non-Negotiables
LLMs never create or promote vulnerabilities. Findings must map to AST detector evidence from Slither.
Native compiler builds (Foundry/Hardhat) are disabled by default to prevent arbitrary code execution from untrusted imports.
Every reported finding links directly to file paths, AST node IDs, source line numbers, and Slither detector rules.
03 · Architecture & Evidence Grounding
Untrusted Solidity is parsed through a guarded ingestion layer into Slither AST analysis. Normalized findings populate the SQLite Evidence Graph, which forms the strict bounding context for local LLM explanations.
Deterministic Screening & Constrained Explanation
01. Ingestion: Safely stages untrusted Solidity source with path-traversal safeguards and disabled-by-default native builds.
02. Slither AST: Runs 27 mapped static detectors producing verifiable AST code spans and call graphs.
03. Normalizer: Maps diverse detector outputs into unified finding schemas with deterministic CWE/SWC identifiers.
04. Evidence Graph: Stores verified findings and relations in SQLite as immutable factual anchor nodes.
05. Constrained Explanation: LLM explains and writes diffs strictly bounded by Evidence Graph nodes; zero hallucinated CVEs.
04 · Three Key Architectural Decisions
1. Static Analyzer Evidence Owns Vulnerability Facts
EVIDENCE BOUNDARYContext: Prompting an LLM with raw Solidity code yields inconsistent results with high false positive rates and hallucinated vulnerabilities.
Decision: Only 27 curated Slither AST detectors can promote findings into the triage report. Unmapped tool output is preserved in SQLite as raw trace evidence. The generative model is restricted to interpreting confirmed findings.
Outcome: Achieved 100% recall on the 50-contract Slither50 v2 benchmark with completely verifiable AST references.
2. Native Compiler Builds Fail Closed by Default
SECURITY BOUNDARYContext: Compiling untrusted Foundry and Hardhat repositories can execute malicious Node.js build scripts or Makefile hooks inside the developer's environment.
Decision: The system defaults to single-file AST parsing via solc-select with disabled build scripts. Full project builds require an explicit --allow-native-builds opt-in flag with documented security warnings.
Outcome: Prevents sandbox escape and arbitrary code execution during automated CI/CD triage scans.
3. Grounded RAG with Deterministic Fallback
RELIABILITYContext: Users without Apple Silicon or local GPU hardware cannot run large LLM models locally.
Decision: Built a hybrid explanation architecture: if Apple MLX is available, it generates contextual walkthroughs grounded in the SQLite evidence graph. If offline or on CPU-only hosts, the system seamlessly falls back to pre-indexed static security patterns and CWE documentation.
Outcome: The CLI and workbench remain 100% functional without an active LLM runtime or external API keys.
05 · The Hardest Technical Challenge
Normalizing Heterogeneous Slither AST Findings into a Unified Graph
Root Cause: Different Slither detectors emit drastically different JSON schemas: some reference individual AST expressions, others reference variable declarations, while reentrancy detectors reference multi-function call traces across inheritance hierarchies.
Engineering Approach: Developed a Python AST Normalization Engine that maps all detector outputs into a normalized SecurityFinding schema with source code snippet extraction, line number ranges, and cryptographic hash deduplication. Stored findings and their parent contracts in a relational SQLite Evidence Graph.
Verification: Validated across 140 pytest test cases, including edge-case Solidity features like Diamond Proxies, assembly blocks, and custom error types.
06 · Disclosed Limitation & Public Benchmark
Unlike marketing claims of "100% automated security", SCSA discloses its precise empirical limitations. On the public Slither50 v2 benchmark, the system achieved zero false negatives (100% recall), but produced 4 false positives.
07 · Testing, Build & Verification Evidence
100% backend test pass rate covering Slither detector parsers, solc version resolution, SQLite queries, and CLI commands.
Frontend test suite validating React reviewer workbench components, finding filters, and diff visualization.
Packaging via Hatchling verified with clean PyPI-compliant source distributions and wheels.
Statistically significant separation between safe contracts (mean 8.2) and vulnerable contracts (mean 53.25) across benchmark runs.
08 · Retrospective & Learnings
Engineering Takeaway: Engineering Takeaway: In high-stakes domains like smart contract security, fluent generative text without verifiable grounding is a liability. Decoupling the "truth engine" (deterministic AST analyzers) from the "presentation engine" (LLMs) eliminated hallucinated vulnerability disclosures while preserving the educational benefit of AI-assisted explanations.
What I Would Do Differently: What I Would Do Differently: Expand beyond Slither by integrating symbolic execution engines (such as Manticore or Halmos) into the deterministic pipeline, allowing automated verification of false positives before human triage.