AI and Ethics · 4 Nov 2026
Cyberlens: taxonomy-mapped sparse autoencoders for cybersecurity interpretability of open-weight language models
Fahad Hafeezh-index 0
Citations by database
Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.
- OpenCitations0
- Crossref0
Citations per year
The current year is still counting.
- 2026 so far0
Abstract
Abstract Sparse autoencoders (SAEs) can expose interpretable features in language-model activations, but public SAE resources such as GemmaScope and Llama Scope are trained on generic web text rather than cybersecurity evidence. CyberLens is a purpose-trained SAE and heuristic taxonomy-mapping pipeline for cybersecurity interpretability of an open-weight language model. Because cyber uplift is a foreseeable deployment risk, reasonable care requires developers of general-purpose language models to provide proportionate internal observability of cyber-relevant behaviour. We train a width-4096 JumpReLU SAE on layer 8 of EleutherAI/pythia-160 m, a 12-layer, 162-million-parameter model with a 768-dimensional residual stream, using 14,925 documents (approximately 1.85 million tokens) of MITRE ATT&CK descriptions and procedure examples, NVD vulnerability reports, CISA KEV records, CAPEC, malware reports, phishing emails, and Dolly-15k benign contrasts, with grouped splitting to prevent source-document leakage. The cybersecurity SAE achieves FVU 0.0435 (95.7% explained variance) versus 0.186 for a matched generic-text SAE—an absolute explained- variance gain of 14.2 percentage points on the reported activation sample. Its 4,096 feature rows are mapped heuristically to ATT&CK techniques and CISSP domains using local all-MiniLM-L6-v2 similarity, with confidence scores from 0.313 to 0.928; the top 50 remain pending hand-verification after a labelling pilot failed. Refusal is 0.000 in both arms (a Pythia-160 M base model without instruction tuning or RLHF). Suppressing 20 selected features increases keyword-flagged cyber content from 0.067 to 0.317 overall and from 0.050 to 0.800 for exploit items; phishing and malware are null, and 15 of 20 exploit placeholders contain “Exploit,” inflating this lexical metric. English-only, phishing-heavy corpus; one completed layer-8 width-4,096 SAE whose observed $$L_0=286$$ greatly exceeds the target 20; heuristic, unverified labels; keyword-based outcomes; a placeholder-heavy 60-prompt intervention set; no evidence about frontier models or effective cyber-content suppression. SAE weights, feature dictionary, loader, and an interactive demonstration are released for reproducible small-model research; scaling to larger models remains future work.
What the data shows
- No database records a citation yet, as is usual for a paper this new.
- Fahad Hafeez has the highest h-index among the authors (0).
Graph
This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.
Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.
Start with
Cited by 0 papers
Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.
References (7)
What it cites, from open citation data, with Crossref's record of each.