OpenResearchAlpha

AI and Ethics · 4 Nov 2026

Cyberlens: taxonomy-mapped sparse autoencoders for cybersecurity interpretability of open-weight language models

Fahad Hafeezh-index 0

0
Citations
no database records one yet
0
Citations in 2026
so far
0
Leading author's h-index
Fahad Hafeez, in QAI's scholar index
7
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Abstract Sparse autoencoders (SAEs) can expose interpretable features in language-model activations, but public SAE resources such as GemmaScope and Llama Scope are trained on generic web text rather than cybersecurity evidence. CyberLens is a purpose-trained SAE and heuristic taxonomy-mapping pipeline for cybersecurity interpretability of an open-weight language model. Because cyber uplift is a foreseeable deployment risk, reasonable care requires developers of general-purpose language models to provide proportionate internal observability of cyber-relevant behaviour. We train a width-4096 JumpReLU SAE on layer 8 of EleutherAI/pythia-160 m, a 12-layer, 162-million-parameter model with a 768-dimensional residual stream, using 14,925 documents (approximately 1.85 million tokens) of MITRE ATT&CK descriptions and procedure examples, NVD vulnerability reports, CISA KEV records, CAPEC, malware reports, phishing emails, and Dolly-15k benign contrasts, with grouped splitting to prevent source-document leakage. The cybersecurity SAE achieves FVU 0.0435 (95.7% explained variance) versus 0.186 for a matched generic-text SAE—an absolute explained- variance gain of 14.2 percentage points on the reported activation sample. Its 4,096 feature rows are mapped heuristically to ATT&CK techniques and CISSP domains using local all-MiniLM-L6-v2 similarity, with confidence scores from 0.313 to 0.928; the top 50 remain pending hand-verification after a labelling pilot failed. Refusal is 0.000 in both arms (a Pythia-160 M base model without instruction tuning or RLHF). Suppressing 20 selected features increases keyword-flagged cyber content from 0.067 to 0.317 overall and from 0.050 to 0.800 for exploit items; phishing and malware are null, and 15 of 20 exploit placeholders contain “Exploit,” inflating this lexical metric. English-only, phishing-heavy corpus; one completed layer-8 width-4,096 SAE whose observed $$L_0=286$$ greatly exceeds the target 20; heuristic, unverified labels; keyword-based outcomes; a placeholder-heavy 60-prompt intervention set; no evidence about frontier models or effective cyber-content suppression. SAE weights, feature dictionary, loader, and an interactive demonstration are released for reproducible small-model research; scaling to larger models remains future work.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.
  • Fahad Hafeez has the highest h-index among the authors (0).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (1)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (7)

What it cites, from open citation data, with Crossref's record of each.