Research

I'm building an evaluation environment to test whether security controls can contain autonomous coding agents across multi-step Linux tasks. The core question is whether actions that are individually allowed can become unsafe when an agent combines them over a longer trajectory.

I evaluated the released EMBER2024 LightGBM malware detector on 1.08 million Windows PE files. The model reached 98.0% accuracy and 0.0031 ECE overall. When I analyzed results by malware family, some families performed far worse; malicord had 0.602 ECE and an 85.2% false-negative rate. The main result was that strong aggregate metrics hid concentrated reliability failures.

Motivated by href="https://www.nist.gov/news-events/news/2026/04/nist-updates-nvd-operations-address-record-cve-growth" target="_blank" rel="noopener noreferrer" >NIST's struggle to keep up with record CVE growth, I built a temporal benchmark using 2022–2023 CVEs for training and 2024 CVEs for testing. From the CVE description alone, I predicted severity, CWE, CVSS score, and vector attributes. TF-IDF baselines scored higher than zero-shot 70B+ models on severity and CWE, while rare classes and attributes that required inference remained difficult.

I built Linux and Microsoft 365 checks to compare declared security configuration with account-level exceptions and runtime state. The checks looked beyond whether a policy existed and tested whether it was actually enforced. The main result was that configured and enforced were not always the same.

I analyzed 200 ICLR 2023 peer reviews to test whether common toxicity tools capture the kinds of harmful feedback that appear in academic peer review. I compared GPT-based toxicity scores with Perspective and VADER using Pearson and Spearman correlations. The GPT-based measures generally aligned more strongly with VADER's general negativity than with Perspective's web-toxicity score, suggesting that academic toxicity and online toxicity capture different kinds of language.