Gabriel Konar-Steenberg

Dedicated to computational research on the world’s most important problems (he/him)

The above plot, Figure 4 from our paper and our blog post, shows the performance of four interpretability techniques applied to three families of LLMs trained to display a particular ‘quirk’ via seven different methods. The interpretability techniques’ success in auditing these quirks is highly dependent on how the models are trained, even though the rate at which each model expresses its quirk is equal within each family.

Quick links: paper presented at the ICML 2026 Mechanistic Interpretability Workshop, blog post on LessWrong, the same blog post on my blog, Twitter thread.

In January 2026, I took a sabbatical from my researcher position at the National Lab of the Rockies (previously the National Renewable Energy Lab) to undertake the LASR Labs AI safety fellowship in London. My team of four, mentored by Stefan Heimersheim (Apollo Research, FAR.AI, Google DeepMind), has been researching the impact of training methodology on the interpretability of LLM “model organisms of misalignment” (MOs) — models deliberately trained to exhibit unnatural or undesired behaviors for scientific purposes. MOs are often used to benchmark how well interpretability techniques work. However, recent work suggests that the way in which MOs are typically trained may make interpretability unrealistically easy. This could give us misplaced confidence in the readiness of these interpretability techniques for auditing of safety properties in LLMs “for real.”

We investigated this concern by training 54 MOs to exhibit three different “quirks” via seven different training methodologies. We used this MO suite to benchmark four different interpretability techniques. We found that:

  • MO interpretability depends strongly and unpredictably on several train-time choices, even after controlling for black-box behavior; and
  • Our novel “integrated” training technique, which incorporates MO training data directly into the original post-training phase, fairly often yields less interpretable MOs than typical fine-tuning methods do.

In response, we recommend that MO-based interpretability benchmarks include models trained in many different ways and that no one MO’s result be taken as individually meaningful. We also suggest incorporating MOs trained with the “integrated” technique into benchmarks where possible.

The original fellowship ended in April, but we received £190k/$260k grant from Coefficient Giving to conduct an extension phase through late summer/fall. We recently presented our first paper at the ICML 2026 Mechanistic Interpretability Workshop and wrote a blog post summarizing the results here. We’re now focusing on improving auditing setups, studying more realistic MO quirks, and testing interpretability techniques that didn’t even exist when we started the project.

Everyone on the team of four was involved in all stages of the project, but I took particular leadership roles in:

  • Translating Stefan’s high-level proposal into concrete experimental design, including a taxonomy of quirk types and a decomposition of the project into discrete phases with a mini literature review for each;
  • The design of methods to eliminate confounders, including equalizing black-box “quirk expression” across training methods (Figure 2a in the paper and blog post, Section 3.4 in the paper), ruling out extraneous sources of auditability (Figure 2b, Section 3.4), and finding logit lens and sparse autoencoder noise floors (Figure 4c-d, Section 4.1);
  • The design and implementation of one of our prototype quirks, including dataset creation and execution of DPO post-training and SFT fine-tuning;
  • The implementation and execution of the steering-with-hypothesis-generation interpretability technique (Figure 4b, Section 3.5), including substantial tuning for statistical validity and computational tractability; and
  • Results interpretation and communication.

I think my attention to scientific rigor and knack for systematizing information are some of my most important professional comparative advantages. I’m proud of how much I was able to exercise these while developing more LLM-specific skills over the course of this project. I’m excited to apply my stronger AI safety research skills to whatever comes next!

Updated: