Linkpost: "Neural Networks Are Already Clean" Followup Technical Paper
Source: LessWrong ·
Yesterday, Goodfire released the companion technical report to Nick Cammarata’s viral interpretability essay "Neural Networks Are Already Clean" , but it seems to have gotten buried in all the recent crazy AI news. The technical report is titled: "Neurons as an Interpretable Basis". Goodfire's Twitter announcement for it can be found here . Like Nick Cammarata's essay, it's full of inventive, oft-interactive data displays. Don't just feed this to an AI agent and assume they can grok the post by pulling a PDF. Quoting from the Abstract : (emphasis mine) We study the neurons of Qwen3.6-35B-A3B u
Frequently Asked Questions
What is Linkpost: "Neural Networks Are Already Clean" Followup Technical Paper?
Linkpost: "Neural Networks Are Already Clean" Followup Technical Paper is a article discovery curated from LessWrong. Yesterday, Goodfire released the companion technical report to Nick Cammarata’s viral interpretability essay "Neural Networks Are Already Clean" , but it seems to have gotten…
How can I access or experience this?
You can experience this content directly on its original host (LessWrong) by clicking the "Open Original" button.
Where does this content come from?
Curated from LessWrong by the OnSuggest team to surface the most interesting and serendipitous content on the web.
More like this
All Culture & ideas discoveries →FDT and the sacrificial volunteer's dilemma
Consider the " twin sacrificial volunteer's dilemma ": [1] 1000 identical agents have the same deterministic decision-making process and the same observations, and can't communicate with each other. They can decide to either do nothing, or pay $1 to volunteer. If at least one agent volunteers, everyone who didn't volunteer gets $1 million [2] . Volunteers never receive anything. All of this is common knowledge. Payoff matrix for clarity: You don't volunteer You volunteer Nobody else volunteers $0 -$1 Someone else volunteers $1M -$1 Because the agents all make decisions the same way, there are
Quine can predict cross-propensity spillover. (imperfectly but significantly better than chance)
A little over a month ago I put forward a challenge in Can You Predict Cross-Propensity Spillover. The results are in and we have a winner. Well actually we have a rules ambiguity, but I'll get into that later. My SPAR group ran a 2 × 24 × 24 [1] propensity SFT spillover experiment (i.e. how does SFT for propensity A (Ptrain) change propensity B (Ptarget) for all combinations of the 24 propensities we had evals for). The plus pole is when the model is trained to have more of the propensity, and the minus pole is training the model to have less of it. Before releasing the results we gave you lo
Chips for Humanity: Compute for AI safety research
TLDR: Compute will likely be a bottleneck in short-timelines, so I am helping launch Chips for Humanity . Please fill out our survey to inform us of your compute needs if you work in AI safety. Our Theory of Change If timelines are short (I personally believe they are short...but I also know one should not make sweeping statements in a pseudo corporate type of post), a lot of the most important safety research will happen around "crunchtime" (no better way to define this than: when things start getting Serious), and a lot of it will be done by AI. That research runs on compute, compute may be
Rage, Rage Against the Dying of the Light
The lowest point of my life was the summer of 2019. Waking up at 4AM every day to go to work as a janitor, and never getting more than five hours of sleep, dark bags permanently lined my eyes. My diet of cereal and frozen pizza grew my gut day after day, and peppered my face with acne. After enduring an injury and not being able to exercise, my body was atrophying and falling apart. Every day was pain. And that was just the physical side of things. Mentally, I was getting pushed closer to the edge by a mental disorder that wouldn't get diagnosed for another six years. On top of that, my anger
From FairBot to other cooperation conditions by substitution
The simplest interesting player in MIRI's provability bot prisoner's dilemma tournament was FairBot, which cooperates if it can prove its opponent cooperates. It cooperates with another FairBot despite the apparent infinite regress, by Löb's theorem ( some might say magic ). A related line of work was how to represent statements like "if I take this action, it will have that consequence" in provability logic. For example there's these old slides by Vladimir Slepnev that I came across on the MIRI website. A post by James Payor brings these together. Well, maybe he didn't think of it like that.
Experience machines and Coxon’s rhetorical failure
Experience machines In the experience machine thought experiment, you choose to either continue your life as you are currently living it, or spend the rest of it inside a machine that cultivates a world perfectly fitted to you. It was introduced to argue against hedonism, but we can imagine the machine pushing for any metric we care about—it doesn’t necessarily have to be pleasure. For example, we can consider a non-hedonistic experience machine which makes you feel a perfect balance of happy, challenged, loved, fulfilled, empowered, or anything else you care about. The philosophical tension i