Where are the adults in AI safety?
Source: LessWrong ·
The following is based on a true story (not mine). Let’s say you’ve been bombarded with enough AI doom that you decide to do something about it! You watched Artificial and don’t trust the frontier labs to steer us somewhere safe on their own, so you want to work on the problem from the outside. You’re a little fuzzy on the acronyms - AGI, RSI, TAI, SVU - but by God you’re going to figure it out. So, you Google “How to get into AI safety.” And it’s encouraging! There are articles about how AI safety needs founders. There are calls for technical people to switch careers. There are career transit
Frequently Asked Questions
What is Where are the adults in AI safety??
Where are the adults in AI safety? is a article discovery curated from LessWrong. The following is based on a true story (not mine). Let’s say you’ve been bombarded with enough AI doom that you decide to do something about it! You watched Artificial and don’t…
How can I access or experience this?
You can experience this content directly on its original host (LessWrong) by clicking the "Open Original" button.
Where does this content come from?
Curated from LessWrong by the OnSuggest team to surface the most interesting and serendipitous content on the web.
More like this
All Culture & ideas discoveries →Reading Speculations Concerning the First Ultraintelligent Machine
In 1964 [1], I. J. Good wrote Speculations Concerning the First Ultraintelligent Machine . I've only ever seen it cited for a single paragraph: Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind ... Thus the first ultraintelligent machine is the last invention that
Where ASI Could Stumble
Before our eyes, forecasts are contracting and RSI is becoming nearer with each day. That's why I decided to publish a wide list of the obstacles for AI progress. I've been collecting these ideas for a long time, trying to make the list as comprehensive as it can be. Some of these obstacles may turn out to be almost irrelevant. Others might slow things down by months or even longer. Or they might not. It's difficult to estimate their impact, so for now my goal was just to briefly list them. List will include more then just obvious things like market correction, hardware shortages, or diminishi
Alignment 'train/test splits' should be sacred
Tl;dr: Alignment techniques can be used for training alignment or for testing it, but they cannot be used for both! As we build out the set of alignment techniques available to us, we should enforce that some are judiciously reserved only for evaluating alignment. See also: Validating against a misalignment detector is very different to training against one . Here is a guiding principle regarding alignment-minded AI training: whatever you do, you must maximally avoid teaching the model to take misaligned actions which are invisible to the user. In fact, you should be shamed if you use training
Bayesian Games with Nukes
How we all learned to stop worrying and love the bomb... for a while Listen to this post on the Non-Zero-Sum James podcast Nearing the end of 1961 the Soviet Union tested a 50-megaton hydrogen bomb equivalent to 3800 Hiroshima bombs. At the same time an intelligence officer by the name of Juanita Moody and her team became aware of Soviet ships delivering large cargo, under cover of darkness in Cuba—what if such a bomb was being delivered to the doorstep of the US? A surprise attack of that magnitude, would devastate the US, and potentially destroy its ability to retaliate, jeopardising the off
Easier To Be Good
I do not consider myself a good person. I am lazy and frivolous and inconsiderate, I care too much about intelligence and status, I am a snob and a hypocrite, and I do not call my mother enough. I don't mean this to read as self-hatred, nor a show of false modesty. Among all the people I know there is only a scant handful of people I would consider genuinely good, and so it is unremarkable to me that I wouldn't find myself amongst their number. You do not need to be good to desire more goodness in the world. It is good for everyone when things are good, and when there is more of it — more thou
Giving Up on Saving the World
When I was 19, I decided I was going to save the world. Now, 25 years later, I’ve decided to give up. To explain why and what giving up means, we first need to go back to how I ever got into the world-saving business. As a kid, I loved computers, and I started teaching myself to program them when I was 8. By 16 I could write real programs that did interesting things, and it was also around this time I got into Linux. And since I was into Linux, that meant I read Slashdot . If you weren’t into Linux in the late 90s, you might not even know what Slashdot was. It still exists, but it’s a shell of