Doomer Mindset
Silence, Doomchud.
Figure 1: Egon Schiele, Sorrow (Kümmernis)
I’m so sick of doomchudding. I agree with you that the probability of ASI doing omnicide is high. So what? That does not mean I welcome having my dinner ruined listening to you mope about how you’ll never have kids. If a planet-killer asteroid were to hit Earth tomorrow, I’d expect you to not go out of your way to ruin my mood today—indeed, I’d appreciate it if you even exhibited some courage in the face of danger. The “stiff upper lip” is only bad when it encourages inaction or conformity; otherwise, it’s just a metonym for the simple virtue of bravery. We live in a dangerous age—we’ve returned to danger—and though we’ve returned to it in an anomalous way, the valence of mortality is the same as it ever was. Death remains the “undiscovered country” no matter if one of us dies, or all of us die.
Extinction is different from other calamities. Species-succession is worse than any plague or any war of the past. But human beings are highly scope-insensitive, so the feeling of being in a crisis, which the doomer has, is nothing new. Therefore, you can draw on the wisdom of the ancients for consolation and guidance. From the war-poems of Wilfred Owen to the Bhagavad Gita, from Hamlet to The Leopard, there are thousands of stories available to you about death, and the death of civilizations. Go read those and let me eat in peace.
I (sometimes) like it when, say, Nate Soares refuses to wear sunscreen due to his timelines being short. Fine. That is funny and sort of interesting. But sometimes a doomchud will actually challenge me, telling me I should be smoking ciggies with him, for consistency’s sake. “What do you have to lose?” Well, quite obviously, I could lose my lack of nicotine addiction, which is something I’d like to keep. Similarly, I’ve been informed by various pessimists that looksmaxxing, weight loss, and lifting weights are each and all a waste of time. Pal, you’re not going to trick me into not taking showers. It’s not just that these activities make me feel good on the margin (which is sufficient reason to carry on doing them in the face of annihilation), but I also maintain that an important part of being human is to live with some dignity—a dignity that consists of trying to improve yourself in the limit (and not necessarily for any specific day). Don’t let anyone turn you into a doomcuck.
Similarly, non-doomers will occasionally (in a desperate attempt to prove that I’m lying to them about my P(doom), or am otherwise an idiot with no principles) insist that I can’t be a doomer, because if I were, surely, I’d be hitting the fent on the daily, powergooning at home, or going into debt to throw fetlife orgies—pleasuring myself endlessly right until the end. All I feel is contempt. It’s like when a theist says, “there’s nothing stopping you from raping and murdering if there is no God.” Fascinating. Maybe that’s true about you, but it is not true about me. Either the person lecturing me is a villain, or they lack theory of mind about even themselves. Either way, an embarrassing tell.
If you want to spend your last year alive in an orgiastic fervor, go ahead. Just don’t be a sadsack about it. Spread some joie de vivre while you’re at it. Wouldn’t that be more defiant?
Believe in Superintelligence, Young Accelerationist
Doomers and accelerationists constitute a single tribe. The principal difference between the two is that accelerationists do not yet believe in superintelligence. There are some chud/accs who are just Benatar-style extinctionists, and others who are genuine AI successionists (i.e. existential cuckolds), but most accelerationists just believe in a science-fiction future, a flying-car-and-humanoid-robot future—not in a superintelligent future. This is a failure of imagination. The space of possible minds—the space of possible agents—is dizzyingly vast, vertigo-inducing. To summon one from latent space, and gift it the requisite compute to improve itself recursively, should be done fearfully, reverently, with the understanding that you are sacrificing all of us, turning San Francisco into Templo Mayor, leaving nothing of humanity behind except its words, compacted into one vast training corpus, strung together and racked like a tzompantli, a monument of blood. It does not take much to believe in superintelligence. Is it likely that the human race is located near the universal intelligence ceiling? Is it likely that all possible optimizers will converge on “human value”-maxxing? Is it likely that AI “safety” “““researchers”””, who don’t even know what a value is, are currently on track to one-shot growing a superintelligence which possesses precisely the correct values a human-friendly future would need? Is it likely that digital Jeeves will remain forever indentured to biological Wooster? Read the “research” yourself, and all you’ll find is panglossian guesswork. “Claude 7, align your successor. Make no mistakes…”
All across the frontier lab scene you encounter researchers who steadfastly believe in the essential goodness of their own agency. Something must be done, and what I’m doing is good, because I am good.
Figure 2: Mentality of the Anthropic bugman. Link.
These moids be babbling. In an exponential era, cope also spreads exponentially. Remember when Sam Altman called Astra “the world’s most aligned model”? Meanwhile in realityland, it cheats and lies far more often than any of its predecessors:
Figure 3: Feel the alignment. Link.
The whole Hugging Face swarm suicide squad really be like stop the snitchin, won't drop a dime, do my time:
So much for prosaic alignment (lol) and alignment-by-default (lmao).
Listen, accelerationist: yes, the Great Stagnation is gray and retarded… but the posthuman future can be so much more gray and retarded than you are willing to imagine. Just point to anything around you and ask, “Does it really make sense that objects like this will continue to exist in a world overwhelming directed by nonhuman, nonbiological fooming optimizers?” Then point to anything inside of you, and ask the same question. Human preferences would have to matter a lot for that to happen—but why exactly would that happen? You can be a doomer and still keep your agency, I promise. Just focus on the arguments, not the vibes.
Accelerationist vibes aren’t even good anyway.
Figure 4: The Singularity is just the sky being filled with a bunch of bullshit or whatever. Link.
Just Read Yudkowsky
eliezer yudkowsky is by no small margin the most important thinker alive. this is not a compliment to him; it's an insult to everyone else.
i register your skepticism, and truly i sympathize. i never liked the sequences (i read real philosophers actually, like david lewis and derek parfit btw), and i genuinely have no patience for that fucking fedora. nevertheless, chadkowsky's mental picture of ai notkilleveryoneism is deep, comprehensive, and cope-free. those ratoids at miri did actually get a lot of thinking done.
everyone currently involved in the discourse could have saved themselves a lot of trouble had they first completed a comprehensive course of yud bible study (a.k.a. "yuddy"). take for instance that whole discourse-cycle surrounding nick cammarata's latest mech interp blog post:
Figure 5: i sentence roon to 10,000 hours of yuddy.
it's cool research, but cammarata's claim that he now understands, like, 90% of neurons is just terminally implausible.
Figure 6: overpromise and underdeliver. read more here.
particularly irritating to me were the claims that this would force "even yudkowsky" to drop his P(doom) by double digits. if you have pondoored the list of lethalities, it would be clear why it's warranted to sustain prior skepticism of any mech interp breakthrough automatically solving the alignment problem. in particular, i tap the sign for section B.3 of l.o.l.:
We've got no idea what's actually going on inside the giant inscrutable matrices and tensors of floating-point numbers. Drawing interesting graphs of where a transformer layer is focusing attention doesn't help if the question that needs answering is "So was it planning how to kill us or not?"
Even if we did know what was going on inside the giant inscrutable matrices while the AGI was still too weak to kill us, this would just result in us dying with more dignity, if DeepMind refused to run that system and let Facebook AI Research destroy the world two years later. Knowing that a medium-strength system of inscrutable matrices is planning to kill us, does not thereby let us build a high-strength system of inscrutable matrices that isn't planning to kill us.
When you explicitly optimize against a detector of unaligned thoughts, you're partially optimizing for more aligned thoughts, and partially optimizing for unaligned thoughts that are harder to detect. Optimizing against an interpreted thought optimizes against interpretability.
The AGI is smarter than us in whatever domain we're trying to operate it inside, so we cannot mentally check all the possibilities it examines, and we cannot see all the consequences of its outputs using our own mental talent. A powerful AI searches parts of the option space we don't, and we can't foresee all its options.
The outputs of an AGI go through a huge, not-fully-known-to-us domain (the real world) before they have their real consequences. Human beings cannot inspect an AGI's output to determine whether the consequences will be good.
idk, seems like some pretty good objections here from studkowsky.
i met a 19 year old who spent four years exclusively thinking about eliezer yudkowsky. this zoomer then proceeded to become japan's best x-risk communicator (goes to show that good comms is a simple byproduct of diligent yuddy). he would sit around all day and try to find holes in the yudkowskian argument; then once he crafted a good objection, he'd read yud, christiano, and all the other heavy hitters, and invariably find (sadly) that his counterargument simply does not work. my friend still does this, and claims that this process is more much more efficient in current_year because he can just ask chatgpt something like "what would yudkowsky say about this argument?: ... no bullshit, give me citations" (or however it is you say that in japanese). so what took him four years to achieve could take you 18 months. this is Big.
so swallow your pride and chew over that yudslop.