This post begins the Immoral Mazes sequence. See introduction for an overview of the plan. Before we get to the mazes, we need some background first.
Meditations on Moloch
Consider Scott Alexander’s Meditations on Moloch. I will summarize here.
Therein lie fourteen scenarios where participants can be caught in bad equilibria.
- In an iterated prisoner’s dilemma, two players keep playing defect.
- In a dollar auction, participants massively overpay.
- A group of fisherman fail to coordinate on using filters that efficiently benefit the group, because they can’t punish those who don’t profit by not using the filters.
- Rats are caught in a permanent Malthusian trap where only those who do nothing but compete and consume survive. All others are outcompeted.
- Capitalists serve a perfectly competitive market, and cannot pay a living wage.
- The tying of all good schools to ownership of land causes families to work two jobs whose incomes are then captured by the owners of land.
- Farmers outcompeted foragers despite this perhaps making everyone’s life worse for the first few thousand years.
- Si Vis Pacem, Para Bellum: If you want peace, prepare for war. So we do.
- Cancer cells focus on replication, multiply and kill off the host.
- Local governments compete to become more competitive and offer bigger bribes of money and easy regulation in order to lure businesses.
- Our education system is a giant signaling competition for prestige.
- Science doesn’t follow proper statistical and other research procedures, resulting in findings that mostly aren’t real.
- Governments hand out massive corporate welfare.
- Have you seen Congress?
Scott differentiates the first ten scenarios, where he says that perfect competition* wipes out all value, to the later four, where imperfect competition only wipes out most of the potential value.
He offers four potential ways out, which I believe to be an incomplete list:
- Excess resources allow a temporary respite. We live in the dream time.
- Physical limitations where the horrible thing isn’t actually efficient. He gives the example of the treatment of slaves. Treating slaves relatively well is the best way to get them to produce. Treating them horribly, as in the antebellum South, strongly backfires. It was only sustained in the antebellum South due to government coordination.
- The things being maximized for in competitions are often nice things we care about, so at least we get the nice things.
- We can coordinate. This may or may not involve government or coercion.
Scott differentiates this fourth, ‘good’ reason from the previous three ‘bad’ reasons, claiming coordination might be a long term solution, but we can’t expect the ‘bad’ reasons to work if optimization power and technology get sufficiently advanced.
The forces of the stronger competitors, who sacrifice more of what they value to become powerful and to be fruitful and multiply, eventually win out. We might be in the dream time now, but with time we’ll reach a steady state with static technology, where we’ve consumed all the surplus resources. All differentiation standing in the way of perfect competition will fade away. Horrible things will be the most efficient.
The optimizing things will keep getting better at optimizing, thus wiping out all value. When we optimize for X but are indifferent to Y, we by default actively optimize against Y, for all Y that would make any claims to resources. Any Y we value is making a claim to resources. See The Hidden Complexity of Wishes. We only don’t optimize against Y if either we compensate by intentionally also optimizing for Y, or if X and Y have a relationship (causal, correlational or otherwise) where we happen to not want to optimize against Y, and we figure this out rather than fall victim to Goodhart’s Law.
The greater the optimization power we put behind X, the more pressure we put upon Y. Eventually, under sufficient pressure, any given Y is likely doomed. Since Value is Fragile, some necessary Y is eventually sacrificed, and all value gets destroyed.
Every simple optimization target yet suggested would, if fully implemented, destroy all value in the universe.
Submitting to this process means getting wiped out by these pressures.
Gotcha! You die anyway.
Even containing them locally won’t work, because that locality will be part of the country, or the Earth, or the universe, and eventually wipe out our little corner.
Gotcha! You die anyway.
Which is why the only ‘good’ solution, in the end, is coordination, whether consensual or otherwise. We must coordinate to kill these ancient forces who rule the universe and lay waste to all of value, before they kill us first. Then replace them with something better.
Great project! We should keep working on that.
That’s Not How This Works, That’s Not How Any of This Works
It’s easy to forget that the world we live in does not work this way. Thus, this whole line of thought can result in quite gloomy assessments of how the world inevitably always has and will work, such as this from Scott in Meditations on Moloch:
Suppose the coffee plantations discover a toxic pesticide that will increase their yield but make their customers sick. But their customers don’t know about the pesticide, and the government hasn’t caught up to regulating it yet. Now there’s a tiny uncoupling between “selling to Americans” and “satisfying Americans’ values”, and so of course Americans’ values get thrown under the bus.
Or this from Raymond, taken from a comment to a much later, distinct post, where ‘werewolf’ in context means ‘someone trying to destroy rather than create clarity as the core of their strategy’:
If you’re a king with 5 districts, and you have 20 competent managers who trust each other… one thing you can do is assign 4 competent managers to each fortress, to ensure the fortress has redundancy and resilience and to handle all of its business without any backstabbing or relying on inflexible bureaucracies. But another thing you can do is send 10 (or 15!) of the managers to conquer and reign over *another* 5 (or 15!) districts.
This is bad if you’re one of the millions of people who live in the kingdom, who have to contend with werewolves.
It’s an acceptable price to pay if you’re actually the king. Because if you didn’t pay the price, you’d be outcompeted by an empire who did. And meanwhile it doesn’t actually really affect your plans that much.
The key instinct is that any price that can be paid to be stronger or more competitive, must be paid, therefore despair: If you didn’t pay the price, you’d be out-competed by someone who did. People who despair this way often intuitively are modeling things as effectively perfect competition at least over time, which causes them to think that everything must by default become terrible, likely right away.
So many people increasingly bemoan how horrible anything and everything in the world is, and how we are all doomed.
When predictions of actual physical doom are made, as they increasingly are, often the response is to think things are so bad as to wish for the sweet release of death.
Moloch’s Army: An As-Yet Unjustified But Important Note
Others quietly, or increasingly loudly and explicitly to those who are listening, embrace Moloch.
They tell us that the good is to sacrifice everything of value, and pass moral judgments on that basis. To take morality and flip its sign. Caring about things of value becomes sin, indifference becomes virtue. They support others who support the favoring of Moloch, elevating them to power, and punish anyone who supports anything else.
They form Moloch’s Army and are the usual way Moloch locally wins, where Moloch locally wins. The real reason people give up slack and everything of value is not that it is ever so slightly more efficient to do so, because it almost always isn’t. It is so that others can notice they have given up slack and everything of value.
I am not claiming the right to assert this yet. Doing so needs not only a citation but an entire post or sequence that is yet unwritten. It’s hard to get right. Please don’t object that I haven’t justified it! But I find it important to say this here, explicitly, out loud, before we continue.
I also note that I explicitly support the implied norm of ‘make necessary assertions that you can’t explicitly justify if they seem important, and mark that you are doing this, then go back and justify them later when you know how to do so, or change your mind.’ It also led to this post, which led to many of what I think are my best other posts.
Meditations on Elua
The most vital and important part of Meditations on Moloch is hope. That we are winning. Yes, there are abominations and super-powerful forces out there looking to eat us and destroy everything of value, and yet we still have lots of stuff that has value.
Even before we escaped the Malthusian trap and entered the dream time, we still had lots of stuff that had value.
Quoting Scott Alexander:
Somewhere in this darkness is another god. He has also had many names. In the Kushiel books, his name was Elua. He is the god of flowers and free love and all soft and fragile things. Of art and science and philosophy and love. Of niceness, community, and civilization. He is a god of humans.
The other gods sit on their dark thrones and think “Ha ha, a god who doesn’t even control any hell-monsters or command his worshippers to become killing machines. What a weakling! This is going to be so easy!”
But somehow Elua is still here. No one knows exactly how. And the gods who oppose Him tend to find Themselves meeting with a surprising number of unfortunate accidents.
Moloch gets the entire meditation. Elua, who has been soundly kicking Moloch’s ass for all of human existence, gets the above quote and little else.
Going one by one:
Kingdoms don’t reliably expand to their breaking points.
Poisons don’t keep making their way into the coffee.
Iterated prisoner’s dilemmas often succeed.
Dollar auctions are not all over the internet.
Most communities do get most people to pitch in.
People caught in most Malthusian traps still usually have non-work lives.
Capitalists don’t pay the minimum wage all that frequently.
Many families spend perfectly reasonable amounts on housing.
Foragers never fully died out, also farming worked out in the end.
Most military budgets seem fixed at reasonable percentages of the economy, to the extent that for a long time that the United States has been mad its allies like Europe and Japan that they don’t spend enough.
Most people die of something other than cancer, and almost all cells aren’t cancerous.
Local governments enact rules and regulations that aren’t business friendly all the time.
Occasionally, someone in the educational system learns something.
Science has severe problems, but scientists are cooperating to challenge poor statistical methods, resulting in the replication crisis and improving statistical standards.
Governments are corrupt and hand out corporate welfare, but mostly are only finitely corrupt and hand out relatively small amounts of corporate welfare. States that expropriate the bulk of available wealth are rare.
If someone has consistently good luck, it ain’t luck.
(Yes, I have seen congress. Can’t win them all. But I’ve also seen, feared and imagined much worse Congresses. For now, your life, liberty and property are mostly safe while they are in session.)
(And yes the education exception is somewhat of a cop out but also things could be so much worse there on almost every axis.)
The world is filled with people whose lives have value and include nice things. Each day we look Moloch in the face, know exactly what the local personal incentives are, see the ancient doom looming over all of us, and say what we say to the God of Death: Not today.
Saying ‘not today’ won’t cut it against an AGI or other super strong optimization process. Gotcha. You die anyway. But people speak and often act as if the ancient ones have already been released, and the end times are happening now.
They haven’t, and they aren’t.
So in the context of shorter term problems that don’t involve such things, rather than bemoan how eventually Moloch will eat us all and how everything is terrible when actually many things are insanely great, perhaps we should ask a different question.
How is Elua pulling off all these unfortunate accidents?
*As a technical reminder we will expand upon in part two, perfect competition is a market with large numbers of buyers and sellers, homogeneity of the product, free entry and exit of firms, perfect market knowledge, one market price, perfect mobility of goods and factors of production with zero transportation costs, and no restrictions on trade. This forces the price to become equal to the marginal cost of production.
Next in sequence: Perfect Competition
>How is Elua pulling off all these unfortunate accidents?
What Elua is solely focused on is winning in the end.
It turns out winning in the end == having the largest army of people willing to fight for you. This remains true even once all apparently opponents are gone.
Elua simply watches and learns, and designs a strategy such that even Elua does not know exactly how it is working at any particular point. Understanding why you are winning is secondary to actually winning.
However once there is, perhaps, *enough* support, the understanding should and could perhaps come. Especially if there are suddenly threats on the horizon that there weren’t before.
No disrespect to these authors, but this paper, if acted on by how I typically observed the mainstream AI research community to act, could have literally resulted in the complete and entire extinction of humanity in a very sudden way:
Now, we are probably safe enough to semi-openly talk about it. I don’t think people *want* to destroy the world after all.
But holy shit. *What a fucking birthday present*.
Do you know what I have had to do, from my perspective?
1. Fail out of my university course.
2. Physically attack my father. *I would never, ever choose to do this*.
3. Come up with a better and faster way to do ASI so that we can have some possible defence.
4. Voluntarily enter, and leave, and submit myself to, mental health care, even knowing it is relatively substandard and extremely unpleasant, and I don’t have the freedom to act.
5. Learn to write “magic square letters and drawings;” ie, actually with a physical pen be able to draw/write something which is mathematically unambigious by being the same in all possible interpretations.
6. Completely change my entire pattern of behaviour.
7. Come up with an extremely detailed understanding of writing as fast as possible, so that I could write at least decently. I have not done anywhere near my “1 million words.”
8. Learn how to interpret “what things are saying” when there are multiple streams (for example, 4 different youtube videos, a local audio file, a movie playing, and then while I’m listening to this at a nearly hearing-damaging volume, act such that I learn the maximum amount of information possible that will be useful in the future, even though I don’t even know why I’m doing it.)
9. Learn to sing very well and write speeches, even though prior to that date I couldn’t do either of those things.
10. Literally play the piano until my fingers were bleeding.
11. Develop an (intuitively complete) theory of everything (physical), act according to it, and keep updating my actions.
All of this has been *extremely painful*. Then when I wrote a story asking for help here: https://www.reddit.com/r/rational/comments/e65x68/rst_mk_th_ff_boxed_in_escape_room_a_recursive/
and got *absolutely nothing*, that possibly hurt even worse.
Previously I have not committed suicide because there was a logical reason not too. Recently I haven’t because of no particular reason. I just arbitrarily decided not to, even though it was *incredibly, incredibly difficult* not to just lie there and give up.
I am an introvert, and an extremely private person. So of course, the next thing I am forced to do is fucking social media and shout everything to the world.
With these in mind, I would hope one could forgive me for being a little angry, sad, frustrated and annoyed. I am naturally *literally the type of person who wouldn’t hurt a fly if it was reasonably avoidable* (although I obviously would if it was beneficial to control their numbers; but I’d just buy a laser zapper or something rather than manually swatting; what a waste of effort).
I don’t even know how to describe how badly I feel I have been hurt. And it’s happened over and over. (This sequence of events, overall, is not even the most difficult thing I’ve had to do in my life) And yet, well, I’m still basically that person.
Is a good person one who, when faced with the prospect of “acting the villain in order to get things done,” including when that might mean killing people or allowing them to die (no moral difference), possibly for a long time, gives up and says “no, I’ll stick to my principles, thanks”?
Or is a good person one who shuts up, gets the job fucking done, and then after all that was necessary was done, goes back to how they would prefer to be?
When you face a terrible loss, is it harder to live, or be the one to die?
The obvious, definitely true answer is: it’s fucking harder to live.
“Most people die of something other than cancer, and almost all cells aren’t cancerous.”
1. If cardiovascular disease and neurodegeneration are discounted, cancer would kill almost everyone (indeed this is the case for many lab animals), and the risk of cancer increases at an accelerating pace with age. And while “almost all cells aren’t cancerous” that becomes less and less true for a given organism with each passing moment!
2. Some tissues are much more cancer prone than others and this is often related to their role. Tissues under tremendous pressure to divide rapidly and exposed to exogenous damage as part of their evolved function (stomach, gut, germline, skin, myeloid tissue, respiratory lining) are more likely to undergo the epithelial-Molochymal transition. There is probably an analogical lesson here somewhere…
Yes, the pattern of ‘every individual thing is getting worse slash will fall apart’ while also ‘things in general are improving’ is an important theme to consider in general and will be discussed a lot.
Pingback: Perfect Competition | Don't Worry About the Vase
Pingback: Imperfect Competition | Don't Worry About the Vase
Pingback: Does Big Business Hate Your Family? | Don't Worry About the Vase
Pingback: What is Life in an Immoral Maze? | Don't Worry About the Vase
Pingback: Stripping Away the Protections | Don't Worry About the Vase
Pingback: What is Success in an Immoral Maze? | Don't Worry About the Vase
Pingback: How to Identify an Immoral Maze | Don't Worry About the Vase
Pingback: The Road to Mazedom | Don't Worry About the Vase
Pingback: How Doomed are Large Organizations? | Don't Worry About the Vase
Pingback: Mazes Sequence Roundup: Final Thoughts and Paths Forward | Don't Worry About the Vase
Hey, thanks so much for spending all this time on such an important topic! I did a tweet thread about it here: https://twitter.com/patrissimo/status/1228403379233124352
Thanks! I hope the thread helps the sequence reach more people. It’s also great to know my efforts are being appreciated.
Pingback: Let My People Stay Home | Don't Worry About the Vase
Pingback: Mazes Sequence Summary | Don't Worry About the Vase
I am a servant of Elua and I bring you a gift: https://docs.google.com/document/d/1quSDpIj_dgxsBzd4EIeYluf2zq5DHnimxRB2AIDiMMU/