AI #16: AI in the UK


It would make strategic sense, even from a selfish perspective, for essentially any nation to get more involved in sensible AI regulation, and to get more involved in capturing AI benefits and being the home to AI and AI-enabled economic engines.

It makes especially good sense for the UK, which is facing numerous economic headwinds and yet still has the legacy of its history and of London and DeepMind, of Oxford and Cambridge, and has broken free of the absolutely bonkers regulatory regime that is the European Union. This is their chance, and Sunak is attempting to take it.

Will it end up being yet another capabilities push, or will we get real progress towards safety and alignment and sensible paths forward? The real work on that begins now.

In terms of capabilities, this was what passes for a quiet week.

The most exciting recent capability announcement was the Apple Vision Pro, which didn’t mention AI at all despite the obvious synergies. I review the reviews here.

Continue reading
Posted in Uncategorized | 9 Comments

On the Apple Vision Pro

Apple is offering a VR/AR/XR headset, Vision Pro, for the low, low price of $3,500.

I kid. Also I am deadly serious.

The value of this headset to a middle class American or someone richer than that is almost certainly either vastly more than $3,500, or at best very close to $0.

This type of technology is a threshold effect. Once it gets good enough, if it gets good enough, it will feel essential to our lives and our productivity. Until then, it’s a trifle.

Continue reading
Posted in Uncategorized | 6 Comments

The Dial of Progress

“There is a single light of science. To brighten it anywhere is to brighten it everywhere.” – Isaac Asimov

You cannot stand what I’ve become
You much prefer the gentleman I was before
I was so easy to defeat, I was so easy to control
I didn’t even know there was a war

– Leonard Cohen, There is a War

“Pick a side, we’re at war.”

– Steven Colbert, The Colbert Report

Recently, both Tyler Cowen in response to the letter establishing consensus on the presence of AI extinction risk, and Marc Andreessen on the topic of the wide range of AI dangers and upsides, have come out with posts whose arguments seem bizarrely poor.

These are both excellent, highly intelligent thinkers. Both clearly want good things to happen to humanity and the world. I am confident they both mean well. And yet.

So what is happening?

A Theory

My theory is they and similar others believe discourse in 2023 cannot handle nuance.

Instead, effectively there is a single Dial of Destiny Progress, based on the extent our civilization places restrictions, requires permissions and places strangleholds on human activity, from AI to energy to housing and beyond.

Continue reading
Posted in Uncategorized | 4 Comments

AI #15: The Principle of Charity

The sky is not blue. Not today, not in New York City. At least it’s now mostly white, yesterday it was orange. Even indoors, everyone is coughing and our heads don’t feel right. I can’t think fully straight. Life comes at you fast.

Thus, I’m going with what I have, and then mostly taking time off until this clears up. Hopefully that won’t be more than a few more days.

The Principle of Charity comes into play this week because of two posts, by people I greatly respect as thinkers and trust to want good things for the world, making arguments that are remarkably terrible. I wrote detailed responses to the arguments within, then realized that was completely missing the point, and deleted them. Instead, next week I plan to explain my model of what is going on there – I wish they’d stop doing what they are doing, and think they would be wise to stop doing it, but to the extent I am right about what is causing these outputs, I truly sympathize.

For a day we were all talking about a Vice story that sounded too good (or rather, too perfect) to be true, and then it turned out that it was indeed totally made up. Time to take stock of our epistemic procedures and do better next time.

Continue reading
Posted in Uncategorized | 17 Comments

Monthly Roundup #7: June 2023

This month’s roundup of non-AI things, with the good, the bad and the uncondoned, along with several additional individual topics. I’m also trying out running some blog-focused surveys near the end, right before the jokes.

Continue reading
Posted in Uncategorized | 14 Comments

AI #14: A Very Good Sentence

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

That is the entire text of the one-line open letter signed this week by what one could reasonably call ‘everyone,’ including the CEOs of all three leading AI labs.

Major news outlets including CNN and The New York Times noticed, and put the focus squarely on exactly the right thing: Extinction risk. AI poses an extinction risk.

This time, when the question was asked at the White House, no one laughed.

You love to see it. It gives one hope.

Some portion of we are, perhaps, finally ready to admit we have a problem.

Let’s get to work.

Continue reading
Posted in Uncategorized | 11 Comments

To Predict What Happens, Ask What Happens

When predicting conditional probability of catastrophe from loss of human control over AGI, there are many distinct cruxes. This essay does not attempt a complete case, or the most generally convincing case, or addressing the most common cruxes.

Instead these are my best guesses for potentially mind-changing, armor-piercing questions people could ask themselves if they broadly accept many concepts like power seeking being a key existential risk, that default development paths are likely catastrophic and that AI could defeat all of us combined, have read and thought hard about alignment difficulties, yet think the odds of catastrophe are not so high.

In addition to this entry, I attempt an incomplete extended list of cruxes here, an attempted taxonomy of paths through developing AGI and potentially losing control here, and an attempted taxonomy of styles of alignment here, while leaving to the future or others for now a taxonomy of alignment difficulties.

Apologies in advance if some questions seem insulting or you rightfully answer with ‘no, I am not making that mistake.’ I don’t know a way around that.

Here are the questions up front:

  1. What happens?
  2. To what extent will humanity seek to avoid catastrophe?
  3. How much will humans willingly give up, including control?
  4. You know people and companies and nations are dumb and make dumb mistakes constantly, and mostly take symbolic actions or gesture at things rather than act strategically, and you’ve taken that into account, right?
  5. What would count as a catastrophe?
  6. Are you consistently tracking what you mean by alignment?
  7. Would ‘human-strength’ alignment be sufficient?
  8. If we figure out how to robustly align our AGIs, will we choose to and be able to make and keep them that way? Would we keep control?
  9. How much hope is there that a misaligned AGI would choose to preserve humanity once it no longer needed us?
  10. Are you factoring in unknown difficulties and surprises large and small that always arise, and in which direction do they point? re you treating doom as only happening through specified detailed logical paths, which if they break down mean it’s going to be fine?
  11. Are you properly propagating your updates, and anticipating future updates?
  12. Are you counting on in-distribution heuristics to work out of distribution?
  13. Are you using instincts and heuristics rather than looking at mechanics, forming a model, doing math, using Bayes Rule?
  14. Is normalcy bias, hopeful thinking, avoidance of implications or social cognition subtly influencing your analysis? Are you unconsciously modeling after media?
Continue reading
Posted in Uncategorized | Leave a comment

Types and Degrees of Alignment

What would it mean to solve the alignment problem sufficiently to avoid catastrophe? What do people even mean when they talk about alignment?

The term is not used consistently. What would we want or need it to mean? How difficult and expensive will it be to figure out alignment of different types, with different levels of reliability? To implement and maintain that alignment in a given AGI system, including its copies and successors?

The only existing commonly used terminology whose typical uses are plausibly consist is the contrast between Inner Alignment (alignment of what the AGI inherently wants) and Outer Alignment (alignment of what the AGI provides as output). It is not clear this distinction is net useful.

Continue reading
Posted in Uncategorized | 2 Comments

Stages of Survival

This post outlines a fake framework for thinking about how we might navigate the future. I found it useful to my thinking, hopefully you will find it useful as well.

Continue reading
Posted in Uncategorized | Leave a comment

The Crux List

This is a linkpost for The Crux List. The original text is included as a backup, but it formats much better on Substack, and I haven’t yet had time to re-format it for WordPress or LessWrong.

Continue reading
Posted in Uncategorized | 2 Comments