AI #52: Oops

We were treated to technical marvels this week.

At Google, they announced Gemini Pro 1.5, with a million token context window within which it has excellent recall, using mixture of experts to get Gemini Advanced level performance (e.g. GPT-4 level) out of Gemini Pro levels of compute. This is a big deal, and I think people are sleeping on it. Also they released new small open weights models that look to be state of the art.

At OpenAI, they announced Sora, a new text-to-video model that is a large leap from the previous state of the art. I continue to be a skeptic on the mundane utility of video models relative to other AI use cases, and think they still have a long way to go, but this was both technically impressive and super cool.

Also, in both places, mistakes were made.

At OpenAI, ChatGPT briefly lost its damn mind. For a day, faced with record traffic, the model would degenerate into nonsense. It was annoying, and a warning about putting our trust in such systems and the things that can go wrong, but in this particular context it was weird and beautiful and also hilarious. This has now been fixed.

At Google, people noticed that Gemini Has a Problem. In particular, its image generator was making some highly systematic errors and flagrantly disregarding user requests, also lying about it to users, and once it got people’s attention things kept looking worse and worse. Google has, to their credit, responded by disabling entirely the ability of their image model to output people until they can find a fix.

I hope both serve as important warnings, and allow us to fix problems. Much better to face such issues now, when the stakes are low.

Gemini Has a Problem

Google’s Gemini 1.5 is impressive and I am excited by its huge context window. I continue to default to Gemini Advanced as my default AI for everyday use when the large context window is not relevant.

However, while it does not much interfere with what I want to use Gemini for, there is a big problem with Gemini Advanced that has come to everyone’s attention.

Gemini comes with an image generator. Until today it would, upon request, create pictures of humans.

On Tuesday evening, some people noticed, or decided to more loudly mention, that the humans it created might be rather different than humans you requested…

Sora What

Hours after Google announced Gemini 1.5, OpenAI announced their new video generation model Sora. Its outputs look damn impressive.

The One and a Half Gemini

Previously: I hit send on The Third Gemini, and within half an hour DeepMind announced Gemini 1.5.

So this covers Gemini 1.5. One million tokens, and we are promised overall Gemini Advanced or GPT-4 levels of performance on Gemini Pro levels of compute.

This post does not cover the issues with Gemini’s image generation, and what it is and is not willing to generate. I am on top of that situation and will get to it soon.

A Tale of Two Restaurant Types

While I sort through whatever is happening with GPT-4, today’s scheduled post is two recent short stories about restaurant selection.

The Third Gemini

[Editor’s Note: I forgot to cross-post this on Thursday, sorry about that. Note that this post does not cover Gemini 1.5, which was announced after I posted this. I will cover 1.5 later this week.]

We have now had a little over a week with Gemini Advanced, based on Gemini Ultra. A few reviews are in. Not that many, though, compared to what I would have expected, or what I feel the situation calls for. This is yet another case of there being an obvious thing lots of people should do, and almost no one doing it. Should we use Gemini Advanced versus ChatGPT? Which tasks are better for one versus the other?

I have compiled what takes I did see. Overall people are clearly less high on Gemini Advanced than I am, seeing it as still slightly to modestly behind ChatGPT overall. Despite that, I have not been tempted to switch back. I continue to think that Gemini Advanced is the better default for my typical queries. But your use cases and mileage may vary. I highly recommend trying your prompts in both places to see what works for you.

AI #51: Altman’s Ambition

[Editor’s note: I forgot to post this to WorldPress on Thursday. I’m posting it here now. Sorry about that.]

Sam Altman is not playing around.

He wants to build new chip factories in the decidedly unsafe and unfriendly UAE. He wants to build up the world’s supply of energy so we can run those chips.

What does he say these projects will cost?

Oh, up to seven trillion dollars. Not a typo.

Even scaling back the misunderstandings, this is what ambition looks like.

Monthly Roundup #15: February 2024

Another month. More things. Much roundup.

More on the Apple Vision Pro

Previously: On the Apple Vision Pro

The reviews are coming in. What say the people, other than the complaining about the two to three hour battery life?

Then later I’ll get to my own thoughts after the demo.

On the Proposed California SB 1047

California Senator Scott Wiener of San Francisco introduces SB 1047 to regulate AI. I have put up a market on how likely it is to become law.

“If Congress at some point is able to pass a strong pro-innovation, pro-safety AI law, I’ll be the first to cheer that, but I’m not holding my breath,” Wiener said in an interview. “We need to get ahead of this so we maintain public trust in AI.”

Congress is certainly highly dysfunctional. I am still generally against California trying to act like it is the federal government, even when the cause is good, but I understand.