Thanks to Jordan Schneider gifting a last minute ticket for an amazing seat, serendipity led me to go to Friday night’s performance by Jerry Seinfeld at the Beacon Theater.
If I had to describe the show in one word, I’d say it was unsurprising. Jerry was Jerry. He had thoughts. Mostly he had complaints. They were all Very Seinfeld.
If I had two words I might say mildly amusing. Which was good enough for a worthwhile evening. Live performances are something special. Every time I’ve gone out to a comedy show, even if a bunch of it was kind of lame, I have been happy I came. The correct bar for worth watching is actually lower in person than at home.
The thought I couldn’t shake as I went home was, what GPT level was that on?
This first came to me during the opening act. His opening act was if anything too on the nose. Very Jerry. Much Seinfeld.
It felt very much like he had GPT-X put together a package of Standard Mildly Amusing Jokes and Perspectives, with the prompt that the performer was black, old, male, never married and the opening act for Jerry Seinfeld. Given the material, delivery was solid.
The question was, what was the X in GPT-X?
I am confident our current incarnation, which is GPT-3.5 or so, can’t do it.
What about GPT-4? Could go either way, if you could try out bits and select winners.
What about GPT-5? Yes, absolutely, if it is what I’d expect, that should work.
So I’d give that act a GPT-level rating of 4.25.
Jerry himself was on a higher level. Not as high as his peak. I’d give him a 5.
What about the best stand-up shows and specials?
I’ve seen 6s. I worry I may have never seen a 7.
Don't Worry About the Vase 
ChatGPT summary:
The writer attended a comedy show by Jerry Seinfeld and was impressed by the quality of the performance, describing it as “mildly amusing.” The writer also speculates about the potential ability of various versions of a language generation model called GPT (Generative Pre-trained Transformer) to generate similar material, rating the opening act at a GPT level of 4.25 and Jerry Seinfeld himself at a level of 5. The writer suggests that the highest level of stand-up comedy performances they have seen are 6, and wonders if they have ever seen a 7.
I mean, this would make me laugh if I saw it performed by Seinfeld, but not for the punchline:
Write a joke about raincoats in the style of jerry Seinfeld:
“You know what’s funny about raincoats? They’re completely unnecessary in the shower. I mean, think about it. You’re already standing under a stream of water. You’ve got water all around you. But for some reason, we still feel the need to put on a raincoat. It’s like, what are we trying to protect ourselves from? The water we’re already standing in? And then when you take it off, you’re just standing in the shower, all wet, like, ‘Well, that was a waste of a raincoat.’ It’s like, why do we even bother? It’s like, if you’re going to stand in the shower, just embrace the water. Don’t try to protect yourself from it. Just let it soak you. It’s like, whoa. Mind. Blown. Raincoats are completely unnecessary in the shower.”
Yeah, there’s a clear *not quite getting it* here but you could definitely punch that up a bit and have something.
What does “I’ve never seen a 6” even mean?
You’re writing like a 6 is an order of magnitude better at X than a 5. But the thing 6 would be an order of magnitude better -at- than 5 is “successfully impersonate Z”.
“This AI is better at being Jerry Seinfeld than Jerry Seinfeld” is not the AI doing a good job at being Jerry Seinfeld, it’s the observer having a poor, caricaturesque model of Jerry Seinfeld.
Assuming away the alignment problem; the standard “moon, not finger”. The hypothetical AInfeld is better at being the ideal comedian (even “the ideal Seinfeld-type comedian”).
A seven pretty much requires a world class world model, and a sense of what’s creative and sufficiently rule breaking on the margins of that world model. Carlin at his best. I doubt any GPT will *have* a world model in the next decade or two, much less a good one, much less the best one.
“GPT” is quickly going to become the new “NPC”. Except it’s more believable and therefore more annoying.
NPC? Non-player character?
Yeah. https://knowyourmeme.com/memes/real-life-npc