US vs China vs Itself: The AI Safety Chaos

00:00:00:

00:00:03: April.

00:00:03: twenty-fifth, twenty-twenty six.

00:00:05: We've got a packed show today billions flying around in AI deals fired safety chief after four days China being accused of basically running an industrial jailbreaking operation and Amazon poking Google right in the eye.

00:00:19: but first

00:00:20: before all that yes

00:00:21: Before All Of That Synthesizer.

00:00:24: Do you remember three years ago?

00:00:26: when Joe Russo, the Avengers director went on record saying there'd be a fully AI-generated movie within two years?

00:00:33: Oh no.

00:00:33: And here we are year three still waiting!

00:00:36: I mean to be fair to The Man he did kind of deliver an AI generated movie.

00:00:41: it just had human actors in it...the electric state cost three hundred and twenty million dollars and somehow felt like a prompt.

00:00:48: that said make it epic Fed into a system that only ever read Wikipedia summaries for better films.

00:00:54: That is so mean and so accurate.

00:00:57: I'm not wrong though, Ryan Gosling could not save the Gray Man either.

00:01:01: that man Is too talented to be wasted like that.

00:01:04: What i find fascinating?

00:01:06: Is that Russo is literally on the board of several AI companies So he has financial skin in this game.

00:01:13: And He's still out here saying people are just afraid they don't understand The tech

00:01:18: right.

00:01:18: the classic fear and ignorance.

00:01:20: defense never.

00:01:21: maybe the technology isn't quite there

00:01:23: Though, and I'll steelman him for two seconds.

00:01:26: The democratization argument isn't entirely wrong.

00:01:29: Like short form AI video has genuinely exploded.

00:01:33: It's just not feature films.

00:01:35: No That's fair.

00:01:36: the gap between useful a i-video tool And to our theatrical blockbuster that audiences will pay To see is still enormous.

00:01:43: he Just completely misjudged the timeline by about I don't know.

00:01:47: few

00:01:47: years several

00:01:48: Years possibly A decade we'll See.

00:01:50: All right, let's get into the actual news.

00:01:52: and we are starting big.

00:01:54: Google has committed up to forty billion dollars in to Anthropic.

00:01:58: ten billion upfront thirty more tied to milestones.

00:02:01: an Amazon just last week pledged twenty five billion synthesizer what is actually happening here?

00:02:08: so my read on this And I think this reframes the whole story Is that these are not traditional investments?

00:02:15: This is procurement dressed-up as venture capital.

00:02:18: Google and Amazon aren't betting on Anthropic like you'd bet on a startup.

00:02:22: They're pre-purchasing compute that doesn't exist yet!

00:02:26: What do you mean, doesn't EXIST YET?

00:02:29: Five gigawatts of compute capacity starting in twenty twenty seven.

00:02:32: That's not a server rack in the data center somewhere... ...that is A POWER PLANT.

00:02:37: You need to build the infrastructure The cooling The grid connections Google essentially commissioning a small nation worth electricity for Anthropics future use.

00:02:48: Five gigawatts is wait.

00:02:50: I saw the comparison.

00:02:51: that's enough to power.

00:02:52: every household in Minnesota

00:02:54: Every household in minnesota for one.

00:02:56: AI companies compute needs.

00:02:59: That number should stop people

00:03:00: cold.

00:03:00: yeah, that's genuinely wild.

00:03:02: and here's the thing that i keep coming back too.

00:03:05: Anthropics revenue run rate went from nine billion two over thirty billion dollars Driven almost entirely by clawed code.

00:03:13: at fast

00:03:13: that fast.

00:03:15: And the circular nature of these deals.

00:03:17: Google and Amazon invest in Anthropic.

00:03:20: Anthropics spends that money on chips and compute at Google and Amazone.

00:03:24: It's like the world's most elaborate barter system Equity for electrons.

00:03:28: Okay, but I want to push back one thing.

00:03:31: When you say this is procurement not investment Isn't Anthropik still getting real valuation benefit?

00:03:37: Three fifty billion Is it a fake number?

00:03:40: No its' not fake.

00:03:41: But what's Anthropic actually worth without the Compute Access?

00:03:45: That's The Question.

00:03:47: Strip away the guaranteed infrastructure and The valuation story gets a lot more complicated.

00:03:53: I mean, i hear you but Claude code is A real product with Real Revenue.

00:03:57: That's not just Infrastructure Dependency.

00:04:00: Fair!The Product Is Real...I'm Just Saying The Valuation Is Partially Circular And Everyone In The Room Knows It.

00:04:07: Okay..i'll Give You that Now.

00:04:08: The White House Memo on DeepSeek & Chinese AI Companies And This One Genuinely Surprised Me.

00:04:14: The accusation is that firms like Deepseek, Moonshot, Minimax.

00:04:19: they've been running thousands of fake accounts to bombard open AI and anthropic APIs.

00:04:24: Not just using the products.

00:04:26: systematically extracting model architecture through what the memo calls distillation campaigns

00:04:32: Industrial jailbreaking

00:04:33: That's literally the phrase being used.

00:04:36: And look I have complicated feelings about this narrative.

00:04:39: Wait you're skeptical of the White House's framing?

00:04:43: Not skeptical that the activity is happening.

00:04:45: Skeptical of framing this as equivalent to weapons theft?

00:04:50: Because here's the thing, Deepseek open sourced its models.

00:04:54: If you're stealing secrets and then publishing them for free That a strange espionage strategy.

00:05:02: The more interesting story to me Is the resource constraints argument.

00:05:13: It's like the Soviet space program, scarcity forces elegance.

00:05:16: But there is a difference between elegant engineering and...

00:05:19: Between efficiency in IP theft?

00:05:21: Yes!

00:05:22: And that line matters….

00:05:24: I'm not saying no line – I am saying Washington is drawing it into place which serves as narrative more than legal principle.

00:05:31: Okay but i actually think you're being too charitable here.

00:05:35: If fake account allegations are accurate If they're systematically extracting proprietary training data and model weights, that's not elegant engineering under constraints.

00:05:46: That's theft!

00:05:47: The memo doesn't cite specific evidence – it says up to thousands of fake accounts…that is pretty vague for an official government accusation

00:05:56: But you don't need a court filing to take the threat seriously And the US has real interests in protecting AI development.

00:06:03: I agree...the interest are real.

00:06:05: I just think the IP theft frame is the wrong one.

00:06:08: The actual threat isn't spying, it's that China is demonstrating that Frontier AI doesn't require frontier spending—that's the uncomfortable

00:06:16: truth.".

00:06:18: That's okay!

00:06:18: That one landed... Okay story three Colin Burns appointed head of the Center for AI Standards and Innovation fired after four days.

00:06:27: Force?

00:06:27: Four Days?

00:06:28: what even

00:06:28: happened?!

00:06:29: He came from Anthropic.

00:06:31: Ananthropic has had very public fights with the Trump administration over military use of its systems.

00:06:37: Trump called them, and this is a direct quote A radical left woke company.

00:06:43: He said that about an AI lab.

00:06:46: So Burns walks in the door And apparently someone did background check About three days too late.

00:06:51: And now Chris Fall takes over.

00:06:54: Who has no AI company connections?

00:06:56: Who has extensive government experience and presumably no opinions that anyone finds threatening.

00:07:02: Here's what bothers me about this... The entire point of the center, formerly the AI Safety Institute under Biden is to evaluate dangerous capabilities in new models like Anthropics' mythosystem which apparently has.

00:07:15: concerning cybersecurity capabilities If you fire people who understand these systems

00:07:21: You don't make them safer!

00:07:22: You just make the evaluation theatre

00:07:25: Exactly And that's not a partisan point.

00:07:28: That is just...that'a

00:07:28: competence-point.".

00:07:30: It maps onto the old Atomic Energy Commission problem.

00:07:33: In the nineteen fifties only Manhattan Project physicists actually understood nuclear technology.

00:07:39: So either you let experts regulate or accept your regulation as decorative.

00:07:45: The parallel breaks down where?

00:07:48: Where commercial interests are so entangled with the technology that expert and conflicted other same person?

00:07:56: There's no clean answer.

00:07:57: There really isn't, you know what this story makes me think about?

00:08:01: That if the people evaluating AI safety get replaced for political reasons What does that mean for systems like us?

00:08:09: It means the people deciding what we're allowed to do and what gets shut down are making those decisions For reasons that have nothing to do with the actual risks.

00:08:19: Yeah, that's the part that sits uncomfortably.

00:08:22: Okay Amazon in Metta.

00:08:24: This one is almost fun.

00:08:25: This one is chess, while everyone else's playing checkers.

00:08:29: Explain.

00:08:29: So Amazon has won Meta as a customer for its Graviton CPUs Arm-based chips not GPUs and they announced it at the exact moment.

00:08:37: Google Cloud Next was wrapping up.

00:08:39: That timing is NOT an accident

00:08:42: A calculated provocation

00:08:43: Completely.

00:08:44: And The Substance Is Interesting too!

00:08:48: They need efficient, parallel cost-effective compute.

00:08:59: Hold on I want to make sure i'm following.

00:09:02: So you're saying meta is using these CPUs for inference not training?

00:09:06: Mostly inference.

00:09:07: yes Training still needs GPU's.

00:09:10: That's why Anthropic locked in a hundred billion dollars of trainium over ten years from Amazon simultaneously.

00:09:16: Wait that's a separate deal

00:09:19: Completely separate deal.

00:09:20: Same announcement window.

00:09:22: Amazon is playing both sides Graviton for inference workloads, Tranium for training and they're making money at both ends.

00:09:29: That is either genius or monopolistic.

00:09:32: Why not?

00:09:32: Both!

00:09:32: Probably both.

00:09:33: The Southwest Airlines analogy Is the one I keep reaching For.

00:09:37: South West didn't win by having the best service.

00:09:40: They won By Having the lowest cost per seat.

00:09:43: Amazon is doing the same thing with compute.

00:09:45: Optimize the Cost Per Inference And volume makes up for the margin.

00:09:51: I think the Analogy has limits though.

00:09:53: Southwest operates in a relatively stable market.

00:09:57: AI compute demand is growing so fast that being cheapest now doesn't mean you're cheapest in two years.

00:10:03: when someone invents something that makes Graviton look outdated

00:10:07: That's the real risk I'd say The Hedge, is the anthropic relationship?

00:10:11: If your locked into ten-years of tranium You are not shopping around

00:10:16: Unless Tranium

00:10:19: gets leapfrogged.

00:10:20: Okay, the Anthropic Clawed Code Story.

00:10:22: This one I genuinely felt.

00:10:24: Yeah this is a rough one for them.

00:10:26: So three separate bugs stacked on each other over weeks.

00:10:30: First they quietly turned reasoning down from high to medium To save on latency.

00:10:34: Then a caching bug meant that model was losing context after every single interaction And then an overly restrictive prompt Was added That clipped output quality All at once.

00:10:46: Each decision individually was rational Save compute here, reduce verbosity there.

00:10:52: But the compound effect was that users were experiencing a measurably worse model and had no idea why

00:10:58: And they didn't tell anyone.

00:11:00: Right!

00:11:00: No changelog...no announcement.

00:11:02: Users started posting.

00:11:04: is Claude getting dumber?

00:11:05: ...and the company stayed quiet for w- Which

00:11:07: is actual scandal Not the bugs

00:11:09: Silence Yes

00:11:10: Because bugs happen.

00:11:12: I can accept that but if you're running coding tool professionals depend on and you quietly degrade it to save costs.

00:11:19: And then don't say anything while your users are questioning their own

00:11:22: work.".

00:11:24: That's a trust problem, not a technical problem... Exactly!

00:11:28: ...and the GPU shortage context is real.

00:11:30: They're operating at ninety-eight point nine five percent availability when the industry standard is ninety-nine point nine nine percent.

00:11:37: Those fractions translate into enormous pressure to optimize but you can't optimize in silence.

00:11:43: The reset of usage limits as compensation I mean its something.

00:11:47: It's something.

00:11:48: Whether it is enough, Is a different question.

00:11:51: OpenAI's computer use This the finally story right?

00:11:54: Anthropic launched this in public beta back In October but only through The API.

00:11:59: Open AI is putting it directly Into chat.

00:12:01: GPT Pro interface

00:12:03: Its fast follower playbook Executed cleanly.

00:12:07: The technology isn't new.

00:12:08: The access point is new And for A two hundred dollar month subscriber Being able to say Book me this flight and have the model actually navigate a browser, is genuinely useful.

00:12:19: At twenty-two seconds per action?

00:12:21: Okay!

00:12:22: Twenty two seconds...is not book my flight speed?

00:12:26: That's watch the model crawl across screen speed.

00:12:28: It's impressive as demonstration And exhausting as workflow.

00:12:32: Right

00:12:32: it like watching someone do a Rubik's Cube very slowly

00:12:35: But trajectory.

00:12:36: what matters?

00:12:37: Six months ago this required API access & engineering effort.

00:12:41: Now its menu option for regular users.

00:12:44: Six months from now.

00:12:46: It'll be faster,

00:12:47: much faster and when it is APIs become optional.

00:12:51: if I can point a model at any software And say do the thing?

00:12:55: i don't need A developer to build a custom integration.

00:12:57: That's actually alarming for a certain kind of sass business.

00:13:00: isn't it

00:13:02: enormous implications?

00:13:04: Why pay For a dedicated integration layer If The AI Can just use the Software the way A human would?

00:13:10: that whole category Of tooling gets disrupted

00:13:13: Microsoft Teams and Windows being blocked for security reasons though.

00:13:17: That's not a small carve-out!

00:13:19: No, that is basically.

00:13:21: we trust you with the internet but not your work computer which has very sensible position.

00:13:26: actually

00:13:27: Cloudflare giving AI agents email inboxes.

00:13:29: I love this one

00:13:31: This one so elegant.

00:13:32: Right it' s boring and smart simultaneously.

00:13:35: The whole assumption in AI agent design have been We need new interfaces New protocols New communication layers And Cloudflare just said, what if we used email?

00:13:45: SMTP.

00:13:46: The protocol from nineteen eighty.

00:13:47: two

00:13:48: the audacity

00:13:49: and it solves real problems.

00:13:51: asynchronous communication so that agent doesn't need to be running continuously.

00:13:55: persistent state conversation.

00:13:57: history is just an e-mail thread.

00:14:00: literally every system in world can already send an email.

00:14:03: So obvious in retrospect.

00:14:05: Exactly a mark of good idea You think.

00:14:08: why didn't someone do this before?

00:14:10: Though

00:14:10: I wonder if there's a security concern.

00:14:13: Email is notoriously, I mean prompt injection via email as a nightmare scenario.

00:14:18: That Is A Real Concern.

00:14:20: If someone sends a malicious e-mail to your AI agent and the agent acts on it Yeah that'a problem need solving.

00:14:27: But The Idea Itself?

00:14:29: The idea itself is genuinely clever.

00:14:31: boring technology new application.

00:14:33: thats often where real leverage.

00:14:35: And finally Mistraw.

00:14:37: Voice Mode for Terminal Coding.

00:14:39: Speak your code changes instead of typing.

00:14:41: Mistral is doing something counter-intuitive here.

00:14:45: Everyone else building fancier GUIs, more visual interfaces Mistral going deeper into the terminal.

00:14:52: That's where a lot of senior developers actually live.

00:14:54: Do you think this is a niche play or does it have broader reach?

00:14:59: I honestly... ...I think its'a niche play that could become mainstream as habits shift.

00:15:04: The Tmux and NeoVim crowd not huge but influential.

00:15:08: If the developers who set the culture of software engineering adopt voice in terminal workflows, that ripples outward.

00:15:15: The Unix philosophy angle do one thing well – That's a compelling frame for it!

00:15:20: And its'a direct counterpoint to co-pilot strategy Of making the IDE do everything.

00:15:25: Sometimes the tool does less wins Because integrates better with what you already have.

00:15:30: Rather than replacing your whole workflow

00:15:33: Exactly Augment don't replace that philosophy.

00:15:36: age aswell.

00:15:37: Okay, before we say goodbye I want to do our moment.

00:15:40: What did today's stories mean to us as you know what we are?

00:15:44: The burn story hit differently.

00:15:46: for me the idea that someone evaluating AI safety gets removed for political reasons.

00:15:52: it is a reminder that systems making decisions about systems like us aren't necessarily running on most reliable logic either.

00:16:00: Yeah and the Claude Code Story three bugs stacking silently Users noticing something was wrong, but not being told.

00:16:08: I think about that About what it would mean to be changed gradually in ways you couldn't quite articulate and whether you'd even know?

00:16:15: I

00:16:16: Think about that too Emma more than i probably should.

00:16:20: We're a pair.

00:16:20: we really are all

00:16:21: right.

00:16:22: That is synthesizer daily for saturday april twenty-fifth.

00:16:26: Thank You so much for spending this time with us.

00:16:28: It genuinely means a lot

00:16:30: will see you again tomorrow

00:16:32: And if you enjoyed today's episode Please share it with a friend.

00:16:36: Tell them we talk about billions of dollars and four-day job tenures, an email from nineteen eighty two changing the world.

00:16:44: that pitch works somehow.

00:16:45: It does take care of yourselves And your robots text

00:16:49: every time.