AI safety testing can be useful and still miss the risks that matter most when capabilities move faster than the institutions built to evaluate them. In this episode, Prakash Narayanan and Nathan Labenz discuss guardrails, fast classifiers, multi-agent coordination, and the challenge of measuring alignment in systems that keep changing shape.
Max Nadeau of Coefficient Giving joins the conversation to unpack why safety audits can become box-checking exercises, how funders think about uncertain technical work, and why talent is often the real bottleneck. The closing segment shifts to AI shopping agents and the broader question of who owns the customer as agentic commerce becomes more real.
Show Notes
Prakash Narayanan and Nathan Labenz open with the limits of AI safety testing, including finite-choice guardrails, fast classifiers, emergent multi-agent behavior, and why alignment measurement gets harder as systems become more capable. Max Nadeau of Coefficient Giving then explains why audits can turn into box-checking, how funders assess uncertain technical safety work, and why talent may matter more than budget. The episode closes with AI shopping agents, Amazon’s response, and the question of who owns the customer relationship when agents become first-class users.
Chapters
(0:00) When safety becomes reflexive.
(0:10) When AI audits become box-checking.
(1:22) Who owns the AI customer?
(2:01) Opening and live setup
(4:20) JEV and fast classifiers
(5:40) Guardrails for AI agents
(7:12) AI moderation use case
(11:25) System one and system two
(13:24) Multi-agent scaling
(21:18) Emergent agent hierarchies
(23:43) The case for slowing down
(34:28) AI and mathematics
(41:10) Capability acceleration
(45:45) Building an AI organization
(57:36) Funding the AI project
(58:17) Meet Max Nadeau
(1:05:21) Why safety research resists metrics
(1:15:13) When AI audits become box-checking
(1:23:07) Funding across uncertain timelines
(1:30:49) Why fund whole organizations
(1:39:06) Preparing for larger grants
(1:39:16) Why talent is the bottleneck
(1:46:48) Opening and agent setup
(1:49:16) The AI services business model
(1:52:59) Muse and autonomous shopping
(1:56:52) Who owns the customer
(1:57:37) Agents as the universal interface
(2:00:01) Why ban shopping agents
(2:05:31) The bot-detection arms race
(2:07:26) A better lane for agents
(2:09:30) Agents as internet users
(2:11:31) Closing and next steps
Guests:
Max Nadeau — Program Officer, Coefficient Giving (𝕏 | LinkedIn)
