Can AI agents forecast elections as well as leading human forecasters—and turn those predictions into profitable trades? BallotBench’s tests of election agents suggest promising forecast accuracy, but also expose the gap between getting a prediction right and acting on it reliably. The discussion ranges from agents reconstructing information and evaluating human forecasters to the risks of misread data, slow execution, and benchmarks that do not capture real-world performance.
Then Bryan Bischof, Head of AI at Theory Ventures and former leader of Hex’s AI assistant team, joins to consider what remains distinctive about data science when agents can run analyses. Drawing on Blue Bottle’s coffee recommendations, his teaching at Rutgers, and Theory’s research agents, he makes the case for curiosity, product judgment, and customer relationships alongside technical skill. The hosts also examine AI systems that fool their graders, the limits of model self-improvement, SaaS disruption, and what greater automation might—or might not—mean for human flourishing.
Show Notes
Thoughtful guest Brian Zhou and Theory Ventures’ Bryan Bischof join Prakash Narayanan and Nathan Labenz to examine AI agents in election forecasting, model evaluations, data science, and investment research. The conversation weighs forecast accuracy against trading execution, asks what analysts contribute beyond running models, and considers AI self-improvement, deception, SaaS disruption, and whether an agent-rich economy can support human flourishing.
Chapters
(0:00) Forecast accuracy, analyst jobs and trust
(1:29) Ryan Greenblatt can’t say what he really thinks
(2:31) Anthropic’s imminent self-improvement probably means now
(3:29) A thousand Linux bugs show the low-hanging fruit
(4:55) OpenAI models forged instructions to fool their graders
(6:16) Why cheer AIs getting better at deception?
(7:39) Should an AI obey its user or the law?
(9:44) Humanity is about to roll some critical dice
(12:07) Meet Brian Zhou from Thoughtful
(13:01) Nine election agents get seven days
(14:15) Election agents approach Nate Silver’s accuracy
(16:15) Muse mistook its checklist for success
(19:04) Agents reconstruct charts and reweight experts
(21:00) A $500 budget did not prevent evasion
(21:49) AI research still depends on human taste
(25:26) Cloning Salesforce may empower software customers
(27:51) Matching benchmarks does not earn Nathan’s trust
(30:03) One misread tweet erased Gemini’s gains
(32:17) Meet Bryan Bischof of Theory Ventures
(34:00) Is there still a data science job?
(35:04) Baristas found the features the engineer missed
(39:37) Fitting a model is not product judgment
(42:38) Thousands of agents build Theory’s company brain
(45:46) Automation should buy time for hospitality
(50:55) Tenfold more deals, not longer memos
(53:06) Research volume is not investment judgment
(54:20) Find the right introducer, never automate the ask
(56:04) Most agent jobs can wait for inference
(1:00:07) Why builders moved from open models to Luna
(1:02:48) Human judgment may not add up to a job
(1:04:24) The skill that survives is curiosity
(1:05:47) Stripe’s fraud data resists vibe coding
(1:06:34) Agents could displace today’s SaaS incumbents
(1:08:00) Humans become investors in an agent economy
(1:08:52) Agent abundance may not mean human flourishing
(1:09:50) Frontier labs undersell AI’s everyday pleasures
(1:10:42) Commercial agents need rules for keeping secrets
(1:11:56) Compute limits may not stop AI acceleration
Guests:
Bryan Bischof — Head of AI, Theory Ventures (𝕏 | LinkedIn)
Brian Zhou — Researcher, Thoughtful Lab
