AI systems do not hallucinate. Humans hallucinate, with Tony Fish

AI governance exposes the limits of traditional fintech thinking: In conversation with Tony Fish, the episode challenges the assumption that AI governance is simply a regulatory or technical problem. As AI systems move into financial services, institutions are discovering that existing governance frameworks were designed for deterministic systems, not adaptive models trained on evolving data. While AI promises efficiency and automation, it also introduces uncertainty into environments where reliability and accountability are essential. The discussion highlights how the rapid adoption of AI is exposing structural tensions between technological innovation and the institutional systems designed to control risk.

Banks operate like railways, tech companies like roads: #

A central idea explored in the episode is the fundamental difference between how banks and technology platforms operate. Banks function like railway networks. Transactions move along predefined routes with strict controls and auditability. Technology companies, by contrast, operate more like road networks where decisions are made in real time and multiple paths are possible. When AI systems built in flexible, road-like environments are introduced into highly deterministic financial infrastructures, governance challenges emerge. The systems were never designed to work together, yet fintech increasingly forces them to converge.

Money and trust create higher stakes than data and attention: #

The conversation also highlights how economic foundations shape risk tolerance. Technology companies operate on data and attention, where errors typically have limited consequences. Financial institutions operate on money and trust, where mistakes can have irreversible effects on individuals and markets. This difference explains why AI deployment in financial services carries a fundamentally different level of scrutiny than similar technologies in consumer platforms. A recommendation engine can be wrong without major consequences, but an AI-driven financial decision can directly affect livelihoods.

Explainability alone cannot solve system complexity: #

Much of the current AI governance discussion focuses on model explainability. Fish argues that this framing risks oversimplifying the problem. Explaining an individual model decision does not mean the overall system is understandable. Financial institutions already run on complex infrastructures built from legacy systems, layered regulations, and interconnected services. As multiple AI models interact within these environments, emergent behaviors can appear that no single model explanation can fully capture. Governance must therefore shift from individual model transparency to understanding system-level behavior.

Regulation is struggling to keep pace with AI development: #

The episode also explores the growing gap between technological capability and regulatory oversight. Traditional financial regulation relies on static audits and deterministic systems. AI introduces probabilistic decision-making that evolves through interaction with data and other systems. As Fish describes it, regulators face an “exponential asymmetry” where the speed of AI innovation outpaces the tools designed to monitor it. This raises fundamental questions about how compliance and oversight should function in an AI-driven financial ecosystem.

Why listen: #

This episode examines one of the most overlooked challenges in fintech today: how to govern AI systems operating inside financial infrastructure. It explores the structural differences between banks and technology companies, the limits of explainability, and why traditional regulatory models may struggle to oversee adaptive AI systems. For fintech builders and financial leaders, it offers a deeper perspective on why AI governance is not just a compliance problem but a systemic shift for the industry.