The Missing Market: Why Agentic AI Needs Its Own Insurance Ecosystem

Our risk infrastructure has yet to catch up with today’s software. Historically, enterprise software was deterministic – the same inputs produced the same outputs. Liability frameworks, insurance products, and regulatory regimes all evolved around that premise. An insurer underwriting an errors and omissions policy for a software company could reasonably assess risk by examining the product's specifications, its intended use, and the company's quality assurance processes. The failure modes were knowable, even if they weren't always known.

Agentic AI breaks that model. These systems don't just surface information for a human to act on. They reason, plan, and execute multi-step tasks with meaningful autonomy. In healthcare, this distinction carries real weight. An AI tool that flags a potential care gap for a clinician to review is qualitatively different from one that initiates outreach to a patient, coordinates referrals across a care team, or generates clinical recommendations that shape treatment decisions in Medicare populations. The first is a sophisticated search engine. The second is an actor in the care delivery chain, and we lack the legal and financial architecture to allocate risk when that actor makes a consequential mistake.

The Gap in Today's Insurance Market

In Principals and Principles: Agentic AI in Healthcare, I explored the foundational liability question that agentic AI poses: are these systems more like products, agents, or something altogether different? I argued that our legal structures are built around a sharp distinction between sentient humans and inanimate things, and that agentic AI models fit neatly into neither category. They are what I called "quasi-principals," entities that act with a measure of autonomy but that our law does not currently contemplate. That legal ambiguity has a direct and practical consequence: it makes these systems very difficult to insure.

Traditional E&O policies contemplate failures in professional services or flawed software deliverables, not autonomous decision-making by a system whose outputs may be non-deterministic and difficult to reproduce. Cyber liability policies address data breaches and system intrusions, not the downstream clinical or financial consequences of an AI agent that reasons its way to an incorrect conclusion. General liability coverage assumes a relatively clear causal chain between a company's conduct and the resulting harm. Agentic AI complicates every link in that chain.

The causation problem I raised in Principals and Principles maps directly onto the insurance problem. When an AI agent operating in a value-based care environment makes a recommendation that contributes to a delayed diagnosis, who bears responsibility? The developer who wrote the code? The data sources from which the model learned? The deployer who put it into a clinical workflow? The physician who had access to the recommendation but may not have reviewed it? As I noted in that piece, the learning nature of AI models makes it difficult to even identify a single "manufacturer" in the product liability sense, let alone allocate fault among multiple parties (jointly and severally or otherwise). If insurers can't determine who is liable, they can't price the risk. And if they can't price the risk, there is no market.

The regulatory picture compounds the uncertainty. Various Federal and State agencies have jurisdiction over different dimensions of AI in healthcare, but none has issued comprehensive guidance on liability allocation for autonomous AI systems. Without regulatory clarity on what constitutes adequate AI governance, insurers can't calibrate risk, and companies can't build compliance programs with confidence that they'll be treated as mitigating factors if something goes wrong.

Early Signs of a Market Taking Shape

The good news is that the insurance industry is beginning to respond, even if the products remain nascent and immature.

On the carrier side, dedicated AI insurance offerings have started to emerge. In 2025, Armilla Insurance Services launched an AI liability policy underwritten at Lloyd's, one of the first products to offer affirmative coverage for AI-specific risks including hallucinations, model drift, and deviations from expected behavior. In March 2026, HSB (part of Munich Re) introduced AI liability insurance for small businesses, covering bodily injury, property damage, and advertising injury arising from AI use, designed to fill gaps that general liability policies exclude. Munich Re has gone further, standing up a dedicated "Insure AI" practice focused on developing underwriting frameworks that assess AI risk based on tolerance, stability, and severity metrics. Google Cloud, Beazley, Chubb and Munich Re have partnered to offer tailored cyber insurance with affirmative AI coverage for its cloud customers.

A parallel development may prove equally significant: major carriers have begun excluding AI liability from standard corporate policies, mirroring the pattern that played out with cyber and ransomware exclusions between 2019 and 2021. When liability is excluded from general coverage, companies become the primary stakeholders in AI governance because they can no longer transfer that risk. This dynamic is already funding AI governance programs that were starved of resources as recently as 2024.

Standards and certification frameworks are emerging alongside these products. For example, AIUC, a San Francisco-based startup, has developed a certification standard for AI agents covering data privacy, security, safety, reliability, and accountability, and pairs certification with insurance so enterprise customers are protected against AI agent failures. ISO/IEC 42001 now provides the first international management system standard for AI governance, and carriers are beginning to reference it as evidence of governance maturity during underwriting. NIST released a concept note in April 2026 for a new profile on trustworthy AI in critical infrastructure, a signal that the AI Risk Management Framework is becoming the "reasonable security" baseline that insurers expect.

In healthcare specifically, the Joint Commission and the Coalition for Health AI (CHAI) released a framework in September 2025 for responsible AI adoption across U.S. health systems. Malpractice carriers like The Doctors Company currently have no exclusion for AI and would still defend a physician if AI played a role in a claim, but some are introducing policy riders for practices that rely heavily on AI tools, often limiting coverage to FDA-approved uses and excluding experimental features. Stakeholder comments to HHS have proposed a "qualified AI" safe harbor, under which providers using validated, HHS-endorsed AI tools would have mitigated liability.

These are encouraging signals, but they are fragments of a market, not a functioning one. GenAI-related litigation in the U.S. has grown dramatically in the past several years. The pace of AI deployment is outrunning the pace at which risk transfer mechanisms are maturing. For the current trendlines to converge into a real market, more foundational work is required.

What Needs to Happen

An insurance market for agentic AI won't emerge spontaneously. It requires coordinated progress on several fronts.

Actuarial foundations come first. Insurers need data to price risk, and that data barely exists today. The industry needs standardized taxonomies for AI incidents, voluntary (and eventually mandatory) incident reporting frameworks, and benchmarks that allow underwriters to distinguish between well-governed and poorly governed AI deployments. Health tech companies have a role here: sharing anonymized incident data, publishing post-mortems on AI failures, and contributing to industry datasets that can support actuarial modeling.

Legal clarity is equally urgent. Legislators and regulators need to address liability allocation for AI-mediated decisions, including whether and how existing doctrines like learned intermediary, product liability, and professional negligence apply when an autonomous system sits between a technology provider and a patient outcome. Statutory safe harbors for companies that adhere to recognized governance standards would give both insurers and deployers a foundation to work from. The American Law Institute's 2024 Restatement of the Law of Medical Malpractice, which shifted toward a patient-centered concept of reasonable care, adds urgency: as AI-enabled workflows become pervasive and demonstrably useful, the standard of what a "reasonable physician" would do may evolve to include AI use, creating liability exposure in both directions.

Technical standards must follow. Certification regimes for AI systems, akin to what exists in medical devices but adapted for the continuous learning and deployment cycles of modern AI, would give insurers a basis for differentiating risk. Audit protocols that evaluate not just model performance but governance practices, human oversight mechanisms, and monitoring infrastructure would allow underwriters to assess operational maturity. Explainability requirements, tailored to the clinical context, would help establish whether an AI's reasoning was defensible at the time a decision was made.

Market structure matters too. Specialized underwriters with genuine AI risk expertise need to enter or be cultivated in the market. Reinsurance capacity needs to develop so that individual carriers can absorb potentially correlated AI failures across their books. And policy forms need standardization so that coverage terms are predictable and coverage disputes don't consume as many resources as the underlying claims.

Why Healthcare Can't Wait

Every industry deploying agentic AI faces some version of these challenges, but healthcare concentrates them. Patient safety creates a moral urgency that other sectors don't share. HIPAA breach exposure extends to any AI system processing protected health information, and agentic systems that coordinate care across providers multiply the attack surface. Algorithmic bias in care delivery doesn't just create reputational risk; it can produce disparate health outcomes across populations. OCR's updated Section 1557 regulations now establish nondiscrimination obligations for providers using AI and "patient care decision support tools," requiring identification and mitigation of discrimination risks. Corporate practice of medicine doctrines in many states constrain how AI can be integrated into clinical decision-making, creating compliance complexity that varies jurisdiction by jurisdiction.

The state regulatory picture is moving fast. In 2025, 47 states introduced over 250 AI bills affecting healthcare, with 33 enacted into law across 21 states. California's SB 1120 now prohibits health plans and insurers from denying, delaying, or modifying healthcare services based solely on AI’ assessment of medical necessity, requiring licensed physician review of any such AI-informed utilization decision. CMS has specified that AI cannot "act alone" to terminate or deny Medicare Advantage services. These are not theoretical risks. They are live compliance obligations, and the insurance market has yet to price them.

At Pearl Health, we build AI-powered tools for primary care providers and confront these issues daily, not as abstractions but as operational realities. Every model we deploy, every workflow we automate, every recommendation our platform surfaces carries both clinical promise and legal risk. We've built our governance practices to manage that tension, but we also recognize that company-level governance isn't sufficient. The ecosystem needs shared infrastructure: common standards, clear rules, and financial products that distribute risk appropriately.

What Comes Next

The path forward requires each stakeholder to act within its domain. Health tech companies should build for insurability now, investing in governance frameworks, audit trails, and explainability before the market demands it, because the market will. Insurers should develop dedicated AI risk practices staffed by people who understand both the technology and the regulatory environment, rather than trying to shoehorn agentic AI into existing product lines. Regulators should prioritize guidance on liability allocation and governance expectations, giving the market the clarity it needs to function. And industry groups should convene the working groups necessary to develop the standards and reporting frameworks that underpin everything else.

In Principals and Principles, I concluded that until technology and law develop appropriate mechanisms to protect the public from autonomous machines acting something more like principals, AI should remain dependent on humans who can be held accountable. I still believe that. But I also believe we need to build the infrastructure, including a functioning insurance market, that will eventually allow agentic AI to operate with greater autonomy in healthcare under proper safeguards. The question is not whether AI will assume more autonomous roles in care delivery; the question is whether we will have the legal, financial, and governance frameworks in place when it does.

At Pearl Health, we use AI to support our provider partners, not to displace them. Professionals continue to make medical judgments and bear ultimate moral and legal responsibility for their medical decisions, while receiving the technological leverage to provide higher quality, more efficient, and more abundant treatment for their patients. That principle won't change. But the ecosystem around it must evolve, and we intend to be part of building it.