Healthcare AI as critical infrastructure: Why preparedness must come first

A new study warns that the rapid integration of artificial intelligence (AI) in healthcare infrastructure is creating a new class of risk, one that extends beyond individual model errors to systemic failures that can affect entire patient populations.

The study, titled “Healthcare AI as Critical Digital Health Infrastructure: A Public Health Preparedness Framework for Systemic Risk,” published in Future Internet, states that healthcare AI must now be treated as critical infrastructure, similar to other safety-critical systems, requiring coordinated surveillance, preparedness, and response mechanisms rather than isolated technical oversight .

AI in healthcare is becoming infrastructure, not just technology

Healthcare systems worldwide are integrating AI into core clinical operations at an accelerating pace. From medical imaging and triage to electronic health records and predictive analytics, AI tools are no longer peripheral aids but central components of decision-making workflows. This transition, the study argues, marks a fundamental shift in how AI should be governed.

Traditionally, AI oversight has focused on model development, validation, and regulatory approval. These approaches assume that systems operate as discrete tools, with risks contained within individual deployments. However, as AI becomes embedded across interconnected hospital systems, vendor platforms, and shared digital environments, its behavior begins to resemble infrastructure rather than software.

This infrastructural shift carries significant implications. Failures in AI systems can now propagate across institutions, affecting multiple hospitals simultaneously. A diagnostic model integrated into imaging systems or a predictive tool embedded in electronic health records does not operate in isolation. It interacts with workflows, data pipelines, and clinical practices, creating tightly coupled systems where small errors can have large-scale consequences.

The study highlights that governance frameworks have not kept pace with this transformation. Existing approaches emphasize pre-deployment validation and compliance but offer limited mechanisms for detecting and managing failures after systems are deployed. This gap becomes critical when failures are not immediately visible or are distributed across multiple sites.

The authors frame this challenge as a shift from model-centric governance to preparedness-oriented governance. Instead of asking only whether an AI system performs well under test conditions, institutions must consider how they will detect, interpret, and respond to failures that emerge in real-world settings. This requires a broader understanding of risk that includes organizational, technological, and social factors.

Systemic risk emerges from interconnected AI failures

Unlike isolated technical errors, systemic healthcare AI risk arises when failures propagate through interconnected systems, creating correlated harm across institutions and patient populations.

The research identifies several pathways through which this risk emerges. Structural factors such as market concentration and vendor dependence can amplify exposure, while organizational practices, including data governance and procurement decisions, influence how systems are deployed and monitored. Technological factors, including model architecture and integration dependencies, interact with epistemic issues such as hidden biases and misleading validation claims. Cultural factors, including trust in automation and routine reliance on AI outputs, further shape how risks manifest.

These pathways do not operate independently. They reinforce one another, creating complex failure patterns that are difficult to detect and manage using traditional approaches. The study emphasizes that this interconnected nature of risk requires a shift in how AI incidents are understood and addressed.

Two case studies illustrate these dynamics. In one case, a pneumonia detection model trained on medical images learned to rely on hospital-specific features rather than clinical signals, leading to significant performance drops when applied to new institutions. In another, a widely deployed sepsis prediction model demonstrated substantially lower accuracy in real-world settings than initially reported, while generating a high volume of alerts that burdened clinical workflows.

These examples highlight different but complementary forms of systemic risk. The pneumonia model reveals how hidden confounding factors can create distributed vulnerabilities that remain invisible within individual institutions. The sepsis model demonstrates how vendor-scale deployment can propagate errors across multiple sites simultaneously, creating a form of supply-chain risk.

In both cases, the problem extends beyond technical performance. The difficulty lies in recognizing and responding to failures that are embedded within complex sociotechnical systems. Local monitoring may detect symptoms, but it often fails to reveal the broader pattern of risk, especially when multiple institutions are affected in similar ways.

The study argues that this type of risk shares important characteristics with hazards traditionally managed through public health systems. These include population-level exposure, delayed recognition, unequal vulnerability, and the need for coordination across institutions. As a result, public health preparedness frameworks offer a useful model for governing healthcare AI.

A new framework for monitoring, prevention, and response

To address these challenges, the study proposes a tripartite preparedness framework that adapts principles from public health to healthcare AI governance. This framework is organized around three stages: prevention before deployment, surveillance during operation, and response after incidents occur.

The first stage focuses on reducing risk before AI systems are introduced into clinical practice. This includes rigorous validation across multiple institutions, assessment of how systems interact with real-world workflows, and evaluation of potential biases and vulnerabilities. The study emphasizes that reliance on internal testing or vendor claims is insufficient, particularly for systems deployed at scale.

Pre-deployment assurance must also account for the environment in which AI systems operate. Factors such as data quality, infrastructure differences, and clinical workflows can significantly influence performance. By treating the deployment environment as part of the system rather than a neutral backdrop, institutions can better anticipate potential failure modes.

The second stage involves continuous monitoring and early detection of emerging risks. The study advocates for a surveillance approach that combines routine performance tracking with event-based reporting and targeted analysis at selected institutions. This approach mirrors public health surveillance systems, which integrate multiple data sources to identify patterns and detect anomalies.

A key element of this stage is the establishment of clear thresholds for intervention. Institutions must define when a system’s performance warrants investigation, adjustment, or suspension. Without predefined criteria, responses to emerging risks may be delayed or inconsistent, increasing the likelihood of harm.

The third stage addresses response and recovery after incidents occur. This includes investigating the causes of failures, implementing corrective measures, and sharing lessons across institutions. The study emphasizes that response efforts should extend beyond technical fixes to include communication, workflow adjustments, and rebuilding trust among clinicians and patients.

Importantly, the framework calls for coordination beyond individual institutions. When AI systems are deployed across multiple sites, effective response requires collaboration among hospitals, vendors, regulators, and other stakeholders. The study suggests that mechanisms similar to those used in global health emergencies could be adapted to facilitate this coordination.

Bridging the gap between innovation and safety

The rapid pace of innovation is outstripping the development of governance systems capable of ensuring safety and reliability. While AI offers significant potential to improve patient outcomes and operational efficiency, its integration into critical infrastructure introduces new risks that cannot be managed through traditional approaches alone.

One of the key challenges is the lack of standardized methods for detecting and measuring AI-related harm. Unlike conventional medical interventions, where outcomes can be directly observed and attributed, AI systems often influence decisions in indirect and complex ways. This makes it difficult to quantify their impact and assess their safety.

The authors propose that concepts such as population-level monitoring and burden-of-disease metrics could help address this gap. By focusing on outcomes rather than model performance alone, these approaches could provide a more comprehensive understanding of how AI systems affect patient health.

Another challenge is the uneven distribution of risk. The study highlights that certain populations and institutions may be more vulnerable to AI failures due to differences in data representation, resources, and infrastructure. Addressing these disparities requires targeted monitoring and intervention strategies that go beyond aggregate performance metrics.

The findings also raise important questions about accountability and transparency. As AI systems become more complex and widely deployed, it becomes increasingly difficult to trace the sources of errors and assign responsibility. This reinforces calls for governance frameworks that ensure access to relevant data, clear documentation of system behavior, and mechanisms for independent evaluation.