Healthcare AI adoption is raising new questions about clinical oversight

Healthcare organizations face pressure from workforce shortages, administrative demands, and complex operations. This contributes to growing interest in artificial intelligence for clinical and administrative work. The TIAA Institute’s October 2026 report found that 52% of health systems had implemented generative AI by late 2025, up from 25% two years earlier.

Yet only 35% of AI proofs-of-concept advanced into full production. The findings suggest adoption is moving quickly, while validation and integration may require more organizational attention.

The implementation gap

The implementation gap becomes more apparent when leaders consider how AI insights enter daily decisions. In a 2026 survey of 281 healthcare leaders, Arcadia reported that 53% said AI insights were only partially embedded in decision-making. Only 14% described them as fully integrated at key decision points.

The survey identified workflow integration as a leading challenge to responsible adoption. Implementation may therefore depend on process redesign, staff preparation, and defined clinical responsibilities alongside technology deployment.

Those questions also extend to the external relationships supporting implementation. Healthcare executives may turn to vendors for specialized capabilities. Coordinating several providers can add communication channels and complicate responsibility for resolving problems.

KLAS Research’s 2026 report describes the ongoing work involved in monitoring external providers, noting that organizations can find vendor oversight difficult to maintain over time. Multiple vendor relationships may lengthen response times as teams handle separate processes and contracts. The strategic question for health systems is which capabilities require outside support and which may be developed within their own workforce.

Clinical knowledge

That question becomes especially consequential when clinical knowledge is absent from AI design and oversight. Technical teams can process large volumes of information, while frontline professionals understand how patient circumstances and workflows affect its meaning.

Pharmacists may contribute particularly useful insight to medication-related AI. They can help identify gaps in patient records, establish appropriate human review, and clarify who is accountable when an output requires intervention.

Dr. Dan Eduardo Gonzalez, founder of ClarityRx Advisory, a healthcare strategy, AI governance, data, and cybersecurity advisory firm, brings a pharmacist’s perspective to these issues. He points to medication histories as an example of how apparently complete digital records can leave out important details.

“Medication information is often more complicated than the record suggests,” Dr. Gonzalez says. “A pharmacist knows to ask about the prescription paid for in cash, the sample provided at an appointment, or the medicine a family member shared. Those questions can reveal information that changes how a clinical recommendation should be interpreted.”

The implications can extend to an organization’s financial assessment of AI. Evaluations that focus primarily on licensing fees, processing speed, or projected labor savings may leave out some costs. These include correcting incomplete information, reviewing questionable recommendations, and providing additional human intervention.

Clinical pharmacists can help identify these requirements during development. They can test workflows against realistic medication scenarios and determine when an AI-generated result should be escalated. Their participation may help anticipate costs and reduce rework.

Error review

A further element is the ability to assess errors consistently and learn from them. The National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) provides a relevant reference through its Medication Error Index.

NCC MERP’s framework classifies medication errors according to their severity, including whether an error reached the patient and whether harm occurred. Similar AI records could identify where an error occurred, its contributing factors and severity, and the corrective action assigned.

“Accountability becomes practical when an organization can trace an event from the original information through the decision and the response,” Dr. Gonzalez says. “Teams need to know where the process broke down, why it happened, who is responsible for addressing it, and how the correction will be documented.”

Such records could support internal review, risk analysis, and audits while helping distinguish data issues from broader design or workflow concerns.

Internal ownership

Establishing that accountability may require changes to organizational roles. Healthcare systems could consider creating internal positions dedicated to AI oversight and training current employees in governance and risk assessment. They could also give clinical professionals defined responsibilities within technology programs.

Incentives for clinicians to develop AI expertise may retain institutional knowledge and encourage participation. Internal ownership can also support faster resolution through familiar teams and clearer escalation routes.

“This further supports and incentivizes internal employees by creating positions that they can own,” Dr. Gonzalez notes. “External vendors may remain valuable partners, while the organization retains responsibility for how tools are used in its clinical environment.”

This is where Dr. Gonzalez’s work through ClarityRx Advisory sits within a wider shift toward clinician-informed technology governance. His work brings clinical, operational, and technical considerations into discussions of AI, data, and cybersecurity.

More broadly, healthcare organizations may need people who can translate clinical realities into system requirements and governance responsibilities. Pharmacists, in particular, can contribute expertise in medication safety, patient communication, and the interpretation of incomplete information.

Healthcare AI may deliver meaningful value when clinical professionals and technical teams participate throughout development, deployment, and ongoing review. Their collaboration can help organizations assess whether a system performs as intended. It can also show whether its information reflects patient circumstances and whether its outputs fit the workflow in which they are used.

As adoption expands, internal expertise, documented accountability, and structured error review may become increasingly important elements of responsible implementation. The opportunity for healthcare leaders is to build systems in which technical capability develops alongside clinical judgment, clear ownership, and a continuing commitment to patient safety.