The hidden cost of an AI pilot that never makes it to production
The cost of an artificial intelligence pilot may seem low: a temporary license, a limited scope, a three-month timeline, and a department willing to test it. When the pilot ends without making it into production, the most common assumption is that little was lost: the cost was contained, the risk was isolated, and the learning was retained.
That learning rarely lasts. What almost never enters the equation is what the operation spent to make the demonstration work. Someone exported the database and cleaned it manually. Someone standardized exam names that routinely appear in different formats. Someone filled in a field that the system allows to remain empty. This effort does not appear on any budget line because it was performed by people who were already on the payroll, during their regular working hours.
This pattern is not exclusive to healthcare. According to the MIT NANDA study, 95% of generative AI projects fail, and this figure is often interpreted as evidence of a problem with the model. In diagnostic operations, that is rarely the case. The model does what it was designed to do. What is missing is the underlying data foundation it needs to operate every day, without special preparation, and continue delivering results.
The pilot that passed the wrong test
Every pilot answers a question, it is worth checking which one, one type of pilot can demonstrate that the model works: given a set of prepared examples, the system classifies, extracts, or generates suggestions with acceptable accuracy. Another, much rarer, type demonstrates that the operation can consistently supply the model with what it needs: under normal operating conditions, with the usual team and without prior data preparation, information arrives complete, in the right format, and at the right time.
Only the second question is relevant to production.
The first is almost always answered with a yes. It is quick to test and fits within a quarter, which is precisely why it is the test most organizations run. The second requires examining registration records, protocols, free-text fields where staff enter information that does not fit anywhere else, and lists of exceptions that exist only in the minds of two people. No demonstration can answer this second question. It can only be answered in live operations. That is why the project fails after the pilot, rather than during it.
The costs no one accounts for
When a pilot is closed, the recorded cost is usually the license fee, the real bill has at least six components.
Unbudgeted operating hours
Coordinators, analysts, and experienced staff were reassigned to prepare data, validate outputs, and explain exceptions. None of these hours were charged to the project, yet all of them reduced the operation’s capacity to serve patients.
Preparation that became a parallel process
The manual data extraction that supported the demonstration continues for as long as the pilot runs. It is new work created by the project, but no one accounted for it because it was never formally recognized as work.
Exceptions that went back to people without becoming rules
Every case the model could not resolve returned to someone’s queue, without a documented criterion for handling it. Staff made the decision again based on what they already knew, and that knowledge remained outside the system. The pilot relied on tacit knowledge but failed to turn it into explicit, documented rules.
The cost of reconciling discrepancies
There was a difference between what the system suggested and what the operation actually did. Someone had to correct that discrepancy later, whether in the patient record, the schedule, or the billing process.
The missed opportunity to structure the operation
Throughout the project, the operation continued recording information in the old way. No fields became mandatory, no protocols were standardized, and no scheduling rules were documented. When the pilot ended, the underlying data foundation remained unchanged, and the next project started from the same point, with one less year to work with.
The cost to internal credibility
This is the most expensive cost, and the only one that cannot be recovered through a budget.
Management approves the first pilot easily, the second with suspicion, and by the third, the organization is already convinced that “it doesn’t work here.” From that point on, the discussion is no longer technical.
How many of these six cost components were included in the evaluation of the last pilot your operation closed?
The rule that should come before the pilot
There is a simple, rarely applied criterion: an AI pilot should only be approved when the data it consumes is already generated by routine operations, in the format it requires, without special intervention. If the data had to be prepared for the demonstration, what was demonstrated was the preparation itself.
Applying this rule often delays the pilot because it may require expanding the scope of the solution. This runs counter to the expectations of those who buy technology to achieve results quickly, as well as those who sell technology promising quick solutions without the need for foundational work. Much of what is purchased today as an AI project should first be approached as a project to structure processes and formalize business rules.
This is something organizations should demand, specifically, from both the technology provider and the business team responsible for evaluating the project. The pilot should run on automated data extraction from the live operational environment, not on a specially prepared dataset. Success criteria should include the proportion of cases the operation can supply without manual intervention. And the contract should specify whether the business rules documented during the pilot remain with the institution or leave with the project.
All of this requires revisiting the project scope and may mean implementing solutions that address more fundamental operational needs. Although these solutions require more time and effort upfront, they are more likely to address the underlying problem and generate real business value.
Where the foundation needs to be in place before AI
At the entry point of a diagnostic operation, this foundation has a clear starting point. WiseTouch is the layer that makes scheduling business rules explicit. With WiseTouch, these rules must be structured within the system. The solution requires information and processes to be organized first, because they form the operational foundation on which it runs.
These rules include sequencing exams according to each patient’s preparation requirements, meeting authorization deadlines, checking coverage by location, accounting for travel time within facilities, managing add-on appointments, and handling multiple requests within a single workflow.
WiseTouch is the only system that automates all business rules for scheduling in SADT (auxiliary diagnostic and therapeutic services). Making these rules explicit transforms individual staff decisions into system rules, creating the prerequisite for any subsequent automation.
The self-service modules in the WiseTouch digital journey are built on this foundation. They include self-scheduling, medical order reading, automated health insurance card reading, online delivery of reports and images, customer service via WhatsApp and chat with an AI agent, and routing between AI and the human team. AI comes into play on top of an already structured process. It takes on repetitive tasks and routes cases requiring human judgment to the team.
Beyond the benefits of AI itself, we have observed that operations already using this structured workflow can achieve reductions of up to 15% in patient visit time for appointments involving multiple exams, as well as a reduction of around 10% in errors and repeat calls to the call center. These results stem from an organized operational foundation, before any AI model is introduced.
If your organization has already completed an AI pilot that never made it to production, the most useful next step is not to choose another model. It is to measure what your operational foundation can deliver today without manual intervention.
Talk to our specialists to learn how we can help your institution assess its current processes.