When introducing a new system, a commonly adopted method in recent IT implementation projects involves first conducting a Proof of Concept (PoC) on a small scale to evaluate whether the system can address the client's challenges. If the PoC gives a GO decision, a pilot project is then conducted to verify the system from a business perspective. If the expected benefits seem achievable, the system is fully implemented and released. This approach to IT implementation projects, where a PoC is conducted initially, has been in practice for the past five to ten years. Prior to this period, when I was actively involved in system implementations, it was more common to carry out a small-scale pilot project as an initial validation step. If the pilot was successful, the system would then be fully implemented.
Observing the current mainstream approach of starting with a PoC, I feel that my experience may be becoming outdated. On the other hand, while this method is effective for traditional systems where a verification environment for the PoC can be built quickly and cost-effectively, it may not work well for projects involving the development of in-house AI models. In such cases, developing the AI model itself becomes the task of the PoC stage, which contradicts the original intent of a PoC. This significant concern arose about two years ago when I began examining the possibility of using AI to streamline the creation of clinical trial protocols.
The reason for this is simple: it is neither quick nor inexpensive to develop an AI model that can secure a GO decision at the PoC stage. If a satisfactory AI model were to be completed during the PoC, the AI development would essentially be finished, with only the addition of the necessary UI and AI training phases remaining. In other words, the AI would be completed by the end of the PoC. However, reaching this point requires considerable budget and time. The worst-case scenario is investing significant funds to create an AI model, only to have it fail the PoC, rendering the entire effort futile and deviating completely from the original purpose of a quick and low-cost PoC.
I was stuck after thinking this far, I sought the opinion of an acquaintance knowledgeable in AI, who simply responded, "Yes, that's right."
For instance, to consider building an AI model to support the optimization of clinical trial protocols by training it on data from ClinicalTrials.gov. The estimated cost for this AI model's development, including data cleaning, is approximately USD 280k. If one opts to purchase pre-cleaned data to avoid the hassle, the cost increases to about USD 380k to 450k. This is far beyond what one would typically allocate for a PoC.
The prudent approach, then, is to purchase an off-the-shelf AI model and adapt your processes to fit the model to some extent. This approach allows for small-scale pilot projects and the validation of new workflows, similar to traditional IT implementations.
In conclusion, not everything needs to be approached with a PoC. Utilizing packaged software for pilot projects is also a viable option.