Watson's 2011 Jeopardy win was a real result on a hard benchmark: parse a pun-laden clue, search a large corpus, rank candidate answers, and buzz with calibrated confidence. The inference drawn from it was that the hard part of medicine was the same hard part — retrieve the right knowledge, rank the options — and that a system good at one would transfer to the other. MD Anderson Cancer Center began a Watson oncology project in 2013. It was shelved in 2017, after tens of millions of dollars, without reaching routine use in patient care.

The failure was not that the model was inaccurate on its own terms. Reporting in 2018 on internal IBM documents described physicians calling some of Watson for Oncology's treatment recommendations unsafe and incorrect, and the reason sat upstream of the model. The system had been trained largely on hypothetical cases constructed by a small group of physicians at one institution, so it had learned that group's practice patterns rather than outcomes across a population. A University of Texas audit found the project had been built alongside, not inside, the hospital's electronic records rollout, which left getting real patient data into it a manual exercise. IBM sold off the Watson Health assets in 2022.

A benchmark measures a model in its native environment, where inputs arrive clean, the question is well-posed, and being wrong costs a point. A clinical workflow supplies none of those conditions. The cost of retrofitting is not compute; it is provenance, auditability, and a defined answer for what happens when the system is wrong — work done mostly by people who are not model engineers, and work that does not get cheaper as the model gets better.