Physician judgement vs. algorithms: which is better in assessing clinical status?

This recent study in the journal Critical Care Medicine gives some useful information about an increasing trend in all of medicine, and certainly in my field of pediatric critical care. It has been well shown a key part in any acute care setting is distinguishing the really sick from the not so sick. We have some screening tools that nurses and other nonphysicians can use to help make that distinction. They use things like heart rate, respiratory rate, and maybe some simple blood tests. In pediatrics, there is a mountain of research trying to answer if some screening tool or test can help. Earlier work has shown really the best tool is for the most experienced person simply to look at the child and say: “This kid is really sick.” We all do this, kind of using gestalt and various soft observations that are hard to quantify and describe. Now we have some data about that, although it studied people of all ages.

The study is from Amsterdam and it assessed if the tool they use, called MEWS, which is similar to many of them, predicts serious illness better than clinical judgement. To me, the results are really what I would expect, which is not so much. All these screening tools tend to have a very high false positive rate, in that they predict serious illness in patients that end up being not seriously ill. I suppose you want that to some extent, since you don’t want to miss serious illness. But in this study the false positive rate was quite high. It was simple clinical judgement by a physician that was the best predicter. The authors conclude:

“Medical professionals can predict the development of critical illness within 3 days significantly better than the MEWS. Although MEWS is able to correctly predict those patients that become critically ill, its use leads to overestimation due to a substantial number of false positives.”

I am sort of reassured by this; there are still some things algorithms can’t do better than an experienced human.