There was a recent paper in Nature that detailed the prospects of radiologist cooperation with deep learning algorithms to analyze chest X-rays. The results and their implications are particularly interesting, and this is the topic I will expound upon in this article, providing additional interpretation and speculation.
The Experiment and Results
In the experiment, a deep learning model, individual radiologists, and a group of radiologists connected with “swarm technology” were tasked to diagnose pneumonia from chest X-rays. Swarm technology sounds fancy, but it is essentially just a real-time collaborative software. It is designed to help a group of radiologists cooperate more effectively than by performing group votes for diagnoses, which is a method prone to unfavorable social influence.
The results were that the deep learning model (called CheXMax) performed the best on some metrics, while the swarm radiologists outperformed the model on other metrics. So, which one is better? The two perform best on different aspects of the task.
Indeed, the model had better _sensitivity _(more precise at detecting pneumonia if it does exist), and the swarm radiologists exhibited higher _specificity _(more precise when pneumonia does not exist). It follows from the model’s sensitivity that CheXMax tends to predict more low-confidence positives when the label is in fact negative. On the other hand, the swarm radiologists will be better at distinguishing those low-confidence positives since they more often err on high-confidence negatives.
So the researchers performed another test. The model was used to initially diagnose all patients, then the swarm radiologists re-diagnosed the low-confidence positives. Remarkably, the swarm correctly changed the prediction label 10/11 times.
The Implications
And hence we have arrived at the optimal pneumonia-diagnosing apparatus: a deep learning model augmented by a group of radiologists for its low-confidence positive predictions.
To be frank, the complementarity of the deep learning model and human diagnoses is probably a happy accident for the jobs of a great deal of radiologists as they are now. Not only does a single radiologist retain their use in a human-augmented deep learning system, but a group is also necessary to produce optimal predictions.
Although the deep learning model will cut the total number of cases that radiologists must review by presenting only low-confidence positives, each case would require more radiologists if swarm technology it to be utilized. The authors suggest that for understaffed hospitals and clinics, a single radiologist augmenting the deep learning model is still better than none, but demand for more radiologists would nevertheless remain.
#deep-learning #medical #artificial-intelligence #radiology #deep learning
