A new study looks at the effectiveness of seven artificial intelligence-based screening algorithms for diagnosing diabetic retinopathy, the most common diabetic eye disease for vision loss.
In a paper in diabetes care, researchers compared algorithms against the clinical expertise of retinal specialists. Five companies produced the algorithms tested in the United States (Ainec, Retina-AI Health), two in China (Airdock), one in Portugal (RateMarker), and one in France (OFTAI).
Researchers deployed algorithm-based techniques on retinal images of approximately 24,000 elderly seeking diabetic retinopathy screening at the Veterans Affairs Paget Sound Health Care System and Atlanta VA Health Care System from 2006 to 2018.
Researchers found that algorithms do not claim performance as well. Many of these companies are reporting excellent results in clinical studies. But his performance in a real-world setting was unknown.
Researchers performed a test in which the performance of each algorithm and the display of human screeners that operate in the VA teleretinal screening system were all compared to diagnoses that expert ophthalmologists had seen while viewing the same images. Were given
Three of the algorithms performed very well and one did worse when compared to physicians’ diagnoses. But the test demonstrated only one algorithm as well as human screeners.
Led researcher Aaron Lee, assistant professor of ophthalmology at the University of Washington School of Medicine, said, “It is dangerous that some of these algorithms are not performing consistently, because they are being used somewhere in the world.”
An explanation may be the difference in camera equipment and technique. The researchers said that their testing shows how important it is for any practice that wants to use the AI Scrurr to do the first test and follow the guidelines on how to properly get patients’ eye images , Because the algorithm is designed to work with minimal quality of images.
The study also found that algorithm performance varied when analyzing images from patient populations in Seattle and Atlanta care settings. This was a surprising result and may indicate that the algorithm needs to be trained with a wider range of images.