Automated DR image assessment software grade fundus images, either instead of or alongside human graders. In this study, the IDx-DR (left) exceeded the sensitivity measures of the Medios AI (right) at the cost of lower specificity. Photo: Digital Diagnostics; Remedio. Click image to enlarge.

With the rising burden of ocular disease, early detection and screening is vital for many widespread conditions, especially those that may manifest at either the level of primary eyecare provider or even the general practitioner. At the same time, artificial intelligence (AI) systems have proven their mettle in recognizing patterns characteristic of disease in large photosets and then using that foundation to screen individuals for a potential match. The result is a gold rush of sorts to develop AI-powered devices offering simpler and more reliable disease screening in many facets of eye care.

Farthest ahead are screening technologies for diabetic retinopathy (DR), with some devices already on the market. Before implementing such AI software in real-world scenarios, it is crucial to conduct comparisons among different solutions available. A research team based in Poland recently analyzed the performance of two automatic screening assessments, IDx-DR and Medios AI. Only Idx-DR is currently available in the United States, and has recently been renamed LumineticsCore. IDx-DR uses a tabletop fundus camera while Medios is smartphone-based.

The retrospective study found that the sensitivity of detecting referable DR against reference standard grading was 95% and the specificity was 80% for Medios. The values were 99% and 68% for IDx-DR, respectively.

Both AI systems are based on convolutional neural networks. The Medios AI algorithm evaluated two possible outputs: “no signs of DR detected” (non-referable DR) and “signs of DR detected” (referable DR). The IDx-DR system also has an image quality and a diagnostic algorithm. The researchers noted that their study looked into using Medios outside of the smartphone application and involved a custom-made deployment.

Retinal images with sufficient image quality were run on both systems. They were captured in 811 consecutive patients with diabetes visiting diabetes clinics in Poland. For each patient, four non-mydriatic images, 45º field of view, i.e., two sets of one optic disc and one macula-centered image using Topcon NW400 were captured. Images were manually graded for severity of DR and categorized as follows:

• no DR
• any DR (mild nonproliferative DR or more severe disease)
• referable DR (moderate NPDR or more severe disease and/or clinically significant diabetic macular edema)
• sight-threatening DR (severe NPDR or more severe disease and/or clinically significant DME) by certified graders.

The automated DR image assessments’ outputs were compared with manual consensus image grading.

Four patients deemed ungradable by the graders were excluded. In total, 807 patients were included for further analysis. On 807 patients, based on consensus grading, there was no evidence of DR in 543 patients (67%). Any DR was seen in 33% patients, of which 22% were referable DR and 5% sight-threatening DR. The IDx-DR exceeded the sensitivity measures of the Medios AI at the cost of lower specificity.

“Another possibility of lower specificity could be referral of patients with similar lesions and concurrent pathologies that were not evaluated as part of this grading,” the researchers wrote in their paper for Ophthalmic Research. “Overall, both systems achieved satisfactory accuracy, particularly when patient safety is concerned with excellent negative predictive values, meaning very few patients receiving a false-negative result.”

The team also highlighted that their study included only patients with good quality, non-mydriatic images, which may not be representative of the whole screening cohort. Using a non-mydriatic protocol may underrepresent patients with smaller pupils or media opacities, particularly the elderly.

“Continued research and validation in larger and diverse patient populations will be essential to strengthen the evidence base and ensure the widespread adoption of these AI screening tools,” the authors concluded. “Our study underscores the promise of these AI systems for DR screening, facilitating early detection and timely intervention for improved patient outcomes.”