These findings open up possibilities of using AI in the clinic or with home monitoring to obtain BCVA efficiently and economically, decreasing the time and expense associated with DME management. Photo: Jay Haynie, OD. Click image to enlarge.

With increasingly more daily tasks being targeted for possible assistance—or even replacement—by artificial intelligence (AI), optometrists and ophthalmologists are eagerly anticipating potential AI applications in eye care.

Take routine follow-up of patients with diabetic eye disease. While obtaining central subfield thickness in-clinic can be quick and accurate, determining BCVA by trained examiners using refraction and VA testing can take up to 20 minutes for each patient. Eyecare providers and assisting technicians might obtain VA using habitual correction rather than BCVA and possibly forgo use of an ETDRS eye chart.

However, researchers at Johns Hopkins University School of Medicine took it a step further, theorizing that if deep learning systems could estimate BCVA accurately from a fundus photograph in an eye with diabetic macular edema (DME), it might obviate the need for protocol refraction and VA measurement by trained examiners, especially if these images could be obtained using phone cameras. Their diagnostic/prognostic study, published in JAMA Ophthalmology, found that the mean absolute error of AI-estimated BCVA from macular fundus photographs across all study visits and treatment groups was within 10 letters of actual BCVA.¹

Deidentified color fundus images taken after dilation were used post-hoc to train AI systems to perform linear regression from image to BCVA and to evaluate the resulting accuracy. Participants were patients enrolled in Regeneron’s VISTA randomized clinical trial through 148 weeks, wherein the study eye was treated with aflibercept or laser.

Analysis included 7,185 macular color fundus images of the study and fellow eyes from 459 participants. Overall, the mean age was 62.2, and 54.5% were male. The baseline BCVA score for the study eyes ranged from 73 to 24 letters (approximate Snellen equivalent: 20/40 to 20/320).

Using an AI tool suited to retinal images, the mean absolute error—a measure of the average size of mistakes in a regression model, with lower scores being better—for the testing set (n=641 images) was 9.66. In clinical terms, 33% of the values were within zero to five letters, and 28% were within six to 10 letters. For BCVA between 20/10 to 20/25 (n=161) and between 20/32 to 20/80 (n=309), the mean absolute error was 8.84 letters and 7.91 letters, respectively. The AI algorithms described in this study tended to perform worse among patients with poorer VA. The study authors proposed that this may have been due to a relatively small number of eyes with poorer VA represented in the training data.

When extrapolating from research trials to the clinical practice setting, the decision to initiate treatment for DME can depend on BCVA. “However, most practices do not obtain a BCVA following a protocol refraction and protocol VA measurement using an ETDRS or electronic ETDRS chart at most visits, if not all,” the researchers wrote in their paper. “Deep learning analysis of macular images might circumvent this problem.”

Furthermore, the time involved to take fundus photos and estimate BCVA “could be less than the time needed to obtain protocol refractions and protocol VA measurements and might be faster than measuring VA with habitual correction if similar outcomes can be obtained from digital images of the macula captured without dilation,” they added. “Using AI algorithms to estimate BCVA also may facilitate determining follow-up intervals or retreatment.”¹

A commentary also published in JAMA Ophthalmology noted, “While the algorithm in this study did demonstrate some ability to make reasonable BCVA estimates, its performance was relatively modest overall.”²

Although data on how race, ethnicity and sex were reported was not recorded in this post-hoc analysis, the commentary authors pointed out that there was a lack of racial and ethnic diversity among the data used to train the algorithm, given that the study cohort was over 80% Caucasian. They suggested that one possible approach to addressing the diversity gap in algorithms based on clinical trial data would be to perform neural network training or validation using data from diverse populations generated in the course of routine clinical care.

An interesting subanalysis in the main study examined outlier cases where AI-predicted BCVA was more than two standard deviations away from protocol-measured BCVA; it found that more than 90% of these cases involved clear pathology besides DME. “Since it will be more common to encounter eyes with multiple pathology in addition to DME in the real world, the use of such algorithms may be limited to a limited pathology for which it was trained,” they emphasized.

“The promise of this type of algorithm would be to offer potential gains in efficiency in both clinic and home settings, and the authors should be commended for pushing the boundaries in applying AI in eye care,” the commentary authors concluded. “However, before this potential can be realized, ongoing work will be needed in this and other AI studies for application in clinical care to ensure that the algorithm has the ability to recognize when predictions may be inaccurate, can perform well in a wide range of disease severities and produces consistent results among diverse populations.”