Intraretinal fluid (white arrow), shown here in a patient with macular neovascularization, was one of the areas of widest disagreement among the experts who assessed OCT scans in the trial.

Intraretinal fluid (white arrow), shown here in a patient with macular neovascularization, was one of the areas of widest disagreement among the experts who assessed OCT scans in the trial. Photo: Jessica Haynes, OD, and Mohammad Rafieetary, OD. Click image to enlarge.

In a new study to appear in Eye, researchers assessed the amount of agreement, or lack thereof, for evaluating neovascular age-related macular degeneration (nAMD), diabetic macular edema (DME) and retinal vein occlusion (RVO). As seen by OCT-certified graders, seven different graders looked at SD-OCT volume scans from 356 eyes of 207 patients.

Included in the grading criteria were the presence of intra- and subretinal fluid, pigment epithelial detachment, epiretinal membrane, vitreomacular interface conditions, central retinal thickness at the foveal center-point and central millimeter, and height and location of intra- and subretinal fluid and pigment epithelial detachment.

Despite using an optimized setting and standardized image acquisition, the researchers found that disease-dependent variability still occurred in biomarker evaluation, being most pronounced for intraretinal fluid in nAMD and DME. However, for nAMD, DME and RVO, the graders did find a consistency for subretinal fluid presence.

Potentially contributing to the reason for the observed intraretinal fluid inconsistency is that intraretinal fluid was significantly more often missed when appearing with an absence of subretinal fluid. This was also missed if the mean retinal fluid volume and number of b-scans showing fluid were lower, as the authors of the study explain these results were found in previous research.

The authors noted that their consistent grading results of subretinal fluid are reasonably expected, since the feature affects a predefined anatomical space of the retina, thus showing less appearance variability. They do note, however, that complicating subretinal fluid grading in cases of nAMD could be association of hyperreflective material in the subretinal compartment. Another complication could be outer retinal degeneration.

Because this study did not include additional training or supervision typically seen in real-world scenarios, the authors of the study relate that “the limited reproducibility seen in our and previous studies raises the question whether OCT image assessment, as we know it, has reached its maximum potential. Despite the resources available to a [reading center], manual gradings remain laborious, and to a certain extent inconsistent and inefficient.”

One of these inconsistencies stems from an approach to image grading that only assesses qualitative aspects, like feature presence or absence, and two-dimensional parameters, like central subfield thickness or feature height, which lack the ability to capture the amount of available structural data generated from OCT imaging in its millions of pixels generated per scan. The authors suggest artificial intelligence-based algorithms could be a  promising tool to allow for more precise and objective evaluations. For example, retinal fluid that is detected by automation can also determine the subtype, location and volume if present.

As such, the researchers ended the paper by articulating their goal of focusing on “the adoption of automated imaging analysis tools for a more precise, efficient and objective image assessment. Furthermore, enhanced collaborations of different reading centers in large-scale clinical studies call for the harmonization and standardization of grading procedures not only within but between centers.”

Michl M, Neschi M, Kaider A, et al. A systemic evaluation of human expert agreement on optical coherence tomography biomarkers using multiple devices. Eye. December 28, 2022. [Epub ahead of print].