The researchers suggested that AI could one day help clinicians draft responses to a patient’s query, saving time and improving consistency. Photo: Getty Images.

Virtual healthcare has boomed since the pandemic, and along with it an increasing need for electronic communication between healthcare providers and patients. Artificial intelligence (AI) may help in this regard, since experts say that the current healthcare systems are ill-equipped to handle this surge in demand for electronic avenues of care and information. Because AI tools are increasingly being used to obtain information, researchers assessed ChatGPT’s capacity for generating accurate information on myopia. They reported in Ophthalmic and Physiological Optics that ChatGPT performed well overall, but a portion of the responses were inaccurate, leading the researchers to conclude that further evaluation is needed.

To test ChatGPT, the researchers generated a series of 11 questions about myopia to ask the AI model. Each question was entered five times into a fresh session. Question categories included a general summary of myopia as well as cause, symptoms, onset, prevention, complications, natural history, treatment and prognosis. Five ODs rated the quality of the responses.

ChatGPT’s 275 responses garnered a median score of 4 (on the 1-5 Likert scale) for 10 questions and acceptable responses (median 3) for one question. Overall, 24% of responses were very good, 49% were good, 22% were acceptable, 3.6% were poor and 1.8% were very poor.

ChatGPT was limited by its “(1) inability to critically analyze results from the literature, (2) knowledge database limited till 2021 (not updated), (3) misinterpretation of medical terms, (4) inability to differentiate between predatory and reputable journal articles, (5) lack of scientific accuracy and reliability, biased and potential misinformation for readers.”

The researchers pointed out in their paper that many patients today seek medical advice from the internet and social media, and as a result receive inaccurate responses. “Misinformation and misalignment between information gathered from the health or eye care practitioner and the internet may result in patient dissatisfaction,” they wrote.

“Despite the limitations of this study and [those] of ChatGPT, AI holds the promise to lower the cost of healthcare by reducing consultation time and provide valuable patient information,” they concluded in their paper. “This may reduce the patient load from the already burdened healthcare and eye care systems worldwide by enhancing patient information, thus refining the workflow and improving patient outcomes.”