Photo Credit: ttsz

The following is a summary of “Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease,” published in the April 2025 issue of Frontiers in Endocrinology by Cheng et al. 

Diabetic kidney disease (DKD), a common complication of diabetes mellitus and leading cause of end-stage renal disease (ESRD), prompted a growing need for early detection and intervention, with increasing attention given to the potential use of artificial intelligence (AI), particularly large language models (LLMs), in patient education.  

Researchers conducted a retrospective study to evaluate the quality of LLM-generated patient education materials (PEMs) for early DKD and assess their feasibility in patient education.  

They selected 4 LLMs—ERNIE Bot 4.0, GPT-4o, ChatGLM4, and ChatGPT-o1—to generate PEMs, the ERNIE Bot 4.0, GPT-4o, and ChatGLM4 produced 2 versions of PEMs: 1 based on American Diabetes Association (ADA) guidelines and 1 without. ChatGPT-o1 generated only a version without ADA guidelines. A PEM based on ADA guidelines was written by an experienced physician. All materials were evaluated using a Likert scale, assessing accuracy, completeness, safety, and patient comprehensibility, 7 medical experts, including nephrologists and endocrinologists, and 50 patients with diabetes evaluated the materials. Basic information from patient evaluators was recorded.  

The results showed that experts rated the PEMs from ERNIE Bot 4.0, GPT-4o, ChatGLM4, and ChatGPT-o1, as well as the physician-sourced PEM. The ERNIE Bot 4.0’s non-guideline PEM and the physician-sourced PEM were rated the highest. Patient evaluations indicated that ERNIE Bot 4.0’s non-guideline PEM performed similarly or slightly better than the physician-sourced PEM in comprehensibility, completeness, and safety. This non-guideline PEM was particularly preferred by patients with over 5 years of diabetes or proteinuria. Additionally, GPT-4o and ChatGLM4’s non-guideline PEMs outperformed their guideline-based versions.  

Investigators concluded that the LLM-sourced PEMs, particularly ERNIE Bot 4.0’s non-guideline PEM for early DKD, performed similarly to physician-sourced PEMs regarding accuracy, completeness, safety, and patient comprehensibility, demonstrating high feasibility and suggesting AI’s potential for wider applications in future patient education. 

Source: frontiersin.org/journals/endocrinology/articles/10.3389/fendo.2025.1559265/full