An artificial intelligence program that used machine learning to diagnosis acute or chronic middle ear infections outperformed standard otoscopy-based diagnostic performance, according to researchers.
Matthew G. Crowson, MD, MPA, MASc, FRCSC, Massachusetts Eye and Ear, Boston, trained a neural network to accurately predict the presence of otitis media in six out of seven cases, which was higher than otoscopy-based diagnostic performances previously reported. Crowson and colleagues suggested that the results of their study, reported in Pediatrics, “may generate value for patients, families, and the health care system by improving point-of-care diagnostic accuracy.”
Otitis media is among the most common childhood infections, with as many as 80% of all children experiencing at least one ear infection by the age of 3. The authors noted that while improved antibiotic strategies and the adoption of the pneumococcal conjugate vaccine have reduced the morbidity and mortality from otitis media, the disease process still places a tremendous burden on public health.
Furthermore, undiagnosed or misdiagnosed otitis media can result in hearing loss, delayed language development, and morbidity from extra- and intracranial complications. However, the authors pointed out that “a technique for the consistent and accurate diagnosis of otitis media, both acute and chronic, has eluded us,” and diagnostic accuracy has yet to “consistently surpass 70% for all providers across the full spectrum of primary care.”
For purposes of this study, Crowson and colleagues developed and trained an artificial intelligence algorithm designed to diagnose middle ear effusion in pediatric patients.
The data set for training the algorithm consisted of tympanic membrane images obtained from children taken to the operating room with the intent of performing myringotomy and possible tube placement. Using these images, the algorithm was trained to classify the tympanic membrane as either indicative of no middle ear effusion (normal) or the presence of effusion (abnormal) in pediatric patients.
Images were included in the modeling data set if they surpassed the authors’ quality criteria (i.e., that 75% of the surface area of the tympanic membrane was visible and was of sufficient resolution to identify major landmarks). The final image database consisted of 126 normal and 212 abnormal ear images.
“The mean training time for the neural network model was 76.0 (SD±0.01) seconds,” they wrote. “Our model approach achieved a mean image classification accuracy of 83.8% (95% confidence interval [CI]: 82.7–84.8). In support of this classification accuracy, the model produced an area under the receiver operating characteristic curve performance of 0.93 (95% CI: 0.91–0.94) and F1-score of 0.80 (95% CI: 0.77–0.82).”
“This diagnostic accuracy performance is considerably higher than human-expert otoscopy-based diagnostic performance,” the authors added.
In a commentary accompanying the study, Michael E. Pichichero, MD, Research Institute at Rochester General Hospital, Rochester, New York, acknowledged the limitations of otoscopy, even in the hands of an experienced clinician: “Most pediatricians are confident they are accurately diagnosing [otitis media] and distinguishing [acute otitis media, otitis media with effusion], or a retracted [tympanic membrane] without middle ear effusion. Indeed, the American Academy of Pediatrics guideline in 2004 that included ’diagnostic uncertainty’ as a consideration in therapy decisions was controversial. But when experienced pediatricians were shown videos of otoscopy that included still frame visualization and pneumatic otoscopy with no cerumen blocking the view, diagnostic accuracy varied from 70% for [acute otitis media] to ~45% for [otitis media with effusion] and retracted [tympanic membrane] without middle ear effusion. Pediatric residents performed similarly.”
So, what is the value of improving the accuracy of diagnosing ear infections? Crowson and colleagues suggested that enhanced diagnostic accuracy could provide the most value in cases where otitis media is actually not present. Ruling it out, for example, “may curtail antibiotic use, limit unnecessary referrals and time spent waiting for specialist consultations, decrease the number of operating room cases, reduce parental leave from work and life responsibilities, or some combination thereof,” they wrote.
However, expectations regarding the value of the machine learning model in this study should be tempered, considering it is not ready to be deployed in the clinic, wrote Crowson and colleagues. “Further work on scaling the technology and adapting for clinic, home, and/or parent-use will be important milestones for realizing the benefits of this approach.”
While a better diagnosis may lead to less antibiotic use and less referral for tubes, Pichichero noted that several questions need to be addressed. For example, will any new technology – including machine learning and AI – be effective if 75% of the tympanic membrane needs to be visualized? Or, how long will an uncooperative child sit still for such an examination? And will the technology be cost-effective if it costs more than a prescription for amoxycillin?
“Will it be necessary to have the technology readout imported to the electronic medical record and become the ground truth for diagnosis such that a diagnosis of [acute otitis media] with associated antibiotic prescription or referral for tubes due to prolonged [otitis media with effusion] needs to be justified when the clinician action contradicts the technology recommendation?” he asked.
-
A machine learning model outperformed human-expert otoscopy-based screening in diagnosing middle ear infection in pediatric patients.
-
The study authors noted that expectations regarding the value of the machine learning model in this study should be tempered, considering it is not ready to be deployed in the clinic.
Michael Bassett, Contributing Writer, BreakingMED™
The study authors had no relevant relationships to disclose.
Pichichero is a member of the clnical advisory board of PhotoniCare, Inc., which has developed optical coherence tomography for application in otitis media diagnosis.
Cat ID: 138
Topic ID: 85,138,730,138,192,925
