Not without Context-A Multiple Methods Study on Evaluation and Correction of Automated Brain Tumor Segmentations by Experts.

Nov 13, 2023

Contributors: Katharina V Hoebel,Christopher P Bridge,Albert Kim,Elizabeth R Gerstner,Ina K Ly,Francis Deng,Matthew N DeSalvo,Jorg Diettrich,Raymond Huang,Susie Y Huang,Stuart R Pomerantz,Saivenkat Vagvala,Bruce R Rosen,Jayashree Kalpathy-Cramer

Brain tumor segmentations are integral to the clinical management of patients with glioblastoma, the deadliest primary brain tumor in adults. The manual delineation of tumors is time-consuming and highly provider-dependent. These two problems must be addressed by introducing automated, deep-learning-based segmentation tools. This study aimed to identify criteria experts use to evaluate the quality of automatically generated segmentations and their thought processes as they correct them.
Multiple methods were used to develop a detailed understanding of the complex factors that shape experts’ perception of segmentation quality and their thought processes in correcting proposed segmentations. Data from a questionnaire and semistructured interview with neuro-oncologists and neuroradiologists were collected between August and December 2021 and analyzed using a combined deductive and inductive approach.
Brain tumors are highly complex and ambiguous segmentation targets. Therefore, physicians rely heavily on the given context related to the patient and clinical context in evaluating the quality and need to correct brain tumor segmentation. Most importantly, the intended clinical application determines the segmentation quality criteria and editing decisions. Physicians’ personal beliefs and preferences about the capabilities of AI algorithms and whether questionable areas should not be included are additional criteria influencing the perception of segmentation quality and appearance of an edited segmentation.
Our findings on experts’ perceptions of segmentation quality will allow the design of improved frameworks for expert-centered evaluation of brain tumor segmentation models. In particular, the knowledge presented here can inspire the development of brain tumor-specific metrics for segmentation model training and evaluation.

Copyright © 2023. Published by Elsevier Inc.

Author

admin

View all posts

ABOUT THE CONTRIBUTORS

Katharina V Hoebel,Christopher P Bridge,Albert Kim,Elizabeth R Gerstner,Ina K Ly,Francis Deng,Matthew N DeSalvo,Jorg Diettrich,Raymond Huang,Susie Y Huang,Stuart R Pomerantz,Saivenkat Vagvala,Bruce R Rosen,Jayashree Kalpathy-Cramer
Katharina V Hoebel

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.); Harvard-MIT Program in Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Massachusetts (K.V.H.).

Christopher P Bridge

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.); Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts (C.P.B., S.R.P.).

Albert Kim

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.); Stephen E. and Catherine Pappas Center for Neuro-Oncology, Massachusetts General Hospital, Boston, Massachusetts (A.K., E.R.G., I.K.L.).

Elizabeth R Gerstner

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.); Stephen E. and Catherine Pappas Center for Neuro-Oncology, Massachusetts General Hospital, Boston, Massachusetts (A.K., E.R.G., I.K.L.).

Ina K Ly

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.); Stephen E. and Catherine Pappas Center for Neuro-Oncology, Massachusetts General Hospital, Boston, Massachusetts (A.K., E.R.G., I.K.L.).

Francis Deng

Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, Massachusetts (F.D.).

Matthew N DeSalvo

Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts (M.N.D., R.H., S.V.).

Jorg Diettrich

Department of Neurology, Division of Neuro-Oncology, Massachusetts General Hospital Cancer Center, Harvard Medical School, Boston, Massachusetts (J.D.).

Raymond Huang

Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts (M.N.D., R.H., S.V.).

Susie Y Huang

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.).

Stuart R Pomerantz

Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts (C.P.B., S.R.P.).

Saivenkat Vagvala

Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts (M.N.D., R.H., S.V.).

Bruce R Rosen

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.).

Jayashree Kalpathy-Cramer

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, Massachusetts (K.V.H., C.P.B., A.K., E.R.G., I.K.L., S.Y.H., B.R.R., J.K.C.); Department of Ophthalmology, University of Colorado Anschutz Medical Campus, 1675 Aurora Court, Mail Stop F731, Aurora, CO (J.K.C.). Electronic address: JAYASHREE.KALPATHY-CRAMER@CUANSCHUTZ.EDU.