Cookie-theft task shows subtle decline before impairment is obvious

Written responses to a simple descriptive task showed subtle signs of cognitive decline before impairment was clinically evident, data from Framingham Heart Study participants suggested.

“Our results demonstrate that it is possible to predict future onset of Alzheimer’s disease using language samples obtained from cognitively normal individuals,” reported Elif Eyigoz, PhD, of IBM Research in Yorktown Heights, New York, and co-authors in Lancet EClinicalMedicine.

The task involved written descriptions of a kitchen scene with three characters, an overflowing sink, and other structures, and ongoing actions including a boy on a stool reaching into a cookie jar (the Cookie Theft Task). Eyigoz and colleagues studied language samples available every 5 or 6 years beginning in 1999 from the prospective Framingham Heart Study.

The research team developed a model that showed whether cognitively normal older adults would develop mild cognitive impairment due to Alzheimer’s by age 85. Using linguistic variables, the model showed which adults would develop Alzheimer’s with an area under the receiver operating curve (AUC) of 0.74 and an accuracy of 70%.

Future onset of Alzheimer’s dementia was associated with telegraphic speech, repetitiveness, and agraphia, including lack of punctuation and misspellings, the researchers reported. Future Alzheimer’s was negatively associated with mention of details in the picture, such as the dishes and dishcloth.

The study results are “among the first to show predictive value of language samples well before the onset of dementia, while its limitations point the way for future work,” noted Jed Meltzer, PhD, of University of Toronto, in an accompanying editorial.

“Dementia onset before age 85 could be identified with 70-75% accuracy,” he wrote. “This is well above chance performance, and slightly below the best performance seen in studies comparing current dementia patients with controls.”

“The accuracy rates of 70-75% are encouraging but not yet satisfactory for a realistic clinical tool, but this is likely to be an inevitable consequence of the limited language samples available,” he added.

Language samples collected in another well-known study published in 1996, the Nun Study, also suggested relationships between early language skills and later cognitive function. At a mean age of 22, nuns wrote autobiographies; they were reassessed at ages 75-95. Results suggested that low early-life linguistic ability (low idea density and low grammatical complexity) predicted poor cognitive function and dementia in late life.

Cookie-theft language samples have been studied in other language disorders including aphasia as well as dementia. A recent Canadian study reported sensitivity of 81% for discriminating controls from those with dementia using cookie-theft description analysis.

“Despite these advances, the predictive power of language samples is largely unproven, given that very few studies have been able to examine participants years before an eventual diagnosis of Alzheimer’s disease, to compare the language output of those who do and do not go on to develop the disease,” Meltzer noted.

In their study, Eyigoz and colleagues defined the onset of Alzheimer’s as the onset of mild cognitive impairment in a participant who later received a diagnosis of Alzheimer’s disease. They looked at three groups in the Framingham cohort: cases (Alzheimer’s patients who developed mild cognitive impairment at 85 or younger), normal aging (people who were dementia-free at 85 or older), and controls (combined normal aging and those with dementia but onset after 85).

“According to this definition, all cases have already developed cognitive impairment due to Alzheimer’s at 85, and none of the controls have developed cognitive impairment due to Alzheimer’s at 85,” the researchers said.

Their analysis included a machine learning approach. Language samples from Framingham participants were divided into a training set (samples from 63 cases and 132 controls) and a test set (40 cases, 40 controls with balanced age, education, and sex).

Half of the participants in the test set developed Alzheimer’s symptoms before age 85; the other half did not. All samples in the test set were collected during the cognitively normal period, and the mean time to diagnosis of mild Alzheimer’s was 7.59 years.

A limitation of the study is that language samples were written only. “Although written samples have a history of predictive value, speech is a more natural form of communication giving access to several important quantitative variables, especially those related to the ease of word finding, including overall speech rate and pausing,” Meltzer observed.

“It remains to be seen how high the predictive accuracy of language samples can be pushed given more extensive data sources, and it is important that such data be collected. The most advanced machine learning algorithms cannot overcome the limitations of sparse input,” he added.

“Fortunately, collection of speech data is simple to implement and simple to include within a larger natural history study like the Framingham Heart Study,” he said. “A number of longitudinal health studies now include detailed speech measures and can be expected to yield new insights into the earliest stages of the evolution of dementia.”

  1. Written responses to a simple descriptive task showed subtle signs of cognitive decline before impairment was clinically evident, data from Framingham Heart Study participants suggested.

  2. A model using linguistic variables from the task showed which adults would develop Alzheimer’s with an area under the receiver operating curve (AUC) of 0.74 and an accuracy of 70%.

Paul Smyth, MD, Contributing Writer, BreakingMED™

Pfizer, Inc. provided funding to obtain data from the Framingham Heart Study Consortium, and to support the involvement of IBM Research in the initial phase of the study. The data used in this study was supported by Framingham Heart Study’s National Heart, Lung, and Blood Institute, and by grants from the National Institute on Aging and the National Institute of Neurological Disorders and Stroke.

Eyigoz is a salaried employee of IBM.

Meltzer is a shareholder and advisor of Winterlight Labs.

Cat ID: 33

Topic ID: 82,33,282,485,494,33,192,255,925