Variability in individual cognitive trajectories may be why cognitive results in most Alzheimer’s disease (AD) trials fell within the level of random differences that occur when no treatment effect is expected, a simulation study showed.
“Our simulation reflects a clinical trial in which there is no real treatment effect, and so the 95% range of group differences between our ’placebo’ and ’treatment’ groups reflect the possible range of scores that can be expected when there is no treatment effect,” wrote Roos Jutten, PhD, of Amsterdam UMC in the Netherlands, and co-authors, in Neurology. “This range of group differences can only be explained by heterogeneity between individuals in their cognitive decline.”
Jutten and colleagues randomized 302 participants in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) research database into “placebo” and “treatment” groups, though no participants actually were treated. They defined treatment effect as the group difference for three commonly used AD trial measures: Clinical Dementia Rating-sum of boxes (CDR-SB), Alzheimer’s Disease Assessment Scale–cognitive subscale-13 (ADAS-Cog) and Mini-Mental State Examination (MMSE) over 18 months.
All participants met inclusion criteria for EMERGE and ENGAGE, the pivotal phase III trials of recently FDA-approved aducanumab (Aduhelm) therapy. All had a diagnosis of mild cognitive impairment (MCI) or dementia, a positive PET amyloid scan, and scores of 24 or more on baseline MMSE.
A given simulation established group differences over 18 months for the three cognitive measures between “treatment” and “placebo” groups. The process was repeated with the cohort re-randomized and the 95% confidence interval of observed differences determined.
By test, the 95% ranges of 18-month group differences with no treatment effect were:
- CDR-SB −0.35 to +0.35.
- ADAS-Cog -1.00 to 1.00.
- MMSE -0.42 to +0.42.
“Comparing our simulation results with reported results from actual clinical trials revealed that these reported differences consistently fell within the 95% ranges of our simulation,” Jutten and colleagues noted.
“Almost all observed group differences showed improvement of treatment vs placebo as captured by negative group differences on the CDR-SB and ADAS-Cog and positive group differences on the MMSE, but most differences fell within the 95% ranges of our simulation for all outcome measures,” they added.
“Clinical meaningfulness has been an elusive concept in the world of AD therapeutics,” wrote Rachel Buckley, PhD, of Massachusetts General Hospital in Boston, and David Knopman, MD, of Mayo Clinic in Rochester, Minnesota, in an accompanying editorial.
“The description by Jutten et al. of the 95% confidence interval for random variation in slope of several widely used outcome measures provides a rational estimate of a floor for what clinically meaningful therapeutic benefit should look like. For instance, for the Clinical Dementia Rating Sum of Boxes, Jutten et al estimate a (treatment vs placebo) group benefit of >0.35 rating points over 18 months as a lower bound.”
In the simulation cohort, mean age was 73 with about 70% APOE4 carriers, 44% female participants, and an average education level of 16 years.
Comparisons with extant literature in addition to ENGAGE and EMERGE included the DAYBREAK-ALZ trial of lanabecestat, IDENTITY-2 trial of semagacestat, EXPEDITION-3 study of solanezumab, and phase III trials of bapineuzumab.
Only the EMERGE aducanumab trial showed worsening of its placebo group compared with aducanumab that was outside the lowest 95% percentile of group differences on all outcome measures, though IDENTITY-2 did show a CDR-SB finding unlikely to be chance, Jutten and colleagues noted.
“This suggests that, even though within some trials differences between placebo and treatment groups were statistically significant, the possibility cannot be excluded that those differences were actually due to oversampling of fast decliners in the placebo group or oversampling of slow decliners in the treated group,” they wrote.
Repeated simulations for separate risk factors associated with disease progression showed that while a positive APOE4 status and baseline abnormal total tau levels were associated with steeper group cognitive decline, it came with greater individual variability in progression.
“This resulted in even broader ranges of effect sizes in these high-risk groups on all outcome measures (e.g., ±0.70 points for the CDR-SB in those with baseline abnormal tau),” the researchers wrote.
“One strategy to overcome the problem of heterogeneity might thus be to further select on risk factors that are associated with rate of decline. However, when we tested this strategy, we indeed observed steeper decline in high-risk individuals at group level, but also more variability at subject level, as compared to low-risk individuals,” they noted.
“Consequently, only enrolling individuals at high risk of fast progression may not solve the issue of heterogeneity, but could increase heterogeneity, making it more difficult to capture a clinical effect on group level,” they added.
“Powering future clinical trials in persons with mild cognitive impairment and mild dementia who have elevated β-amyloid to detect treatment benefits that exceed 0.35 rating points on the Clinical Dementia Rating Sum of Boxes would be a logical approach according to the observations provided by Jutten et al.,” Buckley and Knopman noted. “To be sure, such an effect size might still be too small to be noticeable to patients and their family care partners.”
Limitations include simulation data from a highly educated community sample with little racial or ethnic diversity, limiting generalizability.
-
Variability in individual cognitive trajectories may be why cognitive results in most Alzheimer’s disease (AD) trials fell within the level of random differences that occur when no treatment effect is expected, a simulation study showed.
-
Differences between placebo and treatment groups that were statistically significant in some AD trials may have been due to oversampling of fast decliners in the placebo group or oversampling of slow decliners in the treated group.
Paul Smyth, MD, Contributing Writer, BreakingMED™
The Alzheimer Center Amsterdam is supported by Alzheimer Nederland and Stichting VUMC funds. Research of the Alzheimer Center Amsterdam is part of the neurodegeneration research program of Amsterdam Neuroscience.
Jutter reports no relevant disclosures.
Buckley reports no disclosures relevant to the manuscript. Knopman serves on a Data Safety Monitoring Board for the Dominantly Inherited Alzheimer Network (DIAN) study. He serves on a Data Safety Monitoring Board for Biogen but receives no personal compensation. He is an investigator in clinical trials sponsored by Biogen, Lilly Pharmaceuticals, and the University of Southern California. He serves as a consultant for Roche, Samus Therapeutics, Third Rock, and Alzeca Biosciences but receives no personal compensation.
Cat ID: 130
Topic ID: 82,130,730,130,33,361,192,925
