The following is a summary of “Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study,” published in the January 2023 issue of Neurodevelopmental Disorders by Farmer, et al.
Interventions for genetic conditions associated with neurodevelopmental disorder (GCAND) could focus on improving motor skills, cognitive abilities, social skills, and language. Various scores can reflect how well an individual performed on developmental concept assessments. Norm-referenced scores, developed for use in a clinical setting so that impairments can be identified in relation to age-based norms, can display striking floor effects when applied to people who already have substantial impairments. Available on many standardized exams, person ability scores are developed from Rasch analysis or item response theory and are meant to quantify individual growth. However, they have not been the focus of GCAND clinical trials or evaluated as such.
To objectively examine the power and type I error rate of operationalizing test performance using ability scores rather than norm-referenced scores, researchers simulated a series of parallel-arm clinical trials under multiple chronological age and impairment scenarios. They used the Vineland Adaptive Behavior Scales to illustrate how ability scores have more statistical weight than norm-referenced scores at severe impairment levels. Floor effects in norm-referenced scores likely had a role in this benefit.
Ability scores performed better than norm-referenced scores, but were more similar, for simulated settings when disability was less severe. Rates of type I error were very near to the standard deviation of 5% for both measurements. Based on these simulated data, it is clear that ability ratings have a significant advantage over norm-referenced scores for interpreting the outcomes of GCAND investigations that will include participants with severe disabilities. These findings would generalize to studies of developmental concepts irrespective of the underlying cause or the instrument used to measure them. Nonetheless, for tests with a higher floor than the Vineland, the relative benefit of ability scores is anticipated to be much more significant.
Source: jneurodevdisorders.biomedcentral.com/articles/10.1186/s11689-022-09474-6