A tale of two studies

Results from two embedded clinical trials suggested different conclusions to researchers and editorialists about financial incentives for enrolling in studies.

In their analysis, Scott Halpern, MD, PhD, of the University of Pennsylvania in Philadelphia, and co-authors evaluated the effect of incentives on enrollment—the primary outcome—in two parent randomized controlled trials. To draw conclusions about whether incentives were either undue or unjust, they used a non-inferiority approach to assess whether the upper limit of the confidence intervals for observed interactions exceeded a prespecified non-inferiority margin, an interaction odds ratio of 2.0 with values of <2.0 implying non-inferiority.

“Each trial was powered to test the hypotheses that incentives served neither as undue inducements (based on the interaction between incentive size and perceived research risk, as measured using a 10-point scale, on the primary outcome), nor unjust inducements (based on the interaction between incentive size and participants’ self-reported income),” the researchers wrote in JAMA Internal Medicine.

Participants in parent trials were randomized to a range of incentives in a smoking cessation intervention among people with depression (n=654, 50% women, mean age 51, recruited September 2017 to August 2019) and a walking intervention for hospitalized patients (n=642, 57% women, mean age 47, recruited January 2018 through May 2019).

  • In the smoking trial, incentives increased the proportion of people who consented to participate. Participation rates by incentive were 22% for $0, 36% for $200, and 47% for $500, with an adjusted OR for each increase in incentive of 1.70 (95% CI 1.34-2.17, P<0.001)
  • In the walking trial, incentives did not significantly increase participation. In this study, participation rates by incentive were 45% for $0, 48% for $100, and 43% for $300, with an adjusted OR for each increase in incentive of 0.88 (95% CI 0.64-1.22, P=0.45).

The upper limits of the confidence intervals for observed interactions for undue inducement (interaction between two incentive sizes and perceived research risk) were 1.15 and 0.99. The upper limits of the confidence intervals for observed interactions for unjust inducement (interaction between two incentive sizes and participants’ self-reported income) were 1.21 and 1.26.

The results “provide real-world confirmation of hypothetical studies that also had not identified ethical problems with incentives for research participation,” Halpern and colleagues wrote. “Thus, research regulators should relax restrictions on the use of incentives that are designed to improve enrollment in low-risk trials.”

Nearly half the participants in the smoking trial perceived it to have risks, but fewer than 10% of participants in the ambulation trial perceived risks in their study. It’s likely the ambulation trial had high perceived benefit compared to risks, yielding higher baseline consent rates and fewer patients whose decisions could be altered by incentives, the researchers observed.

In addition, there were no significant effects of incentive size on the secondary outcomes in either trial, including time spent reviewing the risk sections of consent forms, perceived research risks, trial understanding, perceived coercion, or therapeutic misconceptions.

“This work is welcome, as it presents experimental data to a bioethical debate that so far has been largely driven by conjecture and competing suppositions,” noted Winston Chiong, MD, PhD, of University of California, San Francisco, and co-authors in an accompanying editorial.

“The authors regard their study as having settled the practical and normative debate, concluding, ’Thus, research regulators should relax restrictions on the use of incentives designed to improve enrollment in low-risk trials,’” they wrote. “However, interpreting the authors’ findings is complex and illustrates some of the challenges inherent to applying empirical data to ethical problems.”

“For instance, given the potential benefits of monetary incentives for clinical research participation, those who would limit their application may owe us an applicable criterion for what makes an inducement undue or unjust,” Chiong and co-editorialists continued. “Among bioethicists, there is no consensus about what counts as undue inducement or an unjust distribution of research burdens. In this article, the authors have operationalized these constructs based on their own interpretations of undue and unjust inducement, which may not capture all the concerns that scholars have raised about inducement.”

The editorialists’ concerns included the non-inferiority design of the study, which “may be unfamiliar to many bioethicists and can place substantial evaluative demands on readers,” they suggested.

The study design tested a null hypothesis that undue or unjust inducement was present and interpreted the rejection of this null as showing that undue or unjust inducement was not present, but “noninferiority designs do not show that there is no difference between conditions,” Chiong and co-authors argued.

“Instead, they help to evaluate whether one condition is not worse than another by more than some acceptably small margin,” they maintained. “The choice of this noninferiority margin is crucial: choosing too large a noninferiority margin can increase the risk of falsely claiming noninferiority. The authors note that there was no evidence to guide the choice of a noninferiority margin of 2.0 for the odds ratio of the interaction term.”

Chiong and co-editorialists also contrasted a prior concept of undue inducement with the researchers’ interpretation and questioned what extent of effect modification should be considered acceptably small. The work of “illustrates that translating evidence to ethical guidance is not straightforward,” they noted.

“Because there was no evidence to guide the choice of the noninferiority margin, we performed post hoc sensitivity analyses to determine the smallest margins at which undue and unjust inducement were ruled out,” Halpern and colleagues noted. “Similar conclusions that undue inducement was not present would have been reached using noninferiority margins as low as 1.6 and 1.8 in the smoking and ambulation trials, respectively.”

The researchers acknowledged that neither parent trial posed particularly high risks to participants. “Future tests of incentives of different sizes, and in the context of higher-risk parent trials, including trials that test treatments of serious illnesses, are warranted,” they wrote.

  1. Increasing monetary incentives increased participation in a smoking cessation trial, but not a trial of a walking intervention in hospitalized patients.

  2. Results from these two trials suggested different conclusions to researchers and editorialists about paying people to enroll in studies.

Paul Smyth, MD, Contributing Writer, BreakingMED™

This work was supported by the National Cancer Institute.

Halpern reported no disclosures.

Chiong reported research support from the National Institute of Mental Health and National Institute on Aging during the conduct of the study.

Cat ID: 148

Topic ID: 88,148,730,914,143,192,48,148,150,587,590,925

Author