Randomized trials are often regarded as the gold standard for resolving interventional queries and ensuring the dominant hierarchy of medical evidence. But whether the large treatment effect in a randomized trial should be considered an empirical marker for subsequent trials is not clear. This study aims to investigate the same.

This meta-epidemiological assessment of existing publishing data on randomized trials included a total of 2,008 reviews comprising of 85,002 forest plots; however, only 44 of those forest plots (0.05%) satisfied the inclusion criteria. The included plots contained an index randomized trial with a statistically significant very large effect (VLE). The primary outcome was the validation of the VLE.

The index trials in the included plots were small, with 99 being the median sample size. 20% of the trials were rated at low risk of bias. In the subsequent large trials in 43 of 44 cases, the relative risk was closer to null. The subsequent large trial data was not statistically significant. In cases where the large trial data was significant, the additional primary outcomes contradicted the results.

The research concluded that the very large effects followed by large trials were small and were of little to no statistical significance.

Ref: https://www.bmj.com/content/355/bmj.i5432