Methodological quality and homogeneity influenced agreement between randomized trials and nonrandomized studies of the same intervention for back pain

OBJECTIVE: To determine the influence of methodological quality and homogeneity on the agreement between pairs of randomized trials (RCTs) and nonrandomized studies (NRSs) of the same interventions for low-back problems. Homogeneity was assessed regarding settings, population, interventions, and outcomes. STUDY DESIGN AND SETTING: We searched Cochrane Central Register of Controlled Trials, MEDLINE, and EMBASE up to May 2005 for matching pairs of NRS and RCT. Analyses were done using correlation, linear and logistic regression. RESULTS: Forty-eight matched pairs were included with no significant overall correlation between effect sizes (r=0.09). There was a trend showing more agreement among the 22 pairs with higher methodological quality (r=0.33). The correlation among the 20 very homogeneous pairs was 0.59, and among the 28 heterogeneous pairs was -0.09. The agreement of authors' recommendations was influenced by the pair's homogeneity (odds ratio [OR]=2.78, 95% CI=1.44-5.37) rather than by methodological quality of the NRS (OR=0.93, 95% CI=0.67-1.29) or the RCT (OR=1.03, 95% CI=0.73-1.45). CONCLUSIONS: Pairs of low-quality studies disagreed more than pairs where at least one study was of high quality. However, pairs with similar settings, population, interventions, and outcomes showed higher agreement than pairs that were not as homogeneous