Improving clinical trials with propensity score weighting
In an effort to highlight new and exciting work related to the NSRR, we are featuring guest blog posts from authors who have recently published using NSRR data. The following references:
Please contact support@sleepdata.org if you have new work you would like to share in a guest blog post.
Introduction
Propensity score weighting is a widely used metric for analyzing observational studies for treatment comparisons. However, it can also be used for covariate-adjusted analysis of randomized clinical trials. A recent study has introduced a class of propensity score weighting estimators for analyzing randomized trials, and developed an overlap weighting estimator that effectively removes chance imbalance and improves study power. The BestAIR dataset from NSRR was used to demonstrate the application of the overlap weighting method and its comparisons to other analytical approaches.
What is the problem being addressed?
Chance imbalance in baseline characteristics is common in randomized clinical trials; it not only affects the face validity, but also has implications for statistical power. While outcome regression is a common practice for covariate adjustment, it has the potential to for inviting a “fishing expedition”, because one may search for an outcome model that gives the most dramatic treatment effect estimate which jeopardizes the objectivity of causal inference. We address the problem of covariate adjustment in randomized trials by developing propensity score weighting estimators as objective alternative approaches.
What was the approach to solving the problem?
We demonstrate that a larger family of balancing weighting estimators can be used for covariate adjustment in randomized trials and improve statistical power over the unadjusted estimator. This family of estimators include the inverse probability weighting (IPW) estimator and the overlap weighting (OW) estimator as special cases, the latter of which has only been previously used in analyzing observational studies. The overlap weights assign to each unit a weight proportional to their propensity score to receive the other treatment, and emphasizes optimal internal validity. We further show that the OW estimator is consistently more efficient than the IPW estimator in finite-samples, and is the preferred propensity score-based approach for covariate adjustment.
What NSRR data were used?
We used the Best Apnea Interventions for Research (BestAIR) trial for an illustration of the proposed propensity score weighting estimators. The data set for illustration includes 169 participants with 83 patients in the active CPAP group and 86 patients in the combined control arm. A set of patient-level covariates were measured at baseline and outcomes were measured at baseline, 6 months, and 12 months.
What were the results?
The simulation results confirm that the OW estimator is consistently more efficient than IPW (leading to a smaller variance and higher power) than the IPW estimator. The OW estimator is also more robust to model misspecification than IPW and outcome regression estimators. The data analysis results suggest that CPAP intervention leads to lower systolic blood pressure, and Epworth Sleepiness score; however, only the change in Epworth Sleepiness score is statistically significant at the 0.05 level after covariate adjustment.
What were the conclusions and implications of this work?
While the overlap weighting (OW) estimator has been previously used in addressing complexities with observational studies, we provided a rigorous justification of the OW approach for the analysis of randomized trials. In randomized clinical trials, OW can lead to a smaller variance (and higher power) of the treatment effect than IPW, and is more robust to model misspecification than the outcome regression estimator.
Are any tools available?
We have developed an open-source R package "PSweight" that can implement the proposed propensity score weighting estimator. The propensity score weighting estimators can be used in analyzing both randomized trials and observational studies.
Paper Summary
Chance imbalance in baseline characteristics is common in randomized clinical trials. Regression adjustment such as the analysis of covariance (ANCOVA) is often used to account for imbalance and increase precision of the treatment effect estimate. An objective alternative is through inverse probability weighting (IPW) of the propensity scores. Although IPW and ANCOVA are asymptotically equivalent, the former may demonstrate inferior performance in finite samples. In this article, we point out that IPW is a special case of the general class of balancing weights, and advocate to use overlap weighting (OW) for covariate adjustment. The OW method has a unique advantage of completely removing chance imbalance when the propensity score is estimated by logistic regression. We show that the OW estimator attains the same semiparametric variance lower bound as the most efficient ANCOVA estimator and the IPW estimator for a continuous outcome, and derive closed-form variance estimators for OW when estimating additive and ratio estimates. Through extensive simulations, we demonstrate OW consistently outperforms IPW in finite samples and improves the efficiency over ANCOVA and augmented IPW when the degree of treatment effect heterogeneity is moderate or when the outcome model is incorrectly specified. We apply the proposed OW estimator to the Best Apnea Interventions for Research (BestAIR) randomized trial to evaluate the effect of continuous positive airway pressure on patient health outcomes. All the discussed propensity score weighting methods are implemented in the R package PSweight.
Citation
The NSRR thanks guest blogger Fan Li for submitting this article!