In educational research, a randomized controlled trial is the best design to eliminate potential selection bias in a sample to support valid causal inferences, but it is not always possible in educational research because of financial, ethical, and logistical constrains. One alternative solution is use of the propensity score (PS) methods. However, the bias and variance of the estimated causal effect can depend strongly on which covariates are included in the PS model of assignment to treatment. This study uses two simulated examples to understand how inclusion or exclusion of a classical or reciprocal suppressor, improving the in the regression model, affect the estimations of causal effect by using regression, PS as a covariate, PS weighting and PS matching methods. An additional condition of adding different covariates, P's, is also tested in all methods where P's explain the variance of outcome in different levels to approximate unconfoundedness. Findings indicate that both classical and reciprocal suppressors increase the predictive power of the treatment effects and influence the estimations of the treatment effects regardless in regression or PS methods without controlling any P. Although the impacts of the suppressors vary by different types of models applied, the strong enough covariates, P's, can eliminate the impact of suppressors in all models. With the stronger P's applied, the estimates of standard error only decline by using the regression models, but are quite consistent in the example of classical suppression and slightly increase in the example of reciprocal suppressions by using the PS models.
Read
