東華大學圖書館 |

Performance of Parametric Vs. Data Mining Methods for Estimating Propensity Scores with Multilevel Data: a Monte Carlo Study.

紀錄類型:	書目-電子資源 : Monograph/item
正題名/作者:	Performance of Parametric Vs. Data Mining Methods for Estimating Propensity Scores with Multilevel Data: a Monte Carlo Study./
作者:	Fan, Meng.
出版者:	Ann Arbor : ProQuest Dissertations & Theses, : 2020,
面頁冊數:	158 p.
附註:	Source: Dissertations Abstracts International, Volume: 82-05, Section: B.
Contained By:	Dissertations Abstracts International82-05B.
標題:	Educational tests & measurements. -
電子資源:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28091764
ISBN:	9798684669354

Performance of Parametric Vs. Data Mining Methods for Estimating Propensity Scores with Multilevel Data: a Monte Carlo Study.
Fan, Meng.

Performance of Parametric Vs. Data Mining Methods for Estimating Propensity Scores with Multilevel Data: a Monte Carlo Study. - Ann Arbor : ProQuest Dissertations & Theses, 2020 - 158 p.

Source: Dissertations Abstracts International, Volume: 82-05, Section: B.

Thesis (Ph.D.)--University of Delaware, 2020.

This item must not be sold to any third party vendors.

Randomized controlled trials (RCTs) or randomized experiments, have long been considered as the most rigorous method to determine whether causal effects exist between a treatment and an outcome, such as the effect of an educational intervention. However, RCTs are often infeasible due to practical or ethical reasons in educational settings. Under such circumstances, non-randomized observational studies are often used to estimate treatment effects. The propensity score is defined as the conditional probability of receiving treatment given a set of observed pretreatment variables. Under Rubin's causal model, the aim of conditioning on the propensity score is to improve the quality of estimates by attempting to mimic the balance between groups that occurs through the randomization process. Propensity score methods have been developed primarily for single-level data structures. In educational studies, data typically have a clustered or hierarchical structure, where probability of receiving treatment is a function of both individual and cluster-level factors. Using the Monte Carlo simulations, this dissertation aims to compare two tree-based data mining approaches (i.e., generalized boosting modeling [GBM], generalized linear mixed-effects model trees [GLMERTREE]) to two parametric models (i.e., multiple logistic regression [MLR], multilevel logistic regression [RC]) for propensity score estimation under different simulated settings. There are several primary findings in this study. First, hidden bias from unobserved covariates has a very large impact on the estimate of causal effects-missing covariates renders all PSA approaches invalid. Second, under conditions of non-additivity and non-linearity, the data mining approaches can provide better performance on predicting the propensity score. However, all of the four estimation methods with an appropriately specified outcome model can provide unbiased treatment effect estimates. Third, although the MLR and RC outcome models performed similarly on the relative bias of treatment effects, RC offers better precision by producing lower standard errors of treatment effects. Fourth, among the eight estimation and outcome model combinations, GBM-RC combination provided a more accurate and precise treatment effect estimates across the greatest number of simulated conditions. There are several limitations in this study. First, this study did not consider varied correlation between covariates. Future research can be done to incorporate varied correlations among covariates. Second, balanced cluster size scenarios were created in this study. It is worth exploring the effect of the imbalance on the estimation of treatment effect. Third, this study included only propensity score weighting as the conditioning method. Future research can assess the performance of data mining approaches to estimate the propensity score using matching and stratification conditioning methods. Fourth, when using GBM to generate the propensity score in this study, only one algorithm specification was specified. Further research should include different algorithm specifications for GBM with multilevel data.

ISBN: 9798684669354Subjects--Topical Terms:

3168483
Educational tests & measurements.
Subjects--Index Terms:

Data mining

Performance of Parametric Vs. Data Mining Methods for Estimating Propensity Scores with Multilevel Data: a Monte Carlo Study.
LDR:04447nmm a2200397 4500 001 2345854
005 20220613064758.5
008 241004s2020 ||||||||||||||||| ||eng d
020 $a 9798684669354
035 $a (MiAaPQ)AAI28091764
035 $a AAI28091764
040 $a MiAaPQ $c MiAaPQ
100 1 $a Fan, Meng. $3 3550125
245 1 0 $a Performance of Parametric Vs. Data Mining Methods for Estimating Propensity Scores with Multilevel Data: a Monte Carlo Study.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2020
300 $a 158 p.
500 $a Source: Dissertations Abstracts International, Volume: 82-05, Section: B.
500 $a Advisor: May, Henry.
502 $a Thesis (Ph.D.)--University of Delaware, 2020.
506 $a This item must not be sold to any third party vendors.
520 $a Randomized controlled trials (RCTs) or randomized experiments, have long been considered as the most rigorous method to determine whether causal effects exist between a treatment and an outcome, such as the effect of an educational intervention. However, RCTs are often infeasible due to practical or ethical reasons in educational settings. Under such circumstances, non-randomized observational studies are often used to estimate treatment effects. The propensity score is defined as the conditional probability of receiving treatment given a set of observed pretreatment variables. Under Rubin's causal model, the aim of conditioning on the propensity score is to improve the quality of estimates by attempting to mimic the balance between groups that occurs through the randomization process. Propensity score methods have been developed primarily for single-level data structures. In educational studies, data typically have a clustered or hierarchical structure, where probability of receiving treatment is a function of both individual and cluster-level factors. Using the Monte Carlo simulations, this dissertation aims to compare two tree-based data mining approaches (i.e., generalized boosting modeling [GBM], generalized linear mixed-effects model trees [GLMERTREE]) to two parametric models (i.e., multiple logistic regression [MLR], multilevel logistic regression [RC]) for propensity score estimation under different simulated settings. There are several primary findings in this study. First, hidden bias from unobserved covariates has a very large impact on the estimate of causal effects-missing covariates renders all PSA approaches invalid. Second, under conditions of non-additivity and non-linearity, the data mining approaches can provide better performance on predicting the propensity score. However, all of the four estimation methods with an appropriately specified outcome model can provide unbiased treatment effect estimates. Third, although the MLR and RC outcome models performed similarly on the relative bias of treatment effects, RC offers better precision by producing lower standard errors of treatment effects. Fourth, among the eight estimation and outcome model combinations, GBM-RC combination provided a more accurate and precise treatment effect estimates across the greatest number of simulated conditions. There are several limitations in this study. First, this study did not consider varied correlation between covariates. Future research can be done to incorporate varied correlations among covariates. Second, balanced cluster size scenarios were created in this study. It is worth exploring the effect of the imbalance on the estimation of treatment effect. Third, this study included only propensity score weighting as the conditioning method. Future research can assess the performance of data mining approaches to estimate the propensity score using matching and stratification conditioning methods. Fourth, when using GBM to generate the propensity score in this study, only one algorithm specification was specified. Further research should include different algorithm specifications for GBM with multilevel data.
590 $a School code: 0060.
650 4 $a Educational tests & measurements. $3 3168483
650 4 $a Statistics. $3 517247
650 4 $a Educational technology. $3 517670
650 4 $a Information science. $3 554358
653 $a Data mining
653 $a Monte Carlo simulations
653 $a Multilevel data
653 $a Propensity score analysis
653 $a Mathematics education
653 $a Student performance
690 $a 0288
690 $a 0723
690 $a 0710
690 $a 0463
710 2 $a University of Delaware. $b School of Education. $3 1022090
773 0 $t Dissertations Abstracts International $g 82-05B.
790 $a 0060
791 $a Ph.D.
792 $a 2020
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28091764