Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies

Eun-Young Mun, Yan Huo, Helene R. White, Sumihiro Suzuki, Jimmy de la Torre

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Many clinical and psychological constructs are conceptualized to have multivariate higher-order constructs that give rise to multidimensional lower-order traits. Although recent measurement models and computing algorithms can accommodate item response data with a higher-order structure, there are few measurement models and computing techniques that can be employed in the context of complex research synthesis, such as meta-analysis of individual participant data or integrative data analysis. The current study was aimed at modeling complex item responses that can arise when underlying domain-specific, lower-order traits are hierarchically related to multiple higher-order traits for individual participant data from multiple studies. We formulated a multi-group, multivariate higher-order item response theory (HO-IRT) model from a Bayesian perspective and developed a new Markov chain Monte Carlo (MCMC) algorithm to simultaneously estimate the (a) structural parameters of the first- and second-order latent traits across multiple groups and (b) item parameters of the model. Results from a simulation study support the feasibility of the MCMC algorithm. From the analysis of real data, we found that a bivariate HO-IRT model with different correlation/covariance structures for different studies fit the data best, compared to a univariate HO-IRT model or other alternate models with unreasonable assumptions (i.e., the same means and covariances across studies). Although more work is needed to further develop the method and to disseminate it, the multi-group multivariate HO-IRT model holds promise to derive a common metric for individual participant data from multiple studies in research synthesis studies for robust inference and for new discoveries.

Original languageEnglish
Article number328
JournalFrontiers in Psychology
Volume10
Issue numberJUN
DOIs
StatePublished - 1 Jan 2019

Fingerprint

Markov Chains
Feasibility Studies
Research
Meta-Analysis
Psychology

Keywords

  • Bayesian estimation
  • Higher-order IRT
  • Individual participant data
  • Meta-analysis
  • Multi-group IRT
  • Multivariate IRT

Cite this

Mun, Eun-Young ; Huo, Yan ; White, Helene R. ; Suzuki, Sumihiro ; de la Torre, Jimmy. / Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies. In: Frontiers in Psychology. 2019 ; Vol. 10, No. JUN.
@article{78839e69461944369fa5902a04c376cd,
title = "Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies",
abstract = "Many clinical and psychological constructs are conceptualized to have multivariate higher-order constructs that give rise to multidimensional lower-order traits. Although recent measurement models and computing algorithms can accommodate item response data with a higher-order structure, there are few measurement models and computing techniques that can be employed in the context of complex research synthesis, such as meta-analysis of individual participant data or integrative data analysis. The current study was aimed at modeling complex item responses that can arise when underlying domain-specific, lower-order traits are hierarchically related to multiple higher-order traits for individual participant data from multiple studies. We formulated a multi-group, multivariate higher-order item response theory (HO-IRT) model from a Bayesian perspective and developed a new Markov chain Monte Carlo (MCMC) algorithm to simultaneously estimate the (a) structural parameters of the first- and second-order latent traits across multiple groups and (b) item parameters of the model. Results from a simulation study support the feasibility of the MCMC algorithm. From the analysis of real data, we found that a bivariate HO-IRT model with different correlation/covariance structures for different studies fit the data best, compared to a univariate HO-IRT model or other alternate models with unreasonable assumptions (i.e., the same means and covariances across studies). Although more work is needed to further develop the method and to disseminate it, the multi-group multivariate HO-IRT model holds promise to derive a common metric for individual participant data from multiple studies in research synthesis studies for robust inference and for new discoveries.",
keywords = "Bayesian estimation, Higher-order IRT, Individual participant data, Meta-analysis, Multi-group IRT, Multivariate IRT",
author = "Eun-Young Mun and Yan Huo and White, {Helene R.} and Sumihiro Suzuki and {de la Torre}, Jimmy",
year = "2019",
month = "1",
day = "1",
doi = "10.3389/fpsyg.2019.01328",
language = "English",
volume = "10",
journal = "Frontiers in Psychology",
issn = "1664-1078",
number = "JUN",

}

Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies. / Mun, Eun-Young; Huo, Yan; White, Helene R.; Suzuki, Sumihiro; de la Torre, Jimmy.

In: Frontiers in Psychology, Vol. 10, No. JUN, 328, 01.01.2019.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies

AU - Mun, Eun-Young

AU - Huo, Yan

AU - White, Helene R.

AU - Suzuki, Sumihiro

AU - de la Torre, Jimmy

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Many clinical and psychological constructs are conceptualized to have multivariate higher-order constructs that give rise to multidimensional lower-order traits. Although recent measurement models and computing algorithms can accommodate item response data with a higher-order structure, there are few measurement models and computing techniques that can be employed in the context of complex research synthesis, such as meta-analysis of individual participant data or integrative data analysis. The current study was aimed at modeling complex item responses that can arise when underlying domain-specific, lower-order traits are hierarchically related to multiple higher-order traits for individual participant data from multiple studies. We formulated a multi-group, multivariate higher-order item response theory (HO-IRT) model from a Bayesian perspective and developed a new Markov chain Monte Carlo (MCMC) algorithm to simultaneously estimate the (a) structural parameters of the first- and second-order latent traits across multiple groups and (b) item parameters of the model. Results from a simulation study support the feasibility of the MCMC algorithm. From the analysis of real data, we found that a bivariate HO-IRT model with different correlation/covariance structures for different studies fit the data best, compared to a univariate HO-IRT model or other alternate models with unreasonable assumptions (i.e., the same means and covariances across studies). Although more work is needed to further develop the method and to disseminate it, the multi-group multivariate HO-IRT model holds promise to derive a common metric for individual participant data from multiple studies in research synthesis studies for robust inference and for new discoveries.

AB - Many clinical and psychological constructs are conceptualized to have multivariate higher-order constructs that give rise to multidimensional lower-order traits. Although recent measurement models and computing algorithms can accommodate item response data with a higher-order structure, there are few measurement models and computing techniques that can be employed in the context of complex research synthesis, such as meta-analysis of individual participant data or integrative data analysis. The current study was aimed at modeling complex item responses that can arise when underlying domain-specific, lower-order traits are hierarchically related to multiple higher-order traits for individual participant data from multiple studies. We formulated a multi-group, multivariate higher-order item response theory (HO-IRT) model from a Bayesian perspective and developed a new Markov chain Monte Carlo (MCMC) algorithm to simultaneously estimate the (a) structural parameters of the first- and second-order latent traits across multiple groups and (b) item parameters of the model. Results from a simulation study support the feasibility of the MCMC algorithm. From the analysis of real data, we found that a bivariate HO-IRT model with different correlation/covariance structures for different studies fit the data best, compared to a univariate HO-IRT model or other alternate models with unreasonable assumptions (i.e., the same means and covariances across studies). Although more work is needed to further develop the method and to disseminate it, the multi-group multivariate HO-IRT model holds promise to derive a common metric for individual participant data from multiple studies in research synthesis studies for robust inference and for new discoveries.

KW - Bayesian estimation

KW - Higher-order IRT

KW - Individual participant data

KW - Meta-analysis

KW - Multi-group IRT

KW - Multivariate IRT

UR - http://www.scopus.com/inward/record.url?scp=85068668440&partnerID=8YFLogxK

U2 - 10.3389/fpsyg.2019.01328

DO - 10.3389/fpsyg.2019.01328

M3 - Article

VL - 10

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

IS - JUN

M1 - 328

ER -