ISSN 0439-755X
CN 11-1911/B
主办:中国心理学会
   中国科学院心理研究所
出版:科学出版社

心理学报 ›› 2020, Vol. 52 ›› Issue (9): 1132-1142.doi: 10.3724/SP.J.1041.2020.01132

• 研究报告 • 上一篇    

多维对数正态作答时间模型:对潜在加工速度多维性的探究

詹沛达1(), Hong Jiao2, Kaiwen Man3   

  1. 1浙江师范大学教师教育学院心理学系, 金华 321004
    2Measurement, Statistics, and Evaluation, Department of Human Development and Quantitative Methodology, University of Maryland, College Park, Maryland, United States
    3Educational Studies in Psychology, Research Methodology, and Counseling, The University of Alabama, Tuscaloosa, United States
  • 收稿日期:2020-03-02 出版日期:2020-09-25 发布日期:2020-07-24
  • 通讯作者: 詹沛达 E-mail:pdzhan@gmail.com
  • 基金资助:
    * 国家自然科学基金青年基金项目(31900795)

The multidimensional log-normal response time model: An exploration of the multidimensionality of latent processing speed

ZHAN Peida1(), Hong JIAO2, Kaiwen MAN3   

  1. 1Department of Psychology, College of Teacher Education, Zhejiang Normal University, Jinhua, 321004, China
    2Measurement, Statistics, and Evaluation, Department of Human Development and Quantitative Methodology, University of Maryland, College Park, Maryland, United States
    3Educational Studies in Psychology, Research Methodology, and Counseling, The University of Alabama, Tuscaloosa, United States
  • Received:2020-03-02 Online:2020-09-25 Published:2020-07-24
  • Contact: ZHAN Peida E-mail:pdzhan@gmail.com

摘要:

在心理与教育测量中, 潜在加工速度反映学生运用潜在能力解决问题的效率。为在多维测验中探究潜在加工速度的多维性并实现参数估计, 本研究提出多维对数正态作答时间模型。实证数据分析及模拟研究结果表明:(1)潜在加工速度具有与潜在能力相匹配的多维结构; (2)新模型可精确估计个体水平的多维潜在加工速度及与作答时间有关的题目参数; (3)冗余指定潜在加工速度具有多维性带来的负面影响低于忽略其多维性所带来的。

关键词: 题目作答时间, 多维潜在加工速度, 题目作答理论, 计算机化测验, PISA

Abstract:

With the popularity of computer-based testings, the collection of item response times (RTs) and other process data has become a routine in large- and small-scale psychological and educational assessments. RTs not only provide information about the processing speed of respondents but also could be utilized to improve the measurement accuracy because the RTs are considered to convey a more synoptic depiction of the participants’ performance beyond responses alone. In multidimensional assessments, various skills are often required to answer questions. The speed at which persons were applying a set of skills reflecting distinct cognitive dimensions could be considered as multidimensional as well. In other words, each latent ability was measured simultaneously with its corresponding working efficiency of applying a facet of skills in a multidimensional test. For example, the latent speed corresponding to the latent ability of decoding of an algebra question may differ from encoding. Therefore, a multidimensional RT model is needed to accommodate this scenario, which extends various currently proposed RT models assuming unidimensional processing speed.

To model the multidimensional structure of the latent processing speed, this study proposed a multidimensional log-normal response time model (MLRT) model, which is an extension of the unidimensional log-normal response time model (ULRTM) proposed by van der Linden (2006). Model parameters were estimated via the full Bayesian approach with the Markov chain Monte Carlo (MCMC). A PISA 2012 computer-based mathematics RT dataset was analyzed as a real data example. This dataset contains RTs of 1581 participants for 9 items. A Q-matrix (see Table 1) was prespecified based on the PISA 2012 mathematics assessment framework (see Zhan, Jiao, Liao, 2018); three dimensions were defined based on the mathematical content knowledge, which are: 1) change and relationships (θ1), 2) space and shape (θ2), and, 3) uncertainty and data (θ3). One thing to note is that the defined Q-matrix served as a bridge to link items to the corresponding latent abilities, which shows the multidimensional structure of latent abilities. First, exploratory factor analysis (EFA) was conducted with the real dataset to manifest the multidimensional structure of the processing speed. Second, two RT models, i.e., the ULRTM and the MLRTM, were fitted to the data, and the results were compared. Third, a simulation study was conducted to evaluate the psychometric properties of the proposed model.

The results of the EFA indicated that the latent processing speed has a three-dimensional structure, which matches with the theoretical multidimensional structure of the latent abilities (i.e., the Q-matrix in Table 1). Furthermore, the ULRTM and the MLRTM yield adequate model data fits according to the posterior predictive model checking values (ppp = 0.597 for the ULRTM and ppp = 0.633 for the MLRTM). Furthermore, by comparing the values of the -2LL, DIC, and WAIC across the ULRTM and the MLRTM, the results indicate that the MLRTM fits the data better. In addition, the results show that (1) the correlations among three dimensions vary from medium to large (from 0.751 to 0.855); (2) the time-intensity parameters estimates of the two models were similar to each other. However, in terms of the time-discrimination parameters, the estimates of the ULRTM were slightly lower than the MLRTM. Moreover, the results from the simulation study show: 1) the model parameters were fully recovered with the Bayesian MCMC estimation algorithm; 2) the item time-discrimination parameter could be underestimated if the multidimensionality of the latent processing speed gets ignored, which meets our expectation, whereas the item time-intensity parameter stayed the same.

Overall, the proposed MLRTM performed well with the empirical data and was verified by the simulation study. In addition, the proposed model could facilitate practitioners in the use of the RT data to understand participants’ complex behavioral characteristics.

Key words: item response times, multidimensional latent processing speed, item response theory, computer-based testing, PISA

中图分类号: