Please wait a minute...
Acta Psychologica Sinica    2018, Vol. 50 Issue (1) : 130-140     DOI: 10.3724/SP.J.1041.2018.00130
|
 Optimization of test design for examinees with different cognitive structures
 PENG Yafeng1; LUO Zhaosheng1; LI Yujun2; GAO Chunlei1
 (1 School of psychology, Jiangxi Normal University, Nanchang 330022, China) (2 Center for Studies of Psychological Application/School of Psychology, South China Normal University, Guangzhou 510631, China)
Download: PDF(2101 KB)   Review File (1 KB) 
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks     Supporting Info
Guide   
Abstract   Doctors have to use different medical technologies to diagnose different kinds of illness effectively. Similarly, teachers have to use well designed tests to provide an accurate evaluation of students with different cognitive structures. To provide such an evaluation, we recommend to adopt the Cognitive Diagnostic Assessment (CDA). CDA could measure specific cognitive structures and processing skills of students so as to provide information about their cognitive strengths and weaknesses. In general, the typical design procedure of a CDA test is as follow: firstly, identify the target attributes and their hierarchical relationships; secondly, design a Q matrix (which characterizes the design of test construct and content); finally, construct test items. Within that designing framework, two forms of test are available: the traditional test and the computerized adaptive test (CAT). The former is a kind of test that has a fixed-structure for all participants with different cognitive structures, the latter is tailored to each participant’s cognitive structure. Researchers have not, however, considered the specific test design for different cognitive structures when using these two test forms. As a result, the traditional test requires more items to gain a precise evaluation of a group of participants with mixed cognitive structures, and a cognitive diagnosis computer adaptive test (CD-CAT) has low efficiency of the item bank usage due to the problems in assembling a particular item bank. The key to overcome these hurdles is to explore the appropriate design tailored for participants with different cognitive structures. As discussed above, a reasonable diagnosis test should be specific for the cognitive structure of target examinees so to perform classification precisely and efficiently. This is in line with CAT. In CAT, an ideal item bank serves as a cornerstone in achieving this purpose. In this regard, Reckase (2003, 2007 & 2010) came up with an approach named p-optimality in designing an optimal item bank. Inspired by the p-optimality and working according to the characteristics of CDA, we proposed a method to design the test for different cognitive structures. We conducted a Monte Carlo simulation study to explore the different test design modes for different cognitive structures under six attribute hierarchical structures (Linear, Convergent, Divergent, Unstructured, Independent and Mixture). The results show that: (1) the optimal test design modes for different cognitive structures are different under the same hierarchical structure in test length, initial exploration stage (Stage 0), accurately estimation stage (Stage 1); (2) the item bank for cognitive diagnosis computer adaptive test (CD-CAT) we built, according to the different cognitive structures’ optimal test design modes, has a superior performance on item pool usage than other commonly used item banks no matter whether the fixed-length test or the variable-length test is used. We provide suggestions for item bank assembling basing on results from these experiments.
Keywords  cognitive structure      test design      item bank design     
ZTFLH:     
  B841  
Fund: 
Corresponding Authors: LUO Zhaosheng1, E-mail: luozs@126.com    
Issue Date: 28 November 2017
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
PENG Yafeng
LUO Zhaosheng
LI Yujun
GAO Chunlei
Cite this article:   
PENG Yafeng,LUO Zhaosheng,LI Yujun, et al.  Optimization of test design for examinees with different cognitive structures[J]. Acta Psychologica Sinica, 2018, 50(1): 130-140.
URL:  
http://journal.psych.ac.cn/xlxb/EN/10.3724/SP.J.1041.2018.00130     OR     http://journal.psych.ac.cn/xlxb/EN/Y2018/V50/I1/130
[1] CHEN Airui, WANG Aijun, WANG Tianqi, TANG Xiaoyu, ZHANG Ming.  The primary visual cortex modulates attention oscillation[J]. Acta Psychologica Sinica, 2018, 50(2): 158-167.
[2] ZHANG Xiuling, HOU Yanan, ZHANG Fuxu, MEI Songli, KANG Jingmei.  The adaptation to Kanizsa-type illusory contours[J]. Acta Psychologica Sinica, 2018, 50(2): 168-175.
[3] CHEN Guangyao, HE Xianyou, LIU Tao.  The processing of negative sentences in different semantic context[J]. Acta Psychologica Sinica, 2018, 50(2): 186-196.
[4] CHENG Yahua, WU Xinchun, LIU Hongyun, LI Hong.  The developmental trajectories of oral vocabulary knowledge and its influential factors in Chinese primary school students[J]. Acta Psychologica Sinica, 2018, 50(2): 206-215.
[5] WANG Jing, XUE Chengbo, LIU Qiang.  Storage mechanism of same-dimension features in visual working memory[J]. Acta Psychologica Sinica, 2018, 50(2): 176-185.
[6] HUANG Feng, DING Qian, WEI Hua, HONG Jianzhong.  Effects of post thematic characteristics on knowledge sharing in the virtual community: The bystander effect perspective[J]. Acta Psychologica Sinica, 2018, 50(2): 226-234.
[7] TAO Aihua, LIU Yonghe, WANG Pei.  Moderating effects of conflict types on disappointment in interpersonal conflict[J]. Acta Psychologica Sinica, 2018, 50(2): 235-242.
[8] MA Jie, DOU Haoran, ZHUANG Qian, YIN Dongxue, LEI Ming, GAO Chao, ZHANG Yin, LIU Qiang, ZHAO Guang.  The mechanisms of contextual cuing effect based on objects’ topological properties[J]. Acta Psychologica Sinica, 2018, 50(2): 143-157.
[9] PENG Jian, WANG Zhen.  Being a prototypic follower: Burdening or enabling? The paradoxical effect of followership prototype-trait match[J]. Acta Psychologica Sinica, 2018, 50(2): 216-225.
[10] WEI Qingwang, LI Muzi, CHEN Xiaochen.  Social class and social perception: Is warmth or competence more important?[J]. Acta Psychologica Sinica, 2018, 50(2): 243-252.
[11] TIAN Lumei, YUAN Jingchi, LI Yongmei.  Effects of peer presence and self-esteem on adolescent risk-taking behavior: Evidence from an ERP study[J]. Acta Psychologica Sinica, 2018, 50(1): 47-57.
[12] LI BaoZhu, WEI ShaoMu.  The effect of presentation format for consumer responses: Evidence from eye movements[J]. Acta Psychologica Sinica, 2018, 50(1): 69-81.
[13] LIU Min, ZHANG Qinlin, YU Wei, ZHANG Hua.  Preliminary study on creative thinking mechanism of market information integration[J]. Acta Psychologica Sinica, 2018, 50(1): 82-90.
[14] LIANG Yongyi, YAN Ming, CHU Xiaoping.  The double-edge sword effects of leader group prototypicality in the multi-team context[J]. Acta Psychologica Sinica, 2018, 50(1): 58-68.
[15] WANG Yan, QIAN Xiaoyun, TIAN Qian, GAO Jun, LI Xiaoxu.  Parent-child relationship and the comparison between parents and their children on their children’s mate preference[J]. Acta Psychologica Sinica, 2018, 50(1): 91-100.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech