Analysis of learning effect in interactive problem-solving test: Extension and application of process data measurement model

doi:10.3724/SP.J.1041.2025.1677

Abstract

Cite this article

LU Xiangyu, CHEN Ping. (2025). Analysis of learning effect in interactive problem-solving test: Extension and application of process data measurement model. Acta Psychologica Sinica, 57(9), 1677-1688.

share this article

0
/ / Recommend

Add to citation manager EndNote|Ris|BibTeX

URL: https://journal.psych.ac.cn/acps/EN/10.3724/SP.J.1041.2025.1677

https://journal.psych.ac.cn/acps/EN/Y2025/V57/I9/1677

Figures/Tables 6

Table 1 The model fit of the three models to the empirical data

Models	LOO	WAIC	PPP
1P-ASM	21647.9	21626.5	0.520
1P-ASM-R	21637.7	21616.4	0.496
1P-ASM-R*	21595.2	21573.5	0.507

Table 1 The model fit of the three models to the empirical data

Models	LOO	WAIC	PPP
1P-ASM	21647.9	21626.5	0.520
1P-ASM-R	21637.7	21616.4	0.496
1P-ASM-R*	21595.2	21573.5	0.507

Table 2 Posterior estimates of the learning effect parameter in the 1P-ASM-R and 1P-ASM-R* models

Models	Mean	SD	95%HPDL	95%HPDU
1P-ASM-R	0.010	0.004	0.002	0.018
1P-ASM-R*	0.158	0.029	0.101	0.214

Table 2 Posterior estimates of the learning effect parameter in the 1P-ASM-R and 1P-ASM-R* models

Models	Mean	SD	95%HPDL	95%HPDU
1P-ASM-R	0.010	0.004	0.002	0.018
1P-ASM-R*	0.158	0.029	0.101	0.214

Table 3 Posterior estimates of the easiness parameter in the 1P-ASM and 1P-ASM-R* models

State	1P-ASM				1P-ASM-R*
State	Mean	SD	95%HPDL	95%HPDU	Mean	SD	95%HPDL	95%HPDU
A	0.872	0.033	0.809	0.935	0.667	0.049	0.568	0.764
B	1.606	0.048	1.511	1.699	1.331	0.068	1.198	1.459
C	1.415	0.052	1.316	1.515	1.093	0.076	0.947	1.244
D	1.452	0.059	1.339	1.567	1.089	0.088	0.916	1.266
E	1.940	0.074	1.800	2.088	1.536	0.101	1.336	1.733
F	0.329	0.118	0.099	0.553	-0.086	0.138	-0.353	0.180
G	-1.721	0.078	-1.875	-1.571	-1.952	0.091	-2.130	-1.773
H	-2.086	0.083	-2.252	-1.927	-2.375	0.100	-2.571	-2.182
I	-0.860	0.053	-0.964	-0.757	-1.201	0.082	-1.364	-1.047
J	-0.398	0.111	-0.617	-0.178	-0.798	0.130	-1.056	-0.544

Table 3 Posterior estimates of the easiness parameter in the 1P-ASM and 1P-ASM-R* models

State	1P-ASM				1P-ASM-R*
State	Mean	SD	95%HPDL	95%HPDU	Mean	SD	95%HPDL	95%HPDU
A	0.872	0.033	0.809	0.935	0.667	0.049	0.568	0.764
B	1.606	0.048	1.511	1.699	1.331	0.068	1.198	1.459
C	1.415	0.052	1.316	1.515	1.093	0.076	0.947	1.244
D	1.452	0.059	1.339	1.567	1.089	0.088	0.916	1.266
E	1.940	0.074	1.800	2.088	1.536	0.101	1.336	1.733
F	0.329	0.118	0.099	0.553	-0.086	0.138	-0.353	0.180
G	-1.721	0.078	-1.875	-1.571	-1.952	0.091	-2.130	-1.773
H	-2.086	0.083	-2.252	-1.927	-2.375	0.100	-2.571	-2.182
I	-0.860	0.053	-0.964	-0.757	-1.201	0.082	-1.364	-1.047
J	-0.398	0.111	-0.617	-0.178	-0.798	0.130	-1.056	-0.544

Table 4 The model fit of the 1P-ASM and 1P-ASM-R* models to the simulated data

Sample size	Sequence length	Learning effect	LOO	WAIC
200	Short	0	38%	38%
		0.1	70%	72%
		0.3	98%	98%
	Long	0	26%	26%
		0.1	100%	100%
		0.3	100%	100%
1000	Short	0	28%	28%
		0.1	94%	94%
		0.3	100%	100%
	Long	0	26%	26%
		0.1	100%	100%
		0.3	100%	100%

Table 4 The model fit of the 1P-ASM and 1P-ASM-R* models to the simulated data

Sample size	Sequence length	Learning effect	LOO	WAIC
200	Short	0	38%	38%
		0.1	70%	72%
		0.3	98%	98%
	Long	0	26%	26%
		0.1	100%	100%
		0.3	100%	100%
1000	Short	0	28%	28%
		0.1	94%	94%
		0.3	100%	100%
	Long	0	26%	26%
		0.1	100%	100%
		0.3	100%	100%

Figure 1. Distribution of the differences in model fit indices between 1P-ASM-R* and 1P-ASM.

Figure 1. Distribution of the differences in model fit indices between 1P-ASM-R* and 1P-ASM.

Table 5 Ability parameter recovery

Sample size	Sequence length	Learning effect	Bias		RMSE
Sample size	Sequence length	Learning effect	1P-ASM	1P-ASM-R*	1P-ASM	1P-ASM-R*
200	Short	0	0.018	0.017	0.580	0.581
		0.1	0.025	0.015	0.584	0.583
		0.3	0.040	0.017	0.601	0.596
	Long	0	0.027	0.027	0.449	0.449
		0.1	0.040	0.026	0.472	0.464
		0.3	0.058	0.025	0.524	0.501
1000	Short	0	-0.023	-0.023	0.586	0.586
		0.1	-0.021	-0.022	0.590	0.589
		0.3	-0.023	-0.017	0.609	0.605
	Long	0	-0.020	-0.020	0.447	0.447
		0.1	0.012	-0.009	0.471	0.466
		0.3	-0.012	0.005	0.525	0.502

Table 5 Ability parameter recovery

Sample size	Sequence length	Learning effect	Bias		RMSE
Sample size	Sequence length	Learning effect	1P-ASM	1P-ASM-R*	1P-ASM	1P-ASM-R*
200	Short	0	0.018	0.017	0.580	0.581
		0.1	0.025	0.015	0.584	0.583
		0.3	0.040	0.017	0.601	0.596
	Long	0	0.027	0.027	0.449	0.449
		0.1	0.040	0.026	0.472	0.464
		0.3	0.058	0.025	0.524	0.501
1000	Short	0	-0.023	-0.023	0.586	0.586
		0.1	-0.021	-0.022	0.590	0.589
		0.3	-0.023	-0.017	0.609	0.605
	Long	0	-0.020	-0.020	0.447	0.447
		0.1	0.012	-0.009	0.471	0.466
		0.3	-0.012	0.005	0.525	0.502

References

Metrics

Viewed

Full text

Abstract

Comments

Abstract
Figures/Tables
References
Related Articles
Recommended Articles
Metrics
Comments

TOP