Abstract:

Experimental Vignette Methodology (EVM), as a advanced approach to enhancing both internal and ecological validity, involoves researchers manipulating independent variables and controlling confounding factors through the careful construction of realistic vignettes to assess dependent variables such as intentions, attitudes, and behaviors. In recent years, EVM has gained increasing popularity in managerial psychology research. However, many problems have arisen. First, some studies were actually based on the EVM, but utilize the term “contextual priming methodology”. What’s more, some vignettes in EVM have design issues, which can easily lead to framing effects, social desirability and false reactions. Unfortunatly, integrated recommendations for creating and validating vignettes are limited. These issues render the EVM susceptible to experimental material design challenges, effectiveness queries, and potential hindrances to its development in managerial psychology research. Thus, it is essential to give a brief introduction to EVM and focus on the topic of how to properly design and validate the vignette before its deployment in managerial psychology research.

Based on the above considerations, this paper first introduced the definition and types of EVM. Additionally, we clarified the differences between EVM and contextual priming methodology in terms of method characteristics, scenario characteristics, manipulable variables, and commonly used techniques. Second, based on the three stages of pre-design, design and post-design, this study coded and analyzed 93 scenario experiments drawn from 20 major domestic and foreign management-related journals in the past five years (2019.01-2023.05). We sorted out the design issues and systematically summarized the design strategies existing in the present managerial psychology research. We found that the design issues were as follows: a lack of standardization in the design process; a lack of realism in the vignettes; little attention being paid to the evaluation of realism and content validity before vignettes were deployed; and a lack of widespread use of pretests..

Given the design issues and strategies in the present managerial psychology research, this study adopted method application as the guiding principle and addressed the question of how to design the vignettes in EVM from the three stages of pre-design, design and post-design. During the pre-design phase, the focus was on determining whether EVM was the appropriate approach. We found that EVM is a suitable approach when researchers encountered ethical dilemmas, samples were difficult to obtain, and the varibales often manipulated were interaction relationships and related to the all parties involved in the interaction, such as behaviors, cognitions and emotions. The design stage was concerned with how to draft the vignettes consisting of backgroud module and manipulation material module. This process was divided into five sub-stages: determining the number of vignettes, drafting the vignettes, choosing the media, standardizing vignettes and enhancing vignettes’ realism. The post-design stage focused on evaluating whether the vignettes were clear, realistic, complete and effective. Methods such as realism evaluation scale, content validity index(CVI), and manipulation checks could be used to evaluate the validity of vignettes. If vignettes failed to meet standards or another scenario experiment was required, the researcher should return to the design stage to continuously modify or rewrite vignettes until they met criteria such as consistency with the research topic, accurate stimulation of variable levels, inclusion of necessary information, control of confounding factors, and presentation as fully immersive scenarios.

Although the design strategies for the vignettes in EVM in the managerial psychology research have been systematically summarized from existing research, further enrichments are still needed. Specifically, future research could use incident technique and information technology, such as VR, to design the vignettes. Furthermore, vignettes with iterative decision are encouraged to be created. It is also important to note that the design process of vignettes is guided by theoretical principles. By doing so, it will enhance the applicability, richness and realism of vignettes.

Key words: experimental vignette methodology, vignette, design strategy, managerial psychology experiment