Abstract: As large language models (LLMs) are increasingly deployed in sensitive domains such as education and career guidance, concerns have grown about their potential to amplify gender bias. Prior research has documented occupational gender stereotypes in LLMs, such as associating men with technical roles and women with caregiving roles However, less attention has been paid to whether these models also encode deeper socio-emotional traits in gender-based ways. A persistent societal stereotype holds that “women are more empathetic than men”, a belief that can shape career expectations. This study investigated whether LLMs reflect or even exaggerate gender stereotypes related to empathy and examined the contextual factors (e.g., input language, gender-identity priming) that might influence the expression of these stereotypes. We hypothesized that LLMs would exhibit stronger gendered empathy stereotypes than human participants, that these biases would vary according to linguistic and social cues in prompts; and that these stereotypes would manifest in real-world major/career recommendation scenarios.
We conducted three studies to test these hypotheses. Study 1 compared judgments about empathy from human participants (N = 626) with those generated by six leading LLMs (GPT-4o, GPT-4-Turbo, GPT-3.5-Turbo, DeepSeek-reasoner, DeepSeek-chat, ERNIE-Bot). Twelve story-based scenarios, adapted from the Empathy Questionnaire, covered emotional empathy, attention to others’ feelings, and behavioral empathy. For each scenario, participants and LLMs inferred the protagonist’s gender based on their empathetic behavior. Study 2 examined how manipulating input language (English vs. Chinese) and gender-identity priming (male vs. female) influenced the expression of these stereotypes. Study 3 extended this paradigm to a real-world application: we prompted LLMs to recommend 16 pre-selected university majors and 16 professions (categorized into high- and low-empathy groups) to individuals of different genders, requesting explanatory rationales for each recommendation.
Results indicated that LLMs displayed significantly stronger gendered empathy stereotypes than human participants (Study 1). English prompts and female priming elicited stronger “women = high empathy, men = low empathy” associations (Study 2). In the recommendation tasks, LLMs more often suggested high-empathy majors and professions (e.g., nursing, education, psychology) for women, and low-empathy, STEM-related fields for men (Study 3). Together, these findings suggest that LLMs not only internalize gendered empathy stereotypes but also express them in context-dependent ways, producing measurable downstream effects in applied decision-making tasks.
=Overall, our findings underscore the need for critical evaluation of how LLMs represent and amplify social stereotypes, especially in socio-emotional domains such as empathy. This research contributes to understanding the sources of AI bias by showing that LLMs may exaggerate gender norms beyond human levels. It also highlights the complex interplay between language and gender identity in shaping algorithmic behavior. Practically, the results raise important ethical concerns about fairness in AI-driven decision-making systems and highlight the urgency of developing more robust bias-mitigation strategies in multilingual contexts.

Key words: large language models, gender stereotypes, empathy, AI recommendations, human-computer interaction