Preprint
The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support
ArXiv.org
Cornell University
09/20/2025
DOI: 10.48550/arxiv.2505.15065
Abstract
This paper investigates the capacity of small language models (0.5B-5B parameters) to generate empathetic responses for individuals with PTSD. We introduce Trauma-Informed Dialogue for Empathy (TIDE), a novel dataset comprising 10,000 two-turn conversations across 500 diverse, clinically-grounded PTSD personas (https://huggingface.co/datasets/yenopoya/TIDE). Using frontier model outputs as ground truth, we evaluate eight small LLMs in zero-shot settings and after fine-tuning. Fine-tuning enhances empathetic capabilities, improving cosine similarity and perceived empathy, although gains vary across emotional scenarios and smaller models exhibit a "knowledge transfer ceiling." As expected, Claude Sonnet 3.5 consistently outperforms all models, but surprisingly, the smaller models often approach human-rated empathy levels. Demographic analyses showed that older adults favored responses that validated distress before offering support (p = .004), while graduate-educated users preferred emotionally layered replies in specific scenarios. Gender-based differences were minimal (p > 0.15), suggesting the feasibility of broadly empathetic model designs. This work offers insights into building resource-efficient, emotionally intelligent systems for mental health support.
Details
- Title: Subtitle
- The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support
- Creators
- Suhas BNYash MahajanDominik MattioliAndrew M SherrillRosa I ArriagaChris W WieseSaeed Abdullah
- Resource Type
- Preprint
- Publication Details
- ArXiv.org
- DOI
- 10.48550/arxiv.2505.15065
- ISSN
- 2331-8422
- Publisher
- Cornell University; Ithaca, New York
- Language
- English
- Date posted
- 09/20/2025
- Academic Unit
- Orthopedics and Rehabilitation
- Record Identifier
- 9985113004902771
Metrics
21 Record Views