Preprint
Fine-Tuning Large Language Models for Text Summarization: A Practical Guide
Iowa Research Online (University of Iowa)
University of Iowa
10/26/2026
DOI: 10.17077/pp.006814
Abstract
Large language models (LLMs) have shown strong potential for abstractive summarization, yet most prior work in evidence synthesis limits summarization to titles and abstracts and primarily focuses on domains with well-structured reporting formats. These limitations restrict the applicability of automated PICO summarization in social science research, where key information is often embedded in unstructured full-text sections. This study presents a comprehensive end-to-end workflow for fine-tuning LLMs for PICO-oriented summarization of long-form scientific texts in the mental health domain. The proposed pipeline integrates document-level annotation, label-specific instance construction, section-aware data augmentation, long-context model architectures, optimization strategies, and multi-metric evaluation within a unified framework. To systematically examine the effects of model and training choices, we employ a 2×3×3 factorial design varying model architecture, batch size, and learning rate. Performance is evaluated using three complementary metrics (ROUGE, perplexity, MAUVE), supplemented by qualitative comparisons between model-generated and human-written summaries. Results show that LongT5 consistently achieves higher ROUGE and lower perplexity, while BioBART attains higher MAUVE. Across configurations, intermediate learning rates and moderate batch sizes yield the most balanced performance. Further analysis reveals that higher learning rates disproportionately benefit structured and quantitative categories, while gains are more limited for descriptive categories. These findings highlight systematic trade-offs between accuracy, confidence, and diversity in generated summaries. Overall, the proposed pipeline provides practical, evidence-based guidance for applying generative models in long-document evidence synthesis and offers a foundation for future research on finetuning LLMs.
Details
- Title: Subtitle
- Fine-Tuning Large Language Models for Text Summarization: A Practical Guide
- Creators
- Zhifei Li - University of IowaLi Zhu - University of IowaKishlay Jha - University of IowaAriel M Aloe - University of Iowa
- Resource Type
- Preprint
- Publication Details
- Iowa Research Online (University of Iowa)
- DOI
- 10.17077/pp.006814
- Publisher
- University of Iowa; Iowa City, Iowa, United States
- Number of pages
- 55 pages
- Copyright
- Copyright © 2026 the authors
- Comment
- Portions of this research were previously presented at a roundtable session at the 2026 Annual Meeting of the American Educational Research Association (AERA). The current manuscript expands upon that presentation by providing a more comprehensive analysis and full reporting of methods and results. No prior version of this work has been formally published.
- Language
- English
- Date created
- 05/18/2026
- Date posted
- 10/26/2026
- Academic Unit
- Electrical and Computer Engineering; Psychological and Quantitative Foundations; Teaching and Learning
- Record Identifier
- 9985247183202771
Metrics
2 File views/ downloads
6 Record Views