Logo image
Fine-Tuning Large Language Models for Text Summarization: A Practical Guide
Preprint   Open access

Fine-Tuning Large Language Models for Text Summarization: A Practical Guide

Zhifei Li, Li Zhu, Kishlay Jha and Ariel M Aloe
Iowa Research Online (University of Iowa)
University of Iowa
10/26/2026
DOI: 10.17077/pp.006814
pdf
FineTuningLargeLangPICO2.34 MBDownloadView
Preprint (Author's original) This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access Free to read and download

Abstract

Large language models (LLMs) have shown strong potential for abstractive summarization, yet most prior work in evidence synthesis limits summarization to titles and abstracts and primarily focuses on domains with well-structured reporting formats. These limitations restrict the applicability of automated PICO summarization in social science research, where key information is often embedded in unstructured full-text sections. This study presents a comprehensive end-to-end workflow for fine-tuning LLMs for PICO-oriented summarization of long-form scientific texts in the mental health domain. The proposed pipeline integrates document-level annotation, label-specific instance construction, section-aware data augmentation, long-context model architectures, optimization strategies, and multi-metric evaluation within a unified framework. To systematically examine the effects of model and training choices, we employ a 2×3×3 factorial design varying model architecture, batch size, and learning rate. Performance is evaluated using three complementary metrics (ROUGE, perplexity, MAUVE), supplemented by qualitative comparisons between model-generated and human-written summaries. Results show that LongT5 consistently achieves higher ROUGE and lower perplexity, while BioBART attains higher MAUVE. Across configurations, intermediate learning rates and moderate batch sizes yield the most balanced performance. Further analysis reveals that higher learning rates disproportionately benefit structured and quantitative categories, while gains are more limited for descriptive categories. These findings highlight systematic trade-offs between accuracy, confidence, and diversity in generated summaries. Overall, the proposed pipeline provides practical, evidence-based guidance for applying generative models in long-document evidence synthesis and offers a foundation for future research on finetuning LLMs.
large language models structured text summarization evidence synthesis fine-tuning natural language processing

Details

Metrics

2 File views/ downloads
6 Record Views
Logo image