Journal article
Making AI Accessible to Social Work Researchers: An Exploratory Analysis of Using ChatGPT to Screen Articles for Systematic and Scoping Reviews
Research on social work practice, Vol.36(4), pp.364-381
05/2026
DOI: 10.1177/10497315251357190
Abstract
Purpose: Title and abstract screening is one of the most time- and resource-intensive steps in systematic and scoping reviews (SRs); however, artificial intelligence (AI) can accelerate this step without sacrificing methodological rigor. We test ChatGPT-4o's ability to accurately screen citations and present an accessible approach, tailored to social work scholars with limited AI experience. Method: Through prompt engineering, we tested how two vectorization techniques, three algorithms, and class weighting impact ChatGPT-4o's classification accuracy for article inclusion in two SRs compared to models run in a native Python environment (Jupyter Notebook). Results: ChatGPT-4o-generated models were comparable to Jupyter Notebook, with the bag of words or term frequency-inverse document frequency and logistic regression models performing well. Additionally, adjusting for class imbalance improved performance across samples and models. Discussion: This approach is effective, promoting accessibility for researchers and practitioners. We encourage future researchers to replicate and extend the use of chatbots in the screening process.
Details
- Title: Subtitle
- Making AI Accessible to Social Work Researchers: An Exploratory Analysis of Using ChatGPT to Screen Articles for Systematic and Scoping Reviews
- Creators
- Saige M. Addison - University of IowaEmily D. Campion - University of IowaMiriam J. Landsman - University of IowaChristabel L. Rogalin - Purdue University NorthwestChristopher A. Veeh - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Research on social work practice, Vol.36(4), pp.364-381
- DOI
- 10.1177/10497315251357190
- ISSN
- 1049-7315
- eISSN
- 1552-7581
- Publisher
- Sage
- Grant note
- University of Iowa's Scanlan Center for School Mental HealthUSDHHS Children's Bureau Regional Partnership: 90CU0097-01-00 Purdue Northwest's Department of Behavioral SciencesCHESS Summer Scholarship
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported in part with funding from the University of Iowa's Scanlan Center for School Mental Health, by the USDHHS Children's Bureau Regional Partnership Grants-5 (90CU0097-01-00), Purdue Northwest's Department of Behavioral Sciences as well as the CHESS Summer Scholarship (Purdue Northwest).
- Language
- English
- Electronic publication date
- 07/28/2025
- Date published
- 05/2026
- Academic Unit
- Management and Entrepreneurship ; School of Social Work
- Record Identifier
- 9984927083502771
Metrics
12 Record Views