Journal article
Evaluation of ChatGPT for Pelvic Floor Surgery Counseling
Urogynecology (Philadelphia, Pa.), Vol.30(3), pp.245-250
03/01/2024
DOI: 10.1097/SPV.0000000000001459
PMID: 38484238
Abstract
Large language models are artificial intelligence applications that can comprehend and produce human-like text and language. ChatGPT is one such model. Recent advances have increased interest in the utility of large language models in medicine. Urogynecology counseling is complex and time-consuming. Therefore, we evaluated ChatGPT as a potential adjunct for patient counseling.
Our primary objective was to compare the accuracy and completeness of ChatGPT responses to information in standard patient counseling leaflets regarding common urogynecological procedures.
Seven urogynecologists compared the accuracy and completeness of ChatGPT responses to standard patient leaflets using 5-point Likert scales with a score of 3 being "equally accurate" and "equally complete," and a score of 5 being "much more accurate" and much more complete, respectively. This was repeated 3 months later to evaluate the consistency of ChatGPT. Additional analysis of the understandability and actionability was completed by 2 authors using the Patient Education Materials Assessment Tool. Analysis was primarily descriptive. First and second ChatGPT queries were compared with the Wilcoxon signed rank test.
The median (interquartile range) accuracy was 3 (2-3) and completeness 3 (2-4) for the first ChatGPT query and 3 (3-3) and 4 (3-4), respectively, for the second query. Accuracy and completeness were significantly higher in the second query (P < 0.01). Understandability and actionability of ChatGPT responses were lower than the standard leaflets.
ChatGPT is similarly accurate and complete when compared with standard patient information leaflets for common urogynecological procedures. Large language models may be a helpful adjunct to direct patient-provider counseling. Further research to determine the efficacy and patient satisfaction of ChatGPT for patient counseling is needed.
Details
- Title: Subtitle
- Evaluation of ChatGPT for Pelvic Floor Surgery Counseling
- Creators
- Colin M Johnson - University of IowaCatherine S Bradley - From the Division of Urogynecology and Reconstructive Pelvic Surgery, Department of Obstetrics and GynecologyKimberly A Kenne - From the Division of Urogynecology and Reconstructive Pelvic Surgery, Department of Obstetrics and GynecologySarah Rabice - University of IowaElizabeth Takacs - Department of Urology, University of Iowa Hospitals and Clinics, Iowa City, IAAnnah Vollstedt - University of IowaJoseph T Kowalski - From the Division of Urogynecology and Reconstructive Pelvic Surgery, Department of Obstetrics and Gynecology
- Resource Type
- Journal article
- Publication Details
- Urogynecology (Philadelphia, Pa.), Vol.30(3), pp.245-250
- DOI
- 10.1097/SPV.0000000000001459
- PMID
- 38484238
- eISSN
- 2771-1897
- Language
- English
- Date published
- 03/01/2024
- Academic Unit
- Epidemiology; Obstetrics and Gynecology; Urology
- Record Identifier
- 9984572500802771
Metrics
5 Record Views