Using a stakeholder-engaged approach to develop and validate electronic clinical quality measures

Jill Boylston Herndon; Krishna Aravamudhan; Ronald L Stephenson; Ryan Brandon; Jesley Ruff; Frank Catalanotto; Huong Le

doi:10.1093/jamia/ocw137

Back

Using a stakeholder-engaged approach to develop and validate electronic clinical quality measures

Journal article

Open access

Using a stakeholder-engaged approach to develop and validate electronic clinical quality measures

Jill Boylston Herndon, Krishna Aravamudhan, Ronald L Stephenson, Ryan Brandon, Jesley Ruff, Frank Catalanotto and Huong Le

Journal of the American Medical Informatics Association : JAMIA, Vol.24(3), pp.503-512

05/01/2017

DOI: 10.1093/jamia/ocw137

PMCID: PMC7651948

PMID: 28339559

Files and links (1)

url

https://doi.org/10.1093/jamia/ocw137View

Published (Version of record) Open Access

Abstract

To describe the stakeholder-engaged processes used to develop, specify, and validate 2 oral health care electronic clinical quality measures. A broad range of stakeholders were engaged from conception through testing to develop measures and test feasibility, reliability, and validity following National Quality Forum guidance. We assessed data element feasibility through semistructured interviews with key stakeholders using a National Quality Forum-recommended scorecard. We created test datasets of synthetic patients to test measure implementation feasibility and reliability within and across electronic health record (EHR) systems. We validated implementation with automated reporting of EHR clinical data against manual record reviews, using the kappa statistic. A stakeholder workgroup was formed and guided all development and testing processes. All critical data elements passed feasibility testing. Four test datasets, representing 577 synthetic patients, were developed and implemented within EHR vendors' software, demonstrating measure implementation feasibility. Measure reliability and validity were established through implementation at clinical practice sites, with kappa statistic values in the "almost perfect" agreement range of 0.80-0.99 for all but 1 measure component, which demonstrated "substantial" agreement. The 2 validated measures were published in the United States Health Information Knowledgebase. The stakeholder-engaged processes used in this study facilitated a successful measure development and testing cycle. Engaging stakeholders early and throughout development and testing promotes early identification of and attention to potential threats to feasibility, reliability, and validity, thereby averting significant resource investments that are unlikely to be fruitful.

Electronic Health Records

United States

Adolescent

Child

Child, Preschool

Datasets as Topic

Dental Caries - therapy

Evidence-Based Dentistry

Humans

Meaningful Use

Pediatric Dentistry - standards

Quality Indicators, Health Care

Young Adult

Details

Title: Subtitle: Using a stakeholder-engaged approach to develop and validate electronic clinical quality measures
Creators: Jill Boylston Herndon
Krishna Aravamudhan - American Dental Association
Ronald L Stephenson - Stephenson Developers Group, Clearwater, Florida, USA
Ryan Brandon - Exan Group, Vancouver, Canada
Jesley Ruff - American Dental Partners, Inc., Wakefield, Massachusetts, USA
Frank Catalanotto - University of Florida
Huong Le - Asian Health Services
Resource Type: Journal article
Publication Details: Journal of the American Medical Informatics Association : JAMIA, Vol.24(3), pp.503-512
DOI: 10.1093/jamia/ocw137
PMID: 28339559
PMCID: PMC7651948
NLM abbreviation: J Am Med Inform Assoc
ISSN: 1067-5027
eISSN: 1527-974X
Grant note: DOI: 10.13039/100005982, name: Office of the National Coordinator for Health Information Technology, award: #HHSP233201300039C
Language: English
Date published: 05/01/2017
Academic Unit: Public Policy Center (Archive)
Record Identifier: 9984283859302771

Metrics

16 Record Views

6 Times Cited - Web of Science