Indexing dynamic collective constructs using computer-aided text analysis: Construct validity evidence and illustrations featuring team processes

John E Mathieu; Mikhail A Wolfson; Semin Park; Margaret M Luciano; Wendy L Bedwell-Torres; P Scott Ramsay; Elizabeth A Klock; Scott I Tannenbaum

doi:10.1037/apl0000856

Back

Indexing dynamic collective constructs using computer-aided text analysis: Construct validity evidence and illustrations featuring team processes

Journal article

Open access

Peer reviewed

Indexing dynamic collective constructs using computer-aided text analysis: Construct validity evidence and illustrations featuring team processes

John E Mathieu, Mikhail A Wolfson, Semin Park, Margaret M Luciano, Wendy L Bedwell-Torres, P Scott Ramsay, Elizabeth A Klock and Scott I Tannenbaum

Journal of applied psychology, Vol.107(4), pp.533-559

04/2022

DOI: 10.1037/apl0000856

PMID: 34491075

Files and links (1)

url

https://doi.org/10.1037/apl0000856View

Published (Version of record) Open Access

Abstract

Organizational processes have been widely recognized as both multilevel and dynamic, yet traditional methods of measurements limit our ability to model and understand such phenomena. Featuring a popular model of team processes advanced by Marks et al. (2001), we illustrate a method to use individuals' communications as construct valid unobtrusive measures of collective constructs occurring over time. Thus, the purpose of this investigation is to develop computer-aided text analysis (CATA) techniques that can score members' communications into valid team process measures. We apply a deductive content validity-based method to construct CATA dictionaries for Marks et al.'s dimensions. We then demonstrate their convergent validity with subject matter experts' (SMEs) hand-coded team communications and different SMEs' behaviorally anchored rating scales based on video recordings of team interactions, using multitrait-multimethod analyses in two samples. Using a third sample of paramedics performing a high-fidelity mass casualty incident exercise, we further demonstrate the convergent validity of the CATA and SME scorings of communications. We then model the relationships among processes across episodes using all three samples. Next, we test criterion-related validity using a longitudinal dual-discontinuous change growth modeling design featuring the paramedic CATA-scored team processes as related to a dynamic performance criterion. Finally, we integrate behavioral data from wearable sensor badges to illustrate how CATA can be scored at the individual level and then leveraged to model dynamic networks of team interactions. Implications, limitations, directions for the future research, and guidelines for the application of these techniques to other collective constructs are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Computers

Humans

Details

Title: Subtitle: Indexing dynamic collective constructs using computer-aided text analysis: Construct validity evidence and illustrations featuring team processes
Creators: John E Mathieu - University of Connecticut
Mikhail A Wolfson - University of Kentucky
Semin Park - University of Iowa
Margaret M Luciano - Pennsylvania State University
Wendy L Bedwell-Torres - University of Memphis
P Scott Ramsay - University of South Florida
Elizabeth A Klock - University of Connecticut
Scott I Tannenbaum - Group for Organizational Effectiveness
Resource Type: Journal article
Publication Details: Journal of applied psychology, Vol.107(4), pp.533-559
DOI: 10.1037/apl0000856
PMID: 34491075
ISSN: 0021-9010
eISSN: 1939-1854
Grant note: NASA Army Research Institute
Language: English
Date published: 04/2022
Academic Unit: Management and Entrepreneurship
Record Identifier: 9984380436102771

Metrics

11 Record Views

23 Times Cited - Web of Science