Memory mechanisms of hand gesture in communication and learning

Spontaneous co-speech hand gestures robustly affect learning and memory. Viewing or producing hand gestures during conversation facilitates the encoding, consolidation, and retention of the information in speech. Despite these effects, the cognitive and neural mechanisms supporting this relationship remains unknown. In Experiment 1, I explored the memory mechanisms supporting hand gesture by working with patients with damage to their hippocampus and thus their declarative memory system. Participants engaged in discourse tasks that disproportionately engaged the hippocampus. I found that patients gestured less overall than healthy comparisons across all tasks, suggesting that the hippocampus indeed plays a role in gesture production.

In order to test whether non-declarative memory supports gesture production as well, Experiment 2 directly manipulated features of memory representations (both visual and motor) to determine what would guide the form of gesture when participants later explained their experiences. On three visits, amnesic patients, healthy comparison and brain-damaged comparison groups completed a Tower of Hanoi task, involving moving disks between pegs following a set of rules. On each visit, participants completed the task with different visual and motor information. Comparisons' gestures tended to reflect both visual and motor experience, while patients' gestures tended to rely more heavily on their motor experiences. This suggests that gesture may be supported by non-declarative memory as well, particularly in the absence of a declarative memory for what is being discussed.

To directly test which properties of gesture facilitate learning, Experiment 3 examined how gesture affected the learning of novel labels for common, everyday objects. I again worked with patients with hippocampal amnesia, who are severely impaired in the learning of new words, along with healthy and brain-damaged comparisons. Participants were exposed to novel word-object pairing that either was learned with a gesture or not. For the gestured-with trials, the gesture was either viewed and then produced by the participant or passively viewed, allowing me to determine if production of a gesture was necessary for learning. After adequately learning all the word-object pairings, there was a 30-minute delay followed by a free recall and object identification task. Both comparison groups showed good learning of the words regardless of whether they were learned with gesture. The amnesic patients performed poorly on the recall task. On the object identification task, they were significantly more likely to identify the label-object pairing if the pairing had been learned with gesture. This benefit was only seen for those learned by producing gesture. For the pairings learned without gesture and the pairings learned with only viewing gesture, the patients were at chance. These findings demonstrate that gesture can help rescue hippocampal amnesics’ ability to bind labels with objects, and furthermore suggest that the self-production of gesture is critical for learning.

These findings are the first to demonstrate a link between gesture and memory systems. Experiments 1 and 2 demonstrate that gesture can reflect information from both declarative and non-declarative memory. Experiment 3 demonstrates that the link between gesture and non-declarative memory can be exploited to facilitate learning in patients with memory impairment. By understanding how memory and language interact we will be able to exploit this interaction to benefit memory and language more generally.