Logo image
Back
Reward-based option competition in human dorsal stream and transition from stochastic exploration to exploitation in continuous space
Journal article   Open access   Peer reviewed

Reward-based option competition in human dorsal stream and transition from stochastic exploration to exploitation in continuous space

Michael N Hallquist, Kai Hwang, Beatriz Luna and Alexandre Y Dombrovski
Science advances, Vol.10(8), eadj2219
02/23/2024
DOI: 10.1126/sciadv.adj2219
PMCID: PMC10889364
PMID: 38394198
url
https://doi.org/10.1126/sciadv.adj2219View
Published (Version of record) Open Access

Abstract

Primates exploring and exploiting a continuous sensorimotor space rely on dynamic maps in the dorsal stream. Two complementary perspectives exist on how these maps encode rewards. Reinforcement learning models integrate rewards incrementally over time, efficiently resolving the exploration/exploitation dilemma. Working memory buffer models explain rapid plasticity of parietal maps but lack a plausible exploration/exploitation policy. The reinforcement learning model presented here unifies both accounts, enabling rapid, information-compressing map updates and efficient transition from exploration to exploitation. As predicted by our model, activity in human frontoparietal dorsal stream regions, but not in MT+, tracks the number of competing options, as preferred options are selectively maintained on the map, while spatiotemporally distant alternatives are compressed out. When valuable new options are uncovered, posterior β /α oscillations desynchronize within 0.4 to 0.7 s, consistent with option encoding by competing β -stabilized subpopulations. Together, outcomes matching locally cached reward representations rapidly update parietal maps, biasing choices toward often-sampled, rewarded options.
 Animals Humans Learning Memory, Short-Term Reinforcement, Psychology Reward

Details

Metrics

7 Record Views
Logo image