Journal article
Model-free reinforcement learning for motion planning of autonomous agents with complex tasks in partially observable environments
Autonomous agents and multi-agent systems, Vol.38(1), 14
2024
DOI: 10.1007/s10458-024-09641-0
Abstract
Motion planning of autonomous agents in partially known environments with incomplete information is a challenging problem, particularly for complex tasks. This paper proposes a model-free reinforcement learning approach to address this problem. We formulate motion planning as a probabilistic-labeled partially observable Markov decision process (PL-POMDP) problem and use linear temporal logic (LTL) to express the complex task. The LTL formula is then converted to a limit-deterministic generalized Büchi automaton (LDGBA). The problem is redefined as finding an optimal policy on the product of PL-POMDP with LDGBA based on model-checking techniques to satisfy the complex task. We implement deep Q learning with long short-term memory (LSTM) to process the observation history and task recognition. Our contributions include the proposed method, the utilization of LTL and LDGBA, and the LSTM-enhanced deep Q learning. We demonstrate the applicability of the proposed method by conducting simulations in various environments, including grid worlds, a virtual office, and a multi-agent warehouse. The simulation results demonstrate that our proposed method effectively addresses environment, action, and observation uncertainties. This indicates its potential for real-world applications, including the control of unmanned aerial vehicles.
Details
- Title: Subtitle
- Model-free reinforcement learning for motion planning of autonomous agents with complex tasks in partially observable environments
- Creators
- Junchao Li - University of IowaMingyu Cai - University of California, RiversideZhen Kan - University of Science and Technology of ChinaShaoping Xiao - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Autonomous agents and multi-agent systems, Vol.38(1), 14
- Publisher
- Springer US
- DOI
- 10.1007/s10458-024-09641-0
- ISSN
- 1387-2532
- eISSN
- 1573-7454
- Grant note
- ED#P116S210005; ED#P116S210005 / US Department of Education #2226936; #2226936 / National Science Foundation, United States
- Language
- English
- Date published
- 2024
- Academic Unit
- Iowa Technology Institute; Mechanical Engineering
- Record Identifier
- 9984577033802771
Metrics
2 Record Views