Logo image
Back
Modular Deep Reinforcement Learning for Continuous Motion Planning With Temporal Logic
Journal article   Peer reviewed

Modular Deep Reinforcement Learning for Continuous Motion Planning With Temporal Logic

Mingyu Cai, Mohammadhosein Hasanbeig, Shaoping Xiao, Alessandro Abate and Zhen Kan
IEEE robotics and automation letters, Vol.6(4), pp.7973-7980
10/2021
DOI: 10.1109/LRA.2021.3101544
url
https://arxiv.org/pdf/2102.12855View
Open Access

Abstract

This letter investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) with unknown transition probabilities over continuous state and action spaces. Linear temporal logic (LTL) is used to specify high-level tasks over infinite horizon, which can be converted into a limit deterministic generalized Büchi automaton (LDGBA) with several accepting sets. The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP by incorporating a synchronous tracking-frontier function to record unvisited accepting sets of the automaton, and to facilitate the satisfaction of the accepting conditions. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states and can overcome the issues of sparse rewards. Rigorous analysis shows that any RL method that optimizes the expected discounted return is guaranteed to find an optimal policy whose traces maximize the satisfaction probability. A modular deep deterministic policy gradient (DDPG) is then developed to generate such policies over continuous state and action spaces. The performance of our framework is evaluated via an array of OpenAI gym environments.
 Automata Deep Reinforcement Learning Extraterrestrial measurements Linear Temporal Logic Motion Planning Planning Reinforcement learning Robustness Task analysis Uncertainty

Details

Metrics

25 Record Views
69 Times Cited - Web of Science
Logo image