Logo image
Back
Deep Q-learning for same-day delivery with vehicles and drones
Journal article   Peer reviewed

Deep Q-learning for same-day delivery with vehicles and drones

Xinwei Chen, Marlin W. Ulmer and Barrett W. Thomas
European journal of operational research, Vol.298(3), pp.939-952
05/01/2022
DOI: 10.1016/j.ejor.2021.06.021
url
https://arxiv.org/pdf/1910.11901View
Open Access

Abstract

•We address the dynamic same-day delivery problem with fleets of drones and vehicles.•We present a deep Q-learning approach exploiting both state and action space information.•We provide a detailed analysis in the functionality of our method and the resulting policies. In this paper, we consider same-day delivery with vehicles and drones. Customers make delivery requests over the course of the day, and the dispatcher dynamically dispatches vehicles and drones to deliver the goods to customers before their delivery deadline. Vehicles can deliver multiple packages in one route but travel relatively slowly due to the urban traffic. Drones travel faster, but they have limited capacity and require charging or battery swaps. To exploit the different strengths of the fleets, we propose a deep Q-learning approach. Our method learns the value of assigning a new customer to either drones or vehicles as well as the option to not offer service at all. In a systematic computational analysis, we show the superiority of our policy compared to benchmark policies and the effectiveness of our deep Q-learning approach. We also show that the combination of state and action features is very valuable and that our policy can maintain effectiveness when demand data and the fleet size change moderately.
 Transportation Dynamic vehicle routing Reinforcement learning Same-day delivery

Details

Metrics

16 Record Views
111 Times Cited - Web of Science
Logo image