Deep Q-learning for same-day delivery

Same-day delivery (SDD) changes the way people shop as it combines immediate product availability and the convenience of ordering from electronic devices. The SDD market has seen a fast growth over the past decade. These services have further gained popularity during the COVID-19 pandemic. Yet, even before the surge of demand brought on by the pandemic, providing SDD was a challenge. The timing of requests and the delivery locations are not known until a request is made. In addition, SDD requires tight delivery deadlines and immediate response to customer requests. To address these challenges, this thesis seeks to provide an innovative deep Q-learning approach for studying SDD. Deep Q-learning is a form of reinforcement learning that has shown promising performance in artificial intelligence and robotics. We will investigate the effectiveness of the approach to two emerging SDD problems.

First, to complement the inefficient use of conventional vehicles, companies have begun to use drones for SDD. To effectively exploit the different strengths of vehicles and drones, we first consider the SDD problem with vehicles and drones. Over the course of a day, customers make delivery requests, and the dispatcher dynamically assigns vehicles and drones to deliver the goods. The objective is to maximize the expected number of customers served. Computational results demonstrate the effectiveness of the deep Q-learning approach.

Second, as SDD market continues to expand, concerns about the fairness of these delivery services arise. In 2016, Amazon was accused of excluding certain minority neighborhoods from its SDD map. To address concerns about fairness, we then consider the problem of SDD with fair customer service. In addition to serving as many customers overall as possible, we also seek to increase the chance of receiving the service for customers in an otherwise underserved neighborhood. The proposed deep Q-learning approach computationally demonstrates its effectiveness in reducing unfairness in customer service. We also show that ignoring fairness in services results in a long-term loss in overall revenue.