Dissertation
Application of explainable reinforcement learning in optimal drug dosing: a case of warfarin
University of Iowa
Doctor of Philosophy (PhD), University of Iowa
Summer 2023
DOI: 10.25820/etd.007071
Abstract
Explainable Reinforcement Learning (XRL) is a relatively young area of research in Machine Learning with a promising future. It allows the optimization of sequential decision-making by learning the best decision for any situation without prior knowledge of the optimal decisions. It presents the result in an explainable format that makes it easier to diagnose, understand, and use. These characteristics make XRL a useful tool in optimizing healthcare decision processes. However, the downside of XRL is that it is a data-intensive tool.
In this dissertation, we use warfarin, an anticoagulant, as our case and gradually build toward a framework for optimizing medical decision-making. This framework consists of two phases. Phase I uses a mathematical model to simulate the effect of an intervention on a patient and learns the optimal policy, mapping patients' states to the optimal decision. The second phase uses Policy Distillation (PD) to extract the policy and build an explainable one.
Chapter 1 introduces the problem of warfarin dosing and discusses the importance and impact of an individualized dosing protocol for warfarin. It uses a Pharmacokinetic/Pharmacodynamic (PK/PD) model of warfarin to approximate the patients' responses to warfarin and Deep Q-Network (DQN) to learn and optimize the policy. The focus of Chapter 1 is to show the applicability and superiority of RL in warfarin dosing.
Chapter 2 presents the XRL model for warfarin. It utilizes Proximal Policy Optimization (PPO) in Phase I and introduces ``Action Forging'' techniques designed to enhance the dosing quality for Phase II. Phase II employs decision trees to extract the policy and the result is aggregated in a table familiar to practitioners in this field.
Chapter 3 extends the proposed work by incorporating the decision duration in the decision space. It proposes new PPO architectures to address the curse of dimensionality induced by the additional dimension. While demonstrating the practicality of our approach, we acknowledge the need for further improvements to provide a practical dosing protocol. Potential remedies are explored to enhance the results.
The Conclusion Chapter presents the complete framework, discussing its potential and possible future directions. Our research represents a significant step towards XRL-based tools for optimizing medical decision-making, with the promise of improving care quality and reducing medical errors.
Details
- Title: Subtitle
- Application of explainable reinforcement learning in optimal drug dosing: a case of warfarin
- Creators
- Sadjad Anzabi Zadeh
- Contributors
- W. Nick Street (Advisor)Barrett W. Thomas (Advisor)Qihang Lin (Committee Member)Kang Zhao (Committee Member)
- Resource Type
- Dissertation
- Degree Awarded
- Doctor of Philosophy (PhD), University of Iowa
- Degree in
- Business Administration (Business Analytics)
- Date degree season
- Summer 2023
- DOI
- 10.25820/etd.007071
- Publisher
- University of Iowa
- Number of pages
- xi, 87 pages
- Copyright
- Copyright 2023 Sadjad Anzabi Zadeh
- Language
- English
- Date submitted
- 07/24/2023
- Description illustrations
- Illustrations, tables, graphs, charts
- Description bibliographic
- Includes bibliographical references (pages 81-87).
- Public Abstract (ETD)
Warfarin, a blood clot preventive drug, poses dosing challenges due to its narrow therapeutic window, where excess causes bleeding and insufficiency raises clotting risks.
In this study, we employ the innovative Explainable Reinforcement Learning (XRL) technique to optimize warfarin dosing. Unlike traditional methods, XRL doesn’t require preexisting optimal dose examples, ensuring user-friendly outputs.
Our two-phase XRL framework begins with learning the optimal dose for patients via interaction with a mathematical model. In the subsequent phase, we extract optimal dosing, presented in an easy-to-read table format.
Successfully optimizing warfarin dosing in simulated patients, our framework is a crucial step towards XRL-based tools for medical decision-making. These tools promise improved care quality and reduced medical errors.
As we continue refining XRL’s potential, we anticipate safer and more efficient healthcare, transforming medical decision-making and enhancing patient outcomes.
- Academic Unit
- Tippie College of Business
- Record Identifier
- 9984454434802771
Metrics
1 File views/ downloads
38 Record Views