Author:
Krakow Elizabeth F,Hemmer Michael,Wang Tao,Logan Brent,Arora Mukta,Spellman Stephen,Couriel Daniel,Alousi Amin,Pidala Joseph,Last Michael,Lachance Silvy,Moodie Erica E M
Abstract
Abstract
Q-learning is a method of reinforcement learning that employs backwards stagewise estimation to identify sequences of actions that maximize some long-term reward. The method can be applied to sequential multiple-assignment randomized trials to develop personalized adaptive treatment strategies (ATSs)—longitudinal practice guidelines highly tailored to time-varying attributes of individual patients. Sometimes, the basis for choosing which ATSs to include in a sequential multiple-assignment randomized trial (or randomized controlled trial) may be inadequate. Nonrandomized data sources may inform the initial design of ATSs, which could later be prospectively validated. In this paper, we illustrate challenges involved in using nonrandomized data for this purpose with a case study from the Center for International Blood and Marrow Transplant Research registry (1995–2007) aimed at 1) determining whether the sequence of therapeutic classes used in graft-versus-host disease prophylaxis and in refractory graft-versus-host disease is associated with improved survival and 2) identifying donor and patient factors with which to guide individualized immunosuppressant selections over time. We discuss how to communicate the potential benefit derived from following an ATS at the population and subgroup levels and how to evaluate its robustness to modeling assumptions. This worked example may serve as a model for developing ATSs from registries and cohorts in oncology and other fields requiring sequential treatment decisions.
Funder
Cole Foundation. E.E.M.M.
Fonds de recherche du Québec–Santé
The Center for International Blood and Marrow Transplant Research
National Cancer Institute
National Heart, Lung and Blood Institute
National Institute of Allergy and Infectious Diseases
Health Resources and Services Administration
Office of Naval Research
Actinium Pharmaceuticals, Inc.
Amgen, Inc.
Amneal Biosciences
Angiocrine Bioscience, Inc.
Medical College of Wisconsin
Astellas Pharma US
Atara Biotherapeutics, Inc.
Bristol-Myers Squibb Oncology
Celgene Corporation
Cerus Corporation
Chimerix, Inc.
Fred Hutchinson Cancer Research Center
Gamida Cell, Ltd.
Gilead Sciences, Inc.
HistoGenetics, Inc.
Immucor
Incyte Corporation
Janssen Scientific Affairs, LLC
Jazz Pharmaceuticals, Inc.
Juno Therapeutics
Karyopharm Therapeutics, Inc.
Kite Pharma, Inc.
Medac, GmbH
MedImmune
The Medical College of Wisconsin
Mediware
Merck & Co, Inc.
Mesoblast
Meso Scale Diagnostics, Inc.
Millennium, the Takeda Oncology Co.
Miltenyi Biotec, Inc.
National Marrow Donor Program
Neovii Biotech NA, Inc.
Novartis Pharmaceuticals Corporation
Otsuka Pharmaceutical Co, Ltd. -Japan
PCORI
Pfizer, Inc
Pharmacyclics, LLC
PIRCHE AG
Sanofi Genzyme
Seattle Genetics
Shire
Spectrum Pharmaceuticals, Inc.
St. Baldricks Foundation
Sunesis Pharmaceuticals, Inc.
Swedish Orphan Biovitrum, Inc.
Takeda Oncology
Telomere Diagnostics, Inc.
University of Minnesota
Publisher
Oxford University Press (OUP)