Affiliation:
1. Department of Statistics North Carolina State University Raleigh 27695 USA
2. Department of Statistics London School of Economics and Political Science London WC2A 2AE UK
Abstract
Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time‐varying covariates. In mobile health applications, these covariates are typically collected at different frequencies over a long time horizon. In this paper, we propose a deep spectral Q‐learning algorithm, which integrates principal component analysis (PCA) with deep Q‐learning to handle the mixed frequency data. In theory, we prove that the mean return under the estimated optimal policy converges to that under the optimal one and establish its rate of convergence. The usefulness of our proposal is further illustrated via simulations and an application to a diabetes dataset.
Subject
Statistics, Probability and Uncertainty,Statistics and Probability
Reference95 articles.
1. Deep Reinforcement Learning: A Brief Survey
2. Fast learning rates for plug-in classifiers
3. Prediction by Supervised Principal Components
4. Chen E. Y. Song R. &Jordan M. I.(2022).Reinforcement learning with heterogeneous data: Estimation and inference. arXiv preprint arXiv:2202.00088.
5. Information‐theoretic considerations in batch reinforcement learning;Chen J.;International Conference on Machine Learning,2019