Automatic radiotherapy treatment planning with deep functional reinforcement learning

Abstract

AbstractIntensity-modulated radiation therapy (IMRT) is one of the most important modern radiotherapy techniques and is often modeled as an optimization problem. The objective function and constraints consist of multiple clinical requirements designed for different clinical settings. When a tightly constrained optimization problem has no solution, the planner can empirically relax certain constraint parameters and re-solve the problem until a more satisfactory solution is obtained. This process is time-consuming and laborious. Several inverse planning studies have been devoted to automated radiotherapy planning schemes. Reinforcement learning has been used by many studies to model this process, but they suffer from two important drawbacks: 1) designing a sub-network for each organ, which makes it difficult to extend the model to other patients with a different number of organs. Clinically, it is common for different patients to have inconsistent numbers of organs considered for radiotherapy, even for the same type of cancer; 2) directly feeding low signal-to-noise DVH curves as states into the reinforcement learning network, which ignores its functional characteristics and leads to low training efficiency. In this study, within the framework of deep reinforcement learning, a DVH function-based embedding layer was designed to directly extract the effective information of DVH and allow different organs to share a strategic network. The test results on a dataset of 135 patients with cervical cancer find that our proposed model can be applied to radiotherapy planning in real-world scenarios.