Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning
-
Published:2024-06-17
Issue:8
Volume:19
Page:1569-1578
-
ISSN:1861-6429
-
Container-title:International Journal of Computer Assisted Radiology and Surgery
-
language:en
-
Short-container-title:Int J CARS
Author:
Robertshaw Harry,Karstensen Lennart,Jackson Benjamin,Granados Alejandro,Booth Thomas C.
Abstract
Abstract
Purpose
Autonomous navigation of catheters and guidewires can enhance endovascular surgery safety and efficacy, reducing procedure times and operator radiation exposure. Integrating tele-operated robotics could widen access to time-sensitive emergency procedures like mechanical thrombectomy (MT). Reinforcement learning (RL) shows potential in endovascular navigation, yet its application encounters challenges without a reward signal. This study explores the viability of autonomous guidewire navigation in MT vasculature using inverse reinforcement learning (IRL) to leverage expert demonstrations.
Methods
Employing the Simulation Open Framework Architecture (SOFA), this study established a simulation-based training and evaluation environment for MT navigation. We used IRL to infer reward functions from expert behaviour when navigating a guidewire and catheter. We utilized the soft actor-critic algorithm to train models with various reward functions and compared their performance in silico.
Results
We demonstrated feasibility of navigation using IRL. When evaluating single- versus dual-device (i.e. guidewire versus catheter and guidewire) tracking, both methods achieved high success rates of 95% and 96%, respectively. Dual tracking, however, utilized both devices mimicking an expert. A success rate of 100% and procedure time of 22.6 s were obtained when training with a reward function obtained through ‘reward shaping’. This outperformed a dense reward function (96%, 24.9 s) and an IRL-derived reward function (48%, 59.2 s).
Conclusions
We have contributed to the advancement of autonomous endovascular intervention navigation, particularly MT, by effectively employing IRL based on demonstrator expertise. The results underscore the potential of using reward shaping to efficiently train models, offering a promising avenue for enhancing the accessibility and precision of MT procedures. We envisage that future research can extend our methodology to diverse anatomical structures to enhance generalizability.
Funder
Engineering and Physical Sciences Research Council Wellcome Trust
Publisher
Springer Science and Business Media LLC
Reference36 articles.
1. Townsend N, Wilson L, Bhatnagar P, Wickramasinghe K, Rayner M, Nichols M (2016) Cardiovascular disease in europe: epidemiological update 2016. Eur Heart J 37(3232–3245):11 2. Goyal M, Menon BK, Zwam WHV, Dippel DW, Mitchell PJ, Demchuk AM, Dávalos A, Majoie CB, Lugt AVD, Miquel MAD, Donnan GA, Roos YB, Bonafe A, Jahan R, Diener HC, Berg LAVD, Levy EI, Berkhemer OA, Pereira VM, Rempel J, Millán M, Davis SM, Roy D, Thornton J, Román LS, Ribó M, Beumer D, Stouch B, Brown S, Campbell BC, Oostenbrugge RJV, Saver JL, Hill MD, Jovin TG (2016) Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. The Lancet 387:1723–1731 3. Vidale S, Agostoni E (2017) Endovascular treatment of ischemic stroke: an updated meta-analysis of efficacy and safety. Vasc Endovasc Surg 51:215–219 4. Rha JH, Saver JL (2007) The impact of recanalization on ischemic stroke outcome: a meta-analysis. Stroke 38:967–973 5. Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, Schonewille WJ, Vos JA, Nederkoorn PJ, Wermer MJ, van Walderveen MA, Staals J, Hofmeijer J, van Oostayen JA, Nijeholt GJL, Boiten J, Brouwer PA, Emmer BJ, de Bruijn SF, van Dijk LC, Kappelle LJ, Lo RH, van Dijk EJ, de Vries J, de Kort PL, van Rooij WJJ, van den Berg JS, van Hasselt BA, Aerden LA, Dallinga RJ, Visser MC, Bot JC, Vroomen PC, Eshghi O, Schreuder TH, Heijboer RJ, Keizer K, Tielbeek AV, den Hertog HM, Gerrits DG, van den Berg-Vos RM, Karas GB, Steyerberg EW, Flach HZ, Marquering HA, Sprengers ME, Jenniskens SF, Beenen LF, van den Berg R, Koudstaal PJ, van Zwam WH, Roos YB, van der Lugt A, van Oostenbrugge RJ, Majoie CB, Dippel DW (2015) A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 372(11–20):1
|
|