A rapid mixed-methods evaluation of remote home monitoring models during the COVID-19 pandemic in England

Abstract

Background Remote home monitoring services were developed and implemented for patients with COVID-19 during the pandemic. Patients monitored blood oxygen saturation and other readings (e.g. temperature) at home and were escalated as necessary. Objective To evaluate effectiveness, costs, implementation, and staff and patient experiences (including disparities and mode) of COVID-19 remote home monitoring services in England during the COVID-19 pandemic (waves 1 and 2). Methods A rapid mixed-methods evaluation, conducted in two phases. Phase 1 (July–August 2020) comprised a rapid systematic review, implementation and economic analysis study (in eight sites). Phase 2 (January–June 2021) comprised a large-scale, multisite, mixed-methods study of effectiveness, costs, implementation and patient/staff experience, using national data sets, surveys (28 sites) and interviews (17 sites). Results Phase 1 Findings from the review and empirical study indicated that these services have been implemented worldwide and vary substantially. Empirical findings highlighted that communication, appropriate information and multiple modes of monitoring facilitated implementation; barriers included unclear referral processes, workforce availability and lack of administrative support. Phase 2 We received surveys from 292 staff (39% response rate) and 1069 patients/carers (18% response rate). We conducted interviews with 58 staff, 62 patients/carers and 5 national leads. Despite national roll-out, enrolment to services was lower than expected (average enrolment across 37 clinical commissioning groups judged to have completed data was 8.7%). There was large variability in implementation of services, influenced by patient (e.g. local population needs), workforce (e.g. workload), organisational (e.g. collaboration) and resource (e.g. software) factors. We found that for every 10% increase in enrolment to the programme, mortality was reduced by 2% (95% confidence interval: 4% reduction to 1% increase), admissions increased by 3% (−1% to 7%), in-hospital mortality fell by 3% (−8% to 3%) and lengths of stay increased by 1.8% (−1.2% to 4.9%). None of these results are statistically significant. We found slightly longer hospital lengths of stay associated with virtual ward services (adjusted incidence rate ratio 1.05, 95% confidence interval 1.01 to 1.09), and no statistically significant impact on subsequent COVID-19 readmissions (adjusted odds ratio 0.95, 95% confidence interval 0.89 to 1.02). Low patient enrolment rates and incomplete data may have affected chances of detecting possible impact. The mean running cost per patient varied for different types of service and mode; and was driven by the number and grade of staff. Staff, patients and carers generally reported positive experiences of services. Services were easy to deliver but staff needed additional training. Staff knowledge/confidence, NHS resources/workload, dynamics between multidisciplinary team members and patients’ engagement with the service (e.g. using the oximeter to record and submit readings) influenced delivery. Patients and carers felt services and human contact received reassured them and were easy to engage with. Engagement was conditional on patient, support, resource and service factors. Many sites designed services to suit the needs of their local population. Despite adaptations, disparities were reported across some patient groups. For example, older adults and patients from ethnic minorities reported more difficulties engaging with the service. Tech-enabled models helped to manage large patient groups but did not completely replace phone calls. Limitations Limitations included data completeness, inability to link data on service use to outcomes at a patient level, low survey response rates and under-representation of some patient groups. Future work Further research should consider the long-term impact and cost-effectiveness of these services and the appropriateness of different models for different groups of patients. Conclusions We were not able to find quantitative evidence that COVID-19 remote home monitoring services have been effective. However, low enrolment rates, incomplete data and varied implementation reduced our chances of detecting any impact that may have existed. While services were viewed positively by staff and patients, barriers to implementation, delivery and engagement should be considered. Study registration This study is registered with the ISRCTN (14962466). Funding This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (RSET: 16/138/17; BRACE: 16/138/31) and NHSEI and will be published in full in Health and Social Care Delivery Research; Vol. 11, No. 13. See the NIHR Journals Library website for further project information. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.