A rapid mixed-methods evaluation of remote home monitoring models during the COVID-19 pandemic in England

Phase 1

The phase 1 study comprised:

1. an international rapid systematic review of remote home monitoring services for patients with COVID-1932

2. a rapid study of implementation of remote home monitoring services for patients with COVID-19 during the first wave of the coronavirus pandemic in England (mixed methods). 36

The rapid systematic review aimed to analyse the implementation and impact of remote home monitoring models for patients with COVID-19.

The empirical implementation study aimed to develop a conceptual map of remote home monitoring models, to explore the experiences of staff implementing these models during the COVID-19 pandemic, to document impact of services, to understand the use of data for monitoring progress against outcomes, and document variability in staffing and resource allocation.

Phase 2

Building on phase 1, we conducted a multisite, mixed methods national evaluation involving 28 purposively selected COVID-19 remote home monitoring services (October 2020 to November 2021).

This evaluation aimed to explore effectiveness, costs, implementation and experience (staff/patients) of COVID-19 remote home monitoring, and comprised four workstreams:

1. effectiveness

2. cost analysis

3. national survey of implementation and staff and patient experience

4. in-depth case studies of implementation and staff and patient experience.

Phase 1

Systematic review:

1. What are the aims of remote home monitoring models?

2. What are the main components of these models?

3. What are the patient populations considered appropriate for remote monitoring?

4. How is patient deterioration determined and flagged?

5. What are the expected outcomes of implementing remote home monitoring?

6. How have these models been evaluated?

7. What are the benefits and limitations of implementing these models?

Implementation study:

1. What were the conceptual models guiding the implementation of remote home monitoring models during the COVID-19 pandemic?

2. What were the processes that acted as barriers and facilitators in the design and implementation of pilots of these models during wave 1 of the pandemic?

3. What were the expected outcomes of the virtual wards implemented during wave 1 of the pandemic?

4. What data were collected by pilot sites and how has it helped them monitor progress against their expected outcomes?

5. What quantitative evidence did the sites use from national and international experiences of these models to help inform clinical management decisions?

6. How were resources allocated (including staffing models) to implement the remote home monitoring pilots during wave 1 of the pandemic?

7. What are the lessons learned from implementing remote home monitoring models during wave 1 of the pandemic? Can some of these lessons be used for planning care delivery for winter 2020–21?

Phase 2

1. What was the impact of remote home monitoring for patients with COVID-19 on mortality and use of hospital services? (workstream 1)

2. What were the costs of setting up and running CO@h and CVW models? (workstream 2)

3. What were the factors influencing delivery and implementation of remote home monitoring for patients with COVID-19? (workstream 3)

4. What were the experiences and behaviours (i.e. engagement, use of other services) of patients receiving remote home monitoring for patients with COVID-19? (workstream 3)

5. Were there potential impacts of remote home monitoring for patients with COVID-19 on existing health inequalities? (workstream 3)

6. What were the experiences of staff delivering remote home monitoring for patients with COVID-19? (workstream 3)

See Chapter 2 for protocol deviations.

Methods

Methods

Procedure

The evaluation consisted of four workstreams (see Figure 2).

FIGURE 2.

Overview of the methods for each workstream.

Workstream 1: the clinical effectiveness of COVID-19 remote home monitoring services

Workstream 1 aimed to use combinations of aggregate population-level data and hospital administrative data to both describe and estimate the impact of the roll-out of remote home monitoring models for CO@h and CVW services.

Study design

The study of the impact of CO@h on mortality and admissions was designed as an area-level analysis combining aggregated data from different sources. The aim was to use these data as time series to enable us to investigate ‘dose–response’ relationships between the evolving rates of enrolment to the services within each area and their outcomes, ‘enrolment rates’ being defined as the ratio of people enrolled on to services to the number of new cases each fortnight. For the in-hospital outcomes, we used an observational design relating in-hospital mortality and lengths of stay at an individual patient level to the rate of enrolment to CO@h services within the area at the time of admission.

For the analysis of impact of virtual ward services, we designed an admission-level multivariate regression analyses for patients with COVID-19 being discharged from hospital. The aim was to examine the association between various factors – including the availability of a virtual ward – with the length of COVID-19 inpatient spells, and likelihood of subsequent COVID-19 readmissions.

Setting and population

We evaluated the CO@h service across the CCG areas in England (N = 37) where we judged there was complete data on numbers of people enrolled on to services (onboarded) between 2 November 2020 and 21 February 2021. The study population included anyone aged 65 years or over with a laboratory-confirmed positive test for COVID-19 and any hospital admission within that age group for COVID-19 or suspected COVID-19.

For the virtual ward study, we extracted information from inpatient Hospital Episode Statistics (HES) data on all individuals discharged alive from any of 123 English hospital trusts where there had been a confirmed or suspected COVID-19 International Statistical Classification of Diseases and Related Health Problems, 10th revision (ICD-10) primary diagnosis code recorded during the stay. We included all patients discharged between 17 August 2020 and 28 February 2021, a period covering the beginning, the peak, and the start of the decline of England’s second COVID wave. Where a patient had two or more relevant inpatient stays, all stays were included in our analysis.

Data sets used

For our analysis we used data from a number of sources (see Report Supplementary Material 1).

Procedure and data analysis

For CO@h we analysed four main outcomes: mortality from COVID-19, hospital admissions for people with confirmed or suspected COVID-19, in-hospital mortality for these admissions and their lengths of stay. Because we were using population-level data, we first calculated enrolment rates for CO@h over time and then used multivariate regression models to investigate relationships between levels of enrolment and outcomes.

To analyse outcomes for patients admitted to hospital, we used individual-level HES. Again, we used multivariate regression models relating enrolment rates to outcomes, adjusting for individual patient characteristics (see Report Supplementary Material 1).

For the CVW analysis we developed multivariate regression models to examine the impact of the availability of CVW on two outcomes: (1) the LOS of the COVID-19 inpatient stay, and (2) on subsequent readmissions for COVID-19. Our models adjusted for the following person, inpatient spell, and trust characteristics: age at admission, gender, ethnic group, Charlson Comorbidity Index category,54 whether the stay was the person’s first COVID-19 hospital stay, whether the inpatient stay was an emergency admission, Index of Multiple Deprivation (IMD) decile, the time period of the discharge date, hospital trust, and a trust-and-week-specific measure giving the proportion of all acute beds occupied by patients with COVID-19.

Further details are provided in Report Supplementary Material 1.

Workstream 2: economic analysis

The aim of workstream 2 was to conduct a cost analysis of remote home monitoring that comprised an analysis of setting up and running remote home monitoring services for patients with COVID-19. As described in the study protocol, we originally planned to undertake a cost–utility analysis of remote home monitoring services for patients with COVID-19. 55 However, in the absence of any impacts on mortality or hospitalisation being demonstrated by the effectiveness analysis (workstream 1), a cost–utility was considered uninformative and our analysis focused only on analysing the costs of setting up CO@h and CVW services and mean running cost per patient monitored.

Procedure and data analysis

Remote home monitoring models were costed using data on staff (clinical and non-clinical) and non-staff resource use during the second wave. Contacts at each of the 28 sites (see workstream 3/4 for further details) were asked to complete a form in Microsoft Word (Microsoft Corporation, Redmond, WA, USA) (see Project website documentation – Study instruments) and 26 sites returned it electronically to the researchers. The data collection process was conducted between April and June 2021. Information was collected on staff (including the number of staff, their function/seniority and the number of hours worked) and non-staff resources used for setting up and running the sites (including information on the type and costs of digital monitoring platform used, oximeters, other medical equipment, etc.). We also collected information regarding the number of patients involved (triaged, monitored, deteriorated, escalated and deaths). The cost analysis was conducted separately for CO@h and CVW services.

Costs of setting up remote home monitoring services for patients with COVID-19 We calculated the costs incurred for setting up and running the CO@h and CVW models, also examining the variation by whether data submission was undertaken using technology-enabled with analogue modes versus analogue-only modes. The set-up costs were reported separately as mean costs per site. Note that the lifetime of the sites extends beyond the period observed in the study (e.g. some of the equipment, such as digital platforms and thermometers, can be used for longer than the study time) and therefore we did not calculate the mean set-up costs per patient or per unit of time. The running costs were calculated as mean costs per patient monitored and included the costs of staff (clinical and non-clinical) who were involved in running the services, and non-staff costs including costs of pulse oximeters (based on the number of patients monitored and assuming that 70% of pulse oximeters would be returned and reused) and other monthly expenditures related to maintaining the digital platforms. We adjusted the resources used and costs incurred per patient by applying weights based on the number of patients monitored by each service (CO@h and CVW) and the total numbers of patients monitored at each site. To investigate the effect of the type of service, data submission mode, seniority of full-time equivalent (FTE) staff and the total number of patients monitored factors on the mean running cost, we have used an ordinary least square regression model.

Technology-enabled versus analogue-only modes To explore the differences in staff, resources, types of patients and patient outcomes further between tech-enabled and analogue versus analogue-only data submission modes, additional data were collected from four of the 28 sites. All of these four sites used both tech-enabled and analogue data submission modes. We conducted an in-depth analysis of the time spent per patient for specific activities of the programme and calculated the cost per patient for all the activities. The mean cost for each activity per patient was calculated based on the time spent for each activity and staff working on each activity.

Workstreams 3 and 4: implementation and patient and staff experiences

The aim of these workstreams was to (1) understand the development of national COVID-19 remote home monitoring services and, (2) analyse the implementation of COVID-19 remote home monitoring services, and patient and staff experiences of care.

These workstreams included data from: staff surveys and interviews, patient/carer surveys and interviews, interviews with national leaders and documentary analysis of relevant national operating procedures (see Report Supplementary Material 1 for detailed methods).

Sample and recruitment

Selection of sites Twenty-eight services were included in our national evaluation. To obtain maximum variation, we sampled services based on a range of criteria, including the setting (primary care or secondary care), type of model (pre-hospital, early discharge, both), mechanism for patient monitoring (paper-based, app, both), geographic location (across different areas of the country), timing of implementation (implemented since wave 1 of the pandemic or recently implemented) and involvement in the evaluation with the other evaluation partners (Imperial and IAU).

To conduct an in-depth analysis of implementation, staff and patient experiences, 17 of these sites were selected as in-depth case study sites using the aforementioned criteria. In four of these sites, we also conducted in-depth comparisons of technology-enabled versus analogue models (see Economic analysis above). These sites were identified by NHSX to ensure inclusion of different digital platforms.

Sites were recruited through an expression of interest process whereby we presented our study at local and national meetings and asked sites to express interest in participating. Clinical Research Networks facilitated the set-up of sites and local governance approvals. Some sites were identified through our phase 1 evaluation.

Staff (survey and interviews) We conducted a survey of staff involved in delivering COVID-19 remote home monitoring services in 28 services (including clinical leads, delivery staff and data staff). The study coordinator at participating sites distributed links to the online surveys to staff involved in delivering/leading services via e-mail.

For staff interviews, we purposively sampled staff involved in delivering services, leading services and data processes at each of the 17 sites. Potential participants were approached by staff at participating sites and were introduced to the researcher or asked to contact the researcher if they would like to take part. The researcher sent potential participants an information sheet. If they agreed to participate, they were asked to sign a consent form.

Patients and carers (survey and interviews) To take part in our evaluation, participants needed to be

• 18 or over

• proficient in English (or one of Polish, Bengali, Urdu, Punjabi, French and Portuguese)

• eligible to receive remote home monitoring services for COVID-19, and must also have been offered and received COVID-19 remote home monitoring services.

We were flexible within our sampling to consider both national and local eligibility criteria.

A total of 25 of the 28 sites agreed to run the patient survey. NHS staff from participating services sent the patient survey to patients and carers either via post, text or e-mail. Patients/carers completed the surveys and returned these to the researchers either electronically or via post.

For patient interviews, we aimed to purposively sample four to six patients or carers who received services or declined/disengaged from services at each of the 17 case study sites. We sampled using a range of criteria including age, gender, ethnicity, employment status, mechanism for onboarding, type of monitoring and outcomes. Potential participants were approached by staff at participating sites and were asked if they would be interested in taking part. Participants were then sent an information sheet by the researcher. If the patient/carer agreed to take part in the study, they were asked to sign consent forms and send them to the researcher ahead of the interview.

National lead interviews and documentary analysis To understand development of the national service and capture changes in design and implementation over time, we conducted interviews with national leaders who were purposively selected from different roles across organisations. We also analysed key documents (e.g. SOPs).

For the national lead interviews, national leads were contacted via e-mail and asked if they would like to take part. If happy to take part, they were asked to give their consent in advance of the interview. Researchers then arranged a convenient time for the interview.

Measures

Staff (survey and interviews) The staff survey was developed for this study. We developed different questions for different groups (service leads and staff delivering the service). The staff survey included questions about experiences delivering remote home monitoring services for patients with COVID-19. The questions were reviewed and piloted prior to use (see Project website documentation – Study instruments for survey questions).

We developed a topic guide for staff interviews. Different questions were developed for different groups: (1) service leads, (2) staff delivering the service, (3) data. The topic guide included questions about the service, experiences of delivering services and views on patient engagement (see Project website documentation – Study instruments for topic guide). Interviews at four of the 17 sites included think-aloud questions about using the technology.

Patients/carers (survey and interviews) The patient/carer survey was developed for this study. The survey included questions about experiences receiving remote home monitoring services for patients with COVID-19. The questions were reviewed and piloted prior to use (see Project website documentation – Study instruments for survey questions).

We developed a topic guide for patient/carer interviews. Different questions were developed for those who received the service, and those who declined or disengaged. The topic guide included questions on their experiences of receiving services and engagement (see Project website documentation – Study instruments for topic guide).

Survey and interview questions were informed by relevant service documentation,38,39 theoretical frameworks relating to social, political and technical contexts49–53 and behaviour,56 and previous literature on engagement. 57,58 Questions on demographic characteristics were informed by previous literature. 59–64

National leads (interviews) We developed a topic guide for national lead interviews. This included questions about the service, leadership and governance, data and implementation (see Project website documentation – Study instruments for topic guide).

Data collection

Staff (survey and interviews) Staff were asked to follow an online link to complete the survey. The survey began with an information sheet and consent form. Survey sites were asked to keep a record of the number of surveys sent to determine response rates.

Interviews were conducted by one of six postdoctoral researchers (MS, CV, HW, LH, IL, JB); each site had a different lead researcher. Interviews were carried out via telephone or an online platform as preferred by the participant.

Patients/carers (survey and interviews) Patients were asked to complete the survey in one of two ways: (1) if monitored through use of a tech-enabled mode, patients/carers were e-mailed or texted the link to the online survey, or (2) if the patient was monitored using analogue methods, they received the survey in the post, together with a pre-paid envelope. Surveys were distributed at onboarding to the remote home monitoring service or discharge from the remote home monitoring service, depending on service procedures. The survey included an information sheet and consent form.

Interviews were conducted by one of six postdoctoral researchers (MS, CV, HW, LH, IL, JB); each site had a different lead researcher. Interviews were carried out via telephone or an online platform as preferred by the participant.

National leads (interviews) National lead interviews were conducted by four researchers (MS, CV, HW, NJF). Interviews were carried out via telephone or an online platform such as Zoom (Zoom Video Communications, San Jose, CA, USA) or Microsoft Teams (MS Teams) (Microsoft Corporation, Redmond, WA, USA) as preferred by the participant.

Data management Surveys (patients and staff) Electronic surveys were transferred directly into Research Electronic Data Capture (REDCap, IBM Corporation, Armonk, NY). Surveys received via post were returned to the Nuffield Trust or UCL offices, stored securely in locked filing cabinets and inputted into REDCap by members of the research team. Data were securely stored on UCL’s Data Safe Haven portal.

Interviews (national lead, patients/carers and staff) All interviews were audio-recorded (with consent), transcribed verbatim, anonymised and stored securely.

Analysis

Site characteristics Sites were characterised with respect to their population size,65 the proportion in urban versus rural areas,66 the proportion in the most and least deprived areas (with respect to national quintiles),67 and ethnicity. 68 For sites based on CCG areas we calculated these characteristics using publicly available data at lower-layer super output area (LSOA) level mapped to CCGs,69 while for trust-based sites we used data derived from inpatient HES admissions during the financial year 2019–20, in addition to web searches for the trust catchment populations.

Surveys (patients/carers and staff) Survey data were analysed using descriptive statistics, multivariate and univariate analyses on Statistical Package for the Social Sciences (SPSS, version 25). Participating sites were provided with a report on the patient survey responses from their site compared with overall findings.

Interviews (national lead, patients/carers and staff) Data collection and analysis was facilitated through the use of rapid assessment procedure (RAP) sheets. 70 RAP sheets were developed per site to facilitate cross-case comparisons and per population (to make comparisons between subgroups). The categories used in the RAP sheets were based on the questions included in the interview topic guide, maintaining flexibility to add categories as the study progressed. RAP sheets were used to develop coding frameworks to explore themes and subthemes in more depth and identify example quotes. Further details on analysis methods are reported within each chapter (see Chapters 4 and 8–11).

Although our study was rapid, many steps were taken to ensure the transparency, rigour and robustness of our qualitative research methods. For example, our study used purposive sampling to recruit a large number of participants with a range of characteristics. In terms of analysis, our study used both rapid analysis methods (e.g. RAP),70 together with traditional qualitative analysis methods (e.g. in-depth coding of transcripts and development of themes and subthemes). Our study involved a range of experienced qualitative researchers, and our analysis was strengthened by continuous team discussions about the interpretation of our data. We also discussed findings with those providing the service and our clinical advisory group to sense check our interpretation. Additionally, our findings were triangulated with quantitative, economic and other qualitative analyses. Our methodological approaches are consistent with previous recommendations on conducting trustworthy, robust and rigorous qualitative research. 71–74

Integration

Data from all four workstreams were integrated to answer our research questions and to facilitate processes of triangulation. The study team met weekly throughout the study. We included findings from each workstream within our RAP sheet. We produced RAP sheets at site level and also at population level (including for subgroups of staff and patients). For example, findings on local barriers and facilitators to implementation and patient and staff experiences helped the interpretation of findings on outcomes, service use and costs. Quantitative data on resource allocation were understood in relation to qualitative data on staff experiences of planning and delivering services. Data from the case studies were used to explain the survey findings (representing experiences and trends at a national scale). In addition, emerging findings from this study were discussed in relation to those from the two other evaluation partners (Imperial and IAU).

Deviations from protocol

We planned to undertake a cost–utility analysis of the COVID-19 remote home monitoring services but in the absence of any impacts on mortality or hospitalisation, as demonstrated by the effectiveness analysis (see Chapters 5 and 6), the economic analysis only included the costs of setting up and running services (see Chapter 7).

We intended to conduct ‘think aloud’ sessions with patients who had used tech-enabled data submission from the four sites selected for in-depth analysis of tech-enabled platforms. However, patients did not have access to the platforms after discharge and recall of the use of these platforms during their illness was poor. Consequently, the ‘think aloud’ methodology was discontinued for patient interviews.

Patient and public involvement

Members of the study team met with service user and public members of the BRACE PPI group and health and care panel and patient representatives from RSET in a series of workshops. These workshops informed study design, data collection tools, data interpretation and to dissemination for phase 2. For example, patient-facing documents, such as consent forms, topic guides, patient survey and patient information sheets were reviewed by this group. Additionally, PPI members helped to pilot patient surveys and interview guides with the research team. We also asked some members of the public to pilot the patient survey.

Dissemination and formative feedback

Throughout the project, we regularly shared feedback with stakeholders on emerging findings and lessons learned. The study will inform clinical practice for COVID-19 but will also inform the use of remote monitoring models in other conditions and areas of the NHS. Dissemination has been facilitated through various existing and new networks. Findings have been and will be published in peer-reviewed journals.

What was already known

• Previous research has explored the use of remote home monitoring models for non-COVID-19 conditions.

• COVID-19 remote home monitoring services were implemented rapidly around the world, but implementation processes were rarely documented.

What this chapter adds

• The COVID-19 remote home monitoring services varied in relation to the health-care settings and mechanisms used for patient triage, monitoring and escalation.

• Good communication within clinical teams, culturally appropriate information for patients/carers and the combination of multiple approaches for patient monitoring (app and paper-based) were considered facilitators in implementation.

• We could not reach substantive conclusions regarding patient safety and the identification of early deterioration due to lack of standardised reporting and missing data.

Methods

Results

We present a summary of findings below (see Vindrola-Padros et al. for further information). 32

Strengths and limitations of review

The last search was carried on 5 February 2021, so any articles published after this date were not included. However, the inclusion of preprints helped to mitigate this issue. Although we employed multiple broad search terms, it is possible that we missed articles that did not use these terms. Owing to the variability in study designs and the descriptive nature of the articles we did not assess these articles for quality using standardised tools for assessment. However, we feel it is important to note that we found several cases of missing data and inconsistencies in the reporting of evaluations that would lead to low quality ratings.

Future research areas identified

The review highlighted that further research should focus on exploring patient experience, implementation, outcome evaluation and cost-effectiveness.

Methods

Findings

We present a summary of findings below (see Vindrola-Padros et al. 36).

Strengths and limitations

The study was limited in the sense that we could not determine the effectiveness of these models due to lack of comparator data.

Future research areas identified

We identified the sustainability of the models and patient experience (considering the extent to which some of the models exacerbate existing inequalities in access to care) as future areas of research.

Impact

A senior director at NHSEI provided the following feedback on how findings from phase 1 of our evaluation influenced the national roll-out of COVID-19 remote home monitoring services in England:

1. Setting very simple entry criteria [65 and over OR clinically extremely vulnerable (CEV)].

2. Ensured that we did not go down a tech-only or tech-first route.

3. Produced translations of all the patient facing guidance and an easy-read version at the outset. It is possible that we may have missed the importance of this point.

4. National SOP was focused on patient self-escalation and our workforce calculations were informed by this work.

5. Created a minimum standard for out-of-hours to ensure that the links were clear.

6. We required services to be established within two months and had confidence that we could encourage all CCGs to set up by 31 December.

7. Ensured that we were able to get pulse oximeters out to CCGs within three working days.

8. Provide CCGs with a feed of patients that have tested positive and identify people within the cohort (65 years and over or CEV).

Informing phase 2 of the evaluation

The findings from phase 1 of the evaluation were also used to inform the design of phase 2, including the selection of outcome measures, cost analysis and identification of key areas of focus such as: the sustainability of these models during the second and third waves of the pandemic, patient and carer experiences with the services and how these varied according to the type of model (including analogue vs. digital models), factors that could have created or exacerbated inequalities in health and access to care and the costs involved in delivering these services.

What was already known

• COVID-19 remote home monitoring services were implemented internationally during wave 1 of the pandemic.

• During our phase 1 evaluation (wave 1 of the pandemic), some facilitators and barriers to COVID-19 remote home monitoring services were identified.

• It is important to explore local variation to fully understand implementation and to identify factors that might explain variation in effectiveness and cost outcomes across service models.

What this chapter adds

• Rapid development of national remote home monitoring services took place in three phases: (1) local development, (2) national development and roll-out, and (3) local implementation.

• Patient enrolment to COVID-19 remote home monitoring services across England was lower than expected.

• There was large variability in the models implemented for COVID-19 remote home monitoring services.

• Variation was influenced by patient, workforce, organisational and resource factors, and the rapid and evolving COVID-19 situation.

• These factors might help to explain variation in outcomes/effectiveness of different service models.

Design

This study draws upon findings from all four workstreams (see Figure 3).

FIGURE 3.

A summary of how data from each of the four workstreams contributed to Chapter 4.

Sample and recruitment

Twenty-eight sites were recruited to our study. Sites were included from all regions across England, represented CCGs and trusts, varied in urban/rural, deprivation scores, ethnicity, and size (see Chapter 2, Report Supplementary Material 2).

Data collection

To identify when new services started and to estimate the proportion of new diagnosed cases who were onboarded (enrolment rates), we drew on a selection of data sources (see Report Supplementary Material 1), including Kent, Surrey and Sussex AHSN (KSS AHSN) data (see Chapter 6), bespoke onboarding data and data on diagnoses of COVID-19 provided by PHE (now the UK Health Security Agency).

To describe the development and roll-out of services we conducted interviews with national leads (see Chapter 2). To explore implementation, we conducted a documentary analysis of service documentation, surveys with patients and staff and interviews with patients, carers and staff, and a cost survey (see Chapter 2).

Data analysis

Participant characteristics

We received 292 staff surveys (39% response rate) across 28 sites and 1069 patient and carer surveys (18% response rate) across 25 sites (see Report Supplementary Material 2 for demographic characteristics). Interviews were conducted with national leads (n = 5), staff (n = 58 across 17 sites), and patients or carers (n = 62 across 17 sites). See Report Supplementary Material 2 for demographic characteristics.

An overview of the development of COVID-19 remote home monitoring services including the national roll-out and the uptake/coverage of the resultant service

Within this section, we first present an overview of the three phases of development and implementation of these services. Secondly, we present the uptake/coverage of the final service.

Development of COVID-19 remote home monitoring services

The development of COVID-19 remote home monitoring services occurred rapidly, in three stages: (1) local development and implementation, (2) national development and roll-out, and (3) local implementation (see Figure 4 for timeline and key groups involved in each phase).

FIGURE 4.

Development of COVID-19 remote home monitoring services.

Local development and implementation – wave 1 and the first use of pulse oximetry in patients’ homes

Local development and implementation

The use of oximetry to monitor patients with COVID-19 in their home began in wave 1 as several, initially disconnected, local services were established, motivated by attempts to mitigate silent hypoxia (very low oxygen saturations, often without breathlessness). Pulse oximetry was initially used to monitor patients in the community. 36 During phase 1, local clinical leaders within CCGs and secondary care trusts had identified a need for this service and were integral in facilitating the initial development, set-up and implementation of services, and instigated the development of an informal community of practice. NHSX supported a few of the pilot sites to use tech-enabled models by putting them in touch with tech companies. The local development and implementation of services gained interest from national stakeholders.

An evaluation of phase 1 was commissioned and conducted by researchers from NIHR RSET and NIHR BRACE (see Chapter 3). The purpose of this evaluation was to provide lessons for the ongoing development/roll-out of the services.

Supporting local implementation

Two learning communities were set up prior to the national roll-out, to support local implementation: a community of practice and a national learning network.

The community of practice was an informal group, led by key national clinical and policy leads and attended by national and local clinical leaders for local services adopting these pulse oximetry services. The community of practice held regular online meetings (every couple of weeks) to share best practice quickly from national leads to local teams, and between local teams, offering mutual support and leadership. This community of practice grew rapidly.

A national learning network (the National Deterioration and COVID-19 Forum) was set up by the AHSN patient safety collaboratives to support clinical leads and provide resources to facilitate implementation of these services. The set-up of the national learning network was prompted through NHS @home17 (via the national team and regional medical directors). Additionally, regular webinars were held to share learning. 97 The learning network was hosted within the Future NHS collaboration platform (an online platform designed to help services to co-develop services, share materials and work together). 98

The conversations and shared learning from both these networks played a key role in shaping the basic pathway underpinning these services. These communities were set up to support local clinical leads and share experiences.

National development and roll-out

National development

Between wave 1 and 2, there was a period of gathering evidence (research and clinical consensus), discussions around how services should be developed and development of safety netting guidance. This evidence included our evaluation findings from phase 1 (see Chapter 3).

During the summer and autumn of 2020, NHSEI began to develop SOPs for CO@h, with leadership and governance from the NHS @home team. The National Incident Response Board approved operationalisation and national roll-out of the CO@h service as the appropriate next step.

The SOP for CO@h38 (see Appendix 1, Table 24) that emerged was informed by the learning gained from previous services which provided care at home, experience gained from managing wave 1 in the community including the various pilots and case studies. It was shared through the NHS Futures website98 and the pre-existing evidence base. 32,36 It was intended that services would be inclusive by design and enabled by, but not dependent on, technology (to remain accessible to the most vulnerable patient groups).

In January 2021, a SOP for CVW (early discharge from hospital) was published and these services were rolled out. 39 Virtual ward models were designed to be run by acute trusts, with responsibility for services being with secondary care services39 (see Appendix 1, Table 24).

National roll-out

Following service development, the AHSNs were commissioned through the AHSN’s patient safety collaboratives to facilitate regional roll-out. 97 A programme board with all key stakeholders was established and met weekly to discuss a number of workstreams including data management, workforce/training, patient safety, operations, pathways, evaluation and oximeter distribution. The programme was monitored via comprehensive data collection for CO@h which aimed to track how the programme was developing/performing with a focus on using the data as shared learning. As part of this process any safety incidents were recorded, using patient safety groups where appropriate.

National roll-out of CO@h and virtual ward services was supported by 20 highly attended regional launch events, which included question and answer sessions. To support national roll-out, national clinical leads and seven regional clinical leads were funded to support local implementation. NHSEI agreed to purchase pulse oximeters. NHSEI purchased 706,000 oximeters across waves 1 and 2. These pulse oximeters were CE marked (as recommended by MHRA guidance)37 and were available to local sites within three days through an ordering system set out in the SOP. NHS Digital supported data management. 99 NHSX were a partner in the roll-out and implementation of tech-enabled platforms within COVID-19 remote home monitoring services. NHSX committed to supporting (funding) up to 50 organisations to use a tech-enabled model for CO@h and/or virtual ward models during wave 2 (sites bid for this funding and then chose and procured the platform they wanted to use). CO@h and virtual ward services were rolled out between November 2020 and February 2021.

Local implementation

NHSEI were integral in supporting the local implementation of COVID-19 remote home monitoring services, including providing financial support for primary care to establish and implement these services,40 resourcing NHS Digital to supply data to service providers to identify people who may benefit from these services41 and the commissioning and funding of AHSN support.

National leads described the importance of local leads assuming responsibility and accountability for their services, guided by the SOPs for CO@h and CVW.

While there was some variation in responsibility and accountability, local CCGs were primarily responsible and accountable for delivery of CO@h and secondary care trusts were responsible and accountable for delivery of CVW. Each local service had its own local governance and management structure, which were expected to minimise national input and allow local services to be more responsive to and reflective of the needs of local populations. This local governance was overseen and supported by the AHSNs. NHSX managed data reporting on tech-enabled deployment.

With local services responsible for the day-to-day running of CO@h and CVW, services allocated resources, designed staffing models, and distributed equipment and educational materials. The management teams particularly of integrated services (i.e. where CO@h was combined with CVW) consisted of colleagues drawn from across multiple settings including primary and secondary care, but also the ambulance service, mental health trusts, care homes, and patient safety collaboratives. The precise mix depended upon the scope of services within the local environment. Regional ‘clinics’ were established by the local AHSNs to facilitate regular engagement and support the initial set-up. Two-way relationships were developed between policymakers and strategic management and front-line NHS workers as a way of sense-checking the work completed. Significant support and buy-in from senior management within acute trusts and across CCGs helped to set up COVID-19 remote home monitoring services. There was, however, a need to have supportive local authority and public health teams to integrate CO@h and CVW services with existing ways of data sharing.

Supporting local implementation

Throughout the national roll-out, local services were supported by the community of practice and the national learning network (as described earlier).

National coverage of COVID-19 remote home monitoring services and enrolment

COVID Oximetry @home

We judged that the national onboarding data for CO@h were complete for 37 of 135 CCGs (see Report Supplementary Material 1), so our analysis of coverage and enrolment was restricted to these areas.

The dates sites within each of the 37 CCGs started onboarding patients are plotted in Figure 5. Three of these CCGs were onboarding patients in October and all 37 were onboarding patients in the fortnight beginning 11 January 2021.

FIGURE 5.

Number of CCGs with sites operating the CO@h programme by fortnight (data for the 37 CCGs where the onboarding data were assumed to be complete).

Once services started operating and up until 4 April 2021, enrolment across the 37 CCGs among people aged 65 years or over is shown in Figure 6. The CCG with the highest enrolment rates had values that were more than twice the next highest; 22 CCGs (59%) enrolled fewer than 10%. The overall enrolment rate over the period for this age group across all 37 CCGs was 8.7%.

FIGURE 6.

Enrolment of CO@h services among people aged 65 years or over by CCG after implementation started up until 4 April 2021 (enrolment is defined as the ratio of numbers onboarded to new cases over the period and calculated from 19 October 2020 for any services in operation before then).

Virtual wards

Dates acute trusts opened their virtual wards are plotted in Figure 7. A majority of services (62%) launched in January and February 2021. For seven trust-based sites included in our costs survey (see Chapter 7), we were able to estimate an approximate enrolment proportion: that is, the proportion of patients with COVID-19 discharged who were monitored on a virtual ward. Two sites had fewer than 10% of patients monitored, one site had 20% and four had more than 50% (but below 65%) of patients monitored. Otherwise, with national SITREP data not useable because of data quality issues, we had no further information on virtual ward enrolment.

FIGURE 7.

Start dates of CVW services in England’s non-specialist acute hospital trusts (n = 123 trusts); 14 trusts had no CVW by 22 February 2021.

A description of the different typologies and implementation of services between sites

We created a typology (classification system) and categorised sites according to seven domains (see Table 1): (1) type of model (CO@h/CVW/Integrated); (2) sector leading services (primary care/secondary care/both); (3) type of monitoring (analogue-only (paper and telephone)/tech-enabled and analogue); (4) admission criteria (age and clinical vulnerability); (5) workforce (number and type of staff); (6) date service started; and (7) enrolment rates within the CCGs where the sites were located. As above, enrolment rates were defined as the ratio of patients onboarded to new cases among people aged 65 or over by CCG after implementation started up until 4 April 2021. It was calculated from 19 October 2020 for any services in operation before then.

Different models of COVID-19 remote home monitoring services were implemented across the 28 local sites. Below, we describe variability in relation to the stages of services outlined in the national SOPs (see Report Supplementary Material 4). 38,39

Factors influencing implementation of COVID-19 remote home monitoring services

We observed differences in the design and delivery of services between local sites and between the national guidance and local implementation. Implementation may have been influenced by a number of site-specific factors relating to patients, staff, the organisation delivering the programme, and the resources at their disposal (see Report Supplementary Material 4).

Staff factors

A number of staff-related factors influenced the way in which services varied between sites. These related to their level of training and experience, whether they worked remotely or in a shared workspace with more direct clinical oversight, and the overall workload they experienced (Box 2). For example, those that were redeployed to services tended to be nurses working in secondary care, whereas those asked to take on the responsibility of delivering these services alongside their current roles tended to be drawn from primary care.

The knowledge, skills and confidence of the staff charged with delivering services varied between sites, and these differences in confidence and knowledge were particularly evident when services relied on non-clinical staff. Across many sites, the routine availability of senior clinical oversight was deemed important.

Key findings

The rapid development of COVID-19 remote home monitoring services in England took place in three phases: (1) local development and implementation (in wave 1 and 2), (2) national development and roll-out (in wave 2), and (3) local implementation. The proportion of new diagnosed cases who were onboarded on to these services was lower than expected. The implementation of services varied between sites and from national guidance. A range of service typologies emerged across our evaluation including noticeable variation in relation to enrolment rates, the nature of onboarding, referral criteria, monitoring processes, and discharge and pulse oximeter return procedures. Patient factors (e.g. needs of the local population), staff factors (e.g. workload and work environment), organisational factors (e.g. staffing models and collaboration) and resource factors (e.g. software) influenced implementation and enrolment.

How findings relate to previous research

The implementation of COVID-19 remote home monitoring services in England can be seen as an example of major system change or ‘large system transformations’, defined as ‘coordinated, systemwide change affecting multiple organisations and care providers, with the goal of significant improvements in the efficiency of health-care delivery, the quality of patient care, and population-level patient outcomes’ (Best et al.,102 p. 422). A conceptual framework proposes five main stages of major system change: (1) decision to change influenced by drivers to change, governance and leadership of decision-making, (2) decision on which model to implement, (3) the implementation approach, (4) implementation outcomes, and (5) intervention outcomes. 103 There are different ways of implementing major system change (top-down and bottom-up approaches). Previous research suggests that a combination of both top-down and bottom-up approaches may be beneficial. 104 Our findings indicate that rapid implementation processes may follow a bottom-up (local services make decision to change and which model to implement), top-down (national service makes decision to change and provides guidance on which model to implement), then bottom-up process (local services interpret guidance from national service and decide which model to implement) (as described in Chapter 4). Therefore, rapid implementation may vary from major system change in non-rapid contexts (e.g. Fulop et al.). 103 However, our findings support previous research which indicates that this combination of approaches may impact on consistency of implementation approach and uptake. 105

Prior to this evaluation, there was limited evidence regarding how these services have been implemented. Our findings indicated that most remote home monitoring services studied had low rates of enrolment and that there was considerable variation in which model was implemented and how, according to local context. These findings support previous research demonstrating likelihood of variation when implementing national services generally81–83,89–92 and in response to the COVID-19 pandemic. 106–108 Previous research has highlighted differences in the models implemented and the ways in which services are implemented must be considered when interpreting effectiveness and cost-effectiveness of services. 57,81,83,84,88

Strengths and limitations

This study used mixed-methods data from a wide range of sources, including national quantitative data, cost data, national surveys and case study interviews. Additionally, this evaluation is a large rapid scale evaluation, including 28 health-care services throughout England. Therefore, the findings provide a comprehensive picture of the development, coverage and implementation of COVID-19 remote home monitoring services through phase 2 of the pandemic.

Given the rapid implementation of services, we are unable to say whether the degree of variation we witnessed is a result of the rapid roll-out or whether it would have been observed if lengthier implementation processes had been followed.

Different data sources were triangulated to analyse the findings presented in this chapter. Sometimes, different data sources or respondents disagreed and consensus needed to be reached. For example, across several sites, more than one clinical lead/service manager completed the staff survey and there were several instances of responses being inconsistent (to overcome this problem, we sought further information from services as necessary and selected one clinical lead per site using relevance of job title).

Our assessment of service enrolment within CCGs relied on their onboarding data being complete. To judge completeness, we relied on assessments made by individual CCGs, although as they were not necessarily the service providers, and therefore removed from the data entry process, we were not sure about the accuracy of this information in all cases. We were able to cross-check against the data provided by the 28 study sites themselves and found a reasonable match for most but not all, and we were unable to verify the data from the CCGs that were not study sites in the same way.

Implications

Our findings have implications for interpretation of quantitative outcomes of the evaluation. Our findings on low rates of enrolment and variation in implementation can be used to help interpret findings relating to effectiveness and cost of the services (as we discuss in Chapters 5–7). However, we were not able to look at how specific variation linked with particular service outcomes or enrolment more generally.

Several valuable lessons for service development have emerged from our work. Findings indicate that when developing and rapidly implementing national roll-out of services, service planners must consider the context and priorities of local organisations (e.g. their patient population characteristics and resources). It may be helpful to consult with patient groups to ensure that strategies to ensure reach are appropriate and feasible, and that services meet the needs of the local population. Flexibility of workforce is needed for those delivering the service.

Future research

Future research should explore how similar nationally implemented programmes vary from their intended design when not implemented at such speed and the degree to which local variation acts favourably in sustaining the service. Further evidence is needed on self-monitoring models of remote home monitoring for COVID-19, including the groups of patients that this might be appropriate for and whether its use would enable services to onboard more patients. Further research is needed to explore why enrolment was lower than expected, and how to improve enrolment to these services in future.

Conclusions

Local and national leadership were integral in the development and implementation of COVID-19 remote home monitoring services. Enrolment to these services was lower than expected. There was large variability in the models implemented and that variation may have been influenced by patient, workforce, organisational and resource factors. Together, low enrolment and variation in implementation indicate that it is difficult to evaluate the true extent of effectiveness of COVID-19 remote home monitoring services.

What was already known

• Previous quantitative studies have mostly assessed remote oximetry monitoring services for patients with COVID-19 at individual sites and focused on their safety.

• There is some existing evidence that remote monitoring of patients with COVID-19 can be locally effective, particularly among higher-risk patient groups. However, effectiveness of national programmes has been little studied.

What this chapter adds

• We have found no evidence that national implementation of pre-admission home monitoring of older people diagnosed with COVID-19 impacts on mortality, hospital admission rates, in-hospital mortality or length of hospital stay.

• Missing data and lower than expected enrolment of patients to services may have influenced our results, and the effectiveness of some local programmes could have been lost among the analysis of national data.

• Future implementation requires better data collection strategies and effective learning from areas that have achieved higher patient enrolment.

Data completeness and patient enrolment

As we mentioned in Chapter 2, because of the design of our study we were only able to evaluate services within 37 CCGs where we judged there was complete data on numbers of people enrolled on to services (onboarded) between 2 November 2020 and 21 February 2021. The study population included anyone aged 65 years or over with a laboratory-confirmed positive test for COVID-19 and any hospital admission within that age group for COVID-19 or suspected COVID-19.

The included CCGs had no notable differences in mean age, proportions of non-white population or proportions resident in most deprived areas when compared with the remaining 98 that were not included, although included CCGs had a lower incidence of positive test results (Table 2). There were also regional differences: no CCGs from the East NHS region were included, and only one from the North East and Yorkshire (see Figure 8). The South West, North West and Midlands were the best represented regions.

FIGURE 8.

Number of CCGs with complete data by NHS region (% complete shown in each bar).

Fortnightly patient enrolment to CO@h among people aged 65 years or over within the 37 CCGs is shown in Figure 9. Rates were particularly low earlier in the period because many sites had not commenced implementation. The numbers of CCGs where there were sites onboarding patients within each fortnight is shown along the horizontal axis. Sites within nine (24%) CCGs were operating services in the first fortnight, which had risen to 33 (89%) within the fortnight beginning 28 December 2020. Services were operating within all CCGs during the fortnight beginning 11 January 2021. The median enrolment rate only exceeded 10% in the final fortnight, although, from the end of November 2020, the maximum was consistently above 30%, with one or two CCGs each fortnight achieving much higher rates than the rest. The overall enrolment rate over the period across all 37 CCGs was 5.9%. If we exclude fortnights during which services were either not operating or operating for only part of the fortnight, the overall enrolment rate was 8.7% with only one CCG averaging a rate of more than 30%.

FIGURE 9.

Variation in enrolment rates of CO@h among people aged 65 years or over with laboratory-confirmed COVID-19 across the 37 CCGs included in the analysis.

Summary data and outcomes

Summary data and outcomes for the period from 2 November 2020 to 21 February 2021 for the 37 CCGs by age band are shown in Table 3. Although there were many more positive tests recorded among the under 65s, there were approximately half as many deaths and equivalent numbers of hospital admissions. Enrolment rates after implementation was low across both age groups and highest among those aged 65–79 years.

Time series plots of numbers of newly diagnosed cases and numbers onboarded to the programme within the selected 37 CCGs over the study period (2 November 2020 and 21 February 2021) are shown in Figures 10 and 11. New diagnoses peaked in the fortnight straddling the new year, while onboarding peaked in the following fortnight. Profiles of all COVID-19 deaths and admissions for confirmed or suspected COVID-19 are shown in Figures 12 and 13.

FIGURE 10.

New diagnoses of COVID-19 by fortnight; data for the 37 CCGs included within the analysis.

FIGURE 11.

People onboarded to the CO@h programme by fortnight; data for the 37 CCGs included within the analysis.

FIGURE 12.

All deaths from COVID-19 by fortnight within the 37 CCGs included in the analysis.

FIGURE 13.

Hospital admissions for confirmed or suspected COVID-19 by fortnight within the 37 CCGs included in the analysis.

Rates of in-hospital mortality and median lengths of stay for patients admitted to hospital with confirmed or suspected COVID-19 are shown in Figures 14 and 15. Mortality rates peak at approximately 30% over the new year and decline thereafter to around 21%. Median lengths of stay decline from the end of November 2020 while admission numbers increase.

FIGURE 14.

In-hospital mortality by fortnight for patients resident within the 37 CCGs and admitted with confirmed or suspected COVID-19.

FIGURE 15.

Median LOS by fortnight of admission for patients resident within the 37 CCGs and admitted with confirmed or suspected COVID-19.

The characteristics of patients admitted with confirmed or suspected COVID-19 over the study period are shown in Table 4.

Mortality and hospital admissions

Results from our models for mortality and hospital admissions are shown in Table 5, Figures 16 and 17. For every 10% increase in enrolment, mortality fell by 2% [relative risk = 0.98, 95% confidence interval (CI) 0.96 to 1.01] and admissions increased by 3% (relative risk = 1.03, 95% CI 0.99 to 1.07), but neither result is statistically significant. There is, however, a significantly higher risk of mortality and admission among the older age group (p < 0.001) and higher risk of mortality in the months following November 2020 (p < 0.001 for each month in comparison with November).

FIGURE 16.

Impact of each variable used in the multivariate model on the risk of death (mortality rate ratios). Error bars denote 95% CIs.

FIGURE 17.

Impact of each variable used in the multivariate model on the risk of hospital admission (admission rate ratios). Error bars denote 95% CIs.

We also found no significant relationship between rates of enrolment to CO@h and the ethnicity, sex, deprivation or health status of patients admitted to hospital with COVID-19 or suspected COVID-19 (Table 6). The impact on age is of borderline statistical significance with each 10% increase in enrolment associated with a reduction of nearly four months (95% CI −7.9 months to 0.2 months, p = 0.07).

In-hospital outcomes

The results of our analysis of in-hospital outcomes are shown in Table 7, Figure 18 and Figure 19. For every 10% increase in patient enrolment, in-hospital mortality fell by 3% (relative risk = 0.97, 95% CI 0.92 to 1.03) and LOS increased by 1.8% (95% CI −1.2% to 4.9%). Again, neither result is statistically significant. Non-white ethnicity and existing COVID-19 bed occupancy on admission were both associated with shorter lengths of stay.

FIGURE 18.

Impact of each variable used in the multivariate model on the risk of in-hospital mortality for people admitted with confirmed or suspected COVID-19 (OR). Error bars denote 95% CIs.

FIGURE 19.

Impact of each variable used in the multivariate model on LOS for people admitted with confirmed or suspected COVID-19 (% change). Error bars denote 95% CIs.

Sensitivity analysis

Results from sensitivity analyses are shown in Report Supplementary Material 1. None of the changes we made in our assumptions affected our findings with respect to the association between coverage of remote oximetry and outcomes.

Summary of findings

In this study, we have found no significant association between rates of patient enrolment to the CO@h programme, and COVID-19-associated mortality or COVID-19-associated admission to hospital. For such hospital admissions, we found no significant relationship to in-hospital mortality or LOS. Also, for COVID-19-associated admissions, there appears to have been no associated change in the patient characteristics we have tested.

Interpretation

There are several possible explanations for these results, and it would be premature to conclude that the CO@h programme has had no impact. Firstly, limitations in data completeness meant that we were only able to analyse onboarding data for one-quarter of CCGs across England. This, combined with lower than expected patient enrolment within these CCGs, meant that our ability to detect any possible impact was smaller than anticipated. This also meant that some areas of the country were under-represented. Secondly, individual-level associations may not have been seen at the aggregate level. Thirdly, qualitative findings from our wider study revealed that CO@h was implemented in a variety of ways by different services within CCGs, some of which may have had more impact on outcomes than others (see Chapter 4). The possible impact of the intervention on hospital admissions was always uncertain, since the aim of the programme was to ensure people who needed hospital treatment were admitted at the right time. Also, once in hospital, the determinants of LOS are complex and multifactorial, and may have varied during the time course of the second wave of COVID-19 infection in England.

Strengths and limitations

The anticipated value of our approach with aggregated data was that it would complement the other two simultaneous quantitative evaluations of CO@h,43,45 could be carried out more rapidly and, if COVID-19 continues to stretch national health services, it could be more readily repeated as new data become available, provided the right information is routinely collected at source.

We were able to handle small number suppression and the rounding of aggregate data by multiple random sampling throughout the range of possible values and it was encouraging to discover that this uncertainty did not have a large impact on results.

Using aggregated data has some limitations because it does not enable us to trace direct links between the onboarding of an individual and their outcomes. There may also be an ecological fallacy, where individual-level effects are not observable in the aggregated data. 111 However, obtaining linked individual-level data is a complex and potentially long process that may not always be feasible when there is a need to provide rapid feedback to a developing programme and where resources are stretched. Unfortunately, however, our ability to provide rapid feedback was compromised by delays in obtaining onboarding data which proved an understandable challenge for local services in the midst of a pandemic.

The wider mixed-methods study has added insight into some of our findings and provided locally collected bespoke data against which we could verify information we received centrally about patient enrolment and data completeness. However, these checks could only be made against the 28 study sites and we were not able to verify the data from the other CCGs in the same way. Findings from surveys and interviews have helped interpret what we have found and, conversely, our data analyses have helped provide a context against which to understand the relative importance of the qualitative findings (see Chapters 4 and 8–11).

The qualitative study has also shed light on the different ways that the services had been implemented. However, with just 12 of the 28 study sites being among the 37 included in our analysis, in combination with low enrolment, it was not possible to undertake a quantitative study of the relative performance of the different types of services implemented.

We anticipated that finding a suitable comparator group during the national implementation of a programme was likely to be problematic, and we therefore avoided this problem by treating the relationship between rates of enrolment and outcome as a dose–response. However, the power to detect any impact in such an analysis depends on the level of enrolment which, in practice, was lower than we hoped. This approach also assumes any selection bias with those being enrolled is negligible.

During the period of our analysis the vaccination programme was starting and by the end of our study period 88% of people aged 65 years or over had had at least one dose. 112 Although this study investigates outcomes of people after being diagnosed with COVID-19, there is evidence that vaccination changes the subsequent risks of mortality and hospital admission113 which could have had a confounding effect on our analysis.

In the context of fortnightly data, we assumed a minimal lag between the diagnosis of COVID-19 and onboarding to CO@h. However, there was evidence from sites that they sometimes encountered delays in identifying positive cases,114 although the overall impact on this assumption is uncertain.

Comparison with other studies

Prior to this study, very little was known about the quantitative impact of the use of pulse oximeters for COVID-19 remote home monitoring. One of the other evaluations of the CO@h programme in England also found no significant impact on mortality or health service utilisation. 43 However, the study did find reductions in mortality and increases in hospital attendance (yet with lower use of critical care) among people enrolled on to the programme after attending the emergency department. 42 A study of 4384 high-risk patients receiving home monitoring of vital signs, including pulse oximetry, in one region of Galicia, Spain, found lower admissions, lengths of hospital stays and in-hospital mortality when compared with other local regions. 115 A retrospective cohort study from South Africa evaluated the use of pulse oximetry for home monitoring of people diagnosed with COVID-19 and, as with the Spanish study, deemed to be of high clinical risk. 116 They found a significant improvement in mortality but no impact on admissions to hospital; the reduction in mortality being explained by earlier admission. A recent study of CO@h carried out at one site in England demonstrated reductions in 30-day mortality and lengths of stay among people admitted to hospital over the second wave. 117 The control group in this study were patients admitted with COVID-19 who were not on the CO@h programme and it is not clear whether that would be an inherently higher risk group. In another study implemented in the UK during the first wave, patients with suspected COVID-19 attending the emergency department were discharged home with an oximeter. They observed a reattendance rate of 4.7% compared with 22.7% among a retrospective control group. 118 However, this was a younger cohort (median age 41 years) and the absolute numbers of reattendance were small (nine in all). Other studies have reported on the safety of similar programmes, but have lacked comparators. 28,43,110,119–121

Implications and further research

At the start of this study, we anticipated that services would have higher rates of patient enrolment and that complete data would be available from more CCGs. Although the use of aggregated population-level data can enable more rapid evaluation of a new service, these two elements had an influence on the power of the analysis to detect an impact. The resulting shortfall in expected data reflects the challenges of trying to centrally manage a bespoke data collection while services are already stretched. However, sufficient quantities of data are vital to determining whether a service is effective, so it is important to understand how this can be improved, for example, by concentrating data collection in a few sites and using routinely collected data wherever possible.

Furthermore, low patient enrolment raises questions about capacity of both staff and resources amid high infection rates and how it is possible to secure the best value from such a service under the circumstances. The fact that at least one CCG managed to achieve reasonably good rates of enrolment indicates the possibility for learning from others.

Other studies, as described above, have provided evidence of positive impacts on high-risk cohorts, and similar findings may follow from focusing evaluation of the CO@h programme in the same way.

Conclusions

This study provides an evaluation of the national implementation of remote home monitoring of pulse oximetry for people diagnosed with COVID-19 across the English NHS. Although we detected no significant impact on outcomes, there are potential explanations for this finding that are unrelated to the effectiveness of the programme. Taking due account of populations that may respond less well to oximetry, there is no evidence that future implementation of similar programmes would be unsafe. However, the challenges of providing sufficient data so that effectiveness can be adequately measured need to be overcome.

What was already known

• Post-hospital discharge virtual wards have been found to have a positive impact on patient outcomes when focused on patients with specific diseases, for example those with heart disease. There has been less evidence of impact for more heterogenous groups of patients.

• There has been little evidence thus far that post-hospital discharge virtual wards (using pulse oximetry monitoring) have helped to reduce the LOS of patients hospitalised with COVID-19 or to change rates of subsequent readmissions for COVID-19.

What this chapter adds

• This national-scale study has found no evidence that the roll-out of post-hospital discharge virtual ward services for patients with COVID-19 in England reduced lengths of stay in hospital, or impacted rates of readmission.

• While there is currently an absence of evidence of positive impacts for patients with COVID-19 discharged to a virtual ward, our study emphasises the need for quality data to be collected as part of future service implementation.

Participants

We found 139,619 discharges between 17 August 2020 and 28 February 2021 where the patient had a primary diagnosis of COVID-19 recorded during their inpatient stay, and had not died in hospital. These discharges belonged to 129,461 individuals. Table 9 shows characteristics of these COVID-19 patient discharges.

The number of patients with COVID-19 discharged in this time across England (not limited to the 123 analysis trusts) was 142,216. Thus, our analysis includes 98.2% of recorded inpatient stays of patients in England with a primary COVID-19 diagnosis, discharged alive.

Table 9 also shows the characteristics of the patients split into two key analysis groups: patients discharged where no CVW was available (N = 81,174, 58.1%), and those discharged where a CVW was available (N = 58,445, 41.9%). These two sets of patients were well matched in terms of gender and comorbidities. Patients discharged where a CVW was available were more likely to be in the age range 50 to 64 and members of non-white ethnic groups, and less likely to be from the most deprived areas. As a consequence of the accelerated set-up of CVW services from early January 2021 (Figure 20), discharge dates of patients where a CVW was available tended to take place towards the end of the analysis period, with nearly four in five occurring on or after 4 January 2021 (compared with two in five of the discharges where a CVW was not available). Discharges of patients taking place where a CVW was available were more likely to be occurring in the context of higher levels of beds occupied by patients with COVID-19.

FIGURE 20.

Cumulative start dates of CVW services in England’s non-specialist acute hospital trusts (n = 123 trusts); 14 trusts had no CVW by 22 February 2021. The grey box represents the main period of this analysis.

COVID-19 length of stay

The mean LOS for patients discharged where a CVW was available was 10.1 days [standard deviation (SD) 11.3 days], compared with 8.9 days (standard deviation 10.3 days) for patients where a CVW was not available (Figure 21). This was an unadjusted incidence rate ratio (IRR) of 1.13, or 13% longer for discharges taking place from hospital trusts with a CVW. Distributions of LOS for COVID-19 for patients discharged with and without an available CVW are provided in Report Supplementary Material 5.

FIGURE 21.

Mean (unadjusted) length of COVID-19 hospitalisations and adjusted IRRs, with respect to categories of factors in multivariate model.

Fully adjusting for all covariates, we found an adjusted IRR of 1.05 (95% CI 1.01 to 1.09) for discharges where a CVW was available compared with those where a CVW was not available, indicating that the LOS was 5% longer for relevant COVID-19 inpatient stays where a CVW had been available.

We additionally found strong, generally positive gradients of LOS for increasing age and comorbidity (note that these relationships relate to all 139,619 discharges and not just those with a CVW available). Females had shorter lengths of stay than males (adjusted IRR 0.95, 95% CI 0.94 to 0.96), as did all non-white ethnic groupings, compared with the white group (e.g. Asian patients had an adjusted IRR of 0.89, 95% CI 0.87 to 0.91, and black patients an adjusted IRR of 0.93, 95% CI 0.90 to 0.95). Patients resident in the most affluent population quintiles tended to have slightly shorter lengths of stay than those in the most deprived quintile. First COVID-19 inpatient stays were longer than subsequent stays (adjusted IRR 1.07, 95% CI 1.02 to 1.12). The LOS shortened significantly for all time periods after 31 August 2020, until mid-February 2021. As the proportion of acute beds occupied by patients with COVID-19 increased, LOS tended to fall: for every ten-percentage-point increase in this proportion, the adjusted IRR was 0.97 (CI 0.96 to 0.98).

Model variants carried out as sensitivity analyses marginally altered the resulting adjusted IRR of the CVW-available discharges (see Report Supplementary Material 5). Shortening the time period variable to seven days reduced the IRR slightly such that the difference was no longer significant at 95% statistical confidence level (IRR 1.04, 95% CI 0.997 to 1.08), while increasing the period to 28 days increased the adjusted IRR to 1.08 (95% CI 1.04 to 1.13). The addition of earlier (wave 1) data made the two groups effectively indistinguishable from one another (IRR 1.01, 95% CI 0.97 to 1.05). The two alternative LOS outcomes tested had little impact on the adjusted IRRs compared with the base model.

COVID-19 readmission within 28 days

Of the patients discharged from trusts with a CVW available, 13.0% were readmitted with COVID-19 within 28 days, compared with 13.2% from trusts where no CVW was available (Figure 22). Adjusting for all variables, we found a non-significant difference in COVID-19 readmissions within 28 days for discharges where a CVW was available compared to those where a CVW was not available (adjusted OR of 0.97, 95% CI 0.91 to 1.03). Our sensitivity analyses demonstrated the robustness of these findings (see Report Supplementary Material 5).

FIGURE 22.

Numbers (%) of patients readmitted for COVID-19 within 28 days and adjusted OR with respect to factors in multivariate model.

As with LOS, we found generally positive gradients of relative odds of readmission with increasing age and comorbidity. Females had significantly lower odds of readmission than males (adjusted OR 0.78, 95% CI 0.75 to 0.81). Black patients had lower odds of readmission compared with white patients (adjusted OR 0.91, 95% CI 0.84 to 0.99), while no other ethnic categories (for patients with known ethnicity) showed similarly significant differences. There were no apparent differences in the odds of COVID-19 readmission for patients resident in less deprived areas compared with those in the most deprived quintile areas.

For both LOS and readmissions models, adding variables one by one to our multivariate models (an example is shown in Report Supplementary Material 5) revealed that the addition of the 14-day time period categories, and the clustering by hospital trust had large impacts on the estimated adjusted ratios, but all additional variables (age, comorbidity index, and so on) had relatively marginal additional impacts.

Key findings

In this analysis of patients with COVID-19 discharged from hospital during the second wave of the COVID-19 pandemic in England, we found no evidence of a relationship between the availability of hospital-based CVWs and subsequent rates of readmission for COVID-19. We did, however, find that the LOS for patients hospitalised with COVID-19 was 5% longer where a CVW was available.

Comparisons with other studies

There are few studies available against which we can compare our results. Our finding of no difference with respect to readmissions appears to contrast with that of an analysis of a post-discharge remote monitoring model implemented in five Boston (MA) hospitals. This study found that enrolled patients had a reduced odds (OR 0.54, p-value 0.04) of attending emergency department or being readmitted within 30 days of discharge, compared to those not enrolled. 124 However, given the relatively large CIs in their estimate, emergency department attendances being much more common than readmissions for the enrolled patient cohort, and those readmissions being for any cause, our findings may not be inconsistent.

Two other analyses have claimed reductions in length of hospitalisation associated with post-discharge models, on a scale of 30–40%. One, related to implementation in a Netherlands hospital, estimated a reduction in the LOS of 5.0 days (with a resulting mean LOS of 10.6 days),125 and a second, in an English hospital trust, reported a reduced LOS of 10 days, against an average of 17 days prior to implementation. 126 Both analyses, however, were based on small patient cohorts with no formal statistical testing. In the first analysis it was not clear how estimates of reductions were made, and in the second there was no attempt made to control for differences between the compared groups.

More generally, there is evidence of non-COVID-19-related post-discharge virtual wards having a positive impact on readmissions when employed as a disease-specific intervention (at least in the case of heart failure), but there is not similar evidence of impact for more mixed groups of patients at high-risk of chronic disease. 123 Our findings may be consistent with this picture, considering the heterogeneity of patients with COVID-19, and the multiorgan effects of COVID-19 infection. 131–133

Strengths and limitations

With respect to the analysis of post-discharge COVID monitoring services, our study is currently unique in its scope. We have made use of a national administrative hospital data set, and have analysed almost 140,000 COVID-19 admissions in an effort to detect an impact of a national-scale roll-out of post-discharge remote monitoring services. We have been pragmatic in our use of the available data and have controlled for characteristics available to us.

Nevertheless, there are several limitations, and our results should be interpreted with caution. We did not know which patients were enrolled on to a CVW, and so treated all patients as potentially having received the intervention where one was available at a trust. We had no information about important clinical factors (e.g. ICU admission, clinical readings, including oxygen saturation levels, and specific treatments received). The location to which patients were discharged (home, care home, etc.) was not known, and we also had no information about out-of-hospital deaths (and this during a period in which rates of out-of-hospital deaths had been persistently above long-term norms). 134 We extracted data only on patients discharged alive, so the impact (if any) of patients with COVID-19 who died during their hospital stay has not been accounted for. Our analyses have assumed that these factors, which are likely to have been associated to a greater or lesser degree with eligibility for referral, and/or with risks of readmission and long stays, were well balanced between the two key groupings of patients after having adjusted for model factors, but the extent to which this is the case is not known. However, in terms of at least one of these missing factors, an analysis of a cohort of 419 patients with COVID-19 managed on a post-discharge pathway suggests that out-of-hospital deaths may have been relatively rare (<0.1%). 120

As outlined in Chapter 4, for seven hospital-based sites we were able to estimate a range of 4–65% of discharged patients with COVID-19 may have been monitored on a CVW. But nationally, we did not know the scale of each trust’s implementation and our analysis wasn’t able to examine potential dose–response relationships.

We also know that services were implemented with significant differences, depending on the locally adopted models of care. Our analysis treated the intervention as being homogeneous, and was not able to estimate differential impacts by hospital trust, although these impacts may have existed.

Interpretation

We found longer COVID-19 inpatient stays associated with implementation of CVWs; this was not expected for a service that aimed to support the early discharge of patients. One explanation might be that CVW eligibility criteria influenced hospitals’ discharge decisions such that patients were kept in a bed for longer than they otherwise might have been. It is not clear whether local pre-hospital (CO@h) services starting at similar times might have affected admissions in such a way that patients were admitted sooner and stayed for longer. The finding might, in part, also be a consequence of many CVW services launching during the falling (improving) edge of England’s second COVID-19 wave, with longer lengths of stay being a consequence of improving bed capacity.

We did attempt to control for the latter effect in our addition of trust-specific weekly occupied beds data, and with our inclusion of a time period variable in our multivariate analyses. In one sensitivity analysis, when we shortened the time period to seven days (from 14), we no longer found a statistically significant difference in LOS at the 95% level (adjusted IRR 1.04, 95% CI 1.00 to 1.08, p-value 0.07). The time factor used in our fully adjusted models was somewhat arbitrary, and it is not clear that 14 days was better used above seven days. The finding of a longer LOS associated with CVW therefore needs to be considered with some care.

The expected impact of such services on readmissions following discharge is not obvious. These services aimed to detect early deterioration for more rapid escalation to potentially improve ultimate outcomes. It may be the case therefore, that readmissions might have been expected to increase. Unfortunately, with limitations in the available data described above, we were not able to assess any changes in the severity of patients on readmission.

As a result of the limitations in the available data outlined above, there is a strong possibility that any positive impacts – at a local hospital trust level, or for specific versions of the pathway – were not ultimately able to be detected. This limits the extent to which this analysis can contribute to the further development of these, or similar services, and highlights the importance of making routine data collection – appropriate for use in evaluation – an intrinsic part of services’ operations.

Briefly, we reflect on observed relationships between patient characteristics and COVID-19 lengths of stay, and readmissions. Positive relationships between age and comorbidities and these outcomes have been observed elsewhere. 135–138 Meanwhile, differences with respect to gender (in this study’s case both outcomes were lower for females vs. males) tend to have been absent. 131,136,138,139 We found shorter lengths of stay and lower rates of readmissions for patients belonging to specific ethnic groups with respect to white patients (this was the case for patients belonging to black ethnic groups for both outcomes, and also Asian, mixed and ‘other’ ethnic groups for LOS). Evidence from elsewhere is mixed, but at least one US study found shorter lengths of stay for hospitalised non-white patient groups. 138 Ultimately, however, care is needed when interpreting findings for specific characteristics separate from the context of the other model variables. It is possible, for example, that non-white ethnic groups (to give one example) may be acting as fine-scale markers of urban or more deprived areas (in addition to the model deprivation categories) and as markers of younger patients (within modelled age bands).

Conclusion

Our analysis has not shown any evidence of early discharges or changes to rates of readmissions associated with the roll-out of CVWs in England. While this may reflect the true impact of the service, it may be, in part, a consequence of the lack of certain data: on which patients were enrolled (or even how many), what COVID-19 treatments were received while in hospital, and their clinical observations. It is possible that CVW services had a range of impacts that we have ultimately not been able to reveal.

What was already known

• Previous economic analyses of remote home monitoring for patients with COVID-19 have reported the resources used and the amount spent per patient monitored.

What this chapter adds

• The mean running cost per patient monitored under CO@h services was slightly lower compared with CVW services (£527.5 vs. £599.1).

• For both CO@h and CVW services the mean cost per patient monitored at home was lower in sites using both tech-enabled and analogue modes of data submission compared with the sites using analogue-only modes.

• Mortality rates were slightly higher for the patients under CO@h services in comparison with patients under CVW services (0.9% vs. 0.7% respectively).

• The majority of the staff involved in running CO@h and CVW services were clinical staff. Over 50% of staff (clinical and non-clinical staff combined) were employed at band 5 or below in the CO@h service, whereas in CVW services there were slightly more staff at band 6 or above.

Data collection

We gathered retrospective information using a data collection form provided to the 28 sites that was also used for the analyses in Chapters 4–6. Designated site leads and data leads at each research site were contacted and asked to complete the form in Microsoft Word, and return it electronically to a member of the research team. During May to June 2021, we collected data for the period 1 October 2020 to 30 April 2021 (see Chapter 2 for further information). We collected information regarding number of patients involved (triaged, monitored, deteriorated, escalated, and number of deaths), and the number of staff and resources used for setting up and running the sites (see Project website documentation – Study instruments).

To explore the differences in staff, resources, patients involved and patient outcomes further between tech-enabled and analogue versus analogue-only data submission modes, additional data were collected from four of the 28 sites. All of these four sites were using both tech-enabled and analogue data submission modes. We conducted an in-depth analysis of the time spent per patient for specific activities of the programme and calculated the cost per patient for all the activities. The data collection form for these four sites included additional questions regarding the total number of patients monitored using each data submission mode, staffing (role/band, time spent per patient on every activity) and the frequency of contacts made with patients (see Project website documentation – Study instruments).

Data analysis

Patients onboarded and the impact on outcomes

The number of patients triaged for both CO@h and CVW services (26,126 and 1,761 respectively) was higher compared with the number of patients monitored (17,424 and 1,711). The percentage of the monitored patients over total triaged was much higher for CO@h (149.9%) if compared with CVW services (103.3%). This may be because hospitals had a less heterogenous pool of people qualifying for services and had much more prior information about them. The proportion of the patients escalated and deaths over those monitored was higher for the CO@h services compared to the CVW services (16.6% and 10.8% respectively for escalated patients and 0.9% and 0.7%, respectively, for the number of deaths; Table 10).

Of the 22 sites running the CO@h service, 16 used tech-enabled and analogue modes, and 6 used the analogue-only mode (see Report Supplementary Material 2). Of the 13 sites running a CVW service, 6 used the tech-enabled and analogue data submission modes and 7 the analogue-only mode. Our results show that from the total number of patients onboarded in the CO@h services 76.8% were followed up using the tech-enabled and analogue data submission mode while in the CVW services 60.8% of patients onboarded were followed up using analogue-only data submission mode.

The structure of the workforce involved in providing CO@h and CVW services

The total number of FTE staff involved in running the CO@h service was on average 7.4 per site [range: 1.0–39.0] and for the CVW service this was 2.1 per site [range: 0.1–4.8]. Staff involved were a mix of medical consultants, emergency department staff, general practitioners, nurses, advanced nurse practitioners (ANPS) and medical students. Our analysis of the staff employed in running the services showed that there were no clear patterns in the differences in terms of clinical/non-clinical staff or seniority of the staff involved between CO@h and CVW services.

Table 11 provides the mean number of FTE staff by role and band, stratified by type of service and size of site (see Report Supplementary Material 2 for the total number of FTE staff involved). For both CO@h and CVW services, most of the staff involved in monitoring the patients were clinical (68.4% for CO@h service and 86.7% for CVW service).

Table 11 also shows that for all sites running CO@h services, 53.0% of staff were employed at band 5 or below (27.1% non-clinical and 25.9% clinical staff). Sites using tech-enabled and analogue modes had a higher proportion of non-clinical staff at band 5 or below (33.1% non-clinical and 18.6% clinical staff) compared to analogue-only mode (16.8% non-clinical and 38.5% clinical staff).

Across all sites running the CVW services, 51.3% of all staff (50.4% clinical and 0.9% non-clinical) involved were employed at band 6 and above. A similar trend was observed for the sites using analogue-only mode where 54.1% of all staff involved (53.6% clinical and 0.5% non-clinical) were employed at band 6 and above whereas for the tech-enabled and analogue mode most staff involved (51.6%; 34.1% clinical and 17.5% non-clinical) were at band 5 and below.

Our analysis of the effect of other factors (i.e. type of services, data submission mode and size of the site) on the number of FTE staff involved revealed that, as expected, a key determinant was the size of sites. Note that these findings should be considered with caution due to the small sample size (N = 35 sites) (see Report Supplementary Material 6).

The costs of setting-up and running the sites

Table 12 presents detailed information on the mean running cost per patient triaged and monitored over the study period for both CO@h and CVW services (see Report Supplementary Material 6). These data show that most funds during the setting-up phase were spent on non-staff items (e.g. the technology platform, medical and other equipment). Our analysis of running costs showed that the mean cost per patient monitored using the CO@h service was lower compared with the CVW service (£527.5 vs. £599.1). The mean cost per patient monitored was lower for sites using tech-enabled and analogue compared with the sites using analogue-only data submission mode for both CO@h (£515 vs. £561) and CVW (£584 vs. £612) services.

The regression analysis of the effect of factors (i.e. type of services, data submission mode, seniority of FTE staff and the number of the patients monitored) on the mean running cost per patient revealed that, as expected, a key determinant was the number of patients monitored. Another significant factor was the type of service (i.e. CO@h or CVW). Even though the type of the data submission mode and the FTE difference between the senior and junior staff involved were not statistically significant predictors of mean running costs per patient, the direction of their effect was consistent with the descriptive findings. Note that these findings should be considered with caution because of the small sample size (N = 35 sites) (see Report Supplementary Material 6).

We investigated whether the mean running costs per patient at each site were associated with the total number of patients monitored. Figures 23 and 24 show a positive relationship (slope coefficient for the CO@h service 0.54 (95% CI 0.19 to 0.89), and for CVW service 2.10 (95% CI 0.16 to 4.03)) between the number of patients monitored and the mean running cost per patient for the CO@h and CVW services, respectively. This suggests that running costs increase as the number of patients monitored increases. However, this finding should be treated with caution for several reasons. First, the number of sites involved in this analysis is small. Second, as evidenced by the two figures, there are outlying sites in terms of mean costs and number of patients monitored. Third, the figures are based on running costs only; if set-up costs were included, these costs would fall as the number of patients monitored increased. Fourth, we do not have data on the maximum patient capacity of sites on which total running costs would be based.

FIGURE 23.

The mean cost per patient for CO@h services. Source: Based on data provided by each site; 22 sites used CO@h services.

FIGURE 24.

The mean cost per patient for CVW services. Source: Based on data provided by each site; 13 sites used CVW services.

Time spent on monitoring activities

This part of the analysis used data that were available only from the four sites where we undertook a more detailed investigation of the monitoring activities. All four sites used tech-enabled and analogue and analogue-only data submission modes. The total number of patients monitored at these four sites was 3366, of whom 2824 were monitored using tech-enabled and analogue mode and 542 were monitored using analogue-only mode (see Report Supplementary Material 2).

The most time-consuming activity was patient’s monitoring, and on average less staff time was spent on this under tech-enabled and analogue mode compared with analogue-only (68.0 minutes vs. 72.5 minutes respectively; note that this activity by staff might be removed in the case of self-monitoring by patients). Based on time spent and staff working on each activity (see Report Supplementary Material 6), we calculated the average cost of each activity per patient, for the tech-enabled and analogue and analogue-only modes (£115.1 vs. £98.3, respectively; if the patient monitoring costs were removed, as might be the case in a self-monitoring service with no monitoring by staff, and assuming all other staff inputs/costs remained constant, these costs would fall to £80.0 and £60.8, respectively; see Report Supplementary Material 6). Please note that because of the amount of missing data, these findings should be viewed with caution.

These four sites reported on average 8.7 contacts per patient using tech-enabled and analogue mode throughout the time they were in the CO@h service, and 15.5 contacts per patient using the analogue-only option.

Key findings

A total of 26 of the 28 sites contacted provided information for this analysis, of which 13 provided the CO@h service, 4 provided the CVW service and 9 provided both services. In total, 17 sites used tech-enabled and analogue and 9 analogue-only data submission modes. The mean running cost per patient monitored under the CO@h service was lower compared with CVW service. The main driver of the cost per patient was the number of FTE staff and the grade of staff involved, and this was regardless of the type of service adopted. A comparison with findings from wave 1 (March to August 2020)36 showed that the mean cost per patient monitored remained fairly constant from wave 1 to wave 2 for the CO@h service (£553 vs. £527) and increased from wave 1 to wave 2 for the CVW service (£400 vs. £599).

The mean costs per patient in the sites using tech-enabled and analogue were lower compared with analogue-only sites, mainly due to shorter time needed to collect patient medical information. However, as other studies have shown, the potential for technology to save costs must be treated with caution unless the digital assessments are shorter in time and they are proven to decrease follow-up and deterioration of symptoms, thus leading to lower commitment of staff time for overall service delivery. 142,143

Comparison with other studies

Previous studies have found that remote patient monitoring for conditions other than COVID-19 was associated with improved clinical outcomes and medication compliance and reduced the number of emergency room visits, hospital admissions and patient travel. 144–146 A comparison with data from wave 1 (March to August 2020)36 showed that escalated patient rates for the period October 2020 to April 2021 was higher for the CO@h service (16.6% vs. 10.0%). Whereas for the CVW service the escalated patients’ rates were slightly higher for wave 2 in comparison with March to August 2020 period (10.8% vs. 12.2%). The mortality rates for the period October 2020 to April 2021 were 0.9% for the CO@h and 0.7% for the CVW, slightly lower than the rates previously reported for the period March to August 2020 (1.1% and 0.9%, respectively). 36

A previous systematic review found that there was no evidence showing that telemedicine and telecare interventions can be cost-effective compared with conventional health care. 147 However, it has been also highlighted that cost-effectiveness analysis of remote monitoring of patients for different diseases is often integrated into other home health-care services making it challenging to isolate and allocate its cost appropriately. 147 The evidence on resources used and costs of remote home monitoring for patients with COVID-19 specifically is also very limited. Our previous literature review on similar interventions revealed that very few studies have conducted an economic analysis and in most cases this is limited to describing costs incurred and resources used. 32 In general, previous studies have stated that similar interventions may be cost-saving by reducing downstream costs (in terms of avoiding hospital admissions and bed-days). 31 Our findings are consistent with a recent Australian study where staffing costs were the main contributors to the total costs of monitoring. 148 In other health-care contexts, the use of telehealth for the provision of health care has previously shown to be associated with cost savings, especially in reaching remote areas. 149 However, any potential cost saving of technology relies on the assumption that digital assessment is shorter in time and decreases follow-up contacts and deterioration of symptoms, which may result in new admissions to hospital. Caution is warranted as such assumptions may not always hold. 142

Strengths and weaknesses

The main strength of this analysis is that we achieved good geographical coverage of sites across England (see Chapter 4), obtained primary data on CO@h and CVW services use and distinguished between the tech-enabled and analogue versus analogue-only data submission modes. Additionally, this study fills a literature gap regarding COVID-related remote monitoring services, since it is the first to provide this level of detail about costs, workforce and activities.

There are several weaknesses with this study. First, we were not able to perform a full economic analysis (i.e. cost–utility analysis) as the effectiveness analysis found no evidence of the CO@h service on mortality or downstream hospital utilisation (see Chapters 5 and 6). Second, the study design did not allow the use of patient-level data, which means we were unable to examine patient-level variation in service costs. Third, cost- and resource-use data provided by some sites were incomplete. For this reason, the in-depth analysis using the four sites was very limited and the results should be treated with caution. Fourth, the selection of the four sites used for the in-depth analysis was based on purposive and convenience sampling, and this may insert selection bias. Fifth, there was wide variation in service models across sites and much variation in how services were designed and delivered (see Chapter 4). Sixth, the generalisation of our findings to all patients with COVID-19 is difficult as pre-existing conditions and level of severity were not taken into consideration. Seventh, the limited number of sites which have been used for this analysis does not allow further grouping in terms of their size, considering both the number of patients and the length of time establishing this programme. Finally, any interpretation of results should be taken with caution as we do not have the information on the maximum capacity of the sites and can only observe the current flow of patients. For sites with a higher planned capacity the underutilisation of resources will mean that the cost per patient may go down with the increase in number of patients.

Implications

Our analysis showed a lower cost of remote monitoring per patient with COVID-19 during wave 2 of the pandemic in the UK. Our results indicated that the tech-enabled and analogue monitoring was less costly, in terms of mean cost per patient, than analogue-only monitoring. It offers a model for future research in the way it covers a range in the size of sites and raises questions about differential practices within the overall envelope of remote monitoring – differences that are necessarily present in any geographically diverse and large-scale health system.

Future interventions on remote monitoring for other health conditions need to consider their impact on primary and secondary care. Such services may differ in costs, band of the staff involved and worked hours based on the health sector of primary impact. Consideration of the differences between the CO@h and CVW services should also be considered given the different pool of patients they serve and the level of information the staff have on them. Future studies should also gather information on total capacity and how this relates to costs; given the fixed costs in delivering services, this may provide a better view on actual costs and may also possibly help future planning. Future research, including more sites, may provide additional evidence and shed light on the areas where the availability of this study’s data was poor. Availability of routine data on health outcomes (instead of having to use site questionnaires) would also facilitate future research. Overall, there is pressing scope for seeing remote monitoring within a more holistic view of patients’ and staff experiences.

What was already known

• Remote home monitoring may have some benefits for patients, health-care systems and staff (e.g. flexibility/independence/fast access to clinicians).

• Some challenges and concerns with remote home monitoring have been highlighted (e.g. lack of supportive infrastructure within NHS organisations to facilitate successful implementation and delivery, staff training, treating patients living with a range of long-term conditions which may include physical and/or cognitive impairment).

• However, there is a paucity of evidence on staff experiences and the factors influencing delivery of COVID-19 remote home monitoring services.

What this chapter adds

• Delivery of remote home monitoring services for COVID-19 involved delivery of either technology-enabled and analogue services, or analogue-only services. Staff were involved in a range of tasks including monitoring, escalation, and service evaluation.

• Staff generally reported positive experiences of delivering the service, felt that it was easy to deliver and valued support, but staff would have benefited from further training.

• Factors influencing delivery of remote home monitoring services for COVID-19 included: support for delivering the service, NHS resources and capacity on staff workload, multidisciplinary team dynamics, and patient (dis)engagement.

Design

This chapter draws on data from workstream 3 (staff survey data) and workstream 4 (staff interviews). Data collection for this study took place between February and June 2021.

Sample

Staff who led and/or delivered COVID-19 remote home monitoring services (see Chapter 2 and Chapter 4 for site and staff sample details).

Measures

We iteratively developed the survey and semistructured topic guides specifically for this study (see Chapter 2, Project website documentation – Study instruments).

Procedure

We conducted a survey with staff from 28 sites, and interviews with staff from 17 sites (see Chapter 2 for details).

Analysis

Inductive thematic analysis was used to analyse interview data inputted into RAP sheets. 70 Additionally, a subset of transcripts (n = 17) were independently thematically coded to validate interpretations from the analysis of RAP sheets.

Survey data were analysed on a complete case basis (excluding participants with missing data) using descriptive statistics (using SPSS, version 25). Open text responses were coded using inductive thematic analysis.

Interview and survey findings were triangulated to compare consistency of findings relating to research questions. 71,72

Participant characteristics

Surveys were received from 292 NHS staff (70 clinical leads/service managers, 222 delivery staff) and 58 interviews were conducted with staff (23 clinical leads, 28 delivery staff, 7 data staff. See Chapter 4 and Report Supplementary Material 2 for demographic characteristics.

Findings

Key findings

Staff were involved in flexibly delivering a range of technology-enabled and analogue remote monitoring models for COVID-19. Staff generally reported positive experiences of delivering the service, felt that it was easy to deliver and valued support. A range of training and support opportunities were provided but staff would have benefited from further training especially when undertaking clinical assessments remotely, determining how best to escalate care of patients whose condition may be deteriorating from COVID-19, and how best to provide follow-up care. Findings also indicated different factors which influenced delivery of remote home monitoring models, including appropriate support and training, staff knowledge and confidence, capacity and workload, multidisciplinary team dynamics, and patient engagement or disengagement.

How findings relate to previous research

Given the rapid transition to using remote home monitoring models and technology to support people living with COVID-19 and long-term health conditions, our findings extend literature on the training needs and experiences of staff asked to use such methods within a wider context of workforce challenges across primary and secondary care. 154 Staff were generally comfortable pivoting away from their usual front-line services during the pandemic; however, the impact on how staff managed this transition varied with some feeling an additional burden while others welcomed greater responsibility and the opportunity to support the NHS’s response to the pandemic. For some delivery staff, remote home monitoring brought new challenges especially when completing consultations over the telephone compared to face-to-face. In some cases, such a change led staff to reconsider their professional role, beliefs, as well as capabilities when treating patients. Similar learning has been found in previous reviews, citing nursing staff in particular as being concerned about the impact of digitally enhanced modes of working on the staff-patient relationship. 164 However, patient experience findings indicated that the staff-patient relationship was not jeopardised by remote monitoring (see Chapter 9 or Walton et al. 165). Staff experiences on the modality of COVID-19 remote home monitoring and impact of staff are explored elsewhere (see Chapter 11).

Staff received a range of training and support to deliver remote monitoring models as described in our evaluation. Unfamiliarity with nationally designed competency frameworks and a desire for bespoke training to reflect local organisational working meant that some staff may have been better equipped to work more independently than others. Front-line delivery staff, in particular, were asked to rapidly transition from usual methods of providing face-to-face treatment to remote ways of working with limited available evidence of the effectiveness, benefits/challenges and staff experiences of remote home monitoring services. 153 Given the speed at which remote home monitoring services were implemented, this left a significant gap between staff skills, competency and confidence.

The nature and quality of training delivered to front-line delivery staff was dependent on those providing senior clinical insight. Consequently, some delivery staff experienced substantial levels of uncertainty about using their own clinical judgement to independently assess and communicate risk. This was, to some extent, later addressed by the development of SOPs and other working materials. This uncertainty may have impacted on the ability of delivery staff to communicate risks effectively with patients, especially those living with multimorbidities as well as COVID-19 which may have led to patients to revaluate self-management practices. Remote home monitoring may not always remove the need for in-person examinations. 166

Our findings demonstrated workload differences for those balancing their workload with other commitments versus those who were redeployed. This supports findings from the first wave of the pandemic, in which staff were able to focus on delivery of remote home monitoring services due to cancellation of elective care/other activities, and the redeployment of staff. 36 Delivery staff, particularly nurses, remained committed to taking a patient-centred approach to providing care that took account of the emotional and physical impact of COVID-19 on patients. As with previous research,152 our findings highlighted that staff were often not directly involved or lacked a level of autonomy with respect to wider organisational change, despite wanting to share views on how models should be implemented locally.

Given the range of skills required to deliver remote home monitoring models, a multidisciplinary approach is needed for COVID-19 and chronic disease management in general. 167 Previous research from the United States showed that team-based care (physicians, nurses, administrative staff) led to more effective implementation of COVID-19 remote home monitoring services and higher patient satisfaction than single practitioner-led models. 129 Similar findings were found in another study with chronic heart failure patients, who expressed a greater desire to speak with a range of health professionals when being digitally monitored using a range of technology enabled devices. 168

Strengths and limitations

One strength is that we used mixed methods to study staff experiences of COVID-19 remote home monitoring services, and our study included a large sample of participants working across a range of job roles in primary and secondary care. However, variation between case study sites (see Chapter 4) made it difficult to synthesise findings, which may limit transferability of findings.

Our findings may not be representative of all staff views and experiences. For example, the response rate for our staff survey was 39%. Therefore, it is possible that staff who were less engaged with services may have been less likely to complete the survey.

Implications for policy and practice

Current levels of training may not be adequate to support both clinical leads and delivery staff to undertake a wide range of ‘work’ when managing patients with COVID-19. Staff need the opportunity to be better informed for delivering the service; e-resources should be available to staff, that can be accessed as and when required, and staff should receive bespoke training relevant to their service and organisation. Service leads should consider whether they need to introduce formal sign-off mechanisms to ensure staff have been trained adequately and to a required standard. Staff largely felt confident in their ability to deliver remote home monitoring services but found apps and online platforms difficult to navigate as part of monitoring. It would be valuable, where possible, to involve staff when deciding which apps and/or online platforms to adopt during service development and implementation. Notably, delivery staff lacked confidence when communicating the risks associated with COVID-19 to patients. As a result, organisations implementing remote home monitoring services should promote multidisciplinary approaches where delivery staff have ready access to clinical oversight, especially when a patient has deteriorated to a stage where escalation is required.

Primary and secondary care organisations need to consider staff capacity for delivering home monitoring services, given redeployed staff will no longer be available as routine health-care services recommence.

For staff to take a patient-centred approach, staff should be given flexibility within services to discuss wider contextual issues that can contribute to ill health and to provide emotional support to patients experiencing anxiety and other difficulties.

Future research

Further research is needed to explore the following topics: staff experiences of delivering care for COVID-19 and other health conditions using remote monitoring models, the competency and skill set required to deliver remote monitoring services, how training should be designed, and how best to embed training during the set-up and implementation of services across primary and secondary care sectors.

Conclusions

Remote monitoring models can play a crucial role in managing a large number of patients for a range of health conditions; however, the success of delivery is dependent on the right workplace infrastructure for delivering these models of care. While staff in our study show high levels of acceptability for managing patients using remote models, issues remain around confidence when making clinically informed decisions and a dependency on clinical oversight. Delivery of remote home monitoring services is also influenced by patient engagement or disengagement with these services.

What was already known

• Remote home monitoring services for patients with COVID-19 have been implemented throughout England.

• These services may place more responsibility on to patients/families.

• It is not known how patients and carers experience these services, or how they found engaging with the activities.

What this chapter adds

• Patients and carers found the human contact with staff reassuring.

• Patients with COVID-19 (an acute condition) can engage with remote monitoring services but may require support from staff and family/friends to do so.

• Engagement is conditional on a range of factors including patient factors, support and resources, and service characteristics.

• Burden of treatment may be experienced by patients and families with acute conditions.

Design

This chapter draws on data from workstream 3 (patient and carer survey data) and workstream 4 (patient and carer interviews). Data collection took place between February and June 2021 (patient surveys retrospectively included January).

Sample

Patients and carers who had received COVID-19 remote home monitoring services (see Chapter 2 and Chapter 4 for site and patient sample details).

Measures

We developed the survey and semistructured topic guides specifically for this study (see Chapter 2, Project website documentation – Study instruments).

Data collection

We conducted a survey with patients and carers from 25 sites and interviews with patients and carers from 17 sites (see Chapter 2 for details).

Analysis

Survey data were analysed using descriptive statistics (using SPSS, version 25). All cases were analysed (whether carer, patient or unknown). Where data were missing for specific questions, cases were excluded from the analysis and the denominator reported. Open text responses for three questions (434 responses for anything else to say about the service, 61 responses for how carers supported friend/family member while they had COVID-19 and 200 responses for recommendations to improve service).

Inductive thematic analysis was used to analyse interview data inputted into RAP sheets. 70 We then developed a framework based on these themes and subthemes and one researcher used this framework to extract quotes from all original transcripts.

Survey and interview findings were triangulated.

Participant characteristics

Surveys were received from patients (n = 936) and carers (n = 48), and 62 interviews were conducted with patients (n = 59) and carers (n = 3). See Chapter 4 and Report Supplementary Material 2 for demographics.

Most patients were referred to services via community methods (Table 17). Patients and carers used a range of methods to record and report their readings to the service, including analogue (paper and phone) and tech-enabled methods (Table 17).

What type of formal and informal support did patients receive as part of COVID-19 remote home monitoring services?

What are patients’ and carers’ experiences of engaging with COVID-19 remote home monitoring services?

Patients mostly had positive views of the services; 93% (n = 970/1045) of survey respondents rated services as excellent or good, 90% (n = 923/1028) of respondents found the services helpful and 91% (n = 944/1037) would recommend the services to their family and friends.

Findings indicated that most patients and carers found the services reassuring and supportive (91% (n = 946/1040) of survey respondents). Qualitative findings indicated that patients and carers valued the human contact with staff and found it reassuring due to having someone watching over them, particularly for those who were living alone, had no support nearby or had existing conditions.

Because it’s obviously keeping an eye on you, isn’t it really? And I was getting the phone calls every day. How are you feeling? [. . .]. But someone who was on their own, who had no- who was living on their own, you know, it’s a bit of a lifesaver isn’t it?

(Site A, interviewee 4)

A minority of patients and carers felt that there were gaps in the service, and it was not holistic. Some felt that services were narrowly focused on managing known symptoms of COVID-19 which did not always suit those with other symptoms, health conditions or who required wider support. A few patients reported feeling that services were isolating and unsupportive (e.g. they only received a call about the oximeter drop-off/return, but not for monitoring).

Most patients and carers felt the care provided was appropriate and preferred to be at home instead of being in hospital (given the pandemic context). Reasons for preferring home over hospital included freeing up space for others in need, fears of going to hospital during a pandemic, and communication barriers. However, some patients and carers spoke about preferring to be in the hospital rather than at home, to feel more secure, feeling scared and wanting to be seen face-to-face.

And I didn’t feel too embarrassed that I was using up valuable resources because I thought, Well I’m sitting here at home, there’s no reason for me to go in Hospital, bother anyone and waste people’s time.

(Site N, interviewee 3)

A minority of patients and carers spoke about how these services were the only available care and that they would have liked to have received care from other health-care professionals such as their GP in parallel. Many patients and carers were not aware of these services prior to referral.

Some patients and carers also spoke about how these services helped them to monitor their own improvement and that it potentially improved their outcomes.

Patients and carers reported very positive views of the workforce and that they were helpful and put patients at ease and were professional and potentially even life saving. Continuity of staff was thought to be important.

A few patients had negative experiences with individual staff members (e.g. that they were dismissive, did not recognise that they needed help, were not interested or lacked clinical expertise to support patients or answer their queries).

What are the factors influencing burden of treatment and ability to engage with COVID-19 remote home monitoring services?

Findings indicated that patients and carers generally found it easy or very easy to engage with these services and the resulting activities, including understanding information, monitoring using the oximeter, recording readings and providing readings and escalating care (Table 21). Most survey respondents indicated that they did not experience problems with services (72%, n = 771/1069) and did not report barriers to engagement with services (80%, n = 858/1069).

Engagement with service activities was not without challenges, with some patients and carers reporting issues with the information provided or needing further information. Some patients found monitoring difficult due to other health conditions, or that monitoring made them feel worried. Some patients and carers wanted more support or found recording burdensome. Finally, some issues related to escalating care were identified in the interviews. The uncertainty of COVID-19, perceptions of hospital as a frightening place and uncertainty around interpretation of readings and thresholds, meant that some patients and carers were hesitant to self-escalate their care, waited for a member of staff to advise them to escalate their care or reported not wanting to go to hospital or seek further support, even when advised to by staff members.

really I should have probably rung when the readings were that bad but I didn’t. [. . .]And when I did send them through they said, ‘No, get to the doctors now’.

(Site C, interviewee 6)

The most frequent challenges reported within the survey were returning the oximeter, contacting health-care professionals when needed and seeking further help (Table 22). While many survey respondents discussed problems with the team (35%, n = 87/249) or had their problems resolved (33%, n = 76/232), over half of these participants said problems had not been resolved (54%, n = 126/232).

Findings from the surveys and interviews indicated three overarching themes that influenced burden of treatment and patient’s ability to engage, with COVID-19 remote home monitoring services: (1) patient factors; (2) wider support and resources; and (3) service factors (see Report Supplementary Material 8 for example quotes).

Key findings

Patients can engage with remote home monitoring services, even when experiencing acute illnesses (e.g. COVID-19). However, many patients required formal input from staff and informal support from family and friends to complete the necessary tasks. Patients and carers had positive experiences receiving remote home monitoring. The human contact from staff provided patients and carers with reassurance and patients reported services were mostly easy to engage with. However, patients’ ability to engage with services was conditional on a range of factors, including having support from family/friends and staff, being in good health and receiving clear instructions on what they needed to do and how to do it, and the level of commitment from patients while on the service.

How findings relate to previous research

Previous research highlighted that little was known regarding patient experiences of COVID-19 remote home monitoring services in the context of pressured health services and concerned patients (see Chapter 3 or Vindrola-Padros et al.)32,36 These findings extend earlier findings by highlighting patients’ and carers’ positive views of the service, challenges and concerns relating to engagement with remote home monitoring for acute conditions, and tangible recommendations on how to improve remote home monitoring services (see Appendix 3, Table 26), many of which support wider themes reported in patient experience literature (e.g. the importance of information provision). 183,184

Previous research outlines concerns relating to remote care and telemedicine and the loss of interpersonal dimensions involved in caring relationships. 170 Findings extend the evidence base by showing that care does not need to take place face-to-face for patients to feel reassured and supported. Patients largely felt that care provided at a distance was appropriate and that they were being monitored. This is consistent with previous research indicating that technology may support closer contact with professionals. 170 However, findings may have been affected by the pandemic context in that data were collected during the height of wave 2 of the pandemic, so patients may have been more likely to accept remotely delivered services to help minimise risk to themselves, family members and staff. Patients and carers may feel differently about remote home monitoring and care delivered at a distance in non-pandemic contexts.

New models of health care such as COVID-19 remote home monitoring services sought to change the traditional model of in-person care. Instead, within these models, staff engage with patients to share the care burden while equipping patients and carers to self-monitor and manage care in the absence of staff members. Our findings demonstrate that concepts of treatment burden and difficulties engaging with health-care demands171,179,182 also apply to remote monitoring models for COVID-19. Some patients reported problems engaging with remote home monitoring services for a range of reasons, including feeling too poorly, not having enough knowledge on what to do, and lack of support from staff and/or family/friends. Others reported the necessity of support from their family/friends when engaging with the service. This extends knowledge by showing that social networks may undertake self-monitoring tasks on behalf of (potentially very poorly) patients in addition to helping patients cope with burden of treatment171 and self-management of conditions. 51,185 Those who feel more poorly (either due to COVID-19 or existing conditions) may require more formal or informal support to manage care. Yet, this increases the caring burden for family/friends. This finding also raises concerns regarding appropriateness of care for those who do not have informal support networks in place.

This chapter builds on earlier research by providing a nuanced interpretation of the factors that influenced engagement with remote home monitoring services for acute conditions such as COVID-19. Our finding that many non-health related factors influence engagement is consistent with other studies in a range of other conditions and interventions. 51,179,186–189 Our findings add to prior knowledge by demonstrating that many factors were exacerbated due to the acute nature of COVID-19 and policy factors surrounding COVID. For example, physical health factors limiting engagement may be worsened by the acute nature of COVID-19 and the severity of symptoms that some patients faced, thus affecting a patient’s ability to engage. Furthermore, policy regulations and lockdown restrictions imposed within the UK may have meant that physical social support available to patients may have been limited by members of their support networks living elsewhere. As COVID-19 is easily transmissible, social distancing recommendations were in place, therefore, many patients were distancing from family members living in the same space. Findings demonstrate that despite difficulties imposed by COVID-19, social networks were crucial for many patients in facilitating engagement with services and ensuring that care needs were met. This highlights the need for alternative support where necessary (particularly for those living alone or those who are socially isolated). The reliance on informal support networks has implications for burden of treatment and may not be appropriate for all individuals. These findings support previous theoretical frameworks indicating that social, political and technical contexts influence engagement. 49–52

Previous research has explored concepts of self-management,50,51,186 engagement187 and treatment burden179 in chronic conditions, but little research had been conducted on remote home monitoring and self-management in acute conditions such as COVID-19 in which care needs to be urgently escalated in an efficient and time-sensitive manner. Findings extend earlier work by demonstrating challenges of remote home monitoring models for acute conditions. For example, due to uncertainty of COVID-19, perceptions of hospital being a frightening place and uncertainty around readings and thresholds, some patients were hesitant to self-escalate care and in many cases, patients waited until advised by staff to escalate care. This finding indicates that in situations where there is a need for timely escalation, but concerns around infection transmission from going to hospital, it may be suitable to have formal support from staff members. Together, staff and patients can collaboratively decide when to seek further help, rather than placing responsibility on to patients. This finding contrasts with recommendations within the national SOP for COVID-19 remote home monitoring services,38,39 which indicate that pathways should encourage patients to self-escalate care.

Some patients felt that they were able to engage with services because they had manageable symptoms and felt comfortable with tasks. This indicates that different levels of remote monitoring support are needed for different individuals. This finding supports previous research indicating that the success of telehealth services including remote home monitoring may rely on the fit between individuals’ needs and services. 50,170

Strengths and limitations

Integration of mixed-methods data helped to provide in-depth perspectives on patient and carer experiences of, and engagement with COVID-19 remote home monitoring services.

Findings may not be representative of all patient and carer groups and experiences, for the following reasons: (1) our patient sample was under-representative of some groups (e.g. older patients and ethnic minorities) and over-representative of other groups, when compared with patient onboarding data,114 (2) survey response rate was only 17.5%, and (3) we were unable to recruit patients who had declined or dropped out of the service, or who did not want to take part in the research.

The focus of our research was on patients’ experiences of remote home monitoring services. Therefore, it is possible that we have not captured carers’ experiences in detail. However, some carers shared their own experiences during the interviews and in responding to the survey.

Implications

Burden of treatment may not only affect those with multimorbidity or chronic conditions, but can also affect those with acute conditions. Findings indicate that remote monitoring may increase treatment burden for some patients and families.

COVID-19 remote home monitoring services aimed to target patient groups at higher risk from COVID-19, yet many of these groups appear more likely to report difficulties in engagement with these services (e.g. older patients and those with health problems). Remote monitoring may not be appropriate for everyone (e.g. those without support). Services need to gauge a person’s support network and any concerns surrounding remote home monitoring when assessing eligibility for these services. Services must then tailor the health-care offer to enable patients to engage (e.g. providing further support for those from at-risk groups or who do not have informal support, or linking patients with care networks if needed). All patients should be given contact details to contact services should problems arise. Face-to-face support (e.g. for monitoring) from staff and families has implications for infection transmission.

These findings may have implications for remote home monitoring services more generally. Service developers should consider the type of condition when designing pathways. For example, services for acute conditions may require support from staff to ensure that patients are escalated for further care as necessary. Services must plan logistics for delivery and collection of equipment, ensure sufficient information provision and that patients know what they need to do and that they feel able to engage with the service. Some patients felt that services offered were too narrow and do not consider wider social, emotional or condition-related needs. Service adaptations may be necessary for those receiving remote home monitoring for acute conditions in addition to care for other chronic conditions.

Future research

Further research is needed to explore: (1) experiences of those who decline or disengage from these services; (2) burden of treatment for acute conditions versus chronic conditions; (3) which groups are able to engage with and tolerate burden from remote home monitoring services; and (4) the impact of treatment burden from informal caring responsibilities on families.

Conclusions

COVID-19 remote home monitoring services place a large responsibility on patients and carers in relation to monitoring and escalating care. While patients and carers found services reassuring and a positive experience, many factors influenced their ability to engage. This indicates that services may be conditional on a range of factors relating to the patient (e.g. knowledge and memory), their support and resources (e.g. support from family, friends and staff) and service factors (e.g. service scope and frequency of monitoring).

What was already known

• Evidence shows that COVID-19 has a disproportionate impact on certain population groups, such as ethnic minority groups, older adults and those with comorbidities.

• Literature exploring the effectiveness of strategies to reduce health disparities for remote monitoring services and the impact of remote monitoring services on health disparities is scarce.

• The rapid adoption and spread of remote home monitoring services in England must be accompanied by evaluations at a local level to monitor the impact on health disparities.

What this chapter adds

• Increased understanding of how remote home monitoring services can be designed and delivered to address local population needs to increase accessibility and facilitate engagement with the service.

• We identified a number of local service adaptations, including prioritising vulnerable groups, creating referral pathways, offering translation services, offering non-digital options, and providing face-to-face assessments, and we discuss their potential impact on health disparities.

• We found that despite efforts to adapt services, disparities across patient groups in their experience of, and engagement with, the service were reported (related to age, health status, ethnicity, level of education, employment status and gender).

Design

This chapter draws on staff and patient/carer surveys and staff and patient/carer interviews. Data collection took place between February and June 2021 (patient surveys retrospectively included January).

Sample

Patients/carers who received these services and staff leading and delivering services (see Chapter 2 and Chapter 4 for sample details).

Measures

We developed the survey and semistructured topic guides specifically for this study (see Chapter 2 and Project website documentation – Study instruments).

Data collection

We conducted a survey with patients/carers from 25 sites and with staff from 28 sites. We conducted interviews with staff, patients and carers from 17 sites (see Chapter 2 for details).

Analysis

Descriptive statistics were calculated to explore staff survey responses relating to perceived patient groups facing barriers. Open text responses relating to local service adaptations and patient groups facing barriers were analysed; responses were triangulated with service lead interview data, coded and grouped into themes.

For patient survey data, non-parametric univariate analyses (i.e. Mann–Whitney U-tests and Kruskal–Wallis tests) were conducted to explore patient engagement with services and experience of services across patient characteristics. Where statistically significance differences were detected (at the p < 0.01 level) across more than two groups, post hoc analyses were conducted. For both patient and staff survey responses, where data were missing for specific questions, available case analysis was performed and the denominator reported.

We conducted logistic regression modelling to examine whether modality of the service, age, education, health status and ethnicity were associated with likelihood of patients reporting at least one problem with the service. Patient open text survey responses providing feedback about services and any suggested changes or improvements were triangulated with patient interview data, coded and grouped into themes.

Participant characteristics

Surveys were received from 292 NHS staff and 1069 surveys were received from patients and carers. We conducted 23 interviews with service delivery leads and 62 interviews with patients/carers (see Chapter 4 and Report Supplementary Material 2 for demographic characteristics).

Comparison of the characteristics of patient survey respondents with those of more than 26,000 patients onboarded to CO@h services114 indicate the survey sample to be relatively representative of patients engaging with the service. However, several differences should be noted. The survey sample is under-representative of patients over 80 years (5% vs. 10%) and under 50 years of age (21% vs. 33%), and over-representative of patients aged 50–80 years (74% vs. 57%). The survey sample comprises a higher proportion of patients from white ethnic groups (91%) compared to those onboarded to services (76%). Finally, our sample is over-representative of patients in the least deprived deciles of the IMD (18% vs. 13%) and under-representative of patients in the most deprived deciles (24% vs. 28%).

Service adaptations to increase service reach and patient engagement

Staff reported their views of patient groups facing barriers to engaging with the service. Some 39% (n = 113/292) reported patients for whom English was not their first language, 33% (n = 97/292) patients with a visual or hearing impairment, 31% (n = 91/292) cognitive impairment, 22% (n = 63/292) patients with a learning disability, 21% (n = 61/292) those who are digitally excluded, 13% (n = 37/292) older adults, 7% (n = 20/292) ethnic minority groups, and 5% (n = 16/292) unpaid carers. Other patient groups that staff reported in surveys or interviews included those living alone without support, patients unable to collect the oximeter, people with mental health problems, asthmatics, pregnant patients, patients whose GP had not engaged with the service, those with difficulty communicating via phone, and patients without a fixed abode.

Two-thirds of service leads (16/24 sites) reported that their service had been adapted at a local level to accommodate specific service user needs or requirements. Examples included providing information in different languages, offering translation services, offering non-digital monitoring options, face-to-face assessments, and flexibility of monitoring. Service-lead interviews and survey data indicated there was considerable local variation in the number of strategies adopted by sites. For example, several services made considerable efforts to standardise the coverage of the service, invest resources into setting up special pathways targeting hard-to-reach, vulnerable or at-risk groups, and monitor uptake. National strategies and local service adaptations reported by staff are set out in Appendix 4, Table 27.

Disparities in patients’ reports of their ability to engage with the service

In this section, we present findings from patient-reported engagement with these services in terms of the accessibility of information, the achievability of tasks, and problems reported via the survey (drawn from the survey analysis). See Report Supplementary Material 8 for further details. We also use findings from patient interviews and open-text survey responses (referred to as ‘qualitative data’) relating to patient experiences of the impact of service adaptations aimed at addressing inequalities.

Disparities in patients’ reports of their experience of the service

We found evidence of differences in patient reported experience and helpfulness of services with age and employment status, however reassurance provided did not differ markedly across patient groups.

Older patients reported services to be less helpful (p = 0.004); fewer patients aged 80 years and over reported services to be ‘helpful’ or ‘very helpful’ (88%) compared with those in the under 50 years (92%) and 50–64 years age categories (92%). Older patients also reported a less positive experience of services (p = 0.034), although this was not statistically significant (at the p < 0.01 level). The less positive experiences of patients aged 80 years and over might be explained by differences in understanding information about services and ease of completing monitoring tasks; however, they might also be due to the smaller sample size of the 80 years and over age category.

Patients in employment rated services more positively (p = 0.001) and found services more helpful (p = 0.003); 96% rated it ‘good’ or ‘excellent’ and 93% ‘helpful’ or ‘very helpful’ compared with 92% and 87%, respectively, among patients not in employment. The differences in patient experiences based on employment status might be related to other factors, such as health status, level of education or age, with older patients more likely to be retired.

Living situation was not identified in the survey data to be a significant factor in patient reported experience, or reassurance provided. However patient interview data indicated living situation and strength of social network as important factors; provided by services was reported to be particularly important for those socially isolated or living alone.

Key findings

At a local level many sites designed and adapted their service to be more inclusive to the needs of their local population and to expand their reach. Strategies included: providing information in different languages or formats, offering translation services, offering analogue and tech-enabled options for relaying readings, face-to-face assessments, delivering oximeters, providing additional training or support, flexible monitoring processes, and liaising with family or friends. There was considerable local variation in the adaptations employed by services. The adoption of such strategies was often dependent on local leaders as to how patient groups were approached and supported,90 and on the resources available to deliver the service.

Despite efforts to adapt services to meet local needs, there were disparities across patient groups in their experience of, and engagement with, services including age, employment status, ethnicity, level of education, health status and gender.

How findings relate to previous research

Despite previous evaluations investigating models of remote home monitoring and implementation of such services for COVID-19 patients,36,109 to our knowledge, this is the first research that focused on strategies adopted by services to address disparities. Our findings are consistent with previous research of strategies that can be used to reduce disparities in access to and use of health services more broadly,204–207 for example, expanding admission criteria and referral routes, and providing information in a range of formats (although support was provided at a national level, not all services provided information in formats accessible to all patients). Our findings extend research by outlining some strategies not previously identified; for example, staff providing additional support beyond monitoring (linking with social support, or more contact time), amending protocols to patient need and liaising with family/carers. Some possible areas for improvement included the need to routinely collect and analyse data on relevant populations, co-developing platforms with patients and staff (there was limited opportunity due to the rapid implementation) and working with communities to promote the service.

Critiques of remote models of care often focus on possible harms arising from digital exclusion. Having analogue options available as part of COVID-19 remote home monitoring services is, therefore, crucial to prevent increasing health disparities. 208 This is particularly important given the overlap between patient groups at greater risk of severe illness from COVID-19 (e.g. older adults) and those who are at greater risk of digital exclusion. Our finding that mode of service did not impact on patient engagement or experiences might be explained by patient satisfaction or the fact that all sites offered analogue modes (see Chapter 11).

Implications

Our evaluation generates a range of recommendations for the design and delivery of remote monitoring services for patients with COVID-19, and other health conditions. Services must consider the needs of their local population and be adapted accordingly, monitor the impact of services on population groups at risk of health disparities and work with existing local systems (e.g. community groups) to engage harder to reach groups.

Given that many of the groups of patients (e.g. clinically vulnerable, older adults, those without support) that services aimed to reach found it more difficult to engage with the service, patients’ needs and circumstances should be assessed upon referral in a standardised way and services must be tailored to provide appropriate levels of support for patients and carers (see Chapters 9 and 10 results and Miles and Huberman,203 Cooper et al. 209 and Centre for Ageing Better210). Examples of appropriate tailoring could include patient information in a range of formats,211 additional information, education or support in navigating services,212 and flexible monitoring processes.).

When implementing remote models of care, adequate resources and infrastructure should be provided to allow patients to be involved in the design and implementation. Co-design can help to ensure that services are better suited to patient needs and can facilitate adherence and engagement. 213 Holistic health care and collaboration or linking between services have previously shown to be important for reducing inequalities, particularly for those who are unwell or have comorbidities. 209,213

Strengths and limitations

The analysis of experience and engagement across patient groups could have been subject to false positives due to the number of comparisons made, although a more stringent p-value was used in reporting significant results (p < 0.01). When making comparisons between patient groups, the sample size of several groups was relatively small (e.g. patients aged over 80 years), limiting generalisability. The planned multivariate regressions investigating whether mode of monitoring and patient characteristics were associated with patient experience of the service were not possible due to the skew within the dependent variable (and therefore insufficient power to detect any differences). The interviews with patients focused on their experiences of services more broadly and not necessarily the identification of inequalities in access or quality of care, so much of our analysis of the latter draws on staff perspectives or patient survey data.

Future research

There is little published literature on the implementation of remote monitoring and health disparities. There are other populations at risk of health disparities that were beyond the scope of this evaluation and should be considered in any future evaluations (e.g. the homeless community, people with disabilities and unpaid carers).

The scope of our evaluation also did not allow us to examine the impact of specific strategies used to increase accessibility or patient engagement, or the impact of specific adaptations made by local services. We are therefore not able to make any conclusions regarding the effectiveness of strategies in relation to health outcomes for particular patient groups. This is an area for future research.

Conclusions

When health services undergo such a rapid transformation, as we have seen with the shift towards remote models of care, evaluations of the effectiveness of such services must include an impact assessment to ensure services are accessible and available to all patients, and to monitor the impact on health disparities. Addressing health disparities must therefore be a key focus in the planning, design and delivery of remote monitoring models for COVID-19 and other conditions, both nationally and locally, so that remote models of care can be of value to all population groups. Staff and patients from groups typically experiencing disparities in their health care must play an active role in service design to ensure their needs, experiences and expectations are accounted for.

Design

This chapter draws on staff and patient/carer surveys and staff and patient/carer interviews. Data collection took place between February and June 2021 (patient surveys retrospectively included January).

Sample

Patients/carers who received these services and staff leading and delivering services or involved in service data collection/analysis (see Chapter 2 and Chapter 4 for sample details).

Measures

We developed the survey and semistructured topic guides specifically for this study (see Chapter 2 and Project website documentation – Study instruments). Interviews with delivery staff included a ‘think aloud’ section where staff narrated the process of using the platform.

Data collection

We conducted a survey with patients/carers from 25 sites and with staff from 28 sites. We conducted interviews with staff, patients and carers from 17 sites (see Chapter 2 for details).

Data analysis

Inductive thematic analysis was used to analyse interview data (relating to patient views/experiences of relaying symptom data, staff processing data and interactions between patients and clinicians), inputted into RAP sheets. 70 The PERCS domains of technologies and the clinical relationship were used as a sensitising lens. 208 Transcripts were reviewed to gain further in-depth data on reasons for adoption of models and knowledge gained from models. Each code’s data were checked for consistency of interpretation and re-coded as necessary. The data were organised into lower- and higher-order themes. We constructed the typology of the models based on two key characteristics: the modality of the technology and human contact between patients and clinicians. Data relating to modality and frequency of human contact were coded deductively.

Survey data were analysed using SPSS (version 25). The following patient data were analysed: sociodemographic information, patient-reported ease of completing activities as part of their care, frequency and quality of contact with the remote monitoring team, and whether they had received any support from family or friends. Data related to experience of relaying data and the clinical relationship were also coded deductively from two open-ended survey questions. Survey responses were compared with patients who had reported they had relayed readings via tech-enabled or analogue modes. We analysed staff survey respondent characteristics. Staff reported on their experience of delivering the service, capacity/resources to deliver the service, the impact of the role on their workload and their views of patient engagement with the service. Data related to processing and responding to symptom data and the clinical relationship were coded deductively from one open-ended survey question. To make comparisons across models, staff were categorised as delivering a tech-enabled and analogue model if sites adopted a mixed approach, or an analogue-only model if a tech-enabled platform was not used.

For patient and staff survey data, we used descriptive statistics including frequencies and percentages, and univariate analyses were performed. Where data were missing for specific questions, available case analysis was performed and the denominator reported. We conducted Pearson’s chi-square tests and logistic regression modelling to compare the characteristics of patients using tech-enabled and analogue-only models and to determine the association between patient characteristics and mode of submission. We used non-parametric Mann–Whitney U-tests to compare patient-rated experiences of receiving services and staff experiences of delivering services for tech-enabled and analogue-only models.

Participant characteristics

Surveys were received from 292 NHS staff and 1069 patients and carers. We conducted 58 interviews with staff and 62 interviews with patients/carers (see Chapter 4 and Report Supplementary Material 2 for demographic characteristics).

The majority of staff reported delivering tech-enabled and analogue services (81% interview participants, 77% survey participants) compared with analogue-only models (19% interview participants, 23% survey participants).

More patients reported using a tech-enabled mode for data submission (66% of interview participants and 51% of survey participants) than an analogue-only mode (31% of interview participants and 49% of survey participants). Of the patients surveyed: 29% reported providing readings via text, 25% via digital app, and 1% via e-mail.

A comparison of participant characteristics for survey participants who used tech-enabled compared with survey participants who used analogue methods of submission is shown in Report Supplementary Material 8. The logistic regression analysis found statistical evidence of differences in patient characteristics between survey participants who used tech-enabled versus analogue-only models, for age (p = 0.005), level of education (p = 0.011) and ethnicity (p = 0.043) (see Report Supplementary Material 8).

Models of COVID-19 remote home monitoring adopted

Services were categorised into four models (Figure 25). Services within model 1 (10/17 services) and model 2 (4/17 services) offered tech-enabled and analogue services submission modes. Services within model 3 (1/17 service) and model 4 (2/17 services) offered analogue-only modes of submission. Models 1 and 3 outline ‘universal’ services which offered the same submission options and frequency of human contact to all participants, whereas models 2 and 4 outline ‘risk-stratified’ services which adapted the submission method and/or frequency of contact depending on patients’ needs. For example, two services within model 2 offered self-monitoring (analogue-only options to low-risk patients) and some services within model 4 offered more frequent phone calls or face-to-face visits for high-risk patients. See Report Supplementary Material 4 for further details.

FIGURE 25.

Models of COVID-19 remote home monitoring.

Patients’ experiences of using tech-enabled and analogue methods to relay symptoms

We constructed two key themes related to patients’ experiences of relaying symptom data using tech-enabled and analogue, and analogue-only modes: (1) allocation of mode; and (2) ease of relaying readings and the functionality of tech-enabled systems.

Staff experiences of using the models to process symptom data for large numbers of patients

Two themes emerged as key to staff experiences of processing symptom data: the scalability of the models, and the ease-of-use and functionality of tech-enabled systems.

Patients’ and staff’s experiences of the clinical relationship

Three themes emerged relating to patients’ and staff’s experiences of the patient-clinician relationship embodied in the remote home monitoring models: staff knowledge of patients’ condition, reassurance and providing appropriate care, and concordance in the patient-clinician relationship.

Key findings

Tech-enabled care was not a substitute for human contact in COVID-19 remote monitoring services; phone calls and human relationships were a feature of all models of remote monitoring and patients found services reassuring (across models). This indicates that technology cannot completely replace phone calls. Patients using tech-enabled methods tended to be younger, well-educated and more likely to identify as white British compared with those using analogue methods and most patients did not recall being given a choice about the mode offered. Staff considered tech-enabled and analogue services to be more scalable and able to deal with larger volumes of data and patients than analogue-only services.

How findings relate to previous research

Consistent with previous studies,164,216,218 staff noted that better functioning systems that fit with their work practices would have greatly improved efficiency, highlighting the need to involve clinicians in the design and development of technical platforms. Patient involvement was also missing from their development; patient participation groups and patient experience work were considered non-essential early on in the pandemic. 223 As a number of different tech-enabled models have now been established, there is an opportunity for service leads to trial different systems and incorporate patient feedback/involvement.

Patients in all models were contacted by telephone if their symptoms deteriorated or they did not submit readings, with the exception of two sites that offered a self-monitoring service. Staff reported that pursuing non-submitting patients could unnecessarily consume resources. Consistent with previous research showing staff concerns about patient safety in chronic conditions, the acute nature of COVID-19 caused staff anxiety about rapid deterioration. 164 There is a question of whether services’ resources could have been better optimised for patients’ needs, by addressing concerns about risk in staff supervision and training, by offering more lower-risk patients self-monitoring, by adapting tech-enabled platforms to offer varying levels of automated contact, and/or following more stringent protocols for managing non-submitters (e.g. discharging patients back to the care of their GP if they miss three data submission points).

Previous studies have shown that both patients and staff have responded positively to remote home monitoring for chronic health conditions,216,224 and a small number of studies have been conducted on patients’ experiences of COVID-19 remote home monitoring. 211 This is the first study to examine in-depth patient and staff experiences of using tech-enabled and analogue modalities to respectively relay and process COVID-19 symptom data and compare experiences of the patient-clinician relationship. It builds on our work looking more broadly at patients’ experiences of COVID-19 remote home monitoring services (see Chapter 9).

This study supports prior work that has shown that some patients may need additional support to engage with technology or access to analogue options,208,220 and the importance of maintaining human contact. 221,222 Older patients, less well-educated patients and patients identifying as non-white were more likely to relay symptoms through phone calls with the service, with staff primarily determining the allocation of mode. Co-designing digital health services with patients, improving digital health literacy in the population, training and supporting the workforce to build digital skills and confidence, and gathering data on patients’ access to technology and digital skills could widen patient participation in digital health care. 206,215

Staff engagement has been explored in previous studies,164,216 but this study extends previous research by considering the scalability of models, with staff considering tech-enabled and analogue models better equipped for managing large patient numbers without jeopardising clinical relationships. However, future models of remote monitoring should tailor their frequency of contact to patients’ level of risk and include systems which create a dialogue between patients and staff as to their preferences.

Strengths and limitations

There was a high level of variation between different models (see Chapter 4). Few sites adopted analogue-only modes of data submission, and no sites used tech-enabled-only modes, making it difficult to assess whether differences in patient and staff experiences between tech-enabled and analogue models were due to the engagement with the technology, or related to other service factors. This also meant that it was not possible to use multivariate analyses to look at staff experiences (based on survey responses) across models, due to the limited sample size in analogue-only sites. The univariate analyses of staff and patient experiences could therefore have been subject to false positives due to the number of comparisons made. However, in-depth data collection from staff on the functionality of tech-enabled models at four sites enabled us to draw out specific tech-related findings.

We had originally planned to use ‘think aloud’ methodology with patients but in practice, these did not work because patients no longer had access to the tech-enabled platforms they had used and their recall of their use and functionality during ill health was poor.

The language and terminology describing technology was unfamiliar to some staff and patients, for example, understanding the difference between a web-link and a digital app.

Conclusions

Organisations implementing tech-enabled remote home monitoring at scale should consider providing flexible models which can tailor processes and resources to accommodate patients with different needs and requirements. These include risk levels, abilities to engage with and access technology, support available at home from family or carers, and levels of desire for support versus self-management. Services routinely collecting and monitoring data on patients’ access to technological infrastructure and skills could inform digital inclusion strategies for remote home monitoring.

Key findings from phase 1 (see Chapter 3)

Key findings from phase 2

Implementation

Evaluating effectiveness and cost of services

Staff and patient experiences

Strengths

Our evaluation was a large-scale mixed methods evaluation which included analysis of national data sets, and primary data collection from 28 sites that were representative of a range of regions, levels of deprivation, ethnicity, size of population, and a range of urban versus rural sites. However, it is possible that our study may not be representative of all sites. Using mixed methods (consisting of a systematic review, qualitative data collection and analysis, quantitative data collection and analysis and economic data collection and analysis), enabled us to explore a range of topics and triangulate data from multiple perspectives and sources. For example, we were able to explore possible reasons for low patient enrolment demonstrated by the quantitative national data sets within the qualitative case-study findings.

This evaluation was a rapid evaluation completed over two time periods. Our phase 1 study was completed within one months, and our phase 2 mixed-methods study was completed within one year, with data collection taking less than six months. Our rapid approach to this evaluation meant that learnings from the evaluation were able to be used in real time to inform future service development and evaluation (see Project website documentation – Dissemination). Additionally, conducting our evaluation over two time periods (coinciding with waves 1 and 2 of the pandemic) enabled us to capture learnings and constants/changes over time.

Throughout our evaluation, a number of actions were taken to ensure robustness of qualitative findings (see Chapter 2).

Our evaluation team (a multidisciplinary research team) worked collaboratively with, yet independently from two other evaluation teams and key stakeholders from NHSEI and other organisations. This meant that evaluation teams were able to share findings to evaluate services from multiple perspectives, thus providing findings with more credibility. Additionally, our evaluation team worked closely with our PPI group and the 70@70 cohort throughout the study.

Limitations

Data completeness was a limitation of the quantitative and cost elements of this study. The evaluation of effectiveness for CO@h could only be carried out for CCGs where the onboarding data was believed to be complete, which was only one quarter of all CCGs. For CVWs, we were able to undertake a national analysis, but we had no knowledge about which patients were referred to a virtual ward, or even how many. Additionally, cost and resource use data provided by some sites were incomplete.

Our CO@h effectiveness study (see Chapter 5) and the cost study (see Chapter 7) used aggregated data rather than individual patient-level data for key analyses. This meant that patient-level variation in outcomes and programme costs could not be examined. However, patient-level data on outcomes were examined by the other evaluation teams. 42–45

It was not possible to conduct all aspects of the evaluation as planned in the protocol. For example, a cost–utility analysis was not undertaken, given there was no evidence found of an effect of COVID-19 remote home monitoring services on mortality or downstream hospital utilisation. Additionally, for the qualitative study, we were unable to fully embed ‘think aloud’ interviews within our tech-enabled four case study sites as this was not feasible in practice.

Compared with patient onboarding data,114 the sample in the patient experience study was under-representative of some groups (e.g. older patients, ethnic minorities and most deprived) and over-representative of other groups. Additionally, the response rate for the patient and carer survey was fairly low (17.5%). For patient experience surveys and interviews, participants who had declined services could not be recruited. Additionally, our study would not have included the views of staff or patients who were unable or did not want to take part in surveys and interviews. Therefore, our study may not be representative of all patient and staff groups and experiences.

These services were rapidly developed to respond to the global pandemic. Additionally, our evaluation was rapidly conducted within a pandemic. Therefore, our conclusions may lack generalisability to other acute conditions, as staff and patient expectations and attitudes may be different during a pandemic (e.g. perhaps staff and patients may be more appreciative of the service, given the wider context).

Participant representation

Within this evaluation, we aimed to ensure that participants were representative of different locations within England and different backgrounds.

Reflections on research team and wider involvement

Published:

Georghiou T, Sherlaw-Johnson C, Massou E, Morris S, Crellin NE, Herlitz L, et al. The impact of post-hospital remote monitoring of COVID-19 patients using pulse oximetry: a national observational study using hospital activity data. eClinicalMedicine 2022;48:101441. doi: https://doi.org/10.1016/j.eclinm.2022.101441

Greenhalgh T, Knight M, Inda-Kim M, Fulop NJ, Leach J, Vindrola-Padros C. Remote management of covid-19 using home pulse oximetry and virtual ward support. BMJ 2021;372:n677. https://doi.org/10.1136/bmj.n677

Sherlaw-Johnson C, Georghiou T, Morris S, Crellin N, Litchfield I, Massou E, et al. The impact of remote home monitoring of people with COVID-19 using pulse oximetry: a national population and observational study. EClinicalMedicine 2022;45:101318. https://doi.org/10.1016/j.eclinm.2022.101318

Sidhu MS, Ford GA, Fulop NJ, Roberts CM. Learning networks in the pandemic: mobilising evidence for improvement. BMJ 2022;379:e070215. https://doi.org/10.1136/bmj-2022-070215

Sidhu M, Walton H, Crellin N, Ellins J, Herlitz L, Litchfield I, et al. Staff experiences of training and delivery of remote home monitoring services for patients diagnosed with COVID-19: a mixed-methods study. J Health Serv Res Policy 2023;0(0). https://doi.org/10.1177/13558196231172586

Vindrola C, Fulop N, Greenhalgh T. Virtual wards: caring for COVID-19 patients at home could save lives. The Conversation. 2020. URL: https://theconversation.com/virtual-wards-caring-for-covid-19-patients-at-home-could-save-lives-149292 (accessed 18 July 2023).

Vindrola-Padros C, Sidhu MS, Georghiou T, Sherlaw-Johnson C, Singh KE, Tomini SM, et al. The implementation of remote home monitoring models during the COVID-19 pandemic in England. EClinicalMedicine 2021;34:100799. https://doi.org/10.1016/j.eclinm.2021.100799

Vindrola-Padros C, Singh KE, Sidhu MS, Georghiou T, Sherlaw-Johnson C, Tomini SM, et al. Remote home monitoring (virtual wards) for confirmed or suspected COVID-19 patients: a rapid systematic review. EClinicalMedicine 2021;37:100965. https://doi.org/10.1016/j.eclinm.2021.100965

Walton H, Crellin N, Sidhu MS, Sherlaw-Johnson C, Herlitz L, Litchfield I, et al. Undertaking rapid evaluations during the COVID-19 pandemic: Lessons from evaluating COVID-19 remote home monitoring services in England. Front Sociol 2023;8:982946. https://doi.org/10.3389/fsoc.2023.982946

Walton H, Fulop NJ. Virtual wards and Covid-19: An explainer. Nuffield Trust. 2022. URL: www.nuffieldtrust.org.uk/resource/virtual-wards-and-covid-19-an-explainer (accessed 18 July 2023).

Walton H, Vindrola-Padros C, Crellin N, Sidhu MS, Herlitz L, Litchfield I, et al. Patients’ and carers’ experiences of, and engagement with remote home monitoring services for COVID-19 patients: a rapid mixed-methods study. Health Expectations 2022:25:2386–404. https://doi:10.1111/hex.13548.

Preprints/submitted for review

Crellin N, Herlitz L, Sidhu MS, Ellins J, Georghiou T, Litchfield I, et al. Examining disparities relating to service reach and patient engagement with COVID-19 remote home monitoring services in England: a mixed-methods rapid evaluation. medRxiv 2021. https://doi.org/10.1101/2022.02.21.22270793

Herlitz L, Crellin N, Vindrola-Padros C, Ellins J, Georghiou T, Litchfield I, et al. Technology-enabled and analogue models of remote home monitoring care for COVID-19: a rapid mixed-method evaluation of patient and staff experiences. Submitted for publication June 2023.