Modelling of hypothetical SARS-CoV-2 point of care tests for routine testing in residential care homes: rapid cost-effectiveness analysis

Article history

The research reported in this issue of the journal was commissioned and funded by the Evidence Synthesis Programme on behalf of NICE as project number NIHR132154. The protocol was agreed in August 2020. The assessment report began editorial review in January 2021 and was accepted for publication in April 2021. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. This report has been published following a shortened production process and, therefore, did not undergo the usual number of proof stages and opportunities for correction. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2021 Stevenson et al. This work was produced by Stevenson et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2021 Stevenson et al.

The conceptual model: residents and staff

The EAG spoke with topic experts to validate the proposed conceptual model and to highlight potential sources of data that could populate the model. It was clear that there were potential areas in which residential care facilities could differ in interpretation of past and current guidelines, and so the model has been structured to allow an exploration of the changes in clinical effectiveness and cost-effectiveness of SARS-CoV-2 tests under different underlying assumptions relating to SARS-CoV-2 test accuracy, actions taken on the result of the SARS-CoV-2 test, turnaround times for SARS-CoV-2 tests, configuration of the residential care facility and changed assumptions relating to the spread of SARS-CoV-2 within the residential care facility.

The EAG modelled single bubbles within a care home. A simplified schematic of a typical residential care facility ‘bubble’ is shown in Figure 1. It is assumed that a large residential care facility may have multiple bubbles that could comprise different floors or wings, allowing residents to be separated and prohibit mixing between different bubbles. The analyses presented focus on a bubble within a residential care facility. Advice provided by care home representatives and experts indicated that splitting larger residential care facilities into bubbles was commonplace.

FIGURE 1.

A simplified schematic of key elements in the modelling of a residential care facility.

The use of SARS-CoV-2 testing, in particular the differences between laboratory-based tests and POCTs, is not shown in this diagram, but will be discussed in this chapter. The pathway for residents within Figure 1 is relatively simple. Within the residential care facility, residents are assumed to be either in isolation or free to mix, maintaining social distancing where possible. Residents leave the residential care facility when they die or when hospital treatment is required. Residents who have died are replaced, following a delay, by new residents. Further details are provided in Detailing the elements of Figure 1.

As the consequences of a resident contracting COVID-19 can be severe, including, potentially, death, the modelling assumes a lifetime perspective. Discounting is undertaken at a rate of 3.5% per annum for health, as recommended by NICE. 6 Costs were considered in the initial year only and are not discounted.

The conceptual model: introduction of SARS-CoV-2 to the residential care facility

Severe acute respiratory syndrome coronavirus 2 can be introduced into a residential care facility in a number of ways, for example a staff member having caught SARS-CoV-2 in the community, a resident having caught SARS-CoV-2 following a visit from a family member or from new residents to the residential care facility arriving with a pre-existing SARS-CoV-2 infection. The probability of these events happening are uncertain and may be rare. In addition, these events would require many simulations to provide a robust answer, with many of these simulations consisting of a residential care facility that remains free of SARS-CoV-2 throughout the modelling time horizon.

To counter this potential inefficiency, an alternative approach was adopted in which a seed who had just been infected with SARS-CoV-2 infection was sown in the residential care facility. This seed could be a result of an asymptomatic transmission from a visitor, new resident, residents attending outpatient appointments or a member of staff. This seed could be either a resident or member of staff (either attending to isolated or non-isolated residents) who has been infected with SARS-CoV-2; however, for time reasons, the model is run with a seed resident in the base case, although sensitivity analyses explore the impact of the seed being a staff member. Each model run estimates the impact on a residential care facility 90 days (3 months) after the seed penetrates the bubble. The 3-month duration is arbitrary, as any (non-infinite) time point would not consider the full pathway of all potential infections and 3 months was believed to be a reasonable period over which indicative results would be useful. This aligned with the work for SARS-CoV-2 testing in hospitals, where the pressure in winter was likely to be more acute. The seed will then progress through the relative stages of SARS-CoV-2 infection: becoming infectious, potentially becoming symptomatic (or remaining asymptomatic) and then becoming non-infectious, if the seed did not die from COVID-19. While infectious, the seed has the potential to infect both fellow residents and staff members, and these processes are described in more detail in Chapter 3, Parameters relating to the simulated spread of SARS-CoV-2 infection within the residential care facility. Longer durations were not run. However, it is unlikely that a longer time horizon would alter the results because of the assumption that a bubble could be penetrated only once. The results from the 3-month period are believed to be generalisable to longer periods.

The placement of the seed was at a random time point; therefore, the time to the next set of routine testing for staff and residents (see Chapter 3, Testing strategies within the residential care facility) changed between trials (see Chapter 4, The number of different random number streams used per analysis). The seed was randomly chosen from all residents/staff within the residential care facility. It was assumed that only one penetration of SARS-CoV-2 would occur within the 90-day period.

There will be no spread of SARS-CoV-2 for residential care facilities that remain clear of SARS-CoV-2 infection; however, there may still be differences in the results for POCTs and laboratory-based tests. These results will depend on the specificity of the SARS-CoV-2 test used (as a false-positive result can cause isolation) and on the time to a test result if early release from isolation is permitted (see Chapter 4, The strategies modelled). Therefore, results have been run for residential care facilities that are not simulated to have a SARS-CoV-2 infection.

Assumptions relating to the initiation of unit lockdown

Following a suspected SARS-CoV-2 case, it is anticipated that the unit will enter lockdown, with all residents entering isolation following the government guidance. 7 For larger residential care facilities that have multiple floors and wings, it is assumed that only the specific bubble would enter lockdown. The model has the functionality to use different criteria for the initiation of a lockdown, which change the level of certainty required. In the model, the following criteria are available to initiate isolation for all residents: symptoms suggestive of COVID-19 in a resident, symptoms suggestive of COVID-19 in a staff member, a positive SARS-CoV-2 result for a resident and a positive SARS-CoV-2 result for a staff member. Initiating full lockdown on clinical symptoms alone has the advantage that this may reduce the spread of SARS-CoV-2 within the residential care facility; however, it has the disadvantage that residents would spend more time in isolation. Conversely, isolating the individual with symptoms suggestive of COVID-19 but waiting for a positive SARS-CoV-2 test result to isolate all other residents has the advantage that isolation is not as lengthy, but there is danger that SARS-CoV-2 could be spreading within the residential care facility between the time of clinical symptoms and the time of the SARS-CoV-2 test result.

The results presented in this report provide two broad configurations of residential care facilities. The first residential care facility is assumed to be configured with single-occupancy rooms that have en suite facilities. Therefore, it is assumed that, in lockdown, all residents are isolated in their individual rooms and that there would be contact with only those members of staff attending to the resident.

The second configuration is to assume that the residents have shared rooms and bathrooms. In this instance, it is assumed that residents have pre-assigned rooms and that these cannot be changed, even if a resident were to show signs suggestive of COVID-19. In lockdown, people within the pre-assigned groupings would continue to interact and would therefore have a possibility of spreading SARS-CoV-2. This configuration is extreme and residential care facilities would attempt to segregate people with symptoms. However, the results produced are expected to be useful to the NICE Diagnostics Advisory Committee.

The model has the functionality to examine an in-between configuration, where residents have single rooms but need to share common facilities (e.g. a bathroom). In this configuration, the residents were assigned to ‘sharing groups’ to reduce the spread of infection. One sharing group would be for residents deemed to be at higher risk of having SARS-CoV-2 than the average resident. Residents falling into this group would be known to have either a positive SARS-CoV-2 test result or to have symptoms suggestive of COVID-19. All remaining residents would be allocated to the lower-risk sharing group. The higher- and lower-risk sharing groups may also have subgroups if there are sufficient bathrooms to make this possible. Although this functionality exists, results were not produced because of time constraints.

Assumptions related to the release of residents from isolation

Following a resident or staff member developing clinical symptoms suggestive of COVID-19, or a positive SARS-CoV-2 test result in a resident or a staff member, all residents are isolated. The standard time in isolation is assumed to be 14 days, as reported in guidance. 7 However, the model has the functionality for this duration to be shortened or extended in certain circumstances, if these options are selected.

The default option in the model is that all residents stay in isolation for 14 days once isolated, with all residents tested following a resident/staff member displaying clinical symptoms suggestive of SARS-CoV-2 or a resident/member of staff receiving a positive SARS-CoV-2 test result. This is because residential care facility managers know that current SARS-CoV-2 tests have less than perfect sensitivity, meaning that the test result may be a false negative. Some support for this approach is provided in the BGS guidance,4 which states that residents with a negative SARS-CoV-2 test result should not be released from isolation before a discussion with an appropriate physician is held, but this also leaves an option for a resident to be released from isolation if deemed clinically appropriate. To assess the impact of releasing residents from isolation early, the model has the functionality to release residents from isolation provided that two criteria are met: (1) the resident has returned a negative SARS-CoV-2 test and, in the shared facilities model and (2) the resident has not been in shared facilities with a resident who has a positive SARS-CoV-2 test or symptoms suggestive of COVID-19, a resident who is awaiting the result from a SARS-CoV-2 test or a ‘new’ resident, either residing in the residential care facility for the first time or having returned from a stay in hospital (see Chapter 3, The proportion of residents with COVID-19 who experience severe consequences). This option has the advantage that residents have a reduced time in isolation, but does carry risk that some SARS-CoV-2 test results could be false negatives and, consequently, SARS-CoV-2 could spread within the residential care facility. It is anticipated that the default option and the early release option will provide extreme positions with respect to the release of residents from isolation.

The model has the facility to allow all residents to be released from isolation if the resident suspected of being the index case within the residential care facility has a negative SARS-CoV-2 test result. An index case is defined as the first resident who is suspected of having SARS-CoV-2 infection when the bubble is not in lockdown. If the option to release residents early is selected, then residents can be released from isolation, provided that they meet the criteria in the paragraph above; however, this option is more restrictive, as the SARS-CoV-2 test for the index resident must be negative for all residents to be released. The model also assesses whether or not, in the time required for the SARS-CoV-2 test to be returned, a resident has developed clinical symptoms suggestive of COVID-19. In this scenario, the resident would also be required to have a negative SARS-CoV-2 test result before the bubble is released from isolation.

The rules for releasing residents from isolation, based on the possible model functionality, are provided in Table 1. Although these options are implemented in a Boolean fashion, the EAG acknowledges that, in reality, these decisions are likely to be nuanced and depend on resident characteristics and clinical judgement, which cannot be incorporated into the model in the allocated timelines.

The model has an additional functionality to extend the isolation time for people already in isolation when another incident occurs (either development of clinical symptoms suggestive of COVID-19 or the return of a positive SARS-CoV-2 test). This would require people already in isolation to be isolated for a further 14 days, regardless of the time already spent in isolation, increasing the isolation time beyond those in Table 1. This has the advantage that the resident is likely to be protected against SARS-CoV-2 infection, but has the disadvantage that the isolation time could be greatly increased.

Potential uncaptured benefits within the conceptual model

By necessity, the model did not capture all aspects of the decision problem; however, in the majority of cases, this is unlikely to affect comparisons between POCTs and laboratory-based SARS-CoV-2 tests. POCTs may be disadvantaged by the following omissions: the introduction of POCTs could help reduce the administrative burden associated with requesting and registering laboratory-based tests;9 could potentially facilitate easier access to residential care facilities by allowing the required SARS-CoV-2 test to be carried out at the residential care facility and; in the future, could allow potential treatment on identification of SARS-CoV-2 in symptomatic residents to be undertaken in a more timely manner because of the quicker turnaround times associated with the tests. The EAG notes that trials are under way to provide treatment for people with diagnosed COVID-19 to reduce the probability that it develops into severe COVID-19. 10

The demographics of residents in a residential care facility

The model sampled a starting age and determined a resident’s sex in accordance with published Office for National Statistics data on care home populations in England in 2020. 11 As the focus of the main model was on residents in a care home for older people, only those residents aged ≥ 65 years were considered (although younger residents were considered in sensitivity analyses). Table 2 shows the age and sex distribution of the modelled cohort. Interpolation between age ranges was undertaken to estimate the age of individual residents. The age breakdown was quite crude, with no statistics reported for median or mean ages. The EAG has assumed that the age distribution of male residents in the residential care facility can be approximated by a triangular distribution that has a mean of 83 years (minimum 65 years, mode 83 years and maximum 100 years). For female residents in the residential care facility, a triangular distribution that has a mean of 87 years (minimum 65 years, mode 96 years and maximum 100 years) was used in the model.

For simplicity, neither ethnicity nor comorbidities were modelled. Where residential care facilities have residents at a greater or lower risk of complications from COVID-19, this should be considered when interpreting the results. Sensitivity analyses have been conducted to assess the impact of assuming double QALY losses to consider cases where there are potentially worse consequences of COVID-19 and where more residents have severe SARS-CoV-2.

The characteristics of staff working in the residential care facility

At the start of the model, it is assumed that no staff have been infected by SARS-CoV-2 and that no staff exhibit clinical symptoms suggestive of COVID-19. Age distributions, ethnicity and comorbidity status for staff were not required, as the model does not explicitly take into consideration the consequences of infection for staff beyond infecting other staff and residents.

The configuration of the residential care facility used in the main model

As described (see The conceptual model: residents and staff), the model focused on single bubbles in a care home. From discussion with experts, it was clear that the size of bubbles was very heterogeneous between real-world residential care facilities. To provide example results, it was assumed that the bubble consisted of 16 residents (with demographics sampled in accordance with Table 2) and nine staff members, initially divided into six staff attending residents not in isolation and three staff attending residents in isolation. As a simplification, we have assumed that these staff are the full pool of staff across the week and that staff starting and ending shifts are only implicitly considered. Any resident–staff interaction involves a randomly chosen member from the staff pool and so not all staff will be assumed to be working at all times.

Two types of residential care facilities were considered. The first type was a residential care facility in which all rooms were en suite, in which case we assumed that, if lockdown were initiated, there would be no interaction with other residents in the bubble. The second type was a residential care facility in which it was assumed that the 16 residents were divided into four groups of four residents who shared rooms and bathrooms.

New arrivals in the residential care facility

The model considers two types of new residents in the residential care facility bubble: (1) those who have never been in the residential care facility and who are taking the place vacated by a resident who has died and (2) those who are returning to the residential care facility after a stay in hospital.

In the calibration model (see Chapter 4, Calibration of the model), it has been assumed that residents who die are not replaced. For the models tested, it has been assumed that there is a delay of 3 weeks between a resident dying and a new resident entering the room. The new resident is assumed to have had a recent negative SARS-CoV-2 test and to spend 14 days in isolation prior to being allowed to mix freely with other residents within the bubble.

The percentage of residents with SARS-CoV-2 who are assumed to develop clinical symptoms

There is considerable uncertainty related to the proportion of residents with SARS-CoV-2 who become symptomatic. This estimate can range from 70%, reported by Smith et al. ,12 to 19%, in the Vivaldi report,13 which reported that 5455 out of 6747 positive tests (81%) were in residents who were asymptomatic and therefore 19% of residents were symptomatic. Other estimates include the following: 61% of people with confirmed SARS-CoV-2 in a skilled nursing facility had clinical symptoms,14 56.2% of residents with a positive SARS-CoV-2 test in six care homes in London had clinical symptoms15 and 33% of those who tested positive in the community had clinical symptoms. 16 Within these estimates, it is not always clear if repeat testing was undertaken to establish whether or not the resident was pre-symptomatic and, if not, it is likely that these values will be underestimates. In addition, the exact definition of clinical symptoms is unclear. It has been noted in terms of the Vivaldi data that the definition of symptomatic was omitted and that other symptoms in addition to new cough, fever and loss of taste and smell may be appropriate in elderly care home residents. 17

Given the wide range in results, we have assumed in our model base case that 50% of residents would become symptomatic following infection with SARS-CoV-2. Sensitivity analyses were undertaken that assumed that 33% of residents became symptomatic.

The incidence of diseases that are not COVID-19 but have symptoms suggestive of COVID-19

For residents who have not been infected with SARS-CoV-2, the proportion of residents who have symptoms that are indicative of COVID-19, but do not have COVID-19, was estimated from Hayward et al. 18 to be approximately 0.12 in vaccinated residents over a 147-day period, covering the winter months and based on data for influenza-like illnesses. Assuming that a resident could have only one influenza-like illness during this period, this would translate to an estimated daily risk of 0.087%.

A sensitivity analysis was undertaken using data from Smith et al. ,12 who reported a daily incidence of COVID-like symptoms of 1.1%, which was calculated using data from French emergency departments, average daily incidence of influenza-like illness among older adults (50–99 years, 2008–2017) was used as a proxy attack rate for patients and staff presenting with COVID-like symptoms of other aetiologies. 12 The reported source was a conference abstract (Fouillet et al. 19) that does not mention influenza-like illness and, therefore, the robustness and generalisability of the 1.1% value are uncertain.

For simplicity, it was assumed that residents in isolation could not catch an illness that had symptoms suggestive of COVID-19. This could be unfavourable to strategies that allow early release, as it may result in residents having a higher incidence of non-COVID-19 illness than those held in isolation.

The proportion of residents with COVID-19 who experience severe consequences

Smith et al. 12 reported that 20% of residents with COVID-19 have severe COVID-19. We have assumed that this value is independent of age and that severe COVID-19 means that a resident either dies or requires hospital care. It is acknowledged that there may be residents too frail to be transported to hospital and these residents would not leave the residential care facility, but would have the same outcomes as if hospital treatment were provided. The data for these probabilities were from a previous report of exploratory economic modelling of hospital point-of-care SARS-CoV-2 viral detection tests produced by the EAG20 and assumed to be the same as the proportion of people who died in hospital with COVID-19 but who were not moved to the intensive care unit, presumably as they were too frail. These values, for residents aged ≥ 65 years, are shown in Table 3.

Based on the previous report of exploratory economic modelling of hospital point-of-care SARS-CoV-2 viral detection tests produced by the EAG, the probabilities of death for residents hospitalised with COVID-19 were calculated. These estimated values, conditional on age and sex, for people aged ≥ 65 years are provided in Table 4.

In the case of residents who require hospitalisation for COVID-19 reasons and then return to the residential care facility, it was assumed that the length of stay was distributed log-normally (on a log-scale) with a mean of 2.29 (standard deviation 0.59) days. This is in accordance with the time allocated to residents aged > 70 years in the previous EAG report of exploratory economic modelling of hospital point-of-care SARS-CoV-2 viral detection tests. This distribution was estimated from data in Docherty et al. 21 For simplicity, the costs of hospitalisation has not been included in the model. This will be unfavourable to the strategies with fewer SARS-CoV-2 infections.

In the case of residents who die from COVID-19, the distribution of length of stay was represented by a log-normal distribution with a mean of 2.00 (standard deviation 0.79) days, which was the distribution used in the previous work undertaken by the EAG, relating to the use of SARS-CoV-2 tests in hospital, and based on a study by Docherty et al. 21

The risk of non-COVID-19 mortality for residents

Data on the number of people in care homes and the number of deaths per age band not involving COVID-19 were taken from Office for National Statistics mortality data. 22 The population data used wider age bands (10-year bands) than the death data (5-year bands). Therefore, the EAG needed to make the assumption that the population count in a specific 10-year band is equally divided between the two included 5-year bands (e.g. the population data states that there are 15,838 males aged between 65 and 74 years; therefore, we assumed 7919 males for each of the 65–69 years age band and the 70–74 years age band). The estimated risk of death per 28-day period is shown in Table 5.

Immunity from SARS-CoV-2 infection

It has been hypothesised that some people may have immunity to SARS-CoV-2 infection, although this theory is subject to much debate. 23 In the base case, it has been assumed that no person has immunity to SARS-CoV-2 infection unless they had a SARS-CoV-2 infection at model initiation or were infected with SARS-CoV-2 during the simulation. The model has the functionality to allow a proportion of residents without current SARS-CoV-2 infection to be considered immune from infection.

Parameters relating to the simulated spread of SARS-CoV-2 infection within the residential care facility

The model simulates the probability of transmission of SARS-CoV-2 in the following four categories: (1) resident to resident, (2) resident to staff, (3) staff to resident and (4) staff to staff. These are controlled by variables relating to the chance of a contact in which SARS-CoV-2 could be transmitted (termed an effective contact), the likelihood of infection being transmitted given an effective contact (termed an attack rate) and a calibration factor that attempts to ensure that the values used relating to effective contacts and attack rates produce results that would be relatively consistent with those observed, to date, within residential care facilities (see Chapter 4, Calibration of the model). To decrease the running time of the model, it was assumed that the simulation of infections with SARS-CoV-2 occurred in distinct 6-hour periods.

Characteristics of the point-of-care tests and laboratory-based tests

The TPPs for POCTs and laboratory tests for SARS-CoV-2 are not linked to any marketed test, but are values put forward by the MHRA to designate what would be seen as desirable or acceptable characteristics for forthcoming tests. These values may bear no resemblance to the laboratory-based tests that were used in the early phases of the pandemic and so the EAG decided to use values for diagnostic accuracy from its previous work on the use of POCTs for SARS-CoV-2 in a hospital setting. This resulted in an assumed sensitivity of laboratory-based tests of 89%, based on a meta-analysis conducted by Kim et al.,29 with an assumed specificity of 1, following Grassly et al. 25 The turnaround time for laboratory-based tests for care homes in the early stages of the pandemic was estimated by experts in the research team to be 3 days. It is believed that the turnaround times for laboratory-based tests have improved over time, with data from NHS test and trace weekly reports30 reporting a median time to test result of 45 hours. We have assumed that this equates to a mean time of 2 days.

The assumed diagnostic accuracies of the POCTs and laboratory-based tests, the times for the result to become known by the treating physician and the failure rates used in the model base case are shown in Table 8. It is assumed that neither the laboratory-based test nor the POCT can detect SARS-CoV-2 until 0.5 days after infection (based on Jarvis and Kelley33). The model uses a simplifying assumption that SARS-CoV-2 cannot be detected until this point is reached, although false-positive tests could still occur in this situation. Beyond this time point, the sensitivity values reported in Table 8 are considered appropriate, which the EAG acknowledges as a simplification of viral load dynamics. It is believed that the sensitivity of SARS-CoV-2 tests may be reduced in asymptomatic people. Based on data provided in Clifford et al. ,26 it was assumed that the ratio of sensitivities was 0.826, having assumed 0.75 sensitivity in symptomatic cases and 0.62 for asymptomatic cases.

The TPPs published by the MHRA were used for both the POCTs and laboratory-based tests. 34,35 These were divided into desired and acceptable characteristics. The characteristics of these are presented in Table 8. The estimate of 3 hours for the POCTs was made, considering that it is probable that there would be only one POCT machine per residential care facility, and that swabs from residents and staff members would need to be stored and be in a queue before being processed. As detailed in Testing strategies within the residential care facility, a staggered approach to resident testing was employed; however, this would not be possible when there is a suspected case of COVID-19. In the instance of a suspected case of COVID-19, the average time to receive a result from a SARS-CoV-2 POCT was assumed to increase to 1 day.

In addition, real-world evidence is explored, as has been used for the laboratory-based tests in the calibration period. For SARS-CoV-2 POCTs, data from currently available SARS-CoV-2 POCTs were used. These data suggest that the time to obtain the result is 3.8 hours (based on data presented in Collier et al. 36 and with sensitivity and specificity taken from a Cochrane review by Dinnes et al. ,37 which were multiplied by the sensitivity and specificity of the real-world laboratory-based test diagnostic accuracy). It is seen that the real-world evidence for the SARS-CoV-2 POCT has better diagnostic accuracy than the acceptable TPP, with only a small increase in time to the test result, which would not be captured in the model because of the model updating every 6 hours.

Additionally, recent (after the completion of work for NICE) diagnostic accuracy data from testing using lateral flow methods in mass community testing have been reported,31 as has a Cochrane review. 32 These data have been used in sensitivity analyses and assume that the tests would be undertaken every 5 days for both staff and residents, and that the lateral flow test was £10 cheaper than the other SARS-CoV-2 POCTs and laboratory-based SARS-CoV-2 tests. From the mass community testing report,31 the sensitivity for all individuals, irrespective of viral load, was assumed to be 85.85% (mid-way between 77.8% and 93.9%) and specificity was assumed to be 99.68%, meaning that the diagnostic accuracy of the lateral flow tests was similar to real-world evidence for POCTs. The failure rate was maintained at 5%, despite a mid-point estimate of 5.4% (502/9276), as these data were skewed by the observation from ‘phase 4, school 1’, which is reported to had 311 failures out of 2166 tests (14.4%) and was also reported to have a 16.8% failure rate (the discrepancy between these numbers in not clear). The mass community testing report suggests that there might be a difference in failure rates between batches. If this is correct and the problem can be rectified, then the failure rates may decrease.

From the Cochrane report,32 sensitivity data for the four Instructions for Use-compliant evaluations were taken from three evaluations of symptomatic patients (69.1%) and specificity data were taken from two evaluations of symptomatic patients (99.7%). The assumed time to a result when using a lateral flow test was 30 minutes. These data have been added to Table 8.

The lateral flow analyses were not appraised by the NICE Diagnostic Committee. Caveats to this analysis include that the model does not include visitor testing, that confirmatory tests after a lateral flow test have not been included, behaviour change has not been assumed, that incorrect diagnoses are assumed to occur at random (as the diagnostic accuracy has not come from a strategy of repeated tests), that no analyses increasing the frequency of other POCTs have been modelled and that sufficient data to allow diagnostic accuracy specific to residential care facilities are not available. Therefore, conclusions related to lateral flow tests should be considered indicative rather than definitive. Recent work that assessed the implementation of staff and visitor testing protocols using lateral flow devices concluded that there was poor adherence to using the tests and no reduction in either the number or scale of COVID-19 outbreaks with the use of lateral flow tests. 38

The model has not considered the possibility that less well-trained staff could increase the spread of SARS-CoV-2 when performing the tests. We have assumed that training has been undertaken with care home staff to mitigate this.

Testing strategies in the residential care facility

In accordance with guidance,7 it was assumed that residents were tested every 28 days for SARS-CoV-2 and that asymptomatic staff members are tested weekly. In sensitivity analyses looking at lateral flow SARS-CoV-2 tests, the testing strategy was changed to every 5 days for both residents and staff. No analyses that reduced the frequency of testing for non-lateral flow POCTs were undertaken.

For residents, it was assumed that 16 residents are divided into four groups of four residents and that testing was undertaken on 4 consecutive days so that a single POCT is not overwhelmed. It is assumed that all asymptomatic staff members are tested in one batch, although, in reality, this would be scheduled according to their working patterns. It is further assumed that, when there is a suspected case of SARS-CoV-2 infection within the residential care facility, all residents and staff are tested immediately, with the turnaround times for test results increased to 1 day.

The assumed costs of SARS-CoV-2 tests

In the earlier work undertaken by the EAG, the cost of the hypothetical POCT was assumed to be £27.76,39 which is based on the 5.8 million DnaNudge (London, UK) tests ordered by the government at a cost of £161M, although it was acknowledged that, as the deal was likely to include 5000 platforms for running the tests, this could overestimate the cost. For laboratory-based tests, the cost was assumed to be £21.90,40 assuming that the cost of the urine and plasma laboratory-based test (BioPorto, Needham, MA, USA) used to assess acute kidney failure for people considered for admission to critical care is a reasonable proxy. The staff costs associated with POCTs and laboratory-based tests were assumed to be equal.

At the previous Diagnostic Advisory Committee meeting, there was considerable discussion of the relative costs of POCTs and laboratory-based tests for SARS-CoV-2. Some members contended that the costs used were relatively favourable to POCTs, whereas others argued the opposite. In the light of these discussions, and acknowledging the fact that the costs are uncertain, the costs were assumed equal in the base case, and sensitivity analyses were performed to explore the impact of changes in the incremental cost-effectiveness ratios (ICERs). To facilitate a comparison with a strategy of no routine testing (for academic interest), it was assumed that SARS-CoV-2 tests cost £22. All costs used within the analyses are assumed to be in Great British pounds in 2020.

An exception was for the sensitivity analyses using lateral flow methods. In the exploratory analyses, it was assumed that these would be £10 cheaper than the other tests, averaging £12 per test.

Quality-adjusted life-years lost because of COVID-19

The model estimates the additional discounted QALYs lost (assuming a discount rate of 3.5% per annum, as recommended by NICE6) and life-years lost due to COVID-19. These values have been estimated using data from Office for National Statistics life tables41 (2016–18) and age- and sex-adjusted general population utility values reported from Ara and Brazier,42 which used estimated utility values from 26,679 people (aged 16–98 years) who completed EuroQol-5 Dimensions, three-level version (EQ-5D-3L), questionnaires. 42 The EAG acknowledges that these QALY losses will be overestimated if the average utility for residents in a care home is lower than the average of people in England, and these are conditional on the age and sex of people in England.

Example values of the estimated QALY losses and life-years lost for selected ages are provided in Table 9.

For residents who had severe COVID-19, a multiplier was applied to take into consideration the fact that the QALYs gained would be lower than that of the average age- and sex-matched people because of reduced life expectancy and/or reduced quality of life. As no data were found to populate these multipliers, an arbitrary value of 0.9 was applied for residents who had severe COVID-19. This is higher than the value (0.8) used in the earlier work undertaken by the EAG in a hospital setting,20 as the latter value was applied to only those people who had received care in an intensive care unit.

With the previous work undertaken by the EAG, the QALYs lost could be strongly influenced by Monte Carlo sampling error (MCSE) (i.e. the differences in results purely because of the random numbers used) and so an evaluation was undertaken with the full runs completed at the time of the analysis. The results were influenced less by MCSE, but the EAG still thought it prudent to generate a statistical model based on the data so that the effects of MCSE were reduced. The assumed statistical relationship for QALYs lost from death due to COVID-19 is provided in Chapter 5, Estimating the discounted quality-adjusted life-years lost from death due to COVID-19, with the assumed statistical relationship for the expected QALYs that would be gained for those residents with severe, non-fatal COVID-19, if there were no longer-term consequences, provided in Chapter 5, Estimating the discounted quality-adjusted life-years gained after survival from severe, non-fatal COVID-19, assuming no long-term consequences.

Quality-adjusted life-year losses due to isolation within the residential care facility

It was considered that time in isolation could have an important impact on residents of a residential care facility, particularly elderly residents. This was explored by estimating a utility decrement associated with increased depression or anxiety. It was assumed that all residents of the residential care facility had some problems in the mobility, self-care, usual activities and pain dimensions of the EQ-5D-3L (which were represented by a score of 2) and that half of the residents had a score of 2 for the anxiety dimension, with the remaining half having a score of 1. The decrease in utility when increasing the score on the anxiety dimension was then explored. This resulted in a decrease in utility (using the UK tariff) of 0.434 (0.516 – 0.082) for those who started with a score of 2 for anxiety and a decrease in utility of 0.071 (0.587 – 0.516) for those who started with a score of 1 for anxiety. 43 Having considered these values, the EAG deemed it reasonable that a resident’s utility was reduced by 0.25 when in isolation. A sensitivity analysis was undertaken that assumed that there was no disutility associated with isolation.

The EAG acknowledges that isolation could affect utility beyond the isolation period, which means that the QALYs lost because of isolation may be underestimated. However, the extent of any increased anxiety about multiple lockdowns over and above the first lockdown is unknown.

Estimation of the proportion of bubbles penetrated by SARS-CoV-2 infection in the near future

Initial results from the Vivaldi study,13 reported by the Office for National Statistics,44 show that an estimated 56% of care homes reported at least one confirmed case of COVID-19. However, this value may not be generalisable to our decision problem, as some care homes are likely to have more than one bubble, meaning that the reported value will be an overestimate of the penetration for the average bubble. Additionally, it is unclear what effect changes in policy relating to a multitude of aspects, such as allowing visitors to the residential care facility, would have on the anticipated proportion of bubbles that would be penetrated. Given the large uncertainty, the EAG has decided that, in the base case, it should be assumed that 50% of bubbles will be penetrated, although this will be changed in sensitivity analyses.

The strategies modelled

The strategies modelled were replicated for four broad sets of analyses. The first division was based on the type of residential care facility modelled (i.e. en suite vs. shared rooms and facilities). The second division was based on whether or not the bubble would be penetrated by SARS-CoV-2 infection (i.e. a seeded model vs. a model without SARS-CoV-2 infection). It is acknowledged that whether or not a residential care facility is penetrated cannot be known in advance; however, modelling in this manner makes it easier to change the percentage of bubbles estimated to be penetrated in future sensitivity analyses.

For each of the four combinations (en suite vs. shared, and penetrated vs. not), the strategies shown in Table 10 were run. The following elements were common to all strategies: isolating all residents on the identification of clinical symptoms suggestive of COVID-19 and isolating all residents on receiving a positive SARS-CoV-2 test. For all strategies except strategy 1 (the calibration strategy), it was assumed that there was routine testing, that routine testing would be undertaken in residents in isolation and that all residents would be tested when lockdown was initiated. If this option were selected, isolation could be extended if an additional suspected COVID-19 case or positive SARS-CoV-2 test result came back when the bubble was not in lockdown.

Calibration of the model

It is acknowledged that many parameter estimates are uncertain or have no data (such as effective contact and attack rates). To attempt to provide meaningful results, the model was calibrated to obtain results consistent with observed data by applying a calibration factor to the product of effective contacts and attack rates.

Data from the Vivaldi report13 showed that, among the care homes that reported at least one case of COVID-19, an estimated 19.9% (95% confidence interval 18.5% to 21.3%) of residents tested positive for SARS-CoV-2, with this value being 6.9% (95% confidence interval 5.9% to 7.9%) for staff members. However, the calibration target of the proportion of people with SARS-CoV-2 infection within strategy 1 (i.e. the calibration strategy) was chosen to be 40% for residents and 14% for staff. This value is higher than that reported in the Vivaldi report,13 as the EAG acknowledges that many residential care facilities may have more than one bubble and, if only one bubble had been penetrated, then the values reported in the Vivaldi report13 would underestimate the true number of infections in a single bubble. The calibration factors were 25 for en suite rooms and 20 for shared facilities, which resulted in 38% of residents and 14% of staff being infected in the en suite scenario, and 40% of residents and 13% of staff being infected in the shared-facilities scenario. As it was plausible that there would be more infections of residents in shared facilities, this was deemed acceptable.

The number of different random number streams used per analysis

As with all individual patient-based models, there is the possibility of ‘first-order’ uncertainty, which is the phenomenon that identical residents do not get identical outcomes because of the random numbers drawn to simulate whether or not a resident has an event of interest. The results for a given scenario will therefore change depending on the random number stream seed selected, as will the comparative results with alternative strategies/scenarios, even when the same random number seed is used, as the allocation of random numbers will differ between the strategies/scenarios. To reduce the impact of random number streams, sufficient simulations, using different random number seeds, should be undertaken.

Analyses were undertaken to assess the variability in the mean results as the number of simulations was increased using a near-final version of the calibration model. Appendix 1, Figure 24, depicts the average number of residents infected in the en suite residential care facility as the number of trials increases. This analysis is replicated for the time spent in isolation (see Appendix 1, Figure 25). It is believed that both outcomes have stabilised after 1000 random number streams, with no material change in the number of residents infected between 800 and 1000 random number streams. This conclusion was also reached for the shared facilities. There is a trade-off between decreased model computation time and improved accuracy as more simulations with different random number streams are undertaken. In this instance, we believed that 1000 random number streams were sufficient to provide accurate estimates of the number of infections and days in isolation.

Note that the number of cases identified in the care home, as opposed to residents actually infected, is a function of both the number of residents who become symptomatic and the diagnostic accuracy of the SARS-CoV-2 test used.

The use of net monetary benefit

Incremental cost-effectiveness ratios, being ratios of incremental costs divided by incremental QALYs, can be misleading when incremental QALY gains are small, or when the estimated values are very close between two interventions and one is stated to be dominating the other. Incremental net monetary benefit (NMB), defined as the cost per QALY threshold multiplied by the incremental QALY gain minus the incremental cost,45 removes these limitations. For brevity, incremental NMB will be shortened to NMB throughout this report. Under this framework, the largest NMB is associated with the most cost-effective strategy at the stated cost-per-QALY threshold and multiple strategies can be compared simultaneously, as the absolute difference in strategies in terms of NMB can be easily determined. These analyses have been undertaken at assumed cost thresholds of £20,000, £30,000 and £50,000, which are thresholds used in NICE technology appraisals,6 with the highest value used for treatments meeting the end-of-life criteria. ICERs are presented for strategies that are believed to be comparable.

Potential wider societal benefits

In addition to the cost per QALY results required by NICE,6 the EAG is mindful of potential wider societal benefits that may be associated with the decision problem. A number of aspects have not been formally included in the cost and/or QALY calculations. These include the impacts of staff members not being available to work because of symptoms suggestive of COVID-19 and subsequent isolation; the impacts on the family of staff members who have symptoms suggestive of COVID-19 and subsequent isolation; the potential for further spread of SARS-CoV-2 from infected staff members; the increased burden on general practitioners; the increased burden on hospitals from staff members or those in a chain of infection initiated by the staff member who was infected with SARS-CoV-2 and the potential suspension of visitors to the residential care facility.

Some of these costs (i.e. general practitioners and hospital) will fall directly on the NHS, although the EAG notes that the government will bear costs associated with staff members who need to self-isolate through the government’s Adult Social Care Infection Control Fund,8 which can top up statutory sick pay. Depending on the staff member required to self-isolate and their working pattern, this cost is likely to be between £500 and £900 for a 10-day isolation period, based on pay estimates of £10 per hour for care home workers and £16.50 per hour for a registered nurse.

The estimated number of infections per strategy

Figure 2 provides the modelled number of SARS-CoV-2 infections for each of the 13 strategies listed in Table 10 for the seeded en suite model. Figure 3 provides similar information for the seeded shared facilities model. The number of SARS-CoV-2 infections for non-seeded models is, by definition, zero.

FIGURE 2.

The estimated number of infections per strategy for an en suite residential care facility.

FIGURE 3.

The estimated number of infections per strategy for a shared facilities residential care facility.

Strategy 1, which was believed to have been used in many care home facilities in the early response to SARS-CoV-2 infection, has the largest number of infections. This was because there was no routine testing and those with pre-symptomatic or asymptomatic SARS-CoV-2 infection would not have been detected, allowing the spread of SARS-CoV-2 infection. Given the assumption of the model that lockdown is initiated at clinical symptoms or a positive SARS-CoV-2 test result, the numbers of SARS-CoV-2 infections for strategies that do not allow early release from isolation are similar. The strategies that allow an early release have more infections than the counterpart strategies without early release.

The results are not always intuitive; for example, at face value, one may expect that the desirable TPP for a laboratory-based SARS-CoV-2 test would be to produce fewer SARS-CoV-2 infections than the acceptable TPP, as it performs better on sensitivity, specificity and failure rates (see Table 8). However, the decreased specificity for the acceptable TPP for the laboratory-based SARS-CoV-2 test means that the residential care facility would enter lockdown more often than when the desirable TPP was used. More frequent lockdowns decrease the spread of SARS-CoV-2 when the infected resident or staff member is asymptomatic, resulting in fewer infections, but at the expense of days in isolation, which may have a negative impact on residents.

The estimated days spent in isolation

The model records the days spent in isolation by the entire cohort. For the purposes of presentation, we have assumed that there were no deaths within the bubble and have presented totals per resident by dividing the cohort total by 16. These values per strategy for the seeded en suite model are provided in Figure 4. Corresponding results for the seeded shared facility model, the non-seeded en suite model and the non-seeded shared facility model are provided in Figures 5–7, respectively.

FIGURE 4.

The estimated number of days in isolation per resident by strategy in the seeded en suite model.

FIGURE 5.

The estimated number of days in isolation per resident by strategy in the seeded shared facility model.

FIGURE 6.

The estimated number of days in isolation per resident by strategy in the non-seeded en suite model.

FIGURE 7.

The estimated number of days in isolation per resident by strategy in the non-seeded shared facility model.

The estimated number of SARS-CoV-2 tests performed

The number of tests performed in the seeded models were fairly similar, with the exceptions of the calibration strategy, for which the number was smaller, as there was no routine testing and all residents and staff were not tested following a suspected case or a positive SARS-CoV-2 test, and the strategies that had a poorer specificity, for which the numbers were larger, as there was a larger number of false-positive SARS-CoV-2 tests. Table 11 provides, to the nearest integer, the number of tests performed for strategies 1–13 for en suite or shared facilities, and when seeded or not.

Estimating the discounted quality-adjusted life-years lost from death due to COVID-19

The equation for QALYs lost from death due to COVID-19 is seen in Figure 8. The statistical fit was very good, with an R² of 0.986. The statistical model shown was forced to go through the origin to maintain the logic that zero infections would be associated with no QALY losses.

FIGURE 8.

The relationship between the number of SARS-CoV-2 infections and the QALYs lost from death due to COVID-19.

Estimating the discounted quality-adjusted life-years gained after survival from severe, non-fatal COVID-19, assuming no long-term consequences

The assumed statistical relationship of discounted QALYs gained after survival from severe, non-fatal COVID-19, assuming no long-term consequences, is shown in Figure 9. As with the model for QALY losses due to COVID-19 deaths, the statistical model was forced to go through the origin. Again, there was a very good statistical fit, with an R² of 0.981.

FIGURE 9.

The relationship between the number of SARS-CoV-2 infections and the expected QALYs gained after severe, non-fatal COVID-19, assuming long-term consequences.

Estimating the discounted quality-adjusted life-years lost for each strategy and the costs for each strategy for all models

The QALY loss and the cost of testing for each strategy in the seeded en suite model are shown in Table 12. Tables 13–15 provide similar information for the seeded shared facility model, the non-seeded en suite model and the non-seeded shared facility model, respectively. The non-seeded models will not have any QALYs losses due to COVID-19.

Estimating the implications of a bubble being penetrated

When a bubble is penetrated, there will be an increase in the number of SARS-CoV-2 infections and in the time residents spend in isolation. The increase in the numbers of infections is provided in Figures 2 and 3. By definition, there are no infections in the non-seeded group. However, isolation occurs in both groups. Figure 10 provides the increase in isolation days that occurs when a bubble is penetrated by SARS-CoV-2, which is the difference between Figures 4 and 6 for the en suite model and between Figures 5 and 7 for the shared facilities model. It can be seen that the strategies that use the acceptable TPPs for one of laboratory-based SARS-CoV-2 tests or SARS-CoV-2 POCTs only have a smaller increase in isolation time, as more residents were isolated because of the larger number of false-positive tests.

FIGURE 10.

The number of additional isolation days per resident for penetrated bubbles vs. infection-free bubbles.

Estimating the cost-effectiveness of the strategies for given combinations of residential care facility type and whether or not the bubble is penetrated

For the seeded en suite model, the NMB values associated with each strategy are presented in Figure 11. Corresponding results for the seeded shared facility model, the non-seeded en suite model and the non-seeded shared facility model are provided in Figures 12–14, respectively.

FIGURE 11.

The NMB associated with each strategy at stated cost-per-QALY thresholds for the seeded en suite model.

FIGURE 12.

The NMB associated with each strategy at stated cost-per-QALY thresholds for the seeded shared facility model.

FIGURE 13.

The NMB associated with each strategy at stated cost-per-QALY thresholds for the non-seeded en suite model.

FIGURE 14.

The NMB associated with each strategy at stated cost-per-QALY thresholds for the non-seeded shared facility model.

As expected, the NMBs for all strategies in the seeded model are positive compared with strategy 1, indicating that the strategies appear to result in ICERs below the thresholds stated when the bubble is penetrated. Conversely, all NMBs in the non-seeded models are less than zero, indicating that, when a bubble is not penetrated, the strategies do not result in ICERs lower than the thresholds stated when the bubble remains free of SARS-CoV-2 infection. However, it is not known which bubbles will be penetrated and which will not. The weighted average NMB for estimations of the proportion of bubbles penetrated is provided in The weighted net monetary benefit for each strategy.

Results having assumed that the proportion of bubbles penetrated is 50%

The following subsections provide the estimated number of infections, the estimated days in isolation per resident and the expected NMB for each strategy having assumed that 50% of bubbles are penetrated. The results are provided for en suite residential care facilities and those with shared facilities.

The weighted number of expected infections per strategy

The weighted number of infections in the en suite residential care facility model per strategy is shown in Figure 15. Figure 16 provides the same information for residential care facilities with shared facilities.

FIGURE 15.

The weighted number of infections per strategy, assuming that 50% of bubbles are penetrated in the en suite model.

FIGURE 16.

The weighted number of infections per strategy, assuming that 50% of bubbles are penetrated in the shared facility model.

The weighted number of estimated days in isolation per resident

The weighted number of days in isolation for residents in the en suite residential care facility model per strategy is shown in Figure 17. Figure 18 provides the same information for residential care facilities with shared facilities.

FIGURE 17.

The weighted number of days in isolation per resident per strategy, assuming that 50% of bubbles are penetrated in the en suite model.

FIGURE 18.

The weighted number of days in isolation per resident per strategy, assuming that 50% of bubbles are penetrated in the shared facility model.

The weighted net monetary benefit for each strategy

The weighted NMBs, assuming that 50% of bubbles are penetrated, in the en suite model per strategy are shown in Figure 19. Figure 20 provides the same information for residential care facilities with shared facilities.

FIGURE 19.

The weighted NMB per strategy in the base-case en suite model.

FIGURE 20.

The weighted NMB per strategy in the shared facility model.

For en suite residential care facilities, all strategies have a positive NMB compared with strategy 1 and the ranking of the strategies remains largely constant, independent of the cost per QALY threshold value chosen. Strategy 3 (i.e. a SARS-CoV-2 POCT with a desirable TPP) has the highest expected NMB.

For residential care facilities with shared facilities, the NMBs are lower and some are negative, indicating that some strategies (i.e. strategies 6 and 7) have ICERs greater than the £20,000 and £30,000 thresholds, compared with strategy 1. These two strategies are when the laboratory-based SARS-CoV-2 tests and the SARS-CoV-2 POCTs are at the acceptable TPP, questioning whether or not the acceptable TPP requirement needs to be tightened or if this specific-use case should be excluded. The reason for this result is the large NMB loss in the non-seeded scenario (see Figure 14), which is caused by the additional days in isolation associated with the lower specificity of the acceptable TPPs. Strategy 5 (i.e. SARS-CoV-2 POCT with desirable a TPP and early release) has the highest expected NMB.

Sensitivity analyses undertaken

There is considerable uncertainty relating to the parameters in the model. This has been explored in sensitivity analyses, although these analyses were constrained by the deadlines for the report. The change in the difference in key outcomes between the strategies may be informative to the NICE Diagnostics Advisory Committee. Owing to time constraints, only selected strategies, types of residential care facility and seeding options, were run. As there appeared to be no difference in the conclusions between pairs of tests in the en suite or the shared facilities model (see Tables 16 and 17), the sensitivity analyses were largely undertaken in the en suite version only.

Ten sets of sensitivity analyses were undertaken to explore how the results of the model changed when key assumptions related to key elements were altered. Five sets of sensitivity analyses required the simulation model to be re-run. For the other five sets, the analyses could be undertaken with the results produced when deriving the base-case results. The results for sensitivity analyses that did not materially affect the conclusions are presented in Appendix 1.

The five sensitivity analyses that required new runs of the simulation model were (1) changing the age of residents within the residential care facility, (2) changing the underlying rates of non-COVID-19 diseases that had symptoms suggestive of COVID-19, (3) assuming that a proportion of residents within the residential care facility had immunity to SARS-CoV-2, potentially because of vaccination, (4) changing the seed to a member of staff allocated to dealing with non-isolated residents and (5) assuming that a lower proportion of people become symptomatic following SARS-CoV-2 infection.

The following five sensitivity analyses could be applied after the model has been run and therefore results are provided for all of these strategies: (1) altering the proportion of bubbles that are assumed to be penetrated, (2) changing the relative costs of laboratory-based SARS-CoV-2 tests and the costs of SARS-CoV-2 POCTs, (3) assuming that there is no loss of QALYs for those residents who do not die from COVID-19, (4) increasing the severity of consequences after severe COVID-19 infection and (5) varying the QALY multiplier for residents who have had severe non-fatal COVID-19.

Exploring the effects of resident immunity from SARS-CoV-2 from vaccination

The model has the functionality to allow a proportion of residents and staff (simulated at an individual person level) to be immune from SARS-CoV-2 infection. This will reduce the number of simulated infections in two ways: (1) directly (i.e. owing to the immune person not becoming infected with SARS-CoV-2) and (2) indirectly (i.e. if the person had been infected, they could have infected residents and staff within the care home).

Analyses have been undertaken for both residential care facilities with en suites and for residential care facilities with shared facilities; however, because of time constraints and because SARS-CoV-2 tests with acceptable TPPs performed poorly, only the first five strategies and strategies 10–13 have been re-run. For each strategy, three levels of protection, in terms of the chance of becoming immune, from receiving a vaccine have been explored: 50%, 70% and 90%. The method used may represent a simplification, as immunity may not be as clear as fully immune or fully susceptible, and people may be still susceptible after a vaccine; however, the impacts of COVID-19 are less severe. Additionally, it has been assumed that the vaccine has reached full efficacy at the time of seeding the model with SARS-CoV-2 infection. Despite the limitation in the results, it is believed that these analyses provide useful information to the NICE Diagnostics Advisory Committee.

A further set of analyses has been conducted where it was assumed that a vaccine would not stop a person from being infected with SARS-CoV-2, but that, for those who are symptomatic, it would reduce the probability of severe outcomes by 50%, 70% or 90%. All results presented are weighted assuming that 50% of bubbles are penetrated.

The analyses undertaken have shown that, if an effective vaccine is available for residential care facility residents and staff, strategies with early release will become more favourable. Even at high immunity levels (90%), it is still estimated that desirable TPPs and real-world evidence strategies for both laboratory-based SARS-CoV-2 tests and SARS-CoV-2 POCTs can have positive NMB values at willingness-to-pay levels of £30,000 per QALY and greater for the en suite residential care facility, and at willingness-to-pay levels of £50,000 per QALY for the residential care facility with shared facilities. Therefore, vaccination does not necessarily make SARS-CoV-2 POCTs redundant.

En suite residential care facilities

The number of resident infections in each vaccination scenario is provided in Figure 21. As anticipated, the more immunity conferred by the vaccination, the smaller the number of infections.

FIGURE 21.

The weighted number of resident infections in each vaccination scenario for en suite residential care facilities when the vaccine can confer immunity.

The weighted NMB in each vaccination scenario (50%, 70% and 90% immunity) is provided in Appendix 1, Figures 28–30. The rankings of NMB remain similar; however, the NMB is reduced and as immunity increases the number of strategies with a negative NMB compared with strategy 1 increases.

It is seen that, as the efficacy of the vaccination increases, the NMB of the strategies, compared with strategy 1, decreases. In addition, there is a clear NMB advantage for strategies that allow early release from isolation (i.e. strategies 4, 5, 12 and 13). When efficacy is assumed to be 70%, only those strategies with early release had a positive NMB, compared with strategy 1. However, when it is assumed that efficacy was 90%, the willingness-to-pay threshold needs to be £50,000 per QALY or higher for the NMB of early release strategies to be positive. However, these results are dependent on the assumed 50% of bubbles being penetrated, with the NMB of strategies compared with strategy 1 improving as the proportion of bubbles penetrated increases, and vice versa.

The weighted NMB, which assumes that the vaccine reduces the proportion of COVID-19 cases that are critical but does not affect transmission of SARS-CoV-2 for each vaccination scenario, is provided in Appendix 1, Figures 31–33. It is observed that, as the proportion of COVID-19 cases that are critical decreases, strategies that favour less time in isolation (i.e. strategies 4, 5, 8, 9, 12 and 13) become more favourable, as QALY losses due to isolation begin to outweigh the QALY losses associated with COVID-19. However, these results are dependent on the assumed 50% of bubbles being penetrated, with the NMB of strategies compared with strategy 1 improving as the proportion of bubbles penetrated increases, and vice versa.

Residential care facilities with shared facilities

The number of resident infections in each vaccination scenario is provided in Appendix 1, Figure 34. As anticipated, the more immunity conferred by the vaccination, the smaller the number of infections.

The NMB in each vaccination scenario relating to immunity is provided in Appendix 1, Figures 35–37. The interpretation of the results for the residential care facility with shared facilities is similar to that of the en suite residential care facility, although there is a smaller NMB in that residents in shared facilities are less likely to be released early than those in en suite residential care facilities, as release is dependent on all members of the shared room being non-index cases and returning a negative SARS-CoV-2 test. In addition, while under lockdown, SARS-CoV-2 infection can still be passed among residents in shared rooms.

The NMBs in each vaccination scenario are provided in Appendix 1, Figures 38–40.

The interpretation of the results for the residential care facilities with shared facilities is similar to that of the en suite residential care facility, although the NMB is lower in shared facilities because of the increased time in isolation and there are smaller numbers of infections in the en suite model, as infections are reduced in lockdown because of single-occupancy rooms.

Exploring the use of lateral flow tests

In response to comments from peer reviewers and the emergence of new data on lateral flow tests (see Chapter 3, Characteristics of the point-of-care tests and laboratory-based tests), exploratory analyses were undertaken that assumed lateral flow tests were used every 5 days for both residents and staff members. The results were compared with the desirable POCT assuming that there is no early release from isolation (i.e. strategy 3) or early release from isolation (i.e. strategy 5). The NMB results for the en suite residential care facility are shown in Figure 22, with the results in the residential care facility with shared facilities shown in Figure 23. Summary results are shown in Table 18.

FIGURE 22.

Exploratory analyses assessing the NMB of lateral flow tests in en suite residential care facilities.

FIGURE 23.

Exploratory analyses assessing the NMB of lateral flow tests in residential care facilities with shared facilities.

TABLE 18 - Summary results from the exploratory lateral flow modelling

Residential care facility	Number of resident infections	Days in isolation per resident	Costs of testing (£)	QALYs lost
En suite
POCT desirable	2.63	31.7	5083	1.08
POCT desirable – early release	3.94	8.7	5260	1.16
Lateral flow: PHE report31	0.87	23.8	6098	0.51
Lateral flow: PHE report31 – early release	1.51	5.2	6122	0.49
Lateral flow: Cochrane review32	1.39	24.7	6115	0.67
Lateral flow: Cochrane review32 – early release	2.83	6.3	6148	0.85
Shared facilities
POCT desirable	4.91	34.5	5223	1.68
POCT desirable – early release	4.93	15.5	5320	1.48
Lateral flow: PHE report31	2.97	30.3	6182	1.15
Lateral flow: PHE report31 – early release	2.56	12.2	6171	0.85
Lateral flow: Cochrane review32	3.75	31.0	6203	1.36
Lateral flow: Cochrane review32 – early release	3.69	13.4	6191	1.15

Contributions of authors

Matt Stevenson (https://orcid.org/0000-0002-3099-9877) is a Professor of Health Technology Assessment at ScHARR, University of Sheffield. He led the project, constructed the mathematical model and interpreted the results.

Andrew Metry (https://orcid.org/0000-0001-7412-6093) is a Research Associate at ScHARR, University of Sheffield. He reviewed the evidence to populate the model, critiqued the model and helped interpret the results.

Michael Messenger (https://orcid.org/0000-0002-4975-0158) is Head of Personalised Medicine and Health, University of Leeds; and Deputy Director of NIHR Leeds Medtech and In Vitro Diagnostic Co-Operative. He reviewed the results, suggested additional strategies to run and provided guidance throughout the work.

All authors commented on the final monograph.

Publications

For related publications and further information, see the NIHR funding and awards web page for the overall project: https://fundingawards.nihr.ac.uk/award/NIHR133409.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care.

Estimating the number of random number streams at which stable results are produced

FIGURE 24.

Assessing how many simulations are required to stabilise the number of residents infected with SARS-CoV-2 in strategy 1.

FIGURE 25.

Assessing how many simulations are required to stabilise the number of days spent in isolation by the cohort in strategy 1.

Exploring the effects of younger residents within the residential care facility

FIGURE 26.

The NMBs for selected strategies in the seeded en suite model when younger residents were assumed.

Exploring the effects of a higher prevalence of clinical symptoms suggestive of COVID-19 that are not caused by SARS-CoV-2

FIGURE 27.

The NMBs for selected strategies in the en suite model when a higher rate of diseases that are not COVID-19, but produce symptoms suggestive of COVID-19, was used

Exploring the effects of different vaccination protection assumptions related to immunity: en suite facilities

FIGURE 28.

The weighted NMB assuming that the chance of immunity with the vaccine is 50% for en suite residential care facilities.

FIGURE 29.

The weighted NMB assuming that the chance of immunity with the vaccine is 70% for en suite residential care facilities.

FIGURE 30.

The weighted NMB assuming that the chance of immunity with the vaccine is 90% for en suite residential care facilities.

Exploring the effects of different vaccination protection assumptions related to the probability of COVID-19 becoming critical: en suite facilities

FIGURE 31.

The weighted NMB assuming that the chance of critical COVID-19 with the vaccine is reduced by 50% for en suite residential care facilities.

FIGURE 32.

The weighted NMB assuming that the chance of critical COVID-19 with the vaccine is reduced by 70% for en suite residential care facilities.

FIGURE 33.

The weighted NMB assuming that the chance of critical COVID-19 with the vaccine is reduced by 90% for en suite residential care facilities.

Exploring the effects of different vaccination protection assumptions related to immunity: shared facilities

FIGURE 34.

The weighted number of resident infections in each vaccination scenario for residential care facilities with shared facilities.

FIGURE 35.

The weighted NMB assuming that the chance of immunity with the vaccine is 50% for residential care facilities with shared facilities.

FIGURE 36.

The weighted NMB assuming that the chance of immunity with the vaccine is 70% for residential care facilities with shared facilities.

FIGURE 37.

The weighted NMB assuming that the chance of immunity with the vaccine is 90% for residential care facilities with shared facilities.

Exploring the effects of different vaccination protection assumptions related to the probability of COVID-19 becoming critical: shared facilities

FIGURE 38.

The weighted NMB assuming that the chance of critical COVID-19 with the vaccine is reduced by 50% for residential care facilities with shared facilities.

FIGURE 39.

The weighted NMB assuming that the chance of critical COVID-19 with the vaccine is reduced by 70% for residential care facilities with shared facilities.

FIGURE 40.

The weighted NMB assuming that the chance of critical COVID-19 with the vaccine is reduced by 90% for residential care facilities with shared facilities.

Exploring the effects of having the seed as a staff member rather than as a resident

FIGURE 41.

The NMBs for selected strategies in the seeded en suite model when it was assumed that the seed was a staff member rather than a resident.

Exploring the effects of assuming that a lower proportion of residents with SARS-CoV-2 develop symptoms: en suite facilities

FIGURE 42.

The NMBs for selected strategies in the seeded en suite model when it was assumed that 33% of people with SARS-CoV-2 infection became symptomatic.

Exploring the effects of assuming that a higher proportion of residential care facilities were penetrated by SARS-CoV-2

FIGURE 43.

The weighted NMBs for the en suite residential care facility when it was assumed that 80% of bubbles were penetrated by SARS-CoV-2.

FIGURE 44.

The weighted NMBs for the residential care facility with shared facilities when it was assumed that 80% of bubbles were penetrated by SARS-CoV-2.

Exploring the effects of assuming that a lower proportion of residential care facilities were penetrated by SARS-CoV-2

FIGURE 45.

The weighted NMBs for the en suite residential care facility when it was assumed that 20% of bubbles were penetrated by SARS-CoV-2.

FIGURE 46.

The weighted NMBs for the residential care facility with shared facilities when it was assumed that 20% of bubbles were penetrated by SARS-CoV-2.

Exploring the effects of assuming that SARS-CoV-2 point-of-care tests cost more

FIGURE 47.

The weighted NMBs for the en suite residential care facility assuming that SARS-CoV-2 POCTs cost £10 more than in the base case.

FIGURE 48.

The weighted NMBs for the residential care facility with shared facilities assuming that SARS-CoV-2 POCTs cost £10 more than in the base case.

Exploring the effects of assuming that laboratory-based SARS-CoV-2 tests cost more

FIGURE 49.

The weighted NMBs for the en suite residential care facility assuming that laboratory-based SARS-CoV-2 tests cost £10 more than in the base case.

FIGURE 50.

The weighted NMBs for the residential care facility with shared facilities assuming that laboratory-based SARS-CoV-2 tests cost £10 more than in the base case.

Estimating the cost of SARS-CoV-2 point-of-care tests that would produce the same net monetary benefit as laboratory-based tests

Exploring the effects of assuming that there are no quality-adjusted life-year losses from isolation or non-fatal, severe COVID-19

FIGURE 51.

The weighted NMBs for the en suite residential care facility assuming that there are no QALY losses from isolation or non-fatal, severe COVID-19.

FIGURE 52.

The weighted NMBs for the residential care facility with shared facilities assuming that there are no QALY losses from isolation or non-fatal, severe COVID-19.

Exploring the effects of assuming that the quality-adjusted life-year losses associated with COVID-19 are doubled

FIGURE 53.

The weighted NMBs for the en suite residential care facility assuming that QALY losses from COVID-19 are doubled.

FIGURE 54.

The weighted NMBs for the residential care facility with shared facilities assuming that QALY losses from COVID-19 are doubled.

Exploring the effects of assuming that there are no quality-adjusted life-year losses associated with severe, non-fatal COVID-19

FIGURE 55.

The weighted NMBs for the en suite residential care facility assuming that there were no QALY losses from non-fatal, severe COVID-19.

Exploring the effects of assuming that the multiplier for a quality-adjusted life-year gain following severe, non-fatal COVID-19 was 0.8 of the age-matched population

FIGURE 56.

The weighted NMBs for the en suite residential care facility assuming that there were no QALY losses from non-fatal, severe COVID-19.

Exploring the effects of halving effective contacts and attack rate in the product

FIGURE 57.

The NMBs for selected strategies in the seeded en suite model when the effective contacts and attack rate is halved.

Exploring the effects of reducing the number of days in isolation from 14 to 10

FIGURE 58.

The NMBs for selected strategies in the seeded en suite model when the time spent in isolation was reduced from 14 days to 10 days.