Randomised controlled trial and health economic evaluation of the impact of diagnostic testing for influenza, respiratory syncytial virus and Streptococcus pneumoniae infection on the management of acute admissions in the elderly and high-risk 18- to 64-year olds

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 03/39/18. The contractual start date was in November 2005. The draft report began editorial review in January 2012 and was accepted for publication in April 2013. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Karl G Nicholson has been an ad hoc consultant to GlaxoSmithKline and Novartis. He has received funding to speak at meetings organised by Novartis, Baxter, Berna Biotech, Esteves, and the European Scientific Working Group on Influenza, and H5 vaccines from Novartis to support an MRC-funded research project, and H1N1 vaccines from Baxter AG and GlaxoSmithKline to support an NIHR-funded research project. A colleague in Karl G Nicholson’s Department has received research funding from Roche. Maria Zambon has been an investigator of clinical trials sponsored by Novartis, Baxter, Sanofi Pasteur and CSL Australia Ltd. Keith R Abrams has acted as a paid consultant to the health-care industry generally (for the provision of advice and short courses), but specifically has not advised any organisation as regards diagnostics. Tristan W Clark has been an investigator of clinical trials sponsored by Novartis and Roche.

Copyright statement

© Queen’s Printer and Controller of HMSO 2014. This work was produced by Nicholson et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Elderly demographics

Western industrialised nations face a large increase in the number of older people. In 2006 there were about 200,000 more children of < 16 years of age than people at state pension age in the UK. However, in 2007, for the first time ever, the population at state pension age exceeded the number of children. And, despite increases to state pension age, the population at state pension age is projected to exceed the number of children of < 16 years by 400,000 in 2016, and by over 2 million in 2031. 1 Annual figures published by the NHS Information Centre reveal that people aged > 60 years accounted for almost half of all of the 16.8 million hospital admissions (finished consultant episodes) in 2009–10. 2,3 During 2009–10, infections of the respiratory tract accounted for about 1 in 30 of all hospital admissions in England and 1 in 20 of the 51.5 million bed-days. 4 Preparation for this population growth, including the prevention and care of illness, is of paramount importance.

Acute respiratory illness

Illness surveys conducted in general practice indicate an overwhelming importance of acute respiratory illness in comparison with other conditions. During the most recent (Fourth) National Morbidity Study, a higher proportion of people (31%) consulted for respiratory conditions at least once during the year than for diseases in any other single International Classification of Diseases (ICD) chapter. 5 Overall, 67% of patients who saw their general medical practitioner (GP) with a respiratory condition did so because of an acute infection, i.e. ≈ 20% of all consultations in primary care occur because of acute respiratory infections (ARIs), which are mostly viral. The rates of acute respiratory illness were highest among small children. They were lowest among subjects aged 45–64 years and then increased with age, and the percentage that was graded as ‘serious’ reached ≈ 25% in those aged > 65 years.

Owing to the increasing severity of acute respiratory illness in older people, the number of hospital admissions in England for influenza, pneumonia and other acute lower respiratory infections (ICD-10 codes J10-J18, J20-J22) is approximately three times higher for people aged ≥ 75 years than in younger people ( Figure 1 ). 4 Annual figures published by the NHS Information Centre reveal that among those aged > 75 years the number of admissions for influenza and pneumonia and other lower respiratory tract infections has doubled to almost 200,000 in England over the last 10 years (see Figure 1 ). 4

FIGURE 1.

Annual number of admissions by age to NHS hospitals in England for ‘Influenza and Pneumonia’ (ICD-10 codes J10-J18), and ‘Other acute lower respiratory infections’ (J20-J22), years 1998–99 to 2009–10. Source: www.hesonline.nhs.uk/Ease/servlet/ContentServer?siteID = 1937&categoryID = 202 (accessed 30 October 2010).

The average length of stay for acute respiratory conditions increases progressively with age. 6 Although the average length of stay for pneumonia has fallen during the last 20 years, it is 10–15% longer in those aged > 65 years than in younger adults ( Figure 2 ). 7 The annual number of pneumonia and influenza deaths (ICD-10 codes J10–J18) in England and Wales increases with increasing age and exceeds 1000 per annum in each of the 5-year age bands in those aged > 70 years ( Figure 3 ). Strategies that prevent acute lower respiratory infections, ameliorate their severity or shorten the average duration of stay will have the greatest benefit in the elderly.

FIGURE 2.

Average length of stay for ‘pneumonia’ in short-stay hospitals, by age: USA, 1990, 2000 and 2006. Source: www.cdc.gov/nchs/data/hus/hus09.pdf#102oryID = 202 (accessed 30 October 2010).

FIGURE 3.

Annual number of pneumonia and influenza deaths (ICD-10 codes J10-J18) in England and Wales, years 2000–9 (compiled from annual ONS mortality statistics).

In this study, we evaluate rapid diagnostic technologies for three target pathogens – influenza, respiratory syncytial virus (RSV) and Streptococcus pneumoniae – which are key aetiological agents of acute respiratory illness and, collectively, are responsible for considerable morbidity and mortality in the elderly.

There is a paucity of information on the relative incidence of influenza, RSV and pneumococcal disease among elderly cardiopulmonary admissions. Falsey et al. 8 evaluated the number of hospitalisations for RSV infection relative to influenza in several thousand elderly people admitted to six hospitals in New York State between November and April 1989–1992. This and other studies suggest that RSV may be found in up to 5% of patients hospitalised with acute respiratory disease,9–13 although with molecular diagnostic tests, the number identified may be higher. Previous studies8–10,12,13 indicate that about 10% of cardiopulmonary admissions have influenza but the number of admissions is influenced by the severity of epidemics, which have been generally mild since 1999/2000, including the recent 2009 H1N1 pandemic. None of these studies was conducted in the UK, and referral and admission practices may differ in the UK from those elsewhere. About one-third of all patients who are hospitalised in Northern Europe with community-acquired pneumonia (CAP) have S. pneumoniae infection. 14 S. pneumoniae is the most common microbiological cause of CAP,15,16 including the UK,17 and is the most commonly identified cause of CAP death. 16

Diagnostic tests

Rationale for the study

The three respiratory pathogens, influenza, RSV and S. pneumoniae, are responsible for considerable morbidity and mortality, and become increasingly important as pathogens with advancing age and comorbidity. The elderly population of the UK is rapidly increasing in life expectancy and size. People aged > 60 years now account for almost half of the annual number of all hospital admissions, placing huge pressure on the health-care system. RSV and influenza can exacerbate chronic cardiopulmonary disease in adults of working age, adding to the demand for hospital beds and pressure on health-care providers to discharge patients at the earliest opportunity, preferably before admission to a hospital ward.

Vaccines and drugs to prevent RSV transmission and illness are unlikely to be available for use within the next decade. Vaccines against influenza and pneumococcal infection provide incomplete protection. Many pathogens can cause CAP, and the appreciable risk of death from CAP demands that treatment is given empirically at the earliest opportunity. Early treatment with neuraminidase inhibitors (NIs) – within 48 hours of onset of illness – is considered essential.

Conventional diagnostic tests, especially viral culture, provide information too late to influence care and containment decisions. They are expensive and require specialised facilities and expertise. Blood cultures are often negative in patients with CAP caused by the pneumococcus; the result is also influenced by antecedent antimicrobial treatment. Treatment of CAP is usually empirical.

Compared with other diagnostic tests, the rapid point-of-care tests (POCTs) offer the greatest potential to influence antibiotic prescribing, ameliorate illness and prevent nosocomial transmission – but only if the tests are sufficiently sensitive and specific. PCR could provide comparable benefits if its longer turnaround time is compensated for by better test performance.

This study was designed to evaluate the ease and speed of use of the different tests, assess their costs, and identify whether they provide clinical and health-economic benefits, and rationalise the use of single-roomed accommodation.

The three diagnostic strategies assessed in this study were (1) POCTs for influenza A and B and pneumococcal infection; (2) RT-PCR tests for influenza A and B, and RSV A and B; and (3) conventional culture for these pathogens.

Study design

We undertook a prospective RCT and economic evaluation of rapid POC, molecular and conventional diagnostic tests for influenza, RSV and S. pneumoniae in the management and outcome of acute cardiopulmonary admissions in the elderly (age ≥ 65 years) and ‘high-risk’ individuals with underlying chronic heart or lung disease, including asthma, who were 18–64-years of age at the time of presentation in two teaching hospitals in the UK. A summary of the protocol for the study is provided in Appendix 1 . A copy of the full protocol is available from the Principal Investigator, Karl Nicholson.

Setting

The participating hospitals were Glenfield General Hospital and Leicester Royal Infirmary in the University Hospitals of Leicester (UHL) NHS Trust, Leicester, a city in the English East Midlands. The UHL NHS Trust serves a population of approximately one million subjects of all ages. It is the only facility within the county of Leicestershire that provides inpatient emergency medical care to the population of Leicestershire. The laboratory tests were carried out in the Department of Microbiology, Leicester Royal Infirmary.

Participants

We recruited people presenting to medical admissions units, or any ward accepting acute medical admissions, with an acute exacerbation of chronic cardiopulmonary illness of ≤ 168 hours’ (7 days’) duration or an acute cardiopulmonary illness of ≤ 7 days’ duration [including pneumonia, ‘influenza’/ILI, exacerbations of chronic obstructive pulmonary disease (COPD), bronchitis, asthma, congestive heart failure or cardiac arrhythmia], who satisfied the study inclusion and exclusion criteria and could be recruited to the study within a 16-hour period of initial assessment by the patient’s medical team.

The inclusion and exclusion criteria for participants are shown in Box 1 .

Recruitment

Medical and nursing staff on medical admissions units and wards providing acute medical care to patients with acute cardiopulmonary conditions identified eligible patients. Research nurses provided trial information and obtained signed informed consent from the patient or signed informed assent from a relative or carer.

Randomisation

Participants were then randomly allocated to one of three diagnostic study groups: (1) NPTs for pneumococcal infection and influenza; (2) rapid molecular tests for influenza and RSV; or (3) conventional laboratory diagnostic tests. Their investigations, medical care and discharge planning was provided as usual by the medical and nursing teams on the medical admissions units and other wards, not by the investigators. The randomisation process enabled the investigators to evaluate the role of the diagnostic tests on clinical outcomes. Ultimately, all diagnostic tests were performed on specimens from all subjects, providing the means to compare diagnostic accuracy.

The trial statistician generated randomisation codes, stratified by centre and using randomly permuted block sizes of 9, 12 or 15, which were not revealed to any person before randomisation. The randomisation code allocated participants in the ratio 1 : 1 : 1 to one of the three study groups. It was provided in sequentially numbered sealed study envelopes, which were stored securely (within a locked filing cabinet, within a locked office, within a locked department). The randomisation code for an individual patient became known to the research nurse only when signed informed consent or assent was obtained. The randomisation codes were then checked by the trial statistician against the master copy to ensure that the sequences concurred. It was not revealed to the participants or to the medical and nursing team providing care.

Planned interventions

Participants were randomised to receive:

• diagnostic assessment using rapid near-patient diagnostic tests (Quidel for influenza, and BinaxNOW for the pneumococcal antigen), or

• rapid molecular tests (for influenza A and B and RSV A and B), plus laboratory pneumococcal antigen testing, or

• conventional laboratory diagnostic assessment, notably culture for influenza A and B, RSV A and B, and S. pneumoniae, and serology for influenza A and B.

Although all tests were eventually carried out on all participants, clinicians were provided with rapid test results relating only to their randomisation group.

Collection of samples for microbiological analyses

Identical samples were taken from each person but were processed differently depending on the randomisation ( Box 2 ).

Participants in each group gave a venous blood sample on entry to the study for blood culture if it had not Partcipants in each group gave a venous blood sample on entry to the study for blood culture if it had not been collected already by the medical team. Blood cultures were processed in the laboratory according to local protocols. Serum from an ‘acute’ blood sample was collected from participants in each group on entry to the study and stored for titration of antibodies against influenza A and B in paired ‘acute’ and ‘convalescent’ sera. It was originally planned that the ‘convalescent’ sample would be collected 10 days after admission. Because many people were discharged within 10 days of admission, we amended the protocol to collect ‘convalescent’ sera up to 30 days, and subsequently up to 90 days, after admission. Acute and convalescent sera were processed in the Influenza Laboratory at the Health Protection Agency (HPA) Centre for Infections, Colindale, London.

On entry to the study we collected freshly expectorated sputum for Gram stain and culture from participants in each study group with a productive cough. Freshly voided urine was collected from all participants on entry to the study and processed in the BinaxNOW S. pneumoniae test, as shown in Box 2 , and according to the manufacturer’s instructions (see Appendix 2 ). One nasal swab (collected from one nostril only), one nasal swab (collected from the opposite nostril) and one throat swab were collected on entry to the study (see Appendix 3 ), and were processed as shown in Box 2 .

EuroQol quality-of-life assessment (European Quality of Life-5 Dimensions)

We assessed patients using the European quality of life (EuroQol) assessment at baseline on admission, and 7 and 28 days later. The EuroQoL European Quality of Life-5 Dimensions tool (EQ-5D)55 has been used in many cost-effectiveness studies56,57 and is recommended for use in the economic evaluation of health-care technologies within the UK in guidance issued by the National Institute for Health and Care Excellence (NICE). 58 It defines health in five dimensions: morbidity, self-care, usual activities, pain or discomfort, anxiety or depression. Each dimension has three levels: no problems, a moderate problem, or a severe problem. Health states defined by the level chosen for each dimension can be scored using utility weights reflecting the values from a representative sample of the UK population. 59 These utilities are scaled so that full health = 1 and death = 0, and they allow for severe health states for which health-related quality of life (HRQoL) is valued lower than death. The EQ-5D was self-completed by participants or was done by proxy if the participant was incapable of completing the self-administered questionnaire or providing a verbal response. Where patients were discharged, the EQ-5D was assessed by telephone interview of the patient, by postal questionnaire or by proxy.

Record-keeping

The mainstay of the record-keeping for this study was the case report form (CRF) (see Appendix 5 ). Individual CRFs were stored in locked filing cabinets in a secure University of Leicester research laboratory within Leicester Royal Infirmary.

The CRFs captured basic demographic data (including details of residential status, smoking habits, alcohol consumption, household contacts, influenza and pneumococcus immunisation status during the previous 3 years, and hospital admissions during the period 1 September to 30 April of the previous winter), the date and time of admission, the randomisation group, GP details, symptoms of the presenting illness, examination findings on admission, past medical history, and provisional diagnosis information on recruitment. The CRFs also recorded information on prescribed medication, oxygen and intravenous (i.v.) fluids, investigations, isolation status, complications, transfer to the intensive care unit, duration of stay, deaths, quality of life (QoL), and the timing of specimen collection and test results. Information was collected from the participant by research staff on recruitment and during follow-up visits, from medical case notes, computerised hospital records, and by post, and was entered into the CRF. Clinical and QoL data, together with laboratory findings, were entered into a database written in Microsoft Access 2000 (Microsoft Corporation, Redmond, WA, USA), which was maintained on University of Leicester mainframe computers, behind electronic firewalls allowing limited access with passwords. This proved a secure and confidential way of maintaining the records.

Near-patient tests

Molecular diagnostic tests

We used molecular diagnostic tests that were developed at the Centre for Infections, HPA (London, UK), and the HPA Laboratory, Cambridge.

Conventional diagnostic tests

Outcome measures

Antimicrobial spectrum of activity

Sample size

Statistical power of the Three Winters Study (3WS) was estimated for one laboratory end point (sensitivity/specificity), the primary clinical end point (i.e. length of stay), and one secondary end point (i.e. appropriate isolation levels).

The initial sample size calculation proposed that if the average sensitivity/specificity of the tests is assumed to be 80% and a 20% dropout rate was assumed then 2752 patients in total would enable the sensitivity/specificity to be estimated to within two standard errors (SEs), i.e. 7.6% if the disease prevalence was 5% and 5.4% if the prevalence was 10%.

In terms of clinical end points, 2752 patients in total would also enable a minimum clinically significant difference (MCSD), between diagnostic policies, of 1 day in the mean length of stay [assuming standard deviation (SD) = 6 days] to be detected at the 5% significance level with over 80% power, assuming a 20% dropout rate and adjusting for the fact that there are three groups. In total 2752 patients would also enable a MCSD, between diagnostic policies, of an improvement in appropriate use of isolation facilities from 5% to 15% to be detected at the 1% significance level with over 95% power, assuming a 20% dropout rate and adjusting for the fact that there are three groups.

Clearly, the initial sample size calculation was driven by the desire to estimate the sensitivity/specificity with a sufficient level of precision but that this depended crucially upon the disease prevalence. Consequently, the sample size calculation was revisited in 2007 after the first two winters’ data were available. During 2005–6, the overall disease prevalence was 18.7%, and during 2006–7 it was 10.3%, giving a combined disease prevalence across the first two winters of 13.2%. Consequently, if the average sensitivity/specificity was 80% and the dropout rate continued to be 20% then 1200 patients in total would enable the sensitivity/specificity to be estimated to within, i.e. two SE, 7.8% assuming a prevalence of 13%. Hence, the required sample size was revised to 1200 patients in total, as this still enabled both a MCSD of 1 day in mean length of stay (SD = 6 days) to be detected at the 5% significance level with over 80% power, and a MCSD of an improvement in appropriate use of isolation facilities from 5% to 15% to be detected at the 1% significance level with over 95% power, both assuming a 20% dropout rate and allowing for the fact that there are three groups.

Statistical methods

Statistical analyses were carried out using Stata version 11 (StataCorp LP, College Station, TX, USA). The study was analysed as an intention-to-treat (ITT) study; effectively all patients were analysed according to the group to which they were randomly allocated, regardless of whether they were in fact managed according to the results of their designated diagnosis method.

For continuous, non-time-to-event, outcomes the three intervention groups were compared using either parametric [analysis of variance (ANOVA)] or non-parametric (Kruskal–Wallis) test as appropriate, and summary statistics were reported as means (SD) and medians [interquartile range (IQR)], respectively. For categorical outcomes, the three interventions groups were compared using either Pearson’s chi-squared test or Fisher’s exact test, as appropriate. For time-to-event outcomes, these were reported as median (IQR) for each of three intervention groups, and compared formally using Cox proportional hazards regression models to allow for censoring/death and reported as hazard ratios (HRs) [and associated 95% confidence intervals (CIs)]. For the primary outcome, length of stay, the three intervention groups were also compared graphically using cumulative probability plots, i.e. one minus the Kaplan–Meier survivor function. EQ-5D data – collected at baseline, 7 days and 28 days – was initially summarised at each of the three time points and compared using parametric/non-parametric methods as for other continuous outcomes. Further analysis using linear mixed-effect regression models to allow for within-patient correlation enabled an assessment of the change in EQ-5D over time to be made and whether there was evidence of an intervention group interaction with time.

A number of patients were randomised more than once owing to the nature of their age and clinical condition. The primary analyses used their first admission/randomisation. However, to assess the impact that this assumption had on the results, two sensitivity analyses were undertaken – the first using all admissions but assuming that they were independent, i.e. ignoring the fact that some patients appeared more than once, and the second, perhaps more appropriately, including a random effect in the analyses to account for the inherent correlation induced by patients being rerandomised.

All statistical tests were reported at the 5% significance level, and 95% CIs or credible intervals (CrIs) were reported throughout as the sample size calculations explicitly made allowance for the fact that there were three rather two groups and inflated the sample size accordingly to maintain an overall 5% significance level.

Ethical arrangements

This study was sponsored by the University Hospitals of Leicester NHS Trust. It was approved by Leicestershire, Northamptonshire and Rutland Research Ethics Committee on 20 July 2005, reference no. 05/Q2502/76.

Revisions to the protocol

We requested protocol amendments as outlined below. All requests were approved by the Local Research Ethics Committee. Protocol amendments that were implemented are outlined below:

1. On 13 January 2006 we requested the following amendments to the protocol in response to poor recruitment:

   • Approve a one-page ‘Synopsis’ of the six-page Patient Information Sheet to be used by patients who are not known to be suffering from dementia but were too unwell to read the full Patient Information Sheet.

   • Allow recruitment of subjects with an illness of up to 7 days’ duration (i.e. ≤ 168 hours) rather than five days’ duration (i.e. ≤ 120 hours).

   • Change from 6 hours to 8 hours the interval between initial assessment by the admitting medical team and recruitment to the trial.

   • Allow trained medical students to recruit patients to the study during out-of-hours periods.

2. On 14 March 2006, we requested the following protocol amendments.

   • Change the timing of the convalescent blood sample from day 10 to ‘between day 10 and day 30’.

   • Approve recruitment of all 18- to 64-year-old subjects with pneumonia and ILI, rather than just those 18- to 64-year-old subjects with pneumonia and ILI who have underlying chronic heart and lung disease, including asthma.

   • Recruit patients until the end of June, rather than the period ‘September to April’.

   • Approve a 20-ml urine collection and amendments to protocol where the volume was stated differently.

   • Change the labelling of the packs to correspond exactly with the trial identification numbers. For patients recruited in the Royal Infirmary, the packs were changed from RI-HTA-0001, RI-HTA-0002, etc., to 1–0001, 1–0002, etc. For patients recruited at Glenfield Hospital, the packs were changed from GH-HTA-0001, GH-HTA-0002, etc., to 2–0001, 2–0002, etc.

3. On 21 December 2006 we requested an amendment to aid recruitment:

   • Change, from 8 hours (originally 6 hours) to 16 hours the interval between initial assessment by the admitting medical team and recruitment to the trial.

4. On 20 March 2007 we requested an amendment to aid collection of convalescent sera:

   • Extend from day 10 to day 90 (i.e. 90 days after admission) the period when we could collect convalescent blood.

5. On 5 November 2007 we requested an amendment to aid recruitment of incapacitated adults:

   • Allow personal and professional representatives to provide consent for incapacitated adults to take part in the study.

Recruitment and compliance

Between 14 December 2005 and 23 May 2008, 1253 admissions were enrolled and randomised to the rapid near-patient test group (n = 418), the molecular diagnostic group (n = 415) and the conventional diagnosis group (n = 420). One individual withdrew from the study. Altogether a total of 1252 admissions were randomised as shown in Figure 4 , and participated in the study.

FIGURE 4.

Randomisation of admissions, according to diagnostic group. a, The figure 1252 refers to admissions or patient admissions (and not unique patients) – there were 1172 patients of whom 67 were admitted/randomised at least twice.

Demography

The data set comprised data relating to 1252 separate hospital inpatient episodes. There were 1172 unique patients in the study, with 67 individuals having up to four separate inpatients episodes each. The main analyses and results use the first admission for the 1172 patients (see Chapter 2 for other sensitivity analysis details). The number of admissions per patient is shown in Table 1 , together with the frequencies of inpatient episodes by patient numbers. The 1172 first admissions were randomised as shown in Figure 5 .

FIGURE 5.

Randomisation of first cases, according to diagnostic group.

The demographic variables are described for first admissions only (1172 patients) and are set out in Table 2 , as well as other patient background information deemed of interest. The baseline demographics are intended to describe the characteristics of the patients participating in the trial and to assess whether the randomisation process had been applied successfully.

From Table 2 , the only demographic variable that appears to show weak statistically significant evidence of unbalanced distribution across the three trial arms is body mass index (BMI), with a p-value of 0.091. However, the mean values of BMI across the three groups are very similar, and it is unlikely that any difference across the groups is clinically significant.

Regarding hospital admissions in the previous year, out of the 1140 first admission patients with data available, 727 had zero admissions in the previous year. The maximum number of previous admissions was 20 (one patient).

Comorbidity

Pre-existing medical conditions for the 1172 patients at first admission are set out in Table 3 . As can be seen from Table 3 , there is no evidence of any association between trial arm and comorbidity for any disease.

Time to convalescent blood samples

Convalescent blood samples were taken from patients following their admission. Median times to collection of the samples are set out in Table 4 , for first admissions only, and are similar across trial arms.

Introduction

In this section, we report the findings relating to the patient outcomes. The main results reported are based on the first admission for 1172 patients. The following diagnostic tests contributed to the diagnoses made:

1. rapid molecular tests (PCR): influenza and RSV

2. Quidel (NPT for influenza)

3. viral cultures (influenza, RSV)

4. BinaxNOW test (S. pneumoniae)

5. sputum cultures (S. pneumoniae), and

6. blood cultures (S. pneumoniae).

A patient was diagnosed as having any of the three infectious conditions (influenza, RSV or S. pneumoniae) if any of the diagnostic tests (not necessarily a test to which the patient had been randomised) showed a positive result, regardless of the results of the other tests. The Gram stain test was not used in determining diagnosis as none of the outcomes were positive for S. pneumoniae. In Chapter 5 , sensitivity to diagnostic test results is further explored using serology data for influenza.

Diagnostic test results for first admissions

The results of the diagnostic tests for 1172 first admissions are set out in Table 5 . Note that an outcome of ‘Not diagnosed’ does not indicate a definite negative diagnosis, as for the influenza tests, a diagnostic result was not available for two patients (e.g. due to lack of a suitable sample). From the first admissions, four patients were diagnosed with both RSV and S. pneumoniae; also four patients were diagnosed with both influenza and S. pneumoniae. No patients were diagnosed with both RSV and influenza.

Overall, the diagnoses appeared to be relatively evenly spread across the three trial arms, with no evidence of an association between trial arm and diagnoses. No attempt is made in these analyses to distinguish patients with multiple diagnoses (e.g. if a patient has a diagnosis of both RSV and S. pneumoniae, this patient will be counted in both groups).

Prescribing outcomes

Clinical outcomes

Length of hospital stay

Length of hospital stay among survivors was calculated as the time between admission to hospital and discharge for patients who were discharged only (excluding patients who died while in hospital). For the first admissions, only 46 patients died in hospital (39 within 28 days), with the remaining 1126 being discharged.

Using a Cox proportional hazards model to investigate any associations between the duration of hospital stay and diagnostic group, the results are set out in Table 9 , which shows the Cox HR and median hospital stay (days). Two further analyses were performed on the full cohort of all discharged patients (1205), including discharges resulting from admissions subsequent to the first. These analyses used a Cox proportional hazards model, one of which included a frailty (random effect on individual patient across multiple admissions for those patients who had more than one admission, effectively treating patient as a cluster variable).

TABLE 9 - Duration of hospital stay among survivors: median hospital stay (days) until discharge and Cox proportional hazards model (for both survivors and those who died in hospital) by diagnostic group, for all first admissions, and first admissions with RSV, influenza and S. pneumoniae only

Frailty (random effect) on individual patient (theta): 0.129, p-value for theta = 0 0.166.

Investigation	Traditional	Near patient	Rapid molecular	Total
All first admissions discharged (n = 1126)
Median days hospital stay (IQR; n)	3.23 (1.21–7.86; 374)	2.44 (0.90–7.56; 388)	2.81 (0.76–7.58; 364)	2.98 (0.95–7.75; 1126)
Cox HR (95% CI; p-value)	1	1.041 (0.903 to 1.201; 0.577)	1.109 (0.960 to 1.282; 0.160)
All first admissions discharged who were diagnosed with influenza (n = 90)
Median days hospital stay (IQR; n)	3.59 (1.40–7.28; 34)	2.08 (0.37–4.43; 26)	1.96 (0.74–6.82; 30)	2.52 (0.90–6.56; 90)
Cox HR (95% CI; p-value)	1	1.274 (0.758 to 2.143; 0.361)	1.248 (0.762 to 2.043; 0.380)
All first admissions discharged who were diagnosed with RSV (n = 27)
Median days hospital stay (IQR; n)	2.99 (2.98–3.16; 5)	2.56 (2.05–8.60; 11)	6.18 (1.27–11.75; 11)	3.10 (1.32–8.84; 27)
Cox HR (95% CI; p-value)	1	0.625 (0.203 to 1.920; 0.412)	0.559 (0.183 to 1.709; 0.308)
All first admissions discharged who were diagnosed with S. pneumoniae (n = 90)
Median days hospital stay (IQR; n)	4.21 (2.78–8.75; 37)	3.10 (1.60–6.19; 22)	4.02 (1.10–8.30; 31)	4.13 (1.82–8.50; 90)
Cox HR (95% CI; p-value)	1	1.131 (0.665 to 1.924; 0.651)	1.033 (0.637 to 1.676; 0.896)
All patients discharged (all admissions, 1205 discharges; no adjustment for multiple admissions)
Median days hospital stay (IQR; n)	3.23 (1.17–7.63; 404)	2.43 (0.90–7.48; 409)	2.81 (0.79–7.58; 392)	2.99 (0.97–7.52; 1205)
Cox HR (95% CI; p-value)	1	1.035 (0.902 to 1.188; 0.622)	1.098 (0.955 to 1.262; 0.190)
All patients who died or who were discharged (all admissions, 1205 discharges; frailtya on individual patient to adjust for multiple admissions in same patient)
Cox HR (95% CI; p-value)	1	1.055 (0.904 to 1.230; 0.499)	1.118 (0.957 to 1.306; 0.159)

As shown in Table 9 , there was little difference in the HR whether including first admissions only or all admissions. Also, there was little difference in the HR when including frailty in the Cox proportional hazards model. Frailty itself did not appear to be significant (p-value = 0.166), indicating no evidence of an effect on hospital stay, independent of diagnostic group.

Comparing the three diagnostic groups, there did not appear to be any evidence to support any association between duration of hospital stay and diagnostic group. Figure 6 shows a Kaplan–Meier estimated cumulative probability plot of time to discharge or death by randomised group. It shows the overall similarity in length of stay by randomisation group.

FIGURE 6.

Kaplan–Meier estimated cumulative probability of time to discharge (all first admissions discharged n = 1126) by randomised group.

Use of isolation facilities

Across the 1252 admissions, 112 patients were admitted to isolation, and, of the 1172 first admissions, 102 patients were admitted to isolation. Of the 1172 patients at a first admission, 120 were diagnosed with influenza or RSV of whom 14 were admitted to isolation at some time during their admission. In five cases, their time from admission to hospital to admission to isolation was zero or negative, and was replaced by duration of 0.01 hours. Owing to the small numbers of patients with RSV or influenza who were admitted to isolation, a time-to-event analysis is difficult, but the results are set out in Table 14 .

The numbers of patients with influenza or RSV who were admitted to isolation within 120 hours of admission in each group are shown in Table 15 . Owing to the small number of admissions to isolation within 120 hours of the admission, comparisons between the diagnostic groups are considered inappropriate.

Eight patients who experienced a first admission and were diagnosed with S. pneumoniae (but not concomitant influenza or RSV) were admitted to single-room accommodation. Seven were isolated for > 12 hours, including two of 22 (who were diagnosed with streptococcal infection by any means but did not have concomitant influenza or RSV) in the ‘near-patient’ group (9.1%), four of 34 in the ‘rapid molecular’ group (11.8%) and one of 35 in the ‘traditional’ group (2.9%). Overall, taking data relating to RSV, influenza and S. pneumoniae infections into consideration, we found no evidence to indicate that diagnostic group has any association with use of isolation facilities, or that patients with RSV and influenza, who pose a higher threat of nosocomial transmission than S. pneumoniae, are isolated more often.

Quality of life

Data regarding QoL (EuroQoL EQ-5D scores) are available at three time points in the trial: admission, 7 days and 28 days following admission, and was self-completed by participants. At first admission, all 1172 patients were available for EQ-5D assessment, whereas at day 7, 16 patients (with a first admission) were deceased, and at day 28, 50 patients (with a first admission) were deceased. The numbers of patients alive, by diagnostic group, and with EQ-5D data available at different times, is shown in Table 16 . Numbers of patients across the diagnostic groups (first admissions only) with full EQ-5D data available at different times during the study period are shown in Table 17 , which also provides descriptive analyses of EQ-5D scores as combined into one overall score. 66,67

There were no statistically significant differences between the three intervention groups in terms of median EQ-5D score at admission (p = 0.931), 7 days (p = 0.466) or 28 days (p = 0.117).

Repeated measures analyses, with the aim of investigating an interaction between diagnostic group and time of EQ-5D measurement on EQ-5D scores, using a linear mixed-effects model with a random effect for individual patient, are reported in Table 18 , for first admissions only. These analyses were performed initially on all data available at each time point, and subsequently on data from patients with EQ-5D scores available at admission, days 7 and 28.

The analysis of all available data indicated that there was a significant interaction between ‘near-patient’ diagnostic group and time of measurement at day 28 (p-value 0.033) and a borderline significant interaction between ‘rapid molecular’ diagnostic group and time of measurement at day 28 (p-value 0.074). In both cases the coefficient of the interaction was positive, indicating that the EQ-5D scores for day 28 were greater for the ‘near-patient’ and ‘rapid molecular’ groups compared with the traditional group. Under both sets of analyses, the effect of changing EQ-5D levels over time was also highly statistically significant (p < 0.001), with increases in EQ-5D at both day 7 (0.123; 95% CI 0.076 to 0.169), and day 28 (0.096; 95% CI 0.050 to 0.143) compared with admission.

Case reviews

Altogether eight patients in the ‘near-patient’ study group had a positive Quidel QuickVue Influenza A + B result. We examined the CRFs of these eight patients to examine at a patient level whether their medical management could have been influenced by the result of diagnostic testing. We also reviewed the CRFs of the 21 patients in the ‘near-patient’ study group whose urine on admission was positive for pneumococcal antigen using the BinaxNOW test to see if there was a temporal association between the test result and changes to case management and isolation.

Discussion

In this chapter we explored the relationship between the diagnostic group and clinical measures, notably antiviral and antibacterial prescribing, the duration of fever and hospitalisation, treatment with oxygen and CPAP, intensive care support and death. We also examined the use of single-room accommodation to see whether the location of care was influenced by different diagnostic tests. Finally, we explored whether the randomisation resulted in any differences in the QoL of survivors across groups. By each measure, we failed to demonstrate any clinical benefit to first admission patients from POCTs for influenza A and B and S. pneumoniae, or molecular diagnostic tests for influenza A and B, or RSV A and B. We also found no evidence that the microbiological diagnosis, whether obtained by POCTs or by RT-PCR, had any effect on containment.

This study was carried out in a busy teaching hospital environment that caters for a local population of approximately one million people. The patients who were recruited to our study had a broad range of acute cardiopulmonary conditions that are typically seen in acute care facilities around the country. Our cohort included substantial numbers of patients who were diagnosed with acute exacerbations of asthma, bronchitis, acute exacerbations of COPD, CAP – conditions that are associated with influenza A and B, RSV and S. pneumoniae. In our study, influenza A or B occurred in 91 of 1172 first admissions (7.8%, 6.3–9.4%), RSV in 29 (2.5%, 1.7–3.5%), and S. pneumoniae in 99 (8.4%, 6.9–10.2), i.e. infection with these pathogens occurred in more than one in six patients.

Two recent studies68,69 suggest that rapid influenza testing may influence the management of adults with influenza. Falsey et al. 68 retrospectively evaluated the clinical management of patients with positive rapid antigen tests for influenza upon admission, during a period when the hospital infection control policy mandated such testing. Viral culture tests and/or RT-PCR, or serological testing were performed if the test was negative using the Directigen POCT for influenza. Antibiotic use was cut modestly from 98.7% (79 out of 80) in patients whose antigen test was negative or unknown to 86.0% (74 out of 86) (relative risk reduction 12.8%: 95% CI 5.7 to 21.9%) in those in whom it was positive. This reduction was not associated with significant differences in either antibiotic days, length of hospital stay among survivors or antibiotic complications; rather, the mean length of stay was 41 hours longer (p = 0.16) in those found positive using rapid influenza testing. Although antiviral use was high in the antigen-positive test group (63 out of 86, 73%) compared with patients whose test was negative, or in whom the test was not performed (6 out of 80, 6%), 39% of patients treated in the study presented outside the 48-hour therapeutic window recommended for prescribing antiviral therapy. Regarding infection control, the antigen test was falsely negative in 41.5% out of 147 patients tested; it is unclear whether any of these patients were isolated. It is also uncertain how many antigen-positive cases had false-positive results and were isolated and given antiviral therapy treatment inappropriately.

D’Heilly et al. 69 evaluated the influence on clinical care of different diagnostic tests (an immunoassay and two cell culture assays) that were used to detect influenza in adults with a mean age of 57.4 years who presented to an urgent care or outpatient medical clinic at a Veterans Affairs medical centre in Minneapolis, USA. The rapid antigen test had a sensitivity of 65% (84 patients tested positive overall); 20 of the 55 cases of influenza that were detected by rapid antigen testing had symptoms of < 48 hours and received antiviral medication. Thus, in this study, rapid testing led to treatment of 23.8% of the 84 patients with influenza. Altogether, 91% of patients with a positive rapid antigen test who described symptoms of ≤ 48 hours’ duration received antiviral therapy compared with only 8% of patients with a positive rapid cell culture test but a negative rapid antigen test. Individuals with a positive rapid antigen test were significantly less likely to be treated with antibiotics.

Both of the above studies focused on older adults tested under clinical rather than controlled study conditions and illustrate the potential for near-patient influenza tests to influence prescribing practices, with certain provisos.

• First, to meet NICE criteria for treatment of seasonal influenza, it is essential that results are available to clinicians within a period of 48 hours after the onset of symptoms. 70 Our cohort presented with a median duration of symptoms before admission of 120 hours (IQR 72–168 hours) (see Chapter 7 ), and only 12% of all patients who were studied could potentially benefit from treatment with influenza antiviral drugs (on the basis of symptom duration, the turnaround time of the diagnostic tests and NICE guidance), even if they all had influenza. However, as shown in Chapter 4 , influenza occurred in 91 out of 1172 (7.7%) first admissions. Thus, with current referral and admission practice, it seems unlikely that detection and treatment of influenza in the small numbers of patients who are eligible (because of underlying ill health and illness duration of < 48 hours) would impact on the outcomes in our study. However, we did not stratify illness severity as part of the study or take into consideration the possibility that late treatment may benefit severely ill patients who are hospitalised with influenza, as such treatment falls outside NICE guidance. Evidence indicates that oseltamivir (Tamiflu^®, Roche) treatment of patients hospitalised with seasonal71–73 and pandemic 2009 H1N1 influenza74–80 is associated with reductions in radiological pneumonia, illness severity and death. Even when administered > 48 hours after symptom onset, oseltamivir showed considerable potential for reducing pneumonia due to 2009 pandemic H1N1 virus. 79 Survival benefits for H5N1 influenza have also been observed when treatment with oseltamivir was delayed up to 6 days after symptom onset. 81

• Second, the ability to diagnose influenza is dependent on the sensitivity of the test. Many of the studies that have compared rapid tests with RT-PCR and/or cell culture have been carried out in young healthy adults or in children. Children generally present earlier to medical practitioners with ILI and they shed higher titres of virus than older people, who also present later. 82 In our study, the sensitivity of the POCT in comparison with RT-PCR was low at 24.4% (see Chapter 5 ), indicating that opportunities to identify and treat patients like ours could be reduced considerably. The low sensitivity in our study may reflect the length of illness and age of the patients prior to sample collection. Nilsson et al. 83 found the sensitivity of the BinaxNOW influenza A/B test against PCR to drop from 71% to 63% and 14% when specimens were collected from adults presenting to the emergency department of Malmö University Hospital on days 1–3, 1–5 and > 5 days, respectively, following onset of symptoms. Similarly, Stripeli et al. 84 observed a fall in sensitivity from 76% to 65% for The QuickVue Influenza Test when specimens were collected from hospitalised children with disease duration of < 48 hours compared with children with disease duration of longer than 48 hours.

• Third, the sensitivity of near-patient immunoassay tests can also be influenced by virus type, and pandemic 2009 H1N1 virus compared with seasonal influenza. 85 Heinonen et al. 86 observed a sensitivity of 90% for influenza A viruses but only 25% for influenza B viruses when specimens were collected from young children aged 1–3 years who presented within 24 hours of onset of fever, and tested using the Actim Influenza A&B POCT. Reduced sensitivity for influenza B virus has also been noted by other investigators using other rapid tests. 87–89 In our study more than one-third of the specimens positive for influenza by RT-PCR were identified as influenza B virus, which may, in part, account for the low sensitivity of the near-patient QuickVue Influenza A + B test in our study (see Chapter 5 ), and the lack of clinical benefit from the POCT in comparison with molecular or conventional diagnostic tests in our study. Our study was carried out before the 2009 H1N1 pandemic.

• Fourth, the quality of the specimen and method of collection can also impact upon test sensitivity. Scansen et al. 90 compared the performance of the Quidel QuickVue Influenza A + B test on secretions from the anterior nares when a polyurethane foam swab was used for collection to that of a nylon flocked swab. The QuickVue test was positive for 40 foam and 30 flocked swabs, for sensitivities of 71% and 54%, respectively, a difference that achieved statistical significance. Agoritsas et al. 91 showed that the sensitivity of the QuickVue test was 85% with nasopharyngeal swabs, 78% with nasal swabs and 69% with nasopharyngeal washes, when specimens were collected from children with a mean age of 5 years. Anterior nasal swab collection was performed first with an absorbent foam swab, posterior nasopharyngeal swab collection was performed second with a Dacron swab, and nasopharyngeal washing was performed last with sterile saline. The difference in sensitivity between nasopharyngeal swabs and nasopharyngeal washes was significant. There was no difference in sensitivity between anterior nasal swab collection and the two nasopharyngeal collection methods. Walsh et al. 92 compared nylon flocked swab/universal transport medium collection method (sampling at the mid-turbinate level) with nasal aspirates collected from infants and toddlers in a PCR test – the swab collection method significantly outperformed the use of nasal aspirates. Schmid et al. 93 evaluated traditional and rapid culture methods for influenza using nasopharyngeal aspirates and throat swabs from adults with symptoms and signs consistent with influenza. Nasopharyngeal aspirates were twice as sensitive as throat swabs. Similarly, Smit et al. 94 tested upper respiratory tract samples in parallel with the BinaxNOW Influenza A&B combination assay, BinaxNOW Flu A and BinaxNOW Flu B assays, the Becton Dickinson Directigen Flu A + B assay and IF, and the results were compared with viral culture. Altogether 521 samples – including 338 nasopharyngeal swabs, 162 throat swabs, 19 nasal washes and two swabs from unspecified sites – were collected from 448 adults and children. There were no significant differences in the performances of all rapid antigen tests, with sensitivities of 53–59% compared with culture and IF but the sensitivities of all the rapid antigen tests were significantly higher for nasopharyngeal samples than for throat swabs. In our study, nasopharyngeal specimen collection was undertaken by trained research staff, and specimens were handled as per the manufacturer’s instructions. It is therefore unlikely that specimen quality was responsible for the observed outcomes.

Most antibiotics prescribed in general practice are for respiratory tract infections that represent about 70% of all infections treated in primary care. 95,96 In the USA, antibiotic prescribing for acute respiratory illness accounts for over 50% of all antibiotics used in primary care. 97 Antibiotic prescribing has decreased in UK general practices in recent years mainly because there are fewer consultations for common respiratory infections, and partly because GPs are prescribing antibiotics less frequently for conditions that are primarily viral in aetiology. 98 There is a paucity of evidence showing benefit from antibiotic use in many acute respiratory conditions including acute bronchitis,99 acute asthma exacerbations,100–102 and in many cases of acute exacerbation of chronic obstructive airways disease. 103–105 Although overprescription of antibiotics has led to the emergence and proliferation of antibiotic-resistant bacteria,106–109 a meta-analysis of 122 reports of CAP showed that approximately 6000 (18%) of 33,148 cases had a bacterial pathogen, with S. pneumoniae accounting for 73% of all cases and 66% of fatal cases. 16,110 Thus, clinicians face the problem of unnecessary use of antibiotics for some respiratory conditions – and the requirement for prompt (empiric) antimicrobial treatment for others.

National and international bodies including the British Thoracic Society (BTS) provide guidance for empiric treatment of CAP due to the risks from delayed treatment and limitations of diagnostic tests. 17 Isolation of S. pneumoniae from the blood is specific but lacks sensitivity. 111–113 Culturing blood before antibiotic therapy is recommended by the BTS (and similar bodies) for all hospitalised patients with moderate and high-severity CAP, but studies have suggested that blood cultures rarely alter antibiotic therapy for patients presenting with pneumonia because of the low overall positive rate of blood cultures. 112,114–118 Moreover, clinicians did not follow protocols for narrowing an antibiotic spectrum even when appropriate. Sputum culture is often used to help identify aetiological agents and is recommended by BTS to investigate moderate severity CAP and severe CAP that fails to improve. 17 The test lacks specificity, and isolation of S. pneumoniae from sputum may represent colonisation. 119 In our study, sputum culture had a sensitivity of 100% (2.5–100%) (see Chapter 5 ) when compared with blood culture as the reference standard but the CIs were extremely wide due to the infrequency of a productive cough in our cohort. Musher et al. 120 analysed 105 patients with pneumococcal pneumonia proven by blood culture – sputum culture was positive in only 44% of all cases. These investigators reviewed earlier articles and noted that sensitivity of Gram staining in proven cases of pneumococcal pneumonia ranged from 20% to 69% and that the sensitivity of culture ranged from 29% to 94% (see Chapter 6 ).

Given the limitations of conventional diagnostic tests for S. pneumoniae infection, the development of a S. pneumoniae urinary antigen test offered several advantages – ease in getting a diagnostic specimen, ability to detect pneumococcal infection after antibiotic treatment has been commenced, its diagnostic yield, and a rapid turnaround time. Among adult patients with CAP, this test has shown sensitivities of 64–87% compared with blood culture,111,121 but dropped to 54% when cultures of respiratory tract secretions were included in the reference standard. 122 In our study, the sensitivity of the urinary antigen test in comparison with blood culture was 57.1% (18.4% to 90.1%) (see Chapter 5 ), which is comparable with previous reports. In our study, the POCT for pneumococcal infection was positive in 92 (7.8%; 6.4 to 9.5) of all 1172 admissions and, together with POCT for influenza, it had no evident effect on clinical outcome.

Only one patient in this study was prescribed a NI. Reasons for the low use these drugs might include the length of illness pre-admission together with unfamiliarity with NIs and the diagnostic test. The number of patients in the ‘near-patient’ group with a positive QuickVue Influenza A+B test was too small to draw definitive conclusions – none of eight patients with a positive result was eligible for treatment based on their duration of symptoms pre-admission and NICE guidance.

Regarding ‘containment’, two of the eight patients in the ‘near-patient’ group with a positive QuickVue Influenza A+B test were already in single-room accommodation when the test was carried out. One patient was discharged shortly afterwards. None of the remaining five patients was isolated, but four had fever and respiratory symptoms and were probably still shedding influenza virus. 123–125 We may reason that single-room accommodation was either unavailable, or that the nursing and medical carers were unaware of the possible risk of nosocomial transmission. Altogether, 120 of the 1172 patients with a first admission had positive tests for influenza or RSV, of whom only 14 were given single-room accommodation during the admission.

The US Healthcare Infection Control Practices Advisory Committee recommends that patients with influenza are given single-room accommodation, when available, for a period of 5 days, and masked when transported out of the room. 126 The problem for clinicians is recognising patients with influenza, as exacerbations of asthma,127 cystic fibrosis,128 COPD,129 heart failure,130 bouts of pneumonia,17 acute bronchitis131 and other acute respiratory conditions are all associated with influenza, and such complications may dominate the features of influenza. Nosocomial influenza is a well-recognised problem in acute care hospital settings. 132–135 Evidence indicates that nosocomial infection of health-care workers with influenza in acute hospitals is not uncommon136 and that influenza acquired within hospitals can be fatal. 137 In our study, the rate of isolation for patients with influenza or RSV (14/120; 11%, 95% CI 7.1% to 18.6%) was no different to the overall rate of isolation (112/1172; 9.5%, 95% CI 7.9% to 11.4%), suggesting that hospitals like ours have inadequate resources to contain infection.

The extremely low prescription rate for NIs in this study indicates that near-patient testing for S. pneumoniae could not reduce NI use, even if considered appropriate. Bacterial pneumonia is a well-recognised complication of influenza and occurred in 29% of ≈1200 patients hospitalised during the 2009 H1N1 pandemic. The pneumococcus was by far the most common pathogen,138–145 suggesting that withdrawal of NIs from some patients with a positive urinary pneumococcal antigen test may be inappropriate.

Statistical methods

Data on diagnostic performance of the various tests (blood culture, viral culture, sputum culture, PCR, Quidel and Binax POCTs) for the diagnosis of (1) influenza and (2) S. pneumoniae infection, are summarised as:

• percentage diagnostic agreement (the percentage of test results, either positive or negative, that are in agreement) (and associated 95% CI), or

• sensitivity (percentage of true-positives correctly identified)

• specificity (percentage of true-negatives correctly identified) (and associated 95% CI)

• positive predictive values (PPVs) (percentage of positive test results that are truly positive) and negative predictive values (NPVs) (percentage of negative test results that are truly negative) (and associated 95% CI), and

• area under receiver operating characteristic (ROC) curve (area under the curve, AUC) (the probability that two patients – one diseased and one not diseased – would be both correctly classified by the test) (and associated 95% CI).

In the case of influenza, uncertainty remains as to what is currently considered to be the reference standard to which all tests should be compared (i.e. ‘gold standard’). When the 3WS was designed, standard laboratory testing procedures were anticipated to be the then current ‘gold standard’. However, during the course of the 3WS, PCR techniques have been developed and now may be considered to be the ‘gold standard’ – for example in the meta-analysis reported in Chapter 6 of influenza diagnoses, 13 out of 64 (20%) studies used PCR as the ‘gold standard’ and only 24 of 64 (37%) studies considered laboratory culture alone to be the ‘gold standard’. Hence a number of analyses using different ‘gold standards’ have been undertaken to enable comparison between the results from 3WS and those found in the literature.

In the case of influenza, test performance could also be assessed by serology. The primary analysis uses only definite positive serology results and does not allow for month and time between blood samples. However, due to the nature of the 3WS, patients could be discharged from hospital and subsequently have had an influenza vaccination prior to a second (convalescent) blood sample being taken. In order to allow for this, and the fact that positive serology results were graded as definite, probable and possible, two sensitivity analyses were undertaken: (1) excluding those who had their first blood taken between September and December and had > 30 days between samples and (2) including both definite and probable positive serology results.

All analyses were undertaken in Stata version 11.

Results

Discussion

We compared the diagnostic performances of viral culture, Quidel QuickVue Influenza A + B test and RT-PCR, using either PCR or viral culture as the ‘gold standard’. With PCR as the gold standard, viral culture carried out at the UHL NHS Trust laboratory had a sensitivity of just 21.6% (95% CI 13.5% to 31.6%), which, although similar to the sensitivity reported by the Portuguese national surveillance network, during the 7-year period 1992–93 to 1998–99,146 is suboptimal in comparison with sensitivities of 61–96% reported in nine studies90,91,147–153 ( Table 24 ) that were identified for the systematic review and meta-analysis of POCTs for influenza (see Chapter 6 ). However, the sensitivity of viral culture was similar to that of the Quidel POCT (24.4%, 95% CI 16% to 34.6%). Although the performance of both the viral culture and the Quidel POCT appears relatively low in terms of identifying truly positive cases, especially in comparison with the headline estimates of the meta-analysis of POCTs for influenza (see Chapter 6 ), any comparisons need to be approached with caution owing to the extremely heterogeneous nature of the studies reported, and because subgroup and sensitivity analyses indicate that 3WS provides comparable estimates of diagnostic performance to appropriate comparator studies (see Chapter 6 ).

Factors that may contribute to lower rates of influenza virus isolation, include the type of swab collection device, the cell culture system, the passage history of the mammalian cells used, the stability of protease supplement to the media and, possibly, alterations in receptor binding properties of isolates of influenza A H3N2 and H1N1 since the late 1980s. 146,154 Specimens were collected according to the methods recommended by the HPA, and by the manufacturer of the Quidel POCT (see www.cliawaived.com/web/items/pdf/QDL-20183-Quidel_Influenza_Tests_Insert∼619file1.pdf) by research nurses dedicated to the project. Thus, it seems unlikely that the expertise of personnel who collected the specimens or the method of specimen collection were important contributory factors to the low sensitivities of culture and the Quidel POCT observed in our study. In our study specimens were received by the laboratory at a median of 3.6 hours after specimen collection, making delayed transportation and inappropriate specimen handling unlikely as contributory factors. A more plausible explanation is the length of illness prior to sampling, which in our study was a median of 120 hours. Lee et al. 125 evaluated secular shedding of seasonal influenza virus from 147 inpatients aged > 16 years (mean age 72 years, ± 16 years) in Hong Kong. The results of virus isolation showed that among untreated (i.e. by NIs) patients, 38.5% and 21.2% remained culture positive by symptom day ≥ 4 and ≥ 5, respectively, and in patients with comorbidities the proportions were somewhat higher, at 41.7% and 33.3%, respectively. Leekha et al. 155 examined secular shedding of seasonal influenza virus by 50 hospitalised patients by culture. Results showed that by symptom days 3, 4, 5 and ≥ 7, 91%, 75%, 52% and 20% of patients, respectively, remained culture positive. Although these observations have implications for infection control, they show that by day 5 of symptoms (i.e. the median duration of symptoms upon sampling in our study), approximately 50–80% of patients with influenza were unlikely to be shedding infectious virus. Presence of viral nucleoproteins that are detectable by the Quidel POCT is likely to fall at a similar rate.

Despite being the most common pathogen in CAP, S. pneumoniae is underdiagnosed because of the limitations of conventional diagnostic tests. Although blood culture is used as the ‘gold standard’ for diagnosis of pneumococcal CAP in many studies, it has low sensitivity and premedication with antibiotics reduces it further. The observed sensitivity (57.1%, 95% CI 18.4% to 90.1%) and specificity (92.5%, 95% CI 90.6% to 94.1%) for the BinaxNOW urinary antigen detection test in our study are comparable to sensitivities of 75% to 88%49,156–160 compared with blood culture (or culture of blood and pleural aspirate) in previous studies. The BinaxNOW POCT has a high specificity, in excess of 90%, and contributed to a reduction in the spectrum of antibiotic cover of 41 of 474 episodes of CAP in one recent study. 161 In comparison with blood and sputum cultures, the pneumococcal urinary antigen test has advantages of being a simple and rapid method with visually detectable results (see Chapter 7 ), providing speedy diagnosis without any additional equipment or reagents in a clinic or triage setting; is non-invasive; positive results persist over a period of days and occur despite treatment with antibiotics; and the test has high sensitivity and specificity. However, a negative result cannot rule out pneumococcal infection.

In our study, sputum culture had a sensitivity of 100% (2.5–100%) when blood culture was used as the gold standard. The CIs are wide due to the small number of patients with a positive blood culture and productive cough. Musher et al. 120 analysed 105 patients with pneumococcal pneumonia proven by blood culture. Sputum culture yielded a pneumococcus in 44% of cases. Sensitivity data for sputum culture from that review and several recent studies are shown in Table 25 . 119,120,162–167 Random-effects meta-analysis of the data in Table 25 reveals sensitivities of 43% (33–53.0%) when cases who received antibiotics are included, and 59% (21–92%) when they are excluded. Specificity data are not available, but the specificity of sputum culture is generally poor due to colonisation of the respiratory tract. The poor sensitivity and specificity of sputum culture therefore question its value in the management of lower respiratory infections. Sputum culture also has downsides of cost and ease and speed of use, but empiric antibiotic recommendations are ultimately dependent upon the accumulated information available from such tests.

Introduction

Influenza resembles other acute viral respiratory infections with respect to its seasonality, clinical presentation and complications, but it differs from other respiratory viral conditions by being preventable by annual vaccination and ameliorated by antiviral drugs if given within 48 hours of symptom onset. The gold standard for influenza diagnosis is viral culture, which, although specific, had low sensitivity compared with real-time RT-PCR in our study and gave results long after hospital discharge or death (see Chapters 5 and 7 ). In contrast, we were able to correlate the results of PCR and serology (see Chapter 5 ), confirming the accuracy of RT-PCR, and we showed that it gave a diagnosis in a busy clinical setting within a median of 29 hours (IQR 13.5–31.6 hours) (see Chapter 7 ). This turnaround time might facilitate timely antiviral therapy but concerns have been raised that the demands of the test, requiring transportation of specimens to a laboratory with specialised expertise and equipment, and its turnaround time make it too slow for therapeutic or infection control purposes. 168–170

Commercial POCTs were used to manage patients with ILI during the 2009 H1N1 pandemic. 168,169,171,172 However, there have been mixed reports of the diagnostic accuracy of such tests, perhaps reflecting the test used, patient age, the method of sample collection and transport, the ‘gold standard’ used, and the type and subtype of influenza (i.e. whether seasonal or pandemic influenza, or influenza B). According to the manufacturer, the QuickVue Influenza A + B test detected all 24 influenza A viruses, subtypes H1–H15, which were isolated from birds and mammals, although performance characteristics were not established (see www.cliawaived.com/web/items/pdf/QDL-20183-Quidel_Influenza_Tests_Insert∼619file1.pdf).

We used the QuickVue (Quidel, USA) POCT in our study and found that it had a sensitivity of 33.3% and a specificity of 98.6% when compared with culture and 24.4% and 99.7%, respectively, when compared with RT-PCR (see Chapter 5 ). To compare our findings with other studies, we systematically reviewed published articles on the diagnostic accuracies of commercially available influenza POCTs. To assess the quality of the methodology and the completeness of the reporting of each manuscript, we ‘scored’ each publication using the QUADAS (quality assessment tool for diagnostic accuracy studies) tool and the Standards for Reporting of Diagnostic Accuracy (STARD) initiative. 173–176

Methods

Literature searches

On-line searches were made on MEDLINE/PubMed on 28 April 2011 and on the Bioscience Information Service (BIOSIS) and The Cochrane library on 27 May 2011 for publications on influenza POCT diagnostic accuracy studies between 1991 and 2011 (inclusive) that met the following five criteria:

1. Articles written in English.

2. Commercially available test kits.

3. Testing done in human seasonal and pandemic influenza.

4. Sensitivity results with specific numerators and denominators.

5. We had authorised journal access.

Medical subject heading (MeSH) search phrases included:

1. “QuickVue test influenza”.

2. “Rapid influenza test”.

3. “Rapid antigen test influenza”.

4. “POCT influenza”.

5. “Immunochromatographic test influenza”.

6. “Bedside test influenza”.

Figure 7 shows a flow diagram of the manuscript screening process, taken from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline on systematic reviews. 177

FIGURE 7.

The manuscript screening process. a. Partial data from three publications.

Results

More than 2000 publications were found using the MeSH terms and 490 of these were relevant. In total, 70 out of the 490 publications met all five criteria and were selected for the systematic review. Some of the 70 had more than one finding, which we called ‘studies’. There were 143 studies altogether. Twenty-eight of the 70 publications reported full 2 × 2 data and were used for meta-analysis. There were a total of 68 studies in the 28 publications but four studies from three publications were excluded. 88,172,181 Thus 64 studies from 28 publications were included in the meta-analysis. Figure 7 shows a flow diagram of the manuscript screening process, taken from the PRISMA guidance on systematic reviews. 178

Appendix 6 summarises the publications that were screened and Table 26 summarises the sensitivities and specificities of the 64 POC studies that were included in the qualitative synthesis. Table 27 presents the QUADAS results and Table 28 the STARD results. Figure 8 shows the percentage of studies satisfying each of the QUADAS items and Figure 9 displays the distribution of the overall QUADAS score. As can be seen from Figure 9 , 40 of the 64 (62.5%) studies that were included in the meta-analysis scored > 10 indicating that the studies were of a reasonably high quality overall.

FIGURE 8.

Percentage of studies satisfying QUADAS items.

FIGURE 9.

Distribution of QUADAS Score.

Of the 143 studies in Appendix 6 for which data could be extracted, 64 (45%) reported the full 2 × 2 table, and using a bivariate mixed-effects meta-analysis model produced an overall estimate of sensitivity of 0.73 (95% CI 0.66 to 0.80) and of specificity of 0.99 (95% CI 0.98 to 0.99). However, there was a high level of heterogeneity between the studies for both outcomes (sensitivity: Q = 777.5, p < 0.01, I ² = 91.9%, 95% CI 90.5% to 93.3%; specificity: Q = 2128.9, p < 0.01, I ² = 97.0%, 95% CI 96.7% to 97.4%), as can be seen from the Forest plots in Figure 10 . Figure 11 displays the summary receiver operating characteristic (SROC) curve derived using the estimated overall pooled sensitivity and specificity. Superimposed on the SROC curve are the results from 3WS using (1) PCR (sensitivity 24.4%, specificity 99.7%) and (2) viral culture (sensitivity 33.3%, specificity 98.6%) as the ‘Gold standard’ tests. As can be seen from Figure 11 , the 3WS results, although being within the associated prediction region, are nevertheless considerably lower, in terms of sensitivity, than those estimates from other studies in the meta-analysis.

FIGURE 10.

Forest plot of sensitivity and specificity for 64 studies reporting full 2 × 2 data table.

FIGURE 11.

Summary receiver operating characteristic estimated using bivariate mixed-effects meta-analysis model with 3WS results using PCR (a) and viral culture (b) as ‘gold standard’ superimposed.

To explore the between-study heterogeneity observed, a number of subgroup-specific models were estimated using study-level covariates (age of participants, ‘gold standard’ used, geographical region in which study was conducted, type of influenza tested for, type of POCT, and study quality assessed using the QUADAS tool). Owing to the relatively small numbers of studies in some specific subgroups (i.e. < 4) it was not always possible to estimate the associated effects. It can seen from Table 29 that the pooled estimates of specificity across subgroups were consistently high with relatively little subgroup-to-subgroup variation; however, for sensitivity there was variation in the pooled estimates across subgroups with a mixed age distribution, use of PCR as a ‘gold standard’, and testing for influenza of swine origin all yielding lower estimates of sensitivity. There was also some variation in sensitivity depending on the geographical region in which the study was conducted, with Europe and Australasia yielding lower estimates; this was also seen for study quality, with ‘higher’-quality studies producing a lower pooled sensitivity than those of ‘lower’ quality.

In terms of the subgroup specific estimates of pooled sensitivity, even the lower estimates were still higher than those found in 3WS – using PCR as the ‘gold standard’, a Quidel POCT produced a sensitivity of 24.4%, whereas using viral culture as the ‘gold standard’ it was 33.3%. Although the distribution of studies across the various study-level covariates (and their levels) makes it difficult to obtain an estimate of sensitivity that closely matches the characteristics of 3WS – it is possible to estimate a pooled effect for those studies (n = 5), which (1) included both adults and children (as opposed to only children) and (2) compared Quidel with PCR. These produced an overall pooled sensitivity of 34% (95% CI 14% to 62%) and specificity of 99% (95% CI 97% to 100%), which are much more similar to those obtained in 3WS.

As only 64 out of 143 studies (45%) reported sufficient data to permit a formal bivariate analysis to be undertaken, a sensitivity analysis only pooling sensitivities for the 64 and 79 studies separately was undertaken to assess whether there was in fact a selection effect. The 79 studies produced a pooled sensitivity of 60.2% (59.0% to 61.0%), whereas the 64 studies produced an estimate of 69.1% (67.5% to 70.6%) – the latter was slightly lower than that produced by the bivariate model (73%, 95% CI 66% to 80%). A formal test of heterogeneity between the two sets of studies was highly significant (p < 0.001) indicating that had the 79 studies reported both sensitivity and specificity the corresponding bivariate model would have produced an estimate of sensitivity lower than that observed.

Discussion

This systematic review concurs with other reviews of diagnostic studies in different disease areas in that there was considerable heterogeneity – both in terms of reporting of data and clinical and methodological characteristics, thus making formal synthesis of study results using appropriate bivariate meta-analysis methods challenging. 203

Overall, the bivariate meta-analysis produced estimates of sensitivity that were considerably higher than that observed in 3WS. However, exploration of the considerable between-study heterogeneity using subgroup analyses showed that for some subgroup combinations the pooled estimate of sensitivity was considerably lower than that estimated for others. In fact, the subgroup combination most closely resembling the characteristics of 3WS (comparing Quidel POCT with PCR in a mixed-age population) produced an estimate of sensitivity in close agreement with 3WS.

Further sensitivity analysis comparing those studies that reported fully data for sensitivity and specificity with those that only reported sensitivity indicated the possibility of a selection effect that would further reduce the true estimate of sensitivity of POCT for testing for influenza.

Published evidence on the usefulness of diagnostic tests has been summarised in four systematic reviews including this review. Uyeki204 reviewed published evidence on clinically useful diagnostic tests and antiviral treatment for influenza virus infections in children, which were published in the English language from 1966 to September 2002. The topics covered were wide-ranging, including clinical diagnosis, IF and rapid influenza diagnostic tests, as well as antiviral treatment. Altogether 28 studies of rapid influenza tests were identified. This was a descriptive study with no formal assessment of study quality or meta-analysis of the findings, rather the author presented median sensitivity values for the tests in comparison with cell culture as the gold standard. Overall, the POCTs had sensitivities and specificities of 40.4% to 100% and 65.2% to 100%, respectively. The median sensitivity of the Zstat Flu test (Zyme Tx, Oklahoma City, OK, USA) was 68.8% (range 28.1–96%) and median specificity was 83% (range 62.7–92.4%). The median sensitivity of the Directigen Flu A test was 87.2% (range 39–100%), and the median specificity was 98.1% (range 84–100%).The median sensitivity of the FLU OIA test (Biostar, Boulder, CO, USA) was 71.8% (range 36.7–93%), and the median specificity was 82% (range 65.2–95.7%). In five studies of the QuickVue Influenza Test the median sensitivity was 79.2% (range 74–95%) and the median specificity was 91.9% (range 76–98%). The studies were evidently heterogeneous in terms of age and the author concluded that rapid influenza diagnostic tests were ‘moderately to reasonably’ accurate for detecting influenza virus infections, and that false-negative results appeared more common than false-positive results.

Petrozzino et al. 205 were supported by the Quidel Corporation, the manufacturer of the QuickVue Influenza A + B test, to undertake a systematic review and meta-analysis of studies reporting sensitivity, specificity, and effects of ‘rapid flu tests’ (RFTs) and clinical diagnosis on decision-making for patients with ILI. Search results were limited to literature published in English between 1984 and 2009. Results from included studies were stratified according to age categories with an approximate cut-off of 15 years of age. It was not possible to stratify results for older people aged > 60 years. No RCTs were found directly comparing RFTs against the clinical diagnostic skills of clinicians. All included studies used an independent gold standard test for confirmatory influenza diagnosis. Separately, these investigators evaluated the clinical diagnosis of influenza.

Among older subjects aged ≥ 15 years, data on the QuickVue test from five studies showed that this POCT had a sensitivity in a fixed-effects model of 57% (52–62%) and specificity of 96% (95–97%). In a random-effects model, the POCT had a sensitivity of 61% (36–81%) and specificity of 96% (94–98%). Data from 11 studies showed the sensitivity of clinical diagnosis in a fixed-effects model to be 64% (51–75%) and much lower specificity of 65% (63–66%). In the random-effects model, clinical diagnosis had a sensitivity of 64% (51–75%) and specificity of 68% (57–77%). Thus clinicians were as able to diagnose influenza clinically as POC testing but wrongly identified other individuals as having influenza, which could be problematic when isolation facilities for adults are scarce, although it is conceivable that these patients also posed an infection risk to others.

Among younger subjects aged < 15 years, data on the QuickVue test from 14 studies showed that this POCT had a sensitivity in a fixed-effects model of 63% (60–67%) and specificity of 94% (92–95%). In a random-effects model, the POCT had a sensitivity of 76% (65–85%) and specificity of 95% (92–98%). Data from five studies showed the sensitivity of clinical diagnosis in a fixed-effects model to be 70% (66–74%), and the specificity of 61% (59–63%) was again lower than that of the POCT. In the random-effects model, clinical diagnosis had a sensitivity of 69% (44–87%) and specificity of 63% (31–87%). For all age groups combined, the sensitivities of the POCT and clinical diagnosis in both the fixed and random-effects models were similar, with sensitivities of 61% (59–64%) and 62% (60–63%) respectively in the fixed-effects model, and 72% (62–81%) and 65% (55–74%) in the random-effects model. The respective sensitivities were 94% (93–95%) and 63% (62–64%) in the fixed model, and 96% (93–97%) and 67% (57–76%) in the random-effects model.

These authors examined 10 studies reporting outcomes relating to patient management associated with the use of POCT for influenza. This overview led the authors to conclude that in various clinical settings and across a wide age range, RFT use in patients presenting with ILI leads to reduced diagnostic testing, antibiotic use and emergency department length of stay, although increases antiviral prescribing.

Babin et al. 206 did a review and meta-analysis of published literature on the 2009 novel swine flu outbreak to assess the potential utility of POCTs for initiating infection treatment and control for this pathogen. Although these POCTs were not developed for swine-origin virus, their speed and ease of use (EoU) made them attractive for clinical and public health use. These authors identified 14 reports on sensitivity and/or specificity of seven different POC influenza tests for diagnosis of 2009 pandemic H1N1 virus on clinical specimens. The pooled sensitivity and specificity for all studies were 67.5% (95% CI 66.2% to 68.9%) and 80.7% (95% CI 80.0% to 81.4%). Pooled data were provided for three POCTs from different manufacturers: BinaxNOW Influenza A&B, with a pooled sensitivity of 31.4% (95% CI 26.3% to 36.7%) (no specificity data); Directigen EZ Flu A + B, with a pooled sensitivity of 52.8% (95% CI 45.9% to 59.6%)%) (no specificity data); and QuickVue A + B, with a pooled sensitivity of 73.6% (95% CI 72.1% to 75.0%), and specificity of 76.6% (95% CI 75.5% to 77.5%). In conclusion, the authors considered that the relatively poor performance of the POCTs affirmed recommendations by the US Centers for Disease Control and Prevention (CDCP) that caution should be applied in the interpretation of negative POCTs.

Our systematic review and meta-analysis confirms and extends the observations in the above reports. In our study, specificity across subgroups was consistently high but sensitivity was higher in studies involving children and adolescents than in ‘mixed’ populations (i.e. mixed age groups). We may speculate that this might reflect decreased virus shedding in adults than in young children (although virus shedding may be high in very elderly hospitalised patients)125 and the effects of vaccination and past infection. We also found that the test sensitivity was a function of the nature of the gold standard used, with PCR setting a higher target than virus culture. The caution issued by the CDCP regarding the sensitivity of POCTs for the detection of the 2009 pandemic ‘swine-origin’ H1N1 virus was affirmed in our analyses, which also showed better performance of POCTs in the detection of seasonal influenza type A virus than type B virus.

Background

The benefits of rapid diagnosis are increasingly appreciated in diverse health-care settings. Besides direct clinical benefits, political and economic imperatives, including the UK NHS 4-hour patient assessment rule in A&E departments, are likely to accelerate demand for rapid diagnostic tests. 207–209 However, successful adoption of new technology in the health-care sector, and especially in hospitals, depends on its acceptance by end users. 210,211 The latter is a function of the ease of the required associated procedures, extent of demand on staff time, perceived efficiency, quality and added clinical value of speedy results and the actual reduction in time to a result for the new rapid diagnostic test compared with the method in use. Although there have been various studies comparing one or more of these features of diagnostics tests,212–214 to the best of our knowledge only one formal rating system has been described to score the ease and speed of use of diagnostic methods. 215

In the USA, the Secretary of Health and Human Services has delegated to the Food and Drug Administration (FDA) the authority to determine whether particular tests are ‘simple’ and have ‘an insignificant risk of an erroneous result’. The US Clinical Laboratory Improvement Amendments (CLIA) categorisation criteria grade specific laboratory test system, assay and examination for level of complexity by assigning scores of 1, 2 or 3 for each of seven criteria: (1) knowledge; (2) training and experience; (3) reagents and materials preparation; (4) characteristics of operational steps; (5) calibration, quality control and proficiency testing materials; (6) test system troubleshooting and equipment maintenance; and (7) interpretation and judgement. 215 A score of ‘1’ indicates the lowest level of complexity, and the score of ‘3’ indicates the highest level. These scores are totalled to derive a measure of complexity. Here we evaluate the ease and speed of use of selected POCTs, PCR-based rapid molecular assays and traditional culture for the aetiological diagnosis of respiratory infections. As outlined below, we modified the scoring system proposed by CLIA and included additional criteria, specifically: (1) test site requirements; (2) equipment; (3) storage and disposal of waste test materials and reagents; (4) health and safety implications; and (5) time to reporting (TtR) of results.

Specimen handling and diagnostic tests

Specimens were collected and transported as described in Chapter 2 . We evaluated point of care, molecular, and traditional test methods for the diagnosis of respiratory infections as described in Chapter 2 . These included the following.

POCTs:

• QuickVUE Influenza A&B (Quidel, USA).

• BinaxNOW S. pneumoniae test.

Molecular diagnostic tests:

• Semi-nested multiplex conventional PCR for influenza A subtypes H1 and H3, influenza B, and RSV A and B (as used during Year 1).

• HPA Cambridge, one-step, quadriplex, real-time RT-PCR for the detection of influenza (as used during Year 2).

• HPA Colindale, one-step, multiplex, real-time RT-PCR for the detection of influenza (as used during Year 3).

• RSV and hMPV, one-step (Year 2) and two-step (Year 3), multiplex real-time RT-PCR.

Conventional diagnostic tests:

• Virus isolation and culture on continuous cell lines.

• Blood culture.

• Sputum culture.

• Sputum Gram stain and microscopy.

Ease-of-use scores

Each specific test procedure was graded by assigning EoU scores of ‘1’, ‘2’ or ‘3’ for each of the 11 criteria listed below. Like the CLIA system, a low score reflects the lowest levels of complexity of test procedures that can be carried out quickly by personnel with minimal training. A high score reflects high levels of complexity. A score of ‘2’ was assigned to scoring criteria when the characteristics for a particular test are intermediate between those listed for scores of ‘1’ and ‘3’.

Results

As outlined in Chapter 3 , 1252 admissions were enrolled into the study, of which 418 were randomised to the ‘near-patient’ group, 415 to the ‘molecular’ diagnostic group, and 419 to the ‘conventional’ diagnostic group. The median duration of symptoms before recruitment was 120 hours (IQR 72–168 hours), and the median time to discharge or death was 72 hours (IQR 21–180 hours).

Speed of use of test methods for the diagnosis of respiratory infections

Quidel QuickVUE Influenza A&B

The interval between specimen collection and the results of the near-patient Quidel QuickVue Influenza A + B test was recorded for 327 of the 418 admissions who were randomised to ‘near-patient’ test group (Group 1). The median interval from specimen collection to the provision of a positive or negative result for the 327 admissions was 15 minutes (IQR 10–23 minutes) ( Figure 12 ). Altogether, 278 results (85%, 95% CI 80.7% to 88.7%) were reported by 30 minutes.

FIGURE 12.

Time interval between specimen collection and report (QuickVUE test).

BinaxNOW S. pneumoniae test

Urine samples from 412 out of 418 admissions in the ‘near-patient’ group (Group 1) were available on recruitment for POC testing using the BinaxNOW S. pneumoniae test. The interval between specimen collection and the results of the near patient BinaxNOW S. pneumoniae test was recorded for 324 admissions. The median interval from specimen collection to the provision of a positive or negative result was 20 minutes (IQR 15–30 minutes) ( Figure 13 ). Altogether, 270 results (83.3%, 95% CI 78.8% to 87.2%) were reported by 30 minutes.

FIGURE 13.

Time interval between specimen collection and report (BinaxNOW S. pneumoniae test).

Semi-nested multiplex polymerase chain reaction for influenza A subtypes H1 and H3, influenza B and respiratory syncytial virus A and B

The median TtR of results obtained by ‘conventional’ semi-nested multiplex PCR (n = 57, Group 2) was 50.8 hours (IQR 44.3–92.6 hours). Figure 14 shows a plot of the time interval between specimen collection and the report. Altogether 12 results (21.0%, 95% CI 11.4% to 33.9%) were reported by 36 hours; 18 (31.6%, 95% CI 19.9 to 45.2%) were reported by 48 hours; 36 (63.1%, 95% CI 49.3% to 75.6%) were reported by 72 hours, and 45 (78.9%, 95% CI 66.1% to 88.6%) were reported by 96 hours.

FIGURE 14.

Time interval between specimen collection and report (semi-nested multiplex PCR).

One-step, quadriplex/multiplex, real-time reverse-transcriptase polymerase chain reaction for influenza, and one-step and two-step respiratory syncytial virus and human metapneumovirus multiplex real-time polymerase chain reaction

These real-time PCRs are considered together owing to their similarity. The median TtR of results obtained by ‘real-time’ RT-PCR (n = 358, Group 2) was 29.2 hours (IQR 26–46.9 hours) ( Figure 15 ). The difference between the time taken to analyse specimens by the ‘conventional’ semi-nested multiplex PCR and real-time PCR was 21.9 hours (p < 0.0001, Mann–Whitney U-test). Figure 16 shows the percentage of specimens that are reported as positive or negative by real-time PCR in increments of 12 hours (and at 30 hours) after specimen collection. By 30 hours, results were available for 220 (61.3%, 56.0% to 66.3%) admissions.

FIGURE 15.

Time interval between specimen collection and report (real-time RT-PCR).

FIGURE 16.

Percentage (± 95% CIs) of specimens reported as positive or negative by real-time RT-PCR at 12-hour intervals after collection.

Viral culture

Viral culture results were available for 1245 (99.4%, 95% CI 98.8% to 99.8%) of the 1252 nasopharyngeal specimens that were collected from all admissions in Groups 1, 2 and 3. The cumulative percentage of specimens reported by 24-hour time periods for influenza A and B (n = 21), other viruses (n = 49) and specimens that were culture negative (n = 1175) is shown in Figure 17 . The median time from specimen collection to reporting a positive culture result for admissions with influenza A and B was 629.6 hours (IQR 262.5–846.7), which is approximately nine times greater than the median time to discharge or death (72.08 hours). The median turnaround time of 220.4 hours (IQR 172.1–314.5 hours) for reports of viruses other than influenza was significantly shorter than for the time for influenza (p < 0.0001, Kruskal–Wallis test). The time to report the isolation of influenza A or B did not differ from the time to report culture negative results (454.7 hours, IQR 406.0–621.5 hours). Less than 6% (70 of 1245, 5.6%, 95% CI 4.4% to 7.0%) of all virus culture results were reported within 14 days of specimen collection. The results for 17 out of 21 (80%, 95% CI 56.3% to 94.3%) nasopharyngeal specimens that grew influenza virus became available only a median of 459.1 hours (IQR, 290.1–768.2 hours) after death or discharge. For 33 specimens that grew herpes simplex type 1 virus (another virus for which therapy is available), the results for 26 (78.8%, 95% CI 61.1% to 91.0%) became available only a median of 298.9 hours (IQR 135.95–552.35 hours) after death or discharge – which is outside the therapeutic window for treatment.

FIGURE 17.

Cumulative percentage (± 95% CIs) of nasopharyngeal specimens reported as positive or negative by viral culture at 24-hour intervals after collection.

Blood culture

Blood culture results were available for 973 (77.7%, 95% CI 75.3% to 80.0%) patients in Groups 1, 2 and 3. Eighty of the 973 blood cultures grew an organism (including contaminants), and 10 of the 80 grew S. pneumoniae. The median TtR S. pneumoniae cultures was 84.4 hours (IQR 70.7–137.8 hours), but a provisional report was issued a median of 36.8 hours (IQR 22.7–48.75 hours) after specimen collection. The median TtR bacterial growth for the other 70 specimens was 53.8 hours (IQR 47.7–71.8 hours). The median TtR negative culture results for the remaining 893 specimens was 136.2 hours (IQR 124.4–143.25). Blood culture results for three specimens that grew S. pneumoniae were reported a median of 139.2 hours (IQR 27.9–163.4 hours) after death or discharge. The remaining seven pneumococcal culture results were reported a median of 53.2 hours (IQR 30.4–58.1 hours) before discharge. The median interval between specimen collection and reporting all 973 blood culture findings was 130.8 hours (IQR 123.8–142.8 hours). Altogether 619 of the 973 admissions were discharged or died before their blood culture results, i.e. fewer subjects were discharged after the blood culture result was reported than before.

Antimicrobial sensitivity data became available a median of 84.4 hours after specimen collection for the 10 admissions with positive S. pneumoniae blood culture results.

Sputum culture

As in previous studies, we found that a substantial number (941) (75.2%, 95% CI 72.7% to 77.5%) of the 1252 admissions with acute cardiopulmonary conditions were unable to produce sputum for analysis. Sputum was collected from 311 (24.8%) admissions. Test results were only available for 296 (23.6%, 95% CI 21.3% to 26.1%) sputa owing to rejection of 15 samples by the laboratory for reasons of poor quality. Of the 296 specimens, sputum culture and Gram stain results were available for 274 specimens and sputum culture results were available for only an additional 22.

A total of 76 of the 296 sputum cultures grew an organism (excluding Candida species in five additional specimens); of the 76, 10 grew S. pneumoniae after a median interval from specimen collection of 71.4 hours (IQR 69.15–84.0%). The median turnaround time was 60.7 hours (IQR 50.15–71.6 hours) for the remaining 66 culture-positive sputa (p = 0.0065, Kruskal–Wallis test) and 50.5 hours (IQR 48.3–66.9 hours) for the 219 of 220 culture-negative sputa with relevant data (p < 0.0001). The median interval between specimen collection and reporting sputum culture results for 295 out of 296 admissions was 51.4 hours (IQR 48.75–69.2 hours). Overall, 129 of 294 (43.9%) admissions with relevant data were discharged before their sputum cultures were reported.

The median TtR antimicrobial sensitivity was 133 hours (IQR 123.1–148.1 hours) for 8 of the 10 isolates of S. pneumoniae.

Sputum Gram stain

Gram-staining of sputum samples is not routinely performed by the University Hospitals of Leicester NHS Trust, and the median TtR of Gram stain results for 274 sputum samples was 60.7 hours (IQR 50.0–89.4 hours). Figure 18 shows the cumulative percentage of sputa with Gram stain reports at 12-hour intervals after collection.

FIGURE 18.

Cumulative percentage of 274 sputum samples with Gram stain reports at 12-hour intervals after collection.

Discussion

A prime purpose of POCTs and molecular diagnostic tests is to provide information that enables medical and nursing staff to treat patients with optimal medication as soon as possible, and to inform the use of isolation facilities and infection control procedures and equipment. Such measures can potentially cut illness duration, prevent or ameliorate complications, shorten the duration and costs of hospitalisation, improve the cost-effectiveness of health delivery, and also reduce cross-infection. To be of clinical value, such diagnostic tests must have high sensitivity and specificity across all age ranges, be inexpensive and provide test results sufficiently early for a measurable effect. Here we consider the speed and ease of tests rather than their costs, sensitivity and specificity, clinical impact or cost-effectiveness.

The EoU scores used in this study provide a measure of the complexity and requirements of the test procedures that we assessed. We included TtR of results in the scoring system as a possible indicator of EoU. We did a sensitivity analysis and found that exclusion of TtR of results in the scoring system – either alone or with each other component scores – had no appreciable effect on the overall ranking of tests in terms of EoU and/or test requirements.

The patients in our study had a median duration of illness of 120 hours before admission with a lower quartile of 72 hours. Recent systematic reviews and meta-analyses of the therapeutic use of NIs,70,216,217 show that a clinically beneficial effect of oseltamivir and zanamivir (Relenza^®, GSK) depends on treatment starting within 48 hours of first symptoms when seasonal (inter-pandemic) influenza is circulating. In our study, only the Quidel QuickVUE Influenza A&B test provided results soon enough to influence treatment decisions that might benefit patients – and then only to a small percentage (150/1240, 12.1%) of admissions. In our study, the Quidel QuickVUE Influenza A&B test had a median turnaround time of < 30 minutes, and 85% of results were reported within half an hour of specimen collection. The PCR tests were much slower in comparison – the median TtR of results using molecular tests was 50.8 hours for conventional PCR, and 29.2 hours for real-time RT-PCR. For patient cohorts as in our study, these turnaround times preclude RT-PCR from guiding decisions to start antiviral treatment but could influence infection control procedures in hospitals with limited single-room accommodation, as influenza virus can still be detected in nasopharyngeal aspirates by real-time RT-PCR up to 1 week or more among patients who do not receive antiviral therapy. 125,155 In contrast, conventional virus culture provided results of no clinical or public health relevance to the patient or hospital. In our study, the median TtR of results obtained by virus culture was 452.6 hours (IQR 403.6–620.8 hours) and fewer than 5% of results were available within 14 days of specimen collection – too late to initiate antiviral therapy for influenza or recurrent orolabial herpes simplex virus (HSV) infection. The virus culture results usually became available when the patient had either been discharged or died; this occurred for 80% of the specimens that grew influenza A or B, and 57% of those that grew HSV.

In the EoU evaluation of the diagnostic tests for influenza, the Quidel QuickVUE Influenza A&B POCT was allocated a score of ‘11’ (i.e. component scores of ‘1’ for each of the 11 criteria for categorisation), indicating that it was straightforward and undemanding to use, and can be used with minimum levels of training, knowledge and technical skills. The tests kits can be stored at room temperature and have a 24-month shelf life from date of manufacture, are readily transferable and convenient to use. In contrast, the ‘conventional’ RT-PCR test for influenza A and B and RSV A and B had the highest of all EoU scores in the analysis (‘30’), reflecting its complexity and requirements. In comparison with ‘real-time’ PCR tests, the ‘conventional’ RT-PCR test took 20 hours longer when median turnaround times were compared. The EoU scores for the ‘real-time PCRs’ were ‘25’ and ‘26’, indicating PCRs to be ‘complex’ in terms of EoU and requirements. Our observations of turnaround times and EoU scores indicate that the ‘real-time’ RT-PCR method should be used in preference to the ‘conventional’ RT-PCR test, assuming that both methods are otherwise comparable in terms of sensitivity, specificity and cost. However, it is acknowledged that the technology is evolving rapidly, so what is complex now, with multiple steps, is gradually being automated so as to reduce the skill requirements, and improve turnaround times. The EoU score for the conventional cell culture diagnostic test was ‘26’, which reflects its general complexity and requirements in performance and interpretation. As judged by turnaround times, our findings indicate that consideration is given to replacement of conventional virus culture with an alternative test. However, virus culture remains of value in providing specimens for antigenic analysis and antiviral susceptibility, and guiding strain selection for vaccine production.

In the UK, treatment of ILI with NIs is influenced by illness duration and virus activity established through surveillance, rather than POCTs. Similarly, initial therapy of CAP with antibiotics is guided by severity scores – such as CURB-65 – and cannot await ‘early’ microbiology results, although ‘early’ results might subsequently influence the spectrum of antibiotics that are used.

For pneumococcal infection, only the BinaxNOW S. pneumoniae test provided results rapidly – the test had a median turnaround time of < 30 minutes, and 85% of results were reported within half-an-hour of specimen collection. The BinaxNOW S. pneumoniae POCT had an EoU score of 11 (i.e. component scores of ‘1’ for each of the 11 criteria for categorisation), indicating that it was as equally straightforward and undemanding to use as the Quidel QuickVUE Influenza A&B POCT. Like the QuickVUE Influenza A&B test, the BinaxNOW test can be used with minimum levels of training, knowledge and technical skills, can be stored at room temperature, and is readily transferable and convenient to use. The blood and sputum culture results were much slower in comparison. The median TtR growth of S. pneumoniae was 84.4 hours, although a provisional report of the growth of an organism was reported after a median of 36.8 hours. Antimicrobial sensitivity data did not become available for a median of 84.4 hours after specimen collection. Similarly, growth of S. pneumoniae from sputum was reported a median of 71.4 hours after collection, and the median TtR antimicrobial sensitivity was 133 hours.

As noted in the review of cases that tested positive in the pneumococcal POCT (see Chapter 4 ), the much shorter turnaround time of the urinary antigen test for pneumococcal antigen compared with blood and sputum culture, did not lead to any step-down in antimicrobial treatment within 24 hours of the test result, nor did a positive test result lead to the release any single-room accommodation. Similarly, the substantially faster turnaround time of the POCT for influenza compared with virus culture had no impact on the use of NIs, antibiotics or single-room accommodation but the number of patients with positive results was very small, and the data should therefore be interpreted with caution.

Introduction

This chapter reports the results of a trial-based cost-effectiveness analysis. Resource-use data were collected prospectively during the time patients were in hospital via the CRFs, and retrospectively from discharge until day 28 via the 28-day follow-up. UK unit costs obtained from a variety of sources were then applied to the resource use data to estimate total cost per patient. As well as describing the distribution of total costs for the three diagnostic strategy groups, an incremental cost-effectiveness analysis was undertaken with cost per quality-adjusted life-year (QALY) (estimated using the EQ-5D and mortality data reported in Chapter 4 via an AUC approach218) being estimated when appropriate, together with cost per (correct) case detected (using the diagnostic data reported in Chapter 5 ). To simultaneously allow for the potential for correlation between cost and outcome, as well as their inherent uncertainty, together with the fact that some data were missing for both resource use (and therefore cost) and EQ-5D (see Chapter 4 for details), a Bayesian approach was adopted219–221 using Markov chain Monte Carlo (MCMC) methods, implemented in WinBUGS version 1.4.3 (MRC Biostatistics Unit, Cambridge, UK) to estimate the uncertainty surrounding the outcome measures. 222 A series of sensitivity analyses were undertaken, both to assess the sensitivity to the MCMC methods used, and to the prices of the index tests (PCR and POCT), and also the model structure used to estimate cost-effectiveness. The perspective adopted was that of the NHS.

Methods

Results

Table 33 displays the mean/frequency of resource use for the main constituents of the cost components for patients with no missing data, i.e. complete case analysis. As can be seen, the only statistically significant difference between the three strategies was in terms of use of a practice nurse (p = 0.03). However, this should be interpreted with caution owing to the number of hypothesis tests undertaken. Table 34 displays the mean (and SD) cost per patient for the cost components that make up the overall total cost, as well as the mean overall total cost itself, for each of the three diagnostic strategies. It can be seen that the means for each cost component, as well as for the total, are similar across the three strategies, although there is considerable uncertainty surrounding some of these. Formal comparison of the three strategies using one-way ANOVA did not identify any statistically significant differences between them (p = 0.3). For the observed QALYs, i.e. based on patients who did not have any missing data, POCT produced a mean QALY gain of 0.008137 (SD 0.007191), PCR 0.007724 (SD 0.005977) and traditional laboratory culture 0.007631 (SD 0.006244), which were not statistically significant from one another (p = 0.7).