Cancer of Oesophagus or Gastricus - New Assessment of Technology of Endosonography (COGNATE): report of pragmatic randomised trial

Treatment

Staging and treatment selection

It is clear from both SAGOC4 and NOGCA5 that there is variation in the selection of patients for different treatments. Operation rates in the Scottish audit varied by tumour type: oesophageal cancer 31%; gastro-oesophageal junction tumours 38%; and gastric cancers 51%. There was even greater variation between centres in operation rates: oesophageal cancer 20–42%; junction cancer 25–64%; and gastric cancer 32–63%. 4 The NOGCA report5 showed similar variation in patients selected for curative surgery.

The general criteria for treatment selection are stage of the tumour at presentation, along with patient's fitness, and ability and willingness to undergo specific treatments. Patient fitness depends on comorbid disease. Management decisions depend on the interaction of all of these factors. In addition they increasingly depend on markers of the biological behaviour of a tumour. For gastro-oesophageal cancer these may include tumour differentiation, growth characteristics, response to chemotherapy and molecular mechanisms.

Endoscopic ultrasound

Endoscopic ultrasonography was introduced in the early 1980s. However it became accepted practice only in the 2000s. For example the EUS rate was 3% in SAGOC undertaken in the late 1990s,4 but 48% (gastric) and 58% (oesophageal) in NOGCA5 undertaken some 10 years later. In this review we consider the accuracy of EUS for both gastric and oesophageal cancers, and for both T and N stages.

Gastric cancers

We identified 29 studies, with 2500 patients, that reported on the accuracy of EUS staging for gastric cancer between 1988 and 2009;37–65,21,37–41,43–45,47,48,50–53,56,57,59,62–65 used radial ultrasound probes, three49,58,60 used linear array probes and five42,46,54,55,61 did not report the type of probe (Table 1). As the reported accuracy of linear array probes for both T and N stages did not differ from that of the rest, we include them here.

The pooled accuracy of EUS for gastric T stage was 76.2% (range 35–92%). Accuracy was 71.2% for studies reported before 2000, and slightly but not significant less than 80.4% for studies after 2000. This is consistent with Puli et al. ,36 who reported no difference between studies published in the 1980s, 1990s or 2000s. 36 Their review also compared the sensitivity and specificity of EUS at different T stages of gastric cancer with the pathology from resected specimens in 22 studies with 1900 patients (Table 2).

Thus sensitivity for tumour invasion is high for T1, lower for T2, and then improves as tumours become more advanced. In contrast, specificity is very high for all stages of disease, but highest for T1. Hence if EUS shows T1 disease, the patient probably has anatomical T1 disease. In contrast, if EUS shows T2 disease, the patient may have anatomical T1 disease. So EUS can result in overtreatment, subjecting patients to resectional surgery rather than EMR in the first instance.

The pooled diagnostic accuracy of EUS for nodal staging of gastric cancers was 67.9% (range 42–90%), lower than for T stage as reported in previous studies. 34,36 However it is likely that the use of linear array probes and fine-needle cytology increases that accuracy.

Oesophageal cancers

We identified 40 studies, with 2600 patients, which reported on the accuracy of EUS staging for oesophageal cancer between 1986 and 2009 (Table 4). 48,49,53,59,65,69–103 The pooled accuracy for T stage was 78.5% (range 59–93%) and for N stage 76.3% (range 60–90%). As with gastric cancer, there was no significant evidence that accuracy improved with time. Indeed the studies with the highest diagnostic accuracy were all undertaken before 2000.

In a systematic review of EUS in the staging of gastro-oesophageal cancer, Kelly et al. 34 found that non-traversability of oesophageal cancers and tumours at the gastro-oesophageal junction reduced diagnostic accuracy of EUS staging, but not significantly. In contrast Hordijk et al. 74 found that accuracy was about 90% whether tumours were traversable or not, but fell to 46% for tumours that had been dilated. Accordingly we postulate that differences between studies may arise from the percentage of non-traversable tumours and the method of dealing with these. Two other studies78,85 found a significant reduction in diagnostic accuracy in stenosed oesophageal tumours and suggested that this may be because all the tumours were T3 or T4 with a high rate of lymph node involvement. Kelly et al. 34 also identified junctional tumours as a potential, but not statistically significant, source of diagnostic inaccuracy owing to the difficulty in getting contact between the probe and the tumour surface. As few studies report the exact location of tumour sites, there is uncertainty whether this is a confounding variable.

As with gastric cancer, the accuracy of EUS was better in more advanced oesophageal cancers; specificity is high at all tumour stages, whereas sensitivity increases for T3 and T4 tumours (Table 5). 35 This suggests a slight tendency for EUS also to overestimate oesophageal T stage.

Two meta-analyses35,104 have reported sensitivities above 80% and specificities about 80% for EUS in estimating lymph node involvement in patients with oesophageal cancer. Puli et al. 35 also found four studies that combined EUS with fine-needle aspiration cytology (FNAC) and increased sensitivity to 97% (range 92–99%) and specificity to 95% (range 91–98%). In contrast, Van Vliet et al. 104 found no improvement in accuracy by combining FNAC and EUS. However they did find five studies that analysed results for mediastinal lymph nodes separately from those for coeliac lymph nodes, which had a higher sensitivity of 85% (range 72–99%) and a higher specificity of 96% (range 92–100%). It is likely that these high accuracies for FNAC and coeliac nodes were in specialised centres. The identification of coeliac lymph node involvement may have greater potential to improve management as this suggests metastatic disease in patients with oesophageal cancer, and thus precludes surgery.

Endoscopic ultrasound compared with computed tomography scanning

Endoscopic ultrasound staging consistently has higher diagnostic accuracy than CT staging for both T and N stages (Table 6). Unlike gastric cancer, there is no reported comparison of EUS with the most up-to-date CT techniques. Although there are few studies comparing EUS with MRI or PET scanning, these are not generally more accurate than CT for local regional upper gastrointestinal cancers.

Summary

Many studies have assessed the accuracy of EUS in the staging of gastro-oesophageal cancer. The pooled rates for T stage suggest accuracy of 76% for gastric cancer and 78% for oesophageal cancer; those for N stage suggest accuracy of 68% for gastric cancer and 76% for oesophageal cancer. Furthermore accuracy improves for more advanced tumours. These estimates of accuracy are consistently better than those achieved with other imaging modalities, most often CT scanning. However there is little rigorous evidence as to whether increased accuracy translates into improvements in patient management, still less patient outcomes.

However the management of gastro-oesophageal cancer depends, not only on staging accuracy, but also on patient factors like fitness and willingness for treatment, and treatment factors like benefits and risks. Many patients with gastro-oesophageal cancer have substantial comorbid disease. Often this determines treatment selection irrespective of tumour stage. For other patients the differentiation between T2 and T3 tumours may have little influence on treatment selection, as it is unclear how this affects prognosis or whether treatment should differ between these tumours. So increased accuracy from EUS may be most valuable in discriminating between T3 and T4 tumours and judging whether complete resection is feasible; and between T1 and T2 tumours and judging whether endoluminal treatment is feasible.

Nodal status is another important prognostic indicator for gastro-oesophageal cancer, but it is less clear how this should affect management decisions. Patients with tumours that are N-positive and T3 or T4 will generally fare worse than those with less advanced tumours. Nevertheless we do not know whether and how the outcome for patients with more advanced tumours depends on the choice between curative and palliative treatment.

In short, the known accuracy of EUS in staging gastro-oesophageal cancer makes it important to evaluate whether this staging modality significantly affects the management of gastro-oesophageal cancer. Only a RCT of all patients eligible for EUS can answer that question.

Evaluative paradigm

There has been no formal evaluation of EUS, merely recommendations that it was essential in staging oesophageal cancers. 108 Nevertheless the 2008 NOGCA5 showed that even cancer networks do not universally use EUS to stage oesophageal cancers. Staging non-traversable tumours is difficult;34 the majority are T3 or T4 lesions, which need good staging to avoid non-curative resections. EUS is least accurate in carcinomas around the gastro-oesophageal junction,34 incidence of which is increasing rapidly. Another problem is that there are few studies comparing the cost-effectiveness of EUS with that of modern CT protocols. 34

In summary, although there is evidence that EUS improves anatomical staging of patients with gastro-oesophageal cancer, it is not clear how it affects patient management, even less how it affects patient outcome. SAGOC showed that between 1997 and 2000 few patients with gastro-oesophageal cancer underwent EUS. 4 The subsequent growth in use of EUS in gastro-oesophageal cancer, documented in NOGCA, reinforces the case for evaluating the contribution of EUS to management.

It is also important to study which patients with gastro-oesophageal cancer benefit from EUS. At first sight three types of cancer have the best chance:

1. T1 tumours localised to the mucosa, in which endoscopic treatment may avoid unnecessary surgery.

2. Tumours for which EUS may predict the outcome of ‘curative’ surgery, in particular the risk of residual disease.

3. T3 or T4 tumours in which EUS may encourage multimodal treatment, taking the form of chemotherapy alone, radiotherapy alone, both or neither, depending on clinical circumstances.

To address all these issues comprehensively needs a pragmatic randomised trial that assesses patients by a conventional staging algorithm and then randomises them between EUS and not. In designing such a trial, we started from the seminal writing of Schwartz and Lellouch109 (Table 7).

At a time when randomised trials were much rarer than today, Schwartz and Lellouch prepared the way for ‘health technology assessment’ by distinguishing between two distinct scientific paradigms for clinical trials: ‘fastidious’ trials aim to test defined scientific hypotheses and ‘pragmatic’ trials aim to choose between alternative technologies. 109

In practice, trials that keep to either column of Table 7 are rare. For example the proposal that pragmatic trials need no significance test is feasible only if the protocol specifies how analysis will combine the potential criteria to yield a single decision function, and there is enough information about that function to ensure that the resulting sample size calculation yields the required statistical confidence in the simple decision to choose the technology that performed best on that function in the trial. Since trials rarely fulfil both of these conditions, pragmatic trials usually borrow from the left-hand column of Table 7 by specifying the significance tests that they will undertake.

These far-sighted distinctions influenced the pragmatic design of the COGNATE trial in at least four ways:

1. While fastidious trials mimic the laboratory conditions associated with scientific investigation, pragmatic trials take place in normal clinical practice.

2. While fastidious trials define treatments rigidly, so as to keep hypothesis tests free from external influence, pragmatic trials define treatments flexibly because they seek the best decision for the complexities of normal clinical practice.

3. While fastidious trials seek to equalise placebo or non-specific effects, so as to compare like strictly with like, pragmatic trials seek to optimise these effects as one does in clinical practice

4. While fastidious trials exclude from analysis participants who violate the protocol in any way (‘analysis per protocol’), pragmatic trials seek to analyse all participants according to their allocated treatment whatever subsequently happens (‘analysis by treatment allocated’). 110

The value of EUS in staging patients with gastro-oesophageal cancer is not proven. Hence the only ethical means of evaluating this investigation is a randomised trial. As the funders – the National Institute for Health Research (NIHR) – aim to inform decision-making in the NHS, and EUS was already widespread across the NHS, the trial has to be pragmatic. Accepting these arguments, the Multicentre Research Ethics Committee (MREC) for Scotland approved this pragmatic protocol on 14 June 2004. Thus the scientific validity of the COGNATE trial lies in its adherence to the pragmatic scientific paradigm rather than the fastidious scientific paradigm.

It is intrinsic in the pragmatic scientific paradigm that, after randomisation between alternative interventions (in this trial, alternative diagnostic pathways and their therapeutic sequelae), multidisciplinary teams (MDTs) and individual clinicians make optimal clinical decisions for trial participants. Thus the COGNATE trial is evaluating, not an isolated EUS scan seen as a simple diagnostic intervention, but the ‘complex intervention’111 comprising the entire sequence of clinical decisions that flow from that intervention. In particular, the COGNATE economists seek to cost all the consequences for the use of NHS resources that lie downstream from the focal EUS scan or its absence.

Quality assurance

Little guidance is available for assuring the quality of the clinical processes in clinical trials. Most trials, including COGNATE, follow standard operating procedures (SOPs)112 providing rigorous guidance on the conduct of the trial itself. However there is little if any scientific literature on ensuring the quality of the clinical process that is being tested. As EUS is an operator-dependent skill, it was important to assess the quality of the scanning process within the COGNATE trial.

Variation in the interpretation of scans has three main sources. First there are concerns over the accuracy of EUS scans36 and the learning curve of those who interpret them. 113 Secondly the equipment to record and store images is not consistent between centres. Thirdly analytical interpretation of scans varies among observers and even over time by the same observer. Hence the COGNATE trial aimed to develop and report on a prospective system of peer review to assure the quality of EUS scans. In particular, it reviewed the quality of the reports and recommendations made by reporting clinicians.

Intervention

We designed the COGNATE trial to test the effect on quality-adjusted survival of undergoing EUS within the staging process. Before the trial began, we developed a pragmatic, and therefore advisory, staging algorithm from normal clinical practice as characterised by SAGOC:4

1. All patients should receive biochemistry, haematology, pulmonary function tests and cardiac assessment, not least to exclude patients whose World Health Organization (WHO) status is 3 or 4, or who are medically unsuitable for either surgery or chemotherapy.

2. Patients who are medically fit for surgery without evidence of metastases should undergo CT following an agreed protocol using spiral scanner and intravenous contrast.

3. Patients with any suspicion of peritoneal disease should undergo laparoscopy as the best means of detecting peritoneal tumour deposits.

4. Fit patients with localised tumours and no contraindications were eligible for randomisation to EUS or not.

In the resulting control group (or ‘non-EUS group’ in tables), the choice of treatment depended on the results of the completed initial staging investigations, revisited if necessary. In the resulting intervention group (or ‘EUS group’ in tables), the final choice of treatment followed the EUS scan. At the end of staging, with or without EUS, MDTs assigned patients to one of three treatment options. Patients with:

1. tumours that were adjudged to be mucosal underwent EMR with or without argon-beam ablation of the surrounding mucosa

2. tumours that were adjudged to be resectable underwent surgery with or without neo-adjuvant chemotherapy, typically with cisplatin and 5FU

3. advanced localised disease, for which complete resection was adjudged to be impossible, received multimodal treatment, possibly including palliative surgery for gastric cancers.

Thus we randomised no patients who then had evidence of metastases or then had plans for palliative treatment or were then known to be medically unfit for surgery. In a pragmatic trial, of course, subsequent changes in all this information cannot invalidate the randomisation.

Trial flow chart

Figures 1 and 2 summarise the trial design. Randomisation took place after review of the initial staging investigations by the MDT. Clinicians agreed a conditional management plan, sought informed consent and randomised patients between receiving EUS and proceeding to the agreed management plan. They also reported to the North Wales Organisation for Randomised Trials in Health (NWORTH), Bangor University's Registered Clinical Trial Unit, all patients whom the MDT decided not to randomise with reasons for exclusion. These included patients for whom they considered EUS either essential or inappropriate.

FIGURE 1.

Trial design: start to randomisation. MDM, multidisciplinary meeting. PROM, patiented-reported outcome measure.

FIGURE 2.

Trial design: randomisation to conclusion. PROM, patient-reported outcome measure.

Inclusion and exclusion criteria

The COGNATE trial was a trial of patients with proven cancer of the oesophagus, stomach or gastro-oesophageal junction. To be eligible for the trial, patients had to be fit for both surgery and chemoradiotherapy as well as free of metastatic disease. Both their ASA (American Society of Anesthesiologists) grading114 and their WHO performance status115 had to be 1 or 2 (see Figure 1). Following initial staging, clinicians could exclude patients from the trial for clinical reasons.

Patient information and informed consent

Before randomisation, the research professionals or clinicians invited eligible patients to participate in the COGNATE trial, gave them the patient information sheet approved by the Scotland MREC, and allowed them time to consider it and ask questions. We explained the nature of EUS and the process of randomisation to these patients. We stressed that the choice of treatment after EUS was the same in both groups. We then asked consenting patients to sign the consent form.

Randomisation

Once an eligible patient had consented and completed the baseline quality-of-life questionnaire, the recruiting centre telephoned NWORTH in Bangor. NWORTH staff confirmed eligibility and asked for information on both stratifying variables: centre and tumour location – gastric, oesophageal or the gastro-oesophageal junction. As we included only participants with good WHO performance status, we did not need to stratify for this. NWORTH then randomised the participant between intervention and control groups using a dynamic randomisation algorithm designed to prevent subversion. 116,117 We reported regularly on recruitment to the National Clinical Research Network and NCCHTA. NWORTH confirmed the allocation by e-mail to the recruiting centre, which either booked an EUS scan for intervention participants or continued the agreed management plan for control participants.

Sample size

Our original application proposed to consent, randomise and follow up a total of 700 patients in a trial in which the primary outcome was survival. We soon discovered that most centres in the UK wanted to use EUS to stage gastro-oesophageal cancer. Conscious that early participants were able and happy to report on their health-related quality of life (HRQoL), however, we decided with the support of the Trial Steering Committee (TSC) and the approval of the Health Technology Assessment (HTA) programme to change the primary outcome to quality-adjusted survival. This effectively combines the components of health that EUS might improve – survival through better staging and HRQoL through reassurance arising from better staging and planning – in principle reducing the sample size needed.

As there is no easy means of calculating the power of a sample for the primary outcome of quality-adjusted survival, we calculated power for two simple but plausible scenarios. First, if there were no difference between groups in quality of life, the combination of a sample of 400 participants and a log-rank test using a 5% significance level would yield 80% power of detecting a hazard ratio of 0.6, equivalent to a difference between 60% survival at 12 months [derived from SAGOC:4Appendix 1 and Figure 2] and 73% survival. Second, if there were no difference between groups in survival, the combination of the sample of 400 and a t-test using a 5% significance level would yield more than 80% power of detecting a ‘small’ effect size of 0.3118 in quality of life. As the groups were more likely to differ in both survival and quality of life, the power of our primary analysis of quality-adjusted survival would be correspondingly greater. At worst, if we were able to randomise and follow-up only 220 patients, that scenario would yield 80% power to detect a hazard ratio of 0.5 (equivalent to a difference between 60% and 78% in survival at 12 months) or an effect size of 0.4 in quality of life, still ‘small’. 119

Quality-of-life instruments

The COGNATE trial used two instruments to gather information on quality of life as the basis for evaluating both effectiveness and cost-effectiveness: the European Quality of Life – 5 Dimensions (EQ-5D) and its visual analogue scale (EQ-VAS); and the Functional Assessment of Cancer Therapy (FACT) comprising FACT – General (FACT-G) and FACT – Additional Concerns (FACT-AC). Centres administered the questionnaires, including these instruments at baseline and 1, 3, 6 and 12 months after randomisation and at 18, 24 and 36 months where possible.

We used the EQ-5D, developed by the EuroQol Group,120,121 to measure patients’ HRQoL and to ascribe utilities to their health states. The EQ-5D is a preference-based generic measure comprising five domains: mobility, self-care, usual activities, pain and discomfort, and anxiety and depression. Each domain has three levels: no problems, some problems and a lot of problems. The EQ-5D scoring system defines 245 possible health states, namely 3 × 3 × 3 × 3 × 3, plus two additional states – dead and unconscious. EQ-5D gives death a utility of zero and ‘best imaginable health’ a utility of one. For each participant it converts the five item scores into a summary utility based on the ‘time trade-off’ preferences of a UK-wide random sample of 3000 respondents. 122 Some health states have negative utility (‘worse than death’). We included the EQ-5D in our outcome portfolio as the primary means of adjusting survival for quality of life. EQ-5D complements its five items with a visual analogue scale (VAS); this is a single thermometer-like generic quality-of-life scale with zero representing ‘worst imaginable health’ and 100 representing ‘best imaginable health’, on which respondents mark their perceived current health directly. Scores, therefore, require no further processing.

Functional Assessment of Cancer Therapy is a psychometric instrument measuring cancer-specific quality of life. 123,124 The current version of FACT-G comprises four subscales: Physical Well-Being (seven items), Social or Family Well-Being (seven items), Emotional Well-Being (six items) and Functional Well-Being (seven items). FACT sums scores on these subscales to give FACT-G. The COGNATE team derived its FACT-AC scale from Gastric Additional Concerns,125 comprising 19 items, and Oesophageal Additional Concerns,126 comprising 17 items, by removing overlapping items and psychometrically weak items using methods described by Streiner and Norman. 119 In this way we effectively merged the Gastric Additional Concerns and Oesophageal Additional Concerns scales to form a single integrated Gastro-Oesophageal Concerns scale for easy use by all trial patients. We also assessed the extent to which this provided information over and above that provided by EQ-5D.

Data collection

Centres collected data on the due day when possible, but otherwise within a window. Although pre-randomisation data could be collected up to 3 days before randomisation, randomisation could not proceed without these data. Data due at 1, 3 and 6 months could be collected up to 14 days after the due date; data due at 12, 18, 24 and 36 months could be collected up to 28 days after the due date. To avoid bias, for example by anticipating a major event, we did not collect data before the due date except for pre-randomisation data. Similarly, we did not collect data for 7 days after a major procedure.

Our preferred mode of administration was a rigorously defined face-to-face interview. In our experience interviews reduce biases due to sicker patients not responding. Trained research professionals read each question while participants followed laminated versions. The researchers entered their responses directly into a laptop computer. If a face-to-face interview was not possible, we conducted the interview by telephone, having posted the questionnaire to the participant in advance. As a last resort, the participant could complete the questionnaire and return it by post; with that exception, research nurses were interviewers not observers. The researchers also recorded the mode of completion: face to face, telephone or postal. Although we know that interviewers affect responses to questionnaires, there is strong evidence [from a recent systematic review to which two of us (DKI, ITR) contributed] that this effect is consistent across trial arms. 127 Furthermore FACT itself is robust against interviewer effects. 128

Primary outcome measure: quality-adjusted survival

The primary outcome measure was quality-adjusted survival, using the EQ-5D health index to adjust for the quality of life of survivors. We integrated outcomes over time for individuals by calculating the ‘area under the curve’ (AUC). This avoids multiple testing of correlated outcomes. We calculated this area from the graph of HRQoL (EQ-5D or FACT) against time by joining all the intermediate points derived from the follow-up interviews, drawing vertical lines to the horizontal (time) axis at randomisation and at death, complete withdrawal or censoring at the end of the trial (August 2009), whichever occurred soonest, and then calculating the area of the resulting polygon. This area is the standard measure of quality-adjusted life-years (QALYs) in cost-utility analysis. 129

Secondary outcome measures

Follow-up

We followed patients until death or the end of data collection, which was between 12 and 58 months after the end of recruitment for all patients. We collected data at discharge from hospital after initial treatment and at follow-up clinics after 1, 3, 6, 12, 18, 24 and 36 months.

Quality assurance of endoscopic ultrasound scans

The COGNATE team asked investigators to record EUS scans on videotapes and to record staging and explanatory comments on a trial proforma. Anonymised videos of selected EUS scans of intervention participants were reviewed by a panel of investigators during a series of web conferences, each of which reached a blinded consensus on the staging of four tumours. We compared the staging decisions of the individual peer reviewers, their consensus decision and the staging decision of an international expert as external reviewer with that of the original investigator who performed the scan. We used Cohen's kappa or weighted kappa130 to test for agreement.

Independent trial monitoring

The COGNATE trial had a TSC comprising an independent chairperson, two independent members and four members of the COGNATE Trial Executive Group (TEG). Principal investigators (PIs) in each centre reported to the TSC through the TEG. The Data Monitoring and Ethics Committee (DMEC), comprising three independent members with the trial statistician in attendance, acted as a subcommittee of the TSC, reporting to the TSC. Appendix 3 lists members of the TSC and DMEC.

Trial management

North Wales Organisation for Randomised Trials in Health managed the trial. A Trial Management Group comprising the two chief investigators, trial statistician, health economist, outcomes specialist, data manager, trial manager and trial administrator met monthly at NWORTH. Telephone access was available to other investigators by invitation. Minutes were taken and stored in the Trial Master File.

Ethics and participating sites

The COGNATE trial was approved by the Scotland MREC A and 10 Local Research Ethics Committees (LRECs) and associated research governance units for a total of 16 hospital sites. Two centres did not recruit any patients, one owing to competing workload and the other (who had obtained ethical approval from four LRECs for seven hospitals in that area) owing to lack of commitment from staff. Appendix 2 gives details of the eight recruiting hospitals.

One centre screened many patients but randomised only one into the COGNATE trial, and later asked to withdraw from the study. They agreed that their COGNATE patient, if willing, could receive continuing follow-up through a nurse from the local Cancer Research Network. This arrangement was accepted by the patient and the local Research Governance Department.

Protocol amendment

The proposal to change the primary outcome measure from survival to the more sensitive measure of quality-adjusted survival, which reduced the target sample size from 700 to a maximum of 400 and a minimum of 220, was agreed by the DMEC and TSC in April 2006 and reported to the NCCHTA in December 2006. A 6-month, no-cost extension was agreed by the DMEC and TSC in April 2007 and by the NCCHTA in November 2007. The final protocol is in Appendix 1.

Electronic data collection and storage

All data identified patients only by a unique trial number. Each trial centre kept its own index linking trial numbers to patients' names and addresses separate from the laptop computers used to store and transfer trial data, and protected that index by key and password. The trial co-ordinating centre in Bangor had no access to these local indices. Those analysing data had no access to which group of participants received the intervention until the TSC had scrutinised and approved the methods and findings of the main statistical analysis.

North Wales Organisation for Randomised Trials in Health designed a Microsoft Access database (Microsoft Corporation, Redmond, WA, USA) to enable each centre to collect COGNATE data on their trial laptop computer. The electronic forms for data entry were designed to be friendly to local staff. When local staff suggested an improvement to display the dates of tests already entered into the database, the database designer programmed the change and the trial manager implemented it on the next suitable occasion – monitoring visit, investigators’ meeting or over the telephone.

Centres transferred data to NWORTH by electronic file transfer protocols every 2 months throughout the trial. This allowed the data manager and trial manager to monitor data completeness and report to the research team. As Bangor University updated file transfer protocols twice during the COGNATE trial, this necessitated retraining of local staff in methods of data transfer. By all of these means the COGNATE trial can fairly claim to have been a generally paperless trial.

Statistical analysis plan

To avoid bias, we wrote our statistical analysis plan, and the TSC approved it, during the recruitment phase. Although participants and their clinicians knew their randomised allocation, we kept the methodological chief investigator and trial statistician blind to all these allocations until they had presented blinded findings to the TSC in September 2009.

Primary analysis was by ‘whether allocated to endoscopic ultrasound’. 109,110 This reflects the pragmatic nature of the trial, and its primary goal of evaluating that health technology in informing decisions in the real world. We also undertook secondary analysis by ‘endoscopic ultrasound received’ to explore the implications of clinical decisions that diverged from the random allocation.

Quality-adjusted survival (primary outcome) and survival

The primary outcome measure (survival adjusted by EQ-5D), and survival-based secondary outcome measures (unadjusted survival, survival adjusted by FACT-G, and survival adjusted by FACT-AC) all analysed the time between randomisation and the end of the trial or death (if it occurred before the end of the trial). Thus surviving participants recruited at the end of the trial were censored at 12 months, while those recruited near the beginning of the trial were censored only if they survived 48 months or more.

Because of the variable follow-up, applying standard analysis of variance methods or t-tests to these measures is biased against the group with better survival. There are a range of survival analysis methods, all of which allow for censored observations and thus avoid this bias. For descriptive comparisons, we used Kaplan–Meier estimates of mean and median length of survival as numerical summaries, survival curves as graphical summaries, and log-rank tests to compare the survival experience of different groups. However we used Cox regression, which models the simultaneous effect on survival of several characteristics, for the main (quality-adjusted) survival comparisons.

Both primary quality-adjusted survival analysis and secondary survival analyses using Cox Regression models considered several baseline characteristics, including EQ-5D and FACT baseline scores, for inclusion as covariates. We did this, not only to take account of any baseline imbalance between groups despite stratification, but also to improve the precision and generalisability of the model. We always included centre, condensed to three groups of similar size: Aberdeen, Gloucester and the rest. We always used the baseline score of a given measure to predict a later score of that measure. Other characteristics considered in step-wise model building included: age and gender; site, stage and type of tumour; the initial management plan agreed before randomisation; but not WHO status, as most participants had a WHO status of 1. To get the best from ‘initial management plan’ in predicting outcomes, we created a binary variable to distinguish between conservative prior plans (namely chemotherapy, radiotherapy, both or neither) and therapeutic prior plans (namely endoscopic resection or surgery in some form). As we had expected, this later proved very good at predicting outcomes. As conservative plans choose between all possible combinations of chemotherapy and radiotherapy, we followed the example of many MDTs by describing this and the resulting binary variable as ‘multimodal’.

Quality of life at 12 months: sensitivity analysis of primary outcome

We analysed EQ-5D, FACT-G and FACT-AC scores at 12 months by general linear models, using the same approach to covariates as in survival analyses. Although all follow-up times contributed to the EQ-5D- or FACT-adjusted survival analyses, we reported these three measures, and the four FACT subscales, at all follow-up times only descriptively to reduce multiple testing. As in the Cox regressions, we supplemented model-based analyses with basic summaries and graphs.

For cost-effectiveness analysis, we extended the primary QALYs measure beyond the end of the trial to 48 months for all participants by a combination of survival modelling and imputation. When all censored individuals have the same follow-up time, survival analysis methods are no longer needed. Hence this fully imputed measure, and the equivalent truncated to the minimum follow-up of 12 months, were analysed in the same way as the quality-of-life measures at 12 months. This provides both a sensitivity analysis for the main effectiveness measure and a link to the cost-effectiveness analysis.

Other outcome measures

We compared changes in management plan, complete resection rates and cause of death between groups and centres by chi-squared tests of the appropriate proportions. In response to referees, we also investigated the relationship between changes in management plans and selected other outcomes (survival, quality of life, cost and complete resection) for the two allocated groups.

Modelling: covariates and interactions

We used analysis of covariance (ANCOVA) in general, and Cox regression models specifically for survival and quality-adjusted survival, to improve comparisons between intervention and control groups. These techniques enhance the basic analytical techniques – t-tests and log-rank tests respectively – by making allowance for covariates, namely participant characteristics that affect outcome. Most covariates have similar effects on intervention and control groups, therefore including them in models corrects for baseline imbalances if present, and thus improves estimates of the effect of EUS. Whether or not there are any baseline imbalances, covariates can also explain some of the intrinsic uncertainty and thus reduce it.

In choosing covariates we followed the analysis plan that we had defined prospectively and the independent TSC had approved, also prospectively. This plan considered as potential covariates only the pre-randomisation variables listed above [see Quality-adjusted survival (primary outcome) and survival]. First we entered centre and baseline values of the outcome under analysis. Secondly we sought covariates that affect that outcome directly; to avoid subjective choices, we used a step-wise procedure and included only covariates that increased the precision of the estimated effect of EUS.

However covariates may also interact with the effect of EUS in the sense that changing the value of the covariate changes the estimated difference between an intervention participant and a control participant with the same characteristics. Thus an interaction indicates which participants derive the most benefit from EUS. The first two analytical steps we have just described used covariates without interactions. The third step added interactions between the treatment allocated and the covariates chosen in the previous two steps if they improved the model significantly. In the only change to this analysis plan after the TSC had approved it, we also investigated whether or not the unexpected interaction between the effect of EUS on survival adjusted by EQ-5D and the baseline EQ-5D also applied to survival adjusted by FACT and the baseline EQ-5D.

Imputation and missing values

To avoid bias in analysis by treatment allocated, it is desirable not to exclude participants for whom some outcome data are missing;129,131 whenever possible, therefore, we imputed these data from known data about these participants and other participants whose outcome data are known. 132,133

The main trial recruited for 3.5 years. To maximise statistical power, therefore, we followed all participants for as long as possible, namely between 1 and 4.5 years. The main aim of imputation for the effectiveness analysis was to achieve complete data within this design rather than to extend data beyond 31 July 2009, the end of the trial and thus the censoring date. However the bootstrap methods used to assess cost-effectiveness required us to estimate costs and benefits for all participants for the same period. We chose two follow-up times: 12 months, the minimum unless a participant withdrew from the trial; and 48 months, which took account of all information on both survival (as there were no subsequent deaths) and quality of life (as the last questionnaire to participants was at 36 months). Sensitivity analyses of both effectiveness and cost-effectiveness provide explicit links between the effectiveness and cost-effectiveness sections of this report.

Potentially there was complete information for all participants who died during follow-up. Dead participants did not use any more resources, and we set their subsequent quality of life to zero, rather than missing. We also received survival data on all participants up to complete withdrawal or the end of the trial. As survival analysis allows for variable follow-up, it does not need imputation.

Data were of two types:

1. Clinical data, including demographic and resource use, abstracted by research professionals from hospital notes and entered retrospectively onto the electronic database. In general we did not need to impute these data, because we asked research professionals to collect complete data except for pre-randomisation tumour stage, for which we permitted ‘missing’. However we imputed resource use in secondary care from the end of the trial to 48 months.

2. Patient-reported outcome measures at baseline and follow-up. We imputed the few missing data for the main effectiveness analysis. For the cost-effectiveness analysis and effectiveness sensitivity analysis, we needed to impute quality of life to 48 months.

We imputed missing quality-of-life data in three phases. Each phase used SPSS (SPSS Inc., Chicago, IL, USA) Missing Values Analysis (MVA) procedure,132 which simultaneously estimates all missing values in a data set, on one or more data sets. The final phase also used estimated survival probabilities from the Cox regression model that we have described.

Rationale

We assessed quality of life by the EuroQol120,121 and FACT123,124 (see Design, Quality of life instruments). We chose the EQ-5D as the primary means of adjusting survival for quality of life. We included the EQ-VAS as a natural adjunct to the EQ-5D. We added FACT as an alternative means of adjusting survival, conditional on psychometric analysis within the trial. When the primary outcome measure changed from survival to quality-adjusted survival during the trial, thorough quality-of-life assessment became even more crucial.

The standard way to score FACT data is as follows. Each subscale is the sum of its items. FACT-G is the sum of Physical, Social, Emotional and Functional subscales. FACT Total is the sum of FACT-G and Additional Concerns. FACT Trial Outcome Index (TOI) is the sum of Physical, Functional and Additional Concerns. The rationale for the TOI is:

It is a common endpoint used in clinical trials, because it is responsive to change in physical or functional outcomes. While social and emotional well-being are very important to quality of life, they are not as likely to change as quickly over time or in response to therapy. 134

To assess the best use of FACT in the COGNATE trial, we posed three research questions and conducted psychometric analyses separately on the data collected before randomisation, and at 1 and 3 months, when enough individuals attempted the questionnaires to make the analyses feasible.

Questions and corresponding analyses

How to score FACT-G?

The scoring method of FACT-G – summing item scores to produce four subscale scores that can be summed into a single score – implies four intercorrelated dimensions. Previous factor analyses have typically found such a structure. 136 However we felt it prudent to check the factor structure of the FACT-G items and to consider the implications for scoring. We achieved this by subjecting the FACT-G items, that is to say the Physical, Functional, Social, and Emotional subscales, to confirmatory factor analyses with maximum likelihood estimation, using AMOS™ version 16137 (SPSS Inc., Chicago, IL, USA). The factors were free to correlate. For each model, we considered the global fit, the residual covariances, the loadings of items on their intended factors, the modification indices for the loadings of items on their non-intended factors, and the correlations between factors. We measured global fit by the comparative fit index (CFI), root-mean-square error of approximation (RMSEA) and standardised root-mean-square residual (SRMR). Values of CFI above 0.9, SRMR below 0.10 and RMSEA below 0.08 represent adequate fit. 138 Ideally, we sought a significance level of at least 0.05 for the RMSEA test of close fit, where close fit is defined as an RMSEA value no greater than 0.05. We modified models only when there was both statistical and theoretical justification for so doing.

What FACT summary measure to use in COGNATE?

Although we included FACT as an alternative means of adjusting survival, it was not clear a priori which summary score would be best – General, Total or TOI. To make an informed choice, we examined relationships among the subscales and with the EuroQol measures using structural equation modelling with AMOS. We modelled the EQ-5D construct as an observed variable, and the FACT constructs as latent variables. It was not possible to use multiple indicators for these latent variables, given the size of the model and the size of the sample. Therefore we used an alternative means of adjusting for measurement error. 139 For each FACT latent variable there was one indicator, the corresponding FACT subscale. We fixed the path from the latent variable to the indicator at 1, and the measurement error at the variance of the indicator multiplied by 1 minus the reliability of the indicator. In the initial model, Additional Concerns was free to influence each of four FACT-G constructs, and each of the FACT-G constructs was free to influence EQ-5D, but the Additional Concerns construct was not free to influence EQ-5D directly. We allowed the disturbances of the four FACT-G constructs to co-vary. The criteria for adequate fit were as for the confirmatory factor analysis. We modified these models only when there was both statistical and theoretical justification for so doing. Finally we deleted non-significant paths and re-ran the model.

Introduction

The primary objective of the health economic analysis was to assess the cost-effectiveness of EUS staging in the management and treatment of patients with gastro-oesophageal cancer compared with its absence. Our reporting of economic analysis in COGNATE is consistent with that in the Multi-Institution Nurse Endoscopy Trial (MINuET),140 our published SOP for economic evaluation alongside RCTs,141 and published guidance. 129,142

Existing economic evidence

We found five relevant economic studies on staging cancer of the oesophagus. 143–147 Two were review articles143,144and three were American decision-analytic modelling studies. 145–147 Of these, Wallace et al. 147 is the most robust, arguing that CT, EUS and fine-needle aspiration (FNA) was the least costly strategy (US$40,000) and offered more QALYs on average (0.965) than all other strategies with the exception of PET, EUS and FNA (US$45,000 for 1.034 QALYs). Thus the latter was slightly more effective but also more expensive, yielding a marginal cost-effectiveness ratio of US$61,000 per QALY, less than that of many medical treatments but above accepted thresholds in the USA and UK. We did not find any National Institute for Health and Care Excellence (NICE) guidance on this topic.

Outcome measures

The outcome measure for the economic analysis was the QALY. Both NICE and NCCHTA support the use of QALYs as an outcome measure in technology assessment. 140,148 QALYs measure health gain, combining survival and HRQoL. 149 The COGNATE trial measured HRQoL by the EQ-5D questionnaire. 120,121 We calculated the difference in mean QALYs between the intervention and control groups.

Analysis by time intervals

As participants were in the trial for different lengths of time, we adjusted estimates of costs and QALYs to allow for censoring. Following Lin et al. ,133 we used five time intervals covering 4 years (see Statistical methods). In the first interval (up to 12 months) both cost and QALY data were uncensored in principle. As we expected, the majority of costs fell during that first year. Hence we decided to conduct economic analysis over two different periods – for 12 months after randomisation and for 48 months, essentially the whole study period (see Statistical methods). In both we used bootstrapping, first to derive confidence intervals (CIs) around our point estimates of incremental cost-effectiveness ratios (ICERs) when appropriate, and then to draw cost-effectiveness acceptability curves (CEACs) to quantify uncertainty and convey to policy-makers the probability that EUS is cost-effective at different thresholds. In our primary analysis at 48 months, we discounted costs and QALYs at 3.5% per year. 148 We undertook sensitivity analyses at 12 and 48 months by varying the costs of EUS. We also re-ran our analysis using undiscounted QALYs at 48 months, again varying the costs of EUS. We undertook exploratory subgroup analyses at 48 months to test the effect of baseline EQ-5D and age on our results.

Resource use in secondary care

The COGNATE trial measured each participant’s type and frequency of contacts with NHS secondary care over 4 years – contacts very likely to account for almost all their direct NHS costs. Within cancer care we focused on the EUS procedure, surgery, chemotherapy, radiotherapy, other drugs, inpatient stays, and day care and outpatient appointments. We integrated the collection of all these data on resource use into this essentially paperless trial. In particular we designed an electronic version of the Client Service Receipt Inventory (CSRI)150,151 – a trial-specific structured form that enables research staff to report on the type and frequency of participants' contacts with health care. We took special care to check and maintain the quality of these data, unfamiliar to many trial practitioners. This led to a complete set of secondary care use data for the 213 participants who contributed to our analysis of clinical effectiveness and cost-effectiveness.

We excluded primary care costs for two reasons. First, the COGNATE team, including surgeons, economists and other health services researchers, were confident from our combined experience that these would account for a very small proportion of direct NHS costs. Second, although we initially tried to collect primary care data with quality-of-life data during structured interviews in secondary care, practical constraints led us to collect most of these primary care data by post. Subsequent rigorous validation established that the resulting primary care data, unlike the simpler quality-of-life data, were not good enough for robust imputation. After the unsuccessful validation, we followed the principles of Good Clinical Practice (GCP) by discarding the flawed primary care data, and changed the perspective of our economic analysis to secondary care.

As resource use is generally skewed, our published SOP for economic analysis advocates converting resources to costs, which are commensurable and facilitate analysis of uncertainty. 141 As distributions of resources and costs are especially skewed in cancer care, we always planned to impute censored data and bootstrap skewed data, rather than adopt the inappropriate method of adjusting observed costs by ANCOVA. Moreover, if COGNATE were to find significant differences in survival, and thus follow-up time, between allocated treatments, any simple comparison of observed costs would be biased.

Unit costs

Our general approach to costing all these data on resource use was to apply NHS reference costs for 2008152 and drug costs from the Prescription Cost Analysis 2008 England (PCA). 153 We undertook detailed costing with oncologists, radiologists, surgeons and others at COGNATE sites for high-cost items such as multimodal treatment, neo-adjuvant therapy, surgery, chemotherapy, radiotherapy and positron emission tomography (PET) scanning. We derived the cost of a stent from Shenfine et al. ,154 and inflated it to 2008 prices using the Hospital and Community Health Services Pay and Price Index. 155

We undertook detailed costing of the EUS procedure through a time-and-motion approach and discussion with oncologists, radiologists and surgeons at trial sites. We included variation between local and national estimates as part of our sensitivity analysis. Table 8 summarises the sources of data on resource use and unit costs for secondary care.

Cost-effectiveness analysis

Our analysis of cost-effectiveness took an NHS perspective and compared effects by treatment allocated. As the trial randomised participants, we compared their costs between intervention and control groups. Cost-effectiveness analysis calculates ICERs, calculated as (C2–C1)/(E2–E1), where C2 is the mean total cost of the intervention group (EUS staging), C1 is the mean total cost of the control group (clinical management as usual), E2 is the mean effect in the intervention group and E1 is the mean effect in the control group. However interpretation of ICERs that cover more than one quadrant of the cost-effectiveness plane requires caution. In particular negative ICERs, which describe situations where one group (either intervention or control) is both less costly and more effective than the other, are not useful. As cost gains and effect gains are not competing but combining, their ratio is no longer relevant.

One alternative is the net monetary benefit (NMB) equal to the product of net QALYs and (WTP minus cost), where WTP is the decision-makers' maximum willingness to pay for a QALY. 129 The main disadvantages are that interpretation is difficult without separate knowledge of the costs and that the NMB for one condition cannot easily be related to NMBs for other conditions or to the NICE threshold. Instead we present CEACs, which combine the wider applicability of NMBs with a more user-friendly presentation. The CEAC is a graphical representation of the probability that an intervention is cost-effective over a range of monetary values for decision-makers' willingness to pay for a QALY. A value of zero for a QALY leads to comparison of the costs of EUS staging and usual care alone, and infinite or very large values result in a comparison of QALYs alone.

For COGNATE we use the values £20,000 and £30,000 – the range of threshold values used by NICE in the UK. 148 This enables policy-makers to compare the cost-effectiveness of EUS in gastro-oesophageal cancer with the range of estimated health gains per ‘NHS pound’ in other cancers and conditions.

We used bootstrapping to generate cost-effectiveness planes, CEACs and, where appropriate, 95% confidence intervals around our point estimates of ICERs. 156 Bootstrapping draws repeated samples from the trial data set with replacement. It calculates mean costs and effects in each group for each sample, and estimates the incremental cost-effectiveness of the intervention. We used Stata version 10.1 (StataCorp LP, College Station, TX, USA) to perform this non-parametric bootstrapping by generating 5000 samples.

Imputation, censoring, discounting and sensitivity analysis

In Statistical methods, Imputation and missing values we described the methods we used to impute missing cost and QALY data, and to allow for the censoring caused by variable follow-up in the COGNATE trial. We discounted all costs and QALYs beyond 12 months at 3.5% per year, as recommended by NICE. 148

We conducted sensitivity analyses at both 12 months and 48 months to explore whether, and to what extent, the estimated ICERs for EUS relative to conventional staging are robust to key assumptions. There was uncertainty about the cost of EUS between local and national tariffs and estimates from the literature, so we explored the implications of the three different estimates we obtained. We also present undiscounted QALYs as another sensitivity analysis, not least for consistency with the effectiveness analysis.

Our analyses at 48 months include two subgroup analyses. First, we estimated CEACs for participants whose baseline EQ-5Ds were below or above the median score of the whole sample. Second, we estimated CEACs for participants whose ages at randomisation were below or above 65 years.

Objective

To develop a rigorous process of peer review to assure the quality of EUS scanning in a trial to evaluate the use of that technology in staging and managing gastro-oesophageal cancer.

Methods

The COGNATE site investigators reported the EUS scans of all participants in the intervention group by completing an anonymisable form (see Appendix 8.1) specifying tumour location and tumour, nodal and metastatic (TNM) stages. Our quality assurance panel comprised: an experienced surgical endosonographer and trialist as independent chair; six assessors from six sites – four radiologists, one gastroenterologist and one surgeon – who performed scans within the trial; with trial staff, notably the information technology specialist, in support.

The COGNATE site investigators recorded scans for peer review on to videotapes or compact discs (CDs) using trial number as sole identifier. They posted them by recorded delivery to the COGNATE team, who digitised and anonymised them, and stored them on the Bangor University website. The panel reviewed sampled scans at planned web conferences. One month before each conference we gave the panel links to sampled videos and asked them to study those videos on our website. When NHS firewalls denied access to the website, we posted them to members on CDs by recorded delivery. For each scan we asked members to complete and return a numbered assessment form (see Appendix 8.2) covering: the quality of the recording; their estimate of T, N and M stages; and the specific video times that informed those estimates. Trial staff collated results before each web conference.

Our IT specialist organised the conferences through www.webex.co.uk and transmitted videos and related documents to the office computers of all panel members. We recorded discussion both on audiotape and as text in the ‘chat box’ on the screen. If broadband problems affected the conference, the panel viewed CDs contemporaneously. Thus the panel viewed scans together but discussed staging blind, i.e. without knowing the source of the scan, the original assessment or those of fellow assessors. Once they had reached consensus on TNM stages, they could compare that consensus with the original assessment and review it if necessary. They also assessed the general quality of each scan and recording (Figure 3).

FIGURE 3.

Process of quality assurance. BU, Bangor University.

Conscious of the danger that the panel, although blind to the source of the scans they were assessing, were favourably disposed to those scans, we recruited an experienced endosonographer from the Commonwealth as external assessor. We compared assessments between the original operator, panel members, the panel consensus and the external assessor, in principle by Cohen's kappa or weighted kappa. 130 Finally, for participants in the intervention group who had resection without pre-treatment, we compared EUS findings with pathological findings at operation.

Recruitment

Identification and eligibility

Recruitment started in Aberdeen and Gloucester in September 2004, became fully established on 1 February 2005, and finished on 31 July 2008. Eight centres participated. The COGNATE team identified 1152 potential participants and recruited 223 (19.4%) to the trial (Figure 4, elaborated in Appendix 4.1).

FIGURE 4.

CONSORT flow chart to point of randomisation.

The most common reason for exclusion was that patients were ‘not medically fit for surgery’ (n = 383). This reflects the physiological fitness of the patient, not the suitability of the tumour for surgery. Although only eight exclusions cited WHO status, many excluded as ‘not medically fit for surgery’ also had WHO status 3 or 4 so we combined these two categories. About half of those identified (n = 567) were initially adjudged eligible, given information about the trial and asked for consent. Of these 90 (15.9%) refused immediately; we sought no further details for them. The remaining 477 were discussed as potential participants at a multidisciplinary meeting (MDM) (see Figure 1), after which centres identified further exclusions. These included 24 further refusals, some of whom withdrew previous consent. Thus 114 people (9.9% of those identified; 20.0% of those potentially eligible) refused consent to the trial, leaving 453 potentially eligible.

Centres excluded a further 228 people after the MDM – a larger proportion than first predicted. We grouped reasons in free text into categories. Centres excluded 120 people because they regarded EUS as essential; for half of these, no reason was given. In contrast, only eight people apparently did not need an EUS. Another 36 patients had already received EUS or a booking for it. In all, we randomised 225 participants. Subsequent validity checks established that two were in error; we removed them from analysis, leaving 223.

Eligibility by centre

Differences between centres in numbers of patients identified, proportions recruited, and reasons for exclusion were highly significant. Figures 5 and 6 summarise these differences after combining the four centres with fewest identified patients (full details are shown in Appendix 4.2). Although centres 20 and 22 identified more potential participants than other centres, they excluded many more than Aberdeen or Gloucester. Almost all of those with negative histology were from centres 29 and 30. ‘Not fit for surgery’ was least common in Gloucester, Aberdeen and centres 24 and 26. Gloucester had fewest exclusions for metastases. Almost all post-MDM refusals, but few pre-MDM refusals, came from Gloucester; this may reflect differing interpretations of consent. Unlike other centres, Aberdeen and Gloucester did not exclude people for ‘mandatory’ EUS scans. These findings led us to decide provisionally to combine all centres except Aberdeen and Gloucester into a single group for all analyses, and to test the validity of this plan through those analyses, notably by assessing how well data fitted this three-centre model.

FIGURE 5.

Patients identified by category within centre.

FIGURE 6.

Patients excluded by category within centre.

Attrition, withdrawals and missing data

Figures 7 and 8, and Appendix 4.3, show progress through the trial.

FIGURE 7.

CONSORT flow chart (from randomisation to follow-up at 12 months). a, One participant died in the time period in which he or she withdrew.

FIGURE 8.

CONSORT flow chart (from follow-up at 12 months until end of trial). QoL, quality of life.

In summary, six participants (three in EUS group; three in non-EUS group) withdrew completely; 25 (13 in EUS group; 12 in non-EUS group) withdrew from questionnaires, but not from collection of hospital data. We lost baseline data for three participants when a laptop computer was stolen; baseline data were also unavailable for another participant. Although none of the four completed another questionnaire, none withdrew completely; all four later died. Another participant withdrew completely before any information could be input.

In accordance with the general principles of pragmatic trials (see Table 7), and the need to analyse by treatment allocated enshrined in the CONSORT (Consolidated Standards Of Reporting Trials) statement,111 we sought to analyse as many participants as possible. Although 14 participants allocated to the EUS group did not receive a scan and three allocated to the non-EUS group did receive a scan, we analysed all 17 as allocated. Only a fastidious trial would label these two natural events as ‘deviations from the protocol’ (see Table 7). In contrast, we judge that these natural events strengthen the findings of our pragmatic trial as they represent normal clinical practice; to prevent them, even if possible and ethical, would transform COGNATE into a laboratory investigation of EUS that would be unable to yield the practical evaluation of the two policies – ‘include EUS in management plan after MDT meeting’ and ‘do not include EUS in this plan’ – that the NIHR HTA programme commissioned.

Furthermore, rather than remove participants who withdrew or provided incomplete information from analysis, we imputed their missing data whenever possible. In particular we know that 21 participants who withdrew died by the end of the trial. As EQ-5D is taken as zero after death, this information contributes to the primary outcome measure, QALYs, for these participants. However we excluded from primary analysis 10 participants with few or no useful data. Two withdrew completely before the 1-month interview and provided no information on outcome. The remaining eight provided no follow-up data, usually through non-attendance or withdrawal from interview, and either survived beyond the end of the trial or until they withdrew completely. Most of these participants also had little or no cost data. Thus 213 participants contributed to primary outcome analysis. To maximise power, however, we can compare survival and management plans (which do not require imputation of EQ-5D utilities from questionnaires or early deaths) on up to 221 participants.

Demographic and baseline characteristics

Table 9 compares baseline characteristics between intervention and control groups. As we stratified randomisation by centre and tumour site, these match very closely; the balance in other characteristics provides reassurance that the random allocation process was valid.

Over 75% of participants were male, and nearly half were aged over 65 years. The most common tumour site was oesophagus, the most common tumour type adenocarcinoma, and the most common tumour T stage was T3. Slightly more participants had nodal stage N1 than N0. As almost all were WHO status 1, we did not use this variable as a covariate. Similarly we reduced tumour site, T stage and type to binary variables for use as covariates. However pre-randomisation tumour stage was not recorded for one-quarter of participants. Very little other baseline information was missing among the 213 participants in the primary analysis.

Table 10 summarises the same baseline characteristics in the three centre groups, with the third group combining six separate centres. There were some significant differences between centres. Aberdeen participants were more likely to be aged under 65 years; those from Gloucester were the least likely to have WHO status 2. There was no record of pre-randomisation tumour stage for about half the Gloucester cases; among those with recorded T stage, Aberdeen and Gloucester had the highest proportion of T3 tumours. Despite the exclusion criteria, two biopsies from Aberdeen (both in the non-EUS group) were coded M1 and 11 more were equivocal.

Effective sample sizes

We immediately excluded one of the FACT Social items – ‘I am satisfied with my sex life’ – because the proportion of patients answering the question was low (40%, 43%, 37% at months 0, 1 and 3 respectively). After that exclusion and imputation (see Chapter 2, Statistical methods), the effective sample sizes for psychometric analyses were 220 at month 0, 173 at month 1 and 150 at month 3. These are fewer than the numbers of survivors because some survivors did not attempt one or both questionnaires. Beyond month 3, the effective sample sizes were too small for psychometric analysis. Hence we based decisions about how to score FACT on the data for months 0, 1 and 3, and extrapolated our conclusions to the whole trial.

Principal components analysis of FACT-AC

Confirmatory factor analysis of FACT-G

Structural equation modelling of relationships between measures

Findings

The internal consistencies and correlations between scales for month 0 are in Table 13 (with similar results for months 1 and 3 in Appendices 5.5 and 5.6): the internal consistencies of all multi-item scales were at least 0.70 at each time point; and EQ-5D and EQ-VAS correlated 0.40, 0.55 and 0.63 at months 0, 1 and 3 respectively. We ran separate structural equation models for EQ-5D and EQ-VAS. Those for EQ-5D in month 0 are in Figure 9 (with similar results for months 1 and 3 in Appendices 5.7 and 5.8): EQ-5D was predicted by Physical Well-Being at each time point, and also by Functional Well-Being at months 1 and 3. Additional Concerns predicted Physical, Functional, and Emotional Well-Being at all three time points, and also Social Well-Being at month 1. There was no evidence for a direct effect of Additional Concerns on EQ-5D, that is an effect not mediated by Physical and Functional Well-Being. The models for EQ-VAS in month 0 are in Figure 10 (with similar results for months 1 and 3 in Appendices 5.9 and 5.10). They show the same structure as for EQ-5D.

TABLE 13 - Internal consistency and intercorrelations of scales at month 0a

*p < 0.05; **p < 0.01.

n = 220.

Scale	Internal consistency	Correlations
		1	2	3	4	5	6
1. FACT Specific Concerns	0.91	–
2. FACT Physical	0.75	−0.50**	–
3. FACT Social	0.73	−0.11	0.11	–
4. FACT Emotional	0.78	−0.26**	0.19**	0.12	–
5. FACT Functional	0.84	−0.50**	0.61**	0.26**	0.29**	–
6. EQ-5D	–	−0.28**	0.53**	0.06	0.25**	0.44**	–
7. EQ-VAS	–	−0.45**	0.62**	0.15*	0.04	0.50**	0.40**

FIGURE 9.

FACT predicting EQ-5D at month 0. [χ²₄ = 6.52; p  = 0.16; SRMR = 0.04; CFI = 0.98; RMSEA = 0.07; p (close fit) = 0.21]. **p < 0.01.

FIGURE 10.

FACT predicting EQ-VAS at month 0. [χ²₅ = 11.03; p  = 0.05; SRMR = 0.04; CFI = 0.98; RMSEA = 0.07; p (close fit) = 0.21]. **p < 0.01.

Survival

Survival curves: univariate comparisons

Figure 11 shows the observed survival curves for the two groups. These take account of withdrawals and different lengths of time in the trial (between 1 and some 4 years) to provide unbiased estimates of the proportion of the original sample still alive at each time. Because of censoring, the lower portion of the curve suffers more random variation. For example a single death represents a change in survival of about 0.01 in the first year of the trial and about 0.02 at 3 years.

FIGURE 11.

Survival curves by allocated treatment group. Note: rectangular boxes above the x-axis record the number of participants available to estimate survival curve, subdivided into participants still alive + cumulative deaths; thus these boxes exclude participants censored when the trial ended or they withdrew completely.

The intervention survival curve lies almost completely above the control curve. Survival is similar in the two groups up to about 1 year, after which the curves diverge. Figures 12–16 subdivide survival curves by other baseline characteristics.

FIGURE 12.

Survival curves by centre.

FIGURE 13.

Survival curves by tumour type.

FIGURE 14.

Survival curves by tumour stage (T).

FIGURE 15.

Survival curves by tumour site.

FIGURE 16.

Survival curves by management plan.

Survival was significantly better in early tumour stages (univariate log-rank test; p = 0.017), and raised but not significantly better for non-squamous tumours and participants under 65 (p = 0.11 and 0.14 respectively). However centre, gender and tumour site do not contribute to survival in isolation (p = 0.48, 0.51 and 0.37 respectively). As expected survival was strongly related to management plan before randomisation and hence before EUS (p = 0.005): those planned for surgery or EMR survived longest, whereas those scheduled for chemotherapy or radiotherapy fared worst. Some survival curves are more open to random variation, for example those comparing participants from ‘other’ centres (n = 43) or with gastric or junctional tumours (n = 60). Unfortunately, more than 50 baseline tumour stages were absent from the patients’ notes from which we extracted our electronic data; their survival pattern was similar to that of the T3–T4 group.

Table 14 compares estimated mean and median survival times in intervention and control groups. Like survival curves, these estimates take account of varying follow-up times among survivors, so differ from the average length of survival. As there were no deaths after 4 years, we truncated the survival times of the few who were known to have survived beyond that time to 1461 days for consistency with the cost-effectiveness analysis. Estimated mean values still depend on the length of the trial, so medians are more robust. The median survival times were 20 months for the control group, and 24 months for the intervention group.

TABLE 14 - Estimated mean and median survival time by allocated group

Group	Mean survival in days (truncated at 1461 days)	Median survival in days
	Mean (95% CI)	Median (95% CI)
Non-EUS	700 (601 to 799)	596 (415 to 777)
EUS	813 (707 to 919)	717 (533 to 901)
Both	755 (683 to 828)	647 (520 to 774)

Table 15 gives a similar comparison between centres. The median survival times were 20 months in Aberdeen and 23 months in both Gloucester and the other centres.

TABLE 15 - Estimated mean and median survival time by centre

Centre	Mean survival in days (truncated at 1461 days)	Median survival in days
	Mean (95% CI)	Median (95% CI)
Aberdeen	709 (583 to 834)	592 (484 to 700)
Gloucester	761 (655 to 867)	688 (396 to 980)
Other centres	836 (672 to 1001)	683 (530 to 836)
All centres	755 (682 to 828)	647 (520 to 774)

Survival models with multiple predictors

Survival analysis takes account of censoring of observations at the time of withdrawal from the trial or other loss to follow-up. Our analysis therefore included all 221 people who had useful survival data, namely those who survived longer than the first participant who died.

Cox regression uses a logarithmic model to estimate the hazard ratio, which is the ratio of the instantaneous chance of dying in intervention and control groups. We specifically chose a proportional hazards model, in which this ratio is the same at all time points. If that hazard ratio were 1.0, there would be no difference between groups: at any time intervention and control participants would have an equal chance of dying. If we were to use only the allocated treatment to predict risk, the estimated hazard ratio would be 0.76, with a 95% confidence interval from 0.54 to 1.06 (see Appendix 6.1). At any time after randomisation, the estimated risk of death for an intervention participant would be only three-quarters of that for a control participant. As the confidence interval for that hazard ratio includes 1.0, however, we could not be 95% certain that the risk of death is lower with EUS than without.

Consistent with our agreed analysis plan (see Chapter 2, Statistical methods), we extended our model to allow for covariates. As potential covariates we considered centre, age, gender, tumour type and site, biopsy tumour (T) stage, baseline EQ-5D and management plan before randomisation. To keep the model parsimonious, we dichotomised pre-randomisation T stage into T2 or less versus T3 or more [including not recorded (NR)], and management plan into multimodal versus not. Although Figure 12 suggests that survival may be similar across centres, the known differences between centres (see Figures 5 and 6) led us to split them into two contrasts – Gloucester and Aberdeen versus the other centres, and Gloucester versus Aberdeen. Adding T stage, multimodal plan and centre thus defined to the model improved the EUS hazard ratio, which became significant (Table 16, plus see Appendix 6.2).

TABLE 16 - Survival by allocated group: Cox regression estimates without interaction

Tis, tumour in situ.

Rows in bold type contribute significantly to the model.

Model component or contrast	Hazard ratio	95% CI	Significance
EUS vs non-EUS	0.706	0.501 to 0.996	0.048
Other centres vs (Aberdeen or Gloucester)	0.929	0.723 to 1.194	0.567
Gloucester vs Aberdeen	0.858	0.707 to 1.041	0.120
(T3, T4 or NR) vs (Tis, T1 or T2)	1.353	1.097 to 1.670	0.005
Multimodal plan vs not	1.415	1.147 to 1.746	0.001
Baseline EQ-5D	0.716	0.277 to 1.849	0.490

Two examples may help interpretation. First, if we compare two participants who differ only in whether allocated to EUS, the chance of the intervention participant dying at any time is 71% of the chance of the corresponding control participant with the same centre, initial T stage, management plan and baseline quality of life. Second, if we compare a control participant with a multimodal plan (and personal characteristics associated with that) with an intervention participant with a different plan (and personal characteristics, although in the same centre with the same initial T stage and baseline quality of life) then our model tells us that the latter's relative chance of death is 0.706 divided by 1.415, namely one half.

Building on the model in Table 16, Figure 17 shows the estimated population survival curves when all covariates are at their mean. The pattern is similar, but not identical, to the corresponding empirical survival curves in Figure 12. This similarity confirms that our data are consistent with the proportional hazards assumption in the Cox regression model.

FIGURE 17.

Survival curves estimated from Cox model without interaction by group.

We considered whether or not to add interactions with covariates to this model. None of the main covariates in that model had a significant interaction with the effect of EUS on survival. However baseline EQ-5D, although not significant in its own right, was significantly associated with the effect of EUS, yielding the hazard ratios shown in Table 17. This interaction improves the fit of the model, and the main effect of EUS remains significant.

TABLE 17 - Survival by allocated group: Cox regression estimates with interaction

Tis, tumour in situ.

Rows in bold type contribute significantly to the model.

Model component or contrast	Hazard ratio	95% CI	Significance
EUS vs non-EUS	0.704	0.499 to 0.993	0.046
Other centres vs (Aberdeen or Gloucester)	0.938	0.741 to 1.188	0.596
Gloucester vs Aberdeen	0.865	0.713 to 1.051	0.144
(T3, T4 or NR) vs (Tis, T1 or T2)	1.286	1.039 to 1.591	0.021
Multimodal plan vs not	1.443	1.168 to 1.782	0.001
Baseline EQ-5D	0.518	0.189 to 1.424	0.203
Interaction: EUS and EQ-5D	13.18	1.672 to 107.0	0.016

To help interpretation, we considered three participants with the same centre, pre-randomisation T stage and management plan, but different baseline EQ-5D scores – low (0.6), average (0.8) and high (1.0). Hence most of the hazard ratios in Table 17 do not contribute to relative chances. By analogy with the first example above, the second participant has an instantaneous risk of dying after EUS that is 70% of the risk without such a scan. In contrast the first participant has an EQ-5D score of 0.6, that is −0.2 below the mean EQ-5D. Thus the interaction term multiplies the hazard ratio by (13.18)–^0.2 = 0.597, thus changing it to 42%, namely 0.597 × 0.704. Also in contrast the third participant has an EQ-5D score of 1.0, that is 0.2 (= 1.0 – 0.8) above the mean EQ-5D. Thus the interaction term multiplies the hazard ratio by (13.18)^0.2 = 1.675, thus changing it to 118%, namely 1.675 × 0.704. Lest an increase of 18% in mortality risk seem alarming, we recall that this estimate is subject to enough random error that we cannot conclude that EUS increases the risk of dying even for the subgroup with baseline utility of 1. In short survival averaged over all participants is better in the intervention group; and lower baseline EQ-5D scores give greater advantage to the intervention.

Quality of life

Table 18 shows no significant differences in baseline quality-of-life scores between groups. However there is evidence of systematic differences between centres in quality of life at baseline; in particular Gloucester had the worst baseline (see Appendix 6.3). However the difference is only significant for FACT-G, FACT Social and EQ-VAS. We judge that the different proportions excluded by the three ‘centres’ and the resulting differences in case mix explain these baseline differences.

In comparing EQ-5D utility scores at baseline and follow-up between groups, we initially used only participants with interview data (face to face, telephone or postal) at those times (see Appendix 6.4). We imputed the few individual EQ-5D items omitted before calculating their utility scores (see Chapter 2, Imputation and missing values, Imputing quality of life: phase 1 – psychometric and effectiveness analyses). In general, respondents' EQ-5D scores fell in the first 3 months and then remained stable until 24 months, although the number of respondents was decreasing. There were no consistent differences between intervention and control groups.

Other quality-of-life scores had changed less at 12 months (see Appendix 6.5), with FACT and EQ-VAS means similar to those at baseline. Mean FACT subscale scores varied from about 2.5 for FACT Functional to about 3.5 for FACT Social. The EQ-VAS means were at about three-quarters of their maximum value. Adding imputed scores for surviving non-responders increases the number of participants at 12 months from 112 to 143 (see Appendix 6.6). Mean scores are similar to, but marginally worse than, the corresponding means for responders only (see Appendix 6.5). This suggests that non-responders have slightly worse health.

Throughout follow-up more control participants than intervention participants had died (Figure 11 and see Appendix 6.1). Hence comparisons of survivors are likely to be biased against the intervention group. As EQ-5D explicitly assigns a utility of zero to death, Table 19 uses this for known non-survivors at all times between their deaths and the end of the trial. Combining these zeros with the non-zero scores of survivors gives an unbiased comparison between groups. Table 19 has two rows for each of the 18-, 24- and 36-month follow-ups. The first uses only participants whose survival status (i.e. whether dead or alive) is known, but imputes quality of life if an interview has been missed. To avoid bias, it excludes deaths recruited near the end of the trial who would not have reached this time point if they had survived. This is sufficient for the primary analysis of QALYs (i.e. area under EQ-5D curve) using Cox regression. The second row uses all 213 participants, as required for both the main 48-month cost-effectiveness analysis and our sensitivity analysis of QALYs using ANCOVA. ANCOVA relies more heavily than Cox regression on imputation: more than 25% of the EQ-5D scores at 36 months arose from multiplying imputed ‘live’ scores by estimated survival probabilities (see Chapter 2, Design).

As expected, the resulting mean quality of life declines faster over time for all participants (including those who have died) than for survivors only (see Appendices 6.6 and 6.8). Although at all time points except 1 month the mean is higher in the EUS group, none of the differences between EUS and non-EUS groups is significant. Table 19 reports imputed scores at 12 months as secondary outcomes, to reduce the chance of false-positive results due to multiple testing. By analogy, from the FACT data we chose only four more secondary outcomes – the corresponding AUC and 12-month scores of FACT-G and FACT-AC (see Recruitment, participant flow and CONSORT). For description rather than inference Appendix 6.7 reports the corresponding raw data and basic t-tests at all time points, thus elaborating Table 19 by including all eight quality-of-life measures shown in Table 18 – at all seven follow-up points.

From death we scored EQ-VAS, FACT-G and its four subscales as zero, and FACT-AC (for which the worst score is 4.0) as 3.66, the worst reported within the COGNATE trial (see Chapter 2, Statistical methods, Imputation and missing values). The FACT scales and EQ-VAS then follow patterns similar to that of EQ-5D utilities in Table 19. By 36 months mean quality of life measured by FACT or EQ-5D had dropped below one-quarter of its optimum value. Although none of the 56 differences in Appendix 6.7 is significant, all 56 are in favour of EUS. Appendix 6.8 focuses on findings at 12 months from Appendix 6.7. The more powerful ANCOVA also shows no significant differences between intervention and control groups. Appendices 6.9–6.11 show the models for EQ-5D, FACT-G and FACT-AC respectively – those suggested by the structural equation modelling (see Refinement of quality-of-life measures). Table 20 shows the corresponding adjusted means and confidence intervals.

In the search for covariates, baseline measurements are highly significant for the two FACT summary measures and close to significance for EQ-5D. In all models the only other significant or near-significant covariates are biopsy T stage (one of the two covariates for survival) and age. Initial management plan is less significant in predicting quality of life than in predicting survival, especially for the two FACT models.

Table 21 shows the result of including interactions in the ANCOVA, illustrated by the same three participants used in Table 17, with low, medium and high baseline EQ-5D respectively. Appendices 6.12–6.14 show the corresponding models for EQ-5D, FACT-G and FACT-AC respectively. At 12 months there was no significant difference in EQ-5D between EUS group and non-EUS group (see Table 20 and Appendix 6.9). If we add interaction between EUS group and baseline EQ-5D to the model, both baseline EQ-5D and the interaction are significant (see Table 21 and Appendix 6.12). Unlike survival, however, adding the interaction does not make the main effect of EUS group on EQ-5D significant. Replacing age by multimodal plan would again have little effect on the model, with management plan now a marginally significant predictor (p = 0.048), but only one of these two covariates is needed. Table 21 shows that as baseline EQ-5D increases, the advantage of EUS decreases and is eventually reversed. Although an EUS participant's expected quality of life at 12 months is similar for the three examples, non-EUS participants’ quality-of-life outcomes depend strongly on their initial EQ-5D.

Thus the analysis of both survival and EQ-5D shows highly significant interactions between participants’ allocated treatment (whether they had drawn EUS in the COGNATE lottery) and baseline EQ-5D. EUS may benefit participants in poor health more than those whose health is not yet showing the worse effects of gastro-oesophageal cancer.

In planning the analogous analyses of covariance for the FACT-G (see Appendix 6.13) and FACT-AC (see Appendix 6.14), we predicted that the corresponding baseline FACT scores would also show significant interaction with allocated treatment. However those analyses showed that the baseline FACT scores achieved no interactions. Instead we found that baseline EQ-5D surprisingly continued to interact strongly with allocated treatment. We interpret this as evidence that baseline EQ-5D may be better than other baseline measures at differentiating ‘worse’ patients (helped by EUS) from ‘better’ patients (not helped by EUS) in serious conditions.

Including interactions as well as covariates in the models predicting 12-month EQ-5D, FACT-G and FACT-AC had very little influence on the main effect of allocated treatment: the estimated differences between EUS and non-EUS groups when baseline EQ-5D takes its average value of 0.8 in Table 21, and the significance of those differences, are similar to the differences in Table 20.

Survival adjusted by EQ-5D (primary outcome)

Univariate comparisons (quality-adjusted survival curves and log-rank tests)

Participants in early tumour stages [tumour in situ (Tis), T1 or T2] before randomisation achieved significantly higher QALYs (p = 0.008 by univariate Mantel–Cox log-rank test), as did participants whose original management plan was not multimodal (p = 0.002). However allocated treatment, tumour type, age over 65 years, gender, tumour site and centre do not contribute to survival in isolation (p = 0.09, 0.20, 0.28, 0.44, 0.57 and 0.65 respectively). Table 22 compares mean and median quality-adjusted survival between the two allocated treatments. The more robust median quality-adjusted survival times are 11 months for the control group and 13.5 months for the intervention group.

TABLE 22 - Mean and median QALYs by allocated group

Group	Mean QALYs (years, truncated at 4-year survival)		Median QALYs (years)
	Mean	95% CI	Median	95% CI
Non-EUS	1.44	1.14 to 1.74	0.94	0.51 to 1.37
EUS	1.83	1.49 to 2.17	1.12	0.67 to 1.57
Both	1.65	1.42 to 1.88	1.05	0.76 to 1.34

Figure 18 compares adjusted survival curves by allocated treatment. In contrast to the unadjusted survival curves, these two curves quickly separate, with consistently better experience in the intervention group than in the control group. Only 1 of the 10 people who survived beyond 4 years had maximum quality of life throughout their time in the trial. Similarly, fewer participants reached one, two or three QALYs in the trial than the corresponding numbers for unadjusted survival times.

FIGURE 18.

Quality-adjusted survival curves: EUS and non-EUS groups. Rectangular boxes above the x-axis record the number of participants available to estimate survival curve, subdivided into participants still alive plus cumulative deaths; thus these boxes exclude participants censored when the trial ended or they withdrew completely.

Quality-adjusted survival in the three centres is shown in Figure 19. Although the ‘other centres’ curve is slightly above the curves for Gloucester and Aberdeen, it represents only 41 participants, and the difference could easily be explained by chance.

FIGURE 19.

Quality-adjusted survival curves by centre.

Survival adjusted by EQ-5D with multiple predictors

We undertook survival analyses by Cox regression for 213 participants for whom there was enough survival, FACT and EQ-5D data to estimate quality-adjusted survival; all contributed to the model estimates. As for unadjusted survival, our analysis censored observations at the time of withdrawal or loss to follow-up. We fitted three models: the simplest fitted only the allocated treatment; the second model added the effects of centre and other significant or helpful covariates, but not interactions; and the third added the only significant interaction term.

In the model without covariates, the intervention group had better quality-adjusted survival than the control group, but not significantly better; the hazard ratio was 0.76 (95% CI from 0.54 to 1.06; p = 0.106). Table 23 shows the second Cox regression, including covariates but not interaction. As before potential covariates were baseline characteristics: centre; baseline EQ-5D; aged over 65 years; gender; tumour site, pre-randomisation stage and type; and management plan before randomisation. In accordance with the analysis plan [see Chapter 2, Statistical methods, Quality-adjusted survival (primary outcome) and survival], the first two of these were automatically included. The other two selected – pre-randomisation T stage and multimodal plan – were the same as for unadjusted survival. However age, which replaced multimodal plan as a covariate for quality of life at 12 months, was not a significant predictor. The addition of these four covariates improved the contrast between intervention and control: the hazard ratio fell to 0.705 (95% CI from 0.499 to 0.995; p = 0.047), now significantly different from 1.0.

Table 24 shows that interaction between baseline EQ-5D and allocated treatment in predicting QALYs is still highly significant (p = 0.008). The main effect of baseline EQ-5D also becomes significant (p = 0.011). Although the effect of EUS in the EUS group is no longer strictly significant, the hazard ratio 0.712 (95% CI from 0.504 to 1.005; p = 0.054) differs little from that without interaction. Although the initial management plan (and its covariates) had a large effect on quality-adjusted survival, it, like all other covariates except baseline EQ-5D, did not interact with the intervention. When the interaction was added, pre-randomisation T stage declined in importance as a predictor. As usual we kept centres in Table 24 to show that that had no effect despite very different recruitment strategies.

Survival adjusted by FACT (secondary outcome)

Survival adjusted by FACT-G scale

We undertook another quality-adjusted survival analysis, again using ‘AUC’ but replacing EQ-5D with the more comprehensive FACT-G. For this measure a year of life with the best possible FACT score of 4.0 throughout gives a quality-adjusted survival of 1 year, and a year in which the average FACT score is 2.0 gives a quality-adjusted survival of only 6 months. Results were similar, but not identical, to those using the conventional definition of quality-adjusted survival based on the EQ-5D. Figure 20 shows the observed survival curves for the two groups. However a Mantel–Cox log-rank test shows no significant difference between the allocated treatment groups (p = 0.11).

FIGURE 20.

Survival curves adjusted by FACT-G by allocated treatment group.

Table 25 gives the corresponding means, medians and confidence intervals. The median survival times adjusted by FACT-G are 11.5 months for the control participants and 15 months for the intervention group. However Cox regression using allocated treatment group as single predictor did not show a significant effect of EUS: the hazard ratio was 0.774 with a 95% confidence interval from 0.550 to 1.089 (p = 0.142). Although adding management plan, pre-randomisation T stage, baseline FACT-G and centre to the covariates decreases the estimated hazard ratio to 0.72, and improves significance, the difference is still not significant (see Appendix 6.15).

TABLE 25 - Survival adjusted by FACT-G: mean and median by allocated group

Group	Mean QALYs (years)		Median QALYs (years)
	Mean	95% CI	Median	95% CI
Non-EUS	1.46	1.19 to 1.72	0.94	0.55 to 1.34
EUS	1.78	1.49 to 2.06	1.27	0.79 to 1.76
Both	1.63	1.43 to 1.82	1.17	0.90 to 1.43

We then analysed models with interactions. As for 12-month FACT-G scores, we considered interactions with baseline FACT-G and the usual covariates (see Appendix 6.14). As none of these were significant, we added baseline EQ-5D and its interaction with allocated treatment to the model (see Appendix 6.15). There was significant interaction with baseline EQ-5D but, as with the 12-month FACT-G, not with baseline FACT-G. Although baseline FACT-G still had a significant effect on the outcome, that effect was the same in both allocated treatment groups and does not affect the difference between them or the hazard ratio. As before, however, this hazard ratio applies only to participants with baseline EQ-5D equal to 0.8, the mean of the whole sample. Participants with lower baseline EQ-5Ds have an even smaller hazard ratio, whereas those with maximum baseline EQ-5Ds of 1.0 have hazard ratios close to 1.0. Thus EUS improves FACT-G-adjusted quality of life for the sickest participants but not the healthiest.

Quality-adjusted survival: sensitivity analyses with fully imputed data

Table 27 displays means and confidence intervals from comparisons of QALYs fully imputed to 1 or 4 years for all 213 participants, both before and after ANCOVA. The similarity between the estimated differences in the top and bottom of the table confirms that the conclusions of our comparative analyses are not sensitive to choice of method or covariates.

Before ANCOVA, the cost-effectiveness analysis (see Health economics) used these fully imputed estimates in its own analysis to test whether those conclusions are sensitive to the choice of whether or not to discount benefits as well as costs. In this way, sensitivity analyses in this section analysing effectiveness and in the subsection analysing cost-effectiveness (see Health economics; Sensitivity analysis at 48 months with undiscounted quality-adjusted life-years) have used precisely the same data.

Change in management plans (secondary outcome)

Having found that EUS improves quality-adjusted survival, especially for those with low quality of life at baseline, we explore the mechanism of this effect by considering changes in management plans. Table 28 compares plans at the time of randomisation into the trial with those recorded after EUS (or its alternative in the control group), but before treatment started; the diagonal, unshaded cells represent no change.

Table 28 shows no difference between intervention and control groups in the number of changes from initial, pre-randomisation management plan to updated plan: about one-quarter of each group changed – 29 out of 109 in the intervention group compared with 27 out of 110 in the control group. Given the improved survival and quality-adjusted survival, we did not expect this similarity. Hence we scrutinised the two subgroups who changed plans, although conscious that the small numbers in both meant that even large differences could not be statistically significant. Over 60% of initial plans in each group were for neo-adjuvant chemotherapy before surgery, with very few planned EMRs. Most changes in each group were from neo-adjuvant therapy before surgery to another option. In the intervention group 61% (11/18) of these changes (in the third row of Table 28) were to therapeutic treatment (EMR or surgery), compared with only 38% (8/21) in the control group. Among all changes of plan, more in the intervention group changed to a more optimistic plan (those below rather than above the leading diagonal of Table 28) than in the control group [52% (15/29) vs 41% (11/27)]. Half of the changes in the intervention group were recorded, explicitly or implicitly, as due to EUS results. Some changes in the control group cited other staging tests done after randomisation, typically PET, laparoscopy or CT.

Furthermore changes in plans improved survival more in the intervention group than in the control group. As we expected, those whose final plan was multimodal had significantly worse survival in both groups (see Figure 16). However this difference was more marked in the intervention group than in the control group (mean survival shorter by 354 and 205 days respectively). Table 29 shows that this contrast between groups stemmed from those whose plans changed: in the intervention group mean survival was 650 days (namely 452 vs 1102) shorter in those who changed to multimodal than in other changers, but in the control group the difference was only 58 days (namely 564 vs 622). For those with unchanged plans the corresponding survival differences were close – 253 days (namely 605 vs 858) in the intervention group and 305 days (namely 455 vs 760) in the control group.

Although none of these differences was significant, they are consistent with EUS identifying participants whose original management plans were inappropriate, especially those originally scheduled for multimodal treatment but able to benefit from surgery, and those originally scheduled for surgery despite poor prognosis (see Chapter 1, Literature review, Treatment). In contrast those who changed plan in the control group experienced intermediate survival, arguably because they were on the borderline between alternative plans.

Table 30, restricted to 145 participants from the primary analysis who survived to 12 months (with the result that four of its eight cells contain fewer than 10 participants), illustrates how EUS also improved quality of life. All four sets of survivors who had changed plan (namely intervention or not by whether final plan was multimodal or not) reported higher mean EQ-5D scores than the corresponding set of survivors whose plan remained unchanged, and all four sets of survivors in the intervention group (namely plan changed or not by whether final plan was multimodal or not) reported higher mean EQ-5D scores than the corresponding set of control group survivors. When we extended Table 30 to all 213 participants in the primary analysis by ascribing zero quality of life to those who did not survive to 12 months, we found similar benefits from EUS: 17 intervention participants who changed to non-multimodal plans had mean EQ-5D of 0.642; 16 control participants with confirmed multimodal plans had mean EQ-5D of 0.284, and the other six subgroups lay between these two extremes.

Table 31, restricted to the 213 participants for whom we could estimate costs, illustrates how EUS reduced costs. Control participants who changed plan in either direction incurred intermediate mean costs – above confirmed multimodal plans but below confirmed non-multimodal plans. In contrast intervention group participants who changed plan in either direction incurred lower costs than intervention group participants with confirmed plans of either type. That these findings are analogous to those of Table 29 is not surprising, because cost generally depends on length of survival. However the relationship is not straightforward, as costs are also subject to other influences; for example, long-term survivors avoid the typically high cost of terminal care.

In summary, Tables 29–31 are exploratory tables, designed to explain the apparent inconsistency between major differences in survival and quality of life between intervention and control groups yet similar numbers of changes in management plans. Unlike the definitive analyses of outcome, they do not seek to adjust for covariates, even though the groups are not comparable: instead they use raw means and medians to estimate utility and cost, and simple univariate Kaplan–Meier analyses to estimate survival. Although changes of management plan, and the resulting final plan, are themselves trial outcomes dependent on covariates, traditional adjustment by ANCOVA would obscure the relationships we seek to understand.

Table 32 shows that treatments actually delivered did not always coincide with management plans at randomisation. In particular several participants in both groups started neo-adjuvant chemotherapy but did not progress to surgery, usually because the tumour remained unresectable. Their actual treatment was therefore multimodal. Again there was no significant difference between the pattern of change in the intervention and control groups. However Figure 21 shows considerable differences between centres, both in amendments to the initial treatment plan and in the actual treatment delivered. In both Aberdeen and Gloucester some 30% of participants were originally scheduled for neo-adjuvant chemotherapy followed by surgery but were switched from it. However most changes in Gloucester occurred before the updated plan was confirmed, while in Aberdeen such changes occurred mostly between the updated plan and actual treatment. Other centres changed less often than Aberdeen and Gloucester, and ended with fewer participants receiving only multimodal treatment, as one might expect from their better initial tumour T stage (see Table 10) and baseline quality of life (see Table 25).

FIGURE 21.

Management plan changes and treatment received by centre (three groups).

Other secondary outcome measures

Analysis by ‘treatment received’

Fourteen participants allocated to EUS did not receive it, for varied reasons: early withdrawal (excluded from most analyses) or failure to attend accounted for six; appointments could not be arranged in time for another three; and equipment failed twice. Three allocated to the control group did receive EUS, only two at the request of medical staff. As a form of sensitivity analysis we repeated our main analyses – survival with and without adjusting for quality of life, changes in management plan and complete resection – by ‘EUS received’.

Other analyses

Our models have consistently shown that, apart from random allocation to EUS, the main prognostic indicators for both survival and quality of life were pre-randomisation tumour stage, planned multimodal treatment, baseline quality of life, and the interaction between that baseline and random allocation to EUS. In particular there is no hint in any of our many multivariate analyses of different types that centre, tumour site or tumour type had any influence on outcome.

Palser et al. 5 have strongly recommended EUS for oesophageal tumours but not for gastric tumours. As none of our extensive analyses provides any support for this, in particular any evidence of interaction between allocated treatment and tumour site, we offer two case studies of patients with gastric cancer to illustrate our analyses:

A patient aged 61 with adenocarcinoma of the stomach was randomised after screening barium study, chest radiography, CT scan and endoscopy. The management plan of neo-adjuvant chemotherapy plus surgery was changed to multimodal treatment after endoscopic ultrasound revealed ‘a bulky, locally advanced tumour unsuitable for resection’. Staging at randomisation was T3/T4 N1 but endoscopic ultrasound confirmed that staging was T4 N1. The patient died 7 months after randomisation of cancer as the main cause with local tumour only.

A patient aged 70 with gastric adenocarcinoma was randomised in October 2007 to the EUS arm of COGNATE after endoscopy and chest radiography. The earlier biopsy report was ‘highly suspicious for but not diagnostic of invasive adenocarcinoma of intestinal type’. The original management plan of surgery was changed to multimodal treatment as ‘Endoscopic ultrasound showed a T3 lesion; patient therefore not for surgery’. Pre-randomisation staging was T2/T3 and endoscopic ultrasound confirmed T3 N1. The patient, who was frail with a lot of comorbidities, opted for palliative care and died 8 months later; cause of death was unrelated to cancer, with local tumour only.

Who might benefit from endoscopic ultrasound?

We had hoped to estimate the general proportion of gastro-oesophageal cancer patients who benefit from EUS. However differences between our centres (see Figure 4) and our unsuccessful attempts to recruit more than two centres unequivocally in equipoise prevented us from estimating that proportion with any confidence.

Therefore restricting this estimation to the COGNATE study population in Figure 4, we see that, of the 1124 patients with positive histology, 567 were potentially eligible for the trial and therefore for EUS. However the 114 not consenting to the trial could still potentially benefit from EUS. Of the 453 who did consent, eight were no longer eligible by the time of the MDM, and eight more proved unsuitable for EUS. At first sight, therefore, we would expect (567 × 437/453) = 547 (49% of 1124) patients with gastro-oesophageal cancer to benefit from EUS in the way shown by the trial. To illustrate the differences between centres, we estimate by subdividing Figure 4 that this proportion varies from 70/172 (41%) in Aberdeen to 101/163 (62%) in Gloucester.

However the interaction between EUS group and baseline EQ-5D shows those with the best initial quality of life do not benefit from EUS. Estimating from our interaction models that about one-quarter may not benefit, we predict that only 410 (36%) of the original 1124 were likely to benefit. In short this trial suggests that, by undertaking EUS for about 48% of patients with histologically proven gastro-oesophageal cancer, one can benefit about 36%.

Unit costs

As we finished data collection in July 2009, we chose to use unit costs from 2008 for all our analyses. These covered two time periods: up to 12 months after randomisation and up to 48 months after, by which time many costs and QALYs had inevitably been censored. Table 38 shows these unit costs and their sources.

Missing cost and quality-adjusted life-year data

Table 39 shows numbers of participants with partially or completely missing data over time periods. We pursued missing data throughout the trial (see Chapter 2, Design, Electronic data collection and storage). Our electronic database alerted researchers interviewing participants to missed questions and overdue interviews, and verified data quality. Researchers collecting and entering cost data from hospital records also received database prompts, and regular checks from the trial centre. By these means over the first 12 months we collected costs and survival data for all but three participants who withdrew early.

Frequency of resource use

Table 40 (full table in Appendix 7.1) summarises the mean frequencies of resource use for secondary care contacts and hospital prescribed drugs over six time periods for the 213 participants in the primary analysis. This information comprises data put into the COGNATE database via laptop computers by research professionals at each study site. The effective sample throughout comprises those still in the trial, namely survivors observed throughout the period, and those who died before the end of the period and before 31 July 2009. Participants, dead or alive, for whom the end of the trial occurred during that period, get half the weight of those in the trial for the whole of that period.

Hospital inpatient stay was considerably shorter in the EUS group (21.6 days) than in the non-EUS group (29.9 days). Almost all these stays were cancer related (20.8 and 29.5 days respectively). Both patterns were also present in the first year of the trial. In contrast outpatient visits were slightly more frequent in the EUS group than in the non-EUS group over 48 months but not over 12 months.

Costs over 12 and 48 months

We estimated participants’ costs in each time period and resource category by multiplying individual resource use in Table 40 by corresponding unit costs in Table 38. We summed estimates across categories to yield the cost of each period during which participants remained in the trial, and discounted costs in later periods at 3.5% per annum (summarised in Appendix 7.2). We imputed missing costs (as described in Chapter 2, Statistical methods, Imputation and missing values), and summed participants’ costs over 48 months. Table 41 summarises these costs for the primary cost-effectiveness analysis, starting with costs for the first 12 months. The mean cost of secondary care including hospital prescribed drugs was £2860 lower for the EUS group than for the non-EUS group, even after including the cost of EUS scans. This difference is 9% of the mean cost over 48 months in the control group. In the first year of the trial the difference of £3432 (14% of the control group's mean cost) was even bigger. Both analyses used bootstrapping with 5000 replicates to estimate a 95% confidence interval around this mean difference in costs between groups.

Cost-effectiveness at 12 months

Primary analysis

By 12 months, 107 participants in the EUS group had on average gained 0.0304 more QALYs and cost £3432 less than 106 participants in the non-EUS group. This suggests that, over the 12 months from randomisation, EUS is more effective and less costly than usual management. However neither difference is significant on its own (see Table 41). To enhance the statistical power of such evaluations, cost-effectiveness analysis focuses on the joint distribution of costs and effects. When the intervention under evaluation yields clinical benefits but costs more, the ICER compares these benefits and costs by assessing whether the NHS gains from this trade-off. When the intervention yields both clinical benefits and resource savings, both Briggs et al. 132 and Glick et al. 131 warn that the resulting negative ICER is meaningless.

Instead we used bootstrapping with 5000 replicates to generate a cost-effectiveness plane (CE plane) plotting the joint distribution of costs and benefits (Figure 22a): 3799 (76.0%) points lie in the south-east (‘win–win’) quadrant, 911 (18.2%) in the south-west quadrant (net costs and QALYs negative), 187 (3.7%) in the north-east quadrant (net QALYs and costs positive) and 103 (2.1%) in the north-west (‘lose–lose’) quadrant. To summarise the CE plane, we introduce the ‘cost-effectiveness threshold’ – the most that decision-makers are willing to pay to gain one QALY across patients. By varying this threshold, and thus the proportion of points that achieve cost-effectiveness, we construct the CEAC shown in Figure 22b. This shows that: even if decision-makers put no value on an extra QALY, EUS has 94.2% probability (the proportion of points below the x-axis in Figure 22a) of being cost-effective; when the threshold reaches the lower NICE criterion of £20,000, this probability rises to 95.0%; when the threshold reaches the upper NICE criterion of £30,000, it reaches 95.2%; and, if decision-makers put infinite value on a QALY, it falls to 79.7% (the proportion of points to the right of the y-axis in Figure 22a). To assess the stability of these bootstrapped probability estimates, we ran another replication; this confirmed that EUS is likely to be cost-effective at 12 months, yielding very similar probabilities – 95.1% at £20,000 and 95.0% at £30,000.

FIGURE 22.

Cost-effectiveness plane and CEAC at 12 months by unit cost of EUS.

Cost-effectiveness at 48 months

Primary analysis

To extend cost-effectiveness analysis from 12 to 48 months, we kept the unit cost of EUS for day cases at £551, discounted both costs and QALYs at 3.5% per year, and used survival analysis to model likely outcomes for participants who were still alive at the end of data collection (Figure 23). Thus by 48 months the 107 participants in the EUS group would on average have gained 0.1969 more QALYs and cost £2860 less than the 106 participants in the non-EUS group. This suggests that, 48 months after randomisation, EUS is still more effective and less costly than usual management.

FIGURE 23.

Cost-effectiveness plane and CEAC at 48 months by unit cost of EUS.

Again, neither finding is significant. So again we bootstrapped 5000 replicates to generate CE planes and corresponding CEACs. Figure 23b shows that the probability of EUS being cost-effective is 86.3% at a threshold of zero; 96.6% at £20,000; 96.8% at £30,000; and 92.6% when the threshold is infinite. To assess stability, we ran another 5000 replicates; these confirmed that EUS is probably cost-effective at 48 months, with very similar probabilities: 97.0% at £20,000 and 96.7% at £30,000.

Sensitivity analysis at 48 months with undiscounted quality-adjusted life-years

For consistency with undiscounted effectiveness analyses above (see Effectiveness, Quality-adjusted survival: sensitivity analyses with fully imputed data), we undertook further sensitivity analysis at 48 months, consistent with Figure 23 in discounting costs at 3.5%, but without discounting QALYs. This increased the average QALY gain for patients in the EUS group a little from 0.1969 to 0.2086, identical to that estimated in Table 27. Figure 24b shows that not discounting hardly changes the probability of cost-effectiveness, which is: 87.2% at a threshold of zero; 97.0% at £20,000; 96.9% at £30,000; and 92.6% at infinity. As before, we varied unit costs for EUS among those offered by the Department of Health. Figure 24d shows that a unit cost of £1477 rather than £551 slightly reduces the probability that EUS is cost-effective – to 79.1% at a threshold of zero, 95.6% at £20,000 and 96.2% at £30,000. Figure 24f shows that a unit cost of £3781 further reduces the probability that EUS is cost-effective – to 50.1% at a threshold of zero, 87.2% at £20,000 and 90.1% at £30,000. However the standard deviation (SD) is larger in undiscounted QALYs than in discounted QALYs, so as the threshold increases to infinity, this counteracts the slightly larger QALY gain. Thus all three probabilities converge on the same 92.5% probability of cost-effectiveness as in the primary analysis. In the NICE threshold range of £20,000 to £30,000, each of the three CEAC curves is marginally above the corresponding curve in the primary analysis. Hence all features of Figure 24 are very similar to those of Figure 23.

FIGURE 24.

Cost-effectiveness plane and CEAC at 48 months by unit cost for undiscounted QALYs.

Subgroup analysis at 48 months

As Figures 22b and 23b provide strong evidence that EUS is likely to be cost-effective in gastro-oesophageal cancer, we ask whether some participants benefit more than others. Our effectiveness analysis addressed this question by adding covariates to primary analysis (see Table 24). However cost-effectiveness analysis with bootstrapping is less flexible. Although subgroup analysis reduces sample sizes and statistical power, we examined subgroups defined by baseline EQ-5D (Figure 25) and age (Figure 26) within the primary analysis to 48 months, in which we discounted both costs and QALYs.

FIGURE 25.

Cost-effectiveness plane and CEAC by whether baseline EQ-5D below or above median.

FIGURE 26.

Cost-effectiveness plane and CEAC by whether age below or above median of 65 years.

Summary

Table 42 summarises our health economic findings. All analyses took account of censoring and used bootstrapping with 5000 replicates. Analysis at 12 months after randomisation, covering the period when most treatment occurred, suggests EUS had saved costs (but not quite significantly – see Table 41) and gained QALYs (but not significantly – see Table 41). Combining these findings we conclude that EUS is significantly cost-effective, in the sense that the probability of being cost-effective exceeds 95% at the NICE threshold of £20,000 to £30,000 per QALY when the cost of a scan is set at the national unit cost of £551 for day cases (Figure 22b). However this probability falls below 95% at the less plausible national unit cost of £1477 for outpatient scans (Figure 22d) and markedly below 95% at the much less plausible national unit cost of £3781 for inpatient scans (Figure 22f). We judge that a cost close to £500 is much more plausible because most trial participants received their scans as day cases and, more importantly, our own detailed analysis of the staffing and time needed to deliver EUS to trial participants as day cases was close to £500.

Analysis at 48 months after randomisation, covering our entire period of useful data collection, with costs discounted at 3.5%, suggests that EUS had saved costs (but not significantly – see Table 41) and gained QALYs (but not quite significantly – see Table 41). Combining these findings we conclude that EUS is significantly cost-effective at the two lower national unit costs – both £551 for day-case scans and £1477 for outpatient scans. However the probability of cost-effectiveness falls below 95% at the highest national unit cost of £3781 for inpatient scans. Whether or not one discounts QALYs does not affect these conclusions.

Table 24, using Cox regression, shows that participants reporting poorer health at baseline gained significantly more QALYs over a range of follow-up periods averaging 24 months than those reporting better health at baseline. We found a similar pattern, although less significant, in fully imputed 48-month QALYs (see Effectiveness, Quality-adjusted survival: sensitivity analyses for fully imputed data, and Appendix 6.18). Exploratory subgroup analysis at 48 months suggests that these sicker participants saved fewer costs than the healthier participants (but far from significantly – see Figure 25b). Combining these findings suggests that EUS is probably more cost-effective for sicker patients. Exploratory subgroup analysis also hints that EUS could be slightly better for younger patients. However these weak subgroup analyses do not detract from the unequivocal finding that EUS is almost certainly cost-effective for gastro-oesophageal cancer.

Process

We requested images from all seven sites which performed EUS scans within the trial and received images from five. Initially only two sites could record scans and forward them by file transfer protocol. Thereafter the trial agreed to buy recording equipment for any site that recruited ten patients into the trial. Variable recruitment meant that only one more site took up this offer. To extend quality assurance to more sites, we obtained still images from one site without recording equipment and, to maintain balance, another site that provided videos.

Figure 27 shows that we assessed 18 videos and two sets of stills (21% of scans performed in the study). Our quality assurance panel held five consensus conferences over the course of the trial, all web-based and audio-facilitated. Contributors included between two and four members but not always those who had provided pre-conference assessments. To maintain objectivity they remained blind to the source of the scans and other assessments until they had reached consensus.

FIGURE 27.

CONSORT ‘flow chart’ of EUS scans for web conferences.

Findings

Images varied in quality, and were generally less clear than an operator would see. The panel judged them adequate for staging, stills less so than videos. Over the five conferences they refined the pre-conference assessment form (see Appendix 8.2), for example by adding information about margins. On finding that members differed slightly in classifying borderline nodes, they agreed criteria for nodal involvement: diameter of 1 cm suggests a positive node, as does hyperechoic state. Where the coeliac axis was visible in images, the panel asked assessors to focus on coeliac nodes.

Table 43 displays original assessment, individual pre-conference assessments and blinded consensus. Before the conferences only 6 of the 20 scans showed general agreement between assessors across all stages. Nevertheless the panel reached consensus for all but one T stage and one N stage, including that two T stages and two N stages were inconclusive. For T stage there was a range of individual findings between T1 and T4 but none that varied by more than one from original or consensus; three scans individually graded as T4 became T3 in consensus. As M stage may depend on information outside the scan, both individuals and consensus often graded it as unknown.

Given the consensus that four scans were equivocal, there was no real disagreement between original staging and the final unblinded consensus for 19 T stages and 17 N stages out of 20 tumours (see Table 43, footnote ‘b’). Even before unblinding, the consensus agreed with the original operator about all T3 classifications, but there was some uncertainty about the boundary between T1 and T2, notably in scans K and R, the only two gastric tumours assessed by the panel. The panel could not see any tumour or node in K but excision revealed a T1 tumour. They could see an involved node but no tumour in R and S. In S, the only tumour originally scored as T1, biopsy had shown a tumour, and the operator had commented ‘difficult to see on EUS – slight thickening at 36 cm’; thus the panel understood the original minimum T stage but could not themselves find the tumour. There was occasional disagreement about nodal status: the panel changed Q from N1 to N0, S from N0 to N1, and remained confident that D was N1 even after seeing the original NX. Table 44 tabulates agreement on T stage between original and blinded consensus, and Table 45 does the same for N stage.

Thus there was excellent agreement between original observer and panel on T stage (weighted kappa = 0.866; p < 0.001), and moderate agreement on N stage (kappa = 0.562; p = 0.012). We also asked the panel whether or not each individual scan was a contraindication to resection. They judged that the nine scans that they agreed to stage as T3 N1, all but one in complete agreement with the original staging, made a strong case against resection.

Table 46 shows that staging by the external assessor, using an adapted video assessment form (see Appendix 8.3) and an anonymised summary of the original operator's proforma (see Appendix 8.1), differed more from the original operator than the panel did. The external assessor agreed with the T stages of 11 scans, but upstaged seven and was unsure about the other two. Of the 12 originally graded as T3, he agreed with seven but thought that five were T4. Hence he had a lower threshold for scoring pleural involvement. He upstaged two other scans from T2 to T3; and four nodal assessments, three in common with the consensus panel.

Three internal assessors re-examined five videos in the knowledge that the external assessor had scored them as T4 (Table 47). One assessor commented ‘Pleural involvement does not seem to be a factor in patient management at our MDT meetings although I appreciate it does alter the T stage from T3 to T4.’

Our assessors agreed that T4 was possible in four of these five scans. Once one assessor suggests pleural involvement, perhaps others are more likely to find it than if they viewed the films blind.

Of these five patients, four had chemoradiotherapy and one had neo-adjuvant chemotherapy before surgery. Hence there is no evidence from excised tumours that could confirm or refute the T4 scores. Thus the only corroborative evidence comes from survival. As mean survival was 371 days (median 188, range 141 to 810), it is unlikely that all had T4 cancers.

Comparison of endoscopic ultrasound staging with pathology of excised tumours

Our secondary outcome measures include ‘pathological reporting of resected specimens using SAGOC criteria’ (see Chapter 2, Design, Secondary outcome measures). To this end Table 48 compares staging before randomisation, staging by EUS and excisional staging for the only 21 intervention group participants who had surgery without prior treatment, as they alone can contribute without bias to quality assurance of EUS. As there may have been a gap of a few weeks between staging and resection, the true stage may have advanced. However spontaneous reduction in stage is so unlikely that preoperative overstaging is the more likely explanation for reductions in stage. Although we have pathology reports for tumours resected after chemotherapy or radiotherapy, they are biased by that intermediate treatment and cannot contribute usefully to quality assurance.

Of 21 participants in Table 48, four staged as T3 by EUS were T2 on histopathology, and two staged as T2 became T1; three staged as T2 were T3 on histopathology, one staged as T2/3 became T3, and one staged as Tis became T2. As one scan and one pathological examination were incomplete, only eight participants showed agreement on T stage. Of the 19 participants for whom EUS could estimate N stage, seven missed nodal involvement and two over-reported it – not a significant difference; thus only 10 patients showed agreement on N stage.

Again quality assurance needs the three-dimensional Tables 49 and 50 to focus on the same 21 uncontaminated participants undergoing EUS as Table 48 and to exclude participants receiving neo-adjuvant therapy, because intermediate treatment biases estimated stage; where T or N stage is missing from early staging, endoscopic staging or excisional staging, numbers fall below 21. Table 49 tabulates agreement on T stage between these three estimates, and Table 50 does the same for N stage. There is no statistical evidence here that EUS is a better predictor of histopathology than early staging is.

Psychometric analyses

From the FACT portfolio of outcome measures we combined existing gastric and oesophageal site-specific modules into a single comprehensive scale. We gave equal weight to the four subscales when combining them for FACT-G. Following internal analysis of FACT scales we dropped two items – one social and one functional – which did not perform well.

The psychometric analyses supported both our choice of EQ-5D as the primary means of adjusting survival in this trial and our decision to supplement it by FACT measures which include aspects of individuals' experience not captured by EQ-5D, namely their social and emotional well-being. Hence our effectiveness analyses used EQ-5D and two FACT summary measures: FACT-G, an average of the four subscales, and FACT-AC, combining the site-specific gastric and oesophageal modules. Structural modelling indicated that the alternative FACT summaries (TOI and FACT Total) were less appropriate than the two selected; while EQ-VAS correlated moderately well with EQ-5D and the physical and functional FACT scales.

Our original plan for analysing each FACT scale considered its own baseline values but not the baselines of the other scales. However the existence of a strong interaction between EUS and baseline EQ-5D in the QALY and 12-month EQ-5D models prompted us to include EQ-5D as an extra covariate in analysing FACT. To reduce the chance of false-positives we did not consider all baseline scale and subscale values as potential covariates.

Although comparisons of FACT between groups are not significant with or without interaction, those for FACT-adjusted survival favour EUS. There are highly significant interactions with baseline EQ-5D but not with the baseline of the corresponding FACT scale or subscale. Even when this is the only interaction in the model, the effects of EUS and baseline FACT-G on FACT-G at 12 months are unrelated. Although each baseline FACT has a considerable effect on the corresponding 12-month follow-up and on FACT-adjusted survival, that effect is present equally in EUS and non-EUS groups, and so does not affect the difference between them. Although the finding that EQ-5D is a better covariate for FACT than FACT itself is unusual, the findings from FACT are entirely consistent with, and serve to reinforce, those from EQ-5D.

Effectiveness

At the end of the trial, 44% of intervention participants, allocated at random to EUS, and 32% of control participants, allocated to usual staging, were alive. EUS improved survival adjusted for generic quality of life (EQ-5D), i.e. QALYs. The resulting hazard ratio was 0.705 (95% CI from 0.499 to 0.995), reflecting a difference in the estimated median quality-adjusted survival between 1.12 years in the intervention group and 0.94 years in the control group – a difference of 66 days in perfect health.

Both components of the composite primary outcome measure – survival and patient-reported HRQoL in the form of EQ-5D scores – also differed between groups. The estimated median survival time was 1.96 years in the intervention group, compared with 1.63 years in the control group – a difference of 121 days, albeit at reduced quality of life. Allowing for covariates would increase that difference; the corresponding hazard ratio was 0.706 (95% CI from 0.501 to 0.996) compared with 0.758 in a simple comparison. The mean participant-reported EQ-5D scores at 12 months were 0.509 in the intervention group and 0.449 in the control group – a non-significant difference of 0.061 (95% CI from £0.043 to 0.164). However the effectiveness of EUS did not differ between centres, cancer sites or cancer stages.

In contrast, the benefits of EUS were markedly greater for participants with poor initial quality of life; in statistical terms there was significant interaction between EUS and generic quality of life (EQ-5D) for all these outcomes. There was also significant interaction between initial generic quality of life and the effect of EUS on all relevant scales within FACT, both at 12 months and in the form of FACT-adjusted survival. Again sicker patients benefitted more from EUS. However there was no interaction between initial FACT scores and the effect of EUS on FACT-based outcomes, and no significant difference between intervention and control groups in mean FACT scores adjusted for covariates.

Both management plans and final treatment varied between centres. Although EUS changed the management plan for several participants, differences between groups in the proportion who received each treatment were not significant. Although the proportion of attempted resections that were incomplete was twice as high in the control group (20%) as in the EUS group (9%), this was not significant because there were only 108 real attempts at resection. Nevertheless the contrasting survival experience of intervention and control participants whose management plans changed is consistent with the hypotheses that: EUS identifies participants whose initial plans were not appropriate; and the resulting change of treatment contributes to improved survival and quality of life.

Cost-effectiveness

For both intervention and control participants we estimated the total cost to the NHS of health care during their time in the trial. We aggregated the costs they incurred in outpatient clinics, as day patients, during inpatient stays and through prescribing. Even allowing for the cost of the EUS scan itself, participants in the intervention group generally consumed fewer resources in secondary care and pharmaceuticals. The average total cost of care over 48 months was £29,200 (SD £14,900) in the intervention group but £32,000 (SD £22,000) in the control group. Hence allocation to EUS saved an average of £2860. The large standard deviations show that these cost data are skewed, as usual. So the best way to estimate confidence intervals is by ‘bootstrapping’, i.e. by resampling 5000 times from the original distribution of costs. This leads to an estimated confidence interval for the saving due to EUS of between −£2200 and £8000.

Bootstrapping is also useful for combining the data on quality-adjusted survival with those on resource costs. We used the same 5000 sample points to compare the estimated net benefit in QALYs from EUS with the estimated net cost of EUS. Of these, 3988 lie in the south-east quadrant of the cost-effectiveness plane where EUS both increases QALYs and saves NHS costs – best summarised as ‘win–win’ for EUS; a mere 46 points lie in the north-west quadrant where EUS both loses QALYs and increases NHS costs (‘lose–lose’); 640 points lie in the north-east quadrant where EUS improves benefits but increases costs; and 326 in the south-west quadrant where EUS makes savings but loses QALYs.

Since NICE sets the threshold for cost-effectiveness between £20,000 and £30,000 per QALY,148 bootstrapping leads to the conclusion that the probability that the EUS gives ‘value for money’ lies between 96.6% and 96.8%. Even if the NHS is unwilling to pay for an additional QALY, the probability that EUS is cost-effective is (3988 + 326)/1000, namely 86.3%. Finally, if the NHS were to set its cost-effectiveness threshold very high, there would still be (3988 + 640)/1000 = 92.6% probability that EUS would be cost-effective.

Quality assurance

This study showed the feasibility of undertaking quality assurance as part of a multicentre randomised trial of diagnostic technology. Five web-based consensus conferences allowed peer review of 20 anonymised scans (21% of 97 scans performed within COGNATE) that would not have been possible face to face because of the physical distances between reviewers. The conferences facilitated discussion, consistently leading to consensus among the panel representing six centres.

The process established that interpretation of scans varied between centres, and that peer review could alter final assessment of T (tumour) and N (nodal) stages. Most critical decisions relate to the diagnosis of N1 or T4 disease, which change management if the tumour has previously been diagnosed as less advanced (namely N0 or T3). Against this background the panel upgraded two scans to N1 and downgraded another to N0. They also reclassified one T stage from T2 to T0; thus it may have been suitable for a less radical resection such as EMR. Changes in M (metastatic) stage from M0 to M1 would also have changed clinical decisions but this did not occur within our web conferences. In short, variation between assessors had little effect on clinical decision-making, in particular whether to resect cancers.

To enhance this quality assurance we recruited an external assessor from another Commonwealth country who was very familiar with EUS scanning. Of the five tumours that the external assessor graded as T4, only one went on to resection. Our panel graded fewer scans as T4. Although two panel members graded three different tumours as T4, the panel later agreed that all three were T3. Hence there is benefit in reviewing scans when the decision is likely to affect clinical decisions.

The panel could not always verify the nodal assessment of the coeliac axis. Yet all but two of our consensus assessments stemmed from full EUS scans. This may highlight a training issue in using EUS in this context. Nevertheless kappa statistics showed a moderate level of agreement between assessors and original investigators on nodal involvement. This is important because the presence of nodal disease was the commonest reason why EUS changed treatment decisions away from cancer resection.

General

In 2001 the HTA programme responded to the systematic review by Harris et al. 143 by issuing a commissioning brief for a pragmatic randomised trial of EUS as an adjunct to the usual staging tests for gastro-oesophageal cancer. The successful bid came from an alliance between the leaders of SAGOC4 and a small team of trialists in the emerging North Wales Clinical School. Experience of SAGOC and the sparse literature had convinced them that the case for EUS was unproven.

Unfortunately the commissioning process took longer than usual, for reasons beyond the control of both the HTA co-ordinating centre and the COGNATE team. As a result COGNATE began recruitment 7 years after the systematic review that had triggered it. Not surprisingly most UK centres had by then decided that EUS should be part of routine practice for gastro-oesophageal cancer. The original COGNATE bid in 2002 had included genuine expressions of interest from eight centres. Another eight seriously considered taking part after COGNATE had achieved funding, often by inviting the trial team to present their case to the local MDT. Nevertheless in most centres there were clinicians whose personal experience had led them to believe that EUS was effective and who would not forgo EUS for any patient. However two centres – Aberdeen and Gloucester – remained in equipoise about the practical value of EUS and recruited participants effectively over 3.5 years. Another six centres allowed individual clinicians in equipoise to recruit participants as and when they could. By analysing these six as one centre, the COGNATE team has in effect delivered the proposed trial in the equivalent of three centres rather than the target of 10. Although the trial team has therefore done its best under difficult circumstances, it was arguably risky to proceed with the COGNATE trial when only two UK centres were truly in equipoise.

In the face of opposition to the proposed trial, the COGNATE team, supported by the TSC and DMEC, maintained its commitment to rigorous evaluation to underpin evidence-based practice. In particular we continued to encourage the remaining centres to recruit whenever they could. We also revised our power calculations in two ways. We responded to the initial realisation that many centres had already adopted EUS by replacing our primary outcome of survival by quality-adjusted survival, often called quality-adjusted life-years, for two reasons: (1) this reduced the target sample size from 700 to 400; and (2) QALYs is the criterion preferred by NICE. We judged that this was the best we could do to generate trial-based evidence in the face of the widespread adoption of EUS. As the number of actively recruiting centres continued to fall, we increased the target effect size for both survival and quality of life from 0.3 to 0.4 (still generally regarded as a ‘small’ effect). That enabled us to reduce the target sample size to 220. To achieve even this minimal target we had to extend the period of recruitment by 6 months – from 3 to 3.5 years.

Thus the main weakness of the COGNATE trial is that it was some 10 years too late. Committed to the practice of evidence-based medicine, however, we judged that rigorous evaluation was still essential. So we strove in four distinct ways to deliver the best trial we could in difficult circumstances.

First we used information technology in the form of laptop computers and file transfer protocols to facilitate data collection in all eight centres.

Secondly we complied with GCP throughout. Over the life of COGNATE we translated the principles of GCP into a portfolio of practical SOPs, thus laying the foundations of NWORTH, now a Registered Clinical Trials Unit. In particular, the economic analysis of COGNATE follows our SOP for economic evaluation alongside RCTs,141 based on the authoritative text by Glick et al. 129

Thirdly we successfully included an assiduous representative user within trial management – both TSC and DMEC – at a time when ‘patient and public involvement’ (sic) was in its infancy.

Finally we pursued the principle of ‘analysis by treatment allocated’112 in the face of a modicum of missing data, notably by following the four steps later advocated by White et al. 159

1. Try to follow up all randomised participants, even if they withdraw from allocated treatment.

We defined two levels of withdrawal – complete withdrawal and withdrawal from questionnaire interviews. Hence local researchers could stress that withdrawing from allocated treatment or active participation did not require withdrawal from data collection, and continue to collect data from hospital records. Thus only six participants withdrew completely and 25 partial withdrawals contributed to secondary analysis (see Figure 8). Rigorous data collection was generally successful in minimising missing data (see Chapter 2, Design, Electronic data collection and storage).

2. Perform main analysis of all observed data under plausible assumption about missing data.

Loss of data occurred mainly through censoring at the end of the trial, one to five years after randomisation. As it is safe to treat censoring as ‘missing at random’,129,131 both survival analysis for effectiveness and survival-based imputation for cost-effectiveness are valid (see Chapter 2, Statistical methods, Imputation and missing values).

3. Perform sensitivity analyses to explore departures from this ‘plausible assumption’.

Our main sensitivity analysis tested the validity of imputing missing data on the assumption that they are ‘missing at random’. The first confirmed that our primary 48-month cost-effectiveness analysis was consistent with secondary analysis restricted to the first year of the trial, and with the primary effectiveness analysis of quality-adjusted survival, both of which used little imputation. The second confirmed that estimated quality of life at later follow-up times yielded consistent findings whether analysed for all participants with imputation beyond the end of the trial where necessary, or else for all (dead or alive) who reached that time-point before the end of the trial (31 July 2009), which used less imputation.

4. Account for all randomised participants, at least in the sensitivity analyses.

Only two participants (one from each group) withdrew so soon that they could not contribute to any analysis; moreover, both primary analyses – effectiveness and cost-effectiveness – used 213 participants (96% of 223). So sensitivity analysis of the missing participants would thus have been trivial.

Trialists, notably Schwartz and Lellouch109 and the CONSORT group,110 have consistently argued that ‘analysis by treatment allocated’ (previously known as ‘analysis by intention to treat’) is the only unbiased approach to analysing trials. Nevertheless we undertook ‘analysis by treatment received’ as another form of sensitivity analysis (see Table 37). The 14 participants who did not receive EUS as allocated included three for whom there was no time to arrange appointments. The implication is that surgery was the unequivocal management plan. Hence transferring these three from the EUS (allocated) group to the non-EUS (received) group increases the proportion of changed plans in the former and reduces it in the latter, thus biasing analysis with changes in plan as criterion. Nevertheless the sensitivity analysis showed findings consistent with the primary analysis.

These developments in the conduct of randomised trials enabled the two main clinical centres to achieve high standards of recruitment and data collection and the team in Bangor to create a data set of high quality in a sick population. That COGNATE achieved largely unequivocal findings despite recruiting only a minimal sample is a measure of the rigour of these processes.

Effectiveness and cost-effectiveness

By changing our primary outcome prospectively, we identified significant positive findings, both for survival (the original primary outcome) and for quality-adjusted survival (the eventual primary outcome). How reliable are these findings in the light of the small sample? As the six centres outside Aberdeen and Gloucester contributed only 20% of participants, we evaluated EUS mainly in two UK centres still in equipoise about its role when we began the trial in 2005, perhaps the only two. These two centres remained in equipoise about the practical value of EUS and recruited participants enthusiastically and effectively over 3.5 years. The other six centres allowed individual clinicians in equipoise to recruit participants as and when they could. To get the most out of these 43 participants, we treated them as coming from a virtual third centre.

Nevertheless there were 120 potential participants who could not enter the trial because EUS was considered essential for them (see Figure 4). As a result, COGNATE validly randomised only 223 (47%) of 477 patients adjudged eligible to join the trial. So who was left in the trial? What does this mean for the routine use of EUS?

While the circumstances limiting the COGNATE trial to 223 randomisations were disappointing, only one of the 120 excluded because EUS was felt essential was in Aberdeen or Gloucester; these two centres therefore provide a representative picture of the care of gastro-oesophageal cancer under clinicians in equipoise. Thus COGNATE evaluated EUS wherever we could find equipoise about its value – specifically in two centres in equipoise and through individual clinicians in equipoise in six other centres.

Logically one would expect the other six centres to exclude from the COGNATE trial the patients considered likely to benefit from changed management plans following EUS; thus it would have been unethical for them to randomise these patients. In contrast, these centres were likely to randomise only those for whom there was uncertainty about likely benefit. This implies that the expected benefit of EUS for trial participants from those centres was less than for those whom they excluded from the COGNATE trial. Nevertheless we found no significant differences between this virtual third centre and Aberdeen or Gloucester. If we respect the clinical judgement of those who made these decisions to exclude potential participants, we can infer that the expected benefit in those excluded was even greater than estimated by COGNATE. In short, this argument suggests that the benefit of EUS for gastro-oesophageal cancer may be even larger than the significant positive differences – in survival, quality-adjusted survival and cost-effectiveness – estimated by the COGNATE trial.

Referees interested in exploring these findings asked for subgroup analyses. Therefore we repeated survival analysis for several clinical subgroups and economic analysis for subgroups based on initial health status and age (see Figures 25 and 26); with the exception of sicker participants in Figure 25, we found effects to be less significant than in the full sample. Otherwise we avoided subgroup analyses because they reduce the power of comparisons: larger differences or smaller hazard ratios are needed to establish significance in a subgroup than in the full sample. Instead, if effects do differ significantly between subgroups, this will appear as an interaction with the relevant covariate in the full sample. The only such significant interaction was with baseline EQ-5D. There is no hint in any of our various multivariate analyses that centre, tumour site or tumour type had any influence on outcome. Given that ANCOVA is more powerful than subgroup analysis, we judge that these non-significant findings provide further support for the generalisability of our findings.

In summary, faced by difficulty in recruiting to a trial of a health technology no longer in equipoise in most centres across the UK, we decided to seek a smaller sample than originally planned – capable of detecting standardised differences of 0.4 rather than 0.3. Then COGNATE achieved statistical significance in favour of EUS on all three of our main criteria – survival, quality-adjusted survival and cost-effectiveness. Hence, as the concept of statistical power is prospective, this strategy overcame the undoubted prior weakness of a small sample.

Quality assurance

We undertook quality assurance – successfully despite difficulties of recruitment. Unfortunately the absence of a true gold standard with which to compare the consensual process limited the value of this work. Not only was no pathology available for patients not resected, but chemoradiotherapy may have downstaged the tumours of those resected. Furthermore few in this field would advocate using EUS to reassess tumours after chemoradiotherapy because of the difficulty of distinguishing residual tumour from the fibro-obliterative process that occurs with successful chemoradiotherapy. 71,160 Hence quality assurance within COGNATE did not address the whole process of care, but concentrated on scans and their interpretation. As EUS was the intervention, it was important to check the quality of reporting.

The role of quality assurance in pragmatic trials like COGNATE is to characterise ‘best practice’, ensure that the intervention achieves and maintains it, and thus reassure observers that the trial is robust. So it needs, not to influence the research, but to optimise best clinical practice. Hence it needs to be independent of the research but close enough to the clinical activity to perform this role. So we chose our panel of six from the centres who undertook the COGNATE trial and added an expert endosonographer from another continent. Although we had hoped to receive 40 usable videos and sample a random half, we reviewed the only 18 usable videos and added two sets of still images, one of which was from a centre without video equipment. Thus we achieved our aim of reviewing 20 (21%) of 97 scans performed in the EUS group. This needed five web-based and audio-facilitated consensus conferences of our panel over the course of the trial, remotely validated by the international assessor.

If the resulting opportunity sample of 20 scans was close to random, then we can be 95% confident that the overall proportion of acceptable scans lay between 87% and 100%. As the opportunity cost of these high-technology consensus conferences exceeded £10,000, we believe that the lower confidence bound of 87% is sufficient for a pragmatic trial in which EUS also achieved significant benefits, not only in survival and quality-adjusted survival, but also in cost-effectiveness. Both the results and the consensus among contributors supported the proposition that quality of the scanning process within the COGNATE trial was reliable and generalisable.

Thus our prospective quality assurance programme achieved substantial consensus among six participating centres in inferring a common standard for reporting the TNM stages of gastro-oesophageal cancers through EUS. It reassured us about the quality of our intervention and helped us to meet the standards set for trials through GCP. It also fulfilled a useful educational role; for example, on finding they differed slightly in classifying borderline nodes, the panel agreed criteria for nodal involvement.

In short, we believe we improved consistency across centres and developed a model suitable for wider use after COGNATE. It is reassuring that other recent national trials, such as the US lung cancer screening trial reported by Gierada et al. ,161 have also monitored scans centrally. We therefore advocate central quality assurance for technologies like EUS, both in normal clinical practice and a fortiori in research.

Effectiveness

A combination of technical expertise and hard work enabled us to address many of the issues posed by a trial that was late and therefore small (see Strengths and weaknesses of COGNATE, General). Nevertheless COGNATE is not as generalisable as it would have been 10 years earlier. The effective study population comprised 70 participants from a teaching hospital in Aberdeen, 108 from a district general hospital in Gloucester, and a further 43 from centres that were not able to recruit in full. So it was reassuring to find no hint that the evidence from these three groups was at all heterogeneous. In particular we consistently tested: first whether the performance of EUS differed between the other centres and Aberdeen or Gloucester; and then whether it differed between Aberdeen and Gloucester; all without hint of statistical significance. Thus we are confident that COGNATE has estimated the true value of EUS in mainstream clinical practice. How could a trial that only just achieved minimal size achieve such clear benefits?

After EUS, one-quarter of management plans changed; participating clinicians explicitly attributed many changes in the intervention group to EUS. Although we had not expected much change in the control group, the proportion of changes in management was the same as in the intervention group. Those attributed to other diagnostic tests in the control group balanced those attributed to EUS in the intervention group, even though the results of most tests other than EUS had been available to the original MDM that had set the management plan. Given the encouraging findings about differences in outcome, notably survival, the apparent similarity in process was initially surprising.

Further analysis showed that participants allocated to EUS who then transferred to a therapeutic treatment, namely EMR or surgery in some form, survived much better than control participants who made this change, and better than intervention participants confirmed for one of these therapeutic treatments. The few intervention participants who transferred to conservative treatment – namely chemotherapy, radiotherapy or both – survived worse than control participants who made this change and worse than intervention participants confirmed for conservative treatment (see Table 29). In contrast, control participants who changed in either direction experienced intermediate survival, arguably because they lacked the discriminatory power of EUS. Furthermore intervention participants reported higher mean EQ-5D scores than the corresponding control participants. Finally, intervention participants who changed to multimodal treatments incurred lower costs than multimodal non-changers in the intervention group, possibly because many of them were already in advanced disease stages and hence died quickly; and intervention participants who changed to other treatments also incurred lower costs, arguably because their surgery or EMR was successful, leading to quicker recovery, less time in hospital and fewer recurrences. In short, changes were consistently more appropriate in the intervention group than in the control group.

Although all these unplanned analyses have low power, and are therefore not statistically significant, they stemmed from the need to explain significant differences in outcomes, yielded plausible reasons for those differences, and helped to explain them. They also yielded plausible explanations for the similar proportions of management changes in each group: changes arising from EUS were larger in effect and more discriminatory in nature. The better resection rates in the intervention group, even among those whose plans remain unaltered, confirm these observations. Our quality assurance data are consistent with the propositions that EUS triggers switches to better treatment and also helps surgeons undertaking resection and oncologists devising multimodal treatment without changing their basic plans. Furthermore our data on the use of NHS resources in both groups show that, although the findings of EUS usually led to action without delay, its absence led to additional tests and extended average stay by one-third. Although we have no data on the confidence of clinicians and participants in both groups, it is possible that some of the benefits of EUS stemmed from the psychological boost of prompt discharge, in addition to technical improvements in staging, management plans and resection.

The higher proportion of resections completed in the intervention group suggests that EUS improves selection for surgery by finding more resectable tumours. If so, that is one biological reason why EUS has a beneficial effect on survival and quality of life. The way in which changes are consistently more appropriate in the intervention group than in the control group suggests a more pervasive effect: even when EUS does not lead to a change of plan, it nevertheless enables clinicians to confirm their chosen plan, and helps them to implement it; and they convey the resulting increase in confidence to their patients. As this was a pragmatic trial, however, it did not directly address the mechanisms by which EUS might improve survival and quality of life. Our supplementary indirect analyses help to fill this gap.

Cost-effectiveness

Since 2008, NICE has recommended that the appraisal of health technology should consider benefits only to patients and resource costs only to the NHS. 148 Many have argued that benefits and costs to others, notably patients' families, are equally important in such appraisals. In recent elegant economic analysis, however, Claxton et al. 168 have underpinned the NICE policy, showing that for them to ignore the natural constraint on the NHS budget would lead to greater distortion in public decision-making than it corrects. Although we are conscious that in the field of cancer the NICE policy does generate anomalies, we have been happy to follow their prudent advice, not least because it simplifies analysis.

To cost NHS resources we used published national unit costs and refined these in the light of detailed consultation with the Nuclear Medicine and Finance Departments of Grampian Universities NHS Trust. This led us to estimate the cost of EUS as £550 for day patients, £1500 for outpatients and £3800 for inpatients. Hence we used sensitivity analysis to test whether this wide variation affected our conclusion about the cost-effectiveness of EUS.

The National Institute of Health and Care Excellence has suggested that interventions costing less than a threshold of between £20,000 and £30,000 (depending on circumstances) per QALY are likely to use NHS resources cost-effectively. 148 At all thresholds in this range EUS has a more than 95% chance of being cost-effective. Given that EUS appears to save costs, we also considered a threshold of £0 per QALY, which implies that the NHS is unwilling to pay for extra QALYs but insists that innovations such as EUS should finance themselves: even then it has more than 90% chance of being cost-effective. Finally we considered an infinite threshold per QALY – even that suggests that EUS is likely to be cost-effective (85% chance).

We also found significant interaction in cost-effectiveness analyses: participants with low baseline EQ-5D scores gained an average of 0.31 QALYs from EUS, whereas those with high baseline EQ-5D scores gained an average of only 0.018 QALYs (see Table 42). Nevertheless detailed analysis shows that EUS is probably cost-effective even for these healthier participants because they have lower costs. So there is no scientific reason to restrict access to EUS to patients who are less fit.

Synthesis

We judge that the consistency of our findings provides convincing evidence that EUS is effective and cost-effective. As we achieved the minimal sample size for our primary outcome, but only some secondary outcomes, we could not achieve statistical significance throughout. Nevertheless we judge that the combination of: increasing the complete resection rate from 80% to 91%; significant improvements in survival and quality-adjusted survival; and consistent improvements in patient-reported outcomes, in survival across subgroups defined by changes in management plans, and in resource use – all resulting in 95% probability that EUS is cost-effective – is conclusive.