A pragmatic multicentre randomised controlled trial comparing stapled haemorrhoidopexy with traditional excisional surgery for haemorrhoidal disease: the eTHoS study

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 08/24/02. The contractual start date was in September 2016. The draft report began editorial review in October 2016 and was accepted for publication in April 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Jonathan Cook reports membership of the Health Technology Assessment (HTA) Efficient Study Designs Board outside the submitted work; Malcolm Loudon states that between 2002 and 2004 his research fellowship was sponsored by Ethicon Inc. to carry out a randomised controlled trial evaluating rubber band ligation against stapled haemorrhoidopexy; he received no direct financial support. John Norrie reports membership of the National Institute for Health Research (NIHR) Journals Library Editorial Group; non-financial support from the HTA commissioning board; and is a member of the NIHR HTA Editorial Board.

Copyright statement

© Queen’s Printer and Controller of HMSO 2017. This work was produced by Watson et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2017 Queen’s Printer and Controller of HMSO

The eTHoS study: either Traditional Haemorrhoidectomy or Stapled haemorrhoidopexy for haemorrhoidal disease

In 2010, the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme funded the eTHoS (either Traditional Haemorrhoidectomy or Stapled haemorrhoidopexy for haemorrhoidal disease) study. The study evolved in response to a HTA programme-commissioned call issued in 2008 inviting proposals for a study. The call itself arose after completion of a HTA-commissioned and -funded evidence synthesis written by Burch et al. 2 One of the key recommendations was the need for a well-designed, multicentre, prospective randomised controlled trial (RCT) to inform the evidence for the management of haemorrhoidal disease. This report describes the research.

The eTHoS study was a major multicentre UK-based RCT investigating the clinical effectiveness (including safety) and cost-effectiveness of stapled haemorrhoidopexy (SH) and traditional excisional haemorrhoidectomy (TH) for adult patients with grade II, III or IV haemorrhoids.

This chapter reflects clinical practice, particularly within the NHS, and the evidence available at the time of study inception, and discusses changes in NHS practice and any relevant literature published since 2009.

Background

Participants

Potential participants were adults referred to NHS hospitals for consideration of surgical treatment and identified according to the inclusion and exclusion criteria specified below. Participants could have previously had RBL or HAL, but not undergone any other form of haemorrhoid surgery.

Outcome measures

Research ethics and regulatory approvals

The eTHoS study was given favourable ethics approval by the North of Scotland Research Ethics Committee (REC) on 18 June 2010 subject to receiving individual research approvals from each host organisation prior to the commencement of the study at site (reference number 10/20802/17). The study was co-sponsored by NHS Highland and the University of Aberdeen. The study was registered and was given an International Standard RCT Number (ISRCTN80061723).

Project management group

The study was supervised by a project management group (PMG). This consisted of the grant holders and representatives from the study office, who provided day-to-day support for the study. Observers were invited to attend at the discretion of the PMG. The PMG met every 6 months, on average, during the course of the study.

Study oversight committees

The study Data Monitoring Committee (DMC) comprised three independent individuals who met initially at the beginning of the study when terms of reference and other committee procedures were agreed. The committee then met an additional four times during the course of the study to monitor unblinded baseline and outcome data provided by the study statistician and details of serious adverse events (SAEs). The DMC reported any recommendations to the chairperson of the Trial Steering Committee (TSC). The TSC comprised four independent individuals who oversaw the study. At the invitation of the chairperson, grant holders and observers/members of the host university (University of Aberdeen) and the funder (HTA programme) could also attend. Terms of reference for the TSC were agreed at the commencement of the study and the group met annually throughout the duration of the study. The terms of reference for the DMC and TSC can be accessed on request from the eTHoS study office.

Patient and public involvement

An independent patient and public involvement representative was a member of the TSC. A patient representative was also involved in the grant application and throughout the study as a grant holder. Patient and public involvement was sought in the development of the participant newsletter. Both members reviewed the paperwork and provided constructive feedback in its development.

Participants

Study flow

The flow of participants through the study is detailed in Figure 1.

FIGURE 1.

Participant study flow.

Identification of patients and recruitment

Patients were identified by colorectal clinicians in participating sites, and were supported by local clinical research teams.

Colorectal clinicians assessed the eligibility of the participants for surgery according to standard practice. At the initial consultation, the colorectal clinician discussed the various treatment options available and provided information regarding the eTHoS study. Risks and benefits of all treatments were also discussed as part of standard clinical practice. Patients who had undergone RBL or HAL but still had symptoms were eligible for the study. Patients who had received any other form of haemorrhoid surgery were excluded from the study. Participants who expressed a strong preference for one of the interventions were also excluded. Identified eligible patients were then given a patient information leaflet (PIL; see Appendix 2), which contained details of the study and context. The clinician, or a trained member of the local clinical team, was able to answer any questions/discuss the study with the potential participants. Patients were encouraged to take time to read through the PIL and to take it home if they should wish. If the patient agreed to be contacted at home, then a member of the local research team would call to discuss the study. Patients who gave verbal agreement to take part in the study were then requested to complete documents [consent form and baseline questionnaire, which collected data on the EQ-5D-3L, the SF-36, the CIS and the HSS, and patient preference for surgery outcomes (see Appendices 2 and 3)] and return these by post to the local study team at their treating hospital or bring them when they returned to hospital for preoperative assessment or at the time of the operation. The consent form was then countersigned on receipt by the local study team member.

Patients who were able to make a decision to join the study while they were at the initial consultation were provided with the eTHoS study participant baseline questionnaire. This was completed only once a patient was eligible and happy to take part in the study and had signed the consent form. Contact details for both the local and central team were provided on the PIL. Further checking against eligibility criteria was performed by local research staff. Current clinical status was also captured on the baseline case report form (CRF) (see Appendix 3).

Following full written consent and baseline data completion, patients were randomised as near to their surgery date as possible. For those patients who did not consent to participate, an ‘Ineligible/Declined’ form (see Appendix 3) was completed by a local study team member, detailing non-personal data, including the reason(s) for the participant declining (if provided), or the ineligibility criterion. These data were recorded on the study database to inform the consort.

Randomisation

Participants were randomised to one of the two study groups on a 1 : 1 basis. The minimisation algorithm used centre, grade of haemorrhoidal disease (II, III or IV), baseline EQ-5D-3L score and sex. A remote telephone interactive voice response randomisation application hosted by CHaRT, HSRU at the University of Aberdeen was used to perform randomisation. The first participant was randomised on 13 January 2011 and the final participants were randomised on 1 August 2014 when the randomisation system was closed.

Study interventions

Eligible participants who consented were placed on the appropriate waiting list by the treating colorectal surgeon or his/her designated team member. Most participants received the allocated intervention, which was either SH or TH. Some received alternative treatment or did not receive treatment; however, participants were followed up regardless of the intent-to-treat analysis. Treatment data were captured on the hospital and surgery CRF (see Appendix 3). Each centre’s participating surgeons had to have undergone appropriate recognised training for both SH and TH. Surgeries were performed by surgeons in training; either independently, if signed off by their supervising consultant, or under the direct supervision of their consultant. Pre-operative and postoperative care followed the respective surgeon’s and centre’s standard care policies. Although the two procedures being compared are offered as routine care, not all sites provided postoperative information (what to expect after the haemorrhoid treatment) to participants. Therefore, we prepared a participant postoperative leaflet that sites could make available to participants if they wanted to (see Appendix 2).

Data collection

Follow-up consisted of clinical follow-up at 6 weeks after treatment. Postal questionnaires were distributed at 1 week, 3 weeks and 6 weeks after treatment, and 12 months (or 18 months if EQ-5D-3L not completed at 12 months) and 24 months after randomisation (Table 2). The main outcome assessment was planned at 24 months’ (from the date of randomisation) follow-up.

Consent was sought from all participants recruited to the study to be flagged for notification of haemorrhoidal recurrence. To evaluate long-term safety, the participants who had further haemorrhoidal surgery were identified via registry databases: Hospital Episode Statistics (HES) in England, Patient Episode Database Wales (PEDW) in Wales and Information Services Division (ISD) data in Scotland, when all participants had reached 12 months post randomisation.

Safety reporting

We reported SAEs in accordance with the guidance from the National Research Ethics Service, which is a subdivision of the National Patient Safety Agency. As such, non-SAEs were not collected or reported. All SAEs were recorded on the SAE report form (see Appendix 3). In addition, SAE forms recorded all deaths during the course of the study whether or not these were related to the research procedures. The following occurrences were potentially expected: intraoperative occurrences associated with the intervention, including anaesthetic-related problems, intraoperative instrument failure, damage to adjacent organs and bleeding. Possible (expected) occurrences associated with either type of surgery occurring at any time during the study included haemorrhage, requirement for blood transfusion, anal stenosis, anal fissure, pain, urinary retention, residual anal skin tags, anal fistula, prolapse, difficult defaecation, faecal urgency, wound discharge, pelvic sepsis, systemic complications and pruritus.

The details of any of the occurrences listed above were recorded on the CRFs and on participant-completed questionnaires and reported to the DMC in annual reports, but were not recorded as a SAE unless defined as an event occurring to a research participant that was related (resulted from administration of any of the research procedures) and was expected or unexpected, which caused death, was life-threatening, required hospitalisation, resulted in significant incapacity/disability or was otherwise considered medically significant by the investigators. Expected and unexpected SAEs were recorded.

Change of status/withdrawal

Participants remained on the study unless they chose to withdraw consent or if the principal investigator, chief investigator or study office felt that it was no longer appropriate for the participant to continue (i.e. the participant became unable to complete the study documentation). Reasons for participant withdrawal from the study were recorded on the ‘withdrawal/change of status’ form (see Appendix 3). Participants could withdraw from treatment but remain in the study for follow-up questionnaires and/or to have relevant outcome data collected from NHS records if consent remained for this.

Data management

Clinical data were collected at the individual hospital centres using hospital-based records and hard-copy CRFs (see Appendix 3). Clinical data were then input onto the eTHoS study database by local researchers using an electronic web-based data capture system [in addition relevant clinical data being collected from registry databases (HES, PEDW and ISD)]. Extensive range and consistency checks enhanced the quality of the data. Staff in the study office provided periodic data queries to local research staff to ensure that the data were as complete and accurate as possible. Questionnaires returned by post to the study office were entered onto the database by data-trained members of the central research team.

Data collected during the course of the research were kept strictly confidential and accessed only by members of the study team. Participants’ details were stored on a secure password-protected and user-specific database under the guidelines of the 1998 Data Protection Act,24 including encryption of any identifiable data. Participants were allocated an individual-specific study number and all data, other than personal data, were identified only by this unique study number.

The content of approximately 10% of CRFs and participant questionnaires entered at sites were checked to ensure accuracy. To ensure that sites had understood the data-entry processes and that they were efficiently entering the data, a sample from the first CRFs entered by each site were selected. Further paper CRFs and questionnaires were selected at random. This was completed at the study office (a copy of the paper forms were sent/faxed from the individual sites to the University of Aberdeen to facilitate this). An error rate of < 5% required no further action; however, if the error rate was > 5%, a 100% check of all forms from that site was undertaken.

Some questionnaires were returned directly to the University of Aberdeen. A 10% random sample of questionnaires returned to the study office were checked by a different data co-ordinator/clerk than the person responsible for the initial data entry, and the action on identified error rate was identical to that described above.

Statistical methods and study analysis

Economics methods

Study recruitment

In total, 777 participants were recruited for surgical treatment from 29 out of the 32 UK-wide NHS hospitals that took part in the study (Table 5). Twenty-seven of these centres randomised 389 participants to the SH arm and 28 randomised 388 participants to the TH arm. Participants were recruited to the study between 13 January 2011 and 1 August 2014 and followed up to July 2016; however, not all the centres recruited within this period, owing to staggered introduction and early closure of centres. The trajectory of recruitment from all centres is shown in Figure 2.

FIGURE 2.

Recruitment over time.

Participant flow

Figure 3 shows the Consolidated Standards of Reporting Trials (CONSORT) diagram for the study. In total, 1127 participants were screened and 1056 were found to be eligible. For the 279 excluded participants, 225 declined, 33 were excluded because of clinical reasons and 21 were missed. The main reasons for ineligibility resulted from the participants having had previous surgery (26.8%) and being unable to complete study questionnaires (23.9%). The most frequent reasons for participants declining resulted from treatment preference (22.2%) and not wanting surgery (14.7%). Further details on reasons why screened participants were not randomised can be found in Table 6. Three participants were excluded post randomisation. One was under the age of 18 years and two withdrew consent within a short period following randomisation. Therefore, 774 participants were included in the study analysis and presentation of findings.

FIGURE 3.

The CONSORT flow diagram.

All the participants randomised attended the baseline clinical assessment and completed the baseline questionnaire. A total of 31 participants withdrew from the study; one withdrawal was caused by the surgeon’s decision, whereas the remaining were at the participants’ request. Details of the treatment received are described in Chapter 5. Six weeks post surgery, 704 participants attended clinical follow-up appointments and 614 responded to the questionnaire. At the 24-month post-randomisation follow-up, 562 participants responded; a total of 31 participants withdrew from the study by that time point. Two deaths were reported, and none was caused by a study intervention.

Baseline characteristics

The participants’ baseline characteristics are shown for the 774 participants in Table 7 (excluding the post-randomisation exclusions). Sex, grade of haemorrhoids and EQ-5D-3L score were included as minimisation variables; therefore, the balance between the two treatment groups for these covariates was ensured. The median age was 50 years old and the quality-of-life outcomes (SF-36; physical and mental component summary) were balanced between the groups. Over 60% of participants had grade-III haemorrhoids, and 35.8% of participants in the SH arm and 30.1% in the TH arm had had previous haemorrhoid treatment. The majority of participants had either no preference for the type of surgery or preferred an intervention associated with a lower risk of haemorrhoid recurrence.

Table 8 summarises the level of experience of the surgeons and is broken down by surgical procedure and not randomisation group. The number of SH and TH operations performed in the previous year per surgeon are similar, whereas the number performed during a surgeon’s career differs.

Description of sample

The surgical procedures received by each treatment group are shown in Table 9. Of the 388 participants randomised to receive SH, 358 (92.3%) had surgery and, of these, 301 (84.1%) received their allocated intervention, 37 (10.3%) received TH, 18 (5.0%) had a technique other than the study interventions and two (0.6%) are unknown as they underwent private treatment. For the 386 participants randomised to receive TH, 363 (94.0%) had surgery, 323 (89.0%) received their allocated treatment, 29 (8.0%) received SH and 11 (3.0%) received other treatments. Across both arms 53 (6.8%) participants did not receive surgery, and the reasons are shown in Table 10.

Primary outcome

The EQ-5D-3L values at each time point are shown in Table 11. Figure 4 shows the profile for the two treatment groups over the study period, whereas Figure 5 shows the profile for the earlier follow-up time points. The SH arm had higher EQ-5D-3L scores for 1 week and 3 weeks, but this changed to higher scores in the TH arm from 6 weeks onwards. The primary outcome, EQ-5D-3L AUC over 24 months post randomisation, reported as QALYs (Table 12), favoured TH (–0.073 QALYs, 95% CI –0.140 to –0.006 QALYs; p = 0.034). The secondary analysis of AUC over 6 weeks post surgery favoured SH and the AUC over 12 months post randomisation showed no evidence of a difference between the two interventions. The analyses restricted to participants with at least one long-term measure were similar, and exploring the sensitivities to assumptions (per-protocol and MI analyses) gave similar results (Tables 13–16).

FIGURE 4.

Participant EQ-5D-3L scores up to 2 years. Participants need to have at least one short-term and one long-term follow-up score over 2 years.

FIGURE 5.

Participant EQ-5D-3L scores up to 6 weeks. Participants need to have at least one short-term follow-up score over 6 weeks.

Secondary outcomes

Adverse events

In total, there were 61 SAEs during the study; 59 participants [28 (7.2%) in the SH arm and 31 (8.0%) in the TH arm] had an event (Table 27). Table 28 gives details by randomisation group and Table 29 by the treatment received. There were two deaths in the SH arm, both unrelated to haemorrhoid surgery. One patient died while abroad 1 year after surgery, with the cause unknown; the other participant was randomised to receive SH but did not receive surgery, and the cause of death was ketoacidosis. The majority of SAEs by treatment received were related to pain (six in the SH arm and 10 in the TH arm), bleeding (six in the SH arm and one in the TH arm) and participants requiring catheterisation for urinary retention (seven in the SH arm and four in the TH arm). Two participants in each surgery arm had two episodes of pain, two required a blood transfusion; one had a low haemoglobin concentration prior to surgery and the other was readmitted 1 week post surgery with severe postoperative bleeding, which required extensive hospital stay. A combination of pain, constipation and bleeding occurred in a small number of participants, and one participant who received SH and two who received TH had pain caused by fissure.

Subgroup analysis

Two subgroup analyses were planned on the primary outcome (EQ-5D-3L AUC over 24 months): sex and grade of haemorrhoid.

A stricter level of statistical significance (two-sided 1% significance level) was applied given the exploratory nature of the analyses.

There was no evidence that any of the subgroups were statistically different at the 1% level; the results are shown in Tables 30 and 31.

Resource use

The details of the resources used for the surgical procedure, such as type of surgery, type of stapler used, operating surgeon grade, anaesthetist grade, type of anaesthesia, duration of operation and length of hospital stay, are reported (see Table 9). On average, the use of resources was similar across both arms. Details of the number of participants receiving further surgical interventions, such as removal of skin tags, TH, SH and RBL, are reported (see Table 21). The number of further interventions, was low over the 24-month period. For example, at the 6-week time point, eight participants received TH: six in the SH arm and two in the TH arm. Three participants (in the SH arm) and one (in the TH arm) received SH, while four (in the SH arm) and six (in the TH arm) had RBL. Five participants (in the SH arm) and four (in the TH arm) had further treatments for skin tags. The number of participants receiving non-surgical treatments, such as Botox and medical treatments, are reported (see Table 22). Further non-surgical interventions resource use was similar for both arms at 6 weeks post treatment. Further non-surgical interventions resource use was higher in the SH arm at 12 months (54 vs. 31 participants) and 24 months (39 vs. 19 participants). Table 33 below details participant-reported visits to outpatient department and GP. On average there were more visits in the SH arm.

The average surgery time used in the economic analysis was estimated using all available data. A few participants were missing either time of entry into the operating room and/or the time of leaving the operating room, so their time was estimated using available data. No data were collected on the time spent in the recovery room as it was assumed that this would have been similar in both treatment arms; it was therefore not included in the estimation of resource use.

Tables 34 and 35 detail the conversions of all the resource use into costs to the NHS and cost differences between the two arms. The estimates are reported in terms of NHS costs (see Table 34) incurred after the participant received the treatments. The mean total cost per patient in the SH arm was £921.80 (SD £586.90) and the mean cost of treatment in the TH arm was £620.65 (SD £582.98). There was a statistically significant difference in the (adjusted) total mean costs (£323, 95% CI £237 to £409) of the interventions.

The cost results reflect resource use results. A low level of resource use means that total cost data were skewed to the right, as most of the participants had low or no costs at all, but a few had high costs. Figures 6 and 7 illustrate the distribution of SH and TH costs, respectively.

FIGURE 6.

Distribution of cost data in the SH arm.

FIGURE 7.

Distribution of data in the TH arm.

Although cost of resource use measured during the intervention was similar in both arms, the mean cost of interventions was £272.62 higher in the SH arm (95% CI £239.74 to £305.66) because of the additional costs of the staplers (see Table 35). The costs reported at the 6-week visit were £25.34 lower in the SH arm (95% CI –£67.80 to £17.12), but not statistically significant. The other patient-reported costs between 6 months and 12 months were higher for SH (MD £32.96, 95% CI –£1.82 to £67.74) but not statistically significant, and the total 12-month costs were significantly higher in the SH arm (MD £308.50, 95% CI £237.08 to £379.93) than in the TH arm. The 12- to 24-month costs were significantly higher in the SH arm (MD £47.57, 95% CI £13.32 to £81.82). Total mean costs over the 24-month follow-up period were significantly higher in the SH arm (MD £323.38, 95% CI £237.11 to £409.64) than in the TH arm. The incremental differences are based on linear regression models, namely ordinary least squares with adjustments for baseline covariates, including baseline EQ-5D-3L score.

Quality-adjusted life-years

The EQ-5D-3L scores for the study arms at baseline, at 1 week, 3 weeks, 6 weeks, and at 12 and 24 months are shown in Table 36. The EQ-5D-3L scores were higher in the SH arm at 1 week and 3 weeks. However, the scores were lower at 6 weeks, 12 months and 24 months. From these data it was estimated that the mean QALYs over the 2 years were 1.676 (SD 0.384) in the SH arm and 1.738 (SD 0.334) in the TH arm.

The MD between the arms in EQ-5D-3L score at 1 week, 3 weeks, 6 weeks, and at 12 and 24 months are reported below (Table 37). The MD in EQ-SD-3L scores after adjusting for minimisation factors and baseline EQ-SD-3L scores was statistically significantly higher for SH at 1 week (MD 0.135, 95% CI 0.082 to 0.188) and 3 weeks (MD 0.05, 95% CI 0.008 to 0.091), but was significantly lower at 12 months and 24 months. The QALY difference was –0.071 (95% CI –0.127 to –0.016) and was statistically significantly lower for the SH arm than the TH arm.

Cost–utility results

The base-case analysis was based on multiple imputed data. The estimated costs and QALYs are reported below (Table 38). Total cost was £337 (95% CI £251 to £423). Total QALYs were lower in the SH group (MD –0.074, 95% CI –0.127 to –0.011). Both these differences are statistically significant.

The CEAC generated from the base-case cost-effectiveness analysis (Figure 8) shows that there is no probability of SH being cost-effective at the thresholds of £20,000 and £30,000.

FIGURE 8.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds.

The scatterplot (Figure 9) shows the point estimate and the distribution of the joint differences in costs and effects. The ICER point estimate and almost all of the bootstrapped estimates fall in the north-west quadrant of the cost-effectiveness plane, suggesting that SH is significantly more costly and less clinically effective than TH.

FIGURE 9.

Cost-effectiveness scatterplot illustrating the distribution of differences in costs and QALYs.

Cost-effectiveness results: number of recurrences averted

The results of the cost-effectiveness analysis based on the number of recurrences averted were similar to those of the base-case analysis (Table 39). On average, there were more recurrences in the SH arm and SH was more costly than TH. SH was dominated by TH.

Subgroup analysis

The results allowing for a treatment interaction with sex provide similar results to those of the base-case analysis. On average, SH cost £334.78 more than TH and had –0.069 QALYs fewer than TH. There was a 2.4% probability of being considered cost-effective at a £30,000 threshold (Figure 10).

FIGURE 10.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds for subgroup analysis investigating the influence of sex.

The ICER plot estimate falls in the north-west quadrant of the cost-effectiveness plane, suggesting that SH is more costly and less clinically effective than TH (Figure 11).

FIGURE 11.

Cost-effectiveness scatterplot illustrating the distribution differences of costs and QALYs for the analysis interacting treatment with sex.

The results of the subgroup analysis incorporating a treatment interaction with haemorrhoidal grade were broadly similar to those of the base-case analysis. Allowing for grade-specific effects, on average, SH had higher costs and lower QALYs than TH. However, the chance that SH might be considered to be cost-effective at the £30,000 threshold increased to 6% (Figure 12).

FIGURE 12.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds for subgroup analysis incorporating a treatment interaction with haemorrhoidal grade.

The ICER plot estimate falls in the north-west quadrant of the cost-effectiveness plane, suggesting that SH is more costly and less clinically effective than TH (Figure 13).

FIGURE 13.

Cost-effectiveness scatterplot illustrating the distribution of differences in costs and QALY for the analysis incorporating a treatment interaction with haemorrhoidal grade.

Sensitivity analysis

Complete-case analysis

The results of the analysis considering the complete cases (based on participants with both cost and QALY data) are presented in Table 40. On average, SH cost £287.87 (95% CI £190.17 to £385.56) more than TH and had –0.060 (95% CI –0.113 to –0.007) fewer QALYs than TH.

TABLE 40 - Estimation of cost-effectiveness (complete-case analysis)

Dominated = it costs more and is less clinically effective.

Trial arm	Costs (£), n, mean (SD)	Cost (£) difference (standard error), 95% CI	QALY, n, mean (SD)	Difference (standard error), 95% CI	ICER
SH (N = 389)	148, 863.91 (415)	–	148, 1.72 (0.33)	–	Dominateda
TH (N = 386)	149, 573.27 (507)	287.87 (47.33), 190.17 to 385.56	149, 1.77(0.30)	–0.060 (0.026), –0.113 to –0.007	–

The results of the complete-case analysis were broadly similar to those of the base-case analysis. On average, SH had higher costs and fewer QALYs than TH. The probability that SH might be considered to be cost-effective at the £30,000 threshold was 1.7% (Figure 14).

FIGURE 14.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds (complete-case analysis).

The ICER plot estimate falls in the north-west quadrant of the cost-effectiveness plane, showing SH to be more costly and less clinically effective than TH (Figure 15).

FIGURE 15.

Cost-effectiveness scatterplot illustrating the distribution of differences in costs and QALYs for sensitivity analysis (complete-case analysis).

Sensitivity analysis using Short Form questionnaire-36 items data

The utility and QALY scores derived from the SF-36D followed a similar pattern to those of the EQ-5D-3L (Table 41). On average, the 6-week utility score in the SH arm was –0.015 lower than that in the TH arm, but this did not meet statistical significance. However, the utility score was also lower at 12 months (–0.040) and 24 months (–0.034), and these differences were statistically significant (p < 0.05) (Table 42). The number of QALYs was –0.063 (95% CI –0.107 to –0.018) lower in the SH arm than in the TH arm.

TABLE 41 - Sensitivity analysis using SF-36 data converted to SF-6D data

Time point	Trial arm, n, mean score (SD)
	SH (N = 389)	TH (N = 386)
Baseline	376, 0.736 (0.144)	370, 0.745 (0.143)
6 weeks	255, 0.741 (0.142)	268, 0.747 (0.148)
12 months	254, 0.758 (0.154)	241, 0.796 (0.147)
24 months	239, 0.776 (0.144)	222, 0.800 (0.131)
QALYs	161, 1.540 (0.263)	156, 1.600 (0.238)

TABLE 42 - Sensitivity analysis using SF-6D data (MD at different time points)

Discounted at 3.5% QALYs.

Time point	MD (95% CI)	Standard error	t	p-value
6 weeks	–0.015 (–0.37 to 0.007)	0.010	–1.32	0.168
12 months	–0.040 (–0.070 to –0.010)	0.015	–2.73	0.012
24 monthsa	–0.034 (–0.062 to –0.006)	0.013	–2.52	0.019
QALYs	–0.063 (–0.107 to –0.018)	0.021	–2.92	0.008

The CEAC (Figure 16) indicates that there is only a 0.1% chance of SH being considered cost-effective at WTP thresholds of £20,000 or £30,000 (based on QALYs derived from the SF-6D). The ICER plot estimate falls in the north-west quadrant of the cost-effectiveness plane, suggesting that SH is more costly and less clinically effective than TH (Figure 17).

FIGURE 16.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds: sensitivity analysis using SF-6D data.

FIGURE 17.

Cost-effectiveness scatterplot illustrating distribution of cost and QALY differences for sensitivity analysis (SF-6D data).

Sensitivity relating to cost of staplers

Further analysis was undertaken varying the cost of the staplers and using the least expensive of those used in the study (Table 43). The QALY difference remains the same as before (–0.070, 95% CI –0.127 to –0.011) (see Table 38). The results were similar to those of the base-case analysis. For costs of ≥ £125, SH costs significantly more and has fewer QALYs than TH. SH remains marginally more costly than TH, unless the cost of the stapler falls to zero.

TABLE 43 - Cost differences at different costs of staplers

The cost difference indicates that SH costs less than TH (not statistically significant) and is less effective; therefore, it is not likely that it would be considered cost-effective.

Cost of stapler (£)	Cost (£) difference (95% CI)	Standard error	t	p-value	ICER
269	216.45 (123.62 to 309.28)	45.07	4.80	0.000	SH dominated
150	118.55 (30.83 to 206.28)	42.60	2.78	0.010	SH dominated
125	98.10 (11.14 to 185.05)	42.22	2.32	0.029	SH dominated
100	77.64 (–8.67 to 163.93)	41.90	1.85	0.076	SH dominated
75	57.18 (–28.57 to 142.92)	41.64	1.37	0.182	SH dominated
50	36.72 (–48.61 to 122.03)	41.42	0.89	0.384	SH dominated
45	32.62 (–52.62 to 117.87)	41.40	0.79	0.438	SH dominated
0	–4.20 (–89.0 to 80.59)	41.17	–0.10	0.919	60

The results of the analysis in which it was assumed that there was no additional cost of staplers suggest that SH has a 0.1% chance of being considered cost-effective at the £20,000 and £30,000 thresholds (Figure 18).

FIGURE 18.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds: assuming no cost of stapler.

The ICER plot estimate falls in the north-west quadrant of the cost-effectiveness plane, suggesting that SH is more costly and less clinically effective than TH (Figure 19). The cost difference estimate is lower (£216) than that of the base case (£323), so the ellipse is lower than that of the base case.

FIGURE 19.

Cost-effectiveness scatterplot illustrating the distribution of cost and QALY differences: sensitivity analysis assuming no cost of stapler.

Participant costs

The distribution patterns of the participant costs are similar to those of the main analysis (Figures 20 and 21). There are many participants with low costs and very few with high costs. The average participant costs were £31.48 (95% CI £5.00 to £57.00) higher for SH (£69.83, SD 229.33) than for TH (£38.35, SD 72.80). This further increases the total cost difference between the arms.

FIGURE 20.

Distribution of total participant costs in the SH arm.

FIGURE 21.

Distribution of total participant costs in the TH arm.

The cost–utility analysis based on the imputed data is reported in Table 44. On average, the cost indicates that the SH arm has £326.11 higher costs and –0.070 fewer QALYs than the TH arm. Like the base-case analysis, there was no chance that SH would be cost-effective at the £20,000 threshold and a 0.1% chance that it would be cost-effective at the £30,000 threshold (Figure 22).

TABLE 44 - Cost–utility analysis results for societal costsa

Societal means both NHS and participant costs.

Variable	MD (95% CI)	Standard error	p-value	ICER
Costs (£)	326.11 (215.02 to 437.20)	53.94	0	–
QALYs	–0.070 (–0.127 to –0.011)	0.027	0.11	SH dominated

FIGURE 22.

Cost-effectiveness acceptability curve showing the probability that SH is cost-effective, given the different values of WTP thresholds: sensitivity analysis including societal cost.

The ICER plot estimate falls in the north-west quadrant of the cost-effectiveness plane, suggesting that SH is more costly and less clinically effective than TH (Figure 23).

FIGURE 23.

Cost-effectiveness scatterplot illustrating the distribution of cost and QALY differences as per sensitivity analysis: including societal costs.

Discussion

Conclusion

The results of the within-trial analysis show that SH costs more and is less clinically effective than TH. These results were conclusive and supported by the sensitivity analyses and the fact that secondary outcomes such as tenesmus were more prevalent in the SH arm (p < 0.001), and more participants reported recurrence of haemorrhoids at both 12 and 24 months.

Background

Discrete choice experiments (DCEs) were introduced into health economics to value aspects of care beyond QALYs, and thus inform the delivery of public- and patient-centred care. 47,48 Research funding programmes such as the NIHR HTA programme (UK)49 are increasingly using applications of DCEs to assess what the public and patients want from the delivery of health care to inform key stakeholders such as policy-makers (NICE), researchers, NHS health professionals and the general public.

The DCE method can be used to assess the values of outcomes beyond QALYs in the delivery of treatment for haemorrhoids. Burch et al. 2 found that the outcomes of SH and TH differed in terms of short-term outcomes (earlier return to usual activities, pain) and the risk of recurrence. There was some concern that the recommended quality-of-life measurement (and QALYs based on it)25 may not fully measure the change in quality of life, as it would not capture all the aspects that represent patients’ preferences for treatments and their associated outcomes. The objectives of the DCE were to:

• investigate the outcomes of haemorrhoidal treatment that matter to individuals and their relative importance

• estimate the value of such outcomes using WTP, a monetary measure of value for outcomes

• compare costs and benefits in monetary terms, within a cost–benefit analysis framework, using this person and patient-centred measure of value

• compare the results from the standard cost–utility analysis approach (cost per QALY) with the cost–benefit analysis results.

The DCE included an additional surgical intervention, HAL, to take into account all the treatment options currently available to patients with haemorrhoids (and considered within the study).

Methods

Results

Results: web-based sample

Over all of the choices presented in the web-based DCE, 32% were for TH treatment, 30% for HAL treatment, 31% for SH treatment and 7% for no treatment. Very few respondents chose no treatment (8%). Table 48 reports the results of the DCE analyses.

The coefficients for treatment options (TH, 5.043; HAL, 4.995; and SH, 4.709) are all statistically significant and positive. This indicates that everything else being equal, respondents have a general preference for being treated over not being treated. However, the coefficients for TH, HAL and SH are not significantly different from each other, with CIs overlapping. This suggests that respondents did not distinguish between the three treatment options (everything else being equal).

The SDs for TH, SH and HAL (see the lower part of Table 48) are statistically significant, indicating preference heterogeneity for treatment options. Under the mixed-logit model assumptions in which TH, HAL and SH are normally distributed, these results still suggest that almost 100% of the population prefer treatment over no treatment. The coefficients for all the attributes are statistically significant and, as expected, negative. This indicates face validity for the model, with increases in all attributes reducing the utility and probability of take-up of a given treatment option. The magnitude of the coefficients show the impact of a unit change in the attribute. For example, a 1% increase in the chance of recurrence of haemorrhoids reduces utility by –0.568 and a 1% increase in chance of serious complications needing hospitalisation reduces utility by –0.369.

Results from eTHoS participants

The study participants’ results were similar to those of the web-survey group (Table 50). All treatment attributes were significant at the 5% level, suggesting that they all influenced preferences for haemorrhoidal treatments. Although the cost attribute was negative, it was not statistically significant. It was therefore not possible to calculate WTP values for the trial participants.

Cost–benefit analysis

Table 51 shows the results of the cost–benefit analysis. Total WTP for the two treatment options, SH and TH, can be defined as:

While the total WTP estimates were positive for both treatments (£1277 and £1367, SH and TH, respectively), the net benefit was higher for TH than SH (£765 vs. £330).

Discussion

Following the standard approach to valuation in trials (costs per QALY), we estimated a broader measure of value using the DCE. Following this, we generated a monetary measure of value that took account of factors important to users in the delivery of treatments for haemorrhoids (for the general population). Overall, respondents in both groups had a general preference to have treatment over no treatment, but did not distinguish between the three treatment options. However, they also had preferences for the attributes identified, with decreases in all four treatment attributes, as expected, reducing the benefit of treatment. The magnitude of coefficients indicated that the most important marginal change was chance of recurrence, followed by chance of complications.

An interesting finding was that cost was not significant to study participants. This may be explained by study participants having experienced treatment at zero cost to them (at the point of consumption). If DCEs are to be used to estimate a broader measure of value, this raises questions regarding who should complete the DCE and when. We leave this to future research.

The results of the within-study analysis reported in Chapter 6 suggested that the EQ-5D-3L captured differences in quality of life, and that TH costs less (–£337) and is associated with more (0.07) QALYs than SH. The standard cost–utility analysis thus suggested TH as the optimal treatment. These cost–utility analysis results were confirmed by the cost–benefit analysis, which indicated that TH had a higher INB.

Conclusions

Respondents from both the general population and the trial participants valued aspects of treatment beyond those captured in the QALYs. Respondents across groups also had similar preferences for treatment attributes. New haemorrhoidal treatments should aim at reducing the chance of recurrence and complications, as well as the time in postoperative pain and time taken to return to usual activities. The cost–benefit analysis results suggesting TH as the optimal treatment concur with the cost-per-QALY results.

Introduction

Response rates to the questionnaires administered at follow-up in the eTHoS study were routinely monitored at monthly in-house team meetings. Although response rates to the two early questionnaires were acceptable, there was no further contact with participants until the 12-month follow-up questionnaire and no contact between the 12- and 24-month questionnaires. The response rate to the 12-month follow-up questionnaire was observed to be lower than expected part way through the recruitment period. Options to improve the response rates were therefore considered by the study team. Literature reviews looking at methods to increase response rates were reviewed, and it was found that monetary incentives were likely to increase response rates. 56 However, there was limited evidence from studies in the UK and RCTs. It was therefore decided to conduct a formal evaluation of whether or not using a monetary incentive improved the response rate. This study was called ‘to incentivise or not to incentivise’ (IONTI). Surprisingly, the response rates continued to be low, and a need to protect the response rates in the main study was paramount; hence, the IONTI study was stopped early after 15 months. Subsequently, after a further review of the literature, a second incentive approach was implemented and its impact was assessed in the IONTI-2 study. The background, design and findings from both of these ‘studies within a trial’ (SWATs) are described in this chapter. Appendix 4 gives an overview of the timeline for both the IONTI and IONTI-2 studies.

‘To incentivise or not to incentivise’

In early 2013, the response rates to the 12-month follow-up questionnaire were found to be lower than expected at 69% (97/140 as of 25 February 2013); the expected response rate was 85%. Following discussion at the PMG meeting on 26 March 2013, it was decided that action needed to be taken in order to increase the response rate as much as possible. In order to choose the most effective intervention to increase the follow-up response rates in the eTHoS study, the team turned to the literature. A review published by The Cochrane Library56 evaluating strategies to boost response rates looked at 110 different strategies in 481 trials using postal questionnaires and 27 different strategies in 32 trials using electronic questionnaires. Although the eTHoS study follow-up data were collected using self-report by postal and electronic means, the largest proportion of responses were postal and, therefore, this mode of administration was also prioritised when designing the SWAT.

The literature review showed that in postal but not electronic questionnaire studies, monetary incentives were suggested to be the most likely to be effective in increasing response rates (OR 1.87, 95% CI 1.73 to 2.04), although there was significant heterogeneity among results. 56 The amount of the incentive might also affect the impact of monetary incentives. The review showed that in the 37 studies included, in which larger and smaller monetary incentives were compared, larger amounts were more effective. A further variation in studies using monetary incentives is whether the incentive is conditional or unconditional, that is, whether or not a voucher can be forwarded on the condition that the participant returns a completed questionnaire, or the voucher can be sent with the questionnaire to be completed and the participant does not have to do anything before receiving it. The review found that unconditional monetary incentives were more effective than conditional monetary incentives. However, very few studies used gift vouchers, few were clinical trials and only a small number were carried out in the UK; therefore, the generalisability of the findings could be questioned.

One study carried out, in the UK, by Gates et al. 57 showed a positive impact on response rates following the introduction of a monetary incentive, a £5 gift voucher, into their RCT on advice for whiplash patients. Participants (n = 2144) were randomised to two groups: one group received a gift voucher at both follow-up points and the other group did not receive any incentive. Participants in the former group (n = 1070) received the gift voucher unconditionally with their questionnaire at the 4- and 8-month follow-up points. The results showed that the incentive group was more likely to return the questionnaires (relative risk 1.10, 95% CI 1.05 to 1.16), an absolute increase of 7% of questionnaires returned. However, the authors acknowledged that further research was needed in order to establish whether or not the findings were generalisable to different populations and different types of trials.

In addition, the eTHoS study also presented an opportunity to assess the introduction of an incentive part way through a study to address the lower than anticipated response rate to a postal questionnaire; this is a scenario that triallists regularly face. Further steps were also taken in order to protect the response rate of the main study during the IONTI study; however, these were not randomised.

Based on the limited evidence available, the eTHoS study team decided to conduct a SWAT looking at the impact of an unconditional and, therefore, small £5 high-street gift voucher on questionnaire response rates at the 12- and 24-month follow-up points, and whether or not the timing of the incentive has an impact. The specific research questions are explored in detail in the following sections.

Aim

The overall aim was to assess whether or not sending an unconditional £5 high-street gift voucher improves the response rate to the 12- and 24-month patient questionnaires.

The specific research questions were as follows.

1. Does sending an unconditional £5 high-street gift voucher with the 12-month follow-up questionnaire improve the response rate to this questionnaire?

2. Does sending an unconditional £5 high-street gift voucher with the 24-month follow-up questionnaire improve the response rate to this questionnaire?

3. Is there any indication of an interaction when a £5 high-street gift voucher is sent at 12 and 24 months?

Methods: ‘to incentivise or not to incentivise’

Results

Two sites did not approve the amendment for the IONTI study and, therefore, participants from 27 sites were eligible and randomised in the IONTI study. One site objected because of the possibility that participants’ confidentiality might be breached as the sponsor is legally required to share details of individuals who have received payments if requested by HM Revenue & Customs, whereas the other site objected on ethical grounds.

In addition, one participant who was randomised to receive the voucher returned it, as he felt that as he did not receive surgical treatment, he was not eligible to receive the voucher.

Flow of participants

Figure 24 shows the flow of participants who were included in the IONTI study. There were 142 participants randomised to group 1 (voucher at 12 months but not at 24 months); 124 participants randomised to group 2 (voucher at 12 and 24 months); 123 randomised to group 3 (no voucher at 12 months but voucher at 24 months); and 132 participants randomised to group 4 (no voucher at 12 or 24 months). In total, 25 participants withdrew by the end of the study and one had died (unrelated). Baseline data are shown in Table 53.

FIGURE 24.

Flow of participants in the IONTI study. a, Participants eligible while the IONTI study was running from June 2013 until it was stopped in October 2014. Withdrawal refers to withdrawal from main study.

TABLE 53 - Baseline characteristics for participants in the IONTI study

BMI, body mass index.

Characteristic	Trial arm
	No voucher (N = 132)	12 months only (N = 142)	24 months only (N = 123)	Both time points (N = 124)
Age (years), n, median (IQR)	132, 50 (40–59)	142, 48 (38–58)	123, 50 (41–60)	124, 50 (42–61)
Sex (male), n (%)	62 (47.0)	73 (51.4)	64 (52.0)	63 (50.8)
BMI, n, mean (SD)	127, 26.8 (4.9)	137, 27.7 (5.2)	121, 26.6 (5.0)	120, 27.1 (5.3)
Grade of haemorrhoid, n (%)
II	27 (20.5)	37 (26.1)	27 (22.0)	26 (21.0)
III	85 (64.4)	83 (58.5)	81 (65.9)	78 (62.9)
IV	20 (15.2)	22 (15.5)	15 (12.2)	20 (16.1)
Previous haemorrhoid surgery, n (%)	37 (28.0)	52 (36.6)	45 (36.6)	31 (25.0)
Comorbidities, n (%)	1 (0.8)	–	2 (1.6)	2 (1.6)
Systemic medications, n (%)
Aspirin	7 (5.3)	1 (0.7)	7 (5.7)	7 (5.6)
Warfarin	2 (1.5)	1 (0.7)	1 (0.8)	1 (0.8)
Clopidogrel	2 (1.5)	1 (0.7)	4 (3.3)	2 (1.6)
Steroids	–	2 (1.4)	1 (0.8)	–
Other	3 (2.3)	–	1 (0.8)	1 (0.8)
Pain (VAS), n, mean (SD)	131, 2.6 (2.4)	140, 3.0 (2.5)	121, 2.4 (2.7)	121, 2.8 (2.9)
Analgesia, n (%)
Yes	42 (31.8)	44 (31.0)	40 (32.5)	37 (29.8)
Missing	1 (0.8)	1 (0.7)	3 (2.4)	2 (1.6)
Number of days in a week with analgesia, n, mean (SD)	42, 3.8 (2.0)	44, 4.3 (2.2)	40, 5.1 (2.2)	36, 4.4 (2.1)
EQ-5D-3L, n, mean (SD)	132, 0.812 (0.216)	142, 0.735 (0.264)	123, 0.771 (0.262)	124, 0.751 (0.266)
SF-36, n, mean (SD)
Physical component summary	130, 49.6 (9.3)	139, 48.5 (9.5)	119, 48.3 (10.0)	124, 48.3 (10.4)
Mental component summary	130, 50.4 (10.1)	139, 49.1 (11.1)	119, 50.2 (11.7)	124, 47.9 (12.3)
CIS, n, mean (SD)	129, 3.9 (3.6)	137, 4.5 (3.7)	118, 3.4 (3.7)	120, 4.6 (4.5)
HSS, n, mean (SD)	129, 12.9 (3.7)	134, 12.5 (3.9)	116, 12.5 (3.7)	120, 12.5 (4.3)
Patient reporting tenesmus, n (%)
Always	2 (1.5)	6 (4.2)	4 (3.3)	3 (2.4)
Often	18 (13.6)	18 (12.7)	14 (11.4)	14 (11.3)
Sometimes	36 (27.3)	36 (25.4)	33 (26.8)	28 (22.6)
Rarely	19 (14.4)	30 (21.1)	17 (13.8)	19 (15.3)
Never	57 (43.2)	51 (35.9)	50 (40.7)	60 (48.4)
Missing	–	1 (0.7)	5 (4.1)	–
Patient preference, n (%)
Strongly prefer better short-term recovery	11 (8.3)	12 (8.5)	11 (8.9)	10 (8.1)
Prefer better short-term recovery	11 (8.3)	11 (7.7)	6 (4.9)	3 (2.4)
No preference	42 (31.8)	57 (40.1)	51 (41.5)	47 (37.9)
Prefer lower risk of recurrence	33 (25.0)	38 (26.8)	28 (22.8)	28 (22.6)
Strongly prefer lower risk recurrence	35 (26.5)	23 (16.2)	24 (19.5)	34 (27.4)
Missing	–	1 (0.7)	3 (2.4)	2 (1.6)

In general, the groups were well balanced. Table 54 provides the IONTI study response rate results. There were no statistically significant differences between the groups at 12 or 24 months. Similarly, there was no evidence of an interaction effect (Tables 55 and 56).

TABLE 54 - The IONTI study: responses to questionnaires at 12 and 24 months

Response time point	Time point of voucher, n (%)
	12 months		24 months
	No	Yes	No	Yes
Response at 12 months
No	62 (24.3)	69 (25.9)	–	–
Yes	193 (75.7)	197 (74.1)	–	–
Response at 24 months
No	71 (27.8)	74 (27.8)	76 (27.7)	69 (27.9)
Yes	184 (72.2)	192 (72.2)	198 (72.3)	178 (72.1)

TABLE 55 - The IONTI study: ORs of responses to questionnaires at 12 and 24 months

Response time point	OR (95% CI)	p-value
Response at 12 months
Voucher at 12 months	0.99 (0.72 to 1.37)	0.962
Response at 24 months
Voucher at 12 months	0.93 (0.60 to 1.44)	0.760
Voucher at 24 months	0.85 (0.48 to 1.52)	0.586
Interaction	1.39 (0.69 to 2.79)	0.358

TABLE 56 - The IONTI study: before-and-after analysis to response to questionnaires at 12 and 24 months

Time point post randomisation	Responded to questionnaire, n (%)		OR (95% CI)	p-value
	Yes	No
12 months
Before	178 (72.1)	69 (27.9)	–	–
After	408 (77.4)	119 (22.6)	1.31 (0.84 to 2.04)	0.237
24 months
Before	30 (71.4)	12 (28.6)	–	–
After	532 (72.7)	200 (27.3)	1.01 (0.55 to 1.86)	0.980

The ‘to incentivise or not to incentivise’-2 study

Response rates remained lower, at 75% (375/498) for the 12-month questionnaire and 60% (127/212) for the 24-month questionnaire, than the expected rate of 85%, despite the fact that the IONTI study ran for 1 year. The eTHoS study team turned to the literature again for guidance.

A systematic literature review suggested that higher-value incentives were effective in increasing response rates to postal questionnaires in randomised trials. In their review, Brueton et al. 58 looked at randomised SWATs that compared strategies with increase response rates to follow-up questionnaires in RCTs. They identified 38 eligible SWATs, of which 14 tested incentives and two compared size of incentive. The two trials (n = 902) showed that those receiving a higher-value incentive were more likely to respond to questionnaires than those who received lower-value incentives (relative risk 1.12, 95% CI 1.04 to 1.22; p < 0.005), irrespective of how the incentive was delivered.

Based on the evidence reviewed above, the study team decided to conduct another monetary incentive study, IONTI-2. In the IONTI-2 study a high-value gift voucher (£30) was sent to participants who returned a fully completed 12- or 24-month questionnaire or associated reminders. A fully completed questionnaire was defined as one in which the primary outcome measure, the EQ-5D-3L, was completed.

Methods: the ‘to incentivise or not to incentivise’-2 study

Results

Discussion

Findings from the IONTI and IONTI-2 studies were inconclusive with regard to the overall question. The IONTI study was stopped early as a result of concerns that it was having a negative impact on retention, based on anecdotal information and interim results. The overall response rates for the main study (eTHoS) were prioritised over the value of continuing the IONTI study. Findings from the IONTI study were statistically inconclusive. The overall analysis of IONTI-2 was similarly inconclusive, including the sensitivity analysis, which might be viewed as the purer analysis (excluding the group that had received an incentive in the IONTI study), which also showed no difference in response rate at 12 months. The 24-month finding was also statistically significant, but there was a slightly higher post-incentive response rate in both analyses.

As both SWATs were found not to be effective, the cost of these incentives per participant has not been calculated.

Taken together, the findings were, at best, inconclusive, and, from the study team’s perspective, clearly disappointing. No positive effect was observed and there was the suspicion and possibility of a negative impact, despite previous work evaluating incentives as being very favourable. 56,58 It is possible, however, that the negative finding here was only a chance finding. Alternatively, they could reflect something specific to the study and the way in which the incentives were delivered. There are a number of possible contributing factors. Of particular note was the requirement of the sponsor to include a statement in the covering letter regarding possible tax implications in the IONTI study. Anecdotal feedback from participants suggested that this caused concern to some participants. This institutional judgement was later revised on appeal, and the cover letter used was modified accordingly most of the way through the IONTI study intervention period. The eTHoS study participants received multiple questionnaires early in the follow-up period with a number of similar outcomes, which may have resulted in lower response rates. Introducing a relatively low-value incentive at 12 or 24 months might be viewed as too little and/or too late. It was later hoped that the study team could carry out qualitative research to fully explore the cause of the negative impact. This was not followed through, because of the substantial regulatory barriers and lack of resources. Further research in this area is recommended, preferably in a variety of settings to allow disease- and population-specific cohorts, along with features of follow-up, to be explored.

Multiple strategies to improve recruitment and response rate are regularly used in multicentre trials in which recruitment and/or response rates are lower than hoped. Often these are used without much evidence to support their use. It was the investigators’ expectation at the initiation of the IONTI study that the use of incentives could be only positive (or, at worst, that they would have no impact). The findings from the IONTI studies question this assumption and points to the need for care when introducing new interventions part way through a study. Bower et al. 59 suggested that SWATs or ‘nested’ trials are needed to test the effectiveness of retention strategies. However, triallists should be aware of the challenges of implementing SWATs part way through pragmatic trials. As noted by others in Glidewell et al. ;60 ‘Interventions to increase response rates may incur negative consequences’ contrary to the intuitive expectation.

Both studies had a number of limitations in terms of the evaluation of an intervention. A confounding factor during the conduct of the IONTI study was the use of pre-notification newsletters, which were also employed as a method to boost retention. There is some limited evidence that these are beneficial to boosting response rates. 56 Newsletters were issued 1 week prior to the 12- and 24-month questionnaire being distributed across all IONTI study groups; as such, it was not possible to untangle what impact this would have had upon the overall study. The IONTI and IONTI-2 studies were carried out in a fully ‘open’ manner, with the interim results monitored as part of the routine study management process by the study team. This reflected their primary aim to address the lower-than-anticipated response rate, with the main study being the priority. The decision to stop the study was not according to a prespecified stopping rule, but a corporate judgement of the study team. As such, this monitoring process increased the possibility of a misleading finding.

The IONTI-2 study had a number of limitations. As a before-and-after study, its findings are susceptible to secular trends and other concurrent changes. In particular, some participants were involved in both studies, ‘a middle group’, which makes the interpretation more difficult. We carried out a sensitivity analysis that excluded this group; however, such exclusion of data is not without its own risks in terms of bias. More generally, recruitment to trials often has ebb and flows during the life cycle of a trial for various reasons, and this could have led to some bias being included in the comparison.

Participants in the IONTI-2 study were given an ‘opt-out’ option when receiving the £30 voucher. This was taken up by a small proportion [5% (24/481)] of participants, but the true ‘opt-out’ level may include some who did not respond.

Summary

The IONTI studies did not find any evidence in favour of the use of incentives to increase the response rate, and there was some suggestion of a negative impact. There are a number of contextual aspects that may explain this unexpected finding. Care is needed when introducing a new intervention into an ongoing study. Future evaluations of incentives are needed to explore the impact of contextual issues that may moderate their impact.

Introduction

Patients’ preference regarding treatment is well acknowledged to have a potential impact on the generalisability of randomised trials. 61 This can occur if an unrepresentative group of individuals (when compared with the patient population, who would potentially receive the treatments under evaluation) participate in the study. Perhaps less well recognised is the potential for patients’ treatment to have an impact on the study results. 62,63 Participants within trials who receive their preferred treatment may have different outcomes from those who do not, as they may be more motivated to continue participating in the study (e.g. complying with the treatment) or simply perceive their outcome as being better as a result of receiving their preference. Those who do not receive their preferred treatment may experience resentful demoralisation and be more likely to drop out of the study. 62

Various modifications to study design have been proposed to allow exploration of the impact of patients’ treatment preference. These include preference study design (a form of comprehensive cohort in which those who do not consent to take part in a RCT receive their preferred treatment) and the fully randomised patient preference design. 62,64,65 Various advantages and disadvantages of these approaches have been considered elsewhere. 62,65

Previous work has suggested, as noted in Chapter 1, that TH and SH procedures have different outcome profiles; one (SH) potentially leads to less pain and better quality of life in the short term, and the other (TH) potentially leads to less recurrence and need for further intervention. The eTHoS study aimed to evaluate the comparative effectiveness of these procedures over 24 months and, as shown in Chapters 5 and 6, this was confirmed by the study. Given this, there was reason to anticipate that patients’ preferences had the potential to have an impact on the study findings as well as on recruitment. 62 We were unaware of any assessment of the impact of patients’ preference for the treatment of haemorrhoids. To address this, we adopted a fully randomised patient preference design in which participants’ preferences were recorded at baseline, prior to randomisation. This enabled an analysis that takes preference into account. In this chapter, we report the findings of the fully randomised patient preference evaluation. Consideration of the impact of preference is briefly made in Chapter 5, in which reasons for not participating are recorded. An alternative evaluation of patient preferences was undertaken in the DCE reported in Chapter 7.

Methods

Results

Of the 774 participants randomised (after three post-randomisation exclusions), 766 completed the preference question. A total of 307 (40.1%) had no preference, with more participants strongly preferring operation B [159 (20.8%)] over A [61 (8.0%)] (Table 59). Preference groups were balanced between the two randomised interventions.

There was no sign of a difference in outcome between the participants’ treatment preference at baseline in terms of overall or disease-specific quality of life (Table 60). Those who preferred operation A (strongly or not) were slightly younger, and more men strongly preferred A.

The EQ-5D-3L data according to the participants’ preference across all time points are given in Table 61. At baseline, the groups differed noticeably in EQ-5D-3L score, although not in a manner consistent with a preference effect. As was observed in the main study analysis (see Chapter 5), the scores were lower at 1 week and 3 weeks post surgery. At 3 weeks, preference of operation A (but not strongly) had a higher score than the other preference groups, and at 12 months post randomisation, the score was lower in the strongly preferring operation A group. Results of the analyses that assessed the impact of the preference on the randomised group comparison of the primary outcome are given in Table 62.

The EQ-5D-3L preference analyses at 6 weeks and 12 months did not show any clear statistical evidence of an interaction between treatment and preference (at most one of the interaction effects was statistically significant at the 5% level). Although the observed difference within subgroups did differ somewhat, at 12 and 24 months they all tended towards favouring TH. The pattern was not consistent in terms of a preference effect.

The randomised treatment by participants’ preference analyses for HSS and CIS are given in Table 63. The CIS and HSS preference analyses at 24 months did not show any clear statistical evidence of an interaction between treatment and preference (none and one of the interaction effects, respectively, were statistically significant at the 5% level). All within-subgroup effects tended in the direction of favouring TH.

Table 64 shows the patient-reported recurrence preference analyses at 24 months, which did not show any clear statistical evidence of an interaction between treatment and preference (none of the respective interaction effects was statistically significant at the 5% level). All within-subgroup effects tended in the direction of favouring TH.

The compliance with treatment preference analyses (Table 65) did not show any clear statistical evidence of an interaction between treatment and preference (none of the respective interaction effects were statistically significant at the 5% level). All within-subgroup effects tended in the direction of favouring TH. Table 66 shows the association between compliance and retention and participants’ treatment preferences. There was some suggestion of a preference effect (for the ‘strongly prefer B’ group), which might have lead to differential retention.

Discussion

Patient preference was collected in the eTHoS study, prior to randomisation, by asking participants to give their preference for a hypothetical scenario that reflected general belief at the time of the study’s initiation. The majority of participants in the eTHoS study expressed a preference, with more participants favouring longer-term over short-term outcomes. It is worth noting that among all patients who would receive SH or TH, this proportion could well be higher, as some with stronger preferences are likely to not have consented to participate in the study.

Overall, there was no clear sign of a preference effect for any of the outcomes. This finding held for all of the outcomes (EQ-5D-3L, CIS, HSS, recurrence compliance and retention). There was also no sign of a difference at baseline according to preference group. Similar to other work, we have found that the overall RCT results were not modified by treatment preference. 66 There was some suggestion of an effect on retention for strong preference in favour of those who favour long-term reduced recurrence over short-term recovery.

Several trials have been undertaken using this approach to assess the effect of patients’ preference on outcome,63,67–69 with the proportion expressing a preference varying substantially. Findings regarding whether or not the existence of preferences has affected the treatment effect have been mixed. 62,63

Our study is one of the largest to look at within-trial preferences and to assess the possible impact on the treatment effect. 63 We are aware of only one previous fully randomised surgery preference study, which was reported for incontinence surgery. 70 None looked at preference for treatment of haemorrhoids. It may be the condition that explains the lack of preference or that the treatments were of a similar (surgical) nature, though they did differ in expected and realised outcome. The DCE reported in Chapter 7 explored preferences in a more subtle way and showed a specific preference for particular attributes (lower recurrence and complications). Differences in the included participants may explain this difference in findings, or that while there were some preferences for the nature of treatment outcome, this did not affect the treatment comparison of SH and TH.

There were a number of limitations to this work. First, the preference of participants was measured indirectly and fairly crudely. The terminology we used may not have produced a fully balanced response and contributed to some degree to the favouring of B over A, aside from solely the participants’ general preference for longer-term resolution even at the cost of poor short-term outcome. The response would link to the timings we used, which reflected the views on the relative impact of the operation and the anticipated last timing of outcome assessed (6 weeks and 2 years). We asked participants in the eTHoS study directly about any preference, whereas in other studies this was assessed by the health professional. 71

Summary

No clear evidence of a treatment preference effect was observed. These analyses further support the generalisability of the main study results and that TH has a better outcome over 24 months than SH. Further work exploring which types of treatment choices are more likely to have stronger associated patient preferences would be beneficial.

Summary of findings

The eTHoS study is one of the largest studies conducted in the field of haemorrhoidal disease worldwide. The findings from the study, along with those from HubBLe and LingaLongo,16,72 have added greatly to our understanding of the management of haemorrhoids. This study showed better overall generic quality of life over the 24-month follow-up period for TH over SH. Drawing on previous work concerning clinically important differences in the EQ-5D-3L, the magnitude of difference in favour of TH was found to be clinically important. The minimum important difference for the EQ-5D-3L has been estimated to be around 0.07, based on an anchor method applied to a range of patient populations. 73,74 The participants who were randomised to receive TH also reported fewer haemorrhoid symptoms and less incidence of further surgery in the 24-month follow-up period. Incontinence symptoms and tenesmus also occurred less frequently in this arm. In the immediate postoperative period, those who were randomised to receive SH reported less pain, but this difference disappeared by 6 weeks. Short-term post-surgery quality of life (measured by the EQ-5D-3L) scores also favoured SH, reflecting the lower rates of pain and analgesic consumption. At 12 months, the EQ-5D-3L AUC quality-of-life scores were similar; however, the physical and mental health domains of the SF-36 favoured TH (see Table 17). By the end of the 24-month follow-up period, the primary outcome favoured TH. This, along with lower residual symptoms, re-interventions and costs, indicates that TH may be the surgical treatment of choice for grades II–IV haemorrhoids. Overall, this suggests that TH is the better treatment option in terms of clinical effectiveness.

The operating time, length of stay and return to normal activity by 6 weeks were the same in both arms of the study. This differs from the previously published economic studies that suggested that a shorter operating time, and associated length of hospital stay after SH, offset the cost of a stapler. 2 This fact, taken alone, draws into question the continued use of the technique, as it implies higher costs for a similar surgical outcome. The ongoing use of SH within publicly funded health services is also challenged by the other findings from this study that demonstrate a higher recurrence rate, more tenesmus, higher cost and otherwise equivocal incontinence scores and complications. Even the reduced pain after SH was equivalent by 6 weeks postoperatively.

Strengths and limitations