Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

Article history

This issue of the Health Technology Assessment journal series contains a project commissioned by the MRC–NIHR Methodology Research Programme (MRP). MRP aims to improve efficiency, quality and impact across the entire spectrum of biomedical and health-related research. In addition to the MRC and NIHR funding partners, MRP takes into account the needs of other stakeholders including the devolved administrations, industry R&D, and regulatory/advisory agencies and other public bodies. MRP supports investigator-led methodology research from across the UK that maximises benefits for researchers, patients and the general population – improving the methods available to ensure health research, decisions and policy are built on the best possible evidence.

To improve availability and uptake of methodological innovation, MRC and NIHR jointly supported a series of workshops to develop guidance in specified areas of methodological controversy or uncertainty (Methodology State-of-the-Art Workshop Programme). Workshops were commissioned by open calls for applications led by UK-based researchers. Workshop outputs are incorporated into this report, and MRC and NIHR endorse the methodological recommendations as state-of-the-art guidance at time of publication.

The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Lisa V Hampson is an employee of Novartis Pharma AG (Basel, Switzerland) and reports grants from the Medical Research Council (MRC). Catherine Hewitt is a member of the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Commissioning Board since 2015. Jesse A Berlin is an employee of Johnson & Johnson (New Brunswick, NJ, USA) and holds shares in this company. Richard Emsley is a member of the NIHR HTA Clinical Trials Board since 2018. Deborah Ashby is a member of the HTA Commissioning Board, HTA Funding Boards Policy Group, HTA Mental Psychological and Occupational Health Methods Group, HTA Prioritisation Group and the HTA Remit and Competitiveness Group from January 2016 to December 2018. Stephen J Walters declares his department has contracts and/or research grants with the Department of Health and Social Care, NIHR, MRC and the National Institute for Health and Care Excellence. He also declares book royalties from John Wiley & Sons, Inc. (Hoboken, NJ, USA), as well as a grant from the MRC and personal fees for external examining. Louise Brown is a member of the NIHR Efficacy and Mechanism Evaluation Board since 2014. Craig R Ramsay is a member of the NIHR HTA General Board since 2017. Andrew Cook is a member of the NIHR HTA Interventional Procedures Methods Group, HTA Intellectual Property Panel, HTA Prioritisation Group, Public Health Research (PHR) Research Funding Board, Public Health Research Prioritisation Group and the PHR Programme Advisory Board.

Copyright statement

© Queen’s Printer and Controller of HMSO 2019. This work was produced by Cook et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2019 Queen’s Printer and Controller of HMSO

Synopsis

The aim of this document is to provide practical help on the choice of target difference used in the sample size calculation of a randomised controlled trial (RCT). Advice is provided with a definitive trial, that is, one that seeks to provide a useful answer, in mind and not those of a more exploratory nature. The term ‘target difference’ is taken throughout to refer to the difference that is used in the sample size calculation (the one that the study formally ‘targets’). Please see the Glossary for definitions and clarification with regard to other relevant concepts. To address the specification of the target difference, it is appropriate, and to some degree necessary, to touch on related statistical aspects of conducting a sample size calculation. Generally, the discussion of other aspects and more technical details is kept to a minimum, with more technical aspects covered in the appendices and referencing of relevant sources provided for further reading.

The main body of this report assumes a standard RCT design is used; formally, this can be described as a two-arm parallel-group trial. Most RCTs test for superiority of the interventions, that is whether or not one of the interventions is superior to the other (Box 1 provides a formal definition of superiority and of the two most common alternative approaches). A rationale for the report is provided in Rationale for this report and a summary of the research stages that were used to inform this report is provided in Development of the DELTA² advice and recommendations. Appendices 1 and 2 provide fuller details, which have also been published elsewhere. 9 The conventional approach to sample size calculations is discussed along with other relevant topics in Appendix 3. Additionally, it is assumed in the main body of the text that the conventional (Neyman–Pearson) approach to the sample size calculation of a RCT is being used. Other approaches (Bayesian, precision and value of information) are briefly considered in Appendix 4, with reference to the specification of the target difference. Some of the more common alternative trial designs to a two-arm parallel-group superiority trial are considered in Appendix 5.

Rationale for this report

A RCT is widely considered to be the optimal study design to assess the comparative clinical efficacy and effectiveness along with the cost implications of health interventions. 1 RCTs are routinely used to assess the use of new drugs prior to, and in order to secure, approval for releasing new drugs to market. More generally, they have also been widely used to evaluate a range of interventions and have been successfully used in a variety of health-care settings. An a priori sample size calculation ensures that the study has a reasonable chance of achieving its prespecified objectives. 10

A number of statistical approaches exist for calculating the required sample size. 1,11,12 However, a recent review of 215 RCTs in leading medical journals identified only the conventional (Neyman–Pearson) approach in use. 13 This approach requires establishment of the statistical significance level (type I error rate) and power (1 minus the type II error rate), alongside the target difference (‘effect size’). Setting the statistical significance level and power represents a compromise between the possibility of being misled by chance, when there is no true difference between the interventions, and the risk of not identifying a difference, when one of the interventions is truly superior, whereas the target difference is the magnitude of difference to be detected between sample sets. The required sample size is very sensitive to the target difference. Halving it roughly quadruples the sample size for the standard RCT design. 1

A comprehensive review conducted by the original Difference ELicitation in TriAls (DELTA) group14,15 highlighted the available methods for specifying the target difference. Despite there being many different approaches available, few appear to be in regular use. 16 Much of the work on identifying important differences has been carried out on patient-reported outcomes, specifically those seeking to measure health-related quality of life. 17,18 In practice, the target difference often appears not to be formally based on these concepts and in many cases appears, at least from trial reports, to be determined based on convenience or some other informal basis. 19 Recent surveys among researchers involved in clinical trials demonstrated that the practice is more sophisticated than trial reports suggest. 16 The original DELTA group developed initial advice, but this was restricted to a standard superiority two-arm parallel-group trial design and limited consideration of related issues. 20 It did not provide recommendations for practice. Accordingly, there is a gap in the literature to address this and thereby help improve current practice, which this report seeks to address. The Medical Research Council (MRC)/National Institute for Health Research (NIHR) Methodology Research Panel in the UK commissioned a workshop to produce advice on this choice of ‘effect size’ in RCT sample size calculations. From this resulting DELTA project, this report was produced. The process of its development is described in Development of the DELTA² advice and recommendations.

Development of the DELTA² advice and recommendations

The DELTA² project had five components: systematic literature review of recent methodological developments (stage 1); literature review of existing funder advice (stage 2); a Delphi study (stage 3); a 2-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of an advice and recommendations document (stage 5). Full details of the methods and findings are provided in Appendices 1 and 2 and are summarised here.

The project started in April 2016. A search for relevant documentation on the websites of 15 trial-funding advisory bodies was performed (see Appendix 1, Methodology of the literature reviews, Delphi study, consensus meeting, stakeholder engagement and finalisation of advice and recommendations). However, there was little specific advice provided to assist researchers in specifying the target difference. The literature search for methodological developments identified 28 articles of methodological developments relevant to a method for specifying a target difference. A Delphi study involving two stages and 69 participants was conducted. The first round focused mainly on topics of interest and was conducted between 11 August 2016 and 10 October 2016. In the second round, which took place between 1 September 2017 and 12 November 2017, participants were provided with a draft copy of the document and feedback was invited.

The 2-day workshop was held in Oxford on 27 and 28 September 2016, and involved 25 participants, including clinical trials unit (CTU) directors, study investigators, project funder representatives, funding panel members, researchers, experts in sample size methods, senior trial statisticians and patient and public involvement (PPI) representatives. At this workshop, the structure and general content of the document was agreed; it was subsequently drafted by members of the project team and participants from the workshop. Further engagement sessions were held at the Society for Clinical Trials (SCT), Statisticians in the Pharmaceutical Industry (PSI) and Joint Statistical Meetings (JSM) conferences on 16 May 2016, 17 May 2017 and 1 August 2017, respectively. The main text was finalised on 18 April 2018, after revision informed by feedback gathered from the second round of the Delphi study, the aforementioned engagement sessions and from funder representatives. Minor revisions in the light of editorial and referee feedback were made prior to finalising this report.

Target differences and the analysis of interest

Randomised controlled trial design begins with clarifying the research question and then developing the required design to address it. Commonly the population, intervention, control, outcome and time frame (PICOT) framework has been used for this purpose. 21 All of the relevant aspects of trial design (PICOT) should reflect the research questions of interest. The process of determining the design needs to be informed by the perspective(s) of relevant stakeholders, which are discussed in the following section (see Perspectives on the target difference of interest). A key step in the process is the selection of the primary outcome, which is considered in The primary outcome of a randomised controlled trial, given its key role in trial design and its relationship with the target difference. This section focuses on the need for clarity about how the design and intended analysis address the trial objectives.

The need for greater clarity in trial objectives with respect to the design and analysis of a RCT has been noted. 22 This reflects greater recognition of the existence of multiple intervention (or treatment) effects of potential interest, even for the same outcome. For example, we may be interested in the typical benefit a patient received if they are given a treatment, but also the benefit a patient receives if they comply fully with the treatment (e.g. they take their medication as prescribed for the full treatment period, with use of additional treatments). Treatment effects can differ subtly in the population of interest, the role for additional treatment or ‘rescue’ medication, and how the effect is expressed. The concept of estimands has been proposed as a way to bring such distinctions to the fore. An estimand is a more specific formulation of the comparison of interest being addressed. This thinking is reflected in a recent addendum to international regulatory guidelines for clinical trials of pharmaceuticals. Five main strategies are proposed. 23 Of particular note is the treatment policy strategy, which is consistent with what has often been described as an intention-to-treat (ITT)-based analysis. 22–25 That is, the ITT analysis addresses the difference between a policy of offering treatment with a given therapy and the policy of offering treatment with a different therapy, regardless of which treatments are received. Different stakeholders can have somewhat differing perspectives on the comparison of interest and therefore the estimand of primary interest. 22 Corresponding methods of analyses to address estimands that deviate from traditional conventional analyses are an active area of interest26 (see Appendix 3, Other topics of interest for a brief consideration of causal inference methods for dealing with non-compliance).

The target difference used in the sample size calculation should be one that at least addresses the trial’s primary objective and, therefore, the intended estimand of primary interest (with the corresponding implications for the handling of the receipt of treatment and population of interest). In some cases, it may be appropriate to ensure that the sample size is sufficient for more than one estimand, which might imply multiple target differences to address all key objectives. Different estimands may focus on different populations or subpopulations. Estimands will differ in their implications for the magnitude of missing data anticipated (see Appendix 3, Dealing with missing data for binary and continuous outcomes for how missing data can be taken into account in the sample size calculation in simple scenarios). Whatever the estimand of interest, the target difference is a key input into the sample size calculation.

Perspectives on the target difference of interest

The primary outcome of a randomised controlled trial

General considerations

Methods for specifying the target difference

The methods for specifying the target difference can be broadly grouped into seven types. These are briefly described below.

Overview of the case studies

A variety of case studies are provided for different trial designs, including varying types of primary outcomes, availability of evidence to inform the target difference and level of complexity. A short description is provided in Table 1.

Case study 1: the MAPS trial

Radical prostatectomy is carried out for men suffering from early prostate cancer. The operation is usually carried out through an open incision in the abdomen, which may damage the urinary bladder sphincter, its nerve supply and other pelvic structures. Urinary incontinence occurs in around 90% of men initially, but the long-term prognosis varies from 2% to 60%, depending on how incontinence is measured and time after surgery. Successive Cochrane systematic reviews101 have shown that, although conservative treatment based on pelvic floor muscle training may be offered to men with urinary incontinence after prostate surgery, there is insufficient evidence to evaluate its effectiveness and cost-effectiveness. Men After Prostate Surgery (MAPS)100 was a multicentre RCT that aimed to assess the clinical effectiveness (primarily by looking at the presence of urinary incontinence post treatment) and cost-effectiveness of active conservative treatment delivered by a specialist continence physiotherapist or a specialist continence nurse, compared with standard management, in men receiving a radical prostatectomy at 12 months after surgery.

The primary outcome was the presence of urinary continence. No other outcomes were considered. The sample size was based on a target difference of 15% absolute difference (85% specialist treatment vs. 70% control). A Cochrane systematic review101 suggested that the current control group proportion was 70% (average across relevant control groups). This magnitude of target difference was determined to be both a realistic and an important difference from discussion between clinicians and the project management group, and from inspection of the proportion of patients with urinary continence in the trials included in the Cochrane systematic review. 101 Setting the statistical significance to the two-sided 5% level and seeking 90% power, 174 participants per group were required, giving a total of 348 participants prior to considering missing data. Allowing for just under 15% missing data increased the overall sample size to 400. The power (77%) should the control group response turn out to be 40% (i.e. using 55% for the treatment and 40% for the control) was calculated as a sensitivity analysis. As the power was still reasonably high and this was considered a less plausible scenario, the overall sample size was not changed (see Box 4).

Case study 2: the ACL-SNNAP trial

Anterior cruciate ligament (ACL) rupture is a common injury, mainly affecting young, active individuals. ACL injury can have a profound effect on knee kinematics (knee movement and forces), with recurrent knee instability (giving way) the main problem. In the UK, a surgical management strategy has become the preferred treatment for ACL-injured individuals. However, the preference for surgical management (reconstruction) of the ACL-deficient knee had been questioned by a Scandinavian trial,115 which suggested that rehabilitation can reduce the proportion of acute patients requiring surgery by up to 50%.

A two-arm RCT – ACL Surgery Necessity in Non Acute Patients (ACL-SNNAP)115 – was planned to compare a strategy of non-surgical management, with the option of surgery if required (the rehabilitation group), with a strategy of surgical management only (the reconstruction group) in the UK NHS setting treating non-acute patients. The main outcome of interest was the Knee injury and Osteoarthritis Outcome Score (KOOS)-4, which excludes the activities of daily living component of the full KOOS. This decision reflected belief about the impact of ACL rupture and the aim of treatment. 115,116 KOOS-4 seemed to be the most appropriate of the available condition-relevant quality-of-life measures.

Limited work had assessed what would be a minimum important difference (MID) in the overall KOOS and the KOOS-4 variant. The KOOS user guide recommended 8–10 points as the (current) best estimate of a minimum important change. 117 This was based on an anchor method approach, using clinical judgement about the recovery timescale applied to a small cohort of ACL reconstruction patients. 118 Differences that occurred within the recovery period were ≤ 7 points, whereas those that occurred afterwards were ≥ 8 points for three of the four KOOS-4 domain scores. Given the limited data on what would constitute an important difference, estimates from a distribution-based approach [minimum clinically detectable change (MCDC)] were also considered. The MCDC was around 6–12 for individual domains. 119 A value of 8 points was taken as a reasonable value for the MCID in the KOOS-4 overall score.

A standard sample size calculation for comparing two means using a SD of 19 gave a required sample size of 120 in each group, for 90% power at a two-sided 5% significance level. This is how many patients would be required for an individually randomised trial in the absence of any clustering of outcome. The impact of clustering of outcome by the main intervention deliverer (surgeon and/or physiotherapist) was also considered. Given the time of outcome measurement (quality of life at 6 months), previous evidence suggested any clustering effect to be low: circa 0–0.06 for intracluster correlation (ICC) effect estimates from a database of previous surgical trials. 121 Clustering was assumed to occur to the same degree in both arms. Two surgeons from at least 13 sites were anticipated, whereas a priori more physiotherapists (at least 50% more, i.e. around 40) were anticipated to be involved in the study. Credible SDs for the cluster sizes were informally assessed using mock scenarios. Equal allocation was planned.

The sample size was estimated to be 130 patients per group to achieve just over 80% power, based on assuming an ICC of 0.06. With 26 surgeons, the number of patients per surgeon in the surgery management arm was expected to be five on average. With 40 physiotherapists, the number of patients per physiotherapist was expected to be three on average. Some allowance for variance in the number per health professional was also made. Given the anticipated challenges in recruiting to the study, keeping the sample size as small as possible was considered critical. As clustering was not certain, the sample size was increased to ensure at least 80% power if clustering occurred. In the absence of clustering, the power would be > 90%.

To allow for missing data, the sample size was set at 320 (allowing approximately 15% loss to follow-up). The total required sample size was therefore 320 patients. As the funding agency requested an interim check on the degree of clustering, a single planned interim check was set once data for 100 patients had been collected. This planned interim assessment would assess only the ICC magnitude and other sample size assumptions, such as cluster size. A formal interim analysis comparing treatments was not planned. Box 7 provides the corresponding sample size explanation in the trial protocol.

Case study 3: the OPTION-DM trial

A common comorbidity for patients with diabetes mellitus is neuropathic pain. Although there are some pharmacological treatments for this pain, it is unclear which is best. As the first-line treatment often does not work, patients may get second-line treatments as part of a care pathway. In the Optimal Pathway for TreatIng neurOpathic paiN in Diabetes Mellitus (OPTION-DM) trial,110 three care pathways were to be compared in a three-period crossover study. All patients would receive all three patient pathways. Each care pathway reflected a form of clinical practice that a patient might receive for their neuropathic pain. The main candidate primary outcome was the 7-day-average 24-hour pain after 16 weeks of treatment, measured on a numeric rating scale.

There was some experience of using such a pain score within the study team and in the published literature:

• A recent placebo-controlled crossover trial observed a 0.5-point average difference between the active comparator and placebo. 122

• Patients in this population on the active treatment were expected to improve from baseline by, on average, 2 points.

• A 1-point improvement within an individual patient was viewed as a clinically important difference, based on an existing study that used an opinion-seeking approach. 123

These criteria were used to inform the choice of a clinically important difference. The wish was to increase the proportion of patients improving by ≥ 1 point. The proportion of individuals improving can be calculated given the assumed reduction and difference between the groups. We expected a mean improvement of 2 points from baseline. Assuming that the change from baseline followed a normal distribution, 66% of patients were anticipated to improve by 1 point (relevant values are in bold text in Table 2).

If, for example, a clinically important mean difference of 0.5 points between treatments was the target (see bold text in Table 2 for relevant values), this would equate to a mean change from baseline of 2.5 points and 74% of patients showing a clinical improvement of ≥ 1 points in the active group. These calculations suggested a clinically important mean difference of 0.5 points, which we can equate to the proportion of individual patients showing individual clinical improvements of 1 point.

The calculation was then adjusted for multiplicity. Each care pathway was planned to be compared with each of the other pathways at the end of the trial. As three formal comparisons were planned, the Bonferroni adjustment was used to adjust the significance level to maintain an overall two-sided, 5% significance level. The sample size was calculated for 90% statistical power. See Box 8 for the corresponding sample size explanation presented in the protocol.

If the proportion of patients with an improvement of ≥ 1 point had been used as the primary outcome (i.e. dichotomising the pain score) and analysed accordingly as a binary outcome, for an effect of 8%, the corresponding sample size would have required a much larger sample size of 884 analysable patients (n = 1179, allowing for dropout).

Case study 4: the SUSPEND trial

Ureteric colic describes episodic severe abdominal pain from sustained contraction of ureteric smooth muscle as a kidney stone passes down the ureter into the bladder. It is a common reason for people to seek emergency health care. Treatments that increase the likelihood of stone passage would benefit patients with ureteric colic, as they will reduce the need for an interventional procedure.

At the time of planning, two smooth muscle relaxant drugs, tamsulosin (an alpha-adrenoceptor antagonist, or alpha-blocker) and nifedipine (a calcium channel blocker), known as medical expulsive therapy (MET), were considered potentially beneficial treatments. The Spontaneous Urinary Stone Passage ENabled by Drugs (SUSPEND) trial111 was designed to inform the treatment choice. A three-arm RCT was planned to compare tamsulosin and nifedipine with a placebo control to facilitate spontaneous stone passage.

A head-to-head comparison of the two MET agents, nifedipine and tamsulosin, was considered vital. A comparison of the two active arms (combined) with the placebo arm (MET vs. placebo) was also planned, due to uncertainty about the strength of the existing evidence of clinical efficacy. The key outcome of interest was the presence or absence of a stone at 28 days. It was defined as the lack of any further intervention (or planned intervention) to resolve the index ureteric stone.

A review of the evidence base approach was used. Data were available from two systematic reviews125,126 that included RCTs comparing alpha-blockers, calcium channel blockers and a variety of controls (placebo, treatment as usual or prescribed painkillers). 127–129 Only three RCTs compared tamsulosin and nifedipine directly, although there were a number of other trials that compared them to another treatment or a placebo. RCT data from both reviews were combined in a network meta-analysis to maximise the available data to inform the sample size calculation.

The estimated RR effects are shown in Table 3. For simplicity, the uncertainty around the estimates is not shown. The RRs of being stone free, comparing nifedipine and tamsulosin to the mixed control group, were estimated to be 1.50 and 1.70, respectively. Of particular note, the RR of being stone free for tamsulosin compared with nifedipine was estimated to be 1.15.

An estimate of the anticipated control (placebo) group event rate for being stone free was needed before the sample size could be calculated. This was estimated to be 50%, using a random effects estimate of the pooled proportion of the control arms of the RCTs from the two systematic reviews. This was then used as the placebo control group response in the sample size calculation, in lieu of better evidence that might be more relevant to the anticipated population. Using this and applying the corresponding RRs from the network meta-analysis, the stone-free level was anticipated to be 75% and 85% in the nifedipine and tamsulosin groups, respectively.

The study sample size was based on the comparison of the nifedipine and tamsulosin treatments. A standard sample size (for a two-sided, 5% significance level and 90% power with a continuity correction) for comparing two proportions gave a required number of 354 in each group. This sample size was inflated to 400 per group to account for an approximate 10% loss to follow-up. The total required sample size was 1200 (applying this size to the placebo group as well).

The placebo control group size was kept at 400 for the planned comparison with any MET (nifedipine and tamsulosin combined), which provided > 90% power. The size of the placebo group could have been reduced using an uneven allocation ratio, but was instead kept an equivalent size to the two active treatment arms. The funding agency strongly supported the inclusion of a placebo arm, given concerns about the potential risk of bias and the relatively small size of the existing placebo-controlled trials. No adjustment was made to the alpha level for multiple treatment comparisons (and therefore no inflation to the standard sample size), as the different comparisons were considered independent research questions: (1) MET compared with placebo control; and (2) nifedipine compated with tamsulosin. See Box 9 for the corresponding sample size explanation from the protocol. 111

Case study 5: the MACRO trial

Chronic rhinosinusitis (CRS) is a common condition, affecting around 10% of the UK adult population, that can lead to chronic respiratory disease or impaired quality of life. Initial management of CRS in the UK is in the family doctor setting, followed by referral to a hospital setting for medical treatment. Initial management fails to deliver sufficient relief for around one in three patients who attend hospital ear, nose and throat clinics. 130,131 The role of antibiotics for CRS is unclear, although they are commonly used in clinical practice. Endoscopic sinus surgery is a commonly conducted operation. Its use varies from centre to centre due to an insufficient evidence base. Two Cochrane systematic reviews132,133 of treatment of CRS with medical and surgical treatments highlighted the need for new randomised trials. Two main research questions related to treatment for patients with CRS were apparent to the investigators:

1. the relative benefits of surgical compared with medical treatment

2. the role of antibiotics.

Given the lack of clarity about current practice in the UK, two possible trial designs were considered potentially appropriate:

1. A two-stage trial incorporating two linked randomised comparisons:

   1. stage 1 – antibiotic compared with placebo for 3 months

   2. stage 2 – proceeding to receive endoscopic sinus surgery or continued medical therapy for those without significant benefit.

2. A three-arm randomised trial comparing antibiotic, placebo and endoscopic sinus surgery.

The relative merits of the study designs are not considered here. Instead, the focus is on specifying the target difference. The Management for Adults with Chronic RhinOsinusitis (MACRO) trial112,113 was designed to have a sample size sufficient for whichever of the two designs was ultimately chosen. An expert panel subsequently choice a variant of the three-arm design.

The primary outcome was the Sinonasal Outcome Test-22 items (SNOT-22), a validated disease-specific quality-of-life instrument. 134 An anchor approach was used to estimate the MID. A ‘medium’ SES (according to Cohen) was also calculated and used, as there is evidence to suggest that 0.5 SDs would be a reasonable estimate of the MID for this type of outcome. 135,136

Data from an existing study were used to infer what might be realistic to observe. 134 Limited work had assessed what would be a MID in the SNOT-22 score. Based on a large existing study of around 2000 patients receiving surgery for CRS with/without nasal polyps, which used the SNOT-22, a SD of 20 seemed plausible (group change score SDs were in the range of 19–20). An analysis adjusting for baseline was planned. A 10-point difference in the SNOT-22 (0.5 SD with SD of 20.0134) could be considered an important difference to detect.

The anchor method study suggested that the MID could be slightly smaller, at 8.9 points. This estimate was derived by calculating the average difference between those who stated post treatment that they were ‘a little better’ (9.5-point reduction) and those who stated they were ‘about the same’ (0.6-point reduction): 9.5 – 0.6 = 8.9 points mean difference. In the aforementioned study134 of surgical patients, the overall mean change score was 16.2, substantially larger than the aforementioned MID estimates. It is not realistic to expect all of this to be observed in a comparison of surgery with another treatment. If 25% (arbitrary, but based on the judgement of the team) of the response were attributed to regression to the mean or the process of receiving treatment of some kind, this would suggest that a difference of 12.2 might be plausible for surgery compared with an essentially non-effective treatment. A similar value for 15% of the effect was also considered (13.8).

The four mean values (8.9, 10.0, 12.2 and 13.8 points) reflected what might be clinically important differences (8.9–10) and realistic target differences (12.2–13.8) to use in the sample size calculations to look at various potential sample sizes under the two designs. A range of standard sample size calculations for a two-arm trial was produced, looking for 80% or 90% statistical power at the two-sided 5% level, using a pooled SD of 20.0 and assuming around 10% missing data (which was plausible based on two previous studies137,138 of patients in this area).

Case study 6: the RAPiD trial

Increased use of antibiotics is a major contributor to the spread of antimicrobial resistance. Dentists are responsible for approximately 10% of all antibiotics dispensed in UK community pharmacies. Despite clear clinical guidance, evidence demonstrates that dentists often prescribe antibiotics inappropriately in the absence of clinical need. The effectiveness of strategies to change the behaviour of health professionals is variable, but audit and feedback (A&F) has been shown to lead to small but important improvements in behaviour across a range of contexts and settings. The Reducing Antibiotic Prescribing in Dentistry (RAPiD) trial114 randomised all dental practices with responsibility for prescribing in Scotland (n = 795), using routinely collected Scottish NHS dental prescribing and treatment claim data [available through PRISMS (Prescribing Information System for Scotland)], to compare the effectiveness of different individualised (to dentists with practices) A&F interventions for the translation into practice of national guidance recommendations on antibiotic prescribing.

A total of 795 practices were randomly allocated to an intervention or the control (no A&F). Six hundred and thirty-two intervention group dental practices were subsequently evenly allocated to one of eight A&F groups in a 2 × 2 × 2 factorial design. The three factors were (1) receiving feedback with or without a written behaviour change message, (2) providing the graph of monthly practice prescribing levels with or without health board prescribing levels in the graph and (3) receiving feedback reports twice (0 and 6 months) or three times (0, 6 and 9 months). This led to a total of eight equal-sized intervention groups of 79 practices. The remaining 163 practices in Scotland formed the no intervention control group. The addition of this independent no intervention control group led to a ‘partially’ factorial design rather than fully factorial.

The RAPiD trial sample size calculation was unusual, as the population of sample units was fixed by the size of the country [i.e. every dental practice with responsibility for prescribing (n = 795) in Scotland was expected to take part, as part of a national policy to participate in dental service delivery research]. The sample size calculation was therefore based on identifying whether or not adequate statistical power could be achieved for the primary comparisons for target differences that were considered theoretically plausible (realistic) for a fixed size. The cost implications of a larger sample size were nominal and therefore the full population was always going to be used. The analysis was intended to be at the dentist level, adjusted for dental practice. However, the sample size calculation was carried out at the practice-aggregated level and was therefore conservative.

A systematic review139 demonstrated that the interquartile range of effects of A&F across different settings was 0.5–16%. The study team therefore determined that a 10% reduction (or less) would be both plausible and important. The routine prescribing data indicated that the mean number of antibiotic items prescribed per list was 141.1, with a SD of 140.9. Given that past prescribing behaviour is highly predictive of future prescribing data (correlated) both theoretically and empirically (p = 0.91 observed for the two most recent pre-intervention years), correction for the anticipated baseline correlation was used to reduce the precision. 140 A baseline-prescribing data-adjusted analysis was correspondingly planned.

With the sample size calculation for the A&F comparisons estimated, the study sample size for the comparison of A&F compared with no A&F was fixed by the number of dental practices left in Scotland that could be randomised to no A&F intervention. The detectable difference was 12%, which was still considered both plausible (realistic) and important, should it be observed. Given that the intervention group was being modelled twice within the two main hypotheses (intervention vs. no A&F and intervention factors vs. no intervention factors), Bonferroni’s adjustment was used to adjust the significance level to 2.5%, to maintain an overall two-sided 5% significance level. See Box 11 for the sample size explanation presented in the trial results paper.

Summary

Researchers are faced with a number of decisions when designing a RCT, the most important of which are the choice of trial design, primary outcome and sample size. The last is largely driven by the choice of the target difference for the primary outcome, although other aspects of sample size determination also contribute. It is important to be explicit and transparent about the rationale for and justification of the target difference when undertaking a sample size calculation and subsequently reporting it. The target difference, if well chosen, can be used in the interpretation of the trial results.

The DELTA² advice and recommendations on specifying a target difference for a RCT has been developed in response to a recognition that there is a need for practical advice to inform its choice. It was developed by a multidisciplinary group and was informed by multiple stages of preliminary work (see Appendix 1 for further details). This new document aims to address this need, and has used real-world case studies to illustrate how the target difference was chosen in clinical trials and to illustrate how the sample size calculation can be reported. It is intended to be of help to individual researchers and accessible to all relevant stakeholder groups. Engagement with funders is planned to maximise the potential value and usefulness to specific stakeholders. For the document to have the most impact, in terms of raising awareness of alternative approaches, the following would be beneficial: encouraging good practice in undertaking and reporting the sample size calculation, and endorsement from journals, funders of randomised trials and relevant professional groups.

Further research priorities

Producing corresponding recommendations for practice and reporting items for atypical sample size calculations, such as those that are simulation based due to their complexity, and studies that utilise an alternative statistical framework (such as Bayesian and/or decision theoretical) are needed to facilitate greater use of these methods.

Developing complementary recommendations for practice and reporting items that extend the work to cover additional alternative trial designs (e.g. multiarm, multistage, umbrella), which have not been explicitly covered.

Dedication

This work is dedicated to Douglas Altman, an inspirational researcher, friend and colleague.

Contributions of authors

Jonathan A Cook (Professor) drafted the initial version of the manuscript, and read and approved the final version.

Steven A Julious (Professor) drafted the initial version of the manuscript, and read and approved the final version.

William Sones (Statistician) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Lisa V Hampson (Associate Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Catherine Hewitt (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Jesse A Berlin (Vice President and Global Head of Epidemiology) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Deborah Ashby (Co-Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Richard Emsley (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Dean A Fergusson (Senior Scientist and Director) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Stephen J Walters (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Edward CF Wilson (Senior Research Associate in Health Economics) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Graeme MacLennan (Director and Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Nigel Stallard (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Joanne C Rothwell (PhD student) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Martin Bland (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Louise Brown (Senior Statistician) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Craig R Ramsay (Director and Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Andrew Cook (Consultant in Public Health Medicine and Fellow in Health Technology Assessment) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

David Armstrong (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Douglas Altman (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Luke D Vale (Professor) contributed to the development of the document, commented on the draft manuscript, and read and approved the final version.

Publications

Cook JA, Julious SA, Sones W, Rothwell JC, Ramsay CR, Hampson LV, et al. Choosing the target difference (“effect size”) for a randomised controlled trial – DELTA² guidance protocol. Trials 2017;18:271.

Bell M. New guidance to improve sample size calculations for trials: eliciting the target difference. Trials 2018;19:605.

Cook JA, Julious SA, Sones W, Hampson LV, Hewitt C, Berlin JA, et al. DELTA² guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial. BMJ 2018;363:k3750.

Cook JA, Julious SA, Sones W, Hampson LV, Hewitt C, Berlin JA, et al. DELTA² guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial. Trials 2018;19:606.

Sones W, Julious SA, Rothwell JC, Ramsay CR, Hampson LV, Emsley R, et al. Choosing the target difference (“effect size”) for a randomised controlled trial – the development of the DELTA² guidance. Trials 2018;19:542.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.

Disclaimers

This report presents independent research funded under a MRC–NIHR partnership. The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the MRC, NETSCC, the HTA programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the MRC, NETSCC, the HTA programme or the Department of Health and Social Care.

Synopsis

The details of the development of the DELTA² advice and recommendations have been reported elsewhere and are reproduced here. Methods and findings of stages 1–5 of the project, along with associated discussion, are given below.

Methodology of the literature reviews, Delphi study, consensus meeting, stakeholder engagement and finalisation of advice and recommendations

A summary of the methods used in each stage is given below.

Results

Stage 1: systematic literature search

The search identified 1395 potentially relevant reports (Figure 2). Following the screening of titles and abstracts, 73 publications were full-text assessed. Of these, 28 were included in the review as representing a development of one of the previously identified seven broad method types (Table 4 and see Appendix 2). Minor developments were identified for the health economic (including cost–utility and value of information), opinion-seeking, pilot/preliminary study and SES approaches. No new methods were identified. Most developments (17 articles) related to the use of variants of the value of information approach. A number of helpful review articles that summarise different methods and variations in application were identified; these covered willingness to pay66,67 and value of information167,168 health economic-based approaches, and estimation of the smallest worthwhile difference formulation of a MCID, which covered anchor, distribution, opinion-seeking and SES methods. 169 Identified articles on relevant topics (e.g. that address statistical aspects of sample size calculations or an existing method, but contain no new development) were considered as potential references in this document, irrespective of whether or not they were included in this review.

FIGURE 2.

Flow diagram.

TABLE 4 - Included studies from literature review of methodological development in methods for specifying a target difference

ENBS, expected net benefit of sampling; HE, health economic; VOI, value of information.

Study	Year	Journal	Method	Methodological development
Hedayat et al.142	2015	Biometrics	Anchor	Variation in threshold-based approach to estimating the MID
Rouquette et al.143	2014	Journal of Clinical Epidemiology	Anchor	Use of item response model approach to calculate MCID estimate from anchor assessment
Zhang et al.144	2015	Journal of Clinical Epidemiology	Anchor	Assessment of expressing MID as absolute and relative difference
Hollingworth et al.65	2013	Clinical Trials	HE (cost–utility)	Assessment of cost–utility-based sample size approaches
Chen and Willan145	2013	Clinical Trials	HE (VOI)	VOI for multistage adaptive trials from industry perspective
Andronis and Barton146	2016	Medical Decision Making	HE (VOI)	Expected value of sample information variant estimator
Breeze and Brennan147	2015	Health Economics	HE (VOI)	ENBS from pharmaceutical perspective using value-based pricing
Hall et al.148	2012	Medical Decision Making	HE (VOI)	Expected net present value of sample information approach
Jalal et al.149	2015	Medical Decision Making	HE (VOI)	Meta modelling approach to calculating the expected value of sample information
Madan et al.150	2014	Medical Decision Making	HE (VOI)	Efficient approach to calculating the expected value of partial perfect information (applicable to calculating the expected value of sample information)
Maroufy et al.151	2014	Journal of Biopharmaceutical Statistics	HE (VOI)	Method for calculating expected net gain of sampling
McKenna and Claxton152	2011	Medical Decision Making	HE (VOI)	Exploration of the role of value of sample information analysis
Menzies153	2016	Medical Decision making	HE (VOI)	Expected value of sampling information estimator
Sadatsafavi et al.154	2013	Health Economics	HE (VOI)	Expected value of sample information variant estimator
Steuten et al.155	2013	PharmacoEconomics	HE (VOI)	Comprehensive review of VOI methodological developments
Strong et al.156	2014	Medical Decision Making	HE (VOI)	Meta-modelling approach to calculating the expected value of sample information
Welton et al.157	2014	Medical Decision Making	HE (VOI)	Application of expected value of sampling information to a cluster trial
Welton et al.158	2015	Medical Decision Making	HE (VOI)	ENBS accounting for heterogeneity in treatment effects
Willan and Eckermann159	2012	PharmacoEconomics	HE (VOI)	Framework for exploring the perspective of societal decision-maker and industry
Willan and Eckermann160	2012	Health Economics	HE (VOI)	Accounting for between-study variation in value of information approach
Kirkby et al.78	2011	BMC Medical Research Methodology	Opinion-seeking	Survey approach to estimate MCID in trial setting
Ross et al.161	2012	BMC Medical Research Methodology	Opinion-seeking	Survey approach to estimate MCID with three treatment options
Chen et al.162	2013	Clinical Trials	Pilot/preliminary study	Comparison of approaches to using SDs from preliminary study (e.g. pilot or Phase II study)
Fay163	2013	Clinical Trials	Pilot/preliminary study	Variation in approach to using SDs from preliminary study (e.g. pilot or Phase II study)
Kirby et al.164	2012	Pharmaceutical Statistics	Pilot/preliminary study	Variation in approaches to discounting evidence from preliminary study
Sim and Lewis165	2012	Journal of Clinical Epidemiology	Pilot/preliminary study	Inflation factor for pilot study SD estimate for use in trial sample size calculation
Whitehead et al.88	2016	Statistics Method in Medical Research	Pilot/preliminary study	Assessment of size of pilot study needed to inform main trial
Valentine and Aloe166	2016	Journal of Clinical Epidemiology	SES	Alternative effect size metric proposed

Stage 3: Delphi study

Invitations to participate in the Delphi study were sent (by e-mail on 29 July 2016) to 58 methods experts, along with 104 named representatives of key trial groups (including the UK Clinical Research Collaboration network of CTUs, the MRC HTMR, NIHR/MRC/CRUK funding programme panels, the NIHR statistics group and the NIHR RDS). Of the 162 individuals invited to participate, responses were received from 84 (52%), of whom 78 (48%) accepted the invitation and six formally declined to participate. Acceptance of the invitation was allowed up to 10 October 2016 (the last acceptance was received on 4 October 2016).

The round 1 questionnaire was open for completion between 11 August and 10 October 2016. Of the 78 experts and representatives who agreed to participate, 69 (88%) completed the round 1 questionnaire once invited by e-mail, whereas nine did not complete it. The demographics of those who ultimately participated in the Delphi study are given in Table 5. Participants represented a range of RCT roles, with design, analysis and evaluating funding proposals well represented. The majority of participants (57 of the 69 who completed round 1; 83%) were primarily affiliated with an academic institution and the majority of participants were from the UK (55 of the 69 who completed round 1; 80%). Views on whether or not specific topics and alternative designs (i.e. not a ‘standard’ two-arm, parallel-group design) should be covered within the output document are given in Figures 3 and 4. Delphi participants showed strongest support (≥ 25%) for extensive coverage on alternative research questions and handling multiple primary outcomes. Across most topics there was 50–70% support for proportionate coverage, except for mechanistic studies and public and patient perspectives on the choice of the target difference. Regarding alternative study designs, the strongest support for extensive coverage was for adaptive designs, cluster randomised trials and multiarm trials (all > 25%). Across all designs there was 50–60% support for proportionate coverage.

TABLE 5 - Delphi participants’ demographics

Question	Response	Count (percentage of participants)
Your role in RCTs (select all that apply)	Involved in analysis of RCTs	42 (61)
	Involved in RCT design (collaborating clinician)	7 (10)
	Involved in RCT design (lead/chief investigator)	23 (33)
	Involved in RCT design (statistician/methodologist)	49 (71)
	Other (please specify)	16 (23)
	Serves on a funding panel/board which evaluates applications for RCT funding	43 (62)
Primary RCT-related affiliation	Academic institution	57 (83)
	Contract research organisation	2 (3)
	Funder of RCTs (e.g. NIHR in the UK or NIH in the USA)	5 (7)
	Health-care provider (e.g. NHS in the UK)	3 (4)
	Pharmaceutical/medical device company	2 (3)
Where do you work? If you work across Europe or internationally, please choose the category in which the majority of your work is performed	Canada	3 (4)
	Ireland	1 (1)
	Other European country	1 (1)
	UK	55 (80)
	USA	9 (13)
	Australasia	0 (0)
	Other	0 (0)

FIGURE 3.

Round 1 Delphi online questionnaire responses: specific topics to address within target difference recommendations.

FIGURE 4.

Round 1 Delphi online questionnaire responses: alternative trial designs to address within target difference advice and recommendations.

A total of 56 free-text comments were made, covering personal views on specific topics, views on the framing of research questions, and the audience that should be targeted for the advice and recommendations. Comments also included suggestions for additional trial designs to cover references and case study topics. The round 2 questionnaire was open for completion between 1 September and 12 November 2017. Only participants who completed round 1 were invited to participate in round 2, in which assessment of draft guidelines was required. Only two rounds were performed to fit with the project timescale and progress. Of the 69 participants invited to participate in round 2, 38 (55%) completed round 2. Findings from the round 2 questionnaire are summarised in Figure 5. Over 80% either ‘somewhat’ or ‘strongly’ agreed that the document was useful overall for the recommendations, case studies and appendices; 21 suggestions for improving the main text were made (11 regarding the case studies and nine on the appendices). In round 2, 62 free-text comments were provided, which, again, covered a range of suggestions for improving the main text, adding an executive summary, improving the signposting of sections, incorporating views on the case studies and appendices, additional references, raising the issues of estimands and personal views on various topics. Comments made in round 1 and 2 questionnaires, along with feedback from stage 4, led to a substantial number of changes to the document prior to its finalisation. The most substantive being incorporating an executive summary and increasing the number of case studies.

FIGURE 5.

Round 2 Delphi online questionnaire responses.

Discussion

Search strategy details

Search terms used were sample size or target difference or effect size or important difference or detectable difference or power calculation or value of information or value of perfect information or value of partial perfect information or value of sampling information or expected net gain.

The period searched was 1 January 2011 to 31 March 2016.

A search was performed of the articles from the following journals for relevant publications:

American Journal of Public Health, Biometrical Journal, Biometrics, Biometrika, Biostatistics, BMC Medical Research Methodology, Clinical Trials, Contemporary Clinical Trials, Health Economics, International Journal of Epidemiology, Journal of Biopharmaceutical Statistics, Journal of Public Health Medicine, Journal of Clinical Epidemiology, Medical Decision Making, Pharmaceutical Statistics, PharmacoEconomics, Public Health, Statistics in Biopharmaceutical Research, Statistics in Medicine, Statistical Methods in Medical Research, Trials and Value in Health.

List of included studies

For a list of included studies see Appendix 1, Table 4.

Findings from the review of relevant guidance

The findings from the review of relevant guidance is summarised in Table 6.

Sample size calculations for a randomised controlled trial

Statistical sample size calculation is not an exact, or pure, science. 32,172 First, investigators typically make assumptions that are a simplification of the anticipated analysis. For example, the impact of controlling for prognostic factors is very difficult to quantify and, even though the analysis is intended to be adjusted (e.g. when randomisation has been stratified or minimised),173 the sample size calculation is often based on an unadjusted analysis. Second, the calculated sample size can be very sensitive to the values of the inputs. In some circumstances a relatively small change in the value of one of the inputs (e.g. the control group event proportion for a binary outcome) can lead to a substantial change in the calculated sample size. However, the value used for one of the inputs (e.g. control group event proportion) may not accurately reflect the actual value that will be observed in the study. It is prudent to undertake sensitivity calculations to assess the potential impact of misspecification of key inputs (e.g. SD for a continuous outcome, level of missing data, etc.). This would also help inform decision-making about the continuation of a trial in which accumulating data suggest that the parameter will be substantially different from the one assumed in the main sample size calculation.

The role of the sample size calculation is to determine how many observations are required in order that the planned main analysis of the primary outcome, that is the one chosen to address the primary estimand of interest, is likely to provide a useful result. The sample size may also be chosen with reference to further key analyses (e.g. those focusing on other outcomes and subpopulations that address alternative estimands of interest). Most simply, this can be done by choosing the RCT’s sample size to maximise the number of participants required across the various analyses under consideration.

A variety of statistical approaches are available, although, overwhelmingly, current practice is to use the conventional Neyman–Pearson approach. This is so much the case that the specification of ‘effect size’, ‘significance level’ and ‘power’ are common parlance. The Neyman–Pearson approach is explained in Appendix 2 and the rest of this appendix assumes this approach is being used. Alternative approaches to the sample size calculation are briefly considered in Appendix 4 (see Appendix 4, sections Precision; Bayesian; and Value of information approach).

Often a simple formula can be used to calculate the required sample size. 174 The formula varies according to the type of outcome and, somewhat implicitly, the design of the trial and the planned analysis. Some of the simpler formulae are given in Binary outcome sample size calculation for a superiority trial; Continuous outcome sample size calculation for a superiority trial; Dealing with missing data for binary and continuous outcomes; and Time-to-event sample size calculation for a superiority trial, for the standard RCT design (i.e. a two-arm parallel-group RCT) and for the most common outcome types (binary, continuous and time to event).

Neyman–Pearson approach

The most common approach to the sample size calculation for a RCT is based on what can be described as the Neyman–Pearson, or conventional, approach. In essence, this approach involves adopting a statistical hypothesis testing framework and calculating the sample size required, given the specification of two statistical parameters (the power and significance level – see Glossary for definitions). This approach is sometimes referred to as carrying out a ‘power calculation’. This is a frequentist (as opposed to Bayesian) approach to answering the research question (see Appendix 4).

Although it is often not explicitly stated, this approach involves assuming a null hypothesis for which evidence to reject in favour of an alternative hypothesis is assessed. For a superiority trial with a standard design, the null hypothesis is that there is no difference between the interventions, and the alternative hypothesis is that there is a difference between them (i.e. one is superior to the other with respect to the outcome of interest). This leads to four possible scenarios once the trial is conducted and the data have been collected and analysed (Table 7).

There are two scenarios in which a correct conclusion is made and two scenarios in which an incorrect conclusion is made. The chance of these two errors is controlled by the statistical parameters, the significance level and the statistical power. Typically, the probability of the type I error (α) is controlled to be 0.05 (or 5%), which is achieved by using this level as the one with which it is concluded that the result is statistically significant (i.e. a probability of ≤ 0.05 is ‘statistically significant’ and > 0.05 is not). Additionally, this is usually a two-sided significance level, in that it is not prescribed a priori in which direction a difference might be found. In a similar manner, we can also control the type II error rate (β) by ensuring that the statistical power (which is simply 1 minus the type II error rate, i.e. 1 – β) is sufficiently large. Typical values are 0.8 or 0.9 (i.e. 80% or 90% statistical power).

It is worth noting that the presence or absence of a statistically significant result cannot be used to decide whether or not there is an important difference. Often the most that can be concluded from a non-statistically significant result is that there is no statistical evidence of a difference (i.e. a difference cannot be conclusively ruled out). Additionally, it is possible to have a statistically significant result even when the observed difference is smaller than the target difference assumed in a conventional sample size calculation. 175,176 This value can be readily calculated for a continuous outcome. Here, this is described as the minimum statistically detectable difference. It should not be confused with the MCDC or the minimum clinically detectable difference, which are entirely different concepts (see the Glossary for brief descriptions). Some recommend calculating and reporting the minimum statistically detectable difference, as well as the target difference and the required sample size. 176

Both the use of the 5% significance level and 80% or 90% power are arbitrary and have no theoretical justification, but are widely used. However, as excluding the possibility of either error is impossible, and the required sample size increases at a greater rate the closer either error rate is set to zero, these values have become the de facto standards. If well chosen, the target difference is a valuable aid to the interpretation of the analysis result, irrespective of whether or not it is statistically significant. It is essential when interpreting the analysis of a trial to consider the uncertainty in the estimate, which is reflected in the CI. A key question of interest is what magnitude of difference can be ruled out. The expected (predicted) width of the CI can be determined for a given target difference and sample size calculation, which is a helpful further aid in making an informed choice about this part of a trial’s design. 98

Given the assumed research hypothesis, the design, the statistical parameters and the target difference, the sample size can be calculated. Formulae vary according to the type of outcome (see Binary outcome sample size calculation for a superiority trial; Continuous outcome sample size calculation for a superiority trial; and Time-to-event sample size calculation for a superiority trial), study design (see Appendix 5 for some common alternative designs) and the planned statistical analysis (see Other topics of interest). The general approach is similar across study designs. In more complex situations, the frequentist properties (e.g. the type I and II error rates) can be estimated using simulations of data and consequential analysis of simulated results for scenarios in which there is and is not a genuine difference between interventions. 177

The conventional approach to sample size calculations is not without limitations. 168,178,179 Misinterpretation of findings, related at least in part to the statistical approach (such as what a p-value actually is and what can be inferred from it), has been highlighted and various proposal to improve practice have been made. 180 Nevertheless, the conventional approach to clinical trial sample sizes has remained remarkably persistent and is by far the most common currently used. 13,181 This reflects to some degree its ease of implementation and training, as well as the uncertainty about alternatives.

This appendix presumes the conventional approach is to be used for the sample size calculation for a two-arm trial with 1 : 1 allocation. Immediately below, simple formulae for the most common outcome types are provided. For completeness, Appendix 4 briefly summarises alternative approaches to calculating the sample size for a RCT. Statistical issues related to conducting a reassessment of the sample size under a conventional and a Bayesian approach are considered elsewhere. 1,182–184 Adaptive trial design (see Appendix 5) seek to formally incorporate potential changes to the design due to interim data into the trial design.

Binary outcome sample size calculation for a superiority trial

There are a number of commonly used formulae for calculating the sample size for a binary outcome for a superiority trial (i.e. for a study in which two proportions are to be compared). 1 One formula for the required number of participants per arm, n, for a standard trial (assumed equal allocation and therefore group sizes) is presented in Equation 1 and is relatively straightforward to calculate:

where n is the required number of observations in each of the two randomised groups. Z_1 – x is the value from the standardised normal distribution for which the probability of exceeding it is x. π_A and π_B are the anticipated probability of an event in groups A and B. α is the statistical significance level (i.e. the type I error rate), and β is the type II error rate and is chosen so that 1 – β is equal to the desired statistical power. The formula assumes even allocation between the treatment arms and a two-sided comparison.

The target difference can be expressed in multiple ways. It can be expressed as the absolute risk difference (π_B– π_A) or as a ratio, typically the RR: (2)(πB/πA) or OR: (3)(πB/(1−πB)πA/(1−πA)).

Different combinations of π_A and π_B can lead to the same OR or RR, although they may produce very different absolute risk differences. For example, a proportion of 0.4 compared with one of 0.2 represents a RR of 2 and a risk difference of 0.2. Proportions of 0.1 and 0.05 also represent a RR of 2, but the risk difference of 0.05 is far smaller and will require a far larger sample size. Whenever the target difference is expressed as a ratio, the anticipated control (reference) group risk, π_A, should also be provided.

The value assumed for π_A greatly influences the sample size. 1 In this context the control group proportion can be considered as a nuisance parameter with the target difference, δ, fixed regardless of what the control group proportion is. Estimates of this parameter may come from a pilot trial or existing literature (see Chapter 3, Pilot studies and Review of the evidence base). There needs to be an evaluation of the observed response dependent on the study design, population and analysis in the study from which it is being estimated. The planned analysis, particularly the summary measure used, is important for the calculation as adjusted and unadjusted analyses can be estimating different estimands. 185

Continuous outcome sample size calculation for a superiority trial

For ease of presentation, a slightly simplified formula124 to estimate the sample size per arm for a superiority trial with a continuous outcome is:

where Z_1 – β and Z_{1 – α/2} are defined as before, σ is the population SD and δ is the target mean difference. As before, the formula presented here assumes even allocation between the treatment arms and a two-sided test comparison.

In practice, σ is typically assumed to be known, with an estimate from an existing study, S, used as if it were the population value. The formula can be further simplified by replacing δ by δσ, the Cohen’s d standardised effect (d_SES):

Specifying the effect on the standardised scale, d_SES, is therefore sufficient to calculate the required n for a given significance level and power. However, it should be noted that different combinations of mean and SD values produce the same SES (Cohen’s d). See Chapter 3, Standardised effect size, for further discussion. Although sufficient for the sample size calculation, specifying the target difference as a standardised effect alone can be viewed as an insufficient specification as it does not define the target difference in the original scale.

A key component in the sample size calculation of a continuous measure is the assumed magnitude of variance. An estimate of this parameter (usually expressed as a SD) may come from a pilot trial or existing literature (see Chapter 3, Pilot studies and Review of the evidence base). It is possible to get into a ‘Gordian knot’ when looking for an estimate of the variance. Ideally, an estimate of the variance taken from a large clinical study in the intended trial population with the same interventions would be available. However, if such a study was available, a new trial would probably not be necessary. If a new trial is truly needed, that need implies some limitations in the existing evidence. To decide on the relative utility of the variance estimates, various aspects of the study need to be considered (e.g. study design, population, outcome, analysis conducted, etc.), in a similar manner to the control group proportion and any estimate of a realistic target difference (see Chapter 3, General considerations, Pilot studies and Review of the evidence base). 1,124 The accuracy of the variance estimate will obviously influence the sensitivity of the trial to the assumptions made about the variance and will also influence the strategy of an individual clinical trial.

A more accurate, although computationally more demanding calculation if performed by hand, will give a slightly different result from the formula above (see Equation 5) and is used in various sample size software. 186 The difference between the simple and more complicated formulae is that the simple calculation assumes that the population variance, σ, is known for the design and analysis of the trial. The more complicated calculation recognises that, in practice, the sample variance estimate, s, will be used when analysing the trial. The more accurate formula can be found elsewhere. 1

Dealing with missing data for binary and continuous outcomes

In most studies involving humans, it is likely that withdrawals, losses to follow-up and missing data will occur during the trial. 187 Individuals in a trial could decide that they no longer want to take part and completely withdraw from the trial, they could move during the study and not update the study team, and/or they could decide that they do not want to answer a particular question on a questionnaire. Even in the most well-designed and well-executed trial, some losses to follow-up are inevitable. Additionally, intercurrent events (e.g. death or change in treatment) may preclude the possibility of an outcome under the conditions implied by the trial’s aim and corresponding estimand of interest.

Irrespective of the reasons for missing data, sample sizes are frequently inflated to account for a degree of missing data during the study. The estimate of the extent of missing data is often gathered from a pilot trial, previous studies of the intervention, or trials in a similar population. In the presence of missing data, the power of a trial to detect the same target difference is reduced, hence the need for inflation of the sample size. To inflate the sample size to account for missing data, the overall sample size required, 2n, is divided by the proportion of data anticipated to be available for analysis (p_0b):

For example, if 20% attrition is anticipated, then the target sample size is divided by 0.8. A more complex and accurate approach can be used to deal with loss to follow-up over time, which is particularly pertinent for time-to-event outcomes.

It should be noted that adjustments such as above deal with the impact only in terms of precision of the missing data; a substantial number of missing data may also put the study results at risk of bias (e.g. if the reasons for attrition are related to eventual outcomes).

Time-to-event sample size calculation for a superiority trial

Owing to varying time of follow-up across study participants, it is not appropriate to analyse the proportion of participants who experience an event using logistic regression or a similar method. The analysis, and therefore the calculation of the sample size, for time-to-event data is also complicated by the fact that not all individuals will experience the event of interest. As a consequence, it is not appropriate to simply compare mean observation times directly between groups. There are three main approaches to the sample size calculation for this type of outcome:

1. compare Kaplan–Meier survival curves, using the logrank test or one of several other similar methods

2. assume a particular model form without specifying the survival distribution [e.g. the Cox (proportional hazards) regression approach]

3. use a mathematical model for the survival times and hence for the survival curve, such as the exponential or the Weibull distributions.

For ease of discussion, the term ‘survival’ is used to refer to the non-occurrence of the event by a specific time point and does not imply restriction of the methods to looking at mortality. The first two sample size methods are much more common than the third. For either a logrank- or Cox regression-based analysis, the analysis does not imply a specific distribution for the survival curve. The proportion surviving at any time point during the follow-up can be estimated to avoid having to assume one for the purpose of the sample size calculation. A target difference is inferred, explicitly or implicitly, for all of the methods. It is commonly expressed as a HR. 188 Similarly, to a binary outcome, adjusted and unadjusted analyses can estimate different estimands. 189

The difference between the two groups can be expressed as a difference between the survival probabilities at a specified time point. The data can be analysed accordingly, using the Greenwood standard errors to compare survival proportions. 190 However, this is statistically not a good way to compare groups, as it depends on the chosen time point and does not use the data on survival beyond that point. A method that takes all of the observed survival times into account, such as the log-rank test, is more convenient and statistically efficient. This is a test of statistical significance that has no explicit associated estimate of the treatment effect. Despite this, a power calculation can be performed by characterising the two survival curves by their median survival time, the time when half of the population in the group is estimated to have experienced an event.

To infer information about the survival curve from the median survival time, it must be assumed that the survival curve follows a known mathematical pattern, even though this assumption may not be used in the analysis. For example, the survival curve can be (and commonly is) assumed to be an exponential decay curve. The survival proportion (π_A) for treatment A at some time t can then be used to estimate the median survival time m, as follows:

Instead of a difference between mean times and the SD of times that would have occurred if we were comparing the average survival time in which all participants had reached the event, there are two median survival times or, equivalently, the median survival time in one group and the difference between medians, which can be considered the target difference. This is an implicit treatment effect size, although no such estimate is produced by the log-rank test.

Alternatively, an assumption about the difference between the survival curves, the proportional hazards assumption, can be made. This is the assumption that the ratio of the risk of an event in one group over a given short time interval, to the risk of an event in the other group over the same time interval, is constant over the follow-up period. This ratio is the HR and is the parameter that we estimate in Cox proportional hazards regression. This HR can be considered to represent the target difference (albeit on a relative range). However, another parameter is still needed to characterise the survival curve, such as the median survival time in one group.

It is possible to characterise the target difference either as the difference between median survival times or the HR, or by comparing events as an absolute difference in the event rate at a specific time point. Whichever approach is taken, the median survival in the control group or some similar parameter is needed to fully and uniquely specify the target difference. The statistical power of the comparison will depend on the total number of events rather than the total number of participants. A large number of events will imply high power. Participants who do not experience an event contribute little to the power. The median survival time and the planned follow-up time enable the number of events that will occur to be estimated.

Things become more complex if participants are recruited over a time period and then all followed up to the same calendar date. This results in widely varying follow-up times for censored cases. To allow for this, the recruitment period needs to be accounted for in the sample size calculation. If each participant will be followed for the same length of time, such as 1 year, the calculation is as if all were recruited simultaneously.

Methods for estimating the sample size usually rely on the number of events that need to be observed. The additional assumption of an exponential survival curve is typically made. Under these circumstances, the hazard, the instantaneous risk of an event, is a constant over time. The proportional hazards assumption is thus automatically satisfied. The HR can then be calculated as:

Again, under the assumption of an exponential survival distribution for both interventions, we can estimate the required number of events, e_A, in one group by:

This can be doubled for the other group, or the HR can be used to calculate the number of events in the other group. Having calculated the number of events needed, the number of participants required to produce this number of events can be calculated. To do so requires making further assumptions regarding the survival distributions for each group, the length of follow-up and any censoring of data. A varying length of follow-up according to the accrual pattern is also typically assumed to make maximum value of those recruited early in the recruitment period and avoid unnecessarily extending the follow-up of the final participants. This issue is beyond the scope of this document, but further discussion can be found here. 1 Sample size calculations which allow for non-proportional hazards are also possible. 191

The target difference for this type of outcome can be variously expressed as a difference between the median survival times (m_B – m_A), the difference between the proportions surviving at a particular point in time (π_B – π_A), or the HR (which might vary over time). It is worth noting that, as for the other outcome types, how intercurrent events (e.g. change in treatment) are dealt with needs careful consideration and, similarly, the reasons for censoring needs assessing, as not all may be viewed equally. For example, if death is not the event of interest, then the occurrence of a death leads to censoring of the outcome of interest. However, it may be viewed as indicative of the likelihood of such an event occurring or as precluding the event from occurring with no impact on the likelihood (e.g. the death of someone who has had a knee joint replacement precludes the failure of the device due to wear and tear). The handling of such occurrences in the analysis and the corresponding impact on the sample size (and, in this context, the anticipated target difference and event rate) should be considered with reference to the estimand of interest.

Other topics of interest

Introduction

Three main alternative approaches (precision, Bayesian and value of information) to sample size calculations are briefly considered in turn below. Other approaches exist, although at present they are rarely used. 179,197

Precision

The limitations of the conventional approach to the sample size calculation of a RCT are well known. 32,178 One alternative is to base the sample size on the precision of the estimate of the interest, the treatment difference. This can be expressed through the CI, and the sample size can be chosen to achieve a CI of a particular interval width (i.e. difference between the upper and lower limits). The width of the 95% CI for a standard trial design for a binary outcome is related to sample size via the relation:

where p is the expected mean response across treatment groups, w is the width of the CI for the difference in proportions, which could be chosen to exclude the magnitude of difference desired to be detected and n is the number in each group. This formula makes use of the large sample binomial approximation. More complex calculations for the CI can be used instead, although with limited additional value for many situations.

1. The width of the 95% CI for a standard trial design for a continuous measure is related to sample size via the relation:

where w is the width of the CI for the mean difference, which could be chosen to exclude an important difference, S is the population SD (assumed to be known) and n is the number in each group.

For both of the above formulae (see Equations 14 and 15), two-sided CIs with confidence level (1 – α) can be calculated by substituting the corresponding Z_{1 – α/2} in place of 1.96. For example, a 90% CI would use 1.645 instead of 1.96. These calculations implicitly do not take into account statistical power and will lead to a smaller sample size, given equivalent assumptions. This type of approach is increasingly used in the context of pilot trials (e.g. for ensuring that the width of the CI for the group proportion or consent rate is sufficiently narrow). 165,198–200 However, use in the context of a definitive trial is limited to date. 13,178,201 The issue of the magnitude of a difference that is valuable to be observed is still present.

Bayesian

The Bayesian concept of assurance,12 also referred to as ‘average power’202 or a ‘hybrid’ Bayesian–frequentist method,11,203 can be used to inform the sample size calculation for a trial that is to be analysed within a conventional (Neyman–Pearson) framework. In this context, assurance is the unconditional probability that a trial will yield a statistically significant result, calculated by averaging the statistical power across a joint prior distribution for the treatment difference and any unknown relevant nuisance parameters (such as the response variance or control response rate for continuous or binary outcomes, respectively). High assurance implies that the trial is adequately powered to detect a continuum of plausible effects. This increases the robustness of the design but typically leads to larger than conventional sample sizes. 203

When performing assurance calculations, a prior distribution for the treatment difference can be determined from expert opinion (see Chapter 3, Opinion-seeking) or a synthesis of existing data (see Chapter 3, Pilot studies and Review of the evidence base). Adjustment for between-trial heterogeneity204,205 and the bias inherent in existing effect estimates can be made. 206 The latter arises because Phase II trials may be more at risk of internal biases and confirmatory trials are commissioned only after observing promising early phase results. A careful choice of prior distribution (possibly truncated to support only alternative values of the treatment effect) is needed to ensure that sample sizes are not unreasonably large and that assurance approaches one as the sample size becomes infinitely large. 203 A related approach avoiding this last subtlety is ‘conditional expected power’, defined as the average power calculated, assuming that an advantage for the novel intervention must exist. 207 In this setting, one can set the target difference (δ) to the value δ*, which ensures that a conventional RCT designed to high frequentist power to detect δ* also has high conditional expected power. 208

A wide variety of methods for calculating the sample size of a Bayesian RCT also exist. The average power of trials with Bayesian final decision rules can be calculated. 209 Alternatively, the sample size of a Bayesian trial can be chosen to ensure that there is a high prior predictive probability of the trial concluding with definitive levels of evidence, supporting either adoption or abandonment of the novel intervention, thus reducing the region of indecision. 210 Alternatively, precision-based approaches calibrate a trial’s sample size on the basis of the expected length of a 100(1 – α)% posterior credible interval or the expected coverage of an interval of fixed width. 211 Judgements about what constitutes an acceptable length or coverage level will depend on how the trial results will feed into subsequent decision-making.

Value of information approach

Bayesian decision-theoretic designs exist that choose the sample size to maximise the expected utility of the trial. 212,213 This is implemented in health technology assessments as a value of information analysis. An efficient sample size is determined by comparing the (expected) cost of conducting a study of sample size, n, with the expected value of the information that the study will yield. 168,214 As such, it offers a radically different approach to determining the sample size for a RCT from the conventional (Neyman–Pearson) power calculation approach. A key element of the decision-theoretic approach is the focus on expected values rather than hypothesis testing for making decisions. 214

The cost of collecting information is simply the budget for a proposed clinical trial of sample size n. The information the trial yields is valued in terms of its ability to reduce uncertainty; all else being equal, larger trials will yield more information than smaller ones. The value of the information is the ‘expected reduction in the expected loss’ from that study.

The logic is as follows: a decision must be made whether to adopt or reject a new treatment. As the decision is made under conditions of uncertainty, the ‘wrong’ decision could be made. The expected loss associated with the decision is the probability of making the wrong decision, multiplied by the loss (forgone health gain) if the wrong decision is made. More research (i.e. information in the form of a clinical trial or other data-gathering exercise) reduces the probability of error and hence reduces the expected loss. This expected reduction in expected loss is the expected value of sample information. Expected value of sample information can be measured in terms of health gain (e.g. life-years or quality-adjusted life-years) forgone, or it can be expressed in monetary terms. For example, the National Institute for Health and Care Excellence in England values a quality-adjusted life-year at between £20,000 and £30,000. 68 The expected net gain of sampling is the difference between the expected value of sample information and the anticipated cost of the study. The most efficient sample size for the study is that which maximises the expected net gain of sampling. Use of a value of information approach is an active area of research and various modifications to the basic approach have been proposed. 155,168,215

Introduction

Five types of alternative trial (multiarm, cluster, crossover, biomarker and adaptive) designs are considered in turn below, in terms of their implications for specifying the target difference. A huge number of variations in trial designs (e.g. split-plot216 and stepped wedge designs217) exist although, in terms of the implications for specifying the target difference, the relevant issues are typically similar to those addressed below.

Multiarm

There are many different designs and aims of multiarm trials, but the one thing that they have in common is that they all include more than two trial arms. For example, this could involve comparing multiple treatments against a common active control or comparing two or more treatments against a placebo. Specifying the target difference in multiarm trials is more complicated than a parallel two-arm trial, as multiarm trials aim to answer multiple research questions. The sample size has to be sufficient to address each research question and therefore multiple target differences could be appropriate.

The more trial arms there are, the more complicated the process becomes. In a trial with three intervention arms comparing treatments A, B and C, there are seven theoretically possible comparisons that could be made: A versus B; B versus C; A versus C; AB versus C; AC versus B; BC versus A; and A versus B versus C (using a global test). An example is given in Box 12. A key aspect of the design of multiarm trials is specifying what the estimand of interest is, which comparisons are of most interest and which hypotheses will be tested. The selection of comparisons may become simpler if one of the arms (say C) is a control arm, such as usual care or placebo. In this instance, what would be of primary interest would be treatment A compared with the control C, and treatment B compared with C. The simplistic approach to sample size calculation would be to consider each of these pairwise comparisons as if they were separate trials. The target difference for each would require specification and justification in the same manner as a standard trial, even though these might well be the same for both. It might also be of interest to compare A with B directly, although specifying that difference might depend on whether treatments A and B are different types of interventions, minor variations of the same intervention (e.g. doses of the same drug), or an experimental treatment and an active comparator, etc.

Multiple arms allow more than one hypothesis to be explored; however, if it is appropriate to specify the most important hypothesis under study this can then be used to drive the sample size. In the example described above, it could be that treatment arm A compared with the placebo is of most importance and the comparisons of the treatment arms can thus be done in a hierarchy. 219 This would have consequent effects on specifying the target difference, as it would be the same as a parallel two-arm trial as previously outlined, with the comparison of A with the placebo primarily determining the sample size. If the aim is to look at the specified comparisons simultaneously, then there will be multiple target differences that could be used, depending on the comparison being made. In the example above, this would mean that both treatment arm A and treatment arm B must be statistically different from the placebo for the study to be declared a success. Such studies are termed multiple must-win trials. 220

When there are multiple target differences, the smallest target difference will be the one that has the biggest influence on the sample size calculation. 220 When appropriate, the use of global comparison tests, pairwise comparisons and/or statistical multiplicity adjustments can be used to account for multiple comparisons. 221

Cluster randomised

Cluster RCTs involve randomising groups or clusters of individuals to trial arms rather than the individuals themselves. If cluster randomisation is used, then this needs to be accounted for within the design and analysis of the study, including the sample size calculation. 222 Individuals within each cluster will be more alike than they are like individuals within other clusters and cannot be considered independent of each other. The ICC is a measure of this similarity and for a particular outcome represents the amount of variance that can be explained by the variation between clusters. Sample size calculations for cluster trials have been developed and involve inflating the sample size for an equivalent individually randomised trial by a design effect (also called a variance inflation factor):223

where m is the average cluster size and p is the ICC. This formula can be used for binary and continuous outcomes.

For example, in an individually randomised trial of an exercise intervention for low back pain, the target difference was 1.57 points with a SD of 4. Assuming 90% statistical power and 5% two-sided significance level, a target sample size would be 274. If this trial were undertaken as a cluster RCT, then the target sample size, assuming an average cluster size of 20 and ICC = 0.03, would be:

With a sample size of 432 and 20 individuals per cluster, this would require 22 clusters to be randomised (i.e. 440 individuals in total). This increases the sample size substantially compared with individual randomisation, for which a target sample size of 274 would be required.

The ICC, as a ratio, is a difficult quantity to estimate precisely. 224 Pilot trials and most clinical studies are too small to achieve this. Instead, databases of estimates from other data sources, which include similar RCTs, exist, so that the same or at least a similar outcome can be used to provide or inform the choice of a more reliable value. Existing databases of ICC values cover implementation science, organisational interventions and surgical interventions. 121,224,225

The calculations above for the design effect do not take into account variation in cluster sizes and assume that the same, or approximately the same, number of individuals per cluster are recruited. 226 If there is variation in cluster sizes, then the formula above will underestimate the adjustment required. An additional factor that needs to be considered in trials that anticipate variation in cluster sizes is the coefficient of variation (CV). The CV is the ratio of the SD of cluster sizes to the mean cluster size. The design effect is expressed as:

where m is the average cluster size and p is the ICC.

The maximum increase in sample size when accounting for variation in cluster size is CV² + 1. The choice of CV is important to ensure that the appropriate inflation factor is estimated. A number of different methods for different scenarios have been proposed to estimate this coefficient:227 knowledge of CVs observed in previous studies; investigating and modelling sources of cluster size variation; estimating likely minimum and maximum cluster sizes; when all individuals in each recruited cluster participate in a trial; when cluster sizes are identical; and when cluster size follows a roughly normal distribution. It has been recommended that the potential impact of variation in cluster sizes should be explored when planning sample size calculations for cluster trials. In particular, it becomes important to assess this when large variations in cluster size, large ICC values or large mean cluster sizes are anticipated.

It is not always necessary to incorporate the CV into the cluster trial sample size. If the CV is estimated to be < 0.23, then the sample size does not need to be adjusted for the variation in cluster size as the impact on the sample size is negligible. 226

Clustering can also potentially arise in individually randomised trials, when interventions are delivered within a group setting or when individual therapists deliver the intervention on an individual basis to a group of individuals. 228 The same methods outlined above apply in these situations but, depending on the type of interventions and nature of the clustering, this may be required in only one of the trial arms.

Crossover trial

Crossover (randomised) trials involve randomising individuals to a sequence of interventions rather than to a single intervention. 229 In the simplest form, they involve two treatments and two periods (a 2 × 2 crossover trial, also known as an AB/BA trial). More complex designs, with three or more interventions and/or periods, are possible. 1,229 Here, consideration is restricted to the 2 × 2 crossover design and specifically the most common implementation in which AB and BA sequences are used equally. For a binary outcome, the sample size can be calculated in terms of the conditional OR (anticipating an analysis using McNemar’s test) and approximated as:

where p_A and p_B are the probability of an event under treatment A and B, respectively. The number of participants (with outcomes for both treatments) is:

with Z_{1 – α/2} and Z_1 – β defined as before.

It should be noted that the standardised effects (here expressed as ORs) for a parallel-group and crossover trial are not equivalent. In terms of expressing the target difference, the absolute difference along with the control group proportion is the most transparent [i.e. (π_B – π_A) and π_A].

For a continuous outcome, a similar formula to the parallel-group superiority trial can be used:

with δ, Z_{1 – α/2} and Z_1 – β defined as before. However, unlike before, here σ refers to the anticipated within-person variance and not the pooled variance of two independent treatment groups. Here, n refers to the overall sample size, as well as the number receiving each treatment. The crossover design enables the individual variance to be stripped out, leading to a more precise estimate and smaller sample size. In terms of target difference on the absolute level, it remains as before. As for the binary outcome situation above (see Equation 19), it is noteworthy that expressing the target difference as a SES, when calculated simply by using the inputs to the sample size calculation for a crossover, is not directly comparable with a parallel-group trial, even though the absolute target difference is the same. They will be equivalent only in the improbable event that the within-person correlation is zero. For example, given a target absolute difference of 10 points, anticipated intervention group SD of 30 and within-subject variance of 19, we would obtain markedly different SESs of 0.33 and 0.63, respectively. The pooled individual group SD is preferred when expressing an effect size as a standardised mean difference, even where a crossover trial is planned.

It should be noted that the impact of missing data on precision is more marked for a crossover trial than for a similarly sized parallel-group trial.

Biomarkers

The sample size considerations required for biomarker-stratified trials do not differ substantially from those required for non-stratified trials, but there are some additional considerations that are important. The common components of a sample size calculation still apply [e.g. for a binary outcome, the significance level, statistical power, event proportion in the control arm and the target difference being sought (δ)]. However, these components tend to be considered separately within each of the proposed biomarker stratum. Other considerations are briefly covered below.

Adaptive designs

Adaptive designs for clinical trials234 enable the analysis of data as they accumulate during a trial at one or more interim analyses, with the results of these analyses used to modify the trial design in some way. A wide range of design adaptations have been suggested, but perhaps the most common involve either early stopping if the intervention under investigation appears particularly promising or particularly unpromising, or a change in the planned sample size based on early estimates of nuisance parameters or treatment effects. Such designs are considered appealing because of the opportunities they give for increasing flexibility and efficiency. A recent and extensive summary of methodology in the area is given by Wassmer and Brannath. 235

Like almost all confirmatory RCTs, trials with an adaptive design are usually designed to have a fixed type I error rate and power for some specified target difference. The increased flexibility afforded by an adaptive design can, however, have implications for the choice of the target difference used in the construction of the design. In a conventional trial, there is often a compromise between ensuring the trial has sufficient power to detect a small clinically meaningful difference and minimising the sample size if a larger treatment difference is anticipated or hoped for. With an adaptive design, the final sample size can depend on the observed interim data. It can thus be possible to design the study to ensure that statistical power is maintained to detect a small difference, but to also allow the trial to stop with a smaller sample size if a larger treatment difference is observed. In a similar way, if there is uncertainty regarding the SD of the primary outcome at the planning stage, the final sample size can be adjusted depending on interim data, to maintain power for a target difference specified on an absolute scale when this is considered desirable.

Much recent interest in adaptive designs for clinical trials has focused on multiarm, multistage236,237 trial designs, in which more than one experimental treatment is initially compared with a control arm. Less effective treatments are then dropped as the trial progresses. In this case, issues relevant to the specification of target differences in multiarm studies, as described in Appendix 3, Methods for specifying the target difference, should also be considered.