Towards single embryo transfer? Modelling clinical outcomes of potential treatment choices using multiple data sources: predictive models and patient perspectives

Structure of this report

This chapter gives the background to, and motivation for, the project then outlines the project design, informally reviews the relevant literature, and finally introduces some of the methodology used. Subsequent chapters describe the project components, with detailed methodology and results along with discussion. Chapter 2 describes the work on patient perspectives. Chapters 3 and 4 develop predictive models based on two large datasets, and then Chapter 5 uses these models to predict and simulate elective single embryo transfer (eSET) policies. In Chapter 6 we describe patient reactions to the policy options. Chapter 7 discusses specific issues around the use of routine data for studies such as this one, and the final chapter attempts to synthesise the components and draw out the important conclusions, implications for practice and needs for further research.

Throughout this report, additional material giving details of the methodology is provided that is not necessary in order to understand the results and conclusions. The sections containing such material are indicated with an asterisk and can be omitted by the reader without losing any of the cohesiveness of the report.

Research objectives

The project objectives as defined in the protocol were:

1. To collate high-quality cohort data from a series of individual treatment centres to be considered alongside HFEA data.

2. To develop predictive models from each of the data sources for (a) twinning probabilities in patients treated with double embryo transfer (DET) from fresh or frozen embryos, (b) success probabilities in couples receiving SET, and (c) potential singleton and twin rates if couples had been offered SET. In each case, to consider the full range of potentially prognostic factors associated with the couple and the available embryos, including age, fertility history, cause of infertility and embryo quality (the last is not available for the HFEA data).

3. To understand, through qualitative work, the patients’ perspective on these choices as they travel through the treatment process.

4. To involve couples in developing patient-relevant outcome measures for IVF treatment programmes and a range of potential choices and treatment options for consideration.

5. To consider a number of potential outcomes and denominators (including, but not limited to, per couple, per embryo transfer cycle, per stimulated cycle started, per completed cycle) from a clinical and patient perspective, and to predict these for potential treatment scenarios based on proposals in the literature and developed with patients and clinicians.

6. To use the modelling results to investigate with patients the acceptability of the scenarios and the changes in public policy required to make SET acceptable.

7. To suggest appropriate randomised controlled trials (RCTs) to test the effectiveness of the most favourable policies.

Research context

At the time this study was designed (early 2006) there was growing awareness that the high twin birth rate from IVF-treated couples was potentially a significant public health issue and a burden on health-care resources. Standard care was to transfer two embryos if available, in all but exceptional cases. Although a few individuals and centres were advocating SET in order to reduce the chances of twin births, it was rare in practice. The decision whether to have SET or DET was made by the individual couple following advice and counselling from the clinical staff. Thus, it was important to understand the patient perspective on twins and SET. Even if one were to advocate a policy of compulsory SET, in formulating such a policy the patients’ views would need to be considered.

After the start of this project the HFEA commissioned an expert group to consider the incidence and consequences of multiple births3 and undertook a consultation exercise leading to a policy decision in 2008 requiring clinics to reduce their twin rate, incrementally with a target rate reducing to 10% twins per live birth event (LBE) over a number of years. Rather than restrict DET to certain groups of patients, or mandate SET for other groups, the UK policy is based around target twin rates for individual clinics, with each clinic needing to develop its own policy (known as a Multiple Birth Minimisation Plan) to meet the target. This target started at 24% (approximately the national average in 2007) twins per LBE for the reporting year of 2009, with a series of interim targets. The emphasis of this project was therefore centred around the implications of this specific UK policy and the need for clinics to be able to develop strategies to meet the targets, and this formed the basis of discussion with patients and simulations of policy.

Research methods

We undertook an interdisciplinary approach in which quantitative retrospective cohort studies and predictive modelling were embedded within qualitative studies of patient perspectives in an integrated manner. The various components are described below.

Literature review

We informally reviewed the literature to (1) identify studies where SET has been compared to DET, both randomised trials and cohort studies, (2) identify prognostic factors to be included in the models, (3) identify series in which published data are available with sufficient detail to be used in model verification, and (4) identify strategies for the use of SET in clinical practice and the obstacles to their adoption. The results of this review are incorporated into the background section below (Informal review of the relevant literature).

Retrospective cohort studies (objectives 1 and 2)

We undertook two linked cohort studies to determine factors associated with success and twin rates in SET and DET. The sample was designed to include the full spectrum of patient settings, including NHS-funded patients attending a centre offering only NHS treatment, private patients attending a fully private clinic, and NHS-funded, fee-paying NHS patients and self-funded (private) patients all treated within NHS clinics. The centres included cover a range of policies on SET, embryo selection and freezing.

Specifically we collated data from the following sources:

1. Data from the national HFEA register covering 2000–5. This provides outcome data on each embryo replacement cycle conducted in the UK, with a useful, but not exhaustive, set of patient, partner and cycle factors. However, this dataset contains no embryo-level data. The data are anonymised, but records relating to the same couples are linked. There are issues about data quality in such databases (and these are assessed in Chapter 7), but the HFEA Historic Audit project at least validates the quality of the data in cycles that generated a clinical pregnancy or live birth.

2. A collection of single-centre, information-rich datasets with embryo quality measures on all transferred embryos. We extracted a cohort with full outcome data for treatments completed in the 2000–5 time frame. Six centres agreed to take part in the study and provide data, giving an estimated 13,000 cycles. In practice, two of the centres failed to supply data owing to changes in local circumstances, but one additional centre was recruited. The number of patients in the existing centres was significantly larger than originally estimated and these centres gave us a database with 23,582 cycles, which was considered more than sufficient to achieve the stated aims.

3. During the period of the cohorts, two Manchester centres (one NHS only, one private patients only) had a day 1 embryo freezing policy, which means that a maximum of four embryos were available on day 2 for selection of one (SET) or two (DET) for transfer. The other collaborating centres were all NHS centres of excellence with a mixture of NHS and fee-paying patients. They all had an embryo freezing policy that allowed all embryos to be available for selection on the day of transfer, with freezing taking place after selection of embryos for fresh transfer, in contrast to the Manchester centres.

4. We also intended to utilise a dataset from a prospective study of the use of amino acid profiles for the prediction of embryo viability. Unfortunately this study failed to complete owing to technical issues and so could not be used formally, although the project did contribute informally. However, we did utilise the embryo grading data from that study to inform the simulation studies of Chapter 5.

Formal sample size computations were not appropriate here as the aim was to develop predictive models, not to formally test hypotheses. Experience and heuristic arguments suggest that datasets in excess of 10,000 cycles are required for this exercise. Rules of thumb for reliable predictive modelling suggest 10–20 events per considered variable. We expected to have around 40 potential variables, which, with a success rate of 20%, would imply a minimum dataset of 4000 independent cycles, around 8000 patients, given that many patients have multiple cycles and we wished to look at multicycle end points. The sample size was, in practice, determined by the need to have a representative set of centres and a sufficiently long time span to capture treatment histories along with computational feasibility, and the numbers analysed were well in excess of the minimum numbers above.

From these data we developed a series of statistical models for the per-embryo replacement outcomes as a function of the patient, embryo and treatment characteristics.

The aim of this phase of the study was to produce a series of statistical models relating outcome (singleton, twins) to prognostic indicators for fresh and frozen embryo transfer across multiple egg retrieval and embryo replacement cycles. These models identify prognostic factors leading to high risk of twins and high chance of success, and provide the basis for the consideration of the role of SET.

The analysis of the HFEA data is described in detail in Chapter 3 and the individual centre cohorts in Chapter 4.

Patient perspectives (objectives 3 and 4)

In this phase of the study we undertook qualitative interviews with couples who were in the process of undergoing IVF treatment. The aim was to explore the patient perspective of treatment choices as they travel through the treatment process. Therefore interviews covered a range of decision-making stages: (1) waiting list; (2) after the first information meeting and clinic appointment (pre-treatment); and (3) after the second cycle of treatment. This last group allowed for views to be assessed once the outcome of an initial treatment cycle was known and after having the opportunity to reflect on the choices through a second treatment cycle. It was planned that 5–10 couples per stage would be invited to take part in this study. Purposive sampling techniques were employed to ensure maximum diversity of sample including different female ages, parity, duration of infertility and source of funding (which is related to the number of treatment cycles that the couple receive).

Specifically we planned:

1. To assess couples’ knowledge and views on embryo transfer and twin birth prior to treatment, after provision of information and post treatment.

2. To explore the potential facilitators and barriers to eSET.

3. To evaluate the patient perspectives on the decision-making process during key stages of the treatment journey, including consideration of measures of success and attitudes to twin births.

4. To determine the level of involvement couples would prefer in the decision-making process regarding treatment choices.

5. To establish at what stage (pre-treatment) information regarding treatment choices about eSET should be presented, and in what format.

6. To explore couples’ attitudes to research, in particular their understanding of randomisation into a clinical trial.

Predictive modelling (objectives 5 and 6)

Based on our survey of the literature and the qualitative work above, we aimed to identify a limited number of potential treatment policies and choices involving the use of SET, based on a patient perspective of the whole treatment course. These were to include, but not be limited to, SET cycle choices, single DET versus two cycles of SET (with the second fresh or frozen), and include a range of couple prognoses. We used the models developed above to predict the outcomes of the various scenarios for the whole range of prognostic factors, with estimates of their reliability. This predictive modelling encompassed both direct prediction from the models and the use of model parameters (and their associated uncertainties) to make predictions for treatment policies not contained within the source datasets. In developing the models we took care to consider the correlations between cycles, and to assess the errors in the prediction, validating against both internal and external data where these existed. This simulation work is described in Chapter 5.

We planned to establish three focus groups (two NHS, one private sector) of patients and partners who had been through the IVF process to present to them the results from the modelling process. This methodology has been successfully employed to explore sensitive issues. 4,5 A convenience sample of couples who had undergone IVF treatment were to be invited to participate in a structured focus group. Following a general discussion about the various treatment options, a selection of scenarios from the statistical modelling were presented to the groups in a user-friendly format. The groups were asked to rate and discuss the scenarios. This allowed us to explore the responses to the results and determine potential barriers to the proposed solutions and could have led to alternative strategies to be investigated. Owing to changes in circumstances and availability, the composition and conduct of these focus groups was somewhat different from that originally planned and is described in detail in Chapter 6.

Economics

It was planned to synthesise the results of this study with ongoing economic analyses being conducted elsewhere. Unfortunately these studies did not report during the time frame of this study; however, the data generated here have been supplied to one of the groups undertaking economic analysis of SET, and that work will be conducted outside the scope of this project.

Towards randomised controlled trials (objective 7)

Ultimately any proposed treatment strategies would need to be tested in rigorous RCTs. Based on the knowledge gained from these studies we planned to suggest a design for such a trial or trials, defining patient populations, treatments and end points. Such a trial should also include a rigorous health economic assessment. However the failure of the ECOSSE trial (see below), and the fact that, in the UK, policy has now been decided, suggest that any such trial may well be infeasible, and so, although such trials are still needed, we confined ourselves to a general discussion of principles rather than detailed designs (see Chapter 8, Implications for research).

Ethical approval and governance

Ethical approval was obtained from the South Manchester Research Ethics Committee (06/Q1403/254 and 06/Q1403/255). Sponsorship was from the University of Manchester (PS120906). A Project Management Group, consisting of the named investigators, met regularly to oversee the day-to-day running of the project. An Advisory Group consisting of representatives of the centres contributing data, the HFEA and a patient group (Table 1) was established to provide oversight of the study.

Single embryo transfer

Elective single embryo transfer has been widely advocated on the basis that it reduces the number of multiple pregnancies, and the consequent risk to the mother and offspring (e.g. see Pinborg7). Many cohort studies (reviewed in references 8 and 9) suggest that, on a per-transfer cycle basis, SET does indeed reduce twinning rates compared with DET but that this is associated with a reduced success (live birth) rate. This has been confirmed in a limited number of relatively small randomised trials7, although no good-quality randomised data are yet available. 6 The subsequent replacement of single thawed embryos increases the pregnancy rate per episode of IVF on a cumulative basis (e.g. Lukassen et al. 10). Strategies to implement SET are likely to require evaluation across multiple cycles of embryo transfer: in the UK a trial was organised comparing a single fresh cycle of DET with two cycles of SET, one fresh and one utilising a frozen embryo from the first cycle. This trial (the ECOSSE trial, led by Dr Bhattacharya, Aberdeen) was subsequently suspended apparently because of patient reluctance to enter the study (Dr S Bhattacharya, Aberdeen Maternity Hospital, 2009, personal communication).

Subsequent to the start of this project, in 2008 the UK HFEA, after a review and consultation process,3 adopted a policy requiring clinics to reduce the number of twin births as a proportion of live births to 10% over a 3-year period from 2009. The British Fertility Society (BFS) and Association of Clinical Embryologists (ACE) have produced clinical guidelines, with input from the towardSET? project team, reviewing the evidence and supporting this policy change. 11

Elective single embryo transfer trials

Six RCTs, comparing forms of SET with DET in, generally, patients with a good prognosis,10,12–18 and a Cochrane review6 have been undertaken. SET alone gave poorer outcomes in terms of live birth rate per embryo transfer cycle but reduced the incidence of twins to a rate comparable with natural pregnancies. In two small randomised trials, SET with two episodes of embryo replacement was associated with a similar live birth rate as DET but with a significant reduction in the number of multiple births (Lukassen et al. 10 and Thurin et al. 15). There is a lack of large, good-quality trials comparing practical policies with appropriate end points. Cohort studies8,9 show similar conclusions, but these are harder to interpret as the patients undergoing SET are selected by a combination of the clinician and the couple. Most of these analyses used simple per-transfer end points and failed to account for the correlations between cycles from the same patients. Clinical experience in Sweden and elsewhere8 suggests that a legal prescription towards eSET has led to an increased use of SET while maintaining success rates and dramatically reducing twin rates, but these comparisons suffer from the use of historic control data and the use of per-transfer outcomes.

Clinician and regulatory perspectives on eSET

There is widespread agreement among IVF clinicians that, at least in patients with good prognosis, policies to prevent multiple pregnancies, including twin pregnancies, are to be preferred. Although there are counter-arguments, for example Gleicher and Barad,19 claiming that if one assumes that the majority of patients want two babies, a comparison of risk between a single event of IVF twins and two separate IVF singleton pregnancies can be in favour of twins. Many recommendations have been made to increase the proportion of eSET and this is now legally prescribed in Sweden (reviewed in Bergh8). However many centres in the UK are reluctant to adopt policies that might lead to a reduction in pregnancy rates, particularly in the format published by the HFEA, and particularly where patients pay directly for the treatment. The format of outcome data published by the HFEA allows centres to be rated in ‘league tables’. This is widely seen as being of commercial value to centres in the top echelons; SET is more popular where the treatments are publicly funded, as in northern Europe. For example, in 2005 in Manchester within the NHS at St Mary’s Hospital the SET rate was 30%, while in the private sector at Manchester Fertility Services it was 10%. The definition of treatment success rate is crucial here20 but, as yet, there is no consensus on a measure that takes the whole treatment programme, including embryo freezing, into account.

Patient perspectives on SET

Opinions on SET may differ between health professionals/policy makers and patients undergoing treatment; patients often view twins as a positive and not a negative outcome. Blennborn et al. 21 interviewed 272 patients undergoing IVF (males and females) using a semistructured questionnaire to investigate the couple’s decision-making in IVF treatment. Factors associated with opting for SET were previous childbirth and the availability of spare embryos to freeze. DET was more likely in couples who had undergone previous IVF treatment, and in those who held the belief that transferring two embryos would increase their chances of becoming pregnant. They also noted that, in spite of receiving good information on the associated risks, most couples preferred to have two embryos transferred. Pinborg et al. 22 conducted a national survey in Denmark to explore the attitudes of IVF/ICSI twin mothers towards twins and SET. They found that this group of mothers was more likely to desire twins as a first child than either a singleton or a non-twin mother group. This group appeared to accept the known clinical risks of twins, as well as the associated social and physical outcomes of caring for two babies. Acceptance of SET was influenced by the experience of a very low birthweight baby; this presumably heightened awareness of morbidity and mortality outcomes in multiple birth. Many other studies have supported the view that patients undergoing IVF show a preference for twin pregnancies. 23–25

In view of the evidence of patient preference for having two embryos transferred, Murray et al. 26 sought to investigate methods that might improve the acceptability of SET. Couples undergoing IVF treatment were randomised to one of three groups: group 1 (control) received the standard clinic information pack; group 2 received the standard pack plus an extra information sheet outlining the possible adverse outcomes of twin pregnancy; group 3 received the same as groups 1 and 2 with an added 10-minute discussion with a member of staff, focused on the information on twin pregnancy. No new information was given, although couples could ask specific questions. The results revealed that neither method increased couples’ knowledge of the risk of twins or changed their attitudes towards SET. This is perhaps not surprising as attitudes and beliefs develop over extended periods of time and can be resistant to change. 27 This particular intervention was not specifically targeted and did not allow for the member of staff to explore and work with the couple’s belief systems.

The clinical effectiveness and cost-effectiveness of SET cannot be fully assessed without conducting an RCT. Given the strength of feeling concerning choosing between SET and DET, and the emotional effects of treatment, the design and conduct of an RCT with this patient group would require careful consideration. Porter and Battacharya25 undertook an exploration of the opinions of patients undergoing IVF and the views of staff of a proposed trial of eSET. They found that patients lacked awareness of the risks of multiple births. Opinions of the proposed RCT were largely adverse, except in younger women. Although staff appreciated the need for a thorough evaluation of SET, the notion of a double-blind RCT raised both practical and ethical concerns, which often conflicted with their caring role.

Research that has emerged during the conduct of this study has continued to reinforce the strong patient preference for twins, while exploring methods of understanding the complex decision-making process of women and couples in treatment. Twisk et al. 28 presented a range of scenarios to women (in the stimulation phase of IVF or ICSI plus thawed/frozen embryos if available) to assess whether they would be willing to accept a lowered probability of pregnancy from SET in order to reduce twin pregnancies. When women were presented with a scenario in which the pregnancy rate of SET and DET were equal, 46% chose SET. However, once the pregnancy rates were adjusted to show that SET was 1%, 3% and 5% less effective than DET, the preference for SET fell to 36%, 24% and 15% respectively. Similarly, when women were asked to consider additional cycles of SET to optimise pregnancy rates, preference rates began to decline after three treatment cycles. The findings show that the strong desire for a pregnancy outweighed the need to avoid a multiple pregnancy. They also note that treatment preferences may be different in couples who are self-funding.

van Peperstraten et al. 29 used in-depth interviews with 20 patients and 19 IVF health professionals in the Netherlands to explore in more detail the factors that may influence treatment decisions. They found that couples and professionals lacked understanding about SET, and both groups were not clear about the advantages of performing SET in practice. There was concern about the lowered chances of success when using frozen embryos. Furthermore, in order to improve the uptake of SET both groups discussed the need for a positive reimbursement system for this treatment option.

The higher-risk profile associated with multiple pregnancies is not always recognised or acknowledged by couples undergoing treatment. One recent study investigated women’s and men’s understanding of information and the decision-making process in SET. 30 They found that, although women were aware of fetal risks associated with a twin pregnancy, they were less well informed about maternal risks. In this small sample of 54 couples, acceptance of SET was associated with previous pregnancy, younger age and duration of infertility. One interesting study investigated whether the strength of the preferences for twins would be mitigated by providing knowledge of the more severe adverse outcomes. 31 This was compared with the outcome of not achieving a pregnancy. A standard gamble method was used to explore preferences in 74 women waiting for IVF treatment. Scenarios included giving birth to a child with physical or cognitive or visual impairments and experiencing a perinatal death, without a subsequent pregnancy. A further scenario considered another birth outcome, that of a very preterm birth with morbidity and mortality left unspecified. The findings revealed that some women found the risk of severe child disability associated with DET to be more desirable than not having a child at all. The authors note that this construction of ‘treatment success’ is not in line with that of clinicians. They speculate that this may go some way to explain why couples continue to request DET, whether or not they have a good understanding of the associated risks of twin pregnancy.

In order for patients to make informed choices, accurate and relevant information is essential to this process. 32 However, studies in subfertile couples undergoing IVF treatment have shown that information to date has either not been adequate or not been presented in a relevant and targeted way. 26,30 The work of Scotland et al. 31 has crystallised the strength and magnitude of the preference to achieve a pregnancy at all costs. This suggests that for many couples there is a need for a more in-depth information-giving process, which takes into account attitude and belief systems and allows patients to make their own informed decisions. Therefore, further research is required to understand the specific information needs of this patient group and the presentation and timing of this information.

Embryo selection for eSET

The ability to select a ‘top-quality’ embryo for transfer is crucial to the success of eSET (see, for example, De Neubourg et al. 33). Selection is normally made on morphological grounds, with different scoring systems in use in different centres, based predominantly on embryo developmental stage (i.e. cell number, and morphological appearance including cell size, regularity, fragmentation, etc.). There is considerable interest in selection criteria34 and in alternative markers to morphology. 35,36 Selection of embryos by extended in vitro culture to the blastocyst stage has received much attention,37–39 and a Cochrane review,39 and was one approach suggested in the ACE/BFS guidelines. 11 Invasive pre-implantation genetic screening has been advocated as a potential approach and is available clinically. 40 However, there is little evidence for its effectiveness in increasing treatment outcomes and, in contrast, recent prospective RCTs have provided evidence that the invasive nature of the embryo biopsy procedure may be detrimental. 41,42 Treatment policies on the length of time for which embryos are cultured before transfer or freezing and the use of cryopreservation differ between centres; thus, different centres will have differing numbers of embryos at different stages from which to select. To our knowledge, no comparative or modelling studies have been undertaken that consider the impact of different cryopreservation/selection policies, and it is crucial to capture this in any assessment of the impact of SET. Horne et al. 43 prospectively compared day 1 freezing with day 2, with transfer of fresh embryos on day 2 in both arms of the study. They found that the day 2 freezing strategy led to a higher live birth rate from the fresh cycle, presumably as a result of enhanced embryo selection because all embryos were available for selection for fresh transfer, but a lower pregnancy rate per frozen embryo cycle. Both strategies yielded similar final cumulative pregnancy rates.

Economics of eSET

Few studies looking specifically at eSET from the economic perspective have been reported (reviewed in Bergh8 and Fiddelers et al. 44). From a societal perspective, these indicate that the savings in health costs associated with twin pregnancies may offset the direct additional costs of the repeat SET cycles required to maintain the same take-home baby rate. However, in many cases the direct costs of treatment are borne by the patients, while the costs associated with multiple births are (in the UK) met within the NHS. A recent publication45 studied the impact of multiple births from a UK perspective. In addition, there are less readily quantifiable costs associated with a potential requirement for extra treatment cycles per baby in eSET.

Inference from patient cohorts

Three approaches have been used to extract information from retrospective data from patient cohorts:

1. Estimation of pregnancy or live birth rates arising from SET versus DET, with a range of outcome definitions and patient subsets. 46–51 These suffer from inbuilt biases in the selection of patients for SET. In many retrospective datasets it is difficult to know the true reason for SET, unless only one embryo is available. In some studies this is ‘patient choice’, in others it is perceived clinical need (patients for whom twin pregnancies are contraindicated) or some combination of the two.

2. Logistic regression of success rates and twin rates in DET have been commonly used to determine factors that predict a high twinning probability (e.g. Strandell et al. 52). These methods potentially identify high-risk groups but give no information on the potential outcomes if SET were used.

3. Explicit modelling of embryo and recipient (uterine) effects. Within this framework, models derived from DET data can be used to predict SET outcomes. The first published example utilising this embryo–uterus (EU) approach, Hunault et al. ,53 uses the Zhou and Weinberg model,54 but attributes all the prognostic parameters to the embryo, fitting a constant uterine receptivity (U). Analysis of a Manchester series2 also used the EU approach and considered the possibility that factors may influence both the E and U components, but there was insufficient data to identify which of the submodels (E or U) was appropriate for each parameter. A Bayesian approach with the possibility of incorporating a hierarchical structure was examined by Dukic and Hogan,55 although issues surrounding prior distributions and parameter identifiably were observed. These models have the advantage that they allow predictions of SET outcomes from multiple embryo transfer data, avoiding the selection issues in the retrospective comparative studies. The models make other assumptions, particularly around the independence of the embryo and uterine effects, although there is no evidence that these assumptions are inappropriate.

Other methodological approaches that have received attention are case-based reasoning algorithms,56 causal models57 and Markov chain models. 58 The last study provided an alternative to time-to-event analysis for multiple cycles where in each cycle a patient can move across outcome states. These methods are not readily applicable to the questions posed by this study, so are not considered further. They may or may not have advantages in other situations.

In all these types of analysis considerable care and expertise is required in conducting and interpreting the analyses, not only because of the inbuilt biases of the observational data but also to account appropriately for the non-trivial correlation structures between multiple egg-collection and replacement cycles from the same individuals and from centre and cohort effects. Such considerations are rare in the analyses published to date.

Prognostic factors for IVF success

Retrospective studies have identified a number of patient, embryo and treatment factors that are associated with treatment success. In order to derive a list of factors that should be considered in any modelling exercise, a review of the literature was undertaken in May 2007 and updated in November 2008. Searching electronic databases identified 51 papers in which potential prognostic factors were analysed for their influence on an IVF-related response. The tables below were collated for each type of risk (broadly categorised as age, diagnosis, egg and embryo numbers, embryo quality, biological, treatment, and other factors). These show studies where an association between the risk factor and the response was found, but do not include studies where a risk factor was tested but not found to be of significant interest. The results also indicate the nature of the observed effect, the size of the dataset and the methodology employed. There was no attempt made to synthesise this evidence or to assess the quality of the studies and their relevance to the general population of patients undergoing IVF.

As expected, age appears to be the variable with the greatest body of evidence to support an impact on IVF response (Table 2). Though many studies include age as a continuous variable in the linear predictors and universally observe a decline in success as patients get older, studies that allowed more complex representations of age tended to find the greatest success rate for patients in their twenties. There is also strong evidence that various measures of embryo quality can impact on success rates (Table 3). This is despite the fact that most studies adopted an aggregated approach in taking a measure to represent all embryos. This is often taken with a measure of egg or embryo number, such as the number of retrieved oocytes or fertilised embryos (Table 4). Both studies adopting the EU approach, which avoids the necessity for aggregation, found embryo quality measures to be strongly prognostic.

Many studies suggested that diagnosis (Table 5), treatment (Table 6) and biological (Table 7) variables can impact on the success of IVF. Previous success or pregnancy/live birth appear to have an impact (Table 6). The evidence for an impact of biological variables such as follicle-stimulating hormone (FSH) is less clear, and the results of the meta-analysis59 were equivocal. There is some weak evidence for an impact of lifestyle factors such as smoking, alcohol consumption and body mass index (BMI) on outcome (Table 8), but there is a dearth of lifestyle studies with objective measures.

The published results give us a good indication of what variables should be included (where available) in analyses for this study. If confirmed they could provide a basis for selection by showing which characteristics are likely to produce a live birth, and furthermore show which are most likely to differentiate cycles with a singleton live birth versus multiple live births.

Logistic regression models

Most work on prognostic factors has utilised standard logistic regression in one form or another with IVF outcome being modelled as a function of patient and cycle characteristics. Some of this work is reviewed above (see Prognostic factors for IVF success). Utilising standard logistic regression (LR) models using patient-level covariates poses two problems: firstly the nature of the outcome, and secondly the handling of embryo-level covariates.

The usual measure of IVF outcome is the number of live babies produced in some unit of treatment, which may be a singleton, twin or a higher-order multiple birth. Most work in the field uses a binary LBE – one or more babies, but this is clearly not very useful if we wish to look at twin incidence. Some workers have created separate models for live birth and for multiple births. 53,72 Another approach, and the one we adopt here, is to model LBE and then multiple births in those who have an LBE. 85

Characteristics, such as morphological grade, measured on the transferred embryos are difficult to incorporate into a patient/cycle LR model, not least as it is in many cases unknown which of the embryos led to a successful outcome. The usual approach is to create some aggregate embryo covariate using either an average of the embryo parameters or the ‘best’ embryo. The approach here follows that of Roberts1 and uses a simple mean of the parameters across all transferred embryos.

A third issue is the non-independence of multiple cycles from the same couples. Although this can in principle be very complex,97 for most realistic scenarios these can be handled by the addition of standard random effects representing levels in the hierarchy above the cycle. In the work considered here there are couple effects (repeat egg collections from the same couple) and egg collection effect (multiple transfer cycles of fresh or cryopreserved eggs collected in a single procedure).

The EU model

Full details of the approach used here are given in the papers by Roberts1 and Roberts et al. 2 The EU approach was first introduced by Speirs et al. 98 In the EU model the success of an embryo transfer is portioned into two components: the embryo implantation probability, E, and the uterine receptivity, U. For a successful transfer at least one embryo has to implant and the uterus has to be receptive. The E and U parameters give the probabilities of each of these events, and simple probability calculations can then give the probabilities of success (one or more embryos implanting in a receptive uterus), failure (no embryos or a non-receptive uterus) or multiple pregnancies. The Speirs model assumes a constant E and U across all embryos and uteri. The model has been extended to model separately the E and U components using logistic regression models. 1,54

It is to be noted that this formulation explicitly excludes monozygous twins, and such events would be considered single successful embryos within the context of the model. Generally monozygous twins are not identified in the available data, but the incidence is low and not a major bias in the analysis. 99

There is an explicit assumption that the embryos and uterine probabilities are independent (after conditioning on the covariates). There are some claims that embryos exert a ‘helper effect’ on each other,100 but the evidence is weak and compromised by incomplete consideration of covariates and patient selection biases.

The interpretation of the uterine component, U, has broadened since its inception by Speirs et al. 98 In analyses such as these it encompasses a wider range of influences that act on any potentially implanting embryos and includes a range of cycle-specific as well as parental factors that may affect the ability of the mother to carry the pregnancy to term.

*Fitting the EU model

Given the model defined in the section above, we can construct an observed data likelihood function, and obtain parameter estimates for the covariates by direct maximisation of this likelihood. Full details are given in Roberts. 1 For this study the optimisation is performed used custom-developed code in the statistical programming language R101 using a quasi-Newton method. 102

Models are formally compared using standard likelihood ratio tests (LRTs), and these are used to derive significance levels for parameters. Standard errors of parameter estimates are derived using a Wald-style approximation based on the inverse of the Hessian of the log-likelihood function. Profile likelihood limits as used in, for example Roberts et al. 2 are computationally infeasible in datasets of the size being analysed here.

The EU model has two submodels, one for characteristics affecting the embryos, E, and one for ‘uterine’ characteristics, U. It is not clear a priori whether factors such as age should be entered into the E, the U or both submodels. Although patient characteristics such as age are measured at the patient (U) level, they may act through embryo characteristics. In some cases there is a natural level at which a clinical observation should be added: for example, a tubal diagnosis is unlikely to affect embryo quality or the use of donor sperm uterine receptivity. However, even in these cases it is feasible that in observational data effects in submodels that would be considered clinically unreasonable may be mediated through patient selection and treatment effects. In principle, the appropriate submodel can be identified from the data, where there are multiple embryo transfers, as it affects the twin rates differently from success rates, but such identification is weak. 1 Additionally the models being compared are not nested, so no inferential test exists and we have to use model selection criteria such as Akaike’s information criterion (AIC, see below).

*Extending the EU model to include between-cycle correlations

The EU model can be relatively easily extended to include a single random effect in the U submodel. This exploits the fact that the observed data likelihood can be factorised into a product of individual cycle components, and the multidimensional integral necessary to allow for the random effects then becomes a product of one-dimensional integrals, and therefore computationally feasible. Even so, given the sizes of datasets in this project, computationally it is only possible to look at a few restricted models in this way. Extensions to allow for random effects in E or multilevel random effects are not yet feasible.

Presentation of model fits

Tables of the parameter estimates are shown together with their estimated standard errors (derived from a standard Wald approach). Depending on the context these are presented with LRT-derived p-values for removal of each variable from the model. Note that each variable in the model may include a number of model parameters: we generally present significance tests for the removal of these as a set.

Sets of plots such as those shown in Figure 5 are used to visualise the fits. In these plots predicted (fitted) probabilities are shown against selected variables, in this example year of treatment. Panel (a) shows the probability of an LBE, panel (b) that of a multiple birth event, and panel (c) the ratios of multiple births to LBE. Box and whisker plots show the range of predicted probabilities across all the individuals in the dataset with the given values of the selected variable, with a horizontal line for the median value, a shaded box covering the interquartile range and whiskers extending to the full range of values seen in the data. These show the actual values in the full population, allowing for any differences in other variables that may be associated with the given variable. For example, the distribution of patient ages may vary between years, and this will be reflected in the predictions shown. To visualise the pure effect of the selected variable a line is added showing the prediction for a ‘typical’ patient where all other variables are held constant and only the variable in question varied. This typical patient has mean or modal values for the parameters as shown in Table 9. As can be seen in Figure 5, this typical patient may have success or twin probabilities that appear somewhat different than the median of patients; setting each individual parameter to a typical value does not necessarily produce a patient with a totally typical prognosis.

FIGURE 5.

Example plots showing a fit and its predicted values for an EU model. (a) Live births; (b) multiple births; (c) multiple birth rate; (d) U; (e) E.

For an EU model we can produce similar plots for the E and U probabilities as shown in panels (d) and (e) of Figure 5.

Similar plots can be produced showing the spread of predicted values and the observed values, and an example is show in Figure 6, in which the thick line shows the actual probabilities in the dataset used to derive the fit.

FIGURE 6.

Example fitted value plot showing addition of actual probabilities from the dataset. (a) Live births; (b) multiple births.

Assessment of model fits

With any statistical model it is important to assess how closely it fits the observed values in the dataset. Our models have binary (yes/no) outcomes such as LBE and twins. One useful measure of goodness of fit is the predictive accuracy, measured by the area under the receiver operating characteristic (ROC) curve, which in practice takes values from 0.5 (useless) to 1.0 (perfect). Simple visual methods, such as the examination of observed versus predicted outcomes for groups of patients (as in Figure 6), are often informative, along with systematic testing of additional model complexity.

As a model becomes more complicated with additional terms it can only fit more and more closely to the observed data. There is then a danger of ‘overfitting’, in that the model is so specific to the observed dataset that it loses generalisability. The aim of our modelling is parsimony – to include only the important terms so as to find a relatively simple model that describes the observed data. For logistic regression this is often addressed by likelihood ratio testing – formal tests of statistical significance. The addition of each term is assessed by comparison of the goodness of fit with and without that term, and only terms that justify their inclusion (statistically significant) are retained. However statistical significance is not the best indicator of the ‘best’ model, as factors that would not be considered significant (using, say, the usual p = 0.05 criterion) can be useful predictors. Moreover, for some of the more sophisticated models presented here likelihood ratio testing is not always possible.

Thus ‘information criteria’ are used to choose between models. The AIC is the most widely used to trade model complexity against goodness of fit. We use this and the Bayesian information criterion (BIC), the latter having the advantage in large datasets of also accounting for the influence of sample size in determining statistical significance (in large datasets almost any effect is ‘significant’).

For estimating sampling errors in the model parameters and predicted outcomes we have used bootstrapping,103 a relatively recent but increasingly standard method for complex end points. We have also utilised a bootstrap-based calibration procedure104 to assess whether the fitted model displays characteristics of overfitting. A well-fitting model should give a calibration intercept close to 0 and slope close to 1.

Setting and sample

A purposive sampling technique was employed to ensure maximum diversity of sample to include different female ages, parity, duration of infertility and source of funding (which is also related to the number of treatment cycles that the couple receive). Couples and women were recruited from two assisted conception centres (one NHS and one private hospital). Three recruitment strategies were used, one facilitated by the NHS clinic research nurse:

1. Patients were selected by a research nurse from the clinic waiting/appointments list. The research nurse was briefed on the selection criteria for the interviews. The nurse then mailed a letter of invitation and provided information about the study to the patients. Those who expressed a wish to be contacted were then followed up by the researcher. Patients were contacted by telephone or email (LMcG) to discuss the study and answer any queries or concerns. In total 80 patients were mailed. Forty-six couples did not respond to the initial mail request. Owing to ethical and time constraints we were not able to do a follow-up mailing or use other reminders that may have improved the response rate. Seventeen replied stating that they were not willing to be contacted. Of those who agreed to be contacted one declined to take part as she had recently experienced a failed cycle, and three did not respond to telephone calls and/or emails. In total this approach yielded 13 interviews.

2. The qualitative researcher (LMcG) attended one meeting held at the NHS hospital for patients on the waiting list. The purpose of these meetings was to brief patients about the treatment process. At the end of the session the researcher described the study briefly. Those who expressed an interest were given a study pack. This approach resulted in eight positive enquires and four interviews were subsequently conducted.

3. A poster was placed in the waiting room of the private hospitals. Patients who were interested in the study were asked to request an information pack from the nurses in the clinic. They then took this home and replied to the researcher using the invitation slip if they wished to be contacted. It is not possible to gauge an accurate response rate with this approach. However, all the private clinic patients (n = 10) were recruited by this means.

The first two strategies were exclusively used in the NHS setting, and proved to be of limited value. In particular, the mail shot was the least successful method of recruiting patients. The use of a poster (option 3) proved to be a successful mode of recruitment in the private clinic (i.e. more patients recruited in shortest amount of time). The use of a study poster was first suggested by the manager of the private clinic and this was the most successful method.

The final number of interviews conducted was 27. The sample consisted of 14 couples and 13 women (who attended the interview without their partner); 17 were funded by the NHS and 10 were self-funded. Seven were interviewed pre-treatment, 13 during treatment and seven post treatment. It should be noted that three of the women interviewed under the classification post treatment were not pregnant at the time of being approached to do the interview; however, by the time the interview was conducted these women were approximately 5 weeks pregnant. This was because these women initially did not know that they were pregnant following a cycle or they had undertaken a new cycle before agreeing to be interviewed. All these women were receiving treatment from the private clinic.

Characteristics of the sample are displayed in Table 10a and 10b.

Data collection

Data were collected by in-depth semistructured interviews. All interviews were digitally recorded, with permission, and transcribed verbatim. Codes were used to conceal participants’ identity and anonymity and confidentiality were assured. Participants were reassured that the digital recordings would be destroyed after transcription, and that all names referred to in the recordings would be replaced with pseudonyms. Interviews were conducted in a place of the participants choosing, usually at home (eight interviews in a hospital setting). The length of the interviews ranged from 40 minutes to 3.5 hours.

Data were managed using the qualitative software package nvivo 7. 105

Data analysis

The 27 interviews were analysed following the principles of framework analysis. 106 This method was designed to facilitate the systematic analysis of qualitative data, which summarises and classifies data within a thematic framework. The analysis can be based in original accounts of those studied (inductive), or be derived from a priori hypotheses (deductive), or both. Framework analysis was chosen for the following reasons:

It provides coherence and structure to otherwise cumbersome, qualitative data (i.e. interview transcripts).

1. It facilitates systematic analysis, thus allowing the research process to be explicit and replicable.

2. Despite the inherent structure, the process of abstraction and conceptualisation allow the researcher to be creative with the data.

Framework analysis involves a number of stages, but this does not imply linearity, as these stages are highly interconnected. There are five key stages involved in the analysis:

1. Familiarisation The transcripts are read thoroughly by the researcher, so that he or she becomes immersed in the data.

2. Developing a thematic framework The process of familiarisation leads to the development of broad key themes. This initial framework was then taken and applied to the interview transcripts. A numeric colour coding system was developed and applied to this thematic framework. In addition, subthemes were beginning to emerge, which were noted in memos. Thus, the framework was expanded and refined accordingly.

3. Indexing The identification of a thematic framework begins the process of abstraction and conceptualisation, which is further developed by indexing and charting the data. Themes and emerging subthemes are subsequently labelled and indexed.

4. Charting Framework analysis involves devising a series of thematic charts or matrices. The indexed data is entered into the appropriate cell. Charts were constructed using a thematic approach, by which cases were drawn up for each key subject area and entries were made from each respondent (case). This process allows for the checking of emergent themes and cross-referencing to the original data. The researchers’ intention was to keep the original exploratory nature of the study in mind as data was ‘lifted’ from the original context and placed in to the appropriate theme/subtheme. These data were not necessarily direct quotes but notes that captured the context of the discourse. Thus, the charts allowed for the full pattern of views, perceptions and attitudes to be reviewed. ‘Charting’ is an evolving process, the end result being that data are grounded in the respondents own accounts.

5. Mapping and interpretation The aim is to bring out the key characteristics and map and interpret the data as a whole (Ritchie and Spencer,106 p. 186). This process can take several forms in order to define concepts and map the phenomena under consideration. The end point is to provide explanation and meaning, some of which may relate to the initial research aims or emerge directly from the dataset. This is not a purely mechanical process; indeed Ritchie and Spencer106 suggest that each step requires ‘leaps of intuition and imagination’ (p. 186). In order to achieve this aim several strategies were utilised. These included continuous reviewing of transcripts, charts and field notes searching for patterns, interconnections, similarities and dissimilarities. ‘Running ideas’ past those with direct clinical experience in the area of assisted conception proved useful. A benefit of using framework analysis is that strategies and recommendations for policy may be elicited at this stage.

For the purpose of the analysis the interview transcripts were allocated into one of three groups: (1) pre-treatment; (2) in treatment; and (3) post treatment so that any effect of stage of treatment could be more easily identified. During the analytical process the researcher (LMcG) remained cognisant of other potential external factors that may have influenced the couples’ and women’s views on SET (e.g. female ages, parity, duration of infertility and source of funding). The process of analysis was verified by a second external qualitative researcher Lynne Austin at the charting phase to verify the principal researchers’ interpretation (all charts were cross-checked with the original interview transcripts and thematic chart). Any discrepancies were resolved by discussion.

Theme 1: Sources of information

Sources of information included general, medical and specific fertility websites and the internet (see Table 12). It appears that the main sources of information were gained outside formal clinic consultations.

General sources of information included booklets, leaflets concerning fertility as well as books, novels and newspapers. A thirst for knowledge was described for those who had not yet started treatment. The process of obtaining information was likened to studying for a medical degree. However, the majority of persons seeking information were those in treatment. Some had received the HFEA booklet at the waiting list meeting or had contacted HFEA direct. Others described receiving the information packs from the clinics. Descriptions of the packs ranged from basic and just ‘shoved together’ to useful and informative or simply being daunted by a ‘whopping great pack of stuff’. Some described already having a general knowledge in this area, while others noted feeling desperate to know what everything means and needing to have more information.

For those in treatment, views on the information given by medical and nursing staff varied. These ranged from feeling that patients know as much as the doctors to finding the clinic information good and trusting the information given. One woman felt that information is not always given as it may frighten people.

Specific websites for women undergoing fertility treatment were also used to combat feelings of being overwhelmed and to buddy up with like-minded people and make friends as often existing friends were not ‘in the same boat’. Negative thoughts regarding websites included getting confused with the wealth of information, the number of sites and badly designed sites.

Theme 2: Views on policy

Not surprisingly, this was the topic which prompted the most discourse. Five subthemes were identified which related to the potential change in policy to limit the number of embryos from two to one. These are listed below.

Theme 3: Views on multiple birth

Theme 4: Views on frozen embryos

Theme 5: Experience of treatment

Theme 6: Consultation process

Women and couples were asked to reflect on their experience of consultations with doctors, nurses and other relevant health professionals. The aim was to explore what style of consultation people preferred and to explore their experience of consultations that had focused on the decision to choose either SET or DET.

Theme 7: Views on randomised controlled trials of eSET

The majority of women and couples interviewed had limited knowledge about RCTs. When the interviewer explained this research design many could see the value of such a design but did not think it was feasible for those undergoing IVF treatment.

Data source

The original data provided from the HFEA registry had 232,990 treatment cycles covering the period 2000–5. These included a variety of treatments not appropriate to this study. Of the 232,990 cycles, 128,139 were recorded as IVF (with 102,152 of these having one to three embryos transferred) and 91,262 as ICSI (with 84,761 of these having one to three embryos transferred).

Data selection

Cycles were included if they met the following inclusion criteria:

• treatment type: ICSI or IVF

• cycles with one, two, or three embryos transferred

• age 19–54

• patient’s own eggs

• date started trying to conceive or last pregnant after start of 1980.

Cycles were excluded if they met any of the following exclusion criteria:

• donor eggs

• frozen/thawed eggs

• natural or hormone replacement therapy (HRT) induction

• cases with rare, non-standard, ovulation induction regimes (defined as induction types recorded for fewer than 150 cycles in the database)

• cycles not fully identifiable as either fresh or frozen cycles (no mixed cycles), i.e. fresh cycles with frozen embryos and frozen cycles with fresh eggs mixed or cycles classified as fresh and frozen.

After selecting the relevant data we had 172,189 embryo transfers from 104,610 patients in 84 treatment centres.

*Data cleaning and exploration of missing data

Table 13 lists some of the integrity checks made on the database and the number of cycles that failed each test. A number of tests that never failed are not listed. In this table we give the total failing the test, the breakdown by outcome, the mean age of those failing the test and the proportion failing in fresh cycles. Cycles could fail multiple tests so there are more failures noted than cycles excluded.

After data failing these checks had been removed, we were left with 139,848 transfers from 85,349 patients in 84 treatment centres. 19% of the cycles contained some missing or invalid data. The biggest problem was in the recording of the duration of infertility, with significant loss also due to conflicts in the definitions of primary and secondary infertility.

Reassuringly there is no indication that there was differential loss of data according to any of the characteristics we have explored, except that more frozen cycles were lost. Figure 7 shows the values of outcome, age and transfer type (fresh/frozen) for the removed and included cases. It is of note that we can detect no difference in data quality between cycles leading to a pregnancy and those that did not (as demonstrated in the figure); we would expect such a difference owing to the extra data checking that was instituted in the HFEA historic data project.

FIGURE 7.

Cases included and removed from the analysis. Pairs of panels show excluded and included cases, upper panels show percentages by outcome, middle panels the age distributions as a box and whisker plot and lower panels percentages of fresh and frozen cycles.

*Patient history in the HFEA data

One specific issue with the HFEA data is updating of patient history. For individuals with several cycles it has been confirmed by the HFEA that variables such as previous pregnancies, infertility status and previous treatments are not updated from cycle to cycle. In other words, the history recorded on the database is always that at the time the couple first attended for treatment. We have updated the dataset where possible to adjust for this but obviously there may be further errors in the dataset arising from this issue. Attempt number, as defined by the HFEA, includes all previous treatments with and without embryo transfer, being derived from the number of registered cycles.

LR model development for fresh cycles

We adopted a strategy whereby the variables to be included in the model were pre-specified and included as main effects regardless of statistical considerations. As there was an excess of data, rather than assume a functional form for numerical variables such as age and numbers of embryos, these were fitted as categorical variables with a large number of groups. The relationships were expected to be highly non-linear. Preliminary exploratory analysis of the variable distribution was performed to select appropriate categorisation suitable across all the subsets of data to be used, avoiding groups with small numbers of observations. Three of the potential variables were very similar measures of the success of the stimulation and fertilisation process and were highly correlated: eggs collected, eggs mixed and embryos created. Owing to concerns over co-linearity, the most predictive of these variables was considered for the model: each was added individually to a model with number transferred and age group, and embryos created selected as having the lowest AIC.

The variables included in the models with their categorisation are shown in Table 17.

*Interactions

Interactions were added to the model with all variables according to the BIC. The reason for choosing the BIC was to prevent overcomplicated terms being included purely as a result of the size of the dataset. Whereas the AIC penalises according to the number of parameters in the model only, the BIC also penalises on the size of the dataset. We considered interactions of each variable with each of the three most important variables (number transferred, age and embryos created). Thus there were 14 + 13 + 12 = 39 interactions tested. No interactions met the inclusion criteria for the model for all fresh cycles and so these were not considered further.

LR models including frozen cycles

In order to compare the outcomes of fresh and frozen cycles the full model for all cycles was refitted to the combined fresh and frozen cycle data. The number of embryos created was not available for the frozen cycles; for these cycles, this was set to a new category ‘frozen’. Thus, the coefficient for ‘frozen’ is comparable to a fresh cycle with six embryos created. Transfer day was also removed from the model as it does not have a simple interpretation for frozen data.

A nested random effects model to allow for clustering at the centre and patient levels was used, as with the fresh cycles. It would have been desirable to include each egg collection as a clustering variable but this information was not present in the HFEA dataset owing to a lack of linkage between the frozen cycles and the fresh cycle in which the embryos were created.

Each variable was tested for an interaction with a fresh/frozen binary variable (using the fixed model for computational reasons), allowing us to see if individual variables behave differently in frozen cycles. These interactions were added sequentially, with the variable with the most significant value for the LRT being included first. As there was some evidence of different behaviour in frozen cycles for some variables (in particular, age), to enable comparison a model for frozen cycles alone was fitted. This was equivalent to the model for fresh cycles, apart from the number of embryos created which was not present, but a new variable, number of embryos thawed and viable, was included instead. This enables direct comparison of the effects of variables on the fresh and frozen cycles.

Model for LBEs in all fresh cycles

The model parameters are shown in Table 18 and the fitted LBE rates are shown in Figures 8 and 9. Table 18 shows the coefficients for the LR model for a successful treatment outcome from a single fresh cycle. The left-hand columns give the estimates for a fixed effect model – i.e. without any allowance for intercouple correlations – while the right hand columns show the estimates for the model which includes a couple random effect. There is a strong dependence with age as expected (see, for example, Templeton et al. 79), with a steep decline in success rates after around age 32. The number of embryos created is a strong prognostic factor – more embryos created gives more from which the embryos to be transferred can be selected – and so acts as a surrogate indicator of embryo quality. It is likely that the embryo number also acts as a surrogate for hormonal status as this is correlated with the number of eggs collected. 2,113

FIGURE 8.

LBE rates from the LR model fitted to the HFEA dataset.

FIGURE 9.

LBE rates from the LR model fitted to the HFEA dataset.

We note that although there are strong dependencies on age (and the other covariates), there is a wide variation between individuals of any given age, with young patients with poor prognosis and old patients with relatively good prognosis.

Cycles with a single embryo transferred did worse, with an odds ratio for LBEs of about 0.5 compared with DET. This would be expected as a consequence of fewer embryos but also reflects patient selection as the patients who received SET in this series would have different characteristics. In particular, many of these would have clinical conditions contraindicative of multiple pregnancies. Many would have had only a single embryo available: although this is allowed for in the model, embryo quality is not, and patients with few embryos would be likely also to have poorer-quality embryos transferred. Three embryo transfer does not improve LBEs as might be expected; this reflects the restriction on its use to only poor prognosis situations – again the likelihood that these cycles have only poor-quality embryos is not accounted for in the model.

Day 2 and 3 transfers have similar outcomes, and longer culture times do lead to a higher per transfer LBE rate. However the data do not include information on the numbers of cycles in which no embryos survive for transfer, so we cannot conclude from this analysis that the overall success rate per full cycle is improved.

The number of previous attempts is only weakly prognostic, and might be expected to be larger. 114 However, as noted above, the data on attempt number are not totally reliable and so there may be some attenuation of this effect due to measurement error. Additionally, there are strong and complex selection effects due to treatment policies, availability of funding and patient choice, which means that the patient populations who receive more treatments are not the same as those who receive less. The weak effect of duration of infertility is subject to the same caveats. A positive birth history is unsurprisingly associated with better outcomes. 69,79

There are three related variables associated with male infertility. ICSI is associated with marginally worse outcomes. A male cause of infertility is associated with better outcome, as is the use of donor sperm, these two variables being strongly related to each other. Given the strong relationships between these three variables, it is difficult to assign direct causal effects to any specific variable.

A tubal diagnosis is associated with worse outcomes, as has been noted in other work. 52 An idiopathic diagnosis shows a weak association with better outcomes, which is of borderline statistical significance. Even in this very large dataset the effects of other diagnoses did not reach statistical significance once the associated factors were accounted for.

There was a small, but significant, improvement in outcome over the period of the cohort.

Models for LBE and twin rates in fresh DET cycles

Tables 19 and 20 give the parameter estimates for LBEs and twins, given an LBE for fresh DET cycles, and the predicted outcomes for selected variables are plotted in Figure 10. Results for LBEs are consistent with the full dataset. The parameters of the twin model are generally similar to that of the LBE DET model, indicating little differential effect of prognostic parameters for twins, as for LBEs.

FIGURE 10.

Fitted success (LBEs) and twin rates for fresh DET transfers in the HFEA dataset. Live births (left); twins per LBE (right).

*Random effects estimates from the mixed logistic regression models

In order to account for the correlations between multiple cycles from the same patients, and between patients from the same clinics (treatment centre effects), we included patient and centre effects as random effects in the LR models. Inclusion of these random effects did not affect the estimates of the other model parameters. Such models are computationally expensive to fit, and a number of approximations have been developed to speed up the computation. However these approximations are known to perform badly in logistic models: although the fixed effect estimates are reliable, the estimates of the random effects are often biased. 115 Thus, there is a pay-off between accuracy in estimation of random effects and practical fitting of the models. In this dataset it was not practical to fit the full dataset with a reliably accurate method [adaptive quadrature (AQ)] and a Laplace approximation had to be used to make the fitting feasible. Therefore, we derived a smaller dataset of approximately 10% the size by sampling 100 patients from each treatment centre (or all patients if the number was less than 100). Even this relatively small sample of the HFEA dataset took a couple of days to fit via adaptive quadrature, using the stata glamm procedure. 116 In this dataset the Laplace approximation gave a reasonable fitting time but appeared to underestimate the random effects at the patient level.

Table 21 shows the random effect estimates, expressed as the standard deviation of the distribution of effects between centres and patients (within centre). The Laplace approximation does underestimate the magnitude of the effects, particularly the lower-level patient effects. There are reasonably large intrapatient effects, with the standard deviation being equivalent to the difference between a 35- and a 39-year-old patient. There are differences between centres that are of a magnitude to be clinically relevant: the standard deviation of 0.35 on the log-odds scale is equivalent to an odds ratio of 1.4. The random effect for the twin model is small, reflecting the fact that once LBEs are accounted for, the twin rates do not vary much.

*LR model validation

Figure 11 shows the observed and fitted data plotted against age and number of embryos created, showing that, overall, the models were a good fit to the dataset. There is no evidence of any systematic lack of fit in the model, but there is some evidence of overfitting and a somewhat coarser categorisation of the continuous variable might be more realistic.

FIGURE 11.

Fitted and observed outcomes for the LR models for the two most predictive variables in the HFEA dataset. Live births (left); twins per LBE (right).

Table 22 gives the area under the receiver operating characteristic (ROC) curve (AUC) for the various LR models, a measure of the goodness of fit; a value of 0.5 implies that the model prediction is essentially random while a value of 1 gives perfect classification. The fits are reasonable; with the random effect increasing the AUC fit by a modest amount. Clearly the models would not be useful for predicting individual outcomes, but this does not preclude their usefulness for predicting population outcomes. The fitted AUC values are in line with those found by Hunault et al. ,53 who observed AUC values of 0.68 with a logistic regression to model pregnancy and 0.71 when modelling a twin versus no-twin outcome (as opposed to the conditional model used here).

Table 23 shows the results of a bootstrap-based calibration procedure. 104 This procedure is designed to assess whether the fitted model displays characteristics of overfitting, with a well-fitting model giving a calibration intercept close to 0 and slope close to 1. There is no evidence of serious overfitting, despite the large number of parameters included in the model.

It is possible that observations with several previous cycles may not behave the same in the model as those from earlier cycles. Therefore, as a sensitivity analysis, the model was fitted to first cycles only and the parameter estimates compared with the model above (Table 24). No differences were observed, and there was no evidence that variables behave differently in models for all cycles or of an interaction between cycle number and other effects (based on the BIC).

LR models including frozen cycles

If we fit the fresh and frozen data to the same model, allowing for an overall (intercept) difference in LBEs due to freezing, we obtain an estimate of b = –0.52 (SE 0.03) for the freezing effect compared with a fresh cycle with six embryos created. This translates to an odds ratio of 0.6 (95% CI 0.57 to 0.64). It would not be unexpected to see an odds ratio a little less than 1 for frozen embryo transfer as the majority of patients will have already undergone an unsuccessful fresh cycle; however, this result does indicate that frozen cycles are less successful.

There is evidence that the effect of freezing varies according to some of the clinical characteristics, and Table 25 shows the results of the tests for interaction that proved to be statistically significant (with each variable added sequentially – no further variables were significant).

The fit to the frozen data is shown in Table 26, which should be compared with the fresh cycles in Table 18. Figure 12 compares the predicted outcomes for the fresh and frozen transfers. Success rates in frozen transfers are similarly lower for all patient characteristics. Overall, the effects of the clinical characteristics are very similar between fresh and frozen transfers once the much lower overall success rate is taken into account.

FIGURE 12.

Fitted success (LBE) rates for fresh and frozen transfers in the HFEA dataset. Fresh transfers (left); frozen transfers (right).

The decline in success with age appears to be somewhat less in frozen transfers, reflecting the statistically significant interaction. In this data we did not have the age at which the embryos were created (there is no link to the fresh cycle), so it is possible that this is due to the fact that transfers in older patients utilise embryos created when the patients were younger. Three-embryo transfers did relatively better in frozen cycles compared with fresh cycles. During this period three-embryo transfers should have been undertaken only in exceptional circumstances, and it is likely that this effect reflects different selection criteria for fresh and frozen transfers and therefore different patient populations. Also, without a measure of embryo quality, it is not possible to ascertain whether this difference reflects different quality embryos in the three-embryo transfers between fresh and frozen transfers. The third characteristic that showed a difference between fresh and frozen transfers was the attempt number. The effect of attempt number was attenuated in frozen cycles compared with fresh. Attempt number contains a mixture of fresh and frozen transfers. Many clinics and patients do not undertake frozen transfers, so the population of patients who have these will differ from those who do not. Additionally, as the success rate of frozen transfers is low, patients who have these will have a larger number of failed transfers. These biases are likely to be sufficient to explain the differential effect of attempt number seen here.

The interaction effects are all small in nature and, while potentially of interest mechanistically, probably reflect issues in the data structure and are of little practical significance.

Limitations and caveats

The data do contain a large number of errors, and we must assume that there are many we cannot detect by simple consistency checks. There is no evidence that these errors are anything other than apparently random data entry errors, but we cannot rule out appreciable bias. In particular the measurement error will attenuate the effects of covariates. Attempt number is particularly problematic in this context, particularly with respect to frozen cycle attempt numbers, which appear to be incorrectly coded in some cycles (see Characteristics of the dataset). The lack of linkage of frozen cycles to their parent fresh cycle is a major limitation in the use of these data.

There are only a limited number of variables available, but these do include most of the relevant prognostic factors. The major exception to this is the lack of any measure of embryo quality, other than the number of embryos created.

The size of the dataset, with over 100,000 cycles and a large number of covariates, imposes limits on the computational feasibility of the analyses, particularly when we include random effects in the models. Due to these computational constraints and the limited linkage in the data, only simple correlation structures have been considered, and we cannot rule out more complex correlations as might, for example, be represented by random coefficient rather than random intercept models. Models can take many hours to run and, as such, preclude some of the computer-intensive methods that we might otherwise employ.

The data reflect the 2000–5 time period – the data that were available at the start of the project. Practice and success rates are continually improving and the absolute success rates may not apply to current patients; however, it is unlikely that for fresh cycles the differential between patients with different prognoses has changed appreciably in this time period. However, for frozen cycles there have been some additional changes in practice, so some caution is required in applying these results to current patients.

Impact of cryopreservation

The outcomes from frozen transfers in this dataset were generally poor compared with current best practice, and many centres performed few or no such treatments. Thus, it is very likely that the results here provide a lower bound on the success rates that can be achieved with best current practice. The odds ratio for frozen versus fresh transfers of 0.6 reflects a significant loss in viability due to the freezing and thawing processes, although of course selection effects will contribute to this loss. Although there are caveats around the quality of the cryopreservation techniques, there was no evidence of any substantive influence of patient factors on the loss in success due to freezing.

Conclusions

The analyses presented here refer to individual embryo transfer procedures, the data not allowing any meaningful analyses of cumulative outcome beyond the estimation of simple intercycle correlations. We can however demonstrate that there are correlations between outcomes from repeat cycles from the same couples; although these effects do not materially affect estimates of the influence of covariates, they will be important in predicting more relevant outcomes that span multiple transfers across a treatment course. These effects may reflect unmeasured (or inaccurately measured) prognostic factors.

The results for prognostic factors are consistent with other studies, but are estimated here with greater precision due to the large cohort. These factors are similar for live birth outcomes and twin outcomes, with no factors or combinations of factors being identified that specifically predict twin outcomes.

The data on frozen cycles is of poorer quality but does suggest that predictors of success are similar to those in fresh cycles.

Data sources

Data were obtained from five centres (Table 27). Each centre has a database on to which details of every assisted conception treatment are entered, and these data are used as the basis for statutory returns to the HFEA as well as for internal monitoring, audit and, in some cases, costing. Data were requested from 2000 to 2005, this representing a 5-year cohort with full pregnancy outcome at the start of the project.

Centres provided either reports from the databases or raw database tables. Any identifying information was removed by local centre staff and the database/report files transferred electronically to Manchester, where they were assembled into a common format as detailed below.

Data selection

Cycles were included if they met all the following inclusion criteria:

• treatment type: ICSI or IVF

• cycles with one, two, or three embryos transferred

• age 19–54

• patient’s own eggs

• date started trying or last pregnant after 1980.

Cycles were excluded if they met any of the following exclusion criteria:

• donor eggs

• frozen/thawed eggs

• natural or HRT induction.

Additionally we defined a core set of covariates, and cycles were excluded if any of these variables were missing:

• embryo cell number and grade

• age

• number of previous attempts

• number of previous pregnancies and live births

• diagnosis

• duration of infertility

• number of eggs collected in fresh cycles

• number of embryos created in fresh cycles or thawed in frozen cycles

• transfer day (for fresh cycles)

• treatment centre

• year of treatment.

*Data cleaning

*Normalisation of embryo grading across sites

All sites nominally used similar embryo grading criteria, modified from Steer et al. 117 (although the labelling was reversed in some sites relative to the others – here we adopt the convention that increasing score implies increasing quality). However, preliminary exploratory analysis suggested that the grading schemes were not consistent between the sites: the proportions given each grade differed substantially between sites, and preliminary models incorporating grade showed a significant grade by site interaction. We therefore normalised the grades: we based the normalisation on an assumption that the underlying embryo grade distribution was the same across sites and computed a site-specific score for each site to maintain this condition. This normalisation method was justified post hoc by the lack of site by grade interaction in the fitted models.

Embryo cell number (stage) was transformed to log2(cells)/(days in culture), using the recorded time in culture. Where culture time was unavailable, a mean value, given the transfer day, was used. Thus we represent embryo growth by the number of cell doublings per day, allowing day 2 and day 3 embryos to be assessed on the same scale.

The analysis datasets

The dataset as obtained contained 19,373 embryo transfers, 14,373 fresh and 5000 frozen from five centres as shown in Table 27. After cleaning the total number of cycles available for analysis was 16,096: 12,487 fresh and 3609 frozen, from 9040 couples (Table 29).

Logistic regression models for success (all data)

Table 33 shows the coefficients for the LR model for successful treatment outcome from a single fresh cycle. The left-hand columns give the estimates for a fixed effect model – that is without any allowance for intercouple correlations – while the right-hand columns show the estimates for a model which includes a couple random effect. The parameter estimates are almost identical for the two models. The coefficients for the cubic spline functions are not shown as they have no ready interpretation, but plots of the stage and grade functions are shown in Figure 14. The patient random effect has a variance of 0.28 corresponding to a standard deviation of 0.54 on the log-odds scale. The correspondence in parameter estimates of the fixed effect and random effect models suggests that the simpler fixed effect model would be adequate for prediction for individual cycles.

FIGURE 14.

Spline functions for embryo growth rate and grade in the LR model for the full towardSET? dataset. Grade is shown on the normalised scale with 0 being poorest- and 1 best-quality embryos; the vertical lines indicate the mean normalised grade for embryos graded 1:4 as indicated. Note: in the left-hand panel the curve without REs is obscured behind that with REs.

In Figure 15 we show the fitted LBE probabilities plotted against the major prognostic factors. As would be expected, age is the major predictor of success, with an accelerating decline from around age 32 and very low success rates by age 43–45. The number of embryos created is a moderately strong predictor of success: here we note that for any given treatment (solid line in Figure 15) the success rate was lower if only small numbers of embryos were available, but the predicted rate across cycles (boxes) with low numbers of embryos was a lot lower. This reflects the fact that where there are small numbers of embryos to select from, the resultant quality of the transferred embryos is likely to be lower. Couples who have been infertile for a longer time are less likely to have a successful outcome, although this may in part reflect a patient selection effect as patients funded by the NHS will generally only be treated after a longer period of infertility compared with privately funded patients. There was a small decrease in success with later attempts, but it is important to note that the vast majority of later attempts will be from patients with previous failures, so patients having later cycles will be more likely to have a poor prognosis. Having had a previous successful pregnancy (but not an unsuccessful one) is associated with better outcomes. Diagnosis generally only had a small impact on success outcome, with a tubal diagnosis and endometriosis being the only ones that reach statistical significance. A tubal diagnosis led to a poorer prognosis (as seen in other studies such as Strandell et al. 52), while endometriosis was associated with slightly better outcomes.

FIGURE 15.

LBE probabilities from LR model for principal variables.

There were highly significant associations between treatment success and centre and year which are difficult to interpret. There was little evidence of a consistent improvement over time in this cohort. The differences between patients treated in the various centres is not explained by the covariates in the model, but may still reflect patient selection effects and not differences in treatment quality.

As noted by others,2,118,119 successful transfers are associated with embryo cell numbers close to one doubling per day and slow- and fast-growing embryos both do poorly. Embryo grade is consistent with the need for one grade 3 or 4 embryo to have a good chance of success. However embryo quality is hard to assess reliably in a model that requires the aggregation of the transferred embryos.

LR models for success and twins in DET

Here we consider only those patients who had DET. The fixed and random effect models for success had again almost identical parameter estimates (Table 34, Figure 16), and the estimate of the within-patient standard deviation for LBE was 0.53.

FIGURE 16.

Spline functions for embryo growth rate and grade in the LR model for DET. Grade is shown on the normalised scale with 0 being poorest- and 1 best-quality embryos; the vertical lines indicate the mean normalised grade for embryos graded 1:4 as indicated. The arbitrary intercepts have been set to enable comparison between LBE and twin models.

Very few patients had more than one twin birth, so it was not possible to estimate a random effect for the twin model. The parameter estimates for twins, given an LBE, are shown in Table 35 and Figure 16. Figure 17 shows both the fitted models graphically according to patient characteristics, with Figure 18 showing predictions as a function of the averaged embryo characteristics.