Selecting pregnant or postpartum women with suspected pulmonary embolism for diagnostic imaging: the DiPEP diagnostic study with decision-analysis modelling

Study population

Diagnosed PE: the UKOSS research platform was used to identify a sample of pregnant or postpartum women who were diagnosed with PE in the UK after presentation with a suspected PE. We identified and collected data from any woman diagnosed with PE at a hospital participating in the UKOSS platform between 1 March 2015 and 30 September 2016.

Suspected PE: we recruited a sample of pregnant or postpartum women investigated for a suspected PE across 11 participating sites. We anticipated that 98% of women in the sample would have no confirmed diagnosis of PE and would constitute the control group. Those with a diagnosis of PE confirmed would be analysed with the patients with PE.

Inclusion/exclusion criteria

Diagnosed PE: we included women with PE if they met any of the following definitions:

1. pulmonary embolism confirmed using imaging (angiography, CT, magnetic resonance imaging or VQ scan showing a high probability of PE)

2. pulmonary embolism confirmed at surgery or post mortem

3. clinical diagnosis of PE resulting in a course of anticoagulation therapy for > 1 week.

Women meeting criterion 1 or 2 were included as patients with PE in the primary analysis. Women with a clinical diagnosis of PE (criterion 3) were excluded from the primary analysis, but were included as patients with PE in the secondary analysis. This was because of the risk of incorporation bias if the clinical reference standard diagnosis of PE was based on the clinical variables or biomarkers being used as index tests.

We excluded women who did not present with a suspected PE prior to diagnosis (i.e. with PE identified as an incidental finding). We collected data from women who required life support at presentation (chest compressions or assisted ventilation) to allow for the estimation of the incidence of PE in pregnancy, but did not include these women in the analyses in this study.

Suspected PE: we included any pregnant or postpartum woman presenting to the participating hospitals who was considered to require diagnostic imaging for a suspected PE. Women were recruited once the clinician had decided that imaging would be required. However, not all women received lung imaging; in a proportion of patients, the decision that imaging was required was reversed by a more senior clinician, and some women received imaging only for DVT (e.g. venous ultrasound). Women who had PE ruled out clinically (i.e. without diagnostic imaging of the lungs for PE) were excluded from the primary analysis, but included in the secondary analysis. This was because of the risk of bias if the PE was missed as a result of a lack of adequate imaging and the decision not to undertake imaging was based on the clinical variables or biomarkers that constituted the index tests.

We excluded women who needed life support on presentation to hospital (chest compressions or assisted ventilation), women who had been diagnosed with PE earlier in the current pregnancy, women who were unable or unwilling to provide informed consent, women aged < 16 years and women previously recruited to the study. The form used for screening is shown in Report Supplementary Material 1 (the suspected PE screening form).

Setting/context

Sampling

Data collection and follow-up

Sample size

The sample size for the UKOSS data were inevitably determined by the incidence of PE during the data collection period. Based on a previous similar study,27 we anticipated that we would identify 150 patients with diagnosed PE over the 18 months of the study. We aimed to recruit 250 women with suspected PE over the same time period, resulting in around 155 women with PE, and 245 women without PE in the control group, assuming that the prevalence of PE is 2% in those with a suspected PE. This would allow for the estimation of sensitivity or specificity of 90% with a SE of around 2.5% and 2.0%, respectively. Assuming that the ratio of women with PE to women without PE in the control group would be around 0.4, this sample size would be sufficient to identify an odds ratio of a clinical predictor of around 2, with 90% power and 5% two-sided significance. 44

With a limited sample size, the complexity of any statistical model was limited in terms of the number of predictor variables that could be included. This was addressed by selecting only those variables that were felt to be clinically important in the model and by aiming for maximum parsimony in the final model. The statistical analysis outlined in the study proposal stipulated modelling outcomes by splitting the cohort into a training data set and a validation data set. This effectively reduces power, so, in developing the statistical analysis plan, an alternative approach of leave-one-out cross-validation was planned instead.

Data management

Diagnosed PE data and suspected PE data were collected on study-specific case report forms (CRFs) and entered onto a secure electronic data capture system at UKOSS and the Clinical Trials Research Unit (CTRU) at the University of Sheffield. The prospective data were managed via the CTRU Prospect system (epiGenesys, University of Sheffield, Sheffield, UK), which used inbuilt validation to promote high-quality data and security management features to ensure data confidentiality. On completion of the study, pseudonymised UKOSS data were encrypted and uploaded to the Prospect system. Physical data were stored in accordance with good clinical practice and local standard operating procedures.

Ethics and research and development approvals

Ethics approval for the study was obtained from the London Brent Research Ethics Committee (reference 14/LO/1695). A feasibility assessment for each participating site was undertaken by the research team and in accordance with local research governance procedures. Permission to undertake the research study was granted and reviewed in response to project amendments.

Patient and public involvement

Four members of the public, representing interests in obstetric care and emergency medicine, were actively involved in the oversight of the study via membership of the Study Steering Committee. Patient and public involvement (PPI) representatives influenced the development of study documents and data collection, shaped the dissemination strategy and commented on outputs. PPI representatives also engaged with external organisations such as the Sheffield Emergency Care Forum and Thrombosis UK.

Analysis populations

The primary analysis population consisted of women recruited with a suspected PE and women identified through UKOSS with a diagnosed PE. Women who presented to hospital needing life support were excluded from the suspected PE sample and were used in the UKOSS sample only to estimate the overall PE incidence. They were therefore excluded from all analyses reported in this study. We also excluded any women from the UKOSS sample if it was not recorded whether or not they presented to hospital needing life support, unless presenting physiological data showed that life support would not have been needed.

We planned a priori that the primary analysis should be limited to participants with diagnostic imaging, surgery or post-mortem confirmation of PE or in whom PE had been ruled out by diagnostic imaging, and thus included only patients without diagnostic imaging, surgery or post-mortem confirmation in the secondary analyses. We also planned a priori to undertake the secondary analysis excluding isolated subsegmental PE, as the identification of subsegmental PE on imaging may be unreliable and the need for treatment may be uncertain.

We identified any duplicates between the UKOSS data set and the women with a suspected PE. If the woman was recruited with a suspected PE, then the corresponding UKOSS data were removed from the overall data set. If the woman was identified but not recruited with a suspected PE, then the UKOSS data were retained. The anonymised data relating to women’s presentation with a suspected PE were used in a descriptive analysis of the non-recruited patients.

Reference standard classification

We planned that the classification of participants as having PE (PE present) or not having PE (PE absent) should be based on the results of imaging, thromboembolic events and the evidence of treatment for PE, regardless of whether or not participants were recruited with a suspected PE or identified as having a diagnosed PE through UKOSS. Two independent assessors (SG and CNP), who were blind to the clinical predictors and the blood results, used a structured process to classify the diagnostic imaging results, the details of adverse events and the details of treatments given, and thus to classify all participants and non-recruited participants as PE present (women with PE) or PE absent (control group participants). Details of the structured process are provided in Appendix 1. Disagreements were resolved through adjudication by a third assessor (FL). This process also classified how participants would be handled in the primary and secondary analysis.

We structured the process of classification for primary and secondary analysis around the following principles:

Primary analysis –

• Pulmonary embolism was present if lung imaging was reported as showing PE or if venous imaging showed DVT in the presence of symptoms indicating suspected PE (i.e. if the patient met the eligibility criteria), if surgery or a post-mortem examination revealed PE or if the 30-day follow-up identified a subsequent diagnosis of PE.

• Pulmonary embolism was absent if lung imaging was reported as negative for PE, unless the 30-day follow-up identified subsequent PE.

• Pulmonary embolism was absent if lung imaging was non-diagnostic, no treatment was given for PE (defined as therapeutic anticoagulation for at least 1 week) and no subsequent PE was identified on follow-up.

• Women with clinically diagnosed PE were excluded (i.e. if lung imaging was non-diagnostic, but treatment was given for PE).

• Women with clinically ruled-out PE were excluded (i.e. if lung imaging was not done).

The secondary analyses examined the following reclassifications –

• the inclusion of women with clinically diagnosed PE as PE present (i.e. women with no lung imaging or non-diagnostic lung imaging who received treatment for PE)

• the inclusion of women with clinically ruled out PE as PE absent (i.e. women with no lung imaging who did not receive treatment for PE)

• the exclusion of women with isolated subsegmental PE.

Clinical variable classification

Clinical variables that could be diagnostically useful were classified on the basis of a priori categorisation as to whether the variable was present or absent. For most patients, this was on the basis of an expected association between a variable and the presence or absence of PE. If the variable was present, then it was expected that PE would be more likely to be present. Continuous variables were determined from the expert opinion of DiPEP co-investigators, existing criteria used in the relevant decision tools,2 or widely acknowledged physiological definitions (e.g. tachycardia) to give clinically meaningful classifications. Table 4 outlines the classification.

Other previous medical problems and other problems in the current pregnancy were analysed in two ways using the lists above. First, they were analysed using any other previous medical problem or problems with the current pregnancy as the predictor of PE (lists 1 and 2). Then, they were limited to other previous medical problems and problems in the current pregnancy that were known to be associated with an increased risk of VTE, based on the outcome of developing the expert consensus-derived CDRs. Problems specifically tested as a separate predictor (e.g. previous thrombotic event) were not included.

The following ECG abnormalities were classified as being PE related: SI QIII TIII pattern, complete or incomplete right bundle branch block, right axis deviation, simultaneous T-wave inversions in the inferior (II, III, aVF) and right precordial leads (V1–3), right atrial enlargement (peaked P wave in lead II of > 2 mm in height), clockwise rotation (shift of the R/S transition point towards V6, persistent S wave in V6) or atrial tachyarrhythmia (atrial fibrillation, flutter or tachycardia).

Chest radiographs were classified by reviewing the radiology report produced as part of clinical care. If the report mentioned the PE or pulmonary infarction in describing radiographic changes or identified potentially PE-related findings (such as atelectasis) without providing an alternative explanation for the finding, then we classified it as referring to a PE-related abnormality. All other abnormal radiographs were classified as having other abnormality.

The diagnostic impression was reviewed by one of the investigators (SG) and classified by whether or not PE was at least as likely as any other diagnosis, based on with the criterion used in Wells’s PE score. 3 This was done at two levels: (1) a strict judgement in which only diagnostic impressions that clearly stated that PE was at least as likely as any other diagnosis were included and (2) a permissive judgement in which any diagnostic impression that suggested that PE was as least as likely as any other diagnosis was included.

D-dimer measurements were recorded as part of routine care in a proportion of women with suspected PE and women with diagnosed PE in the UKOSS cohort. The biochemical analysis was undertaken in many different hospitals, using different assays with different diagnostic thresholds. Furthermore, we expected specificity to decline with gestational age. We therefore planned to test a threshold for positivity for D-dimer that varied with gestational age and was defined in relation to the threshold used for that assay at the relevant hospital rather than as an absolute value. We used the standard threshold during the first trimester, 1.5 times the standard threshold for the second trimester and two times the standard threshold for the third trimester and post partum. This was based on data showing how D-dimer levels increase during pregnancy45 and evidence that a higher threshold may improve specificity for diagnosing VTE in pregnancy without sacrificing sensitivity. 6

Missing data

The process for classifying the reference standard (PE present or absent) described above outlines how data relating to imaging, follow-up and treatment are handled. In general, if data were missing, it was assumed that imaging was not performed, treatment was not given or follow-up was negative. However, all data were presented to the independent assessors so that they could take the presence or absence of data into account when making their judgement.

For the clinical variables, there was scope for missing data if the attending clinician failed to measure or record the data, or if the UKOSS clinician or research nurse/midwife was unable to access the necessary hospital records. Analyses involving multiple clinical variables (i.e. multivariable regression and analysis of decision rules) have to either impute missing variables or exclude every patient with a missing variable. In these analyses, we included patients with small numbers of missing data, with missing variables imputed as being normal or negative, and excluded patients with large numbers of missing data (i.e. if any of the following criteria were met):

• more than one of heart rate, respiratory rate and oxygen saturation were missing

• more than half of the variables relating to previous medical history were missing

• more than half of the variables relating to the current pregnancy were missing.

Our rationale for this approach was that if large numbers of data were missing, then any assumption about the pattern of missing data would be speculative and it would be best to exclude the patient, whereas, if only a few variables were missing, it would be more likely that these were not recorded because they were expected to be normal or negative. Imputing missing data as normal or negative would tend to overestimate specificity and underestimate sensitivity. We felt that this represented a conservative assumption that would accord with clinician willingness to accept a degree of overinvestigation and unwillingness to accept the risk of missed PE. We felt that imputing abnormal or positive values when variables were missing, especially in validating CDRs, would be met with scepticism by the clinical users of our findings and would undermine the clinical credibility of these findings.

Planned analyses

Demographic and baseline characteristics were presented descriptively for the cohorts with diagnosed PE and suspected PE, and the eligible but non-recruited patients with suspected PE. Demographics, baseline characteristics and prevalence of PE diagnosis were then compared between the recruited women and the non-recruited women with suspected PE to explore whether or not those recruited were a representative cohort.

The cohort with diagnosed PE and the recruited cohort with suspected PE were then combined to form the main data set for analysis. The primary analysis was limited to women with PE confirmed or ruled out by imaging, surgery or post-mortem examination. Secondary analyses examined the effect of (1) including women with clinically diagnosed PE, (2) including women with clinically ruled-out PE and (3) excluding women with subsegmental PE as outlined above.

Clinical variables were compared between women with PE and those without PE. Univariable logistic regression was then used to determine the association between each variable and the presence or absence of PE.

Summaries of the responses to the assessment questionnaires were tabulated for each time point.

The accuracy of each index test was assessed by reporting and comparing the sensitivity and specificity. The combined sensitivity and specificity was assessed by plotting receiver operating characteristic (ROC) curves and quantified by the area under the curve (AUC). By virtue of the case–control design oversampling the proportion of patients with PE, no attempt was made to estimate the positive and negative predictive values.

We retrospectively applied each of the decision rules derived by expert consensus (see Chapter 2) and the following existing decision rules to the data to estimate diagnostic performance:

• the PERC rule5

• the Wells’s score3

• the revised Geneva score. 4

These rules were not developed for use in the pregnant population, so we removed criteria that were not relevant to the pregnant population and adapted criteria when appropriate to be relevant to the pregnant population. We therefore removed exogenous oestrogen from the PERC rule and used the thresholds developed for our analysis of clinical variables to dichotomise age, oxygen saturation and heart rate (see Table 4). The need to design a CRF that would be usable for both prospective and retrospective data collection and that would address the multiple study objectives meant that criteria used in each rule did not always map precisely onto variables in the CRF. Furthermore, data for some of the variables were missing. Appendix 2 outlines how the criteria in each rule were applied to the study data.

The diagnostic performance of CDRs is normally presented as the proportion in each risk stratum with the outcome of interest (in this case PE). This is similar to reporting positive and negative predictive values for a diagnostic test. This approach would be misleading in the DiPEP analysis, because the prevalence of PE in the analysis cohort has been deliberately inflated using the UKOSS data to increase the precision of estimates of sensitivity. Positive and negative predictive values are dependent on sensitivity, so the proportion of PE in each stratum would be much higher than if the rule was used in a typical clinical cohort with low prevalence. We therefore assessed the accuracy of each index test by calculating sensitivity and specificity at the usual or recommended decision-making threshold, plotting ROC curves and quantifying the AUC. The consensus-derived rules each specified a threshold for the rule being positive or negative. The PERC score was considered positive if any criterion was positive. The Wells’s criteria score was considered to be positive if the score was ≥ 4 points (PE likely). The simplified revised Geneva score was considered to be positive if the score was ≥ 4 points (moderate or high risk).

D-dimer (as measured in the hospital laboratory and recorded in the clinical notes) was analysed as a separate index diagnostic test, rather than as one of the clinical variables, and was not included in the CDRs, multivariable analysis or recursive partitioning. The diagnostic accuracy was assessed by calculating the sensitivity and specificity at the hospital laboratory threshold and the pregnancy-specific thresholds outlined previously. We did not use ROC analysis for D-dimer because of the complexity of having to use different thresholds for different assays across the multiple hospitals contributing data to the study.

Methods of statistical modelling

Three statistical modelling approaches were used in the development of new decision rules:

1. Logistic regression.

Univariable logistic regression analyses were undertaken to identify associations between clinical features and diagnosis of PE.

1. The least absolute shrinkage and selection operator (LASSO).

The LASSO regression modelling approach was used to assess the predictor variables and estimate the effect of each variable on the outcome. 46–48 This method is also based on multivariable logistic regression modelling, but addresses some of the problems of overfitting in the presence of multiple correlated covariates. The LASSO adaptation of logistic regression applies a penalisation against higher-dimension models, thereby helping to protect against coefficients being spuriously inflated. 49 We included all clinical variables in the analysis, regardless of their univariable association, but did not include receipt of thromboprophylaxis as a clinical variable. Thromboprophylaxis is targeted at women who are at risk of VTE and is intended to prevent PE. It is therefore likely to have a complex and inconsistent association with PE.

1. Recursive partitioning.

Recursive partitioning50 is a different approach which is used to create decision tree models. Rather than estimating a formula linking outcome to a combination of covariates, recursive partitioning attempts to identify subgroups with higher incidences of PE based on their characteristics, the aim being to derive a rule with optimal sensitivity (ideally of > 95%). This had the advantage that continuous variables did not need to be dichotomised beforehand, as the partitioning algorithm not only determines which subset of variables provides the optimal classification, but also the cut-off points within predictor variables. As is standard practice when performing recursive partitioning, cross-validation is employed at each partition to ensure that fully fitted trees were pruned based on a function of the complexity parameter that minimises the cross-validation error in order to avoid overfitting.

We intended that clinicians would review the derived models to ensure clinical credibility. We planned to purge variables that were considered to be inappropriate (e.g. if it was something that could not in practice be assessed) and then refit the models and recalculate the coefficients of the remaining covariates. Clinical opinion would then be sought to weight/round the coefficients into simple decision rules and decide upon a threshold for decision-making.

In accordance with the principles of reproducible research, all analyses were performed using a literate programming approach, which allowed the recreation of tables and figures at will by anyone experienced in using the software. The scripts were version controlled using the Git version 2.13 [(2017) Github Inc., San Francisco, CA, USA] control system and self-documenting. The statistical programming language R was used to undertake the statistical analysis.

Women with diagnosed pulmonary embolism

A total of 224 women were identified through the UKOSS between 1 March 2015 and 30 September 2016. We excluded 13 women because they were recorded as presenting with life-threatening features and a further eight women because presentation with life-threatening features was not recorded and the absence of life-threatening features could not be inferred from physiological data. We identified five women who had also been recruited with suspected PE (their data were removed from the UKOSS data set) and three whose characteristics matched women in the eligible but non-recruited data set (their data were retained in the UKOSS data set). Thus, we included data from 198 women with diagnosed PE in the study analysis.

Women with suspected pulmonary embolism

We originally planned to recruit across eight sites over 18 months at a rate of two per site per month to achieve a sample size of 250 women. However, we realised after developing the detailed statistical analysis plan and examining initial data that the exclusion of those with clinically diagnosed or clinically ruled-out PE from the primary analysis would potentially leave the primary analysis underpowered. We therefore increased the number of participating sites to 11 to ensure that the planned sample size of 250 women would be achieved for the primary analysis.

A total of 324 women were recruited across 11 participating sites between 15 February 2015 and 31 August 2016. The result of diagnostic imaging for PE was known by the research nurse/midwife at the time of consent for 46 out of 324 women (14%). A further 35 were eligible for recruitment but declined to participate and 95 were not asked to participate despite being eligible, usually because of the lack of availability of an appropriate person to undertake recruitment. Non-identifiable data were collected from this latter group of women, who formed the cohort of eligible but non-recruited women. Appendix 3 provides details of the recruitment process and the flow of participants through the study. The mean monthly recruitment rate per site was 1.7 women per month per site across the study.

Reference standard classification

Full details of the classification process are provided in Appendix 1. The 198 patients in the UKOSS data set consisted of 163 women with PE confirmed by imaging or post-mortem examination (160 by imaging, including seven women with subsegmental PE, two by post-mortem examination alone and one by imaging and post-mortem examination) and 35 women with clinically diagnosed PE (29 with equivocal imaging and six with no imaging recorded; all treated). Thus, 163 women were included as having PE in the primary analysis, 198 women were included in the secondary analysis including those with clinically diagnosed PE, and 156 women were included in the secondary analysis excluding those with subsegmental PE.

The 324 women recruited with suspected PE consisted of 18 women with PE confirmed by imaging (including one with subsegmental PE), five women with clinically diagnosed PE (three with equivocal imaging and two with no lung imaging; all treated), 259 women with PE ruled out after imaging (254 with negative lung imaging and five untreated after equivocal lung imaging) and 42 with PE clinically ruled out without lung imaging (none treated). Thus, 18 women with PE and 259 without PE were included in the primary analysis, 23 women with PE and 259 women without PE were included in the secondary analysis including clinically diagnosed PE, 18 women with PE and 301 women without PE were included in the secondary analysis including clinically ruled-out PE, and 17 women with PE and 259 women without PE were included in the secondary analysis excluding subsegmental PE. The prevalence of PE was therefore 7.1% (23/324) across all women with suspected PE and 6.5% (18/277) when women with clinically diagnosed or ruled-out PE were excluded.

The 95 eligible but non-recruited women with suspected PE consisted of six women with PE confirmed by imaging (including three with subsegmental PE), five women with clinically diagnosed PE (four with equivocal lung imaging and one with no lung imaging; all treated), 73 women with PE ruled out with lung imaging (71 with negative imaging and two untreated after equivocal imaging) and 11 women with PE clinically ruled out without lung imaging (none treated). The prevalence of PE was therefore 11.6% (11/95) across all non-recruited women and 7.6% (6/79) when women with clinically diagnosed or ruled-out PE were excluded.

The total numbers for the analysis were therefore:

• primary analysis – 181 women with PE, 259 women without

• secondary analysis including clinically diagnosed PE – 221 women with PE, 259 women without

• secondary analysis including clinically ruled-out PE – 181 women with PE, 301 women without

• secondary analysis excluding subsegmental PE – 173 women with PE, 259 women without.

Predictor variable completeness

Full details of predictor variable completeness are provided in Appendix 4. Missing data rates were generally higher in the women with diagnosed PE than in women with suspected PE. Only previous pregnancy problems, temperature and D-dimer were missing in > 5% in the suspected PE cohort, whereas employment, previous pregnancy problems, all physiological variables, ECG, likely diagnosis and D-dimer were missing in > 5% of the diagnosed PE cohort.

All seven physiological variables were recorded in 141 out of 198 women (71.2%) with diagnosed PE, whereas 44 out of 198 women (22.2%) had between one and three variables missing and 13 out of 198 women (6.6%) had four or more missing. The corresponding figures were 286 out of 324 (88.3%), 37 out of 324 (11.4%) and 1 out of 324 (0.3%) for recruited women with suspected PE, and 49 out of 95 (51.6%), 44 out of 95 (46.3%) and 2 out of 95 (2.1%) for non-recruited women with suspected PE.

Previous medical history data were complete for 190 out of 198 (96.0%) women with diagnosed PE, 323 out of 324 (99.7%) recruited women with PE and 88 out of 95 (92.6%) non-recruited women with suspected PE. Only 1 woman out of 198 women with diagnosed PE (0.5%) and 2 out of 95 non-recruited women with suspected PE (2.1%) had more than half of their data missing for this category.

Data relating to the current pregnancy were complete for 189 out of 198 women with diagnosed PE (95.5%), 324 out of 324 recruited women with suspected PE (100%) and 90 out of 95 non-recruited women with suspected PE (94.7%). Only 4 out of 198 women with diagnosed PE (2.0%) and 3 out of 95 non-recruited women with PE (3.2%) had more than half of their data missing in this category.

Overall, 15 out of 198 women with diagnosed PE (7.6%), 2 out of 324 recruited women with suspected PE (0.6%) and 2 out of 95 non-recruited women with suspected PE (2.1%) were excluded from the multivariable analysis and the analysis of CDRs, as they met our criteria for exclusion on the basis of having too many missing data.

Characteristics of the cohorts

Table 5 shows the characteristics of the women with diagnosed PE, the women with suspected PE and the non-recruited women. The mean age was 29.3 years for the women with suspected PE and 30.1 years for women with diagnosed PE. The mean age for mothers of live births in England and Wales was 30.3 years in 2015. 51 Most women were white British, but there were significant minorities of Asian Pakistani and Asian Bangladeshi women in the suspected PE cohort, reflecting the minority ethnic populations of the participating site catchment areas. The incidence of both suspected and diagnosed PE increased from the first trimester to the third trimester.

Table 6 compares the characteristics of the recruited women and the non-recruited women with suspected PE. There were no marked differences between the recruited women and the non-recruited women. In December 2016, the mean age at booking of pregnant women in the UK was 29.6 years and 46% were classified as being overweight or obese (BMI of > 25 kg/m²), so both groups were similar to the general UK pregnant population. 52

Follow-up of suspected pulmonary embolism cohort

There were no withdrawals from the prospective data collection and hospital records were viewed at the 30-day follow-up point for all participants. A questionnaire was sent to 321 out of 324 participants (99%) and 265 out of 324 participants (82%) received at least one reminder to return the questionnaire. Questionnaires were returned by 135 out of 324 participants (41%) with a mean response time of 52.4 days. There was insufficient follow-up information to rule out a VTE event for 16 out of 324 women (4.9%), which resulted in contact with the GP by local researchers, and 14 out of 16 questionnaires were returned by the GP.

Table 7 summarises the EQ-5D-5L data at the 30-day follow-up point from the 136 participants who completed and returned the questionnaire. Most women were in good health at 30 days, although a substantial proportion had slight problems with mobility, usual activities, anxiety/depression and pain/discomfort, and around 15% had moderate problems with usual activities or pain/discomfort.

Characteristics of women with and without pulmonary embolism

Table 8 compares the characteristics of women with and without PE in the primary analysis data set. Women with PE were more likely to be older, to be post partum, to have given up smoking during the current pregnancy, to have had a previous pregnancy lasting for > 24 weeks’ gestation, to have had previous pregnancy problems, to have had surgery in the previous 4 weeks, to have a history of VTE, to have a problem with their current pregnancy or to have received thromboprophylaxis. They were less likely to have a family history of VTE, varicose veins, multiple pregnancy or recent long-haul travel.

In terms of presenting characteristics, women with PE were more likely to have reported haemoptysis or syncope and less likely to have reported shortness of breath on exertion. There were only small differences in the proportion of women reporting pleuritic or non-pleuritic chest pain, shortness of breath at rest, cough, palpitations or other symptoms. Women with PE were more likely to have an elevated temperature or low peripheral oxygen saturation. There were only small differences in the proportion of women with a high heart rate or respiratory rate, and very few women had a low blood pressure. Women with PE were more likely to have an abnormal chest radiograph than those without PE, regardless of whether or not the abnormality was considered to be PE related. Women with PE were more likely to have an overall diagnostic impression suggesting PE, but only when a strict interpretation rather than a permissive interpretation was used.

Table 9 compares the mean age, BMI and physiological measurements between women with and without PE in the primary analysis data set. The distributions of these measures are shown in the figures in Appendix 5. Women with PE were slightly older, had a slightly higher mean BMI and mean heart rate and had a slightly lower mean oxygen saturation. There was little difference between women with and without PE, in terms of both the mean and the distribution. Many women with PE had normal physiological measurements and many without PE had abnormal physiological measurements.

Table 10 shows the results of the univariable logistic regression using the primary analysis data set. The only clinical features significantly associated with a diagnosis of PE (p < 0.05) were number of previous pregnancies lasting beyond 24 weeks’ gestation, surgery in the previous 4 weeks (including caesarean section), no history of varicose veins, no long-haul travel during pregnancy, higher temperature, lower oxygen saturation, overall diagnostic impression suggesting PE (strict interpretation) and chest radiograph abnormality (both PE related and non-PE related). Women who had received thromboprophylaxis were significantly more likely to have a diagnosis of PE.

Some of these associations suggest that known risk factors for PE are associated with a diagnosis of no PE. This may be explained by women having a lower threshold for seeking medical attention or clinicians having a lower threshold for investigation in the presence of known risk factors. Whatever the explanation, these counter-intuitive associations are unlikely to be useful in diagnostic decision-making, as they will not be clinically credible and any diagnostic value is likely to depend on implicit selection processes during presentation that may vary between settings.

Diagnostic accuracy of clinical decision rules

Table 11 reports the diagnostic accuracy of each CDR and Figure 1 shows the ROC curve. The sensitivity and specificity are reported for the recommended or usual threshold for decision-making (i.e. any score vs. zero for the PERC score, PE likely vs. unlikely for Wells’s criteria and low risk vs. moderate or high risk for the Geneva score). The ROC figure and the area under the ROC (AUROC) curve relate to the performance of the rule across the range of values, rather than just using the recommended or usual threshold.

FIGURE 1.

Receiver operating characteristic curves for CDRs.

The diagnostic accuracy of the rules was generally poor. The consensus rules were derived specifically for pregnant and postpartum women and were intended to identify a low-risk group of women who could be discharged without imaging, but performed little better than chance. The sensitive rule had good sensitivity (95%) but very poor specificity (4%), showing that sensitivity was achieved only by setting a very low threshold for positivity.

The existing CDRs were developed for the general population with suspected PE. We adapted the relevant criteria to make them appropriate for a pregnant population, but did not otherwise alter the rules. Wells’s criteria may have some modest diagnostic value if the criterion ‘PE is the most likely diagnosis or equally likely’ is applied in a strict way (i.e. it is only positive if PE is clearly considered to be the most likely or equally most likely diagnosis). The other rules performed little better than chance.

Appendix 6 provides figures showing how each rule performed across the range of scores. These show that a non-negligible proportion of women with PE were categorised in the lowest risk stratum of all of the scores. Only the sensitive consensus rule and Wells’s score (permissive application) had < 5% of women with PE in the lowest stratum and both of these methods categorised too few women without PE in the lowest risk category to be clinically useful (i.e. using the rule would not make a practically important difference to patient management).

Wells’s score and the simplified revised Geneva score are usually used to select low-risk patients for biomarker testing, whereas the PERC score is intended to allow discharge without testing if it is negative. Our analysis showed that a substantial proportion of women with PE have a negative PERC score. In applying the PERC score, we did not apply the criterion of exogenous oestrogen. If being pregnant or post partum were considered to carry the same risk as exogenous oestrogen, then the PERC score would be positive in all pregnant and postpartum women.

Appendix 7 shows the coefficients from logistic regression for each of the elements of each rule. This gives an indication of the contribution that each element makes to the diagnostic performance of the rule. The consensus-derived rules included elements that the experts thought would be useful, but added little diagnostic value (pleuritic chest pain, injury, medical comorbidities, raised BMI, tachycardia and tachypnoea) and two elements (being in the third trimester, family history of VTE) that had weak associations with the absence of PE. The existing rules had similar problems related to the inclusion of elements with little diagnostic value, but Wells’s score benefited from the inclusion of the criterion ‘PE is the most likely diagnosis or equally likely’ when this criterion was recorded as positive only if the diagnostic impression clearly indicated a positive diagnosis of PE rather than mentioning it as a possibility.

D-dimer analysis (using measurements from routine care)

D-dimer measurements were recorded as part of routine care for 44 out of 198 women with diagnosed PE (22%) and 156 out of 324 women with suspected PE (48%). After the exclusion of 22 women with clinically diagnosed or ruled-out PE, the primary analysis data set for those with routine care D-dimer measurements consisted of 53 women with PE and 125 women without PE.

We have not reported absolute D-dimer values because the measurements were made using a variety of assays with different thresholds for positivity. Instead, we simply report the sensitivity and specificity of D-dimer measurements using the threshold specified by the hospital laboratory (the conventional threshold) and the pregnancy-specific thresholds we defined a priori (conventional threshold in the first trimester, 1.5× the conventional threshold in the second trimester and 2× the conventional threshold in the third trimester). Ten women with PE did not have a hospital laboratory threshold reported and thus could not be included in the analysis, which therefore involved 43 women with PE and 125 women without PE.

Using the hospital laboratory threshold, the sensitivity (n/N) of D-dimer was 88.4% (38/43, 95% CI 74.1% to 95.6%) and the specificity was 8.8% (11/125, 95% CI 4.7% to 15.6%). Using the gestation-specific threshold, the sensitivity (n/N) of D-dimer was 69.8% (30/43, 95% CI 53.7% to 82.3%) and the specificity was 32.8% (41/125, 95% CI 24.8% to 41.9%).

Multivariable analysis

Details of the multivariable analysis are reported in Appendix 8. Leave-one-out cross-validation was utilised internally at each step of the fitting of the LASSO to shrink the point estimate and inform the next iteration. Optimal values for the parameter lambda were identified as the minimum value or the value corresponding to 1 × the SE of the point estimate of the mean squared error. Models were derived for each of these values and the diagnostic parameters were calculated. Figure 2 shows the ROC curve and Table 12 reports the coefficients for each predictor that contributed significantly to the model and the diagnostic parameters for the model.

FIGURE 2.

Receiver operating characteristic curves for the multivariable models.

The analysis suggests that there is little potential for an accurate and usable CDR. The most accurate model used previous VTE, long-haul travel during pregnancy, multiple pregnancy, oxygen saturation (as a continuous variable), surgery in the previous 4 weeks, temperature (as a continuous variable) and PE-related chest radiograph abnormality to predict PE with an AUC of 0.724 (95% CI 0.669 to 0.779). The ROC curve shows that the specificity would have to be as low as 20% to achieve a level of sensitivity that is acceptable to allow imaging to be avoided (> 95%). This estimate would need to be validated in a new cohort and statistical shrinkage would probably result in worse accuracy. Furthermore, the model includes variables with a counter-intuitive negative association with PE (history of long-haul travel and multiple pregnancy) that may be dependent on referral processes and would therefore vary between settings and over time, if referral processes changed.

In view of the poor accuracy of the model in the derivation cohort, we did not proceed to internal validation or attempting to make the model more clinically credible or usable.

Recursive partitioning

Recursive partitioning resulted in a range of models of increasing complexity, accuracy and risk of overfitting. Details are reported in Appendix 9. The optimal model is shown in Figure 3; it uses BMI, oxygen saturation, heart rate and trimester to categorise women. The percentages show how the study population is split by the partitioning process. The proportions are the proportion of PE in each subgroup. The high proportion of PE in each subgroup reflects the case–control design. The prevalence of PE in the suspected PE group was 6.5%, so the proportion of women with PE in each group is around six times higher than would be expected in a typical population with suspected PE.

FIGURE 3.

Dendrogram of the optimal recursive partitioning model. b.p.m., beats per minute.

Figure 4 shows the ROC curve for the model. The AUC was 0.657 (95% CI 0.611 to 0.703) and the threshold that provided a sensitivity of > 95% had a corresponding specificity of 5%. More complex models with more variables had higher accuracy (see Appendix 9), but with an increasing risk of overfitting. Therefore, although some of the very complex models had apparently acceptable accuracy, statistical shrinkage in the validation analysis would be highly likely to result in unacceptable accuracy in a validation cohort. Highly complex models would be difficult to use in clinical practice without appropriate software and, being based on variables with weak associations with PE diagnosis, would lack clinical credibility.

FIGURE 4.

The ROC curve for the optimal recursive partitioning model. CP, complexity parameter.

Secondary analysis

The results of the secondary analysis are summarised in Appendix 10. There were no meaningful differences between the results of the primary analysis and the secondary analysis.

Feasibility of a prospective cohort study

We aimed to use the recruitment data for women with suspected PE to determine the feasibility of using a prospective cohort design to validate a new CDR or biomarker. The study recruited women with suspected PE at a rate of 1.7 women per site per month and a prevalence of 23 out of 324 women with PE (7.1%). This suggests that a prospective cohort study involving 50 sites (more than one-quarter of all potential sites in the UK) recruiting for 2 years could achieve a sample size of 2040, including 145 women with PE.

Main findings

Our analysis showed that clinical variables have little diagnostic value in the assessment of pregnant and postpartum women with suspected PE. There was very little difference in physiological measures between women with PE and those without PE. Many women with PE had entirely normal physiology while many without PE had abnormal physiology. Higher temperature and lower oxygen saturation were associated with PE, but the association was too weak to be clinically useful. Recent surgery (almost entirely caesarean section) was associated with an increased risk of developing PE, whereas older age, giving up smoking during pregnancy, higher heart rate (measured as a continuous variable), absence of shortness of breath at rest, haemoptysis, previous thrombosis, being post partum, having a single pregnancy and having obstetric problems during pregnancy showed weak associations with PE.

Important negative findings were that presenting features, with the exception of haemoptysis and possibly syncope, had little diagnostic value. Risk factors for VTE, with the exception of obstetric problems and a past history of VTE, also had little diagnostic value. Some risk factors (recent long-haul travel, varicose veins and possibly multiple pregnancy and family history of VTE) were more common in women presenting with suspected PE but in whom the diagnosis was not confirmed, than in women in whom the diagnosis of PE was confirmed. This is likely to reflect selection processes, whereby women with symptoms suggestive of PE and known risk factors are more likely to present, be referred and be investigated than those without risk factors.

There were some counter-intuitive findings, most notably that abnormal chest radiograph appearances (even those not considered to be related to PE) were associated with an increased risk of PE. Diagnostic guidelines for suspected PE in pregnancy2 advise a chest radiograph on the basis of identifying alternative causes and determining whether to use CTPA or VQ scanning. Our findings suggest that an abnormal chest radiograph increases rather than decreases the probability of PE.

Given the poor discriminant value of the clinical predictors, it is not surprising that CDRs also had poor discriminant value, with only Wells’s criteria (with a strict interpretation of PE likelihood) having an AUC that was > 0.7. Critically for clinical practice, none of the rules was able to identify a meaningfully sized low-risk group that could be selected for discharge without imaging.

The same problems arose when we attempted to derive a diagnostic model for PE using multivariable analysis and recursive partitioning. The models and decision trees we derived had poor accuracy, were unable to achieve clinically useful specificity at acceptable sensitivity and included variables with counter-intuitive associations with PE that lacked clinical credibility. The poor performance of these models in the derivation cohort suggests that attempts at validation are not worthwhile. Our findings suggest that CDRs have no useful value in selecting pregnant and postpartum women with suspected PE for imaging.

Comparison with previous studies

Our systematic review identified few studies evaluating clinical variables or CDRs for diagnosing PE in pregnancy and post partum, and those we identified included few women with PE. Previous studies reporting a lack of association between presenting features or physiological measurements and a diagnosis of PE12,15,18 may be explained by a lack of statistical power. The DiPEP study was powered to detect clinically important associations. Our findings should therefore convincingly refute the suggestion that clinical features are diagnostically useful for PE in pregnancy and post partum (in the setting of secondary care at least).

Previous studies suggested that Wells’s PE criteria may be useful in pregnant or postpartum women. 8,17 Our findings suggest that Wells’s PE criteria may have some modest diagnostic value when adapted for the pregnant population and applied with a strict interpretation of the likelihood of PE, but this is unlikely to be clinically useful. The discordance between our study and previous studies may be explained by random error, owing to the small numbers in previous studies or differences in study design.

Strengths and limitations

The DiPEP study is the largest study ever undertaken to evaluate the diagnostic accuracy of clinical predictors and biomarkers for pregnant and postpartum women with suspected PE. In terms of the number of women with PE, it is many times larger than any previous study. This provided us with much greater power to detect associations with PE and allowed us to estimate diagnostic sensitivity with much greater precision. The women with suspected PE were a relatively unselected cohort presenting to a representative group of hospitals, involving both maternity units and emergency departments. The women with diagnosed PE were identified across all UK hospitals and, therefore, were likely to be a representative sample. Data completion rates were generally good and follow-up ensured that the risk of misclassification of women with PE as having no PE was minimised. The primary analysis was limited to women with imaging confirmation to reduce the risk of misclassification, whereas the secondary analysis including women with clinically diagnosed or ruled-out PE explored the applicability of findings to a wider cohort.

The large number of women in the study with PE was achieved by using a case–control design, which has some inevitable limitations. Strictly speaking, the design was a cohort design in which the cohort was augmented by the inclusion of additional patients with PE. However, it shares potential limitations with the case–control design. Patients may be selected and represent the more severe end of the spectrum with the disease, whereas control patients may not be typical of those investigated without the disease and may not have conditions that lead to a false-positive diagnosis. We reduced these risks by including all women with diagnosed PE as participants and then excluding those requiring immediate resuscitations (or who we could not verify had not required immediate resuscitation), so that they would be representative of women diagnosed with PE and presenting with suspected PE. Women in the control group were not healthy control participants, but women presenting with suspected PE who underwent investigation. Crucially, it should be noted that the bias associated with the case–control design tends to inflate the estimates of sensitivity and specificity. This means that our findings of poor discriminant value for clinical predictors and CDRs are unlikely to be undermined by the risk of bias.

A more significant risk of bias may relate to differences in data collection methods between women with diagnosed PE and women with suspected PE. Data from the former group were retrospectively recorded by UKOSS clinicians using the hospitals records, whereas data from the latter group could be collected by the research nurse/midwife directly questioning patients. We encouraged the research nurse/midwife to rely on hospital records rather than patient interview to collect predictor variables, but we cannot be sure that they did not rely on patient interview. Furthermore, to reduce potential screening and data collection bias, the immediate collection of prospective data would be preferable. Missing data rates were higher for the UKOSS data, which suggests that the identification of clinical predictors from the women with suspected PE may have been more rigorous. This potential bias offers an alternative explanation to the one considered above for the counter-intuitive findings that some risk factors for developing VTE (long-haul travel, varicose veins) were associated with an absence of PE. It does not explain the negative associations with easily identifiable predictors, such as multiple pregnancy, or the limited predictive value of physiological variables.

Crucially, it should be noted that the participants in our study had all been through some sort of selection process, either by virtue of deciding to self-present to emergency or maternity services, or by referral by a health professional. This process is likely to have been based on many of the clinical variables that we examined in our analysis. It is therefore possible that the diagnostic value of clinical variables is ‘used up’ during the referral process. For example, if women with risk factors for VTE or abnormal physiology are more likely to self-present or be referred, then these factors will have less diagnostic value than if those presenting are a random or unselected sample of women suffering symptoms compatible with PE during pregnancy or post partum.

This does not undermine the relevance of our findings to secondary care, as clinicians working in emergency departments and maternity units will see similarly selected patients, but does mean that we should be cautious about extrapolating our findings to primary care and other settings outside hospital. Clinical variables may be being used to select women for hospital attendance, which explains their lack of value in a hospital cohort.

It is also important to recognise that our findings do not challenge existing knowledge on what constitutes risk factors for VTE in pregnancy. The DiPEP study was designed to determine the diagnostic value of clinical features (including the risk factors for developing VTE) in women with suspected PE in pregnancy and post partum. It was not designed to determine whether or not these features are risk factors for developing VTE. Previous studies comparing women who develop VTE in pregnancy with those who do not have already answered this question. 27–29,55

Target population

Sample size

The incidence of DVT in pregnancy and post partum is around four times that of PE,28 so we anticipated that we would recruit around 20 women with DVT. Thus, the sample for the biomarker substudy was expected to include 245 women with suspected PE but negative diagnostic testing, five women with diagnosed PE, and 20 women with diagnosed DVT (i.e. 25 women with confirmed VTE).

Blood sample collection, handling, storage and analysis

Serum and citrate blood samples were collected by a member of the clinical team or research nurse/midwife using venepuncture technique, ideally while obtaining routine blood samples for standard clinical assessment in diagnostic workup. Sample preparation was conducted by the research nurse/midwife or a member of the hospital laboratory staff. The samples were centrifuged at 2000 g for 15 minutes at room temperature within 4 hours of being obtained. Citrate samples were further processed to obtain platelet-free plasma.

Plasma and serum samples were stored in aliquots labelled with the patient identification and the storage box co-ordinates were recorded on paper and electronic study documentation, in accordance with local protocols. The samples were stored in –70 °C freezers at each participating hospital (with the exception of one location in which a –40 °C freezer was used) for the duration of the study, until all samples were transported for analysis to Guy’s and St Thomas’ NHS Foundation Trust (GSTT), London, UK.

Biomarker analysis

The biomarkers selected for analysis are outlined in Table 13.

Analytic techniques

Citrated plasma was utilised for PT, APTT, Clauss fibrinogen, D-dimer (Innovance), D-dimer ELISA, thrombin generation (TG), plasmin–antiplasmin, prothrombin fragment 1 + 2 (PF 1 + 2), and tissue factor. Serum was utilised for troponin-1, BNP, MRProANP, and CRP assays.

Thrombin generation was measured by the Thrombinoscope (ThermoElectron Corporation, Cambridge, UK). The samples were tested in batches to minimise variability. The frozen-plasma aliquots were placed in a water bath (at 37 °C) to thaw for 5 minutes. Platelet-poor plasma (PPP)-reagent LOW (Diagnostica Stago UK Ltd., Theale, UK) was used because of expected hypercoagulability; PPP-reagent LOW consists of 1 pM of tissue factor with 4 µM of phospholipids. A measurement of 20 µl of PPP reagent was added to each TG well together with 80 µl of platelet-free plasma and 20 µl of fluorogenic substrate and calcium. The fluorogenic substrate consisted of amino-methyl-coumarin. The calibrator wells consisted of 80 µl of platelet-free plasma, 20 µl of calibrator and 20 µl of fluorogenic substrate and calcium. All of the reagents were from Diagnostica Stago, Reading, UK. The analysis was conducted in an ELISA plate (Diagnostica Stago, Reading, UK) that enables the thrombin formation to be followed in a Fluoroskan (Fluoroskan Ascent™, Thermo Scientific, Loughborough, UK). The coefficient of variation was 2.2% to 3.2% intra-assay and 5.1% to 16.7% interassay for lag time, endogenous thrombin potential, peak and time to peak.

The PT, APTT, and Clauss fibrinogen were measured on the ACL300R (Werfen UK, Warrington, UK) using PT High Sensitivity Plus reagent for the PT, HemosIL APPT Synthetic Phospholipids liquid for the APTT (Werfen UK, Warrington, UK), and fibrinogen C for the Clauss fibrinogen. All reagents were purchased from Werfen UK (Warrington, UK). The tests were measured in accordance with the manufacturer’s instructions for the ACL300R analyser. The coefficient of variation was 3.2% to 3.5% intra-assay and 3.6% to 4.2% interassay for PT, APTT and Clauss fibrinogen.

The latex-based D-dimer was measured on the CA660 analyser from Sysmex UK (Milton Keynes, UK). Innovance D-dimer reagent (Sysmex UK, Milton Keynes, UK) was used to measure the D-dimer in accordance with the manufacturer’s instructions. The coefficient of variation was 6.0% intra-assay and 12% interassay for the Innovance D-dimer.

The Zymutest D-dimer ELISA assay (Quadratech Diagnostics Ltd, Epsom, UK) was used to measure the D-dimers by ELISA. D-dimer was measured in accordance with the manufacturer’s instructions. The coefficient of variation was 4.6% intra-assay and 10.8% interassay for the Zymutest D-dimer.

The plasmin–antiplasmin ELISA (Immunodiagnostics Systems Ltd, Boldon Colliery, UK) was used to measure the plasmin–antiplasmin. The assay was measured in accordance with the manufacturer’s instructions. The coefficient of variation was 4.2% intra-assay and 7.3% interassay for the plasmin–antiplasmin.

The fragment 1 + 2 Micro (Sysmex, Milton Keynes, UK) was used to measure the PF 1 + 2 in the citrated plasmas. The PF 1 + 2 was measured in accordance with the manufacturer’s instructions. The coefficient of variation was 6.0% intra-assay and 9.0% interassay for the PF 1 + 2.

The Immubind tissue factor (Invitech Ltd, Huntingdon, UK), was used to measure the tissue factor by ELISA. The tissue factor was measured in accordance with the manufacturer’s instructions. The coefficient of variation was 6.0% intra-assay and 5.0% interassay for the tissue factor.

The troponin-1 type 3 ELISA (Bio Techne, Abingdon-on-Thames, UK) was used to measure the troponin-1 levels. The troponin-1 levels were measured in accordance with the manufacturer’s instructions. The coefficient of variation was 4.0% intra-assay and 4.6% interassay for the troponin-1 levels.

The BNP ELISA (Bio Techne, Abingdon-on-Thames, UK) was used to measure the BNP levels. The BNP levels were measured in accordance with the manufacturer’s instructions. The coefficient of variation was 10% intra-assay and 15% interassay for the BNP assay.

The human midregional pro-atrial natriuretic peptide ELISA (2B Scientific, Upper Heyford, UK) was used to measure the MRProANP levels. The MRProANP levels were measured in accordance with the manufacturer’s instructions. The coefficient of variation was 8% intra-assay and 10% interassay for the MRProANP levels.

The human CRP Quantikine assay (Bio Techne, Abingdon-on-Thames, UK) was used to measure the CRP levels. The CRP levels were measured in accordance with the manufacturer’s instructions. The coefficient of variation was 5.5% intra-assay and 6.5% interassay for the CRP assay.

Statistical analysis

The biomarker analysis included all women with suspected PE or diagnosed DVT who provided consent and an analysable blood sample. Women with suspected PE were classified as having VTE if they were classified as having PE in accordance with the method described in the case–control study. The primary and secondary analyses were planned along the lines outlined in the case–control study. Women recruited with diagnosed DVT were all classified as having VTE and included in the primary analysis.

Blood samples were analysed at GSTT and the results of the analysis were sent to the Sheffield CTRU. GSTT established normal ranges for the assays using 20 normal plasma/serum samples (depending on the assay), with the 99th percentile used as the top of the normal range.

We calculated the AUC for each biomarker and the sensitivity and specificity at the upper limit of the normal range. We then examined the ROC curve to determine whether or not there was an optimal threshold for clinical practice, whereby sensitivity exceeds 95% but specificity still allows a meaningful proportion of women without PE to have the diagnosis ruled out.

Analysis excluding women who had received anticoagulation treatment

Anticoagulation treatment with heparin is known to interfere with biomarker assays. Unfractionated heparin will prolong the APTT and thrombin time and low-molecular-weight heparin (LMWH) may cause a slight prolongation of the APTT. However, suppressing the activation of both Factor Xa and thrombin will decrease all parameters of the TG assay and PF 1 + 2. Furthermore, decreasing the generation of thrombin, which is a major stimulator of fibrinolysis, will reduce the D-dimer and plasmin–antiplasmin values.

We repeated the analysis, having excluded 240 out of 328 women who had received anticoagulation treatment prior to blood sampling. The primary analysis involved only 66 women, of whom only four had VTE, so the findings were limited by small numbers. Details of the findings are provided in Appendix 11. The differences in mean biomarker levels observed in the main analysis between women with and without PE disappeared or even reversed when those receiving anticoagulation treatment were removed, but this probably reflects the small numbers. ROC analysis suggested that BNP (AUC 0.774, 95% CI 0.670 to 0.878), PF 1 + 2 (0.795 pmol/l, 95% CI 0.644 to 0.947 pmol/l), TG lag time (0.735, 95% CI 0.531 to 0.940) and troponin (0.742 ng/ml, 95% CI 0.453 to 1.000 ng/ml) may have some potential to rule out VTE with acceptable sensitivity, but the CIs were wide and estimates would need to be validated in a larger cohort with VTE.

Study identification

The searches were conducted in MEDLINE, MEDLINE In process and NHS Economic Evaluation Database on the 25 August 2016. The disease-specific search terms were the same as those used in the DiPEP literature review. 7 The Scottish Intercollegiate Guidelines Network economic search filters were added to these search terms. Cost-effectiveness studies on CDRs were identified using the criteria given in Appendix 12.

Results

No studies were found that estimated the cost-effectiveness of the use of selective imaging for the diagnosis of PE in pregnant or postpartum women. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram is given in Appendix 12. The key finding is that no economic study has previously been conducted in a population of women who were pregnant or post partum.

Decision problem and perspective

The economic modelling assessed whether or not using CDRs would be cost-effective compared with scanning all (current care) and scanning none for pregnant or postpartum women who had a suspected PE in the UK. In line with NICE guidance, the analyses took a NHS and personal social services perspective and a lifetime horizon, and future costs and QALYs were discounted at a rate of 3.5% per annum. 60

Model description

A patient-level decision tree was developed in Microsoft Excel 2016 to estimate the cost-effectiveness of the 10 strategies for pregnant or postpartum women with a suspected PE at admission. A decision tree is believed to be an appropriate model for this decision problem, as events do not repeatedly occur over time, representing the likely clinical reality for pregnant/postpartum women with suspected PE. The structure also reflects the fact that the model is estimating the impact of a single decision at a single point in time. A patient-level model was chosen over a cohort model, as this allowed the model to include the fact that women who were more likely to die if untreated were also more likely to have a positive CDR and hence be referred for a scan.

Patient population

The population included in the economic model was pregnant or post partum (up to 6 weeks after birth) women who presented with a suspected PE at a UK hospital. The fetuses were considered to be outside the scope of the model, apart from any imaging-induced childhood cancers.

The primary statistical classification was used to determine whether or not women had PE. In this analysis group, the population was limited to women with PE diagnosed by imaging or post-mortem examination and women with PE ruled out after imaging. After statistical imputation in the suspected PE cohort, 13 women with PE and 248 without PE had complete data for all decision rules. In line with the statistical analysis of rule sensitivity and specificity, these data were supplemented by 144 women with diagnosed PE and from the UKOSS observation data set.

Estimation of outcomes across the populations

The model estimates the costs and QALYs for each strategy in the PE and no PE populations separately. The costs and QALYs for each strategy in the population with a suspected PE were calculated using the following formula:

in which the mean outcomes are derived from the model and the probability of having PE is obtained from the suspected PE data set only.

Determining the number of patients

The model uses a bootstrapping procedure to estimate the lifetime costs and QALYs for each strategy. Within each bootstrap, 157 women with PE and 248 women without PE are sampled from their respective populations with replacement. It was determined that 100 bootstraps were sufficient to produce robust results. Details of how this was determined are provided in Appendix 13.

Model structure

The model was structured as an individual-level decision tree; the key aspects of the model are presented in Figure 8. The key events included in the decision tree were PE-related death, major bleeding events, deaths associated with major bleeding events, chronic thromboembolic pulmonary hypertension (CTEPH), recurrent VTE and deaths from recurrent VTE.

FIGURE 8.

The key aspects of the economic model: (a) true-positive scan; (b) false-positive scan; (c) false-negative scan; and (d) true-negative scan. CTEPH, chronic thromboembolic pulmonary hypertension; ICH, intracranial haemorrhage.

In the base case, it was assumed that recurrent VTE and CTEPH were modifiable by anticoagulation in pregnant/postpartum women with a suspected PE. Furthermore, it was assumed that the risk of major bleeding events was assumed to be solely attributable to anticoagulation. If a woman experienced a major bleeding event following an initial anticoagulation treatment, then it was assumed that they would not receive anticoagulation for any subsequent VTEs or gain a therapeutic benefit from their current anticoagulation treatment. Minor bleeding events were excluded from the economic model, as the definition of major bleeding events included all clinically overt bleeds that could have resulted in a hospitalisation. Finally, it was assumed that each woman was at risk of only one recurrent VTE, which was assumed on the grounds that the PE for these women was related to their pregnancy rather than other underlying conditions, which would typically be present in the population that presents with a VTE.

For all women who were scanned, a QALY decrement and treatment cost were applied for imaging-induced lung and breast cancers in the mother, and a QALY decrement was applied for childhood cancers induced by imaging in the fetuses who would survive anticoagulation to term. These were identified as the key harms from scanning. We did not model any cost or QALY loss associated with teratogenesis, as evidence suggests that this is not a significant risk. 61

Model data

Outcome measures

The main outcome measure for this analysis is the incremental cost-effectiveness ratio (ICER) for each strategy. A full incremental analysis will be conducted, which will compare all of the strategies with each other. The ICER will be compared with a maximum acceptable ICER (MAICER) of £20,000 per QALY gained. This is in line with decision-making processes by NICE. 60

A value-of-information analysis will also be conducted to determine if further research into the cost-effectiveness of the strategies is an efficient use of resources from the health-care system perspective. In the value-of-information analysis, it was expected that the number of women affected per year was 2231. This was based on there being 723,913 live births in England and Wales in 2011 and data from the Scottish Morbidity Record suggesting a combined incidence of 2.0 per 10,000 maternities for antenatal and postnatal PE. 28 This value was then uplifted by assuming that for women with suspected PE, 18 out of 277 women would have PE and 259 out of 277 women would not. On the basis of clinical input, it is expected that any information generated by a future study would be useful for 10 years.

The value of research into particular questions will be assessed by calculating the partial value of information for a particular set of parameters. Previously, this has not been feasible because of time constraints, but recently developed techniques now allow for the calculation of the partial value of information from the model outputs of the PSA,95 estimating multiparameter partial expected value of perfect information from a PSA sample (a non-parametric regression approach). The value of particular study designs to address any research question will not be explored, as performing this calculation would require a known study design to address a particular research question, neither of which was developed in this project. 96

Summary of assumptions

• Long-term evidence from the population with PE is valid for pregnant/postpartum women who have PE.

• The women were only at risk of VTE for 1 year because their suspected PE was caused by their pregnancy. As such, the long-term risk factors that have been shown for other older populations with VTE were not relevant.

• Fetuses are outside the decision problem, apart from the adverse effects of any imaging-induced cancers.

• When initiating anticoagulation treatment, it was assumed that, in the UK, the guidelines by the Royal College of Obstetricians and Gynaecologists would be followed.

• For the base case, it was assumed that an accurate diagnosis of PE/no PE would be made when a woman was referred to imaging.

• In the base case, it was assumed that CTEPH was a risk factor that depended on an individual’s characteristics and was modifiable by anticoagulation treatment.

• The utility of a woman with CTEPH would likely be the same as someone with heart failure.

• The cost of anticoagulation treatment for a second VTE event would be the same as the cost of anticoagulation treatment for the initial VTE.

• All doses are based on a woman’s current weight.

Analysis

Base-case analysis

The results of the base-case health economic analysis are given in Table 16. In the deterministic analysis, ‘scan all’ was the dominant strategy, as it produced the most QALYs (20.3855) at the lowest cost (£1359). In the PSA, the ‘scan all’ strategy also dominated all of the other 10 strategies, as again it produced the most QALYs (20.3832) at a cost of £1360. As well as being dominant on average, in the 1000 PSA replications, the ‘scan all’ strategy was the most cost-effective option in 98.9% of PSA replications when a MAICER of £20,000 per QALY gained was used. This indicates that there is only a very small probability (1.1%) that scanning all women would give an incorrect decision, given the parameter distributions used in the base-case analysis. A cost-effectiveness acceptability curve showing the probability that each intervention is the most cost-effective at different MAICERs is given in Appendix 21 (see Figure 62).

Value-of-information analysis

Table 17 shows the results of value-of-information analysis. The value-of-information analysis found that reducing all uncertainty in the economic model for this decision problem would have a maximum value of £4.57 per person per year. Assuming that 2231 women per year are able to benefit from any research over a period of 10 years, this gives a maximum value of conducting research into this decision problem of £101,952. However, using one piece of research to reduce all of the parameters used in the economic model is highly optimistic, so we conducted an analysis of the value of conducting further research into a group of parameters for which a single study could be designed. The group with the highest value was the effectiveness of anticoagulation treatment, as this had a value of £72.77 for 2231 women with a suspected PE over a 10-year time horizon. As this is the maximum value of future research into this parameter, conducting any research into these values that costs more than this is not indicated in this decision analysis. Given the value-of-information analysis, further research to reduce uncertainty in the parameter estimates used in the economic model does not appear to be indicated for this decision problem. It should be noted that the accuracy of the decision rules could not be included in this analysis, as they were applied to each woman’s modelled characteristics. Therefore, they were not random variables in the PSA, which is necessary to be able to conduct a value-of-information analysis. Instead, the value of conducting the analysis was assessed by conducting the threshold analysis that compared decision rules with hypothetical sensitivity and specificity values to scanning all women. It should also be noted that much of the data in the base case came from the general population with PE and, therefore, these data may not be representative of pregnant/postpartum women with PE. This uncertainty was addressed in the scenario analysis in which the probability of recurrent VTE, CTEPH and bleeding were set to 0%.

Scenario analyses

A summary of the scenario analysis results are provided in Table 18; detailed results for each scenario are provided in Appendix 20. When using a MAICER of £20,000 per QALY gained, scanning all women was the most cost-effective strategy in all scenario analyses. In fact, scanning all women dominated all of the other strategies in every scenario, except when CTEPH was not modifiable by anticoagulation treatment, when the risk of CTEPH was estimated from data presented in the literature and when women were not at risk of bleeding, recurrent VTE or CTEPH. The final scenario is of particular note, as this is the most unfavourable scenario to the ‘scan all’ strategy, as the only benefit of anticoagulation treatment is the prevention of death associated with a woman’s initial PE and there are no harms associated with anticoagulation treatment. This indicates that the results of the economic model are relatively robust to the model assumptions that have been tested.

Threshold analyses

The threshold analyses explored which strategy out of ‘scan all’, ‘scan none, treat all’ and ‘scan none, treat none’ would be the most effective strategy, conditional on the probability that a woman with suspected PE actually had PE. In the first threshold analysis, the model base case was used, and in the second threshold analysis, the scenario in which CTEPH was not modifiable by anticoagulation treatment was used. In the analysis that used the base case, scanning all women is optimal if the probability that they have PE ranges from 0.1% to an upper limit of between 96.5% and 97.0%. At a probability of having PE of 97.0% or above, a woman should be given treatment directly, as the small chance of inducing a bleed in women without PE is outweighed by the QALY losses associated with radiation-induced cancers for the mother and the fetus (if applicable). The results in the scenario in which anticoagulation treatment does not have an impact on a woman’s risk of CTEPH are broadly similar to those of the base-case analysis. However, the threshold at which all women are treated without scanning increases slightly to 98.0%, compared with 97.0% in the base case.

The second threshold analysis calculated pairwise ICERs comparing scanning all women with a series of hypothetical CDRs in which the rules being positive or not is independent of each woman’s characteristics. The sensitivity and 1 – specificity provide the probabilities that the rules are positive for pregnant and postpartum women with a suspected PE, who actually have PE and actually do not have PE, respectively. The results of this threshold analysis are given in Tables 19 and 20. The tables show that even if a decision rule could be developed that had 97.5% sensitivity and 90% specificity, the ICER would be £13,392, which is well below the £20,000-per-QALY-gained threshold. This indicates that scanning all women would probably be considered to be cost-effective, even if such a decision rule could be developed for pregnant and postpartum women with a suspected PE. It should be noted that all of the hypothetical decision rules lead to fewer expected QALYs than scanning all women.

Report authors

Professor Steve Goodacre, University of Sheffield.

Miss Kimberley Horspool, University of Sheffield.

Mr Neil Shephard, University of Sheffield.

Mr Daniel Pollard, University of Sheffield.

Professor Beverley Hunt, Guy’s and St Thomas’ NHS Foundation Trust.

Dr Gordon Fuller, University of Sheffield.

Professor Catherine Nelson-Piercy, Guy’s and St Thomas’ NHS Foundation Trust.

Professor Marian Knight, University of Oxford.

Dr Steven Thomas, Sheffield Teaching Hospitals NHS Foundation Trust.

Professor Fiona Lecky, University of Sheffield.

Dr Judith Cohen, University of Sheffield.

Project management group

Amanda Loban, University of Sheffield.

Professor Beverley J Hunt, Guy’s and St Thomas’ NHS Foundation Trust.

Professor Catherine Nelson-Piercy, Guy’s and St Thomas’ NHS Foundation Trust.

Daniel Pollard, University of Sheffield.

Ellen Bradley, University of Sheffield.

Professor Fiona Lecky, University of Sheffield.

Dr Gordon Fuller, University of Sheffield.

Dr Judith Cohen, University of Sheffield.

Kimberley Horspool, University of Sheffield.

Kiran Parmar, Guy’s and St Thomas’ NHS Foundation Trust.

Professor Marian Knight, University of Oxford.

Professor Matthew Stevenson, University of Sheffield.

Professor Michael Campbell, University of Sheffield.

Neil Shephard, University of Sheffield.

Professor Steve Goodacre, University of Sheffield.

Dr Steven Thomas, Sheffield Teaching Hospitals NHS Foundation Trust.

Dr Wee-Shian Chan, BC Women’s Hospital & Health Centre.

Trial Steering Committee

Dr Andrew Thomson, Royal Alexandra Hospital.

Franchesca Cullinane, PPI lay representative.

Kimberley Horspool, University of Sheffield.

Shan Bennett, PPI lay representative.

Dr Steve Crane, York District General Hospital.

Professor Steve Goodacre, University of Sheffield.

Dr Sue Pavord, John Radcliffe Hospital.

Dr Trevor Cox, Liverpool Cancer Trials Unit.

Local investigators

Professor Steve Goodacre, Sheffield Teaching Hospitals NHS Foundation Trust.

Professor Derek Tuffnell, Bradford Teaching Hospitals NHS Foundation Trust.

Dr Richard Parris, Bolton NHS Foundation Trust.

Dr William Townend, Hull and East Yorkshire Hospitals NHS Trust.

Dr Etienne Ciantar, Leeds Teaching Hospitals NHS Trust.

Dr Richard Body, Central Manchester University Hospitals NHS Foundation Trust.

Professor Catherine Nelson-Piercy, Guy’s and St Thomas’ NHS Foundation Trust.

Professor Tim Harris, Bart’s Health NHS Trust.

Dr Chris Vorwerk, Portsmouth Hospitals NHS Trust.

Dr Gillian Swallow/Dr Judith Moore, Nottingham University Hospitals NHS Trust.

Dr Andy Walden, Royal Berkshire NHS Foundation Trust.