Faecal calprotectin testing for differentiating amongst inflammatory and non inflammatory bowel diseases: systematic review and economic evaluation

Coeliac disease

Coeliac disease is a disease of the small bowel, resulting from an immune reaction to the wheat gluten and similar proteins found in rye, barley and, to a lesser extent, oats. Coeliac disease can be ruled out by testing for autoantibodies at an early stage, so is not relevant to calprotectin testing. It could be classed as an inflammatory disease of the bowel but the inflammatory cells are mainly lymphocytes, so calprotectin is not high [but can be modestly raised in children (D Wilson, Royal Hospital for Sick Children, Edinburgh, 2013, personal communication)].

Ulcerative colitis

Ulcerative colitis is characterised by inflammation of the colon, sometimes intense, with bloody diarrhoea, but is often much milder. The cause is not known, but it appears that some people are more genetically susceptible than others. 12 Around 10% of people with UC have a first-degree relative with the condition. The concordance in monozygotic twins is also around 10%.

Curiously, cigarette smoking may confer some protection, or reduce severity. 12 The risk is also moderately reduced in people who have had appendicitis and appendix removal, under the age of 20 years.

Ulcerative colitis may involve an abnormal immune response to the microbacteria that normally live in the gut, known as commensals. UC is sometimes triggered by episodes of gastroenteritis caused by organisms such as Salmonella, Shigella and Campylobacter, but more by changes in the natural gut flora than direct effects of these organisms.

Ulcerative colitis typically starts in the rectum and spreads upwards through the colon. The natural history is of relapse and remission. At first presentation, most patients have mild disease, and only 10% have severe disease. About half will continue to have mild disease or remission, but in about one-fifth of patients, UC will be chronic and continuous, and more likely to become extensive, throughout the colon.

The aim of treatment in active disease is to secure a remission and then maintain that. Different drugs are used to induce, and then maintain, remission. There is an increased risk of colorectal cancer, so surveillance for that is part of care.

Crohn’s disease

Crohn’s disease can present in different ways, depending on which part of the intestinal tract is affected. Like UC, it is a relapsing and remitting inflammatory disease. However, it can affect any part of the GI tract – it is a much more extensive disease. Also like UC, there is a genetic susceptibility, with concordance in 35% of monozygotic twins. 13 The cause is unknown but it appears to be commoner in those with a ‘westernised’ lifestyle. Like UC, it may occur after infectious gastroenteritis and is associated with disturbances in the usual gut flora. The histological features include some similar to tuberculosis but no mycobacteria have been shown to be responsible. There are around 60,000 people with CD in the UK, of whom 20–30% are aged under 20 years. 14 The incidence is highest in the age range 15–30 years. About 25% of cases have onsets under age 17 years.

The pattern of symptoms in children is different. A prospective survey was carried out in the UK and Ireland by the British Paediatric Surveillance Unit, the BSG Research Unit and the Paediatric Register of Inflammatory Bowel Disease. A total of 739 cases were reported in children under the age of 16 years, making it the largest such study. The commonest presenting symptoms of CD were abdominal pain, weight loss and diarrhoea, but 44% did not report diarrhoea and only 25% reported the classical triad of abdominal pain, diarrhoea and weight loss. Other symptoms at presentation included lethargy and anorexia. Paediatric IBD (PIBD) is often more extensive at diagnosis than in adults.

The UK and Ireland survey found that delays in diagnosis of CD in children were common; 18% had a pre-diagnosis duration of symptoms of 1–3 years, and 9% of more than 3 years. Only 9% had isolated small bowel disease.

The delay in diagnosing PIBD has changed little over the last 20 years. What has changed is the incidence. Henderson et al. 15 reported a rise in Scotland of 76% from 1990–5 to 2003–8, and a fivefold increase over the last 40 years, especially in CD. This rise may not apply to the same extent in the rest of the UK, as there is a north–south gradient within Scotland,16 but, internationally, rates have been increasing. 17

Symptoms of CD include diarrhoea, pain, and blood or mucus in stools. Other presentations include anaemia due to disturbance of iron metabolism, and extraintestinal disease, such as arthritis, which may appear before any intestinal symptoms. Diagnosis is usually based on histology after biopsies taken during endoscopy. Differential diagnosis includes other causes of abdominal pain, such as IBS. Symptoms may be different in children, in whom growth retardation may be a feature that can precede bowel symptoms. 18

The outlook for CD is worse than for UC. Only 10% have prolonged remission. Based on past experience, about 20% require hospital admission each year, and half will have required surgery within 10 years of diagnosis. This compares with the 10–30% of adults with UC who will require a colectomy in the first 10 years. 19,20

The outlook in paediatric UC has been worse, with 20% of 113 children in one study21 having to have their colon removed by 5 years of duration.

Newer drugs such as the ‘biological’ agents (infliximab and adalimumab) may reduce admission rates and the need for surgery. 22,23

There are three main serious intestinal complications of CD. The first is stricture (narrowing) of the bowel. This can lead to intestinal obstruction, and CD can present as an ‘acute abdomen’ requiring surgery, sometimes mimicking appendicitis. The second is fistulas, which are abnormal connections between sections of bowel, or between bowel and bladder. The third is colorectal cancer, and surveillance is required.

In both UC and CD, some people have active disease but no symptoms. This has been noted following the introduction of colorectal cancer screening using faecal occult blood testing (FOBT). Positive screenees are referred for colonoscopy. Butcher et al. 24 reported that amongst 5350 such people who had colonoscopy, 66 were found to have unsuspected IBD (UC–CD 2 : 1), of whom about half had no symptoms. Some had quite extensive UC. 24

Esch et al. 25 reported that some people with CD have no symptoms but are found by chance during investigations for other reasons. However, most developed symptoms over time (mean 3–4 years; range 2 months to 9 years) and one-quarter required surgery. They concluded that initially silent CD requires similar monitoring to initially symptomatic CD.

The treatments and the aims of treatments have changed in recent times. Schoepfer et al. 26 comment that the aims have evolved from relieving symptoms towards mucosal healing. They consider that this has been driven by the arrival of new medications, such as the anti-tumour necrosis factor (TNF) drugs that can induce and maintain mucosal healing. A New Zealand consensus conference concluded that early use of infliximab at induction led to higher rates of mucosal healing. 27 Economic modelling by Ananthakrisnan et al. 28 suggests that treatment aimed at mucosal healing is cost-effective compared with aiming only at relief of symptoms, because over a 2-year follow-up period the mucosal healing group would have fewer hospital admissions and less surgery than the symptom suppression group. This results in a cost per quality-adjusted life-year (QALY) of around £33,000, based on straight conversion of US dollars (US$) to British pound sterling (£).

The arrival of more effective new drugs increases the importance of prompt diagnosis of CD, and it could be argued that they should be used earlier in the treatment pathway. However, National Institute for Health and Care Excellence (NICE) technology appraisal (TA) 18714 recommends use of the anti-TNF drugs, infliximab and adalimumab, only in people whose disease has not responded to conventional therapy with steroids or with immunosuppressive agents, such as azathioprine and 6-mercaptopurine (6-MP). Response means relief of symptoms.

The ratio of CD to UC varies between adults and children. In adults, the ratio of CD to UC is 2 : 3,29 whereas in children the ratio is much higher, at 2.3 : 1. 30

There are other forms of colitis, such as collagenous colitis and lymphocytic colitis, sometimes combined as ‘microscopic colitis’, which can cause persistent non-bloody diarrhoea, but these are usually seen in older people. The mean age at diagnosis in a Swedish study31 was 64 years for collagenous and 59 years for lymphocytic.

Differential diagnosis

Some features of CD, UC, IBS and coeliac disease are compared in Appendix 1. The key point is that distinguishing among them by purely clinical means – signs and symptoms – can be difficult. Ford and colleague carried out a systematic review of the usefulness of symptoms and symptom scores for diagnosing IBS. 32 They concluded that individual symptoms (lower abdominal pain, passage of mucus per rectum, feeling of incomplete evacuation, passage of looser stools at onset of abdominal pain, abdominal pain relieved by defecation and patient report abdominal bloating) have limited usefulness for diagnosing IBS. They also concluded that composite scores such as the Manning and Kruis criteria had only modest accuracy, and noted that these scores were developed based on secondary care populations and might be less applicable to the patient mix seen in primary care. They also noted that around 40% of patients in the studies underlying the scores had some form of organic disease, suggesting an element of spectrum bias.

Jellema et al. 33 carried out a systematic review of the accuracy of symptom-based criteria for IBS (Manning, Kruis, Rome I and II and others). They included 25 studies, but only three were carried out only on primary care patients. Jellema et al. 33 concluded that none of the criteria could reliably exclude organic disease. However, there is a school of thought that asserts that:1

A positive diagnosis of IBS should be reached using symptom-based clinical criteria, not after excluding organic disease by exhaustive investigation.

This is echoed in the NICE scope:34

In the majority of cases the diagnosis of IBS can be made on the basis of clinical history alone.

The systematic review done by the National Collaborating Centre for Nursing and Supportive Care for the NICE guideline group on IBS (p. 101) quotes Jeong et al. :35

It is amazing to see the expensive, dangerous and extensive workups to which healthy patients are subjected by physicians searching for an organic cause in patients who obviously suffer from IBS.

The review lists many possibly investigations (pp. 100–1) but these did not include calprotectin.

General practitioners in the NTAC Durham Dales pilot study36 were good at diagnosing IBS – if a GP thought a patient had IBS, the GP was right in 95%, using a negative FC as confirmation of diagnosis. Note that this does mean that 1 in 20 patients had a diagnosis other than IBS, with raised calprotectin suggesting IBD.

It may be useful to consider new presentations separately. Many GPs will feel confident about the diagnosis in recurrent IBS, when they know the patient well and they have presented with similar symptoms on previous occasions, perhaps after anxiety or stress. They may not feel a need to refer such patients. However, with new presentations there will be more diagnostic uncertainty, and the proportion referred to secondary care to exclude IBD may be higher. Calprotectin testing may be most useful in new presentations.

A survey of GPs from around Bristol found that most GPs were fairly confident (‘8 out of 10’, where ‘10’ was most confident) that they could diagnose IBS at the first visit and most did not investigate the under 45 years age range further. As only a small proportion was referred for specialist investigation, there may have been some false negatives with IBD. 37

However, many patients are referred to gastroenterology, for definite diagnosis, which is usually/often based on endoscopy and histology of biopsies. Some studies report that some patients with IBS are very anxious, and require the reassurance of a hospital ‘check-up’. In one small study (54 patients) from Cardiff, the main reason for referral was diagnostic uncertainty (37/54) but the second reason was for ‘confirmation of IBS’ (17/54). 38

In various studies, the proportion of patients referred for further investigation, in whom abnormal findings are reported on colonoscopy, is low. Kok et al. 39 noted reports that only 22–37% had organic bowel disease (OBD).

The ability of GPs to correctly identify IBS, in a considerable proportion of people with lower abdominal symptoms, has implications for the spectrum of patients in whom calprotectin testing might be used. IBS is very common, and one estimate is that 90% of patients seen, in general practice, with chronic lower abdominal symptoms, have IBS. This high prevalence of IBS in general practice groups has led to concern that results from studies carried out in secondary care may not be applicable to patients seen in primary care. A much higher proportion of patients in secondary care studies may have IBD. However, if GPs are referring only selected patients to specialist clinics, the prevalence of IBD among referrals will be higher, with the spectrum of referred patients more similar to that in the studies from secondary care.

Endoscopy may involve (1) colonoscopy, involving inspection of the whole colon; (2) sigmoidoscopy, inspection of only the distal part of the bowel (the sigmoid colon); or (3) gastroscopy, visualising the oesophagus, stomach and upper part of the small bowel. There are some sections of the small bowel that cannot currently be reached by widely available forms of endoscopy. In those situations, options include capsule camera endoscopy (the ‘camera pill’), and imaging methods including ultrasound and magnetic resonance imaging (MRI).

Long delays in diagnosing IBD have been reported. Burgmann et al. 40 from Manitoba reported that 42% of a group of people with known IBD, had had GI symptoms for more than 3 years before the diagnosis of IBD, with some having symptoms for as long as 11 years before IBD diagnosis. Delays were much commoner in older age groups, with an incorrect diagnosis in around half of the over-64-year-olds compared with only around 10% in younger adults.

Diagnosis may be complicated by some patients having IBS before developing IBD. Because IBS is so common, this is not unexpected. It may also be that some such patients had IBD from the outset. However, it may be that the risk of IBD is raised in people that have had IBS. Porter et al. (2012)41 reported that in patients who had had what they called ‘well-defined IBS’, as confirmed by negative colonoscopies, the relative risk (RR) of later IBD was 15. They suggest that some patients had microscopic colitis with normal appearance on colonoscopy, whereas others might have had CD restricted to small bowel.

Irritable bowel syndrome-diarrhoea

Irritable bowel syndrome-constipation

Population

The population is patients with lower GI symptoms that are chronic, defined as persisting for at least 6–8 weeks. The upper age limit is 60 years, as per the NICE scope. 34 Symptoms in adults include abdominal pain or discomfort, bloating or change in bowel habit. Some will be newly presenting in primary care; others may already have been referred to specialist care.

Children (under 17 years) are a separate group with a different mix of conditions.

The main focus would ideally be in primary care, because that is where people with lower bowel symptoms first present. FC testing has not been widely available in, or to, primary care, and hence much of the differential diagnosis has been done in hospital clinics.

This could potentially give rise to problems reflecting selection for referral. For example, there may be three groups of people with IBS:

1. Those who do not seek help or advice from GPs but self-treat, as required, with over-the-counter medications, such as laxatives and analgesics: ‘self-managed’.

2. Those who do present to their GPs but whose symptoms are not considered such as to require referral: ‘GP-managed’.

3. Those referred by GPs to specialist clinics – the ‘referred’ group. At present it is estimated that only about 25% of patients are referred to secondary care. However, as in the past referral was often followed by colonoscopy, the threshold for testing with calprotectin may be rather lower than the threshold for referral, and many more than 25% may be tested with calprotectin (later, we assume 50%).

Evaluation of tests on only the referred group could, at least in theory, cause spectrum bias problems if the prevalence of IBS was less, and that of IBD higher, as parameters influenced by prevalence might differ from the GP-managed group. This could be important if FC testing was recommended and made more widely available. However, as noted above, GPs are highly selective in whom they refer.

Testing will be used mainly for IBS-D and not IBS-C.

Intervention

Faecal calprotectin tests (Table 2). These are of two types:

1. laboratory testing, using mainly enzyme-linked immunosorbent assay (ELISA) methods

2. point-of-care testing (POCT), which can be used in primary care or secondary care.

Laboratory methods are quantitative. Point-of-care tests may be quantitative or semi-quantitative.

The point-of-care tests can give faster results, within about 30 minutes. Extraction of the faecal sample is always manual, so some time costs are irreducible.

Comparators

The main comparator is clinical assessment, which can be supplemented by ESR and CRP, which can indicate inflammation, but not localise it. There are two options for ESR and CRP testing:

1. If GPs have access to FC testing, they could use that in people with suspected IBS. So FC would replace ESR and CRP testing.

2. If normal ESR and CRP could exclude inflammation of the bowel, they might be used as part of the initial work-up. However, the evidence suggests that normal CRP results can occur in the presence of active inflammation.

The limitations of ESR and CRP are:

• Negative tests do not exclude IBD, so if symptoms persist, patients would still require further investigation.

• Positive tests might be due to other, non-GI inflammations, so further investigations would be needed to localise the inflammation.

In one survey carried out in 2010,26 89% of gastroenterologists considered calprotectin to be more accurate than CRP and ESR for distinguishing between IBS and IBD. A review by Burri and Beglinger (2012)52 noted that ESR and CRP had low sensitivity.

As noted earlier, CRP and ESR have poor sensitivity for IBD.

Therefore, there seems little point in doing these tests even if calprotectin was not available. As noted previously, the YHEC report50 noted that CRP and ESR were economically dominated by calprotectin. These tests are therefore not examined further.

Outcomes

Depending on data availability, these may include:

• referral rates

• numbers of colonoscopies with/without FC testing

• proportion of colonoscopies with no abnormal findings

• duration from onset of symptoms to definite diagnosis of IBD – late diagnosis of CD

• cost

• adverse events such as complications of colonoscopy

• QoL and hence QALYs.

Modelling approach

A set of possible pathways is shown in Figure 2:

• No FC testing available. Clinical assessment and simple tests in primary care followed by decision on referral or symptomatic treatment/therapeutic trial in those thought to have IBS.

• Laboratory testing available to GPs. The laboratory just reports the results.

• ‘Laboratory plus’ where the GP provides clinical details along with the test request and gastroenterologist or clinical biochemist provides commentary and advice.

• POCT available in primary care. If it is negative, the GP manages the patient on a presumptive diagnosis of IBS. If the result is positive, the GP can refer to gastroenterology for further investigation. If indeterminate, the GP can either repeat test or refer.

FIGURE 2.

Service options.

Statistical methods

RevMan was used for data entry and analysis to generate forest plots, and MedCalc version 12.3.0 (MedCalc Software, Mariakerke, Belgium) for producing statistical data based on the 2 × 2 tables, including positive likelihood ratio (PLR), negative likelihood ratio (NLR), positive predictive value (PPV), negative predictive value (NPV) and disease prevalence.

Studies that provided sufficient data for calculation of sensitivity, specificity and other diagnostic outcomes were identified, and data were entered into RevMan for the generation of paired forest plots and receiver operating characteristic (ROC) curves. Further statistical analysis was performed in Stata 12 (StataCorp, College Station, Texas, USA) to produce likelihood ratios (LRs), area under the curve (AUC) and nomograms. Our intention was to examine the performance of calprotectin testing over a range of values, starting with the level recommended by the manufacturers, which is most often 50 µg/g. Where sufficient studies reported results at the same values, we aimed to pool data for each value.

Meta-analysis was performed in accordance with previously reported guidelines for meta-analyses of diagnostic tests using the Stata command ‘metandi’. 55,56 Pooled estimates for values among different diagnoses were obtained with 95% confidence intervals (CIs), assuming a bivariate model.

If there were sufficient studies, we planned to pool data at the same cut-off levels from ELISA and point-of-care tests separately, and compare them. However, only ELISA tests were pooled.

Quality assessment

Quality assessment of studies was done using items adapted from the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) I tool. 57

Reference standard

We took histology after endoscopy to be the definitive reference standard. Some studies used other reference tests. For example, the authors of one study in a paediatric group, quite reasonably, did not consider it justifiable to use endoscopy children with normal calprotectin levels. Instead, they used a 6-month period of observation. In one study in adults, only those with high calprotectin levels had endoscopy. Those with normal levels were managed in primary care.

Note that not all CD can be reached by endoscopy. About 30% of CD in adults is ileal alone, and 50–60% is ileocolonic. But about 20% is in proximal or mid-ileum, and so not accessible by standard colonoscopy of gastroscopy. Small bowel enteroscopy is complex, expensive and available in only a few centres in the UK. So, options include video capsule camera or MRI.

Magnetic resonance imaging of the small bowel (especially after preparation with enteroclysis) is sensitive and has been shown to correlate with FC levels. Zippi et al. 63 reported a good correlation between MRI changes (such as wall inflammation and thickening of bowel) and FC levels.

Ultrasound has been used in several studies. Aomatsu et al. 64 used ultrasound to detect CD in the small bowel in children, using > 3 mm thickening of the small bowel as indicating active CD. Calprotectin levels were much higher (mean 738 µg/g) in children in clinical remission but with active lesions on ultrasound than in children in clinical remission but without activity on ultrasound (mean 18 µg/g). In this study, ultrasound was used as a reference standard for calprotectin testing, but the reverse can apply.

Canani et al. 65 also used ultrasound in children but found some overlap between CD and non-IBD cases. However, they described transabdominal ultrasound of bowel wall thickness to be a useful and non-invasive method in confirming IBD, especially if inflammation was localised to the ileum.

Ultrasound is useful more as screening tool, as it is not sensitive enough to assess location of IBD. Small bowel MRI with contrast follow-through is standard in children. Wireless capsule endoscopy is also used.

Tomas et al. 66 considered that calprotectin showed good correlation with scintigraphy with radiolabelled leucocytes, which they considered was the gold standard for measuring inflammation in the bowel, although undesirable in children because of the radiation and the need for anaesthesia, and thus not used.

Patient groups in studies

The proposed value of calprotectin testing in primary care is to help GPs make decisions on likely diagnosis, in order to decide whom to refer to specialist care for further investigation. Patients with ‘red flag’ signs or symptoms are referred, and so are excluded from the calprotectin pathway. So the value of calprotectin is to guide decisions on whether to refer or not. A low calprotectin level indicates absence of inflammation, suggesting that IBS is the likeliest cause of the symptoms. A high level in someone with chronic symptoms suggests IBD, CD or UC. (FC can be raised in acute bacterial gastroenteritis but that usually resolves rapidly.)

Many studies compared ‘non-organic’ conditions (principally IBS in adults) with any organic condition. However, some organic conditions are not obviously inflammatory, so studies where the organic group included a mixture of conditions could make calprotectin testing look less useful. Calprotectin will therefore appear most impressive in studies that include only IBD and IBS.

Table 3 shows two things. First, the overlap in calprotectin levels between some organic conditions and IBS. Hence comparing only ‘all organic’ and non-organic will make calprotectin testing seem less valuable. Second, that calprotectin levels are raised in colorectal cancer, and to a lesser extent in people with larger adenomas. Adenomas are not usually regarded as being inflamed, in the sense of being infiltrated with white blood cells.

Cut-offs for calprotectin

One problem with the evidence is that many studies used only the manufacturer’s recommended cut-offs. This presents a problem when it comes to assessing optimum thresholds – there is little evidence for levels other than 50 µg/g. We are grateful to Professor KD Bardhan, Dr P Basumani and Dr A Banerjee for providing unpublished data (Rotherham Hospital, 2013, personal communication) from Rotherham on different cut-offs.

There is debate about the minimum number of studies that should be used for pooling data on different cut-offs, with four being regarded as the minimum. 67 We have therefore not pooled studies if there were fewer than four at the cut-off in question but have relied on diagnostic odds ratios (DORs) as the summary statistic when there were fewer than four studies.

Spectrum

Nearly all studies come from secondary care. The secondary care studies will have a different mix of patients from those seen in primary care. (See prevalence data in later tables.) The sensitivity and specificity of testing will be the same, but the different prevalence will give different predictive values.

This may be a particular problem in comparing different tests, such as point-of-care and laboratory tests. These may appear comparable in secondary care populations, but if the calprotectin levels are much higher in those selected populations then the comparability results may not be generalisable to populations with lower calprotectin levels.

Another issue about spectrum of patients arises from another selection effect. The pilot studies of calprotectin use in primary care from the northeast of England have shown that GPs are good at diagnosing IBS. In the Durham Dales pilot, 95% of those predicted by GPs to have IBS, had it. The GPs were also good at predicting IBD – 88% (28 of 32) of patients who had high calprotectin levels, had been predicted by their GPs to have IBD. So GPs may confidently diagnose IBS on clinical grounds in many patients, which implies that those who will have calprotectin testing may be a selected group. They may be more akin to those seen in secondary care – making the results from the secondary care studies more generalisable. A review by Van Roon et al. ,54 described in detail below, concluded that individual GI symptoms could not reliably distinguish between IBS and IBD, but GPs may use non-GI symptoms or signs, and ‘clinical nous’, to diagnose people with IBS. A recent review of IBS concluded that it could be diagnosed on clinical grounds:3

The diagnosis should be reached using symptom-based clinical criteria, rather than excluding underlying organic disease by exhaustive investigation.

Choice of measure

As noted by Harbord and Whiting,55 there is no single measure of diagnostic accuracy. They recommend that the measures most often used are sensitivity and specificity, with the trade-off between these being illustrated graphically.

In the sections that follow, we report:

• brief details of the included studies

• QUADAS quality assessment

• results

• sensitivity and specificity in paired forest plots, for all included studies

• for one study with a range of cut-off points, its own forest plot and ROC curve

• ROC curves with pooled sensitivity and specificity, and AUC

• Forest plots for the studies included in the ROC curve

• Fagan’s nomograms with likelihood ratios

• tables of DORs for different cut-offs, pooled where appropriate.

Summary

Some reviews are now out of date. The most recent ones (YHEC;50 Jellema et al. ;68 Henderson et al. 70) all conclude that FC testing is very useful. Henderson et al. 70 focuses only on use in children but is right up to date, and very high quality.

Conclusions of section ‘Studies of calprotectin in the differentiation of inflammatory bowel disease and irritable bowel syndrome’

Calprotectin testing appears very useful for differentiating between IBS and IBD.

Almost all sensitivities are high, the outlier being Otten et al. 73 with the 60 µg/g cut-off using a POCT. As expected, and shown best by the Basumani et al. 83 data in Figure 5, there is a trade-off between sensitivity and specificity.

The only point-of-care test in the group is the PreventID, which performed well at the 15 µg/g cut-off but not so well at the 60 µg/g cut-off, though it is curious that its specificity should be so high at the lower cut-off. The POCT with a 15 µg/g cut-off had higher specificity (95%) than the ELISA at 50 µg/g (87%).

The variability amongst sensitivities was much less than amongst specificities. Heterogeneity was moderate for sensitivity (I² = 37%) but high for specificity (94%). (However, see earlier note about the I² test in diagnostic reviews.) Even using the same PhiCal ELISA test (Calpro, Lysaker, Norway) with the 50 µg/g cut-off, sensitivities ranged from 83% to 96%.

Figure 6 provides the summary: sensitivity 93% and specificity 94%, for ELISA tests, at a 50 µg/g cut-off. These are based on five studies. 60,73,75,81,83 There was only one study for the 100 µg/g cut-off. 73 With an AUC of 0.97 at 50 µg/g, there is little room for improvement.

The only study using a POCT73 performed well at cut-off 15 µg/g with sensitivity 100% and specificity 95%. At 60 µg/g, sensitivity was only 61%, which is unlikely to be acceptable given the importance of not missing people with IBD.

All studies were on adults.

On this evidence base, it may be unwise to recommend any ELISA cut-off other than 50 µg/g.

Summary inflammatory bowel disease versus non-inflammatory bowel disease

In these mostly paediatric studies, the overall results pooled for IBD versus IBS, show very high sensitivity (99%: 95% CI 95% to 100%) (see Figure 13), but moderate specificity (74%) at a cut-off of 50 µg/g. At a cut-off of 100 µg/g, sensitivity falls to 94% (95% CI 86% to 98%), but specificity improves to 82% (95% CI 67% to 91%).

Calprotectin is therefore a valuable test in children with suspected IBD, and will allow most with non-IBD conditions to avoid invasive investigations, in particular colonoscopy.

Studies comparing faecal calprotectin tests: newly presenting patients

Four studies (Damms and Bischoff;58 Kok et al. ;39 Otten et al. ;73 Burri et al. 85) reported studies in which more than one type of commercial test kit was used, as shown in Table 28. Burri et al. 85 compared two different ELISAs, and the other three studies used a rapid test and an ELISA in the same patients. All of these studies were in adult patients.

Studies comparing faecal calprotectin tests in studies not in newly presenting patients

These are less useful for our purposes because they may reflect comparative reliability at much higher FC levels in active IBD or, conversely, lower levels in those in remission.

Caveat Many of the comparative studies are not yet published in full so details are often sparse. Missing details include those relating to sponsorship.

Studies comparing point-of-care testing and enzyme-linked immunosorbent assay tests

Studies comparing different point-of-care tests

Studies comparing point-of-care testing with enzyme-linked immunosorbent assay

Studies comparing enzyme-linked immunosorbent assay tests

Adults

As noted previously, we lack published data on the use of calprotectin testing in primary care. However, we have the results from the NTAC pilots,36 and these provide data on referral patterns by GPs in the UK (assuming that those in the North East are representative).

The Durham Dales pilot provides data on GP referrals with no calprotectin testing, and the effect that testing would have. The data allow us to explore what might happen if calprotectin testing is made available.

The test used was the POCT PreventID, which divides people into three groups:

• negative < 15 µg/g

• positive > 60 µg/g

• intermediate ≥ 15 µg/g but ≤ 60 µg/g.

General practitioners made diagnoses based on clinical assessment without knowledge of the calprotectin results. They referred those that they thought might have IBD, and managed those that they thought had IBS in primary care.

A final consultant diagnosis was made, based on calprotectin test results and clinical data, including endoscopy. The clinical data came from GP and outpatient data, where patients were referred, or just from GP data, when patients were not referred. Note that those diagnosed as IBS (and not referred) did not have colonoscopy so it is not possible to completely exclude false negatives. These would have IBD but appear clinically to be IBS and have negative calprotectin results. Such false negatives are unlikely given the high sensitivity (100% – see Figure 3) of calprotectin in this POCT at the 15 µg/g cut-off, but not impossible. (The Durham Dales pilot could not be used in our main assessment because of the lack of a definitive reference test.)

For assessing the sensitivity and specificity of GP assessment, there are two options using the Durham Dales pilot data:

1. use calprotectin as reference test

2. use final consultant diagnosis.

If we compare GP diagnosis with calprotectin levels, and assume that a positive calprotectin test implied possible IBD and an indication to refer, then we have a 2 × 2 table as shown by Table 35.

So sensitivity 28/34 = 82% and specificity 79/83 = 95% where ‘positive’ = a positive FC test. If we exclude the two indeterminates (who would be re-tested, rather than referred), sensitivity is 88%. Of the 83 diagnosed as IBS, only four had high calprotectin levels, a 5% error rate giving NPV of 95%. Note that the four are not false negative in the sense of being missed IBD but in the sense of being ‘false non-referrals’. Not all would have IBD. So without FC testing, GPs would not refer 4 of 32 patients with high calprotectin.

Given that the above comparison is of GP, versus calprotectin, the final consultant diagnosis is more useful for our purposes. Table 36 compares the GP diagnosis (without knowledge of calprotectin result) and the consultant diagnosis [with knowledge of calprotectin result and of endoscopy where performed. Note that far more patients (33) had endoscopy than were found to have IBD].

Numbers are slightly less than in the previous table because some patients do not appear to have been followed up. No data are given in the YHEC report on the presumed diagnosis or calprotectin results in five missing cases. 36 The sixth was found to have cancer.

These results show that the GPs referred all those diagnosed as IBD, giving a ‘whole pathway’ sensitivity of 100% (if we assume there were no false-negative IBDs as discussed above.) ‘Whole pathway’ combines GP assessment, calprotectin testing and consultant opinion based on clinical data that included endoscopy (mainly colonoscopy but some flexible sigmoidoscopies).

However this is achieved at a specificity of 79% for GP assessment without calprotectin testing. Without calprotectin testing, GPs refer a group of whom around 25% have IBD (7 of 29) and 75% have IBS. This matches results from routine care – that > 60% of colonoscopies in young people are normal.

This implies that if GPs had access to calprotectin testing, they might be able to reduce referrals by a considerable amount – about three-quarters. The Durham Dales data suggest that GPs refer about a quarter of patients presenting to them with GI symptoms that could be due to IBS or IBD. The number of patients in the study is quite small, but that proportion is similar to the figure of 29% reported in the BSG guideline on IBS, which increases confidence in the pilot data. 106

The prevalence of IBD in the whole population was 6.3% (standard error 2.3%) but amongst those referred it was almost 25% (6.3 of 25%).

In the pilot, a GP decision to refer set a patient on a pathway that could lead to colonoscopy and possible other invasive investigations. This decision would not be taken lightly. However, if FC testing is introduced, we might expect that GPs would consider testing in a wider patient group then they would consider for referral. They refer only about 25% of those that present to them with these symptoms. We can create a scenario analysis assuming that if calprotectin testing becomes available then GPs will test twice as many as they would have referred in the absence of FC testing.

We also note from the Durham pilot that if GPs thought that a patient had IBS, they were right at least 95% of the time because only 5% of those they thought had IBS had high calprotectin and needed referred (NPV 95%).These ‘false non-referrals’ could theoretically include some with IBD. In our scenario analysis, we assume that all patients with IBD will be in the larger group (50% of all patients with symptoms, so 222 patients) that will have calprotectin testing. If we assume that 50% of patients with symptoms will be tested, we get figures as shown in Table 36. All of the 6.25% of patients with IBD are tested, and assuming that the POCT sensitivity of 100%, no patients with IBD would be missed.

If we used an ELISA test, with a sensitivity of 93% (from meta-analysis, we would miss 0.44%, or 0.49 patients, in the numbers in this group).

The extra group consists of those regarded by the GP as less likely to have IBD than the 25% (because the GP did not refer them), and the GP is really doing the test to confirm IBS. The false positive rate among the additional 25% tested, will therefore be much less than in the 25% referred. One option is to assume that there will be no new false positives.

So figures change to those shown in Table 37a.

The prevalence of IBD in the tested group is half that in the referred group – about 12.5%. As all of those with IBD are tested, there are no false-negatives if we assume sensitivity of calprotectin testing to be 100%. Specificity is 90%. If we assumed that there would be more false positives, specificity would be 80% if we double the false positives to 44%, and 85% if we increased them to 33.

If the calprotectin test was the average ELISA with sensitivity 93% and specificity 94%, the figures in the Table 37a would change to those shown in Table 37b.

Only 9% would be referred owing to the greater specificity of ELISA, but 0.49 patients would be missed.

If we assume that only patients with raised calprotectin are referred, and that calprotectin is 100% sensitive for detecting newly presenting (and hence active) IBD, then with calprotectin testing, GPs will refer about 9% (20/222) compared with the 25% referred when they have no calprotectin testing available – a drop of around three-quarters. However, not all of the calprotectin false positives would be referred if GPs, aware of the imperfect specificity of the test, used clinical judgement and a repeat test with the more specific (94%) ELISA test before referral. That would reduce number referred to about 20 (approximately seven true positives and 13 false positives) or 9% – a drop of over 60%.

So for modelling purposes, using the PreventID test, we can use a prevalence of IBD of 6.3% (among all people with symptoms), and in the absence of FC testing, a sensitivity of GP referral of IBD of 100%, and 79% specificity.

Using the North European data from Shivananda et al. ,29 we would expect in this adult group, a ratio of UC to CD of 3 : 2. (Incidence of UC 12.9 in 15- to 44-age group, based on 539 cases; of CD 8.7, based on 365 cases.)

Note that there are some weaknesses in the above arguments:

1. The 50% is a rather arbitrary assumption. We have reasonably assumed that more patients with symptoms would have calprotectin testing than were referred when testing was not available but we cannot say if 50% is correct. Given that GPs are good at diagnosing IBS, we would not expect 100% to be tested.

2. Our base-case assumption is that doubling the number tested would not increase the number of false positives. As the extra 25% tested would have less severe symptoms than the first 25% (referred), it seems reasonable to rule out a doubling of false positives. However, assuming no increase may be too optimistic.

3. The 100% sensitivity for the point-of-care test is based on only one study73 with not very large numbers, and needs to be replicated in a larger study. The mean ELISA sensitivity was 93%. However, GPs would not simply rely on the test results alone, knowing that sensitivity was not perfect, and some of the false negatives on ELISA testing might be referred on clinical nous.

Children

Modelling requires different assumptions in children. Based on the recent UK study by Henderson et al. ,30 48% of referred cases (91/190) had IBD. The ratio of CD to UC is much higher, 2.3 : 1. The potential reduction in colonoscopies is therefore greater.

Irritable bowel syndrome: quality-of-life studies

Akehurst et al. 7 (sponsored by Novartis) undertook a survey of patients with IBS in the UK primary care setting. A sample of 161 patients with IBS was selected from GP lists based upon the Rome I Criteria, with an additional 213 control patients being selected. Controls were matched for age, sex and social characteristics by the patient’s GP. The SF-36, EQ-5D and Irritable Bowel Syndrome Quality of Life (IBS-QOL) questionnaire were administered at baseline and subsequently at 3 months. Patients with IBS reported average baseline SF-36 values that were statistically significantly worse than those reported by the control group for every dimension of the SF-36. Similarly, for the EQ-5D the mean baseline score reported by those with IBS of 67.5 was statistically significantly lower than that of the control group. Unfortunately, it is not clear what EQ-5D rating algorithm was used for this, with it being described as the ‘EQ-5D derived score’, and there is no reference to the UK social tariff. The mean baseline EQ-5D rating scale, presumably the EQ-5D visual analogue scale (VAS), reported by those with IBS of 64.2, was also statistically significantly worse than the 80.3 of the control group. Parallel statistically significant differences were also reported at 3 months, although the mean QoL values between baseline and 3 months were not statistically different.

In what was apparently a follow-up study to Akehurst et al. ,7 Ricci et al. (sponsored by Novartis)115 compared the HRQoL of 305 IBS patients selected from GP lists with 330 controls. Unfortunately, Ricci et al. 115 is only available as an abstract but notes a statistically significant relationship between the severity of IBS and patients’ reported VAS scores. No further detail on this is provided.

Bernklev et al. 116 also report SF-36 scores among IBS patients and compares these with French population norms, finding IBS to significantly adversely affect scores, but no overall QoL values are reported.

Within a broader paper comparing the performance of the EQ-5D and SF-36 across seven patient groups, Brazier et al. 117 apply the EQ-5D and SF-36 to 161 UK IBS patients recruited from primary care. All patients were observed twice. The EQ-5D was evaluated using the UK social tariff, whereas the SF-36 was valued using the Short Form questionnaire-6 Dimensions (SF-6D) algorithm. Based upon 314 responses, the mean EQ-5D index was 0.662 compared with 0.666 for the SF-6D. It appears probable, given the similarity of baseline characteristics and results with those of Akehurst et al. 7 (and that Brazier was a named author of the Akehurst et al. 7 paper) that the IBS patient group and responses were the same for both papers.

Bracco et al. 118 (sponsored by Novartis), in an economic evaluation of tegaserod compared with placebo for IBS, evaluated the EQ-5D responses using the UK social tariff of IBS patients: 247 receiving tegaserod and 238 receiving placebo. The adjusted average baseline utility was 0.726. At week 4, these had improved to 0.795 in the tegaserod group and 0.759 in the placebo group, but by week 12 had fallen back slightly to 0.792 and 0.747, respectively.

Dibonaventura et al. 119 (sponsored by Novartis) compared SF-6D utilities among 109, 83 and 204 patients with IBS-C in the UK, France and Italy respectively with matched controls in the UK, France and Italy. Respondents were recruited through the National Health and Wellness Survey, a self-administered internet based survey. UK patients with IBS-C reported a mean utility of 0.65 compared with 0.71 for their matched controls, the corresponding figures being 0.63 and 0.71 for the French sample and 0.66 and 0.70 for the Italian sample.

Pare et al. 120 (sponsored by Novartis) reported the UK social tariff EQ-5D index among 1555 Canadian primary care patients. The patient group recruited had mainly IBS-C or IBS-alternating type (IBS-A), apparently owing to a desire for the results to be relevant to patients eligible for tegaserod. The mean EQ-5D index was 0.64.

Puhan et al. 121 applied time trade-off (TTO), standard gamble (SG) and the SF-36 to 96 Canadian patients with IBS. Patients were identified through either medical records of the gastroenterology clinic at McMaster Health Sciences Centre or through 10 local gastroenterologists. The SF-36 was valued using the SF-6D transformation. This resulted in three mean estimates for HRQoL: 0.84 for SG, 0.76 for TTO and 0.85 for SF-6D.

Spiegel et al. 5 (sponsored by Takeda) administered the EQ-5D among 257 American IBS patients, with these split into three groups of those with constipation IBS-C, those with diarrhoea IBS-D and those with mixed, IBS-M. 5 Note that the severity of disease within the sample was mixed, with 16% having mild disease, 32% having moderate disease and 55% having severe disease. Patients were followed up at 3 months. It is unclear what algorithm was used to construct the EQ-5D utilities. There was no statistically significant difference in the mean utilities of 0.76 for those with IBS-C, 0.76 for IBS-D and 0.73 for IBS-M. There was a statistically significant difference between the mean utilities of 0.70 for those with severe disease compared with 0.80 for those with non-severe disease, and between the mean utilities of 0.78 for those experiencing considerable relief of symptoms at 3 months compared with 0.73 without considerable relief of symptoms at 3 months.

Wang et al. 4 administered the EQ-5D among 198 people with IBS and 251 people without IBS. These were recruited from those attending the National Foundation for Digestive Diseases Symposium, a free public education symposium held at the Raffles Hotel in Singapore. The EQ-5D was evaluated using the UK social tariff. The mean utility among those with IBS was 0.739, which was statistically significantly lower than the 0.849 of those without IBS.

Summary of modelling approach

The modelling required for a full cost–utility modelling exercise is complicated by there being at least three main conditions under consideration – IBS, CD and UC – and a range of other considerations when PIBD is compared with non-IBD because IBS, although still the commonest non-IBD diagnosis, is less common than in adults. Modelling induction and maintenance subsequent to diagnosis for these three conditions is quite involved. As a consequence, an initial consideration of the immediate QoL impacts for time spent as false negatives is presented, which can be considered alongside the costs of the initial test sequences and likely periods of time spent as false negatives. This QoL impact is restricted to the direct detrimental QoL impacts from not being correctly treated and not entering remission, this being limited by the time spent being incorrectly treated prior to representing for testing. EAG expert opinion suggests that 12 weeks is a reasonable base-case assumption for the duration of false negatives being incorrectly treated prior to the possibility of IBD being reconsidered.

But these immediate QoL impacts among false negatives are formally incorrect, as not all IBD patients when diagnosed with either CD or UC will immediately enter remission after treatment and remain in remission thereafter. Moreover, achieving remission and maintaining it is not costless. As a consequence, it appears that there is a requirement for a full cost–utility modelling exercise that takes into account the costs and benefits of induction therapy and maintenance therapy in both CD and UC, bearing in mind the potential problems of false negatives (IBD missed). This is the approach adopted by the EAG.

The modelling for the full cost–utility approach is eased by the modelling of induction and maintenance of remission for CG152 Crohn’s Disease: Management in Adults, Children and Young People108 and the modelling for the draft clinical guideline for UC being available. 109 Both sets of models adopt a similar framework. Induction therapy with the aim of remission but with subsequent induction therapies for those not achieving remission. Those achieving remission enter a maintenance of remission model, most patients being on treatment but a relatively small minority maintaining remission without active therapy. Remission can be lost, however, which leads to a further sequence of induction therapies. Note that the sequences of induction therapies in the initial induction therapy modelling and the sequences of induction therapies among those having lost remission in the maintenance of remission modelling therapy modelling differ, and even where the same therapy is involved it may have different clinical effectiveness estimates. All induction therapy sequences have as their final option surgery. This, in common with the modelling for the clinical guidelines, is assumed to achieve a permanent remission without the requirement for any further therapy, although this may be optimistic given the 10-year time horizon of the modelling.

Within this modelling, in common with the clinical guidelines’ modelling, there is no explicit consideration of possible disease progression, such as the development of fistula during the period of time spent being incorrectly treated as false negative or during periods of loss of remission. Were this to apply, it is likely that the relative importance of sensitivity over specificity would increase compared with the current modelling approach. We note that in the study by Shaoul et al. 92 two cases had fistulae at diagnosis.

The above has made little reference to the modelling of relief of symptoms in IBS patients. The approach adopted is broadly in line with that of the YHEC model,50 informed by the modelling for CG61: Diagnosis and Management of Irritable Bowel Syndrome in Primary Care. 35 This is simpler than the modelling of induction and maintenance in CD and in UC. But, given the assumed 100% specificity of colonoscopy meaning that there are no false positives at the end of the test sequence, the modelling of IBS and its treatment subsequent to diagnosis is of lesser importance within the overall cost–utility modelling. It is, in effect, a common residual to all comparators. Its main impact is to determine the costs incurred among false negatives being incorrectly treated. Given this, the full cost–utility model can be viewed as both an IBD versus IBS model and a reasonable IBD versus non-IBD model, provided that for the latter the costs among the false negatives are appropriately adjusted to take into account any additional testing and treatments that may occur among the non-IBD patients, noting the lower proportion with IBS in children.

Two scenarios are modelled:

• adult patients in primary care, with test accuracies for IBD compared with IBS

• paediatric patients in secondary care, with test accuracies for IBD compared with non-IBD.

Perspective, time horizon and discounting

The modelling adopts the NICE reference case perspective of patient benefits and NHS and PSS costs, over a 10-year time horizon for the base case, with discounting of costs and benefits at an annual 3.5%.

Model structure

For reasons of space, the cost–utility model is most simply presented as a set of interlinked models:

• the test model

• the induction and maintenance model among true positive patients with CD

• the induction and maintenance model among true positive patients with UC

• the induction and maintenance model among true negative patients with IBS and false-negative patients with IBD.

The model structure for the initial testing sequence is shown in Figure 21.

FIGURE 21.

Model structure of initial test sequences. a, Death after colonoscopy; and b, true negative = IBS.

Owing to the timing of testing and the possible delays between tests, all of the models use a weekly cycle. The delay between referral and colonoscopy is assumed to be 4 weeks and the time to retesting among those testing negative but not responding to IBS therapy is assumed to be 12 weeks, both estimates being based upon expert opinion. This may be optimistic, as noted in Chapter 1, because a sequence of unsuccessful treatments may be pursued for IBS, and so is explored in sensitivity analyses.

The above permits a range of test sequences to be compared. For instance, an initial POCT with a poor specificity could be followed by an ELISA test. FC testing does not have to result in an immediate referral for colonoscopy for all positive results. In a similar vein, the model structure also permits the exploration of rates of indeterminate test results having a follow-up test prior to any referral to colonoscopy. Owing to a lack of data, this latter option has not been formally explored in the analyses that follow, although a consideration of the possible impact of retesting indeterminate results is presented.

The key assumption in all of the above is that all of those who test positive, possible after a sequence of tests, receive a colonoscopy. The current modelling (Figure 22) assumes that referrals to secondary care result in colonoscopy. The model structure allows for referral to secondary care to result in assessment by a gastroenterologist, with only a proportion of those referred going on to colonoscopy. But this requires that the sensitivity and specificity of any gastroenterology assessment be estimated. A lack of data means that this option has not been considered. The sensitivity and specificity of an ELISA test could be seen as the closest available proxy for this.

FIGURE 22.

Model structure of CD true positives.

For the initial induction of remission in CD, the most cost-effective strategy within the modelling of CG152108 was an 8-week course of prednisolone, followed by an 8-week course that adds azathioprine to prednisolone, followed by a 6-week course of anti-TNF. Within this, the more cost-effective anti-TNF was adalimumab, and this is applied within the base-case modelling. This appears to be broadly in line with TA187,14 although this envisages treatment with an anti-TNF for unresponsive disease for up to 12 months.

For the modelling of the maintenance of remission for CD, this is again based upon the most cost-effective strategy identified within the modelling of CG152. 108 This assumes azathioprine as the maintenance therapy, followed by the same induction sequence as in the initial induction of remission modelling.

FIGURE 23.

Model structure of UC true positives.

HASA, high-dose aminosalicylic acid; high-dose aminosalicylic acid with beclometasone; HASB, high-dose aminosalicylic acid with beclometasone; HAST, high-dose ASA with a topical aminosalicylic acid; LASA, low-dose aminosalicylic acid.

In Figure 23, for diagrammatical simplicity, remission from inpatient therapy receives azathioprine maintenance therapy. All other patients receive LASA maintenance therapy if on active treatment.

For the modelling of induction of remission for UC, this is based upon the most cost-effective strategies identified within the modelling for the draft UC guideline: strategy 10 of table 34 of appendix L: High-dose ASA (HASA) followed by addition of a topical ASA (aminosalicylic acid), followed by high-dose ASA with beclometasone (HASB), followed by prednisolone. 109 This appears to be broadly in line with the recommendations of the draft clinical guideline. Whether induction of remission would initiate with the HASA of strategy 10 or perhaps the low-dose ASA (LASA) of strategy 6 is a moot point. The net monetary benefits of the two strategies are similar: £8513 and £8323 at a willingness to pay of £20,000 per QALY, but the cost-effectiveness of strategy 10 versus strategy 6 is estimated to be £2818 per QALY with a likelihood of cost-effectiveness of 47% compared with 18% for strategy 6. The current modelling adopts the sequence of strategy 10.

For the modelling of the maintenance of remission for UC, this is again based upon the most cost-effective strategies identified within the modelling for the draft UC guideline: LASA maintenance, followed by LASA maintenance for any patients losing but then regaining remission, as in table 59 of appendix L.

The rate of response to the various treatments for IBS affects the total costs, in that those not responding to initial treatment with dietary advice followed by medication are assumed to be referred for a colonoscopy if they have not already had one (Figure 24). The economic review and modelling of the IBS clinical guideline suggests a 45% response rate for placebo as drawn from Mearin et al. ,155 which for the base case will be taken to be the response rate to dietary advice. This is broadly in line with the 50% assumed in the YHEC report,50 which was based on expert opinion.

FIGURE 24.

Model structure of IBS true negatives and IBD false negatives.

The IBS clinical guideline also outlines a range of medical therapies for IBS, with RRs of response (compared with placebo), ranging from 1.32 for antispasmodic agents to 2.00 for anti-motility agents. As these can frequently be sequenced, this increases the overall response rates for medical therapies, for example sequencing in the antispasmodics modelling alone increases the estimated overall response rate to 78% in the modelling for CG61. 35 In the light of this, the 5% colonoscopy referral rate for those not responding to initial treatment with dietary advice followed by medication of the YHEC report,50 based upon expert opinion, appears reasonable.

Note that for IBS patients who have already had a colonoscopy it is assumed that the colonoscopy is not repeated. There is some inconsistency of approach in this, in that the small proportion of false-negative patients who have previously had a colonoscopy are assumed to receive a colonoscopy. This can be justified upon grounds of presentation subsequent to incorrect treatment for IBS, but is not without objection.

The above has been applied in the modelling of IBD versus non-IBD diagnosis, the implicit assumption being that similar delay to representation and costs are incurred among false negatives. This may not be the case, and given the other conditions than IBS within non-IBD it may be that the average weekly cost for false negatives in the IBD versus non-IBD modelling should be higher than those for the IBD versus IBS modelling.

For the costs of IBS, Akehurst et al. 7 estimated a net additional annual cost of £188 [£123] compared with a control group of patients. This suggests a weekly average of £9.29. For patients receiving medication for IBS a relatively minor additional £1.41 has been included, based upon a simple average of the cost of generic mebevirine and the cost of the branded mebevirine, Colofac (Abbott Healthcare).

Primary care modelling: base-case test characteristics

The base case considers the cost-effectiveness of GP testing without FC being available; CalDetect at the 15 µg/g cut-off as drawn from Otten et al. ;73 and ELISA at the 50 µg/g cut-off as drawn from Figure 6 of the clinical review (Table 43). The base case using the lower 15 µg/g cut-off of Otten et al. 73 may initially seem surprising, but the data for the 60 µg/g cut-off of Otten et al. suggest only a slight gain in terms of a better specificity, 97.8% compared with 94.5%, but significant loss in terms of a worse sensitivity, 60.9% compared with 100.0%.

The costs of Caldetect are based upon the list price for KST11005 PreventID CalDetect on the AlphaLabs website, coupled with 15 minutes of GP practice nurse time. ELISA testing is based upon an assumption of 40 patient samples per 96-well plate, costed at the list price on the AlphaLabs website, coupled with an average of 11–12 minutes of staff time at grade 6/7 of the NHS terms and conditions of service handbook: Annex C table 12. These staff costs have been proportionately increased for oncosts in line with the oncosts estimated for a hospital pharmacist within the PSSRU 2011 Unit Costs of Health and Social Care. 107 (Note that for the later scenario analyses, a £28.27 cost of Quantum Blue is based upon the list price for Quantum Blue on the AlphaLabs website, the list price for an extraction kit on the Biohit website, coupled with an average of 12–13 minutes of staff time at grade 6/7. Staff times for sample preparation for Quantum Blue and ELISA have been equalised.)

The resulting staff costs are very similar between all FC tests, and the main cost differences are due to the publicly available prices for the consumables. During this assessment it has been consistently noted by both suppliers and those using the tests within the NHS that most if not all tests are sold at a discount to the publicly available prices. These discounts will differ between tests, and possibly even between suppliers of a given test. There will be further geographic variation. Probably the best that the EAG can do is to consider a range of hypothetical discounts common across all tests, but, given the base-case results that follow, there is little requirement for this.

The accuracy of colonoscopy is drawn from expert opinion. We have assumed that colonoscopy is only 95% sensitive, for reasons including failure to intubate the terminal ileum, CD being restricted to parts of the small bowel not accessible by the colonoscope, technical problems, failure of the patient to tolerate the procedure, and inadequate bowel preparation.

Note that in Table 43 the colonoscopy cost includes the cost of an outpatient gastroenterology visit at a cost of £164 (NHS reference cost: 301 Gastroenterology Consultant led First Face to face Non Admitted) and the costs of adverse events. Given the rarity of bleeds and perforations, despite the large cost associated with perforation these add very little to the overall costs of investigation: only around £12 to the cost of each colonoscopy.

Secondary care modelling: base-case test characteristics

The base case considers the cost-effectiveness ELISA at the 50 µg/g cut-off as drawn from Figure 13 of the clinical review; the cost-effectiveness ELISA at the 100 µg/g cut-off as drawn from Figure 16 of the clinical review; and all patients being referred directly to colonoscopy. This results in the test characteristics for the secondary care modelling shown in Table 44.

Characterising uncertainty around the receiver operating characteristic curves

For the probabilistic sensitivity analysis (PSA), a possible and correct approach to characterising uncertainty around the central estimates drawn from the RoC curves is to draw a sufficient number of estimates from iterations of the WinBUGs code (MRC Biostatistics Unit, Cambridge, UK) to characterise the distributions. This has the advantage of correlating the sensitivity and specificity that underlies the RoC curve estimates. But for current purposes this required a large number of iterations for their means to converge to the central estimates of the RoC curves. Given the size and complexity of the other modelling this would have led to each PSA requiring a very long time to run.

In light of this, a simpler approach has been adopted. The sensitivity and specificity, or rather their deviation from 100% accuracy, is simulated using the gamma distribution. Over 1000 iterations, this results in the same central estimates as Figures 6, 13 and 16, but slight differences in the upper and lower confidence limits as outlined in Table 45. Any discrepancies appear minor, with the possible exception of the lower CI limit for the specificity of figure 6, possibly owing to these data being very heavily skewed compared with the other estimates. But it should be borne in mind that these simulations assume independence between the sensitivity and specificity of each RoC.

Baseline patient characteristics: primary care

For the primary care adult population, the model adopts a baseline age of 25 years for those presenting, as drawn from the CD CG152 modelling,108 although this may be quite low for IBS patients. In line with the CD CG152 modelling,108 the female proportion is taken to be 50% for both CD and UC. IBS appears to have a higher proportion of women presenting, the Brazier et al. (2004)117 sample being 86% female, although this estimate may be towards the upper end. The base case adopts a 75% female proportion for IBS. Note that these estimates affect only the all-population mortality risks. As these are low during mid-adulthood for both women and men, the average age and proportion of women inputs have minimal impact upon results.

The base case 6.3% (7/111) prevalence of IBD is drawn from the Durham data, whereas the 60% (539/904) prevalence of UC among IBD patients is drawn from Shivananda et al. 29

Baseline patient characteristics: secondary care

For the secondary care paediatric population, female proportions of 38% (35/91) for IBD patients and for 44% (44/99) non-IBD patients are drawn from Henderson et al. 30 An average age of 16 years is assumed, although as for the adult modelling this has minimal impact upon results.

The base case 48% (91/190) prevalence of IBD and the 75% (62/83) prevalence of CD among IBD patients are drawn from Henderson et al. (2012). 30

Other model inputs

Given the extent of the downstream modelling, the full set of model inputs is presented in Appendix 7, coupled with their treatment within the PSA.

Primary care modelling: sensitivity and scenario analyses

The prevalence of IBD within the presenting patient population determines the relative importance of sensitivity and specificity. This is explored through sensitivity analyses that vary this from 5% to 25% in 5% increments.

A scenario analysis that speculates that FC testing might be used in a wider patient group than would be referred in the absence of FC testing is then presented.

Further sensitivity analyses are then presented, which:

• vary the time to representation among false negatives from the base case 12 weeks to 8 weeks, 16 weeks and 24 weeks

• change the source of utility estimates from Stark et al. 124 and CG6135 to Gregor et al. ,133 Poole et al. 141 and Spiegel et al. 5

• remove the cost of the gastroenterology outpatient appointment from the cost of colonoscopy (this was the approach used in the YHEC 2010 report50 and is included here to allow comparison with their figures; it also provides a sensitivity analysis around the cost of referral and colonoscopy; usual UK practice would be to refer for a gastroenterology opinion that would lead to colonoscopy, but, in some countries, direct referral to colonoscopy appears to apply)

• vary the assumed non-response to medication among IBS patients from the base case 5% to 0% and 10% (this applies to those in whom dietary advice has failed)

• remove the mortality associated with colonoscopy.

The clinical effectiveness section also presents a range of sources that include estimates of the accuracy of FC testing at various cut-off levels. These are considered within individual scenario analyses, with the different cut-offs being directly compared, although these estimates are not integrated into the primary care base case.

Primary care modelling: other estimates of test characteristics

Additional effectiveness estimates for further analyses are drawn from Otten et al. 73 (Table 47) for CalDetect at the 60 µg/g cut-off; from Hessells et al. 98 for CalDetect at the 15 and 60 µg/g cut-offs, and for Quantum Blue at the 30, 40, 50 and 60 µg/g cut-offs (Table 48); and, from Basumani et al. 83 for ELISA at the 50, 100 and 150 µg/g cut-offs (Table 49). The central estimates from Otten et al. 73 are shown in Table 47.

In order to characterise the sensitivities and specificities of Hessells et al. 98 for the probabilistic modelling, the numbers of true positives, false negatives, true negatives and false positives has to be calculated. Unfortunately, Hessells et al. 98 only report the overall sample size, the numbers of correct diagnoses and the sensitivities and specificities. Given this, on the basis of a sample size of 85 the EAG has calculated the number with IBD to be 23, which implies the following numbers of true positives, false negatives, true negatives and false positives. These in turn imply sensitivities and specificities. In the main these correspond with those of Hessells et al. ,98 although there is some very minor disagreement of the order of 1% for a few of the percentages.

Secondary care modelling: sensitivity and scenario analyses

Sensitivity analyses are presented which:

• vary the prevalence of IBD within the presenting patient population from the base case 48% to 40% and 60%

• vary the time to re-presentation among false negatives from the base case 12 weeks to 8 weeks and 16 weeks

• change the source of utility estimates from Stark et al. 124 and CG6135 to Gregor et al. ,133 Poole et al. 141 and Spiegel et al. 5

• doubling the annualised net cost amongst false negatives from £188 to £376

• remove the mortality associated with colonoscopy.

Base-case results: primary care

For the primary care base case, the patient numbers receiving the initial test and being referred for colonoscopy are as shown in Table 50.

Note that the above relates to the initial test sequence of, for example, CalDetect 15 µg/g followed by colonoscopy. Within this test sequence, among those with IBD the initial Caldetect test identifies all 6.3% of patients with IBD as true positives. The colonoscopy subsequent to this identifies 6.0% of the 6.3% referred by Caldetect as true positives, owing to its 95% sensitivity. Among those with IBS, the initial Caldetect test identifies 5.1% of the 93.7% of patients with IBD as false positives. These are referred on, with the subsequent colonoscopy identifying all these as true negatives owing to its 100% specificity. As a consequence, although the initial test referred on a proportion of false positives, these are all eliminated by the colonoscopy and at the end of the test sequence there are no false positives.

Immediately apparent from the above is that GP opinion (without calprotectin) results in a somewhat larger number of false positives being referred for unnecessary colonoscopies: 19.8% of the total patient population or 21.2% of those with IBS, as would be anticipated from the 78.8% specificity. CalDetect 15 µg/g is somewhat better: 5.1% of the total patient population or 5.4% of those with IBS, as would be anticipated from the 94.5% specificity. The ELISA test, although perhaps marginally cheaper than the CalDetect test, is estimated to have an inferior sensitivity and a very slightly inferior specificity. The proportion correctly referred to colonoscopy is lower than for CalDetect, although the proportion incorrectly referred to colonoscopy is slightly higher.

Given the above, when coupled with representations for testing among false negatives and IBS patients not responding to IBS therapy who have not previously been scoped, this results in the following test costs, with the other costs from downstream modelling of treatment for induction and maintenance of remission yielding the total estimated costs for the cost–utility modelling.

Within Table 51, in part due to the quite low base-case prevalence assumed for IBD within the presenting population, the average QALYs and downstream costs of treatment are broadly in line between the three comparators. There are very slight differences between the comparators’ QALYs, with very slight gains from CalDetect over ELISA, and larger, although still slight, QALY gains over the GP with no FC testing. But the main differences are in the upfront average test costs, with CalDetect and ELISA having similar test costs, both of which are somewhat less than those of the GP in the absence of FC testing due to their superior specificity.

The central estimates and cost-effectiveness acceptability frontiers (CEAFs) from the probabilistic modelling run over 1000 iterations are shown in Table 52 and Figure 25, respectively. Within this, it should be borne in mind that the prevalence of IBD is also treated as being probabilistic within the PSA.

FIGURE 25.

Cost-effectiveness acceptability frontiers: primary care – base case.

In the above, the probability of being cost-effective for the comparator of the GP without FC testing never rises above the horizontal axis, i.e. it is estimated that there is no probability of GP without calprotectin testing being cost-effective compared with calprotectin testing.

As for the deterministic modelling, the probabilistic model central estimates suggest small QALY gains from FC testing coupled with cost savings compared with GP referrals in the absence of FC testing. There are very minor differences between CalDetect and ELISA in terms of the central estimates for costs and QALYs. Interestingly, the CEAFs suggest that ELISA is the most likely to be cost-effective, although, again, the differences between the two tests are not marked, but that CalDetect has the highest expected net monetary benefit at all but very low willingness-to-pay values. Further sensitivity analyses that increase the prevalence of IBD suggest that CalDetect will remain at the frontier for all but very low willingness-to-pay values, but, that as the prevalence of IBD rises, the likelihood of CalDetect being the most likely to be cost-effective increases; i.e. the crossover point moves towards lower willingness-to-pay values. But perhaps not too much should be read into this given the similarity of the central estimates, and that a few outlier iterations could have a more marked effect than usual.

Sensitivity analyses: primary care: prevalence of inflammatory bowel disease

Varying the prevalence of IBD in the presenting patient population between 5% and 25% provides the results shown in Table 53.

As in the base-case modelling, the GP with no FC testing is estimated to have the smallest overall QALYs and also to cost more than the other comparators for IBD prevalences up to 25%. At an IBD prevalence of 25%, the GP with no FC testing is still estimated to result in QALY losses compared with CalDatect, but in very slight patient gains compared with ELISA owing to the less-than-perfect 93.0% sensitivity of ELISA. The average cost savings from ELISA also fall from £83 at a 5% prevalence to £63 at a 25% prevalence, as there are lower cost offsets from avoiding fewer incorrect referrals for colonoscopy. But, owing to the very limited differences in QALY estimates and the additional average £63 cost, at a prevalence of 25% the cost-effectiveness of the GP without FC testing compared with ELISA is estimated to be £378,000 per QALY.

For CalDetect an increase in the prevalence of IBD increases the net QALY gain over ELISA, this mainly being due to fewer false negatives incorrectly receiving treatment for IBS. This also slightly increases the costs of CalDetect, to the extent that if the prevalence of IBD is ≥ 20% it is no longer cost-saving compared with ELISA. At a prevalence of 25% the incremental cost-effectiveness ratio (ICER) for CalDetect compared with ELISA is estimated to be £1697 per QALY but the differences in terms of the net costs and net QALYs between CalDetect and ELISA remain slight.

Sensitivity analyses: primary care: population tested and general practitioner sensitivity and specificity

As reviewed in the clinical effectiveness section, if FC is made available in primary care the patient group being tested may widen beyond that which GPs previously considered for referral. If patient numbers being tested with FC are double those who would previously have been seriously considered for referral in the absence of FC testing being available, this will affect specificity of the scenario of GP referral in the absence of FC testing. Assuming a doubling of the patient group for FC testing compared with that previously seriously considered for referral, coupled with a small percentage of additional IBD patients within the additional patient group, could suggest a prevalence of only 3.4% among those being tested with FC, and a sensitivity and specificity for GP referral in the absence of FC testing of 94.1% and 89.7% respectively in this wider patient group. This results as shown in Tables 54 and 55.

With the following results from the full cost–utility modelling.

Faecal calprotectin testing within the wider patient population increases the absolute number of false positives, whereas by construction it has been assumed to remain constant for those assessed by the GP in the absence of FC testing. This tends to reduce the difference between the costs of the test sequences between calprotectin testing and GP assessment in the absence of FC testing. But despite the assumed doubling in the size of the patient group from those who would seriously be considered for referral by the GP in the absence of FC testing to those who would be tested were FC made available to GPs, the introduction of FC testing is still estimated to be cheaper or at worst broadly cost neutral compared with the previous situation of FC testing not being available. Small QALY gains still accrue from FC testing as well.

Sensitivity analyses: primary care – test cut-offs

For the cut-offs for CalDetect reported within Otten et al. 73 the results shown in Table 56 are estimated.

The slightly better specificity of the 60 µg/g cut-off results in slight cost savings of £27 compared with the 15 µg/g cut-off, but gains of 0.0012 QALYs are anticipated from the 15 µg/g cut-off. This suggests a cost-effectiveness of £23,635 per QALY for the 15 µg/g cut-off which could be seen as borderline cost-effectiveness. Whether the 60 µg/g cut-off with an estimated sensitivity of only 61% would be acceptable in practice is a doubtful. Note also the ICER is almost exactly inversely proportionate to the prevalence if IBD in the presenting population, i.e. if it doubles then the ICER halves.

For the cut-offs for CalDetect reported within Hessells et al. 98 the results shown in Table 57 are estimated.

The results from the estimates of Hessells et al. 98 for CalDetect are almost the opposite of those estimated using those of Otten et al. 73 CalDetect with a cut-off of 60 µg/g is estimated to dominate CalDetect with a cut off of 15 µg/g. The specificity is notably better for 60 µg/g at 74.2% compared with 53.2% for 15 µg/g, and the sensitivity, although worse at 87.0%, is still somewhat closer to the 95.7% for 15 µg/g than the corresponding figures within Otten et al. 73 But it should be borne in mind that Hessells et al. 98 did not fit the eligibility criteria of the clinical effectiveness review.

For the cut-offs for Quantum Blue reported within Hessells et al. 98 the results shown in Table 58 are estimated.

As would be anticipated, the 50 µg/g cut-off is dominated, as is the 30 µg/g cut-off, by the 40 µg/g cut-off. The 40 µg/g cut-off is estimated to cost an additional £24 on average compared with the 60 µg/g cut-off, whereas small QALY gains of 0.0003 QALYs suggest an ICER of around £87,000. But these QALY differences are extremely minor and the ICER will swing possibly quite wildly if the underlying inputs and assumptions are changed. Note also that the better specificity of the 60 µg/g cut-off could have resulted in some further QALY gain from avoidance of the minor adverse effects of colonoscopy had these been included in the modelling. But the better sensitivity of the 40 µg/g cut-off would see increases in the prevalence of IBD increase its net QALYs further, the converse being true for a lower IBD prevalence.

For the cut-offs for ELISA reported within Basumani et al. 83 the results shown in Table 59 are estimated.

The 100 µg/g cut-off is only slightly inferior to the 150 µg/g cut-off in terms of specificity, but better in terms of sensitivity. This leads to a very slight QALY gain, but an additional £30 cost. The poor specificity of the 50 µg/g cut-off leads to it being dominated by the 100 µg/g cut-off.

Other sensitivity analyses: primary care

The univariate sensitivity analyses results in the estimates shown in Table 60.

The changes appear to broadly affect the three comparators in a like manner, such that although the estimates of costs and QALYs change there is only a limited impact upon net costs and net QALYs. FC testing remains cost saving compared with no-FC testing, and confers some small additional patient benefits. The costs of the POCT CalDetect FC testing and ELISA FC testing remain very similar throughout, with very slight patient gains from POCT CalDetect FC testing being estimated.

Scenario analyses: outpatient appointment sensitivity and specificity

As previously noted, the base case assumes that all those referred receive both an outpatient appointment and a colonoscopy. Owing to a lack of data, the possibility that the outpatient appointment results in some of those referred not receiving a colonoscopy cannot be given one formal estimate but it can be explored through additional scenario analyses.

For these scenario analyses, given the limited importance of test sensitivity to the net costs for the base case, in order to avoid too many permutations a 100% sensitivity is assumed for the consultant’s clinical assessment at the outpatient clinic (outpatient assessment). The specificity of this outpatient assessment can then be explored using values of 25%, 50%, 75% and 95%. As the specificity of the outpatient assessment rises so the importance of the specificity of the initial test falls. If the outpatient assessment has a very high specificity such as 95%, a good specificity for the initial test still avoids the costs of unnecessary referrals and outpatient assessments but it is no longer instrumental in avoiding the costs of unnecessary colonoscopies.

These analyses have been undertaken for two scenarios: the base-case referral scenario with an IBD prevalence of 6.3% and a resulting 100% sensitivity and 79% specificity and 24% PPV for GP nous without calprotectin testing, and the testing scenario analysis of a larger tested population (50% of those with symptoms) with an IBD prevalence of 3.4% and a resulting 94% sensitivity and 90% specificity for GP nous without calprotectin testing as shown in Table 61.

The key to this is that it assumes that there is a sequence of independent ‘tests’, for example the GP clinical assessment (‘GP nous’) followed by the outpatient assessment, or CalDetect followed by outpatient assessment. The sensitivity and specificity of the first test in the sequence are as per the table above. The outpatient assessment specificity is applied subsequent to this. For instance, GP nous refers 100% of those with IBD and 21.2% (100% – 78.8%) of those with IBS. By assumption, the outpatient assessment passes all the 100% of the referred IBD patient on to colonoscopy. But for the scenario of a 95% specificity for the outpatient assessment, only 5% (100%–95%) of the 21.2% of IBS patients who were referred for outpatient assessment are passed on to colonoscopy; i.e. a total of 1.06% = (100% – 95%) × (100% – 78.8%) of those with IBS among the whole population assessed by the GP. The parallel calculation for CalDetect would be 0.275% = (100% – 95%) × (100% – 94.5%) of those with IBS among the whole population assessed by the GP. (Note that in terms of the model structure the additional ‘test’ of the outpatient assessment also slightly alters the timing of tests within the sequence. In itself, this has minimal impact upon results.)

Note that for the options involving calprotectin testing, the above assumes that the specificity of the outpatient assessment does not vary by calprotectin level. It may be reasonable to assume that someone with IBS and calprotectin level 125 µg/g is more likely to be further investigated that someone with calprotectin of 55. That is, the outpatient assessment specificity may fall as calprotectin rises.

For the base case, this results as shown in Table 62.

The total costs reported in the above include not only the immediate test, referral and colonoscopy costs, but also the downstream costs of induction of remission and maintenance of remission therapy. However, the net costs are mainly composed of the net costs of the immediate test, referral and colonoscopy. The net QALY differences between the comparators are relatively minor.

Calprotectin testing is estimated to remain cost saving, although the cost savings dwindle as the specificity of the outpatient assessment improves. This is for the reasons outlined above: a high test specificity avoids referrals for outpatient assessment, but with a high specificity for outpatient assessment the high test specificity is no longer instrumental in avoiding unnecessary colonoscopies.

As the specificity of the outpatient assessment improves, ELISA becomes marginally cheaper than CalDetect owing to its slightly lower staff and consumables cost. This aspect should be treated with some caution in the light of the staffing time estimates being drawn from different expert opinion and publicly available list prices being used, when all tests are apparently sold at some discount to these.

In the alternative presenting population scenario, it is assumed that calprotectin testing will be carried out in double the number of patients previously considered for referral in the absence of calprotectin testing. This results as follows in Table 63.

The relatively small cost savings from calprotectin testing of the original analysis are reversed, as the specificity of the outpatient assessment rises above 50%. The QALY differences are relatively insignificant.

But Table 63 illustrates that results are sensitive to any assumed increase in the proportion of patients GPs may choose to test with calprotectin if it becomes available.

The Basumani data83 permit a comparison of the different ELISA cut-offs. But note that the EAG cost-effectiveness estimates using the Basumani data83 suggested that the 50 µg cut-off was dominated by the 100 µg cut-off owing to the poor specificity of the 50 µg cut-off. The specificities for the three Basumani data83 cut-offs – 50 µg, 100 µg and 150 µg – are also somewhat worse than that drawn from figure 6 for the pooled 50 µg cut-off. Applying these within the base-case results as follows in Table 64.

For the comparison between ELISA with the 150 µg cut-off and ELISA with the 100 µg cut-off, the slight net additional costs from ELISA with the 100 µg cut-off fall slightly as the outpatient assessment specificity improves, this lessening the importance of the better specificity of ELISA with the 150 µg cut-off. Very small net patient gains are estimated for ELISA with the 100 µg cut-off compared with ELISA with the 150 µg cut-off, although these are not sufficient to render it cost-effective at conventional thresholds.

For the scenario of the alternative presenting population, associating the outpatient assessment with a specificity results as shown in Table 65.

The lower prevalence of IBD increases the importance of specificity. Given the tests’ sensitivities, as outlined in Table 65, as the specificity of the outpatient assessment increases the better specificity of ELISA with the 150 µg cut-off relative to the specificity of ELISA with the 100 µg cut-off falls in importance. The increase in net costs associated with ELISA with the 100 µg cut-off is reduced, and very small patient gains result.

Indeterminate results

In addition to the model structure permitting a sequence of tests to be explored as outlined above, it also permits indeterminate test results to be treated differently than determinate results. For instance, those whose first calprotectin test was within a range deemed to be indeterminate, such as 50–125 µg, might receive a second calprotectin test after a period of time, and be only referred if the result of this second calprotectin test was > 50 µg. Expert opinion indicates that perhaps 10–15% of first calprotectin test results might be indeterminate.

Retesting those who receive an indeterminate result from their first test will increase the cost of the calprotectin testing element. But whether this increases or decreases overall costs will mainly depend upon what tends to happen during the period between the tests to the calprotectin level among the IBS patients who received an indeterminate first test result. If for the vast majority of these patients it remains within the indeterminate range then referral rates would remain largely unchanged and total costs would increase. But if for a reasonable proportion of these patients it falls back to < 50 µg by the time of the second test, referrals to secondary care will be avoided. Given relative costs, only a minority of these patients would have to fall back to < 50 µg for the additional costs of the second test to be more than offset by the reduction in the costs of unnecessary referrals and further investigations. The secondary care costs avoided will be a function of the outpatient assessment specificity, as outlined above. Given the uncertainty around the impact of the outpatient assessment specificity, building further speculation around the impact of indeterminate results upon this seems to be of limited worth.

Base-case results: secondary care – inflammatory bowel disease versus non-inflammatory bowel disease

For the primary care base case, the patient numbers receiving the initial test and being referred for colonoscopy are as shown in Table 66.

The ELISA with 50 µg/g cut-off results in 13.5% false positives being referred onward for colonoscopy, or 26.0% of the non-IBD patient population, as would be anticipated given the 74.0% specificity. This is at the minor cost of 0.5% false negatives not being referred on for colonoscopy at first presentation, or 1.0% of the IBD patient population as would be anticipated given the 99.0% sensitivity. The final results after colonoscopy are 45.0% true positives, 2.8% false negatives and 52.1% true negatives.

The ELISA with 100 µg/g cut-off results in only 9.4% false positives being referred onward for colonoscopy, or 18.0% of the non-IBD patient population, as would be anticipated given the 82.0% specificity. This is at the slightly larger cost of 2.9% false negatives not being referred on for colonoscopy at first presentation, or 6.0% of the IBD patient population as would be anticipated given the 96.0% sensitivity. The final results after colonoscopy are 42.8% true positives, 5.1% false negatives and 52.1% true negatives.

The differences between the two ELISA cut-offs are more marked in terms of false positives with the ELISA cut-off of 50 µg/g, resulting in 13.5% false positives being referred to colonoscopy compared with 9.4% for the 100 µg/g cut-off. But there is also a difference in the end results in terms of true positives being diagnosed at first presentation: 45.0% for the 50 µg/g cut-off and 42.8% for the 100 µg/g cut-off, a net difference of 2.3% of the presenting population or 4.4% of the presenting IBD population. Given the above this results in the following estimates (Table 67).

Faecal calprotectin testing with ELISA is estimated to be both cost saving and more effective than all patients receiving a colonoscopy. There are limited differences between the two ELISA cut-offs, with the 50 µg/g cut-off being slightly more expensive on average by £35, owing to an additional average £22 for tests among non-IBD patients and an additional average £13 among patients with CD owing to earlier diagnosis. It is also marginally more effective by 0.001 QALYs, which suggests a cost-effectiveness estimate of £33,982 per QALY, but it should be stressed that the estimated net QALYs are extremely small and that any change in the underlying inputs would have a large swing effect upon the ICER.

The central estimates and CEAFs from the probabilistic modelling run over 1000 iterations are as shown in Table 68 and Figure 26, respectively.

FIGURE 26.

Cost-effectiveness acceptability frontier: secondary care – base case.

The central estimate of net cost of ELISA with the 50 µg/g cut-off compared with ELISA with the 100 µg/g cut-off remains in line with the deterministic modelling at £35 as are the net QALYs at 0.001. The probabilistic central estimate of the cost-effectiveness of ELISA with the 50 µg/g cut-off compared with ELISA with the 100 µg/g cut-off is £33,088 per QALY.

Up to a willingness to pay of around £30,000 per QALY it is estimated that the ELISA with the 100 µg/g cut-off is most likely to be cost-effective and has the highest monetised health benefits net of costs. Thereafter, as the willingness to pay rises further it is estimated that the ELISA with the 50 µg/g cut-off is most likely to be cost-effective and has the highest monetised health benefits net of costs.

Sensitivity analyses: secondary care

The univariate sensitivity analyses for secondary care result as shown in Table 69.

As for primary care, most of the changes appear to broadly affect the three comparators in a like manner. The main difference arises from varying the prevalence of IBD, which tends to reduce the cost savings from FC testing as the prevalence rises, as would be anticipated. The source of utilities also has an impact upon the anticipated net gain from ELISA with the 50 µg/g cut-off compared with ELISA with the 100 µg/g cut-off, the ICER for which worsens to £117,000 per QALY. But this may be to overstate the effect given the prevalence of CD within the presenting population and the perhaps rather small QoL decrement sourced from Gibson et al. 134

Summary and discussion

Previous economic analyses have typically concluded that FC testing is cost saving compared with the situation without it. Given test specificities and the assumed prevalences of IBD in the presenting population, the additional cost of the FC testing is more than offset by the reduction in the cost of unnecessary colonoscopies. The YHEC 2010 report50 for the CEP concluded that FC testing not only saved money through the diagnostic pathway, but that it also resulted in more true positives and true negatives owing to its superior sensitivity and specificity and so dominated the situation of the GP referral in the absence of FC testing.

Sensitivities

A distinction of the EAG modelling from that of the literature is that for primary care the GP in the absence of FC testing is estimated to have at least as good a sensitivity as the FC tests, and for some comparisons a better sensitivity. In this circumstance, the GP results in as many or more true positives than FC testing. As a consequence, despite quite large cost savings and fewer false positives still being estimated for FC testing compared with the GP referring in the absence of FC testing, dominance for FC testing cannot be definitively concluded on the basis the diagnostic pathway alone. There is an argument that the cost and QALY impacts among the false negatives also need to be considered.

The costs and QALY impacts among the false negatives is in the first instance dependent upon the prevalence of IBD in the presenting population and the sensitivities of the tests under consideration. A low IBD prevalence and high test sensitivities mean that there will be few false negatives, although higher IBD prevalences and lower test sensitivities will increase the number of false negatives and so the importance of considering the costs and QALY impacts among them. These latter are dependent upon the average time spent as false negative prior to re-consideration of IBD within a diagnostic pathway. The QALY impacts are also dependent upon the source of the QoL estimates for false negative being incorrectly treated and for those correctly diagnosed, the latter requiring QoL estimates for remission and no remission. In the absence of other data it has been assumed that the QoL among those remaining as a false negative and being incorrectly treated is the same as among those correctly diagnosed but not in remission. The longer the period of time spent as a false negative and the larger the QoL gain from achieving remission, the larger the impact of false negatives and so the greater the importance of tests’ sensitivities.

Modelling induction of remission and maintenance of remission is eased for the current assessment by the relevant models for both CD and UC being available as appendices to the respective clinical guidelines, although the UC guidelines is still under consultation. A key assumption within these is that there is no disease progression, such as the development of fistula, when patients are not in remission. Were this to apply, it would also increase the importance of tests’ sensitivities.

Adverse events

The modelling also needs to consider the adverse impacts of unnecessary colonoscopies. Owing to data constraints, the cost impacts have been limited to modelling the cost impacts of the relatively rare serious adverse events of bleeds and perforations. The QoL impacts are limited to the mortality associated with perforations. Although perforations are rare, so resulting in a very low mortality rate, the QALY impact of this persists for the duration of the model.

There is evidence from the literature that colonoscopies result in minor adverse events among a reasonable proportion of patients; for example, de Jonge et al. 143 suggest that perhaps around 40% of those investigated with colonoscopy have some effects persisting 30 days subsequent to the colonoscopy. In common with the CG118 guideline on screening for colorectal cancer with colonoscopy,149 these minor adverse events have not been taken into account in the modelling principally because of a lack of QoL data. The effects of minor and transient colonoscopy side effects seem unlikely to affect the conclusions for the comparisons of no FC testing with FC testing, but they may take on a greater significance in the context of comparing different FC tests or different cut-offs. Depending upon the prevalence of IBD in the presenting population, inclusion of these minor adverse events would increase the importance of tests’ sensitivities.