Contrast-enhanced ultrasound using SonoVue® (sulphur hexafluoride microbubbles) compared with contrast-enhanced computed tomography and contrast-enhanced magnetic resonance imaging for the characterisation of focal liver lesions and detection of liver metastases: a systematic review and cost-effectiveness analysis

Incidentally detected focal liver lesions

A focal lesion in the liver refers to any tissue abnormality occurring in one specific area of the liver. FLLs can be classified into two main categories, namely benign or malignant. Benign FLLs include haemangioma, focal nodular hyperplasia, focal fatty sparing and adenoma. Malignant FLLs include primary cancer of the liver, known as HCC, and secondary cancers of the liver (metastases) resulting from primary cancers occurring elsewhere in the body (e.g. CRC, breast cancer, lung cancer and pancreatic cancer).

Once a lesion has been incidentally detected in an individual the foremost concern is to differentiate between benign and malignant lesions. This distinction determines the individual's prognosis and the subsequent treatment strategy. Benign liver lesions, because of their asymptomatic nature, often require no treatment. In such cases it is common for the individual to be monitored and the lesion rescanned in 6–12 months. Once a malignant lesion is identified it is important to distinguish between primary and secondary cancers, as this is likely to impact how the individual is managed. Malignant lesions may be treated by a range of interventions including chemotherapy, liver resection (surgery), radiofrequency ablation (RFA) and transarterial therapies such as selective internal radiation therapy for metastatic lesions secondary to CRC. A fine needle aspiration biopsy to assist in the diagnosis is not always needed and involves the risk of bleeding and the seeding of neoplastic cells (along the needle tract). It has been argued that the biopsy provides little additional information beyond what can be established from a patient history, medical examination, laboratory testing and imaging. 12

Cirrhosis surveillance

Guidelines from the UK Hepatocellular Group advise that, for all patients with cirrhosis who might be suitable candidates for treatment for HCC, surveillance using abdominal US and alpha-fetoprotein (AFP) estimation should be considered. 2 If surveillance is offered it should involve abdominal US assessments in combination with serum AFP estimation at 6-month intervals. US is used for surveillance because it is low risk, non-invasive and has good acceptance by patients. However, fibrous septa and regenerative nodules characteristic of cirrhosis produce a coarse US pattern that can inhibit detection of small HCCs. 13 If the US is inconclusive, confirmatory testing will take place using CECT or CEMRI. The decision about whether to use CEMRI or CECT as the next imaging modality following the initial US scan is highly dependent on clinician preferences and local availability. Although CEMRI in general has a better sensitivity and specificity than CECT for the detection and characterisation of FLLs, the main disadvantage of MRI is the often long waiting times; it can sometimes take up to 6 months for the presence or absence of a FLL to be confirmed. A focal lesion in the liver of a patient with cirrhosis is highly likely to be HCC. 2 Biopsy is rarely required for diagnosis as this can usually be established radiologically, and seeding of tumour in the needle tract occurs in 1–3% of cases. Therefore, it is advised to avoid biopsy of potentially operable lesions when possible. Clinical practice guidelines from the European Association for the Study of the Liver (EASL) state that non-invasive diagnostic criteria for HCC (hypervascular in the arterial phase with washout in the portal venous or delayed phases) can be applied in cirrhotic patients; one imaging technique is needed for lesions of > 1 cm diameter while two techniques are recommended in suboptimal settings and biopsy is recommended only when a diagnosis cannot be reached using non-invasive criteria. 13 HCC can be curatively treated with surgery, either hepatic resection or liver transplantation. 2 Palliative treatments include percutaneous ethanol injection (PEI), RFA and transarterial chemoembolisation (TACE).

Surgical resection is the treatment of choice for HCC in non-cirrhotic patients. Cirrhotic patients need to be carefully selected for resection because they are especially prone to postoperative liver failure and increased risk of death. Survival after resection improves if the disease is diagnosed during the very early stages when liver function is preserved, the patient is asymptomatic and the nodule size is small (single, < 2 cm); it can then exceed 50% at 5 years. Taking liver function into account can help to identify patients in whom the resection could lead to decompensation of the liver and death, when resection might not be the treatment of choice. In contrast, more advanced liver tumours preclude resection. Commonly, the indication for resection is limited to patients with single tumours in the liver, without signs of vascular invasion and dissemination by the tumour. Benefits from other treatment options, such as adjuvant chemotherapy, are uncertain. Recurrence of HCC is very frequent and exceeds 70% at 5 years. Repeated resection is possible if intrahepatic dissemination of the tumour has not occurred. Liver transplantation is an option for early-stage HCC (< 5 cm or with up to three nodules < 3 cm) but is not recommended for more advanced stages. If resection or transplantation is not appropriate, percutaneous ablation (local tumour cell destruction by chemicals or temperature) can be applied to patients with early-stage HCC.

Non-curative (palliative) treatment options may be considered when disease has progressed to medium or more advanced stages and surgery or percutaneous ablation is not considered appropriate. During tumour growth the tumour becomes highly arterialised, meaning that most blood that supplies the tumour is from the hepatic artery. During transarterial embolisation (TAE), acute arterial obstruction is provoked, which causes ischaemic tumour necrosis. If TAE is combined with a chemotherapeutic agent, which is injected into the hepatic artery prior to the procedure, the procedure is called TACE. TACE is indicated if the tumour has multiple nodules, without affecting blood vessels or dissemination outside the liver. Complete necrosis of the tumour is rarely achieved after one treatment, thus treatment needs to be repeated several times. Response to treatment improves survival, which varies from 20% to 60% at 2 years depending on tumour stage, liver function and general health status. Systemic chemotherapy in treating HCC is sometimes used although it is not recommended by the American Association for the Study of Liver Diseases (AASLD). 1 EASL clinical practice guidelines recommend sorafenib (Nexavar^®, Bayer Schering) as the standard systemic therapy in patients with well-preserved liver function (Child-Pugh class A), advanced HCC or tumours progressing after locoregional therapies. 13

Patients at an advanced stage of the disease, characterised by failure of liver function, tumour growth and dissemination or physical impairment, will not benefit from the above treatments and might therefore be enrolled in trials of new agents. In the terminal stage symptomatic treatment is appropriate. 1

Liver metastases for colorectal cancer

For cancers of both the colon and the rectum, surgical resection is the mainstay of definitive treatment. 14After surgical resection, patients may present with metastases. Metastases often first occur in the liver and this may be the only site of spread in 30–40% of patients with advanced disease. 15 For a patient discovered to have isolated liver metastases, CT of the chest, abdomen and pelvis should be performed to determine whether or not metastases at multiple sites are present. Isolated liver metastases of colorectal origin are commonly resected, with or without preoperative chemotherapy. In cases of small liver metastases, colon and liver resection might be combined in one surgery. Metastases at multiple sites may also be resected, with or without chemotherapy, or will be palliatively treated. If resection is not appropriate, systemic treatments such as chemotherapy in combination with other medication may be used; however, response to treatment is generally poor. Ablative therapy may also be considered; however, this is recommended only in the context of randomised controlled trials. As with HCC, recurrence of metastases after liver resection occurs in up to 60% of patients. 15

Patients without metastases are advised to undergo regular surveillance with a minimum of two CTs of the chest, abdomen and pelvis in the first 3 years and regular serum carcinoembryonic antigen (CEA) tests (at least every 6 months in the first 3 years). 14 Follow-up after liver resection is very dependent on local protocols but may include CT of the chest and liver and CEA testing for 5 years.

Participants

Study populations eligible for inclusion were adults (≥ 18 years) in whom previous liver imaging has been inconclusive, including patients being assessed for:

• suspected primary HCC

• suspected secondary malignancy (liver metastases)

• response to treatment/recurrence of known liver malignancy.

Setting

Relevant settings were secondary or tertiary care.

Interventions

The intervention (index test) was SonoVue CEUS.

Comparators

Comparator tests eligible for inclusion were:

• CECT

• CEMRI.

Reference standard

Studies reporting the diagnostic accuracy of SonoVue CEUS for the characterisation of FLLs (identification of liver malignancy) or the detection of liver metastases were required to use histology, following biopsy or surgical excision, to confirm the diagnosis in patients with positive index test results. Patients who test negative on the index test will generally not undergo biopsy or surgical treatment; clinical/radiological follow-up for a minimum of 6 months was therefore considered an acceptable reference standard in these patients.

Outcomes

Studies reporting the following outcomes were considered relevant:

• effect of testing on treatment plan (e.g. surgical or medical management, or palliative care), when information on the appropriateness of the final treatment plan is also reported

• effect of pretreatment testing on clinical outcome (e.g. overall survival, progression-free survival)

• prognosis – the ability of the test result to predict clinical outcome (e.g. overall survival, progression-free survival, response to treatment)

• test accuracy and number of patients/lesions classified as non-diagnostic by SonoVue CEUS.

For included studies reporting any of the above outcome measures, the following outcomes were considered, if reported:

• the acceptability of tests to patients or surrogate measures of acceptability (e.g. waiting time and associated anxiety)

• adverse events associated with testing (e.g. claustrophobia, reaction to contrast media)

• additional FLLs detected by CEUS, over and above those seen on unenhanced US.

Radiation exposure was not considered a relevant outcome as the population is mostly older adults in whom additional incident cancers due to imaging-related radiation are likely to be minimal. In addition, a previous technology assessment (new-generation CT for cardiac imaging) showed that including radiation exposure in modelling did not influence the results of cost-effectiveness analyses. 19

Study design

The following study designs were eligible for inclusion:

• Randomised or non-randomised controlled trials in which participants are assigned to the intervention or comparator test, for treatment planning, and outcomes are compared at follow-up.

• Observational studies that report the results of multivariable regression modelling, with clinical outcome (e.g. survival, response to treatment) as the dependent variable and the index test result as an independent variable. Included studies should control adequately for potential confounders (e.g. age, tumour stage, previous treatment, results of other imaging).

• Test accuracy studies in which the index test is compared with one or more of the comparators and the reference standard. Test accuracy studies of the index test alone were included when these were conducted in patients who had previously undergone one or more of the comparator tests (e.g. a study of the accuracy of SonoVue for the diagnosis of HCC in patients with inconclusive findings on CECT).

Included test accuracy studies were required to report the absolute numbers of true-positive, false-negative, false-positive and true-negative index test results or sufficient information to allow their calculation.

The following study/publication types were excluded:

• preclinical and animal studies

• reviews, editorials and opinion pieces

• case reports

• studies reporting only technical aspects of the test or image quality

• studies with < 10 participants.

Accuracy of SonoVue contrast-enhanced ultrasound for the characterisation of focal liver lesions detected on surveillance of patients with cirrhosis

Seven studies reported comparisons of SonoVue CEUS with other imaging techniques for the characterisation of FLLs detected on unenhanced US surveillance of patients with known cirrhosis. 42–48 One study, by Sangiovanni et al. , was reported as both a full paper47 and a conference abstract. 61 All of the studies in this section reported accuracy data for the differentiation of HCC from other liver lesions only and one study45 reported that there were no imaging-related adverse events. In total, the seven studies in this section reported 369 diagnoses of malignant liver lesions, of which 366 were HCC; the remaining lesions comprised two CCC and one liver metastasis. All studies in this section reported per-lesion data; three studies reported data for one lesion per patient, equivalent to per-patient test performance. 44,45,47 Studies generally focused on the characterisation of small to medium FLLs. Four studies prespecified the size of FLLs considered: < 30 mm42,45,46 or < 20 mm. 47 In two studies the mean size was 15 ± 3 mm43 and 14 mm (range 7-20 mm). 44 The remaining study did not specify lesion size as an inclusion criterion or report mean lesion size. 48 Two studies explicitly excluded lesions of < 10 mm42,47 and one study reported stratified data for different lesion sizes (< 10 mm and 11–30 mm). 45 Two studies compared SonoVue CEUS with CECT,43,46 three studies compared SonoVue CEUS with CEMRI45,48,62 and the remaining two studies compared SonoVue CEUS with both CECT and CEMRI. 42,47 One study included in this section explicitly reported that patients had an uncertain diagnosis following unenhanced US. 43 Five studies had previous unenhanced US examination as an inclusion criterion, and the ‘concern regarding applicability’ criterion for quality assessment was rated ‘unclear’ for these studies (see Table 3). 42,44–47 The remaining study was a retrospective analysis of information derived from a radiology database; inclusion criteria specified only that patients should have received both CEUS and CEMRI and histological confirmation of diagnosis (examinations prior to contrast-enhanced imaging were not specified), and the ‘concern regarding applicability’ criterion was therefore rated ‘high’ risk of bias for this study48 Comparators and imaging criteria used to define a positive test for HCC varied across studies and no meta-analyses were therefore undertaken. All but one42 of the studies in this section used histological confirmation in all patients or histological confirmation of imaging-positive patients and follow-up of imaging-negative patients as the reference standard.

All studies in this section were rated as ‘low’ or ‘unclear’ risk of bias for the ‘index test’ and ‘comparator test’ domains of the quality assessment tool. Two studies recruited consecutive samples of patients without inappropriate exclusions and were rated as ‘low’ risk of bias for ‘patient selection’. 43,45 Four studies were rated as ‘high’ risk of bias for the ‘patient selection’ domain because of the retrospective study design48 or inappropriate exclusions. 42,46,47 Two studies excluded very small lesions (< 10 mm);42,47 as these lesions may be more difficult to characterise, their exclusion may result in overestimations of test performance. One study excluded lesions with peripheral enhancement on CECT, which was considered to be indicative of a high probability of haemangioma. 46 Two of the three studies were also rated as ‘high’ risk of bias for the ‘flow and timing domain’ of the assessment, in one case because the reference standard used was not independent of the imaging test results42 and in the other because a high proportion of lesions (approximately 40%) were excluded because a histopathological reference standard was not performed. 46 One study was also rated as ‘high’ risk of bias for the ‘reference standard’ domain because a suboptimal reference standard (concordance between at least two imaging test results) was used in the majority of cases. 42

The two studies that compared CEUS and CECT had slightly differing definitions of a positive imaging test (hyperenhancement in the arterial phase followed by portal venous washout43 and hyperenhancement in the arterial phase with or without portal venous washout). 46 Neither study reported a significant difference in performance between imaging modalities for the differentiation of HCC from other liver lesions and neither study specified exclusion of very small FLLs. However, no data for very small FLLs were reported; in one study 46% of lesions were 10-15 mm and 54% were 16–20 mm43 and in the other study all lesions were in the range 10–30 mm. 46 The study by Dai et al. 43 reported slightly higher estimates of test performance, particularly for CECT specificity (see Table 4). The sensitivity estimates for CEUS and CECT were 91.1% (95% CI 80.4% to 97.0%) and 80.4% (95% CI 67.6% to 89.8%), respectively, and the corresponding specificities were 87.2% (95% CI 74.3% to 95.2%) and 97.9 (95% CI 88.7% to 99.9%). 43 The definition of HCC used by this study corresponded most closely with that reported in the EFSUMB guidelines on the use of CEUS. 6Table 1 summarises the typical enhancement patterns seen in various malignant FLLs. Quaia et al. 46 reported sufficient data to allow calculation of sensitivity and specificity for the combination of CEUS and CECT, with a positive finding on either imaging technique treated as ‘test positive’; they reported an increase in sensitivity for combined imaging compared with either CEUS or CECT alone with no change in specificity.

Three studies compared CEUS and CEMRI; two used gadolinium contrast-enhanced magnetic resonance imaging (Gd-CEMRI)44,45 and one used Gd-EOB-DTPA-CEMRI,48 a ‘combined’ vascular and hepatocyte-specific contrast agent. The two studies that compared CEUS and Gd-CEMRI used different definitions of a positive imaging test result and only Forner et al. 44 reported data for a definition of HCC, which corresponded with that given in the EFSUMB guidelines,6 which they described as ‘conclusive’ HCC. Forner et al. 44 also reported data for a definition of ‘suspicious’ HCC (hyperenhancement in the arterial phase without portal venous washout). Sensitivity and specificity were similar for CEUS and Gd-CEMRI using either criterion. Specificity tended to increase and sensitivity to decrease for both imaging modalities when the stricter ‘conclusive’ definition of HCC was used. This study did not stratify data by lesion size; however, very small lesions (≤ 10 mm) were included (15% of lesions were < 10 mm, 49% were 10–15 mm and 36% were 16–20 mm). The authors also stated that use of the AASLD criteria (concordant, ‘conclusive’ findings on CEUS and CEMRI) resulted in 100% specificity but low sensitivity (33%) (data not reported). Giorgio et al. 45 used (arterial phase) hypervascularity as the definition of a positive test and stratified data by lesion size. There was no significant difference in the performance of CEUS and Gd-CEMRI for the differentiation of HCC from benign lesions, for FLLs between 11 and 30 mm, and both techniques had sensitivity and specificity values > 85% (see Table 4). For very small FLLs (≤ 10 mm), the sensitivity of CEUS was lower than that of CEMRI (27% vs 73%); for both imaging techniques, sensitivity was poor when the analysis was restricted to very small FLLs. 45 Imaging test performance estimates were similar for the ‘all lesion’ data set from Georgio et al. 45 and the ‘suspicious’ diagnostic criteria data set from Forner et al. ;44 these data sets were similar in terms of diagnostic criteria and distribution of lesion size. The study that used Gd-EOB-DTPA-CEMRI did not report any information on lesion size. 48 The criteria used to define a positive imaging test result matched the definition of HCC given in the EFSUMB guidelines. 6 Sensitivity estimates were similar and high (> 90%) for both CEUS and Gd-EOB-DTPA-CEMRI (see Table 4). Specificity appeared lower for CEUS than for Gd-EOB-DTPA-CEMRI; however, the small number of patients with benign lesions in this study resulted in high imprecision in specificity estimates: 50% (95% CI 42% to 88%) for CEUS and 83% (95% CI 36% to 100%) for Gd-EOB-DTPA-CEMRI.

The two studies that assessed all three imaging modalities42,47 both reported data using a definition of HCC that broadly corresponded to that given in the EFSUMB guidelines,6 although Leoni et al. 42 stated ‘typical enhancement pattern’ without specifying portal venous/late phase washout and Sangiovanni et al. 47 also reported data using arterial hyperenhancement and portal venous washout separately as the definitions of HCC. Both studies assessed Gd-CEMRI and one study also assessed CEMRI using SPIO, a contrast agent that is selectively taken up by Kupffer cells in the normal liver and benign lesions and can therefore be used to identify HCCs, which are generally deficient in Kupffer cells. 47 When the EFSUMB-consistent definition of HCC was used, the two studies reported similar specificity estimates for all imaging modalities and for both MRI contrast agents; however, Leoni et al. 42 tended to report higher estimates of sensitivity. Sensitivity estimates from these studies were generally lower than those from studies with an EFSUMB-consistent definition of HCC that compared only CECT with CEUS43 or CEMRI with CEUS. 44,48 Leoni et al. 42 reported that Gd-CEMRI had the highest sensitivity of the imaging modalities assessed [81.8% (95% CI 69.1% to 90.9%)]. Both studies reported sufficient data to allow calculation of sensitivity and specificity estimates, with a positive result on any of the three imaging modalities treated as index test positive. Data from Leoni et al. 42 indicated that combining the three imaging modalities in this way could increase sensitivity [98.2% (95% CI 90.3% to 100%)] and decrease specificity [75.0% (95% CI 50.9% to 91.3%)] relative to any of the three imaging modalities alone. By contrast, combined imaging modality data from Sangiovanni et al. 47 did not appear to indicate significant improvements in sensitivity.

Table 3 provides a summary of the QUADAS-2 assessments for studies in this section and Table 4 summarises individual study results.

Accuracy of SonoVue contrast-enhanced ultrasound for the detection of liver metastases in patients with known primary malignancy

Two studies compared SonoVue CEUS with both CECT and CEMRI (SPIO-CEMRI in one study and both SPIO-CEMRI and Gd-CEMRI in the other study) for the detection of liver metastases in patients with known CRC. 49,50 Both studies reported per-lesion accuracy data and one study49 also reported per-patient data. These two studies reported a total of 46 diagnoses of metastatic liver lesions. One of these studies included only patients with known liver metastases who were being considered for curative surgery and was therefore rated as having ‘high’ concerns regarding applicability. 50 One study, which compared CEUS and CECT and reported data on the detection of any liver malignancy, was included in this section because the diagnostic status of participants at baseline was unclear and 52 of the 59 positive final diagnoses were liver metastases (primary tumours: colon 43, breast 5, neuroendocrine 2, renal 2); this study was rated ‘unclear’ for concerns regarding applicability. 51 One further study, which did not include a comparator test, was included in this section. 39 This study was included in the review because it reported an inclusion criterion of ‘indeterminate MDCT [multidetector computed tomography]-detected FLLs in patients with known primary cancers’ (various locations) and could therefore provide information on how SonoVue CEUS performs in patients who have had previous imaging other than US and in whom the diagnosis remains uncertain. All studies in this section used histological confirmation in all patients or histological confirmation of imaging-positive patients and follow-up of imaging-negative patients as the reference standard.

Two of the four studies included in this section were reported only as conference abstracts,39,50 resulting in a frequent judgement of ‘unclear’ risk of bias on quality assessment domains (see Table 5). Of the two full papers in this section,49,51 Clevert et al. 51 was rated ‘high’ risk of bias for the ‘flow and timing’ domain of QUADAS-2 because 21% of participants were excluded from the CECT analysis; both studies were judged to be at ‘low’ or ‘unclear’ risk of bias for all other domains. The study by Jonas et al. 50 was rated as ‘high’ risk of bias for the ‘patient selection’ domain because it aimed to assess the ability of imaging modalities to detect liver metastases while including only patients with known liver metastases.

When definitions of a positive imaging test were reported, studies that assessed imaging tests using vascular contrast media (CEUS, CECT and Gd-CEMRI) gave various descriptions of peripheral rim enhancement as the criteria for liver metastases. In addition, two studies reported data for CEMRI using the hepatocyte-specific contrast agent SPIO. 49,50 Jonas et al. 50 reported 100% specificity and, similarly, high (83–97%) estimates of sensitivity for all three imaging modalities (CEUS, CECT and SPIO-CEMRI). Mainenti et al. 49 also reported high (83–100%) specificity values for all imaging modalities and for both per-lesion and per-patient data. Per-patient sensitivity estimates were also consistent across all imaging modalities (83% in all cases);49 however, for both CEUS and CECT, the sensitivity estimates appeared lower for per-lesion data (50% and 69% respectively) than for per-patient data. 49 For both CEMRI methods, the per-lesion estimate of sensitivity (81%) was similar to the per-patient estimate. 49 By contrast, Clevert et al. 51 reported per-patient data and found similarly high (> 95%) estimates of sensitivity for both CEUS and CECT; however, specificity appeared lower for CECT than for CEUS [71.4% (95% CI 47.8% to 88.7%) and 97.6% (95% CI 87.1% to 99.9%) respectively] and images were non-diagnostic in approximately 15% of CT examinations.

Table 5 provides a summary of the QUADAS-2 assessments for studies in this section and Table 6 summarises individual study results.

Accuracy of SonoVue contrast-enhanced ultrasound for the characterisation of incidentally detected focal liver lesions

Five studies reported comparisons of SonoVue CEUS with other imaging techniques for the characterisation of incidentally detected liver lesions identified by unenhanced US. 41,52,54–56 All of these studies reported accuracy data for the differentiation of malignant from benign liver lesions and three studies also provided stratified data for the identification of HCC and the identification of liver metastases. 52,55,56 All but one of the studies in this section reported data on one lesion per patient and the remaining study41 reported per-lesion data for 694 lesions in 686 patients. Therefore, although data are reported per lesion, all results reported in this section can be considered equivalent to per-patient test performance. Four studies compared SonoVue CEUS with CECT41,52,54,55 and one of these52 also reported data on the combined performance of SonoVue CEUS and CECT, with a positive result on either test treated as positive. One study compared SonoVue CEUS with CEMRI. 55 No study reported comparative accuracy data for all three imaging modalities. None of the comparative accuracy studies described in this section explicitly stated that patients had an uncertain diagnosis following unenhanced US, although all patients had a prior unenhanced US examination and therefore the applicability criterion for the quality assessment was rated ‘unclear’ in all cases.

One further study, which did not include a comparator test, was included in this section. 53 This study was included in the review because it reported an inclusion criterion of ‘previous US and/or CT that had suggested the possibility of malignant liver lesions (not sufficiently proven benignancy)’ and could therefore provide information on how SonoVue CEUS performs in patients who have had previous imaging other than US and in whom the diagnosis remains uncertain. Altogether, the six studies included in this section reported 805 diagnoses of malignant liver lesions; these included 459 HCC, 333 liver metastases and 13 CCC. It should be noted that overlap between the study populations of Seitz et al. 55 and Seitz et al. 56 is highly likely as these two publications by the same group reported a very similar study design and identical recruitment periods; Seitz et al. 55 reported a comparison of SonoVue CEUS with CECT and Seitz et al. 56 reported a comparison of SonoVue CEUS with CEMRI in a smaller group of patients. All but one41 of the studies in this section used histological confirmation in all patients or histological confirmation of imaging-positive patients and follow-up of imaging-negative patients as the reference standard.

Studies were generally poorly reported, resulting in a judgement of ‘unclear’ risk of bias for many of the QUADAS-2 domain assessments. No study in this section reported recruiting a consecutive or random sample of participants and the ‘patient selection’ domain of QUADAS-2 was consequently rated ‘high’ or ‘unclear’ risk of bias in all cases. In addition, one study53 excluded patients who were unable to undergo biopsy and both Seitz et al. studies55,56 divided participants into two subgroups based on probable diagnoses after unenhanced US (‘suspected benign’ and ‘suspected malignant’). For the Seitz et al. studies, accuracy data could be extracted only for the ‘suspected malignant’ subgroup; this may have resulted in a higher than usual prevalence of malignancy and possible overestimation of test performance. Two studies were also rated as ‘high’ risk of bias for the ‘flow and timing’ domain, in one case52 because more than half of the participants initially recruited were excluded from the analyses (either because more than 1 month had elapsed between SonoVue CEUS and CECT or because positive lesions could not be confirmed by pathology) and in the second case41 because the reference standard used was not independent of the index test results. This study was also rated ‘high’ risk of bias for the ‘reference standard’ domain because a suboptimal reference standard (concordance between at least two imaging modalities) was used in the majority of cases.

All of the comparative accuracy studies in this section reported no significant difference in the accuracy of CEUS and CECT or CEMRI for the characterisation of focal FLLs. 41,52,54–56 The primary analysis in all studies was for the differentiation of malignant from benign lesions. Studies used similar criteria to define HCC (hyperenhancement in the arterial phase followed by portal venous/late phase washout) and liver metastases (peripheral rim enhancement in the arterial phase, decreasing in the portal venous and late phases). These criteria are consistent with the typical enhancement patterns described in the EFSUMB guideline on the use of CEUS6 (see Table 1). Pooled estimates of test performance for distinguishing malignant from benign FLLs, derived from the four studies that compared CEUS with CECT,41,52,54–56 indicated that sensitivity and specificity were similar for the two imaging modalities. The pooled estimates for the sensitivity of CEUS and CECT were 95.1% (95% CI 93.3% to 96.6%) and 94.6% (95% CI 92.7% to 96.1%) respectively. The pooled estimates for the specificity of CEUS and CECT were 93.8% (95% CI 90.4% to 96.3%) and 93.1% (95% CI 89.6% to 95.8%) respectively. I2 values were moderate (50–75%) for CEUS and high (> 75%) for CECT. Figures 4 and 5 illustrate the sensitivity and specificity values for each study comparing CEUS and CT, with pooled estimates. Sensitivity analyses excluding the study that used a suboptimal reference standard41 showed a trend towards lower estimates of test performance and reduced heterogeneity (I2 values were low, < 50%, in all cases). The new pooled estimates for the sensitivity of CEUS and CECT were 92.3% (95% CI 88.2% to 95.3%) and 87.4% (95% CI 82.7% to 91.3%), respectively, and the new pooled estimates for specificity were 88.2% (95% CI 79.8% to 93.9%) and 82.8% (95% CI 73.6% to 89.8%) respectively. It should be noted that exclusion of the study by Solbiati41 resulted in a large reduction in sample size (694 FLLs from a total sample size of 1038 FLLs) and hence greater imprecision (wider CIs) in the estimates of sensitivity and specificity.

FIGURE 4.

Forest plot of sensitivity and specificity of CEUS for the identification of any liver malignancy in patients with incidentally detected FLLs.

FIGURE 5.

Forest plot of sensitivity and specificity of CECT for the identification of any liver malignancy in patients with incidentally detected FLLs.

The single study that compared CEUS with CEMRI found no significant difference between the performance of the two imaging modalities for the differentiation of malignant from benign FLLs. The reported sensitivities were 90.0% (95% CI 80.0% to 97.0%) and 81.8% (95% CI 69.1% to 90.9%), respectively, and the reported specificities were 66.7% (95% CI 46.3% to 83.5%) and 63.0% (95% CI 42.4% to 80.6%) respectively. This study used gadolinium-enhanced MRI in all patients, with the addition of SPIO-MRI in an unspecified number of patients.

One study reported sufficient data to allow calculation of sensitivity and specificity for the combination of CEUS and CECT, with a positive finding on either imaging technique treated as ‘test positive’. 52 These data indicated that the addition of CECT to the imaging workup would not increase the accuracy of diagnosis over that obtained by CEUS alone; the sensitivity and specificity of CEUS for differentiating malignant from benign lesions were 91.1% (95% CI 78.8% to 97.5%) and 93.8% (95% CI 79.2% to 99.2%), respectively, and for CEUS and CECT combined were 93.3% (95% CI 81.7% to 98.6%) and 93.8% (95% CI 79.2% to 99.2%) respectively. Three studies reported sufficient data to derive estimates of test performance by lesion type (HCC and liver metastases), two comparing CEUS and CECT52,55 and one comparing CEUS and CEMRI. 56 The sensitivity and specificity of CEUS and CECT were similar for the characterisation of HCC; however, one study indicated that CEUS may be more sensitive than CECT for the characterisation of metastases [92.9% (95% CI 82.7% to 98.0%) compared with 67.9% (95% CI 54.0% to 79.7%)]. 55 The sensitivity and specificity of CEUS and CEMRI were similar for both HCC and liver metastases. 56

Table 7 provides a summary of the QUADAS-2 assessments for studies in this section and Table 8 summarises individual study results. Figure 6 shows the results for differentiation of malignant from benign FLLs for all studies in this section, plotted in the ROC plane.

FIGURE 6.

Receiver operating characteristic plane plot comparing performance of imaging tests for the differentiation of malignant from benign lesions in patients with incidentally detected FLLs.

Accuracy of SonoVue contrast-enhanced ultrasound for the determination of treatment success in patients with known liver malignancy

Three studies reported comparisons of SonoVue CEUS with other imaging modalities for the assessment of treatment success (complete response) in patients with malignant liver lesions (mainly HCC). 40,57,58 Two were Chinese-language publications57,58 and the other was published only as a conference abstract. 40 The two Chinese studies reported per-lesion data, with one57 reporting only one lesion per patient, and the remaining study reported only per-patient data. 40 The studies assessed patients following cryosurgery,57 RFA40 and ‘non-surgical treatment’. 58 Sample sizes were small: in total, studies reported data for 105 lesions (102 HCC and three liver metastases) in 97 patients. All three studies included only patients who were undergoing treatment for known liver malignancies and all studies were therefore rated as having ‘low’ concerns regarding applicability.

Studies were generally poorly reported and all QUADAS-2 risk of bias domains were rated ‘unclear’.

One of the two Chinese studies compared CEUS with CECT or CEMRI (numbers of patients receiving CECT and CEMRI, respectively, were not specified)57 and the other compared CEUS with CECT. 58 Both studies reported similar, high sensitivity (95.5–100%) and specificity (83.3–100%) for all imaging modalities, although small sample sizes resulted in wide CIs. One study reported sufficient data to allow the calculation of sensitivity and specificity for the combination of CEUS and CECT, with a negative finding on either imaging technique treated as ‘test negative’ for complete response. 58 These data indicated that the addition of CECT would not increase the accuracy of the assessment of response to treatment over that obtainable by CEUS alone; the sensitivity and specificity of CEUS for detecting complete response were 97.8% (95% CI 88.5% to 99.9%) and 94.4% (95% CI 72.7% to 99.9%), respectively, and for CEUS and CECT combined were 97.8% (95% CI 88.5% to 99.9%) and 100% (95% CI 81.5% to 100%) respectively. The remaining study compared CEUS with Gd-CEMRI and included only 15 patients undergoing RFA, with five final diagnoses of ‘complete ablation’. 40 The results of the two techniques were identical; sensitivity for the detection of complete ablation was 80% (95% CI 28.4% to 99.5%) and there were nine false-positives, resulting in a very low estimate of specificity [10.0% (95% CI 3.0% to 44.5%)].

Table 9 provides a summary of the QUADAS-2 assessments for studies in this section and Table 10 summarises individual study results.

Effectiveness of SonoVue contrast-enhanced ultrasound for treatment planning in patients with known liver malignancy

One controlled clinical trial compared SonoVue CEUS with unenhanced US (control) when added to routine imaging (CECT or CEMRI) for pretreatment assessment of patients undergoing RFA for HCC. 59 This study assessed the effect of CEUS on treatment effectiveness (successful ablation) as the primary outcome measure. Secondary outcomes were incidence of tumour progression, new HCC, repeat RFA and post-therapy complications, and duration of local progression-free survival and new tumour-free survival. The CEUS and control groups were similar at baseline in terms of age, gender distribution, numbers who had CECT and numbers who had CEMRI, TNM (tumour, lymph node, metastases) stage, tumour size and number, and numbers who had Child–Pugh class A cirrhosis.

This non-randomised study was considered to have ‘risk of bias’ in a number of areas. Alternate allocation of patients to the CEUS and control groups means that clinicians could predict patient allocation before recruitment. The nature of the study precluded the blinding of patients, and the blinding of assessors and/ or clinicians planning RFA protocols was not clear. Finally, 14 patients who were considered unsuitable for RFA after imaging assessment (nine in the CEUS group and five in the control group) were excluded from the analyses.

There were no significant differences in the rates of successful ablation (primary outcome) or post-therapy complications between the CEUS group and the control group. Use of CEUS in the pretreatment imaging protocol was found to significantly reduce incidence of tumour progression, new HCC and repeat RFA over a 2-year follow-up period; ORs were 0.35 (95% CI 0.13 to 0.95), 0.34 (95% CI 0.16 to 0.72) and 0.33 (95% CI 0.17 to 0.66) respectively. The use of CEUS also increased local progression-free survival [mean difference 7.2 months (95% CI 6.6 months to 7.8 months)] and new tumour-free survival [mean difference 11.7 months (95% CI 11.1 months to 12.3 months)].

Table 11 provides a summary of the risk of bias assessment for this study and Table 12 summarises the results.

Cirrhosis surveillance model

The cirrhosis surveillance model is a modified version of a model produced by the Health Economics Group, Peninsula Technology Assessment Group (PenTAG), Institute of Health Service Research, Peninsula Medical School (the PenTAG cirrhosis surveillance model). 70 This model was developed to assess the cost-effectiveness of several surveillance strategies in cirrhotic patients to identify HCC, using periodic serum AFP testing and/or liver US examination with CT as a confirmatory imaging technique, followed by treatment with liver transplantation or resection when appropriate. One of the research recommendations made by the authors was to assess the value of CEUS in surveillance strategies for cirrhotic patients. For the assessment of the value of CEUS in cirrhosis surveillance, this model required adaptation, because it did not allow for a confirmatory test with less than perfect accuracy. Also, the original model did not allow the comparison of different confirmatory tests.

The population of interest in the cirrhosis surveillance model in this assessment consisted of those with a diagnosis of compensated cirrhosis deemed eligible to enter a surveillance programme [aged ≤ 70 years with no pre-existing medical conditions that would preclude treatment with a liver transplant or hepatic resection (including current alcohol or intravenous drug abuse)]. The model allowed separate analysis of each of three cirrhosis aetiologies: alcoholic liver disease (ALD), hepatitis B virus (HBV) and hepatitis C virus (HCV). In the base-case analysis, results were produced for a mixed cohort weighted according to the following proportions: 57.6% ALD, 7.3% HBV and 35.1% HCV (expert opinion; as in the PenTAG model70). A probabilistic state transition (Markov) cohort model, constructed using Excel (Microsoft Corporation, Redmond, WA, USA), was used. The time horizon was lifetime and the cycle duration was 1 month.

The model diagram is shown in Figure 8. States are shown as boxes and allowable state transitions are shown as arrows. The basis of the model was the disease process or ‘natural history’ of cirrhosis. Within the natural history model, a distinction was made between those with compensated and those with decompensated cirrhosis. Those with compensated cirrhosis can progress to decompensated cirrhosis, which is irreversible and associated with excess mortality, costs and quality of life decrements. The rate of incidence of HCC is the same in those with compensated and decompensated cirrhosis. HCC can be either diagnosed or occult. Three classes of tumours were distinguished: small tumours (< 2 cm), medium tumours (2–5 cm) and large tumours (> 5 cm). Tumour size was used as a surrogate measure of all characteristics of tumour progression. Hence, tumour progression was modelled by a tumour growth rate. Both test performance in identifying tumours and treatability of the tumour are dependent on the tumour size. For example, for larger tumours there is a greater likelihood of identification. Incidental/symptomatic presentation of HCC is possible for those with both compensated and decompensated cirrhosis, for all tumour sizes, although with significantly lower probabilities for small and medium-sized tumours.

FIGURE 8.

Model diagram for cirrhosis surveillance, based on Thomson Coon et al. 70 Each cycle all patients who are alive can stay in the same health state, die from non-HCC related causes, or move according to the shown transitions. Patients with compensated cirrhosis can decompensate, by moving to the corresponding decompensated cirrhosis health state. Patients who are palliative or untreatable can die from HCC. Light grey states represent occult HCC. Dotted lines represent detection of HCC after screening or showing symptoms; stacked lines represent tumour growth; solid lines represent transplant.

The surveillance programme and treatment components are superimposed onto the disease process. The technical performance of each testing strategy was modelled using decision trees. The testing strategies consisted of unenhanced US followed by CEUS, CECT, Gd-CEMRI or SPIO-CEMRI as a confirmatory imaging test. In the base-case analysis surveillance was every 6 months and stopped for people who reached the age of 70 years. It was also assumed that compliance was 100%. The decisions trees are shown in Figure 9.

FIGURE 9.

Decision tree structure for the cirrhosis model. Note: The confirmatory tests are the comparators in this analysis: CEUS, CECT, Gd-CEMRI and SPIO-CEMRI.

The treatments considered in the model are liver transplantation and liver resection. People can enter the transplant waiting list following diagnosis of either surgically treatable HCC or decompensated cirrhosis. There is no prioritisation of people waiting for a transplant. During the time on the waiting list people are subject to the same natural history process as during prelisting. There is no waiting list for liver resection for HCC. Some people are deemed unsuitable for surgical treatment, including those whose tumours are large or whose tumours become large while on the transplant waiting list. Small tumours are deemed more amenable to surgical treatment than medium-sized tumours. People who undergo successful liver transplant or resection enter a simplified disease process in which post-transplant or post-resection mortality, costs and utilities are taken into account. People with small and medium-sized tumours that are deemed to be surgically untreatable enter a series of states to model palliative care. Palliative care includes PEI, RFA and TACE and supportive care. Once people progress to untreatable large HCC, an excess mortality and associated costs and utilities are applied to reflect the palliation provided by TACE for a proportion of these people. An overview of the key structural assumptions is provided in the following section. A more detailed description of the model structure can be found in Thompson Coon et al. 70

Liver metastases of colorectal cancer model

The CRC metastases model is a modified version of the metastatic model developed by Brush et al. 71 This model was developed to assess the cost-effectiveness of fluorodeoxyglucose positron emission tomography (FDG-PET)/CT as an add-on device in detecting metastatic cancer compared with conventional imaging (CT). The model was adapted to assess the cost-effectiveness of CEUS compared with CECT, Gd-CEMRI and SPIO-CEMRI in detecting metastases from CRC after an inconclusive unenhanced US scan. In addition to changing the comparators in the model, we added the cost of a whole-body CT scan for all patients with a positive test to detect whether or not metastases at extra sites are present. We also changed the way that false-positives were handled, and changed the watch and wait strategy to correspond with latest guidance. The watch and wait strategy was given not only to patients without metastases but also to those patients treated and still alive. A final addition was that we assigned false-negatives poorer survival in the first year because they are not treated immediately. These adaptations are described in more detail below. A decision tree combined with a probabilistic state transition (Markov) cohort model, constructed using Excel, was used. The time horizon was lifetime and the cycle duration was 1 year.

Figure 10 depicts the decision tree structure used for the metastases model. Patients who had previously had surgical treatment for primary CRC and in a routine follow-up assessment (involving a clinical examination and CEA testing) were found to have rising CEA levels and were identified as potentially having a metastatic recurrence received an unenhanced abdominal US scan. When this US scan was deemed inconclusive, the patient entered the decision tree and could receive CEUS, CECT, Gd-CEMRI or SPIO-CEMRI. Similar to the Brush et al. model,71 the decision tree splits the patient population according to true disease status (metastatic recurrence or no metastatic recurrence) before applying the DTA estimates, so that accurate and inaccurate diagnoses can be identified.

FIGURE 10.

Decision tree structure for the liver metastases from CRC model.

In this model, imaging (CEUS, CECT, Gd-CEMRI or SPIO-CEMR) will identify either metastases (test positive) or no metastases (test negative). After a positive test, patients receive a whole-body CECT scan to identify whether there are metastases at one site or at multiple sites. In the base case it was assumed that all patients in the model receive a biopsy to confirm the metastases before treatment, and it was assumed that biopsy is 100% accurate. Thus, in contrast to the Brush et al. model,71 patients with a false-positive test result will not receive treatment. Patients with a positive biopsy (true-positives) receive treatment. In line with Brush et al. 71 it was assumed that all patients with metastases at a single site will receive preoperative chemotherapy and surgery for metastases, and that patients with metastases at multiple sites are assumed to be non-curable and will receive either preoperative chemotherapy followed by surgery and palliative care, or chemotherapy and palliative care. In line with the Brush et al. model, patients with a negative test result are followed up in a watch and wait strategy for 3 years. Also in line with the Brush et al. model, for patients who are inaccurately diagnosed as having no metastases (false-negatives), the true diagnosis is assumed to be identified within a year if the patient is still alive. These metastases can be detected during scans in the watch and wait strategy or because the patient becomes symptomatic. This delayed detection involves a second scan (CEUS, CECT, Gd-CEMRI or SPIO-CEMRI, depending on the comparator), a whole-body CT and a biopsy.

After the decision tree phase, a state transition (Markov) model was used to follow up the patients (Figure 11). After the second year, when every patient is correctly diagnosed, patients can either stay in their health state or die. In the first 3 years, patients without metastases and those who were treated were assumed to be followed up using the watch and wait strategy.

FIGURE 11.

Simplified schematic diagram of the Markov model for follow-up of patients in the CRC metastases model.

Incidentally detected focal liver lesion model

Patients with incidentally detected FLLs can have a variety of diseases, ranging from malignant lesions such as HCC and metastases to different types of benign lesions. Figure 12 illustrates the different combinations of test results and lesion types. The choice of lesion categories was based on similarities and differences in treatments, costs and prognosis.

FIGURE 12.

Description of patient categories and their treatments used in the incidentally detected FLL model.

The prognosis, costs and QALYs seen among patients diagnosed with HCC were modelled using the cirrhosis model, whereas the prognosis, costs and QALYs among patients with liver metastases were modelled using the liver metastases model. The incidentally detected FLL model therefore incorporated elements of the cirrhosis model and elements of the liver metastases model as well as some new elements. The cirrhosis model required adjustments before it could be incorporated into these analyses. One important issue related to when HCC is diagnosed. In particular, although none of the patients in the cirrhosis surveillance model has HCC at the start of the simulation, all HCC patients in the incidentally detected FLL model will have HCC at the start of the simulation.

The economic and health consequences of false-positive and false-negative results were modelled in the following ways. First, it was assumed that patients with HCC who were not correctly identified at baseline would be correctly diagnosed within several months, as essentially all of these patients will have important risk factors (e.g. alcohol misuse, newly diagnosed cirrhosis or hepatitis) that are identified at baseline. Patients with a false-positive diagnosis (in particular, patients with a benign tumour that was misclassified as a malignant tumour) were assumed to undergo one additional follow-up consult as a result of this misclassification. This was viewed as a conservative assumption that would bias the assessment against CEUS and in favour of the comparators (CECT, CEMRI), as a false-positive result might lead to even greater costs than the cost of simply one extra visit and as CEUS was found to have a lower rate of false-positives in the DTA studies.

The costs, life-years and QALYs for patients having a malignancy other than HCC or metastases were assumed to be equal to those for HCC patients (see Figure 12). These other types of malignant lesions (e.g. lymphoma) were infrequently seen among patients with an incidentally detected FLL and the studies comparing CEUS with CECT or CEMRI provided little information about these lesions. Given the heterogeneity in costs and QALYs within this group (and even among patients with the same malignancy), we chose to set the base-case values to the costs and QALYs seen with HCC patients and emphasise that this was an assumption. However, it was known in advance that the costs and QALYs of these patients would have a limited effect on the cost-effectiveness of CEUS compared with the comparators for two reasons: the values for the sensitivity of CEUS and the comparators were very similar and the prior probability of other malignancies was small. In fact, the only possible way in which the values for costs and QALYs of other malignancies could have any effect on the overall cost-effectiveness was if the costs and QALYs changed dramatically if the malignancy were to be incorrectly classified as a benign lesion (i.e. a false-negative test result). The impact of this false-negative effect was therefore examined using sensitivity analysis.

Cirrhosis surveillance model

Liver metastases of colorectal cancer model

Incidentally detected focal liver lesion model

Cirrhosis surveillance model

The proportion of patients receiving confirmatory imaging (the proportion of patients with an inconclusive unenhanced US scan: 43%) was an uncertain parameter in the model. Therefore, we performed a sensitivity analysis in which CEUS, CECT and Gd-CEMRI were used for a proportion of patients equal to the proportion of patients with a positive unenhanced US scan (as a minimum estimate of the patients requiring confirmatory imaging). Second, we reduced the proportion of inconclusive unenhanced US scans considerably (20% instead of 43%). Next, we conducted sensitivity analyses on the age limit of surveillance (90 years instead of 70 years), the frequency of screening (every year instead of every 6 months) and the tumour sizes for which the accuracy data were applied (small only instead of small and medium-sized).

Finally, scenario analyses were conducted using other sources for the accuracy of the tests. As alternative sources we used the articles by Dai et al. ,43 Quaia et al. ,46 Blondin et al. 48 and Giorgio et al. 45 (using data for 11- to 30-mm lesions). Dai et al. and Blondin et al. were included as other examples of studies that used a standard (EFSUMB guidelines6) definition of HCC, and Giorgio et al. and Quaia et al. were included to explore the effects of using other definitions of HCC. The study by Forner et al. 44 was not used because it included a significant proportion of patients with very small (< 10 mm) FLLs and the study by Sangiovanni et al. 47 was not used because it was considered to be an ‘outlier’ (accuracy results differed substantially from those of other, apparently similar studies).

Liver metastases from colorectal cancer model

First, we analysed the impact of not having a biopsy before treatment on the expected costs and effects. This would imply that patients who were inaccurately detected as having metastases would receive treatment, as was assumed in the Brush et al. model. 71 Second, we examined the impact of a 80% instead of a 40% probability of having metastases. We did this because our population of patients who have already received an unenhanced US scan may be slightly different from the population in Brush et al. 71 and may consist of more patients with metastases.

Next, we performed scenario analyses using other sources as input for the accuracy of the tests. Although the results refer to lesions instead of patients, we used the sensitivity and specificity reported in Jonas et al. 50 to assess the impact on the expected costs and effects. We also used the sensitivity and specificity reported in Clevert et al. ;51 this study included some patients with primary cancers other than CRC, but the majority (> 80%) of metastases diagnosed were from CRC.

Incidentally detected focal liver lesion model

A number of different parameters were varied to investigate their impact on the cost-effectiveness of CEUS. First, we increased the probability of a malignant lesion. We also examined the impact of basing the values for the sensitivity and specificity of CEUS and CECT on individual studies rather than on the meta-analysis. We then examined whether or not assuming that all patients with HCC had medium-sized lesions instead of small lesions would have an effect on the results. Lastly, we analysed the impact of changing the costs and health loss from an incorrect diagnosis of HCC or metastasis.

Cirrhosis surveillance model

Additional analyses for surveillance

Sensitivity analyses

First, we analysed the impact of using CEUS, CECT and CEMRI as confirmatory imaging for a proportion of patients equal to the proportion of patients with a positive unenhanced US scan (Table 30). In line with the base-case analysis, CEUS was as effective and less costly than CECT. Gd-CEMRI was also more costly (£321) and more effective (0.025 QALYs) than CEUS, resulting in an ICER of £12,806 per QALY gained. Based on a willingness-to-pay threshold of £20,000–30,000, this indicated that Gd-CEMRI was cost-effective compared with CEUS in this analysis.

TABLE 30 - Results of sensitivity analysis for cirrhosis surveillance: imaging used as confrmatory after all positive non-enhanced US examinations

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	35,828	10.220
CECT	35,867	10.220	39	0.000	Dominated	CEUS	39	0.000	Dominated
Gd-CEMRI	36,148	10.245	321	0.025	12,806	CEUS	321	0.025	12,806

Second, we changed the proportion of inconclusive US scans from 43% to 20% (Table 31), the age limit of surveillance to 90 years instead of 70 years (Table 32), the frequency of screening to every year instead of every 6 months (Table 33) and the accuracy data to use only those that applied to small tumours instead of small and medium-sized tumours (Table 34). Only when changing the proportion of inconclusive US scans was Gd-CEMRI cost-effective compared with CEUS, with an ICER of £16,121 per QALY gained. In all other sensitivity analyses CEUS dominated CECT and was cost-effective compared with Gd-CEMRI.

TABLE 31 - Results of sensitivity analysis for cirrhosis surveillance: proportion of inconclusive US scans 20%

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	35,784	10.192
CECT	35,959	10.192	176	0.000	Dominated	CEUS	176	0.000	Dominated
Gd-CEMRI	36,408	10.216	624	0.024	16,121	CEUS	624	0.024	16,121

TABLE 32 - Results of sensitivity analysis for cirrhosis surveillance: age limit for screening 90 years

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY(£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	36,163	10.164
CECT	36,593	10.164	430	0.000	Dominated	CEUS	430	0.00	Dominated
Gd-CEMRI	37,367	10.188	1204	0.023	51,619	CEUS	1204	0.023	51,619

TABLE 33 - Results of sensitivity analysis for cirrhosis surveillance: annual screening

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	34,431	10.093
CECT	34,629	10.093	198	0.000	Dominated	CEUS	198	0.000	Dominated
Gd-CEMRI	35,025	10.109	594	0.016	37,619	CEUS	594	0.016	37,619

TABLE 34 - Results of sensitivity analysis for cirrhosis surveillance: accuracy data for small tumours only

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	36,054	10.191
CECT	36,432	10.191	378	0.000	Dominated	CEUS	378	0.000	Dominated
Gd-CEMRI	36,966	10.195	913	0.004	244,840	CEUS	913	0.004	244,840

Scenario analyses

Scenario analyses were conducted using other sources for data on the accuracy of the tests. As alternative sources we first used the articles by Dai et al. 43 and Quaia et al. 46 These studies both compared CEUS and CECT. Dai et al. used a definition of a positive test for HCC which was comparable with that used in the EFSUMB guidelines,6 whereas Quaia et al. did not. Using data from either study, CEUS was found to be less costly and more effective than CECT (Tables 35 and 36).

TABLE 35 - Results of scenario analysis for cirrhosis surveillance: Dai et al.43 used as source for accuracy data

Strategy	Cost (£)	QALYs	Compared with CEUS
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	36,203	10.188
CECT	36,332	10.184	129	−0.004	Dominated

TABLE 36 - Results of scenario analysis for cirrhosis surveillance: Quaia et al.46 used as source for accuracy data

Strategy	Cost (£)	QALYs	Compared with CEUS
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	36,479	10.185
CECT	36,767	10.180	288	−0.005	Dominated

Next, we used Blondin et al. 48 and Giorgio et al. 45 as sources for input for the accuracy of CEUS and Gd-CEMRI (Tables 37 and 38 respectively). Blondin et al. used a definition of a positive test for HCC which was comparable with that used in the EFSUMB guidelines,6 whereas Giorgio et al. did not. Based on Blondin et al. , Gd-CEMRI was found to be more costly and less effective than CEUS. Based on Giorgio et al, using only data for lesions between 11 and 30 mm, Gd-CEMRI was found to be more costly, but also more effective than CEUS. However, the resulting ICER of £297,695 was very high.

TABLE 37 - Results of scenario analysis for cirrhosis surveillance: Blondin et al.48 used as source for accuracy data

Strategy	Cost (£)	QALYs	Compared with CEUS
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	36,248	10.190
Gd-CEMRI	36,948	10.187	700	−0.003	Dominated

TABLE 38 - Results of scenario analysis for cirrhosis surveillance: Giorgio et al.45 used as source for accuracy data

Strategy	Cost (£)	QALYs	Compared with CEUS
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	36,034	10.189
Gd-CEMRI	37,078	10.192	1044	0.004	297,695

Probabilistic sensitivity analysis

Over 5000 replications, CEUS has the highest probability of being cost-effective for thresholds < £55,000 (Figure 13). Above this threshold, Gd-CEMRI has the highest probability of being cost-effective. At a threshold of £20,000–30,000, the probability that CEUS, CECT or Gd-CEMRI is cost-effective is 99%, 0% and 1% respectively.

FIGURE 13.

Cost-effectiveness acceptability curves: cirrhosis surveillance (effects are QALYs; both costs and effects are discounted).

Table 39 provides an overview of the results of all of the sensitivity and scenario analyses.

TABLE 39 - Overview of the sensitivity and scenario analyses for cirrhosis surveillance

Analysis	Comparator	Compared with CEUS
		Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
Base-case analysis
	CECT	379	0.000	Dominated
	Gd-CEMRI	1063	0.022	48,454
Sensitivity analyses
Imaging used as confirmatory after all positive non-enhanced US examinations	CECT	39	0.000	Dominated
	Gd-CEMRI	321	0.025	12,806
Proportion of inconclusive US scans 20% instead of 43%	CECT	176	0.000	Dominated
	Gd-CEMRI	624	0.024	16,121
Age limit for screening 90 years instead of 70 years	CECT	430	0.000	Dominated
	Gd-CEMRI	1204	0.023	51,619
Annual screening instead of every 6 months	CECT	198	0.000	Dominated
	Gd-CEMRI	594	0.016	37,619
Accuracy data for small tumours only instead of for small and medium-sized tumours	CECT	378	0.000	Dominated
	Gd-CEMRI	913	0.004	244,840
Scenario analyses
Dai et al.43 used as source for accuracy data	CECT	129	−0.004	Dominated
Quaia et al.46 used as source for accuracy data	CECT	288	−0.005	Dominated
Blondin et al.48 used as source for accuracy data	Gd-CEMRI	700	−0.003	Dominated
Giorgio et al.45 used as source for accuracy data	Gd-CEMRI	1044	0.004	297,695

Liver metastases of colorectal cancer model

Additional analyses for diagnosing liver metastases

Sensitivity analyses

When it is assumed that patients with a positive test do not undergo biopsy but are treated for their disease, implying that patients without metastases can receive unnecessary treatment, the lower specificity of CEUS leads to a loss in QALYs (Table 43). CEUS now yields the lowest number of QALYs (8.343) and is most expensive (£8335), while Gd-CEMRI, which is the most accurate, yields the highest number of QALYs (8.364) and is the least expensive (£7158). In this sensitivity analysis, Gd-CEMRI dominates the other tests because of its better accuracy.

TABLE 43 - Results of sensitivity analysis for metastases from CRC model: no biopsy if test is positive

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	8335	8.343
CECT	7321	8.359	−1015	0.016	Dominant	CEUS	−1015	0.016	Dominant
Gd-CEMRI	7158	8.364	−1177	0.021	Dominant	CECT	−162	0.005	Dominant
SPIO-CEMRI	7537	8.359	−798	0.016	Dominant	Gd-CEMRI	−379	0.005	Dominated

If CEUS is combined with biopsy (see Table 42), and CECT, Gd-CEMRI and SPIO-CEMRI are not followed by biopsy (see Table 43), then CEUS and Gd-CEMRI are most effective, both yielding 8.364 QALYS. However, CEUS is more costly than, and is thus dominated by, Gd-CEMRI. CECT and SPIO-CEMRI are now dominated by Gd-CEMRI.

If it is assumed that, instead of 40%, 80% of the initial population has metastases, the expected number of QALYs is 4.078 for all tests (Table 44). CEUS is now the least costly strategy, being £71 less costly than CECT. Because there is no difference between the tests in QALYs, the least costly test, CEUS, dominates all other tests.

TABLE 44 - Results of sensitivity analysis for metastases from CRC model: proportion of patients having metastases = 80%

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	14,419	4.078
CECT	14,490	4.078	71	0.000	Dominated	CEUS	71	0.000	Dominated
Gd-CEMRI	14,700	4.078	281	0.000	Dominated	CEUS	281	0.000	Dominated
SPIO-CEMRI	14,711	4.078	292	0.000	Dominated	CEUS	292	0.000	Dominated

Scenario analyses

We examined the expected costs and effects using different sources for the accuracy of the tests. First, we incorporated the accuracy data of Jonas et al. 50 (Table 45). This study compared CEUS, CECT and SPIO-CEMRI and found perfect specificity for all tests, with sensitivities of 87%, 83% and 97% respectively. CECT was slightly more costly (£7) and slightly less effective (0.005 QALYs) than CEUS and thus was dominated by CEUS. SPIO-CEMRI was more costly and more effective than CEUS, resulting in an ICER of £43,318 per QALY gained.

TABLE 45 - Results of scenario analysis for metastases from CRC model: Jonas et al.50 used as source for accuracy data

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	7468	8.369
CECT	7475	8.364	7	−0.005	Dominated	CEUS			Dominated
SPIO-CEMRI	8055	8.382	587	0.014	43,318	CEUS	587	0.014	43,318

The slightly lower sensitivity and specificity of CECT compared with CEUS found by Clevert et al. 51 resulted in CEUS being £300 less costly and yielding 0.002 more QALYs than CECT (Table 46).

TABLE 46 - Results of scenario analysis for metastases from CRC model: Clevert et al.51 used as source for accuracy data

Strategy	Cost (£)	QALYs	Compared with CEUS			Compared with next most cost-effective strategy
			Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)	Comparator	Incremental cost (£)	Incremental QALYs	Incremental cost per QALY (£)
CEUS	7821	8.384
CECT	8121	8.382	300	−0.002	Dominated	CEUS	300	−0.002	Dominated

Probabilistic sensitivity analyses

Based on the probabilistic sensitivity analysis with 5000 replications, we found that CEUS and CECT have a similar probability of being cost-effective across all willingness-to-pay thresholds (Figure 14). CEUS has a slightly higher probability of being cost-effective up to a threshold of £20,000, after which CECT has a somewhat higher probability of being cost-effective. At a threshold of £20,000 per QALY, CECT has the highest probability of being cost-effective (48%), followed by CEUS (47%), Gd-CEMRI (3%) and SPIO-CEMR (2%).

FIGURE 14.

Cost-effectiveness acceptability curves: liver metastases from CRC (effects are QALYs; both costs and effects are discounted).

Incidentally detected focal liver lesion model

Additional analyses

Additional analyses changed the absolute costs and effectiveness of the different strategies; however, this did not lead to any dramatic changes in the incremental costs and effectiveness of CEUS compared with CECT or CEMRI. The most critical factor in the analyses related to the costs of the tests. The impact of any other elements (e.g. prior probabilities of a particular diagnosis, costs of treatment) was minimal because the accuracies of the tests were so similar.

Sensitivity analyses

The first sensitivity analysis involved varying the prior probability of malignancy to a value much higher than that used in the base-case scenario. In this analysis, the prior probability was raised from the base-case value of 2.89% to 94% [based on the highest percentages for HCC and metastasis reported in the individual studies (58% of patients with HCC52 and 36% of patients with metastases)]. 55 Although this exceptionally high probability of malignancy was not viewed as realistic in daily practice, it was seen as a way to explore the degree of robustness of the results. As expected, the higher probability of malignancy reduced the absolute number of QALYs and increased the costs (Table 50). However, it increased the incremental QALYs only slightly and had no effect on incremental costs and therefore essentially had no effect on the cost-effectiveness of CEUS compared with CECT or CEMRI.

TABLE 50 - Results of sensitivity analysis for the incidentally detected FLL model: prior probability of malignancy increased to the maximum observed frequencies of HCC and metastasis (any type)

Comparison	QALYs	Cost (£)	Incremental QALYs	Incremental cost (£)	Incremental cost per QALY (£)
CEUS (vs CECT)	6.654	17,121	0.005	−56	Dominant
CECT	6.649	17,177
CEUS (vs CEMRI)	6.614	17,160	0.086	−202	Dominant
CEMRI	6.529	17,362

When the data source for the performance of CEUS and CECT was switched from the meta-analysis to one of the four studies used in the meta-analysis, the cost-effectiveness results changed only slightly.

We also examined the effect on the results of assuming that all patients with HCC had medium-sized lesions instead of small lesions. When we applied this in the model and also increased the risk of HCC to the highest value seen in the DTA studies (58% of patients with HCC52), there was no effect on the cost-effectiveness of CEUS compared with CECT or CEMRI.

When the consequences of an incorrectly diagnosed malignant lesion were made more severe (i.e. by reducing QALYs or increasing costs), this improved the cost-effectiveness of CEUS compared with CECT or CEMRI. For example, if an incorrect diagnosis of HCC and metastases led to a doubling of the costs (compared with the costs following a correct diagnosis) and the QALYs were set to zero, CEUS remained the dominant strategy. Table 51 shows the results of this analysis.

TABLE 51 - Results of sensitivity analysis for the incidentally detected FLL model: more severe consequences of an incorrect diagnosis of HCC and metastases

Comparison	QALYs	Cost (£)	Incremental QALYs	Incremental cost (£)	Incremental cost per QALY (£)
CEUS (vs CECT)	13.321	486	0.001	−54	Dominant
CECT	13.320	540
CEUS (vs CEMRI)	13.312	541	0.020	−162	Dominant
CEMRI	13.293	702

As expected, when an incorrect diagnosis of HCC or metastases did not result in any health or economic consequences, there was no difference in effectiveness between CEUS, CECT and CEMRI. However, because a difference in costs was still observed, this could be viewed as a situation of extended dominance in both comparisons (Table 52).

TABLE 52 - Results of sensitivity analysis for the incidentally detected FLL model: less severe consequences of an incorrect diagnosis of HCC and metastases

Comparison	QALYs	Cost (£)	Incremental QALYs	Incremental cost (£)	Incremental cost per QALY (£)
CEUS (vs CECT)	13.332	469	0.000	−52	Extended dominance
CECT	13.332	521
CEUS (vs CEMRI)	13.332	509	0.000	−130	Extended dominance
CEMRI	13.332	639

Probabilistic sensitivity analyses

Probabilistic sensitivity analyses revealed that there was no uncertainty about the cost savings of CEUS compared with CECT (mean difference −£52, 95% CI −£81 to −£22) but some uncertainty about their differences in effectiveness (mean difference 0.00014, 95% CI −0.00100 to 0.00130). Note that these CIs were based on symmetrical beta PERT distributions for the cost parameters. When the original beta PERT distributions were used, a mean difference of −£46 (95% CI −£71 to −£21) was found.

Figure 15 shows the cost-effectiveness acceptability curve comparing CEUS with CECT. This curve shows that the probability of CEUS being cost-effective compared with CECT is > 95% at willingness-to-pay thresholds of up to £20,000.

FIGURE 15.

Cost-effectiveness acceptability curve comparing CEUS with CECT: incidentally detected FLL model (effects are QALYs; both costs and effects are discounted).

When the differences in costs and effects between CEUS and CEMRI are visualised on the cost-effectiveness plane, it is clear that there is little doubt about the cost savings of CEUS compared with CEMRI, but some uncertainty about their differences in effectiveness.

The results of probabilistic sensitivity analyses comparing CEUS with CEMRI were similar to those shown above for CEUS compared with CECT. There was less certainty about the expected amount of cost savings of CEUS compared with CEMRI (mean difference −£131, 95% CI −£194 to −£69) and some uncertainty about their differences in effectiveness (mean difference 0.0039, 95% CI −0.0058 to 0.0135). Once again, these calculations were made using symmetrical beta PERT distributions for cost parameters to ensure that the point estimate for the cost difference would correspond with the point estimate based on the deterministic analysis. When the original beta PERT distributions were used, a mean difference of −£125 (95% CI −£183 to −£67) was found.

Figure 16 shows the cost-effectiveness acceptability curve comparing CEUS with CEMRI. Here we see that the probability of CEUS being cost-effective compared with CEMRI is > 95% at all willingness-to-pay thresholds between £0 and £20,000.

FIGURE 16.

Cost-effectiveness acceptability curve comparing CEUS with CEMRI: incidentally detected FLL model (effects are QALYs; both costs and effects are discounted).

Clinical effectiveness

Twenty of the 21 studies included in the systematic review were DTA studies: seven compared the performance of imaging modalities for the characterisation of FLLs detected on surveillance of cirrhosis patients using unenhanced US; four compared the performance of imaging modalities for the detection of liver metastases in patients with known primary cancers; six compared the performance of imaging modalities for the characterisation of incidentally detected FLLs identified by unenhanced US; and three compared the performance of imaging modalities for the determination of treatment response in patients with liver cancers.

The only controlled clinical trial identified indicated that the inclusion of CEUS in pretreatment imaging protocols for patients undergoing RFA for HCC may result in a reduced incidence of disease progression, new HCC and repeat RFA, and increased local progression-free and new tumour-free survival, compared with unenhanced US. However, this was a small non-randomised study that had a number of methodological weaknesses and no difference was found in the primary outcome of successful ablation. High-quality RCTs are needed to determine the relative effectiveness of different imaging strategies for treatment planning.

Test accuracy studies varied in terms of target condition (HCC, liver metastases or ‘any malignancy’), definitions of a positive imaging test used by studies of the same target condition, and lesion size assessed. Overall, there was no clear indication that any of the imaging modalities considered (CEUS, CECT or CEMRI) offered superior performance for any of the clinical indications assessed. This is consistent with two other recently published systematic reviews, which found no significant difference in the performance of CEUS, CECT and CEMRI for the characterisation of FLLs. 11,104 Neither of these two reviews reported details of the clinical application of imaging in the included studies (i.e. were FLLs incidentally detected, detected on surveillance or detected during the assessment for liver metastases in patients with known primary cancers) or of the target conditions (e.g. HCC, liver metastases or ‘any liver malignancy’), and one review104 did not specify the use of SonoVue as the contrast agent for CEUS.

The majority of included test accuracy studies were judged to be at ‘low’ or ‘unclear’ risk of bias with respect to the ‘index test’, ‘comparator test’ and ‘reference standard’ domains. ‘Unclear’ ratings for these domains most frequently arose from insufficient detail in the reporting of how tests were interpreted, particularly blinding of interpreters to other test results. Reporting quality was generally poor and a number of studies were reported only as conference abstracts, resulting in a high proportion of ‘unclear’ risk of bias ratings across QUADAS-2 domains (see Figure 7). ‘High’ risk of bias ratings for the ‘patient selection’ domain arose from the use of a retrospective study design or from inappropriate exclusions of particular patient groups (e.g. exclusion of patients with a low probability of malignancy); exclusion of patients with a low probability of disease might result in underestimations of test accuracy, although this was not apparent from the results observed. ‘High’ risk of bias ratings for the ‘flow and timing’ domain arose from exclusion of > 10% of patients from analyses or, in two cases, from incorporation of index test results in the reference standard. The last two studies were also rated as ‘high’ risk of bias for the ‘reference standard’ domain.

Test accuracy studies included in this review were grouped by clinical application: characterisation of FLLs detected on routine unenhanced US surveillance of patients with known cirrhosis, detection of liver metastases in patients with known primary tumours (CRC), characterisation of FLLs in patients with incidentally detected lesions and assessment of response in patients treated for liver malignancy.

Studies conducted in cirrhosis patients undergoing routine surveillance all concerned the differentiation of HCC from other lesion types in small to medium-sized (< 30 mm) FLLs. The definition of a positive test for HCC varied across studies. Studies assessing CEMRI used three contrast agents: gadolinium, a vascular contrast agent; SPIO, a hepatocyte-specific contrast agent which is taken up by Kupffer cells in the normal liver and benign lesions and may therefore aid identification of HCC, which are generally deficient in Kupffer cells, particularly when such lesions are hypervascular;8,9 and Gd-EOB-DTPA-CEMRI, a ‘combined’ vascular and hepatocyte-specific contrast agent. 10 There was no consistent evidence for any significant difference in test performance between the three imaging modalities and three MRI contrast media assessed. When a definition of HCC consistent with that given in the EFSUMB guidelines6 (arterial phase enhancement followed by portal venous/late phase washout) was used, estimates of the sensitivity and specificity of each of the imaging modalities assessed varied across studies. There was some evidence from one study that compared CEUS and Gd-CEMRI that these imaging techniques may be better at ruling out HCC in FLLs of between 11 and 30 mm (sensitivities for CEUS and Gd-CEMRI were 92% and 95% respectively) than in small FLLs of ≤ 10 mm (sensitivities of 27% and 73% respectively), although this study did not use an EFSUMB-consistent definition of HCC. It is therefore possible that some of the variation in sensitivity estimates seen across studies of FLLs of < 30 mm may be due to differences in the size distribution of FLLs included. There was also some evidence from two studies that combined imaging using CEUS and CECT or all three imaging modalities, in which any positive imaging result was treated as ‘test positive’, that combined imaging may increase sensitivity. Inconsistent estimates of sensitivity mean that it is unclear whether or not CEUS alone is adequate to rule out HCC for FLLs of < 30 mm in this population; CEUS alone may be adequate to rule out HCC for FLLs of 11–30 mm, with very small FLLs (< 10 mm) not considered.

Studies of the diagnosis of liver metastases using imaging with vascular contrast media (CEUS, CECT and Gd-CEMRI) in which definitions of a positive imaging test were reported gave various descriptions of peripheral rim enhancement as the criteria for liver metastases. Two studies also reported data for SPIO-CEMRI. There was no consistent evidence for any difference in test performance between the three imaging modalities and the different contrast media assessed. Per-patient sensitivity estimates from two studies were generally high [83% for all imaging modalities and both MRI contrast agents in one study of patients with CRC and > 95% for both CEUS and CECT in a second study of patients with various primary cancers (majority CRC)]. The only previous systematic review identified, which assessed SonoVue CEUS for the diagnosis of liver metastases, did not include any comparator tests and reported sensitivities for CEUS ranging from 79% to 100%. The limited data available indicate that CEUS alone may be adequate to rule out liver metastases in patients with known primary malignancies.

The primary outcome measure reported by studies conducted in patients with incidentally detected FLLs was test accuracy for the differentiation of malignant from benign liver lesions. Studies consistently used definitions of the imaging criteria for HCC and liver metastases that were similar to those reported in the EFSUMB guidelines on the use of CEUS. 6 All studies reported no significant difference in the accuracy of CEUS and CECT or CEMRI for the characterisation of FLLs. All but one study reported data for one lesion per patient and the remaining study reported data for 694 lesions in 686 patients; data were therefore treated as per patient. The pooled estimates of sensitivity for the identification of ‘any liver malignancy’ were approximately 95% for both CEUS and CECT and the pooled estimates of specificity were 94% and 93%, respectively, based on data from four studies. The single study comparing CEUS with CEMRI used Gd-CEMRI in all patients, with the addition of SPIO-CEMRI in an unspecified number of cases, and reported sensitivity estimates of 91% and 82%, respectively, and corresponding specificity estimates of 67% and 63%. Data from one study indicated that combined imaging using both CEUS and CECT, in which a positive result on either modality was treated as ‘test positive’, did not increase sensitivity. This, combined with the high estimates of sensitivity, indicates that CEUS alone may be adequate to rule out liver malignancy in this population.

Two Chinese-language studies comparing imaging modalities for the assessment of response to treatment (cryosurgery and non-surgical treatment) in patients with HCC reported per-lesion sensitivity estimates of > 95% and specificity estimates of > 80% for complete response, using CEUS, CECT and CECT or Gd-CEMRI.

These data indicate that CEUS may provide information on response in patients treated for HCC. However, these data are very limited and may not be directly applicable to UK clinical practice; further studies, ideally conducted in a UK setting, are required to confirm findings. The possibility of the rapid detection of residual tumour tissue using CEUS has the potential to allow the immediate extension of interstitial therapy; however, no data were identified on any therapeutic consequences of using CEUS to assess response to initial treatment.

Cost-effectiveness

The cost-effectiveness analysis of the use of CEUS in patients with an inconclusive unenhanced US test indicated that the use of CEUS instead of CEMRI was considered cost-effective. The use of CEUS instead of CECT was considered cost-effective in the surveillance of cirrhosis and the characterisation of incidentally detected FLLs, whereas the two techniques were similar in terms of costs and effects in the detection of liver metastases from CRC.

In the surveillance of cirrhosis, CEUS was found to be as effective as but £379 less costly than CECT. This indicates that CEUS dominates CECT. Gd-CEMRI was found to be £1063 more costly than CEUS and gained 0.022 more QALYs. This resulted in an ICER of £48,545 per QALY gained. This ICER is deemed unacceptable given the currently used willingness-to-pay thresholds of £20,000 and £30,000 per additional QALY. CEUS can therefore be considered the most cost-effective option after inconclusive unenhanced US. These base-case results were based on one source for accuracy, the study by Leoni et al. 42 Using the two other studies that compared CEUS and CECT corroborated the dominance of CEUS over CECT, showing even lower effectiveness for CECT. Compared with Gd-CEMRI, CEUS was cost-effective in most sensitivity analyses, except when all positive unenhanced US examinations were subject to confirmatory testing instead of the inconclusive US examinations, and when the proportion of patients having an inconclusive US scan was considerably lower (20% instead of 43%). These two analyses resulted in acceptable ICERs for Gd-CEMRI compared with CEUS of £12,806 and £16,121 respectively.

In the diagnosis of liver metastases from CRC, CEUS was found to have similar costs and effects to those of CECT. Using a lifetime time horizon the two techniques yielded equal QALYs per patient, with CEUS costing £1 more than CECT. Both Gd-CEMRI and SPIO-CEMRI were dominated by CECT in this population because they were more costly and equally as effective. However, in the base-case analysis it was assumed that patients who were incorrectly diagnosed with liver metastases would receive a biopsy before they were treated and that this mistake would be discovered. If this is not assumed and patients could receive unnecessary treatment, the lower specificity of CEUS had larger consequences. Under this assumption, CEUS is both the most costly and the least effective option, and Gd-CEMRI dominates all other tests. However, it is questionable whether or not this would occur in practice. If the proportion of patients having metastases were higher, CEUS would dominate the other tests. Based on the two other studies that reported accuracy data in this population,50,51 CEUS was found to dominate CECT. Gd-CEMRI yielded 0.014 more QALYs but was also £587 more costly than CEUS, resulting in an ICER of £43,318 per QALY gained. As this is above the willingness-to-pay threshold of £30,000 per QALY, Gd-CEMRI is deemed not cost-effective compared with CEUS.

The final evaluation involved the comparison of CEUS with CECT and CEMRI in the characterisation of incidentally detected FLLs. In the base-case analysis, no large differences in effectiveness were found between the three imaging strategies (incremental QALYs: CEUS vs CECT: 0.00016; CEUS vs CEMRI: 0.0026). However, a difference in costs was found (CEUS vs CECT: −£52; CEUS vs CEMRI: −£131) and this resulted in a situation of dominance. Probabilistic sensitivity analysis revealed that there was little uncertainty about the cost-effectiveness of CEUS compared with the other two tests. Additional analyses changed the absolute costs and effectiveness of the different strategies but did not lead to dramatic changes in the incremental costs and effectiveness of CEUS compared with CECT or CEMRI. One critical factor in the analyses related to the costs of the tests. This could mean that local conditions may play a role in deciding which test is preferable, assuming that the costs of these tests can be influenced by local conditions.

Clinical effectiveness

Extensive literature searches were conducted in an attempt to maximise retrieval of relevant studies. These included electronic searches of a variety of bibliographic databases, as well as screening of clinical trials registers and conference abstracts to identify unpublished studies. Because of the known difficulties in identifying test accuracy studies using study design-related search terms,20 search strategies were developed to maximise sensitivity at the expense of reduced specificity. Thus, large numbers of citations were identified and screened, many of which did not meet the inclusion criteria of the review.

The possibility of publication bias remains a potential problem for all systematic reviews. Considerations may differ for systematic reviews of test accuracy studies. It is relatively simple to define a positive result for studies of treatment, for example a significant difference between the treatment and control groups that favours treatment. This is not the case for test accuracy studies, which measure agreement between index test and reference standard. It would seem likely that studies finding greater agreement (high estimates of sensitivity and specificity) will be published more often. In addition, test accuracy data are often collected as part of routine clinical practice, or by retrospective review of records; test accuracy studies are not subject to the formal registration procedures applied to RCTs and are therefore more easily discarded when results appear unfavourable. The extent to which publication bias occurs in studies of test accuracy remains unclear; however, simulation studies have indicated that the effect of publication bias on meta-analytic estimates of test accuracy is minimal. 105 Formal assessment of publication bias in systematic reviews of test accuracy studies remains problematic and reliability is limited. 105 We did not undertake a statistical assessment of publication bias in this review; however, our search strategy included a variety of routes to identify unpublished studies and resulted in the inclusion of a number of conference abstracts.

Clear inclusion criteria were specified in the protocol for this review and the one protocol modification that occurred during the assessment has been documented in Chapter 3 of this report (see Inclusion and exclusion criteria). The eligibility of studies for inclusion is therefore transparent. In addition, we have provided specific reasons for excluding all of the studies considered potentially relevant at initial citation screening (see Appendix 5). The review process followed recommended methods to minimise the potential for error and/or bias. 16 Studies were independently screened for inclusion by two reviewers and data extraction and quality assessment were carried out by one reviewer and checked by a second (MW and VG). Any disagreements were resolved by consensus. Chinese-language studies were extracted by one reviewer (MW) working with a native speaker (KL) and the only German-language study was extracted by one reviewer (VG).

With one exception, all studies included in the review were test accuracy studies. The methodological quality of these studies was assessed using a modification of the QUADAS-2 tool. 25 The QUADAS tool has been recommended for assessing the methodological quality of test accuracy studies16,18 and has been widely adopted by researchers and key organisations such as The Cochrane Collaboration, NICE and the Institut für Qualität and Wirtschaftlichkeit im Gesundheitswesen (IQWiG) in Germany. It has been mentioned in more than 200 abstracts in the DARE database and has been cited more than 500 times. The revised version of QUADAS (QUADAS-2) has recently been published. 25 QUADAS-2 more closely resembles the approach and structure of the Cochrane risk of bias tool. It is structured into four key domains covering participant selection, index test, reference standard and the flow of patients through the study (including the timing of tests). Each domain is rated for risk of bias (low, high or unclear) and the tool provides signalling questions in each domain to help reviewers in reaching a judgement. The participant selection, index test and reference standard domains are also separately rated for concerns regarding the applicability of the study to the review question (low, high or unclear). However, the QUADAS-2 tool does not currently include domains specific to the assessment of studies comparing multiple index tests; further development of QUADAS-2 in this area is planned. This assessment used a modified version of the QUADAS-2 tool that includes an additional domain for the comparator test and additional signalling questions in the flow and timing domain. It should be noted, however, that these components of the tool were not developed using the same rigorous evidence-based approach as for the core QUADAS-2 tool. The inclusion criteria for this review were considered to largely match the review question and questions of applicability were therefore relevant only to the patient selection domain. The review-specific guidance used in our QUADAS-2 assessment is reported in Appendix 2. The results of the risk of bias assessment are reported in full for all included studies in Appendix 3 and in summary in Chapter 3 (see Results of the assessment of clinical effectiveness). However, the usefulness of this assessment was limited by poor reporting of primary study methods, particularly with respect to how the index and comparator tests and the reference standard were applied. This issue was exacerbated because four of the 20 test accuracy studies (20%) were reported only as conference abstracts.

The systematic review conducted for this assessment represents an improvement on previously published systematic reviews11,104,106 in that it focuses on studies that directly compared the performance of SonoVue CEUS with at least one other imaging modality, as well as clearly distinguishing between both the clinical application and the target condition of imaging.

Hierarchical or bivariate models are considered the optimal methods for estimating SROC curves and pooled estimates of sensitivity and specificity. 16,27 The bivariate model analyses sensitivity and specificity jointly, retaining the paired nature of the original data, and has been shown to produce equivalent results to the hierarchical SROC model in the absence of other study-level covariates. 28 However, the fitting of this model requires a minimum of four data sets. There was only one group of four studies in this assessment for which meta-analytic pooling was considered potentially appropriate (similar clinical application, target condition and comparator test). One of these studies used a suboptimal reference standard and a sensitivity analysis was used to investigate the influence of this study upon the overall estimate of test performance, reducing the data set to three studies; for this reason, a random-effects model was used to generate pooled estimates of sensitivity and specificity, with 95% CIs.

In addition to the limited potential for meta-analysis and the general methodological quality issues outlined above, there were a number of reporting/methodological problems specific to this review. Of particular concern for this assessment was the way in which data were reported in terms of the unit of analysis. The main reason for undertaking liver imaging in the populations considered is likely to be to rule out primary liver cancer or liver metastases. Therefore, patient-level analyses of test performance are of particular interest. Some of the studies included in this review reported per-patient analyses; however, no study clearly stated how per-patient test results were defined (e.g. was the presence of any positive lesion regarded as a positive test for the whole patient). Some of the included studies reported per-lesion data (multiple lesions per patient). These type of within-patient ‘clustered’ data are a common feature of test accuracy studies and are likely to result in a correlation between results within each patient, which should be accounted for in any statistical analysis. 107 Uncorrected estimates of sensitivity and specificity derived from such data are likely to be accurate, but imprecision will be underestimated. 107 Of greater concern are those studies that reported data for one lesion per patient (treated as per-patient data in this assessment) but in which multiple lesions per patient were present, as was the case for the majority of studies evaluating SonoVue CEUS for the characterisation of incidentally detected FLLs. 52,54–56 These studies generally selected the largest lesion or the lesion ‘most suspicious for malignancy’ for inclusion in analyses, with the result that estimates of test performance may have been exaggerated. It might be argued that, when considering the ability of a test to rule out malignancy, performance for the characterisation of smaller ambiguous lesions is an important consideration.

All assessments of diagnostic accuracy are underpinned by the assumption that the reference standard, against which the index and comparator tests are evaluated, is 100% sensitive and 100% specific. The inclusion criteria specified by the protocol for this assessment allowed the use of different reference standards for test-positive and test-negative patients (histology and clinical follow-up respectively). This approach was used because it may be considered unethical to perform biopsy of test-negative patients or lesions. However, delayed verification, as represented by clinical follow-up, is inherently flawed in that follow-up must be of sufficient duration for any false-positive or false-negative test results to become apparent, but prolonged follow-up may also result in changes in disease state and hence misclassification of test results. In addition, a protocol modification allowed the inclusion of studies on the characterisation of FLLs (suspected HCC) that used EASL/AASLD non-invasive diagnostic criteria (two concordant imaging test results) as the reference standard. Two additional studies were included in the review as a result of this protocol modification. 42,41 Studies using this type of reference standard may be subject to incorporation bias. However, the implications of this are unclear; the review of sources of variation and bias in test accuracy studies, conducted as part of the development of QUADAS, found no evidence on the effects of incorporation bias,23 and the update of this review, conducted during the development of QUADAS-2, found two contradictory studies, one reporting no effect of incorporation bias on accuracy and one reporting increased sensitivity and reduced specificity in the presence of incorporation bias (unpublished data).

The clinical applicability of accuracy data included in this review may have some limitations. The inclusion criteria for this assessment specified that SonoVue CEUS should be used for the characterisation of FLLs when unenhanced US examination was considered inconclusive. Although all study participants had imaging-detected FLLs before undergoing SonoVue CEUS, only one study43 explicitly stated that unenhanced US was inconclusive. Perhaps more importantly, the prevalence of malignancy appeared high in studies assessing the accuracy of CEUS and other imaging modalities for the characterisation of incidentally detected FLLs; these study populations may not be representative of the population with incidental FLLs seen in clinical practice. When any information on the interpretation of CEUS was reported (see Appendix 4), the majority of DTA studies included in this review reported consensus interpretation by multiple experienced operators (experience ranging from > 2 years to 20 years). It should be noted that operator training and experience may have important effects on the diagnostic performance of CEUS, but insufficient data are currently available to explore these potential effects.

The majority of included studies reported no information on funding; two studies reported funding from the manufacturer of SonoVue. 55,56

Cost-effectiveness

In this study we built three separate models for the three different potential uses of CEUS: surveillance of cirrhosis, detection of liver metastases from CRC and characterisation of incidentally detected FLLs. All three models were based on existing models that had previously informed NICE guidance. 70,71 When necessary we updated and improved these models. The model for incidentally detected FLLs was a combination of the two updated and improved models.

In each of the three analyses we used evidence to inform parameters that was relevant for the UK and as up-to-date and of as high a quality as possible. When evidence was not available from published studies or databases, we used the most likely and plausible ranges based on expert opinion.

As expected, the main driver of the models was the accuracy of the different tests. There was only one group of four studies in this assessment for which meta-analytic pooling was considered potentially appropriate (similar clinical application, target condition and comparator test): the use of CEUS to characterise incidentally detected FLLs. As a consequence, the estimated cost-effectiveness of CEUS for the surveillance of cirrhosis and the diagnosis of liver metastases from CRC had to be based on single studies. Scenario analyses were performed using other available studies and these analyses showed that in general the source for accuracy influences the costs and effects of the different tests. However, the use of different sources resulted in similar conclusions. CEUS was found to be the most cost-effective test for the surveillance of cirrhosis, and the two alternative sources for the liver metastases model produced favourable results for CEUS.

In general, the studies used to estimate test accuracy appeared to involve different types of patient populations. The studies used for the incidentally detected FLL model, for example, defined incidentally detected FLLs in different ways. Interestingly, regardless of the variation in composition of the patient populations, there was never an instance when the test accuracy results of CEUS and CECT were very different. All studies concluded that the two tests were comparable in performance.

Another main driver was the clinical pathway of incorrectly diagnosed patients. Although the pathway may be straightforward for false-negative patients, as their disease may be correctly diagnosed at a later stage of the initial workup, it is not as clear for false-positive patients. In the liver metastases from CRC model we assumed that patients who are inaccurately diagnosed as having metastases would receive biopsy before treatment. This implies that patients were not unnecessarily treated. However, it is unclear what happens to these patients in practice. Therefore, we performed a sensitivity analysis in which patients without metastases were treated if they were incorrectly diagnosed. In this sensitivity analysis CEUS was found to be the least effective and most costly option. Although we do not expect it to be realistic that patients without metastases will actually receive treatment, it is important to note this factor.

Besides being less costly, CEUS has the advantage compared with CECT and especially CEMRI that it is highly accessible. All patients already receive an unenhanced US examination and can be immediately diagnosed using CEUS as part of the same examination. A potential benefit of CEUS is therefore the reduction in anxiety in patients because a malignant lesion is ruled out sooner as a result of not having to wait too long for another test. This benefit was not taken into account in the analysis as little evidence is available on the effect of anxiety on quality of life. It might be expected that the effects of using CEUS are therefore underestimated. Although the length of wait associated with other imaging modalities is uncertain, the consideration of this anxiety factor would only further support the use of CEUS over CECT or CEMRI.

Clinical effectiveness

None of the clinical applications of liver imaging considered in this review was evaluated by a large number of studies; the maximum was seven studies on the performance of imaging modalities for the characterisation of FLLs detected on surveillance of cirrhosis patients using unenhanced US. Although this review benefits from focusing on studies that directly compared the performance of SonoVue CEUS with that of other imaging modalities, as noted above in the strengths and limitations section, only two studies on the characterisation of FLLs detected on surveillance of cirrhosis patients42,47 and two studies on the detection of liver metastases in patients with known primary cancers49,50 compared all three imaging modalities under assessment (CEUS, CECT and CEMRI). Most studies that assessed CEMRI used gadolinium-based vascular contrast agent, which has a comparable mode of operation with that of CEUS and CECT. However, CEMRI of the liver can also be conducted using hepatocyte-specific contrast agents such as SPIO or ‘combined’ vascular and hepatocyte-specific agents such as Gd-EOB-DTPA; only four of the studies included in our systematic review reported data for these types of contrast agent. 42,49,48,50 Studies were generally small (15 of the 20 DTA studies included fewer than 100 participants) and, within clinical applications, studies varied in terms of target condition (HCC, liver metastases or ‘any malignancy’), definitions of a positive imaging test used by studies of the same target condition, comparator imaging technologies and lesion size assessed. In addition, four of the 20 test accuracy studies were reported only as conference abstracts,39–41,50 which further limited the available data. These factors meant that, as detailed above in the statement of principal findings for the clinical effectiveness assessment, only one meta-analysis was undertaken (studies comparing CEUS with CECT for the characterisation of incidentally detected FLLs). Based on the available data, SonoVue CEUS appeared to offer similar diagnostic performance to that of other imaging modalities (CECT and CEMRI) for all clinical applications considered, but data were generally insufficient to support firm conclusions.

SonoVue CEUS is generally used for the characterisation or detection of liver lesions in patients for whom unenhanced US examination has proved inconclusive. In addition to test accuracy, it is therefore particularly important to assess the proportion of patients in whom US examination remains inconclusive even after contrast-enhancement compared with the proportion in whom comparator imaging technologies are inconclusive. Four of the 20 DTA studies included in this review explicitly reported the numbers of participants in whom imaging was inconclusive; three studies indicated that SonoVue CEUS was inconclusive in slightly fewer patients than CECT (0%,51 3%46 and 3%54 for SonoVue CEUS compared with 14%,51 8%46 and 6%54 for CECT). One study reported 11% inconclusive imaging results for both SonoVue CEUS and CEMRI. 56 Although not explicitly stated, all other included studies appeared to report complete data sets and hence may be inferred to have had no inconclusive imaging examinations.

When diagnostic accuracy is comparable across imaging modalities, comparison of adverse event rates associated with the different imaging options, as well as consideration of patients' preferences, are also of particular importance. Only one of the DTA studies included in this review reported any information on adverse events related to testing; the authors of this study stated that there were no adverse events associated with SonoVue CEUS but did not report any information about the comparator technology Gd-CEMRI. 45 A large, retrospective safety study of SonoVue CEUS in abdominal applications, which did not meet the inclusion criteria for this review, reported data from 23,188 investigations in 29 centres in Italy. 7 This study found 29 cases of adverse events, of which two were graded as serious, one as severe, three as moderate and 23 as mild. There were no fatal adverse events. One of the serious adverse events occurred in a patient with prostate cancer who was being investigated to characterise a liver lesion suspected of metastases; this patient complained of dyspnoea with signs of bronchoplasm, slight hypotension and bradycardia within 1 minute after injection of SonoVue. The majority of non-serious adverse events resolved without intervention and included itching, mild dizziness, moderate hypotension, headache, sensation of warmth and nausea and vomiting. None of the studies identified reported any information on patient preferences.

Acceptability to patients and the potential for reduced anxiety provided by the ability to conduct CEUS at the same appointment as the initial US examination are also likely to be important factors in the choice of imaging modality; however, no studies were identified that reported these outcome measures.

It should be further noted that, although this review provides some evidence on the accuracy of SonoVue CEUS for the characterisation of FLLs and the detection of liver metastases and response to treatment of liver cancers, only one study59 was identified that reported the effects of imaging with SonoVue on patient outcomes; the ultimate aim of any research on clinical tests should be to determine impact on patient management and clinical outcomes. As described earlier in the statement of principal findings for the clinical effectiveness assessment, this study indicated that the inclusion of CEUS in pretreatment imaging protocols for patients undergoing RFA for HCC may result in some improved outcomes compared with unenhanced US. Overall, the effects, if any, of imaging with SonoVue CEUS on management and outcome of patients with FLLs remain uncertain.

Cost-effectiveness

Many studies emphasised that the participating clinicians had years of experience in the use of CEUS. It is possible that the diagnostic accuracy of CEUS may be poorer if the user has little experience. However, widespread implementation of CEUS might also improve the experience with CEUS and ultimately improve accuracy.

The main uncertainty surrounding the cost-effectiveness of CEUS is how patients who are incorrectly diagnosed are managed. Arguably, this is very different across locations. In the cirrhosis surveillance model, patients are screened twice a year and it is expected that a lesion, although it may have grown and therefore be potentially less treatable, will be detected eventually. In the liver metastases from CRC model, patients with metastases will have associated symptoms and it is therefore justifiable to assume that metastases will be detected within a year. Patients with incidentally detected lesions also often have associated risk factors or evidence of liver disease, which may have been the indication for initial testing with unenhanced US or which may have been identified at this examination; hence, it is expected that their complaints will worsen and that their lesions will be detected within several months. How patients with a false-positive test result are managed might be more complex. We assumed that, in all models, these patients would receive additional costs of unnecessary additional diagnostic tests but would not undergo inappropriate treatment as the correct diagnosis would be determined after additional diagnostic workup. In the liver metastases of CRC model we examined the extreme situation in which all patients who were incorrectly diagnosed with metastases would receive treatment for these metastases. As this involves the costs of the treatment as well as reduced quality of life, this has a considerable impact on the results.

In the cirrhosis surveillance model, the actual use of CEUS impacted the results. If CEUS were used after all positive instead of after all inconclusive unenhanced US examinations, or if the proportion of inconclusive unenhanced US tests were lower, Gd-CEMRI would be cost-effective compared with CEUS.