Imaging tests for the detection of osteomyelitis: a systematic review

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 16/103/03. The contractual start date was in July 2017. The draft report began editorial review in October 2018 and was accepted for publication in March 2019. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Copyright statement

© Queen’s Printer and Controller of HMSO 2019. This work was produced by Llewellyn et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2019 Queen’s Printer and Controller of HMSO

Osteomyelitis

Osteomyelitis is an infection of the bone and bone marrow. 1,2 Left untreated, it may result in bone infarction and loss of limb or joint function and, in extreme cases may require amputation of the affected limb. If the infection spreads, it may lead to potentially fatal septicaemia. 3 In children, osteomyelitis may also inhibit limb growth, requiring extensive orthopaedic intervention in later childhood. Staphylococcus aureus is the most common organism causing osteomyelitis, but other common organisms such as Streptococcus spp. or Escherichia coli may also be responsible in some cases. Bone infections occur most commonly in people aged < 20 years or > 50 years. It accounts for around 1% of all childhood hospital admissions. The incidence of osteomyelitis has increased over recent decades, notably in children and in patients > 60 years of age. This growing incidence has been associated with increased prevalence of meticillin-resistant S. aureus (MRSA) in children and an increase in diabetes mellitus-related infections in adults. 4

Osteomyelitis may be acute, subacute or chronic and is divided between haematogenous osteomyelitis, in which infection transfers from a remote location in the body via the bloodstream, and contiguous osteomyelitis, in which infected material comes into direct contact with the bone. 5 The haematogenous type is more common in children whereas the contiguous type is more common in adults, usually as a result of trauma or surgery. 6 Osteomyelitis is also common in people with vascular deficiency, such as adults with diabetes, as a complication of diabetic foot ulcers. 7 Osteomyelitis may lead to infection of the adjacent joint (septic arthritis) or occur secondary to septic arthritis by contiguous spread.

Patients usually present with a range of symptoms including swelling, joint pain and fever. These symptoms are often not specific to osteomyelitis, leading to delays in correct diagnosis. Blood tests are used initially to assess inflammatory markers indicative of infection in the body, including white blood cell (WBC) count, C-reactive protein (CRP) levels and erythrocyte sedimentation rate (ESR). 8 When these tests show evidence of possible infection, patients are referred for further diagnostic testing. The most accurate diagnostic tool is a bone biopsy or aspiration of a pus collection from the bone or tissue surrounding the bone, with a microbiological assessment of the sample to identify the organism causing the infection. Biopsies are invasive and generally require a local or a general anaesthetic. The analysis of the results may take several days. Alternative diagnostic tools include blood or tissue cultures, which may be less accurate but are useful in identifying the organism causing an infection in the body, which enables the selection of the appropriate antibiotic for treatment. The primary treatment for osteomyelitis is a course of antibiotics, but surgery may also be used. 9

Diagnostic imaging for osteomyelitis

Diagnostic imaging of the affected area before performing a biopsy may help improve diagnosis and avoid unnecessary biopsies in people who may have an infection but are unlikely to have osteomyelitis. It can also be useful to identify pus collections, assess the need for drainage procedures and establish the best way for surgical access.

A range of diagnostic imaging methods are available, including radiography, magnetic resonance imaging (MRI), computed tomography (CT), scintigraphy, positron emission tomography (PET), single-photon emission computed tomography (SPECT) and ultrasound. 9–12 These imaging methods each have their advantages and disadvantages.

Radiography are easily available and cheap to perform, but are poor at detecting osteomyelitis in its early stages. Radiography may be most useful in identifying and ruling out other causes of the patient’s symptoms, such as bone fractures. 11 MRI is probably the most widely recommended and used technique. It is more accurate than radiography and able to detect osteomyelitis in its early stages, but is more expensive to perform12 and when used in children necessitates the use of sedation or general anaesthesia. PET and bone scintigraphy are more expensive and less widely available than MRI or radiography. 11,13 These methods expose patients to ionising radiation. Ultrasound avoids the radiation exposure and is readily available, but its diagnostic accuracy is currently uncertain. 12 There is also a distinction between methods that provide two-dimensional images (radiography, scintigraphy) and those producing three-dimensional images (PET, MRI, CT, SPECT). Some tests (e.g. MRI) may be less suited to patients with hip replacements or other indwelling metalwork because the metalwork can alter the reliability of the imaging.

Current diagnostic and treatment practice

Once osteomyelitis is suspected on the basis of physical examination and blood tests, MRI is currently generally recommended as the imaging test of choice because it can detect osteomyelitis early and it can identify pus collections within bone that might require surgical drainage. Radiography is not usually recommended in isolation, because of their failure to detect early osteomyelitis, but are generally used as a first-line investigation to rule out or confirm bone fractures or other causes of symptoms. 12 CT, scintigraphy and PET are less widely recommended, but are an alternative for patients in whom MRI is not possible.

Ultrasonography is suggested as an alternative to radiological tests7,9,11,12,14 and is widely used in paediatric practice to exclude joint effusions and pus collection next to bone. 14 This is especially helpful in young children (aged < 6 years), who would require a general anaesthetic for MRI. Ultrasound is also used to guide aspiration and biopsy.

Little formal guidance [such as guidelines produced by the National Institute for Health and Care Excellence (NICE)] exists about which imaging techniques to use to diagnose osteomyelitis. The only current NICE guidance is for the treatment of diabetic foot ulcers. 1 In those patients, radiography is recommended to either confirm advanced osteomyelitis or confirm that the symptoms are due to other causes (e.g. broken bones). Radiography is followed by MRI if osteomyelitis is suspected but not confirmed by radiography. In children, ultrasonography is sometimes used in place of MRI. Antigranulocyte Fab fragment antibody scintigraphy should not be used in patients with diabetic foot ulcers. 15 Recommendations about its use have also been published in the USA. 16,17

Osteomyelitis is treated with a 4- to 6-week course of antibiotics. 18,19 Treatment is initially intravenous, switching to oral antibiotics after around 2 weeks. The choice of antibiotics will depend on the infecting organism, as determined by tests such as microbiological culture and the patient’s medical history. Surgery may also be used for debridement of necrotic tissue and affected bone, to drain pus and to reduce bacterial load.

Pathway to diagnosis in the NHS

There are a number of ways in which a patient might be referred for imaging to diagnose osteomyelitis. Patients may present with fever and be admitted as inpatients, or may be referred directly by their general practitioner (GP) to an orthopaedic clinic. This pathway to clinic is slower than presenting directly to accident and emergency and such patients often have less virulent infection or subacute osteomyelitis. Patients may be referred from other hospitals, particularly those that lack the facilities to treat children (e.g. if the hospital does not offer MRI under general anaesthesia). Patients presenting with acute symptoms may have a musculoskeletal issue (often limping or joint pains) or non-specific systemic symptoms and sepsis (e.g. immunodeficient patients as a result of underlying chronic condition). Generally unwell patients with sepsis are more difficult to diagnose because they might be in intensive care and joint symptoms could initially be missed while the focus is on treating severe symptoms.

The range of symptoms and possible causes of these symptoms mean that osteomyelitis may not be suspected at first. Patients may undergo one or more radiographic assessments (and ultrasound scans in children) and repeated blood tests prior to final diagnosis. Patients may also have received a course of antibiotics before diagnosis, with osteomyelitis suspected only because that treatment course was not successful. This complicates the diagnostic process, and the practical pathway to most diagnoses of osteomyelitis in many patients will differ from that used in formal diagnostic accuracy studies.

Existing review evidence

Several systematic reviews or meta-analyses have been performed to assess diagnostic imaging techniques for osteomyelitis. 20–30

Four of these reviews considered primarily, or only, people with diabetic foot ulcers. 20,22,23,26 Their conclusions varied, depending on the tests included, but MRI, PET and WBC scintigraphy were all suggested as suitable imaging tests. Three reviews of osteomyelitis in the general population mostly recommended PET and SPECT as having the best diagnostic accuracy. 25,27,28 One review focused on patients with peripheral post-traumatic osteomyelitis and concluded that WBC scintigraphy with SPECT/CT or ¹⁸F-FDG (fludeoxyglucose)-PET/CT had the best diagnostic accuracy in this population. 21 Another review of MRI in patients with pressure ulcers was inconclusive as a result of insufficient evidence. 29 Two reviews were conducted in children. One focused on calcaneal osteomyelitis and was inconclusive because of the limited evidence. 24 The other children’s review focused on haematogenous acute and subacute paediatric osteomyelitis, and found that MRI had the highest sensitivity and specificity compared with radiography, scintigraphy, CT and ultrasound. 30

Literature searches

The search strategy was developed by an information specialist with input from the review team and clinical advisors. The strategy was developed in MEDLINE (via Ovid) and included search terms for osteomyelitis and relevant diagnostic imaging techniques. No language, date, geographical or study design limits were applied. The MEDLINE strategy was adapted for use in the other resources searched.

The searches were carried out during August 2017 and updated in July 2018 to capture more recent studies. The following databases were searched: MEDLINE (including Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE Daily and Ovid MEDLINE), Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Database of Systematic Reviews (CDSR), Cumulative Index to Nursing and Allied Health (CINAHL) Plus, Database of Abstracts of Reviews of Effects (DARE), EMBASE, Health Technology Assessment (HTA) database and PubMed.

In addition, ClinicalTrials.gov and PROSPERO were searched for ongoing and unpublished studies. Relevant guidelines were identified through searches of the National Guidelines Clearing House, NHS Evidence, the NICE website and the Trip database. The reference lists of relevant systematic reviews were manually checked to ensure that all relevant studies from previous reviews were identified.

The search results were imported into EndNote X8 [Clarivate Analytics (formerly Thomson Reuters), Philadelphia, PA, USA] and de-duplicated. The complete search strategies can be found in Appendix 1.

Study selection

Titles and abstracts and the full texts of studies were independently assessed for inclusion by two reviewers using the inclusion criteria outlined in this section. Disagreements were resolved through discussion and, where necessary, consultation with a third reviewer. Study selection was performed using EPPI-Reviewer 4 software (Evidence for Policy and Practice Information and Co-ordinating Centre, University of London, London, UK).

Data extraction

A mapping exercise informed the development of the data extraction form, which was then piloted on a small selection of studies by two reviewers. Data extracted included details of patient characteristics, diagnostic tests and reference standard tests. Data were extracted by one reviewer and independently checked for accuracy by at least one other reviewer. Discrepancies were resolved by discussion, with involvement of a third reviewer when necessary. Given the high number of studies, authors were not contacted if relevant data appeared to be unreported. In cases of multiple reports for a given study, the most recent or most complete report was used as the main source of data.

Study characteristics were extracted, including design, year, country and patient eligibility criteria. Patient characteristics that were extracted included age, comorbidities, diabetic status, location of osteomyelitis and reason for referral. Data on study intervention (e.g. characteristics of imaging test used, radioisotope, contrast agent, diagnostic cut-off point and thresholds), unit of analysis (e.g. patient, body part) and data on exclusions from study/analysis with reasons were recorded. Types of reference standards used for confirming positive and negative cases were recorded. The numbers of patients confirmed to be positive or negative in accordance with the reference standard, and the numbers of true-positive, true-negative, false-positive and false-negative test results, were extracted, if reported. If not reported, sensitivity and specificity estimates [with their 95% confidence intervals (CIs)] or other reported diagnostic accuracy data were extracted. Where possible, sensitivity, specificity, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated and checked against reported values in the publications to check for any discrepancies. When more than one test was performed for the same participant in the same body part (i.e. repeat test, or follow-up test), only the result of the first test was used.

Inter-rater reliability estimates were extracted from the papers and tabulated. For the implementation review, relevant results on cost-effectiveness or use of machinery or data from surveys of clinicians were extracted and summarised narratively.

Quality assessment

The quality of the included diagnostic accuracy studies was assessed using the QUADAS-2 [quality assessment of diagnostic accuracy studies (version 2)] tool designed for diagnostic accuracy studies. 31 Critical appraisal was performed by one reviewer alongside data extraction and independently checked by at least one other reviewer. The QUADAS-2 tool was adapted to ensure that it is applicable to assessing the quality of studies of imaging tests for detecting osteomyelitis. The tool consists of four key domains: (1) patient selection, (2) index test, (3) reference standard and (4) flow of patients through the study and timing of the index test(s) and reference standard. Each domain was assessed in terms of the risk of bias. The first three domains were also assessed for concerns regarding their applicability, that is, whether or not the participants and setting, the index test, its conduct or interpretation and the target condition (as defined by the reference standard) were applicable to the review question. Further details on the critical appraisal tool, including signalling questions to inform the assessment of the key domains, are reported in Appendix 2.

No validated instrument is available for appraising the quality of studies on inter-rater reliability. We used a modified version of the tool reported by van de Pol et al. 32

Synthesis

Role of patient and clinical advisors

Clinical advisors for this project attended meetings (in person or remotely) over the course of the project to ensure that it met clinical needs. This included approving the protocol, discussing and commenting on the provisional results and meta-analyses, and commenting on this report.

Two patient representatives, both parents of children who had been treated for osteomyelitis, were contacted through telephone meetings over the course of the project to discuss their experience of imaging tests and to discuss the findings of the project.

Quantity and quality of research available

Included studies

Figure 1 is a flow diagram outlining the screening process with reasons for exclusion of full-text papers. The literature searches of bibliographic databases identified 18,386 unique references. After initial screening of titles and abstracts, 597 were considered to be potentially relevant and were ordered for full-paper screening. In total, 77 studies37–113 (from 79 reports) were included in the diagnostic review, of which 69 were also included in a meta-analysis. 38–42,44–51,53–71,73–77,79–95,97,99–113 Eleven studies were included in the review of inter-rater agreement,37,58,59,65,68,86,92,114–117 and one was included in the review of implementation. 118 Seven studies were included in both the diagnostic and inter-rater reliability reviews. 37,58,59,65,68,86,92

FIGURE 1.

A PRISMA flow diagram showing study selection.

Assessment of diagnostic accuracy

Synthesis of diagnostic accuracy in adults

In this section we consider the synthesis of diagnostic accuracy for studies of imaging tests in adults. Studies exclusively in children are considered in Synthesis of studies in children. Studies in mixed adult/children groups or where the population was unclear are included in the analysis of adults.

Excluded from this analysis are studies where the combined diagnostic accuracy of two or more imaging tests was reported (e.g. the accuracy of scintigraphy in combination with SPECT); these are discussed in Studies reporting on combinations of imaging tests. Also excluded are studies that did not report sufficient data to calculate 2 × 2 tables of diagnostic accuracy; these are discussed in Studies not included in the quantitative synthesis.

This analysis included 69 diagnostic accuracy studies. 38–42,44–51,53–71,73–77,79–95,97,99–113 Appendix 4, Table 19, summarises these studies. Many of these studies compare two or more imaging tests. Comparisons between tests are discussed in Comparisons between tests. Many studies compare imaging tests with scintigraphy. Scintigraphy was not specified as a test of interest in the protocol, but it is included here because of the number of studies considering it. The main analysis only considers scintigraphy in studies where it was compared with other tests; for studies of scintigraphy alone, see Studies of scintigraphy.

Some studies reported diagnostic accuracy at multiple thresholds of a single test, or for a single test under different conditions. Where that was the case, the main analysis uses only the result with the highest DOR, as an indicator of the ‘best’ diagnostic accuracy for that test in that study. Where appropriate, data at multiple thresholds are considered in the sections on individual tests (see Synthesis of specific imaging tests).

Figure 2 summarises the sensitivity (proportion of people/scans with osteomyelitis correctly diagnosed) and specificity (proportion of people/scans without osteomyelitis correctly diagnosed) for each test in each study in a ROC plot. This figure shows considerable variation in the diagnostic accuracy both within and between tests. MRI and SPECT both generally have high sensitivity (> 80%), but with a wide range of specificities. PET is generally more consistent with high sensitivity and specificity, but with some studies having lower sensitivity. Scintigraphy results vary widely in both sensitivity and specificity. Radiographic results generally have poor sensitivity.

FIGURE 2.

Summary ROC plot of diagnostic accuracy in adults.

In Appendix 4, Figure 21 shows a similar plot for PPVs and NPVs. These results are also variable and difficult to interpret. MRI, SPECT and PET generally have high PPV and NPV, but with many outliers. Radiography generally has a poorer NPV.

We now consider the bivariate meta-analysis of sensitivity and specificity. Figure 3 shows the summary bivariate meta-analysis results for each test (with 95% CIs for sensitivity and specificity). Figure 4 shows the summary HSROC curves for each test.

FIGURE 3.

Bivariate meta-analysis of sensitivity and specificity: all adult studies.

FIGURE 4.

Summary HSROC curves: all adult studies.

These results show that sensitivity and specificity are highest for SPECT, but with a wide CI for specificity. MRI and PET also have high sensitivity and specificity. CT, radiography and scintigraphy all have poorer sensitivity and/or specificity. The summary HSROC curves for MRI, SPECT and PET are very similar and overlapping, suggesting that they have similar diagnostic accuracy. The curves for CT, radiography and scintigraphy are lower, showing poorer diagnostic accuracy.

Appendix 4, Figure 22, gives the results of the bivariate meta-analysis of PPV and NPV, which shows that MRI, PET and SPECT have higher PPV (88–93%) and NPV (> 85%) than MRI and PET, with SPECT having the highest NPV (97%). Scintigraphy and radiography have lower PPV and NPV.

Figure 5 shows forest plots of sensitivity and specificity for MRI. These highlight the generally high sensitivity and specificity of MRI, and the overall consistency of sensitivity across studies. As noted earlier, the specificity of MRI varies substantially across studies, although many studies are small with correspondingly wide CIs. This raises concerns as to the consistency of MRI performance across studies. Given the large number of tests, and hence forest plots, no further forest plots are presented in this report, but they are available on request from the authors.

FIGURE 5.

Forest plots of sensitivity and specificity for MRI.

Table 4 shows the results of univariate meta-analyses of sensitivity and specificity (i.e. meta-analysed separately rather than jointly) and also meta-analyses of DOR (to summarise overall diagnostic accuracy) and the PR (proportion of people testing positive for osteomyelitis based on the imaging test result alone).

The results for sensitivity and specificity match those of the bivariate analysis in Figure 3, as would be expected. The I² values for sensitivity and specificity are low or zero, which conflicts with the apparent heterogeneity observed in Figure 2. This may be because most studies are small, and so there is considerable within-study variability in estimates, which reduces I², rather than an absence of heterogeneity.

Diagnostic odds ratios are similar for MRI, SPECT and PET, suggesting similar diagnostic accuracy for these tests, as seen in Figure 4. The odds ratios are considerably lower for scintigraphy, radiography and CT, suggesting that these tests have poorer diagnostic accuracy. MRI and SPECT have high PRs of > 70%, so most people have an imaging test result that would lead to being diagnosed with osteomyelitis. This, in part, reflects high incidence in the studies. PET, however, has a lower PR of 52.7%. This may be because the higher specificity and lower sensitivity of PET reduce the number of positive PET scans overall, and in particular the number of false-positive diagnoses.

Comparisons between tests

Forty of the included studies of adults reported diagnostic accuracy data for two or more imaging tests. To determine whether or not these studies show evidence that one test is superior to another, we compared the DORs for each test within each study. Appendix 4, Figure 23, shows these DORs for each imaging test in each study. These results show that, when radiography was included, it was generally inferior to other tests (particularly MRI and scintigraphy); scintigraphy was generally inferior to MRI.

In Figure 6, each dot is the difference in log-DOR between the test in the title of each subplot and the comparator test, indicated by the colour of the dot. Points to the right of the central black line are where the ‘title’ test has greater DOR than the ‘comparator’ test. These results suggest that SPECT and PET were generally superior to whatever test they were compared with (but with few studies). MRI was superior to scintigraphy and radiography, but not to PET or SPECT. Scintigraphy was superior to radiography, but not MRI, and radiography was generally inferior to all other tests.

FIGURE 6.

Comparing DORs within studies.

To formally assess whether or not these apparent differences are genuine, regression modelling was used to compare studies. This regressed the diagnostic accuracy data to estimate differences in DORs and specificity between tests across studies (see Chapter 3, Comparison of imaging tests). Two models were fitted: one using radiography as the baseline test and one using MRI as the baseline test. The formal results of these models are given in Appendix 4, Table 20, and a summary is presented in Table 5.

TABLE 5 - Summary of regression model to compare imaging tests

Parameter	Radiography worse than	Radiography similar to	Radiography better than
DOR	CT, MRI, PET, scintigraphy and SPECT	None	None
Specificity	PET	CT, MRI and SPECT	Scintigraphy
	MRI worse than	MRI similar to	MRI better than
DOR	None	PET, SPECT	CT, scintigraphy and radiography
Specificity	PET	CT, SPECT and radiography	Scintigraphy

These results largely confirm what was observed from the main diagnostic analysis in Diagnostic meta-analysis. Radiography has the lowest diagnostic accuracy (in terms of DOR) of all the tests. Its specificity is similar to that of CT, MRI and SPECT, but it has lower sensitivity than any of these. It has lower sensitivity and specificity than PET, but has higher specificity than scintigraphy.

Magnetic resonance imaging has similar diagnostic performance to SPECT and PET, but has lower specificity (and consequently higher sensitivity) than PET. MRI has higher DORs than CT, radiography or scintigraphy.

Synthesis of specific imaging tests

This section considers in more detail the diagnostic accuracy of each of the included imaging tests.

Studies of scintigraphy

Scintigraphy was not specified as an imaging test of interest in our protocol, but was included in many of the eligible studies as a comparator with other tests. Hence, we included it in our main analysis in Synthesis of diagnostic accuracy in adults. To facilitate a more complete examination of the diagnostic value of scintigraphy, we identified larger, recent studies (> 20 participants and published since 1990) of scintigraphy for analysis, where scintigraphy was the only imaging test reported in the publication. Nine studies132,136,142,148–152,165 from our database searches met these criteria and reported sufficient data for meta-analysis. Because they were off-protocol, we note that the additional nine studies have not been formally assessed for risk of bias. These nine studies were combined with the studies of scintigraphy included in the main meta-analysis, in order to explore the impact of this additional evidence on scintigraphy, and as a sensitivity analysis compared with using only the primary included studies.

A summary of all scintigraphy studies is given in Appendix 4, Table 21. As with other imaging tests, most of the studies are in people with diabetic foot ulcers. A range of types of scintigraphy have been reported, including (hydroxy)methylene diphosphonate [(H)MDP], WBC, hexamethylpropyleneamine oxime (HMPAO) and gallium scans. Some studies reported combinations of these tests. Three radioisotopes were used: technetium-99m (^99mTc), indium-111 (¹¹¹In) and gallium-67 (⁶⁷Ga).

Figure 7 presents a summary ROC plot for the sensitivity and specificity results from these studies, classified by scintigraphy type used. This suggests that there is considerable variation both within and between types of scintigraphy.

FIGURE 7.

Summary ROC plot for scintigraphy studies.

A bivariate meta-analysis of all scintigraphy studies gave an overall sensitivity of 85.6% (95% CI 80.9% to 89.2%) and an overall specificity of 71.7% (95% CI 62.1% to 79.6%). This is similar to the result seen in the main analysis (see Figure 3), but with a slightly higher sensitivity, suggesting that the additional studies (of scintigraphy only) have higher sensitivity than those where scintigraphy was compared with other tests. This could be as a result of either bias against scintigraphy in comparative studies, or poorer quality of the scintigraphy-only studies.

Figure 8 shows the results of bivariate meta-analyses categorised according to the type of scintigraphy performed. This suggests that ^99mTc HMPAO WBC scintigraphy and mixed ¹¹¹In WBC with ^99mTc MDP (methylene diphosphonate) scintigraphy can have very high diagnostic accuracy, with sensitivity > 85% and specificity > 90%; this is similar to the diagnostic accuracy of PET, as seen in Figure 3. The ^99mTc (H)MDP scintigraphy, by contrast, has a much lower specificity, of < 50%.

FIGURE 8.

Diagnostic accuracy of scintigraphy according to type of test performed.

Studies of magnetic resonance imaging scans

Most studies reported only one diagnostic accuracy result for MRI, and there was insufficient information to distinguish between different types of MRI scan. One study86 reported results for four types of MRI, with no evidence of any difference in diagnostic accuracy among them.

The MRI results in the main analysis had a very wide range of specificities, despite fairly consistent sensitivity. We sought to identify any characteristics of the included studies that might explain this variation in specificity. Appendix 4, Figure 24, shows the association between specificity and incidence of osteomyelitis for each study. This shows that specificity declines as incidence increases. This association was statistically significant (a decline in log-odds of specificity of 0.31 per 10% increase in incidence, p = 0.008). This suggests that osteomyelitis may be over-diagnosed using MRI in populations in which osteomyelitis is common. The model was not a good fit (adjusted R² = 0.17), so incidence does not explain all the variation in specificity. No other possible causes of this variation could be identified.

Studies of single-photon emission computed tomography and single-photon emission computed tomography/computed tomography

As with scintigraphy studies, SPECT studies used a range of radioisotopes and test types to perform SPECT imaging. Figure 9 shows the ROC plot of sensitivity and specificity according to the type of test used. There are too few studies to distinguish between tests, or to perform a bivariate meta-analysis. The results show no clear evidence of difference between the types of test. When comparing results with those for scintigraphy (see Figure 8), the results suggest that SPECT produces higher sensitivity than planar scintigraphy for the same test type.

FIGURE 9.

Diagnostic accuracy of SPECT studies according to type of SPECT used.

There was no evidence that studies of SPECT/CT had differing diagnostic accuracy to those of SPECT alone, although the number of studies was limited (results not shown).

Studies of positron emission tomography and positron emission tomography/computed tomography

There was no evidence that studies describing the test used as PET/CT differed in diagnostic accuracy from those describing the test only as PET (see Appendix 4, Figure 25). All but one study used ¹⁸F-FDG PET.

There was no evidence that quantitative analysis (such as measuring the standardised uptake value) improved the accuracy of PET (results not shown).

Studies of ultrasound

Only two studies of ultrasound for diagnosing osteomyelitis in adults were identified, so they could not be included in any bivariate meta-analysis. The results of these studies are summarised in Table 6. The two studies are not consistent in their estimated sensitivity and specificity. Both have DORs that suggest poorer diagnostic performance than MRI, PET or SPECT (see Table 4).

TABLE 6 - Summary of diagnostic accuracy of ultrasound studies

	Study
	Enderle et al. (1999)49 (n = 19)	Nath and Sethu (1992)87 (n = 25)
	Cause: diabetic foot ulcers	Cause: long bone pain or swelling
Sensitivity (%) (95% CI)	78.6 (57.1 to 100)	96.8 (88 to 100)
Specificity (%) (95% CI)	80 (44.9 to 100)	30 (1.6 to 58.4)
DOR (95% CI)	14.7 (1.2 to 185.2)	12.9 (0.6 to 292.8)
PPV (%) (95% CI)	91.7 (76 to 100)	68.2 (48.7 to 87.6)
NPV (%) (95% CI)	57.1 (20.5 to 93.8)	85.7 (49.1 to 122.4)

Studies reporting on combinations of imaging tests

Twelve studies40,43,54,55,64,65,69,75,84,96,109,141 reported data on the diagnostic accuracy of combining two tests. These were mostly combinations of scintigraphy with SPECT. Two studies combined radiography with probe-to-bone tests. These are summarised here, although we note that probe to bone is not an imaging test and so is not covered in this review.

Figure 10 shows the sensitivity and specificity estimates for all the combination tests. This shows considerable variation in the diagnostic accuracy of combination tests. Most tests have too few studies to draw any conclusions. A bivariate meta-analysis of the SPECT + scintigraphy studies gives a summary sensitivity of 87.3% (95% CI 87.2% to 87.5%) and specificity of 83.4% (95% CI 83.3% to 83.6%). This is a lower sensitivity and specificity than was estimated for SPECT alone (see Table 4).

FIGURE 10.

Diagnostic accuracy results for combination tests.

Figure 11 compares the DORs for the combination tests with those for the individual tests within each study. This shows no evidence that combining tests improves diagnostic accuracy; in most studies the combined test (circle) has a DOR no higher, and often lower, than the best single test (triangle).

FIGURE 11.

Diagnostic odds ratios for combination tests.

One study (Aragón-Sánchez et al. 40) of radiography combined with probe to bone obtained very high diagnostic accuracy (sensitivity 97.7%, specificity 94.6%). However, this study also achieved much higher diagnostic accuracy for radiography alone than was observed in most other studies. It is unclear whether this result is meaningful or a chance result in one study.

Other factors and subgroups

This section considers other factors that may affect diagnostic accuracy: the choice of reference standard, risk of bias, the presence of indwelling metalwork, the use of imaging tests prior to the main test and whether osteomyelitis is acute or chronic.

Choice of reference standard

The eligible reference standards for confirming the absence of osteomyelitis were histopathology, microbiology or clinical follow-up of at least 6 months. It is possible that the choice of reference standard could affect the diagnostic accuracy.

In Appendix 4, Figure 26 shows the ROC plots of diagnostic accuracy in accordance with the reference standard, for each imaging test. These results suggest that, when only clinical follow-up was used, specificity might be higher than average, particularly for MRI. A formal regression model to assess this, however, found no statistically significant evidence of a difference between reference standards in either specificity or DOR. We conclude that choice of reference standard is unlikely to lead to bias in the main meta-analyses.

Study quality and risk of bias

The impact of the QUADAS-2 assessment of each study on diagnostic accuracy was investigated. In Appendix 4, Figure 27 shows the ROC plots of sensitivity and specificity for each study according to whether the study was rated as being at a high, unclear or low risk of bias for patient selection. In Appendix 4, Figure 28 repeats this for quality of the index test.

Neither figure suggests that there is any evidence that diagnostic accuracy results were biased by quality factors. Reference standard and patient flow issues were rated as being at a low risk of bias for most studies, and there was no evidence that these had any impact on diagnostic accuracy (results not shown).

Indwelling metalwork

Few studies reported on the number of patients (if any) who had indwelling metalwork at the time of their scan. In Appendix 4, Figure 29 shows the diagnostic accuracy data for each imaging test according to whether indwelling metalwork was present in some patients, not present in any, or unreported. This figure shows no evidence that indwelling metalwork alters diagnostic accuracy.

Prior use of other imaging tests

In some studies, patients were included only after performing a prior imaging test, usually radiography. Patients would go on to the main imaging test only if osteomyelitis had not been ruled out by the initial test. This extra testing could affect the diagnostic accuracy observed.

Figure 12 shows the sensitivity and specificity for all imaging tests, according to the prior imaging test used (if any). This suggests that prior use of radiography might improve specificity in MRI studies, and both sensitivity and specificity in scintigraphy studies.

FIGURE 12.

Sensitivity and specificity according to whether or not any imaging test was used before the main test.

Formal regression analysis to test this possibility found no statistically significant evidence that radiography prior to MRI improved specificity (p = 0.472) or DOR (p = 0.201). There was some evidence (but not conventionally statistically significant) that radiography prior to scintigraphy increased the DOR (p = 0.141). Hence it is possible, but uncertain, that using radiography as a precursor test to MRI or scintigraphy may improve diagnostic performance.

Other than radiography, few tests were used in addition to the main test. This analysis considered only instances when prior tests were explicitly reported. In practice, it is likely that radiography will be used as an initial test in most patients.

Acute and chronic osteomyelitis

Studies were divided according to whether the suspected osteomyelitis was acute or chronic. Studies were categorised as:

• acute or subacute osteomyelitis

• COM

• COM and AOM

• not reported.

Figure 13 shows the sensitivity and specificity results according to the test used and acute or chronic status. There appears to be some pattern in the choice of test used. PET and SPECT have not been evaluated in people with AOM; almost all studies of PET are in people with COM. This may be a consequence of the types of people in these studies; nearly all studies of AOM were in people with diabetic foot ulcers. From these data, there is no clear evidence that diagnostic accuracy for any of the tests differ between COM and AOM.

FIGURE 13.

Sensitivity and specificity according to whether the study was of AOM or COM.

Figure 14 shows the results of bivariate meta-analysis according to acute or chronic status. There is no compelling evidence that diagnostic accuracy for any test varies with acute or chronic status. It is possible that MRI is more specific (91%) but less sensitive (89%) in acute cases than in chronic cases (78% and 98%, respectively). The results are consistent with those in the overall analysis (see Figure 3), with MRI, PET and SPECT generally having greater diagnostic accuracy than radiography, CT or scintigraphy.

FIGURE 14.

Bivariate meta-analysis according to acute or chronic status.

Synthesis of diagnostic accuracy in people with diabetic foot ulcers

This section considers the 35 studies in which diabetic foot ulcers were the primary reason for testing for osteomyelitis. 37,38,40–43,47,49,51,53,55,56,62,67,70,71,74,76,77,82,84,86,88–90,92–95,97,99,102,107,111–113 This includes studies in which at least 60% of patients had diabetic foot ulcers; studies with smaller numbers of people with foot ulcers were excluded. Studies primarily of patients with diabetic foot ulcers constitute around half of the total studies in this review. One study did not report sufficient data to be included in this analysis and is presented in Studies not included in the quantitative synthesis. 37

The sensitivity and specificity estimates from these studies are presented in Figure 15. The results suggest high sensitivity (generally > 80%) for MRI and scintigraphy, but with a wide range of specificities. PET showed high specificity, but a range of sensitivities. Radiography generally had low sensitivity. There were too few studies of SPECT or ultrasound to draw any conclusions, and none of CT. In Appendix 4, Figure 30 shows the results for PPV and NPV, with considerable diversity in PPV and NPV values across studies.

FIGURE 15.

Sensitivity and specificity in studies of patients with diabetic foot ulcers.

Figure 16 shows the results of a bivariate meta-analysis of sensitivity and specificity. The results here are similar to those for all studies (see Figure 3). Owing to non-convergence of their respective bivariate analyses, the univariate results are presented for PET and SPECT. Both MRI and PET have high accuracy, with MRI having higher sensitivity but lower specificity. Scintigraphy and radiography have lower specificity, but here scintigraphy has higher sensitivity than PET. The summary HSROC curves (see Appendix 4, Figure 31) suggest that MRI and PET (and possibly SPECT, although data are limited) have similar overall HSROC curves, and so similar diagnostic accuracy, whereas radiography and scintigraphy have lower accuracy.

FIGURE 16.

Bivariate analysis of sensitivity and specificity in people with diabetic foot ulcers.

In Appendix 4, Figure 32 gives the results of the bivariate meta-analysis of PPV and NPV, which shows that both MRI and PET have high PPV (88%) and NPV (> 85%), with MRI having a higher NPV (95%). Scintigraphy and radiography have lower PPV and NPV rates.

In Appendix 4, Table 23 gives the results of univariate meta-analyses. The results are consistent with bivariate analyses. As in the overall analyses, PET has a markedly lower PR (46%) than MRI (70%). This means that fewer people are considered to have osteomyelitis using PET; this is a consequence of its higher specificity and lower sensitivity.

In studies using scintigraphy in patients with diabetic foot ulcers, a range of different types of scintigraphy were used. In Appendix 4, Figure 33 shows the results of bivariate meta-analysis of these scintigraphy studies according to the test used. This shows substantial variation in diagnostic accuracy, with ^99mTc HMPAO WBC scintigraphy being notably most accurate (89% sensitivity for 92% specificity). This makes it more sensitive than PET, and more specific, but less sensitive than MRI.

Synthesis of diagnostic accuracy for osteomyelitis not related to diabetes

The systematic review identified 31 studies in adults where osteomyelitis was not explicitly related to diabetes. 39,44–46,48,50,54,57–61,63–66,68,73,75,79–81,83,85,87,101,103,105,106,108,109 This excludes studies where some of the participants had diabetes, but includes studies where diabetes was not reported, or where the studies had a range of inclusion criteria, which may have included people with diabetes.

The studies were classified according to the location of the suspected diabetes as follows:

• axial skeleton (skull and spine)

• long bone (leg)

• pelvis, hip or knee

• multiple locations or unreported location.

Studies were also classified according to the suspected cause of osteomyelitis:

• infections and ulcers

• trauma and surgery

• contiguous disease

• multiple causes

• other or unstated cause.

Figure 17 shows the sensitivity and specificity estimates by imaging test and scan location. Similarly, Figure 18 shows results by suspected cause. There are generally few studies of any imaging test within each category, so it is not possible to draw firm conclusions. The results are broadly similar to those obtained when considering all studies and to the results in studies of diabetic foot ulcers. There is no immediate evidence that diagnostic accuracy varies according to location or cause. The choice of imaging test may vary according to location, with SPECT and CT being used more commonly to image the axial skeleton, where MRI is used less.

FIGURE 17.

Sensitivity and specificity according to scan location.

FIGURE 18.

Sensitivity and specificity according to likely cause of osteomyelitis.

Plots for PPV and NPV by location are given in Appendix 4, Figure 35, and by cause in Appendix 4, Figure 36. There is no clear evidence that PPV or NPV varies by location of imaging or cause of osteomyelitis.

Given the limited numbers of studies in each location or cause category, bivariate meta-analysis of sensitivity and specificity could be performed for only a few tests. Results of the analyses are shown in Appendix 4, Figure 37, by location and in Appendix 4, Figure 38, by cause. The limited number of studies means that CIs for most meta-analysis results are wide, so it is difficult to draw any firm conclusions. There is no clear evidence that diagnostic accuracy varies by location or cause, or that results differ from those seen in the overall meta-analysis of all studies (see Figure 3).

Given that there was no clear evidence that diagnostic accuracy for any of the tests varied by cause or location, a combined meta-analysis of all studies in adults without diabetes was performed. The results of the bivariate analysis of sensitivity and specificity are shown in Figure 19. Results from univariate meta-analyses are provided in Appendix 4, Table 23.

FIGURE 19.

Bivariate meta-analysis in studies of people without diabetes.

These results are broadly comparable to the results using all studies (see Figure 3). There is some suggestion that scintigraphy has slightly higher specificity and MRI has slightly lower specificity than the overall results, or to those in diabetic foot ulcer studies (see Figure 15), but 95% CIs are too wide to draw any firm conclusions.

Studies not included in the quantitative synthesis

Of the two adult studies that were not included in any of the above quantitative syntheses, one evaluated the accuracy of MRI37 and one compared radiography, CT and radiography + CT72 (Table 9). The results of both studies differed somewhat from the meta-analysis and showed worse diagnostic accuracy overall, although these results should be interpreted with caution owing to the relatively small size of the studies.

Abdel Razek and Samir37 conducted a prospective study that evaluated MRI in adults with suspected acute diabetic foot osteomyelitis. MRI had a sensitivity of 65% (95% CI 46% to 85%) and a specificity of 61% (95% CI 39% to 84%). These results differ from the meta-analysis, which found higher sensitivity and specificity for MRI (see Table 4). Larson et al. 72 undertook a retrospective study conducted in patients with suspected osteomyelitis associated with pelvic pressure ulcers. Results for radiography (sensitivity 88%; specificity 32%), CT (sensitivity 50%; specificity 85%) differed from the meta-analysis, which found lower sensitivity and higher specificity for radiography, and higher sensitivity for CT overall (see Table 4). The combination of radiography and CT increased the sensitivity of CT alone, but reduced specificity (sensitivity 61%; specificity 69%). This differs from the results of the only other study of radiography + CT reported earlier, which found that radiography and CT combined had a sensitivity of 38% and a specificity of 85% (see Studies reporting on combinations of imaging tests). 75

Synthesis of studies in children

The three studies of children were conducted in single centres in patients with suspected AOM. 52,78,98 Two studies evaluated ultrasonography,52,78 and two used MRI. 78,98 Radiography, scintigraphy and CT were evaluated in only one study. 78

The evidence for the accuracy of ultrasound and MRI in children was mixed and limited overall. Ultrasonography had moderate sensitivity (55%, 95% CI 43% to 67%) and specificity (47%, 95% CI 24% to 70%) in 82 children with suspected acute haematogenous osteomyelitis,78 but had perfect (100%) sensitivity and specificity in a study of 27 children with negative or equivocal initial radiographic findings. 52 MRI had a sensitivity of 81% (95% CI 64% to 93%) and a specificity of 67% (95% CI 22% to 96%) in 38 children with suspected acute haematogenous osteomyelitis,78 but preoperative MRI had 38% sensitivity and 95% specificity in a study of 54 patients with septic hip. A study-by-study summary is provided in Table 10.

Malcius et al. 78 conducted a prospective study of 183 hospitalised children with suspected acute haematogenous osteomyelitis. A total of 169 early radiography (median first day of hospital), 142 late radiographic assessments (15th day of hospital stay), 82 ultrasonographies (second day), 76 scintigraphies (third day), 38 MRI scans (seventh day) and 17 CT scans (15th day) were conducted. Scintigraphy (^99mTc-MDP) was conducted in patients who had an unclear diagnosis based on early radiography and/or ultrasound, and MRI and CT were performed when results from scintigraphy were inconclusive. The reference standard included clinical follow-up, microbiology (blood or bone culture) and/or surgery. Acute haematogenous osteomyelitis was confirmed in 156 (85.2%) patients.

Early radiography had low sensitivity (16%, 95% CI 100% to 23%) but high specificity (96%, 95% CI 78% to 100%). Late radiography had high sensitivity (82%, 95% CI 75% to 88%) and specificity (92%, 95% CI 62% to 100%).

Ultrasonography had moderate sensitivity (55%, 95% CI 43% to 67%) and specificity (47%, 95% CI 24% to 70%). Bone scintigraphy had high sensitivity (81%, 95% CI 68% to 90%) and specificity (84%, 95% CI 60% to 97%).

Magnetic resonance imaging had a sensitivity of 81% (95% CI 64% to 93%) and a specificity of 67% (95% CI 22% to 96%), and CT had a sensitivity of 67% (95% CI 38% to 88%) and a specificity of 50% (95% CI 1% to 98%), although results should be interpreted with caution because of the small number of patients tested.

Overall, Malcius et al. 78 found that scintigraphy and MRI are most valuable for detecting acute haematogenous osteomyelitis at the onset of the disease, and late radiography is the most valuable radiological method. The comparability of index tests is limited by the fact that more advanced tests were performed only in harder-to-diagnose subgroups.

Schlung et al. 98 conducted a retrospective study of 83 children with suspected septic hip arthritis, of whom 54 underwent preoperative MRI. Osteomyelitis was confirmed by positive joint fluid culture or positive peripheral blood culture with joint aspirate. The sensitivity of MRI was 38% and the specificity was 95%. Of the 10 false-negative cases missed by MRI, six had a positive femoral aspiration that resulted in appropriate management, but three had significant morbidity owing to missed or delayed diagnosis; results for the remaining patient were not reported.

Ezzat et al. 52 conducted a small prospective study of children with suspected AOM with negative or equivocal conventional plain radiography. The accuracy of greyscale and power Doppler ultrasound was evaluated in 27 children, of whom 23 underwent surgery. Osteomyelitis was confirmed as positive in 25 out of 27 patients. Ultrasonography had a sensitivity of 100% and a specificity of 100%. The results of this study need to be interpreted with caution because of the limited number of patients.

Review of inter-rater reliability

Review of implementation

One study of implementation was identified by the review. 118

Govaert et al. 118 was a survey of orthopaedic and trauma surgeons, radiologists and nuclear medicine physicians in the Netherlands on preferred imaging strategies for diagnosing post-traumatic osteomyelitis. The 16-question survey included four patient-based clinical cases of fracture-related osteomyelitis. Each case included relevant medical history of the patient combined with a clinical picture of the affected limb and radiographic results. Participants were asked to select which imaging test(s) they considered most appropriate to diagnose or exclude the presence of post-traumatic osteomyelitis.

There were 346 responders, comprising 153 trauma surgeons, 104 orthopaedic surgeons, 56 nuclear medicine physicians and 33 musculoskeletal radiologists. There was consensus on the usefulness of radiography in patients with a late infection, but not in patients with acute infection. There was a marked difference between trauma surgeons, and orthopaedic surgeons’ choice of nuclear medicine imaging for late infection: trauma surgeons preferred PET and orthopaedic surgeons favoured WBC scintigraphy. Similarly, there was a consistent difference between radiologists and nuclear medicine physicians: musculoskeletal radiologists favoured MRI whereas nuclear medicine clinicians preferred PET/CT. CT and three-phase bone imaging for late fracture-related infections were favoured by orthopaedic surgeons and some musculoskeletal radiologists, but not by trauma surgeons or nuclear medicine physicians. Ultrasound-guided biopsy was regarded by all physicians to have some role in patients with early infection but was not popular for late infections. Preference of imaging was influenced by access to equipment, with clinicians selecting imaging methods they had access to in their own hospital.

Summary of previous systematic reviews

The systematic database searches identified 11 previous systematic reviews of the diagnosis of osteomyelitis. 20–30 Most of the studies included in these reviews were also included in this review, although some did not meet our inclusion criteria. The previous reviews are summarised in Table 14.

Four of these reviews considered primarily, or only, people with diabetic foot ulcers. 20,22,23,26 Their conclusions varied, depending on the tests included, but MRI, PET and WBC scintigraphy were all suggested as suitable imaging tests.

The Lauri et al. 23 review, which is the most recent review of patients with diabetic foot osteomyelitis, concluded that MRI, PET and WBC scintigraphy have similar sensitivity, but MRI has lower specificity. This is similar to the findings of this review.

Three reviews of osteomyelitis in the general population mostly recommended PET and SPECT as having the best diagnostic accuracy,25,27,28 but that leucocyte scintigraphy could be useful in some cases, such as for COM in the peripheral skeleton,25 or when PET is unavailable. 28 Only one of these three reviews considered MRI. 25 One review focused on patients with peripheral post-traumatic osteomyelitis and concluded that WBC [or antigranulocyte antibody (AGA)] scintigraphy with SPECT/CT or ¹⁸F-FDG-PET/CT has the best diagnostic accuracy in this population. 21 Another review of MRI in patients with pressure ulcers was inconclusive owing to insufficient evidence. 29

Only two reviews considered osteomyelitis in children. One focused on calcaneal osteomyelitis. The review included only three diagnostic accuracy studies and was inconclusive owing to the limited evidence. 24 One review focused on haematogenous acute and subacute paediatric osteomyelitis. 30 The review included three studies of radiography that showed low sensitivity (16–20%) but high specificity (80–100%). ⁹⁹Tc bone imaging had variable sensitivity (53–100%) and specificity (50–100%) (three studies). MRI had the highest sensitivity (80–100%) and specificity (70–100%) (five studies), compared with CT (sensitivity 67%, specificity 50%, two studies) and ultrasound (sensitivity 55%, specificity 47%, one study). Only one of the studies included in this review met our inclusion criteria, which was the largest and most comprehensive study in children. 78 Overall, the diagnostic accuracy findings of this review broadly agree with the evidence in adults.

Clinical effectiveness summary and conclusions

Statement of principal findings

This systematic review identified 81 studies relating to imaging tests for the diagnosis of osteomyelitis. Seventy-seven of these were studies of diagnostic accuracy, of which 69 reported diagnostic accuracy data sufficient to be included in a quantitative synthesis. Thirty-six diagnostic accuracy studies were in patients with diabetic foot ulcers, and 40 were in patients without diabetes.

Eleven studies evaluated the inter-rater reliability of at least one imaging test and one study provided data on clinician opinions on imaging tests for osteomyelitis.

Overall, most diagnostic accuracy studies were small (80% had < 50 participants) and nearly one-quarter were considered as being at a high risk of bias, although poor reporting means that there is significant uncertainty about the quality of most of the studies. The studies of inter-rater reliability were considered of fair quality overall, although most were small and none provided satisfactory measures of intra-rater reliability.

Only one study from the Netherlands on the implementation of diagnostic test imaging for osteomyelitis was included. 118 The applicability of its findings to other health-care systems is uncertain.

Strengths and limitations of the assessment

Uncertainties

The main uncertainties remaining from this review arise largely because of limitations in the identified studies.

The diagnostic accuracy of imaging tests in children remains highly uncertain owing to the very limited nature of the evidence. We could reach no firm conclusions on the diagnostic accuracy of any imaging test in children.

The diagnostic accuracy of ultrasound is currently unknown as the only two studies in adults had conflicting results. However, DORs in both studies were low, suggesting that ultrasound is unlikely to have similar or better diagnostic accuracy than MRI, PET or SPECT. Ultrasound, however, is a test of choice in patients with metalwork in situ as this causes artefacts in CT and MRI.

Although we found no evidence that diagnostic accuracy varied across subgroups of patients, limited or inconsistent reporting of some characteristics, such as AOM versus COM, or the cause or location of osteomyelitis, means than differences between tests or between subgroups cannot be ruled out.

Considerations from patient representatives

Both patient representatives were parents of children who were treated for osteomyelitis. Both discussed the fact that diagnosis was generally a long process, with multiple imaging tests performed, with preliminary radiography and then ultrasound tests, both of which might be repeated. MRI was used for the ‘definitive’ diagnosis, and MRI was also used to confirm that treatment had been effective. We note that this process of multiple tests and their interactions could not be easily captured in this systematic review, and the impact and value of multiple imaging tests remain uncertain.

Antibiotics were given early in the diagnostic process, before ultrasound or MRI was performed. This may affect evaluations of diagnostic accuracy, because osteomyelitis is treated before being formally diagnosed. In future studies, the impact on diagnostic accuracy of treatment, given the ethical need for prompt treatment, will need to be considered.

Both parents noted that there was little time to discuss the tests or decide which was best for their child and they accepted whatever their doctors recommended. Both felt that the need to correctly diagnose osteomyelitis outweighed any concerns about the tests themselves. So, for example, they were willing to accept MRI requiring general anaesthesia.

Implications for health care

This review found that MRI, PET and SPECT all have broadly similar and high accuracy when diagnosing osteomyelitis. All three tests correctly diagnose most people with, and most people without, osteomyelitis. All three are therefore suitable imaging tests for reliably diagnosing osteomyelitis. No clear reason to prefer one test over the other in terms of diagnostic accuracy was identified. The wider availability of MRI machines, and the fact that MRI does not expose patients to harmful ionising radiation, may mean that MRI is preferable in most cases, unless it is unsuitable for a particular patient. MRI also gives additional information as to the location of fluid collections and anatomical detail for surgical planning not given by SPECT or PET in such detail. PET or SPECT may be required if MRI is inconclusive.

Magnetic resonance imaging generally had poorer specificity than PET or SPECT, with a wide range of specificities across studies. This means that there may be substantial numbers of ‘false-positive’ cases when using MRI and clinicians should be aware of this potential for overdiagnosis. This may mean that MRI may be best suited to cases where false-positive cases are of lesser concern, perhaps such as when patients would proceed directly to antibiotic treatment. This might, however, be an artefact of the diagnostic studies; actual diagnoses of osteomyelitis will consider the clinical background of the patient (e.g. fever and infection markers in the blood test), so diagnosis is not based on the imaging test alone.

Position emission tomography had poorer sensitivity, but higher specificity, than MRI, with more consistent results across studies. This may make PET better suited to situations where avoiding false-positive diagnoses is important, for example when the test would be followed by surgery or other invasive procedures. However, MRI may still provide greater visual detail to guide the surgical approach than PET/CT alone.

Scintigraphy, in general, is less accurate than MRI or PET, but current methods of scintigraphy, particularly ^99mTc HMPAO WBC scintigraphy, may have comparable accuracy, and this should be the preferred approach. However, planar scintigraphy may have poorer diagnostic accuracy than 3D SPECT imaging, so should perhaps be used only when SPECT is unavailable or unsuitable.

Data on ultrasound and CT were limited, but suggested that these tests have poorer diagnostic accuracy and perhaps should not currently be used in adults.

Radiography on its own has poor diagnostic accuracy so should probably not be used in isolation. This review could not confirm whether or not using radiography prior to MRI or other tests improves accuracy or reduces costs. It is, therefore, unclear whether or not the common practice of using radiography as a first test and proceeding to a MRI or another test if the radiography is inconclusive maximises diagnostic accuracy. It is, however, likely that radiography can be usefully used to rule out osteomyelitis in more obvious cases (e.g. when symptoms are the result of bone fracture). Radiography is probably more useful in chronic or subacute osteomyelitis, when bone destruction is already visible, but not in acute haematogenous osteomyelitis, when early changes are looked for.

The review found that diagnostic accuracy in adults does not appear to vary with the potential cause of osteomyelitis, or with the body part scanned. In particular, there was no evidence that diagnostic accuracy was different in patients with diabetic foot ulcers. This suggests that MRI, PET and SPECT are all reasonable tests to use in adults, regardless of the underlying conditions the patient has. The choice may depend primarily on suitability for the patient, such as potential difficulties in interpreting MRI scans in patients with indwelling metalwork. However, data on specific locations or causes of osteomyelitis other than diabetic foot ulcers were limited, so the possibility that diagnostic accuracy is different for some locations or causes remains.

The review identified very limited data on diagnosing osteomyelitis in children, so considerable uncertainty remains over the diagnostic accuracy of imaging tests in children. Clinicians should be aware of this limitation in the evidence base.

There was some evidence to suggest that MRI has an acceptable degree of agreement among radiologists. The evidence for the inter-rater reliability of other imaging tests is too limited to draw conclusions. There is also very limited evidence on the implementation of imaging tests for the diagnosis of osteomyelitis. Notably, there is a lack of evidence on patient preferences and on the most cost-effective diagnostic imaging pathways for osteomyelitis.

Suggested research priorities

The most urgent research priority is to perform diagnostic accuracy studies of imaging tests in children, given the limitations of the current evidence base. Large diagnostic accuracy studies are needed to assess imaging tests in children. These should be of high quality, with proper blinding of test assessors and consecutive recruitment of patients, and reporting of the appropriate reference standard used to confirm positive and negative diagnoses. The priority tests should be MRI and ultrasound, ideally comparing the two tests in the same children.

Ultrasound has not been widely assessed in adults. Current results suggest that ultrasound on its own may not be sufficiently accurate to diagnose osteomyelitis, but further accuracy studies are needed to resolve the uncertainty. It may be more appropriate to investigate the diagnostic accuracy of ultrasound as a precursor to MRI or other tests (e.g. as a replacement for radiography).

Consistency of test interpretation is an important area in diagnostic testing that is often overlooked. Further investigation of inter- and intra-rater reliability for all tests is needed. Ideally this should be included in diagnostic accuracy tests, but it could also be investigated in case series or clinical audits.

Given the similarities in diagnostic accuracy of MRI, PET and SPECT, suitable investigation of patient and clinician experience and opinion of these tests, through surveys or focus groups, would be useful to identify practical reasons for the choice of test. Similarly, a formal economic evaluation of these tests, accounting for test cost, sequencing of tests, availability and risk of radiation exposure, would also help to clarify the choice between these tests.

Contributions of authors

Alexis Llewellyn (https://orcid.org/0000-0003-4569-5136) performed the systematic review, including screening, study selection, data extraction, quality assessment and narrative synthesis. He wrote most sections of this report.

Julie Jones-Diette (https://orcid.org/0000-0003-1769-8612) performed data extraction and quality assessment and prepared data tables for the report.

Jeannette Kraft (https://orcid.org/0000-0003-2705-1302) and Colin Holton provided clinical expertise and commented on drafts of the protocol and this report.

Melissa Harden prepared and performed all database searches for the systematic review.

Mark Simmonds (https://orcid.org/0000-0002-1999-8515) oversaw the project as a whole, performed the statistical analyses and wrote all sections of the report pertaining to those analyses.

Data-sharing statement

This is a systematic review of published data, and no primary data were created for this report. All data used are as reported in the publications listed throughout this report. Data-extraction forms are available on request from the corresponding author.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care.

Database search strategies

Ongoing studies search strategies

Guideline searches