Lead-I ECG for detecting atrial fibrillation in patients with an irregular pulse using single time point testing: a systematic review and economic evaluation

Types of atrial fibrillation

Three types of AF (based on presentation and duration of the arrhythmia) are described in Table 1.

Atrial fibrillation can be categorised as valvular or non-valvular for the purposes of choosing the most suitable treatment. Categorisation as valvular or non-valvular refers to the underlying condition causing AF (i.e. whether or not there is valve disease present) rather than the duration of AF episodes. Both valvular AF and non-valvular AF can be paroxysmal, persistent or permanent. Patients diagnosed with paroxysmal AF can develop persistent or permanent AF. 2 It is also possible, but most unusual, for patients with persistent AF to revert to normal sinus rhythm. 2

Symptoms of atrial fibrillation

Patients with AF may experience palpitations, dizziness, shortness of breath and tiredness. However, AF can be asymptomatic and may be identified only during medical appointments for other conditions. Because the symptoms are intermittent, many cases of paroxysmal AF remain undiagnosed. 2 Cases of paroxysmal AF may be detected only after a prolonged monitoring period, rather than from a single examination. 2

Epidemiology

Atrial fibrillation is the most common type of cardiac arrhythmia. Estimates from 2010 suggest that, worldwide, 20.9 million men and 12.6 million women are living with AF. 2 Higher rates of AF are recorded in developed countries than in undeveloped countries; however, this may be explained by differences in reporting. 2 Higher rates of AF are recorded in people living in Western countries (estimated incidence rate of 9.03 per 1000 patient-years)4 than in people living in Asian countries (estimated incidence rate of 5.38 per 1000 patient-years). 5 Despite a higher exposure to potential AF risk factors, such as hypertension and obesity, African American people were found to have a lower age- and sex-adjusted risk of being diagnosed with AF than white American people. 6

In the 2016 European Society of Cardiology (ESC) guidelines,2 the prevalence of AF in the European Union was reported to be 3%.The ESC also notes that one in four middle-aged people in Europe and the USA will develop AF. 2 The prevalence of AF in Europe is projected to increase over time because of the ageing population, an increase in incidence of conditions associated with AF and the improvements in the detection of AF. 2

The overall age-adjusted incidence of AF per 1000 patient-years in the primary care setting in the UK has increased from 1.11 [95% confidence interval (CI) 1.09 to 1.13] in 1998–2001 to 1.33 (95% CI 1.31 to 1.35) in 2007–10, with a constant increase in incidence reported in people aged ≥ 75 years. 7

In the NHS Quality and Outcomes Framework for 2015–16,8 the prevalence of AF in England is estimated to be 1.7%, which equates to 985,000 people. However, as noted, AF can be asymptomatic, which suggests that 1.7% may be an underestimate of the true prevalence. 9 Based on a reference population in a region of Sweden, Public Health England has estimated that the true prevalence of AF in England is likely to be 2.5% and that 1.4 million people in England are living with AF. 10 In the most recent data from the NHS Quality and Outcomes Framework for 2016–17, the prevalence of AF in England is estimated to be 1.8%, equating to 1,066,000 people. 11 An assessment of electronic primary care records identified an increase in the prevalence of AF in the UK from 2.14% in 2000 to 3.29% in 2016 in those aged ≥ 35 years. 12

The prevalence of AF increases with age and a higher proportion of men than women live with the condition (2.9% and 2.0%, respectively). 10 The median age at which people are diagnosed with AF is 75 years. 10 The largest numbers of AF diagnoses in men and women occur between the ages of 75 and 79 years and 80 and 84 years, respectively. 10 Although fewer women than men have AF, women experience higher mortality rates owing to AF-related strokes. 10

Paroxysmal AF is estimated to account for between 25% and 62% of patients with AF treated in hospitals and general practitioner (GP) practices. 13 Patients with paroxysmal AF tend to be younger and have fewer comorbidities (e.g. hypertension or congestive heart failure) than patients with persistent or permanent AF. 13,14

Impact of atrial fibrillation

Untreated AF is a major risk factor for stroke. AF is associated with a fivefold increase in the risk of stroke and a threefold increase in the risk of congestive heart failure. 15 Strokes with AF as the underlying cause may be more severe than strokes unrelated to AF. 16 Furthermore, each year in the UK, 100,000 people have a stroke and one in five of those strokes has AF as the underlying cause. 17

There is evidence to suggest that there are differences in the risk of stroke between patients with paroxysmal, persistent and permanent AF, with patients with paroxysmal AF having a lower risk of stroke than those with persistent or permanent AF. 18,19 The risk of stroke in patients with symptomatic AF is similar to that in patients with asymptomatic AF. 20

The ESC reports that, annually, between 10% and 40% of patients with AF are hospitalised and that patients with AF have impaired health-related quality of life (HRQoL), regardless of co-existing cardiovascular conditions. 2 Cognitive decline and vascular dementia are conditions suggested to develop from the onset of AF. 2

Current diagnostic and treatment pathways

The National Institute for Health and Care Excellence (NICE) clinical guideline CG1803 provides recommendations for the diagnosis and management of AF. An update of CG1803 is in progress.

Diagnosis of atrial fibrillation

In CG180,3 NICE recommends the use of manual pulse palpation (MPP) to detect the presence of an irregular pulse that may indicate underlying AF in people who have symptoms such as breathlessness/dyspnoea, palpitations, syncope/dizziness, chest discomfort, previous stroke or suspected transient ischaemic attack (TIA).

During the scoping stage of this assessment, clinical experts commented that people presenting with a stroke or TIA would undergo electrocardiogram (ECG) testing for AF in secondary care and are, therefore, outside the scope of an assessment that focuses on diagnosis in primary care.

If AF is suspected because of an irregular pulse, NICE3 recommends that the diagnosis should be confirmed based on the results of an ECG. Patients who are suspected of having paroxysmal AF that is not detected by the ECG should be monitored using either a 24-hour ambulatory monitor or an event recorder ECG. Patients with confirmed AF may also undergo echocardiography to further inform the management of their condition. The current diagnostic pathway for people presenting to primary care with signs or symptoms of the condition and who have an irregular pulse is depicted in Figure 1.

FIGURE 1.

Current diagnostic pathway.

Management of atrial fibrillation

An overview of the treatment pathway described in CG1803 is provided in Figure 2. As shown in Figure 2, the management of AF is subdivided into four algorithms.

FIGURE 2.

Overview of AF algorithms. Source: NICE CG180. 3 © NICE 2014 Atrial fibrillation: management. Available from www.nice.org.uk/guidance/cg180. All rights reserved. Subject to notice of rights (www.nice.org.uk/terms-and-conditions#notice-of-rights).

© NICE 2014 Atrial fibrillation: management. Available from www.nice.org.uk/guidance/cg180. All rights reserved. Subject to notice of rights (www.nice.org.uk/terms-and-conditions#notice-of-rights).

The aim of treatment is to reduce the symptoms of AF and prevent the potential consequences of undiagnosed AF, such as stroke. 3

Reducing stroke risk

In CG180,3 NICE recommends that patients with AF should be assessed for both their risk of stroke and their risk of bleeding. The risk of stroke should be assessed using the CHA₂DS₂-VASc21 algorithm [history of congestive heart failure, hypertension, age ≥ 75 years (doubled), diabetes mellitus, prior stroke or TIA (doubled), vascular disease, age 65–74 years, female] and the risk of bleeding should be assessed using the HAS-BLED22 algorithm (hypertension, abnormal liver/renal function, stroke history, bleeding predisposition, labile international normalised ratio, age, drug/alcohol use).

Depending on the age of the patient, the results of the CHA₂DS₂-VASc21 assessment and the results of the HAS-BLED22 assessment, patients with non-valvular AF may be offered stroke prevention treatment with either a vitamin K antagonist (usually warfarin) or a non-vitamin K antagonist oral anticoagulant (NOAC) [i.e. apixaban (Eliquis^®; Bristol–Myers Squibb, NY), dabigatran etexilate (Pradaxa^®, Prazaxa^®, Pradax^®; Boehringer Ingelheim GmbH, Germany), rivaroxaban (Xarelto^®; Bayer Health Care, Germany) or edoxaban (Lixiana^®; Daiichi Sankyo, Japan)].

Rate and rhythm control

In CG180,3 NICE recommends (with some exceptions) that people with AF who need drug treatment as part of their rate-control strategy should be offered either a standard beta-blocker or a rate-limiting calcium-channel blocker. Exceptions include people whose AF has a reversible cause, those who have heart failure thought to be primarily caused by AF, those with new-onset AF, those with an atrial flutter whose condition is considered suitable for an ablation strategy to restore sinus rhythm or for those whom a rhythm control strategy would be more suitable based on clinical judgement. Digoxin may be offered to sedentary people who have non-paroxysmal AF. If monotherapy does not control the AF symptoms, and the symptoms are a result of poor ventricular rate control, dual therapy with any two of a beta-blocker, diltiazem and digoxin is recommended. 3 For rhythm control, NICE3 recommends pharmacological treatment with or without electrical rhythm control (cardioversion).

In CG180,3 NICE also recommends strategies for left atrial ablation to control AF.

imPulse

The lead-I ECG device is provided with downloadable software for data analysis (imPulse Viewer) and a cable for charging the device. The ECG readings are taken by holding the device in both hands and placing each thumb on a separate sensor on the device for a pre-set length of time (from 30 seconds to 10 minutes). To be operated, the device requires the associated software to be installed on a nearby PC or tablet. Data are transferred to hardware hosting the analytical software using Bluetooth (Bluetooth Special Interest Group, WA, USA), with the recorded ECG trace being displayed in real time.

Once the recording has finished, the generated ECG trace can be saved in the imPulse viewer. Previously recorded readings can also be loaded into this viewer and ECG traces can be saved as a PDF (Portable Document Format). The software has an AF algorithm that analyses the reading and states whether AF is unlikely, possible or probable. In the event of a ‘possible’ or ‘probable’ result, the company recommends that the person should undergo further investigation, and that the algorithm should not be used for a definitive clinical diagnosis of AF.

Kardia Mobile

The Kardia Mobile lead-I ECG device works with the Kardia Mobile app to record and interpret ECGs. In addition to the Kardia Mobile device and app [www.alivecor.com/ (accessed January 2018)], a compatible Android (Google Inc., Mountain View, CA, USA) or Apple (Apple Inc., Cupertino, CA, USA) smartphone or tablet is required.

Two fingers from each hand are placed on the Kardia Mobile device to record an ECG that is sent wirelessly to the device hosting the Kardia Mobile app. The default length of recording is 30 seconds; however, this can be extended up to 5 minutes. The measured ECG trace is then automatically transmitted as an anonymous file to a European server for storage as an encrypted file.

The app uses an algorithm to classify measured ECG traces as (1) normal, (2) possible AF detected or (3) unclassified. The instructions for use state that the Kardia Mobile app assesses the patient for AF only, and the device will not detect other cardiac arrhythmias. Any detected non-AF arrhythmias, including sinus tachycardia, are labelled as unclassified. The company states that any ECG labelled as ‘possible AF’ or ‘unclassified’ should be reviewed by a cardiologist or trained health-care professional. ECG traces measured by the device can be sent from a smartphone or tablet by e-mail as a PDF attachment and stored in the patient’s records. The first version of the Kardia app did not have automatic diagnostic functionality. The AF algorithm was added to the app in January 2015. The Kardia Mobile has previously been available as the AliveCor Heart Monitor.

MyDiagnostick

The MyDiagnostick lead-I ECG recording is generated after a patient holds the metal handles at each end of the device for 1 minute. A light on the device will turn green if no AF is detected or turn red if AF is detected. If an error occurs during the reading, the device produces both an audible warning and a visible warning from the light on the device. Up to 140 ECG recordings can be recorded on the device before it starts to overwrite previous recordings. The MyDiagnostick device can be connected to a computer via a USB (universal serial bus) connection to download the generated ECG trace for review and storage using free software that can be downloaded from the MyDiagnostick website [www.mydiagnostick.com (accessed January 2018)].

RhythmPad GP

The RhythmPad GP lead-I ECG readings are taken by placing the palms of both hands on the surface of the device for 30 seconds. Alternative configurations can be used if a person is unable to place their hands flat on the device, for example if they have arthritis. The software needs to be installed on a device running Windows XP (Microsoft, WA, USA) or a later version, and that has a USB port. Data are transferred directly to a computer using the USB connection to be stored on the device’s hard drive in PDF format.

The software includes an algorithm that can determine if a person has AF, and can additionally detect if a person has bradycardia, tachycardia, sinus arrhythmia, premature ventricular contractions or right bundle branch block. The recorded ECG trace is also available for further analysis by a health-care professional. The company recommends that a 12-lead ECG device is used to confirm a case of AF detected by the RhythmPad GP device.

Zenicor-ECG

The Zenicor-ECG is a system with two components: a lead-I ECG device (Zenicor-EKG 2) and an online system for analysis and storage (Zenicor-EKG Backend System version 3.2). The online system is not locally installed; the device transmits data to a remote server that can be accessed using a web browser, without prior installation of software, and requires a user licence. ECG readings are taken by placing both thumbs on the device for 30 seconds. The instructions for use state that the electrodes in the Zenicor EKG-2 should be replaced after every 500 measurements. The device is powered by three alkaline batteries that the company states are expected to last for at least 200 measurements and transmissions.

Once a measurement is made using the Zenicor-EKG 2 device, the ECG measurement can be transferred from the device (using a built-in mobile network modem) to a Zenicor server in Sweden. Here, the ECG trace is analysed using the Zenicor-EKG Backend System, which includes an automated algorithm. The algorithm categorises an ECG into one of 12 groups corresponding to potential arrhythmias, one of which includes AF. The algorithm will also report if the recorded ECG trace cannot be analysed. The company states that a clinician needs to manually interpret the ECG trace generated by the Zenicor-ECG to make a final diagnosis of AF.

The measured ECG trace can be downloaded or printed as a PDF report. The company states that the ECG is available via the web interface approximately 4–5 seconds after the ECG has been transmitted from the device.

The company states that the Zenicor EKG-2 does not store, contain or transmit any patient-identifying information. ECGs are sent via the built-in mobile network modem to the Zenicor server labelled with the device’s identity number. Communication between the Zenicor server and the web browser accessing it is encrypted.

Statistical analysis and data synthesis

Sensitivity analyses

We had planned to conduct sensitivity analyses by excluding studies judged as having a high risk of bias or studies where the appropriateness of inclusion in the primary meta-analyses was uncertain. Sensitivity analyses stratified by risk of bias were not performed owing to the small number of studies included in the meta-analysis with similar risk-of-bias judgements.

Characteristics of the included studies

The characteristics of the nine included DTA studies are summarised in Table 3.

The studies included in the DTA review were either case–control studies38,39,43,45,47–49 or cross-sectional studies. 50,51 Two of the studies were based in the UK. 39,50 Only one study was conducted in primary care,49 with the remaining studies being conducted in either secondary39,43,45,47,48,50 or tertiary care. 38,51 All of the studies included either patients with a known history of AF or patients recruited from cardiology clinics. Only one study38 presented the reasons that patients were admitted to a cardiology department. Eleven patients (3.4%) were admitted because of symptomatic AF, all of whom had a known history of AF. The study by Haberman et al. 45 included a community-based population comprising healthy young adults and elite athletes. The results for the healthy young adults and elite athletes were excluded from the analysis because these participants did not meet the population inclusion criteria for this review and do not represent the typical population with AF (i.e. those aged > 75 years). 10 The study by Lau et al. 47 included a ‘learning set’ and data from this group were used to optimise the algorithm. The ‘learning set’ data were excluded from the analysis because, according to the author of the study (Ben Freedman, University of Sydney, 15 June 2018, personal communication), two separate cardiologists interpreted the rhythm strips, and the interpretation by cardiologist A seemed to have a bias towards sensitivity, with a resultant lower specificity, while the interpretation by cardiologist B had a slightly lower sensitivity, with a resulting higher specificity.

Only one study included results based on lead-I ECG interpretation by the device algorithm and a trained health-care professional presenting the results separately. 38 One study39 reported data for a lead-I ECG trace that was interpreted both by a cardiologist and by a GP with an interest in cardiology; the results were presented separately for each interpreter. In four studies,47–50 the lead-I ECG was interpreted by the device algorithm alone.

The lead-I ECG devices used in the included studies were Kardia Mobile,39,45,47,51 MyDiagnostick,48,49 RhythmPad GP50 and Zenicor-ECG. 43 The study by Desteghe et al. 38 used both Kardia Mobile and MyDiagnostick and presented the results separately for each device.

The trained health-care professional interpreting the 12-lead ECG in all of the studies included in the DTA review was a cardiologist,39,43,47–49,51 an EP38,45,50 or a GP with an interest in cardiology. 39

Quality assessment of diagnostic accuracy studies

All of the included studies were assessed for risk of bias and applicability using the QUADAS-2 tool. 32 A summary of the results that assessed risk of bias and applicability concerns across all studies is presented in Table 4. The full assessment for each included study is presented in Appendix 5.

All of the included studies were judged as being at an unclear risk of bias for the patient selection domain. Only one study48 reported the method used for patient inclusion. There was an overall lack of information regarding patient eligibility for participation in the studies, and whether or not any patients were excluded at the stage of study selection. All of the included studies were judged as having a high applicability concern for patient selection as none of these studies was performed in the population of interest. One study38 included a proportion (3.4%) of patients admitted to a cardiology department because of symptomatic AF; however, all of these patients had a known history of AF.

Three studies45,50,51 were judged as having an unclear risk of bias in the index test domain because there was lack of information regarding whether or not the index tests were interpreted without knowledge of the reference standard test result. The remaining six studies38,39,43,47–49 were judged as having a low risk of bias on the index test domain. Studies in which the index test was interpreted by a trained health-care professional were judged to be more applicable (low concern)38,39,43,45 than those interpreted by the lead-I ECG device algorithm alone. 47–49 Two studies50,51 were judged as having an unclear applicability concern because of a lack of information in the publication.

Three studies45,50,51 were judged as having an unclear risk of bias for the reference standard domain because they did not explicitly report whether or not the interpreters of the reference standard were blinded to the results of the index test. The reference standard for all of the included studies was the results of a 12-lead ECG, which were interpreted by a trained health-care professional; therefore, all of the studies were judged to have a low concern regarding applicability of the reference standard.

Risk of bias was judged as being unclear for three studies39,49,50 for the flow and timing domain because not all patients were included in the study analyses.

Diagnostic test accuracy results

Interpreter of lead-I electrocardiogram: trained health-care professional

All lead-I electrocardiogram devices: main analysis

We investigated the sensitivity and specificity of a lead-I ECG device when the trace was interpreted by a trained health-care professional and the reference standard was a 12-lead ECG interpreted by a trained health-care professional. Data from four studies38,39,43,45 were included in a meta-analysis. Two studies39,45 had data for Kardia Mobile alone, one study43 had data for Zenicor-ECG and one study38 had data for MyDiagnostick and Kardia Mobile. One additional study51 had data for Kardia Mobile but was not included in the pooled analysis because the numbers of true-positive, false-negative, false-positive and true-negative test results were not reported. The sensitivity and specificity values reported in this study51 were 92.8% and 100%, respectively.

Four meta-analyses were conducted to investigate the impact of using data for each combination of type of lead-I ECG device (MyDiagnostick or Kardia Mobile) and interpreter (EP1 or EP2) from the Desteghe et al. study38 from the results of the meta-analysis. Both EPs interpreted the lead-I ECG trace and the 12-lead ECG trace. The data based on the use of Kardia Mobile lead-I ECG device and interpretation by EP1 were randomly selected to be included in the main analysis. Additional meta-analyses are presented as sensitivity analyses (see Appendix 6, Figure 13).

One study39 reported data for one lead-I device (Kardia Mobile) and two different interpreters (a cardiologist and a GP with an interest in cardiology) of lead-I and 12-lead ECG traces. The data interpreted by the cardiologist were used in the main analysis because the interpreters in the other included studies were either cardiologists or EPs. The analysis with data interpreted by the GP is presented as a sensitivity analysis (see Appendix 6, Figure 17).

A forest plot displaying the results of the individual studies included in the meta-analysis is presented in Figure 4.

FIGURE 4.

Forest plot of individual studies included in the meta-analysis of all lead-I ECG devices; trace interpreted by a trained health-care professional (Kardia Mobile and EP1 data from the Desteghe study). FN, false negative; FP, false positive; TN, true negative; TP, true positive. Reproduced from Duarte et al. 67 © The Authors, 2019. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Reproduced from Duarte et al. 67 © The Authors, 2019. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

A SROC plot that displays the individual study results as well as the meta-analysis result is presented in Figure 5. A visual inspection of Figure 4 and the individual study results presented in Figure 5 shows that the results were relatively homogeneous across the included studies in this meta-analysis. However, owing to some potential heterogeneity between studies, we adopted a conservative approach and used a bivariate model with random effects in the meta-analysis.

FIGURE 5.

Summary receiver operating characteristic plot for lead-I ECG device as index test with trace interpreted by a trained health-care professional and 12-lead ECG interpreted by a trained health-care professional as reference standard (using Kardia Mobile lead-I ECG device and EP1 data from the study by Desteghe et al. 38).

This meta-analysis included 580 participants, of whom 118 had AF. The pooled sensitivity was 93.9% (95% CI 86.2% to 97.4%) and the pooled specificity was 96.5% (95% CI 90.4% to 98.8%).

All lead-I electrocardiogram devices: sensitivity analyses

Forest plots displaying the results of the individual studies included in the meta-analyses are presented in Appendix 6, Figure 13.

Summary receiver operating characteristic plots are presented in Appendix 6, Figure 14. A visual inspection of the forest plots (see Appendix 6, Figure 13) and the individual study results (see Appendix 6, Figure 14) shows that the results were relatively homogeneous across the included studies in these meta-analyses. However, owing to some potential heterogeneity between studies, we adopted a conservative approach and used a bivariate model with random effects in the meta-analysis.

Pooled sensitivity values from these additional meta-analyses ranged from 89.8% to 91.8%, while pooled specificity values ranged from 95.6% to 97.1% (Table 5). Overall, the use of either Kardia Mobile or MyDiagnostick lead-I ECG and an interpretation by different EPs does not seem to make a difference to the pooled results. Considering only the study by Desteghe et al. ,38 specificity is similar across all combinations, whereas the sensitivity of Kardia Mobile seems higher than the sensitivity of MyDiagnostick and EP1 seems to show slightly higher sensitivity than EP2.

TABLE 5 - Results from the meta-analyses of all lead-I ECG devices (trace interpreted by a trained health-care professional)

Data input from the study by Desteghe et al.38	AF cases (n)	Total number of patients (N)	Pooled sensitivity (95% CI)	Pooled specificity (95% CI)
MyDiagnostick device and EP1 data	118	582	90.8% (83.8% to 95.0%)	95.6% (89.4% to 98.3%)
MyDiagnostick device and EP2 data	118	582	89.8% (82.7% to 94.1%)	96.8% (90.6% to 99.0%)
Kardia Mobile device and EP2 data	120	584	91.8% (85.1% to 95.7%)	97.1% (90.8% to 99.1%)

One study39 also presented data for one lead-I device (Kardia Mobile) that were interpreted by a GP with an interest in cardiology, and these data were included in a sensitivity analysis. The forest plot displaying the results of the individual studies included in the meta-analysis is presented in Appendix 6, Figure 17.

The SROC plot that shows the individual study results as well as the meta-analysis result is presented in Appendix 6, Figure 18. When the results presented in Appendix 6, Figure 17, and the individual study results presented in Appendix 6, Figure 18, were studied, they were found to be relatively homogeneous; however, in the study by Williams et al. ,39 specificity was lower when the lead-I ECG trace was interpreted by the GP (76%), than when it was interpreted by a cardiologist (86%) (see Figure 4). Owing to some potential heterogeneity between studies, we adopted a conservative approach and used a bivariate model with random effects in the meta-analysis.

For this meta-analysis (number of AF cases, 118; total number of patients, 580), the sensitivity was 94.3% (95% CI 87.9% to 97.4%) and the specificity was 96.0% (95% CI 85.4% to 99.0%).

Kardia Mobile lead-I electrocardiogram device

Data for the Kardia Mobile device were derived from only three studies. 38,39,45 We conducted two meta-analyses to investigate the impact of using data for each interpreter (EP1 or EP2) from the study by Desteghe et al. 38 on the results of the meta-analysis. Forest plots displaying the results of the individual studies included in each meta-analysis are presented in Appendix 6, Figure 19.

For both meta-analyses, we fitted a univariate random-effects logistic regression model for specificity and a univariate fixed-effects logistic regression model for sensitivity because minimal variability in sensitivity was observed across the studies.

For the meta-analysis that included EP1 data from the study by Desteghe et al. 38 (number of AF cases, 67; total number of patients, 480), sensitivity was 94.0% (95% CI 85.1% to 97.7%) and specificity was 96.8% (95% CI 88.0% to 99.2%). For the meta-analysis that included EP2 results from the study by Desteghe et al. 38 (number of AF cases, 69; total number of patients, 484), sensitivity was lower, at 91.3% (95% CI 82.0% to 96.0%), and specificity was slightly higher, at 97.4% (95% CI 88.3% to 99.5%). As only three studies38,39,45 were included in this analysis, it was not possible to produce confidence regions.

There were insufficient data to generate pooled estimates of sensitivity and specificity for other types of lead-I ECG device based on the interpreter of the lead-I ECG being a trained health-care professional, or to assess differences between types of devices. The sensitivity and specificity estimates for Zenicor-ECG and MyDiagnostick lead-I ECG devices are presented in Appendix 6, Figure 13.

Interpreter of lead-I electrocardiogram: algorithm

All lead-I electrocardiogram devices

We investigated the sensitivity and specificity of the lead-I ECG device when the trace was interpreted by the device algorithm alone. The reference standard used was interpretation of the 12-lead ECG trace by a trained health-care professional. Data from four studies38,47–49 were included in a meta-analysis. Two studies48,49 had data for MyDiagnostick alone,48,49 one study47 had data for Kardia Mobile alone and one study38 had data for MyDiagnostick and Kardia Mobile. One study50 reported sensitivity (67%) and specificity (97%) for RhythmPad GP. Although the authors of this study50 provided the numbers for true-positive, false-negative, false-positive and true-negative test results, these were not included in the pooled analysis because the authors reported that the algorithm had since been modified (Chris Crockford, CardioCity, 3 August 2018, personal communication via NICE). We conducted two meta-analyses in order to investigate the impact of using data for each type of lead-I ECG device (MyDiagnostick or Kardia Mobile) from the study by Desteghe et al. 38 on the results of the initial meta-analysis. In the study by Desteghe et al. ,38 the same patient cohort was tested using both lead-I ECG devices. We performed multiple analyses in order to investigate the impact of varying the type of lead-I ECG device on the results of the overall pooled analysis with no set of patients double-counted. Forest plots displaying the results of the individual studies included in each meta-analysis are presented in Figure 6.

FIGURE 6.

Forest plots of individual studies included in each meta-analysis of all lead-I ECG devices (trace interpreted by the device algorithm). (a) MyDiagnostick data from the Desteghe study; and (b) Kardia Mobile data from the Desteghe study. FN, false negative; FP, false positive; TN, true negative; TP, true positive.

The SROC plots are presented in Appendix 6, Figures 20 and 21. The results were relatively homogeneous across the included studies in both meta-analyses. However, owing to some potential heterogeneity between studies, we adopted a conservative approach and used a bivariate model with random effects in the meta-analysis.

For the meta-analysis that included MyDiagnostick data from the study by Desteghe et al. 38 (number of AF cases, 219; total number of patients, 842), sensitivity was 96.2% (95% CI 86.0% to 99.0%) and specificity was 95.2% (95% CI 92.9% to 96.8%). For the meta-analysis that included Kardia Mobile data from the study by Desteghe et al. 38 (number of AF cases, 219; total number of patients, 842), the pooled estimates for sensitivity were 95.3% (95% CI 70.4% to 99.4%) and for specificity were 96.2% (95% CI 94.2% to 97.6%), which were similar to those obtained from the meta-analysis including MyDiagnostick data from the study by Desteghe et al. 38

MyDiagnostick lead-I electrocardiogram device

A forest plot displaying the results of the individual studies included in this meta-analysis is presented in Appendix 6, Figure 22.

As only three studies38,48,49 were included in this analysis, it was not possible to produce a SROC plot with a confidence region.

For MyDiagnostick, data from three studies38,48,49 (number of AF cases, 171; total number of patients, 638) were included in the meta-analysis; sensitivity was 95.2% (95% CI 79.0% to 99.1%) and specificity was 94.4% (95% CI 91.9% to 96.2%). For this meta-analysis, we fitted a univariate random-effects logistic regression model for sensitivity and a univariate fixed-effect logistic regression model for specificity because minimal variability in specificity was observed across the studies. The results were relatively homogeneous across the three included studies.

Kardia Mobile lead-I electrocardiogram device

We estimated sensitivity and specificity for the Kardia Mobile device, and for the MyDiagnostick device separately. A forest plot displaying the results of the individual studies included in this meta-analysis is presented in Appendix 6, Figure 23. In the study by Desteghe et al. ,38 sensitivity was 55% (95% CI 32% to 76%), much lower than that in the study by Lau et al. ,47 which was 98% (95% CI 89% to 100%).

As only two studies38,47 were included in this analysis, it was not possible to produce a SROC plot with a confidence region.

For Kardia Mobile, data from two studies (number of AF cases, 70; total number of patients, 469) were included in the meta-analysis; sensitivity was 88.0% (95% CI 32.3% to 99.1%), and specificity was 97.2% (95% CI 95.1% to 98.5%). For this meta-analysis, we fitted a univariate random-effects logistic regression model for sensitivity and a univariate fixed-effect logistic regression model for specificity, because minimal variability in specificity was observed across the studies.

Data were not sufficient to pool estimates of sensitivity and specificity for other types of lead-I device based on the interpreter of the lead-I ECG being a trained health-care professional, or to formally assess the differences between types of devices.

Summary of findings: diagnostic test accuracy

No studies were identified that evaluated the DTA of lead-I ECG devices in people presenting to primary care with signs or symptoms of AF and an irregular pulse.

Of the nine included studies, only one study49 was conducted in primary care. The remaining eight studies were conducted in secondary care, tertiary care or community settings.

Of the nine included studies, only one study38 explicitly stated that some patients (n = 11, 3.4%) had signs or symptoms of AF on admission to a cardiology ward. Another study49 included a large proportion of people with known AF (83.4%); however, it is not clear whether or not the patients had signs or symptoms of AF at the time of the assessment and/or if the patients had been previously diagnosed with AF.

As prespecified in the protocol,68 owing to a lack of evidence, we next focused the reviews on an asymptomatic population in any health-care setting. We considered an asymptomatic population to be people not presenting with signs or symptoms of AF, with or without a previous diagnosis of AF. These patients could have had co-existing cardiovascular conditions or could have been attending a cardiovascular clinic but did not present with signs or symptoms of AF. We identified 13 publications38,39,41–51 reporting on nine studies assessing the DTA of lead-I ECG devices in an asymptomatic population. However, all of these studies were judged as having a high applicability concern for patient selection, as none was performed in the population and setting of interest.

We included studies in which the interpreter of the lead-I ECG trace was a trained health-care professional38,39,43,45,51 and studies that included interpretations of the lead-I ECG trace by the lead-I ECG device algorithm only. 38,47–50 The lead-I ECG devices used in the studies were Kardia Mobile,39,45,47 MyDiagnostick48,49 and Zenicor-ECG. 43 The study by Desteghe et al. 38 used both Kardia Mobile and MyDiagnostick.

The results from the meta-analyses are summarised in Table 6. Across all meta-analyses where the interpreter of the lead-I ECG trace was a trained health-care professional, the sensitivity ranged from 89.8% to 94.3% and the specificity ranged from 95.6% to 97.4%. Across all meta-analyses where the interpreter of the lead-I ECG trace was the device algorithm, the sensitivity ranged from 88% to 96.2% and the specificity ranged from 94.4% to 97.2%. Pooled sensitivity and specificity values were similar across the different meta-analyses, irrespective of the interpreter of the lead-I ECG trace or the lead-I ECG device used. However, it should be noted that studies in which the index test was interpreted by the lead-I ECG device algorithm alone were judged as having a high applicability concern for the index test domain. This judgement was based on the consideration made by all manufacturers of the lead-I ECG devices that the diagnosis of AF should not be made using the algorithm alone, and that the ECG traces measured by the devices should be reviewed by a qualified health-care professional.

Details of the excluded studies that report sensitivity and specificity values for the lead-I ECG devices investigated in this assessment, and the reasons for exclusion, are presented in Appendix 7.

Characteristics of the included studies

The characteristics of the 18 quantitative studies included in the clinical impact review are summarised in Table 7. One qualitative study53 included in the clinical impact review reported the results of semistructured interviews with patients, receptionists, practice nurses and GPs.

Eleven of the studies included in the clinical impact review were cross-sectional studies,51,52,54,56–59,61,63–65 seven were case–control studies38,43,45,47,48,60,62 and one study was qualitative. 53 Seven studies were conducted in primary care,53,54,57–60,62 five were conducted in secondary care,43,47,48,63,65 two were conducted in tertiary care38,51 and the remaining four were conducted in a community setting. 52,56,61,64 One study45 included participants recruited from secondary care, but also included (as separate groups) elite athletes and healthy young adults. As discussed in Characteristics of the included studies, the results for these populations45 were excluded from the analysis as these participants did not meet our inclusion criterion for population and do not represent the typical population with AF (i.e. those aged ≥ 75 years).

Four studies included only people without known AF. 52,58,61,65 Three studies54,59,64 may have included only people without known AF as either participants were attending a primary care clinic or the study was conducted in a community setting. However, these studies were available only as conference abstracts and did not provide sufficient information to enable us to determine whether or not the population had a history of AF. The remaining 11 studies38,43,45,47,48,51,56,57,60,62,63 recruited people with known AF or cardiovascular comorbidities or who were attending a clinic for cardiovascular-related reasons.

Quality assessment

The methodological quality of the four cross-sectional52,56,57,61 and the two case–control studies60,62 included in the clinical impact review of lead-I ECG devices, was assessed using the Newcastle–Ottawa quality assessment scale. 33,34 The results of the quality assessment of cross-sectional and case–control studies are presented in Appendix 8, Table 40.

The methodological quality of the diagnostic accuracy studies included in the clinical impact review was assessed using the QUADAS-2 tool. 32 A summary of the results for the risk of bias in the studies38,43,45,47,48,51 that were included in the clinical impact review but had already been assessed as part of the DTA review is presented in Table 4 and a full assessment is reported in Appendix 5. A summary of the risk of bias for one diagnostic accuracy study63 not eligible for inclusion in the DTA review is presented in Appendix 8, Table 41; the full quality assessment for this study63 is presented in Appendix 5.

Five studies54,58,59,64,65 that were available only as conference abstracts and were assessed as meeting the study eligibility criteria for inclusion in the clinical impact review were subjected to data extraction only and not to quality assessment, because there was not enough information to allow a judgement to be made on some of the quality assessment criteria.

Overall, the quality of the four cross-sectional52,56,57,61 and the two case–control studies60,62 was similar across the different domains. None of the included studies was considered to be representative of the target population. Only one study61 included a sample size calculation. In all studies, the test failure rate was low; therefore, the response rate was considered satisfactory. All of the included studies described the intervention. None of the studies accounted for confounding factors in the analyses presented. An assessment of the outcome was described in all of the studies; however, those studies with independent blind assessment or record linkage were judged as being of better quality than the studies without blind assessment or record linkage. The statistical tests used to analyse the data were clearly described and appropriate in all included studies.

The diagnostic accuracy study63 was judged as being at an unclear risk of bias and having a high applicability concern for patient selection. This study63 was judged as being at low risk of bias on the index test domain as the test results were interpreted without knowledge of the reference standard test result and, therefore, there was also low applicability concern for this domain. All of the interpreters of the reference standard test results were blind to the results of the index test; therefore, the study63 was judged as being at low risk of bias for the reference standard domain. However, there were two reference standards: (1) a clinical ECG diagnosis based on additional information not available to the assessors, and (2) consensus (three of the four assessors) that matched this clinical ECG diagnosis. Therefore, this study63 was judged as having a high concern regarding applicability of the reference standard test.

The methodological quality of the qualitative study53 included in the clinical impact review was assessed using the CASP tool36 and the results are presented in Appendix 8, Table 42. In the qualitative study,53 semistructured interviews were conducted with two receptionists, one nurse, three GPs and eight patients across three GP practices. The aim of the study was to investigate the feasibility of using practice nurses and receptionists to systematically screen patients aged ≥ 65 years for AF using a lead-I ECG device (Kardia Mobile) prior to the GP consultation. No details were available about the selection of the interviewees; although the study aim was to investigate the feasibility for practice nurses and receptionists to use the lead-I ECG device, these were the least represented groups in the interviews. The researchers do not discuss their own potential biases, such as relationships with participants or choice of locations for the study. Although the methods are not described in depth, the publication clearly states how the interviews were analysed and how themes were derived from the data and that interviews ceased once information saturation was reached. The duration of the interviews ranged from 5 to 40 minutes. Considering that there were four different groups of participants (i.e. receptionists, nurses, GPs and patients), it is unclear how information saturation was reached, especially for nurses’ views, as only one nurse was interviewed.

Clinical impact results

Summary of findings: clinical impact

As per the DTA review, no studies were identified that evaluated the clinical impact of lead-I ECG devices in people presenting to primary care with signs or symptoms of AF and an irregular pulse, which limits the applicability of the results presented. Therefore, the 24 publications38,41–48,51–65 reporting on the 19 studies that were included in the clinical impact review were focused on an asymptomatic population. Four studies included only people without known AF. 52,58,61,65 Three studies54,59,64 may have included only people without known AF, as either participants were attending a primary care clinic or the study was conducted in a community setting. However, the information describing these studies was limited and the data were available only as conference abstracts.

Test failure rate was reported in nine studies38,45,52,56,57,60–63 and ranged from 0.1% to 9%. Results for test failure rate included both failure of the lead-I ECG algorithm to produce a result and poor quality of the lead-I ECG trace. Diagnostic yield was reported in 13 studies. 38,48,52,54,56–62,64,65 The percentage of new patients diagnosed with AF ranged from 0.38% to 5.84%. Two studies59,61 reported a change in treatment management following the use of the Kardia Mobile lead-I ECG for new patients diagnosed with AF. Acceptability of lead-I ECG devices was reported in four studies,54,58,59,62 with generally positive views. Time to initiation of preventative treatment and HRQoL were not reported in any of the identified studies.

The ‘real-world’ data submitted by Kent Surrey Sussex AHSN reports on the use of Kardia Mobile lead-I ECG device for people with signs or symptoms of AF and an irregular pulse during a 2-year project. Although the information available was limited [Microsoft PowerPoint presentation (Microsoft Corporation, Redmond, WA, USA) and a one-page summary], we considered it relevant to the population of interest. Data from this 2-year project showed that the percentage of new patients diagnosed with AF during the project was 69.9%, which is considerably higher than the diagnostic yield reported in our included studies (0.4–5.8%).

Search strategy

The EAG undertook a systematic review to identify published full economic evaluations of lead-I ECG devices for detecting AF. A search filter to identify economic evaluations was applied to the search strategies and the electronic databases were searched from inception until 24 April 2018. The search strategy used in MEDLINE is presented in Appendix 10. The MEDLINE search was adapted to enable similar searching of the other relevant electronic databases. The following databases were searched for relevant studies:

• MEDLINE (via Ovid)

• MEDLINE Epub Ahead of Print and MEDLINE In-Process & Other Non-Indexed Citations (via Ovid)

• EMBASE (via Ovid)

• PubMed

• EconLit (via EBSCOhost)

• NHS Economic Evaluation Database.

The results of the searches were uploaded to, and managed using, EndNote X8 software. The reference lists of relevant systematic reviews and eligible studies were hand-searched to identify further potentially relevant studies.

Broader searches were carried out to identify existing economic models of ECG devices when used for the detection of AF. Separate searches were carried out to identify supporting information on costs and health-state utility data.

Eligibility criteria

In stage 1, all titles and abstracts identified via searches of the electronic databases were screened for relevance according to prespecified eligibility criteria (Table 9). Any studies that did not meet the criteria were excluded. The EAG planned to obtain full-text manuscripts for all economic evaluations identified at stage 1 to assess relevance against the prespecified eligibility criteria (stage 2).

Data extraction and quality assessment strategy

The EAG planned to extract data relating to bibliographic information [author(s) and year of publication]; general information [country, condition, intervention and comparator(s)]; methodological characteristics (type of economic evaluation, perspective, time horizon, discount rate, key cost categories, year of valuation and key outcomes) and main findings. The EAG planned to assess the quality of all economic evaluations identified for inclusion in the review using the Drummond 10-point checklist. 69

Results of the systematic review of existing cost-effectiveness evidence

The searches of electronic databases identified 40 unique citations after de-duplication. Following screening of titles and abstracts, all 40 records were excluded as they did not include the relevant interventions or comparator, did not consider an eligible study population or were not full economic evaluations.

Conclusions of the systematic review of cost-effectiveness evidence

The EAG did not identify any published papers that met the inclusion criteria for the systematic review.

Approach to modelling

The EAG did not identify any studies in a systematic review of the economic literature that evaluated the cost-effectiveness of single time point lead-I ECG devices compared with MPP followed by a 12-lead ECG in primary or secondary care (prior to initiation of anticoagulation therapy) in people presenting to primary care with signs or symptoms of AF who have an irregular pulse. The EAG therefore undertook a de novo economic analysis.

The economic analysis follows the diagnostic pathway for patients presenting to primary care with signs or symptoms indicative of AF plus an irregular pulse. The results are presented over a time horizon of 30 years, with patients entering the model at the age of 70 years.

The economic evaluation is relevant only to primary care practices where patients have to wait ≥ 48 hours between an initial consultation with the GP and a 12-lead ECG; this allows the benefit of early anticoagulation and rate-control treatment for those patients who receive a positive lead-I ECG to be considered.

A decision tree and two cohort Markov models were built in Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA). The decision tree describes the pathway that a patient presenting to primary care with signs or symptoms of AF and an irregular pulse follows in the initial GP consultation. The first Markov model captures the differences in the costs and benefits of treatment (standard diagnostic pathway compared with lead-I ECG pathway) during the first 3 months after the initial appointment. During this period, some patients will have a diagnosis of AF and start treatment for AF, whereas other patients will have further tests to diagnose or to rule out AF (where ‘rule out’ means that no diagnosis of AF is recorded in the patient’s notes and no treatment for AF is started). The second Markov model captures the differences in lifetime costs and benefits after diagnosis of AF or the time when AF is ruled out.

Population

The modelled patient population is adults presenting to primary care with signs or symptoms of AF who have an irregular pulse. The DTA data included in the model are based on the results of a systematic review (see Chapter 3, Study selection). However, no studies included in the systematic review were carried out in the population of interest. All studies included asymptomatic patients who either had a known history of AF or were recruited from cardiology clinics, except for one study49 that was carried out in primary care. It has been recognised that diagnostic accuracy test specificity and sensitivity values may be affected by prevalence; the use of a test in a more severely diseased population is associated with better performance of the test. 70 It is therefore possible that the sensitivity and specificity data from the systematic review do not represent the true DTA of lead-I ECG devices in the population of interest. It is not possible to know how the sensitivity and specificity of lead-I ECG devices would be affected if different populations were tested. The economic evaluation is therefore limited by the lack of DTA data in the population of interest.

The symptomatic population with an irregular pulse is assumed to consist of people with AF and people with atrial or ventricular ectopy. Clinical advice to the EAG is that the only other condition that would produce an irregular pulse similar to that found with AF is atrial or ventricular ectopy. It is assumed that the symptoms of patients with AF, or atrial or ventricular ectopy, are not severe enough to require urgent referral to cardiology. Advice from the NICE Clinical Knowledge Summary71 on managing atrial and ventricular ectopy for patients without underlying heart disease is to reassure patients.

The mean age of patients in the base-case model is 70 years. The proportions of men and women are based on the age-adjusted ratio in the general population. 72

Comparators

Diagnostic test accuracy data were not available for the population of interest (symptomatic patients with suspected AF and an irregular pulse presenting to primary care) for any of the devices listed in the final scope issued by NICE9 (see Chapter 3, Characteristics of the included studies). The EAG therefore searched for DTA data in an asymptomatic population as prespecified in the protocol to use as a proxy for the population of interest. The economic model includes only the diagnostic strategies for which proxy DTA data were available. The diagnostic strategies (following MPP and before 12-lead ECG) included in the economic model are:

• standard diagnostic pathway (no further testing)

• any lead-I ECG device (interpreted by trained health-care professional)

• imPulse (interpreted by trained health-care professional)

• Kardia Mobile (interpreted by trained health-care professional)

• MyDiagnostick (interpreted by trained health-care professional)

• RhythmPad-GP (interpreted by algorithm)

• Zenicor-ECG (interpreted by trained health-care professional).

Model structure

The model comprises decision trees and two cohort Markov models that describe the patient pathway over a lifetime horizon of 30 years. A decision tree covers the patient pathway in the initial consultation. Patients then feed into a cohort Markov structure with daily cycles for 3 months. This first Markov model includes all testing for AF after the initial GP consultation (12-lead ECG and Holter monitoring for paroxysmal AF). By the end of the first 3-month Markov model, all patients either have an AF diagnosis or have AF ruled out. Patients then move into the second Markov model. All patients in the second Markov model have AF diagnosed or ruled out (either correctly or incorrectly). Patients remain in the second Markov model until death. The cycle length is 3 months in the second Markov model. Costs and benefits are discounted at 3.5% per year.

Diagnostic phase

The diagnostic phase of the model encompasses the initial consultation and the following 3 months. At the end of the first 3-month period in the model, all patients who remain alive have had AF either diagnosed or ruled out (whether correctly or incorrectly; here ‘ruled out’ means that no diagnosis of AF is recorded in the patient’s notes and no treatment for AF has started).

A decision tree structure describes the pathway that a patient presenting to primary care with signs or symptoms of AF and an irregular pulse follows in the initial GP consultation. Patients then enter a cohort Markov model either in a testing state (while waiting for the results of a 12-lead ECG or paroxysmal test) or in a diagnosed state (AF diagnosed or ruled out). Patients may stay in the testing period for a maximum number of days, depending on the test. Clinical advice to the EAG is that patients who cannot have a 12-lead ECG in the GP practice immediately would have to wait between 2 and 14 days for the test. Patients receiving testing for paroxysmal AF using a Holter monitor will stay in the paroxysmal testing state for 7 days.

At the end of the testing period, patients who received a 12-lead ECG may move to another testing state (paroxysmal test), to a diagnosed state (AF diagnosed or ruled out) or to the death state. At the end of the testing period, patients receiving a paroxysmal test may move to a diagnosed state (AF diagnosed or ruled out) or to the death state.

Patients may move out of a testing state before the end of the testing period by experiencing a cardiovascular event (CVE) or death. Patients who experience a CVE and who have not had AF diagnosed or ruled out are assumed to receive a diagnosis as part of treatment for the CVE. CVEs included in the model are TIA, ischaemic stroke (IS) and haemorrhagic stroke (HS). Patients can experience up to two CVEs. Clinically relevant adverse events (AEs) are included in the model (e.g. non-major bleeds). An AE can be experienced in any state and does not affect the risk of transition to another state.

The schematics for the decision tree element of the diagnostic phase of the model are shown in Figures 7–9. The schematic for the Markov element of the diagnostic phase of the model is shown in Figure 10.

FIGURE 7.

Diagnostic phase: decision tree – standard diagnostic pathway.

FIGURE 8.

Diagnostic phase: decision tree – lead-I ECG diagnostic pathway (positive result).

FIGURE 9.

Diagnostic phase: decision tree – lead-I ECG diagnostic pathway (negative result).

FIGURE 10.

Diagnostic phase: Markov model. Note: transition to the death state is possible from all health states.

Standard pathway

All patients in the standard pathway are sent for a 12-lead ECG. No patients receive treatment for AF while waiting for the 12-lead ECG test.

All patients with a positive result from a 12-lead ECG are assumed to be correctly diagnosed with AF and begin treatment. A proportion of patients with a negative result from the 12-lead ECG are sent for further testing for paroxysmal AF and a proportion of patients have AF ruled out at this point in the pathway. All patients with a positive result from further testing for paroxysmal AF with a Holter monitor are correctly diagnosed with AF and begin treatment. All patients with a negative result from a paroxysmal test have AF ruled out. A proportion of patients who have AF ruled out after either a 12-lead ECG or a paroxysmal test will have false-negative results owing to patients with paroxysmal AF not being in AF at the time of the 12-lead ECG or paroxysmal test.

Lead-I electrocardiogram pathway: positive result

All patients in the lead-I ECG pathway with a positive result from a lead-I ECG (who are either true positives or false positives for AF) are diagnosed with AF and sent for a 12-lead ECG following the initial consultation. Clinical advice to NICE, as reported in the final scope,9 is that a 12-lead ECG is important so that any additional abnormalities can be identified in people diagnosed with AF, such as left ventricular hypertrophy. All patients in the lead-I ECG pathway with a positive result from a lead-I ECG begin rate-control treatment for AF, but do not receive Holter monitoring to test for paroxysmal AF, before the 12-lead ECG. As per the final scope issued by NICE,9 patients with positive lead-I ECG test results begin rate-control treatment and NOACs after the initial GP consultation unless contraindicated.

Patients with a positive 12-lead ECG result retain the (correct) diagnosis of AF and continue treatment. Patients with a negative 12-lead ECG result are assumed to have paroxysmal AF and continue treatment, have AF ruled out and discontinue treatment for AF or are sent for further testing for paroxysmal AF. The last group of patients remains on treatment during further testing.

All patients with a positive result from a paroxysmal test are correctly diagnosed with AF and stay on treatment. Patients with a negative result from a paroxysmal test either have AF ruled out and discontinue treatment or are assumed to have paroxysmal AF based on the original lead-I ECG diagnosis and continue treatment despite the negative 12-lead ECG and paroxysmal test result. A proportion of patients who have AF ruled out after either a 12-lead ECG or a paroxysmal test will have false-negative results owing to patients with paroxysmal AF not being in AF at the time of the 12-lead ECG or paroxysmal test.

Lead-I electrocardiogram pathway: negative result

No patients begin treatment following a negative result from a lead-I ECG test. Clinical advice to the EAG about whether patients who receive a negative result from a lead-I ECG device would be sent for a 12-lead ECG or for further testing for paroxysmal AF (see Appendix 11) indicated substantial variation in clinical practice. The EAG has assumed in the base case that 80% of patients who receive a negative result from a lead-I ECG device would be sent for a 12-lead ECG, 10% would be sent for ambulatory Holter monitoring and the remaining 10% of patients would have AF ruled out. The EAG acknowledges that this base case may not represent clinical practice anywhere in the UK; however, it considers that these assumptions may represent ‘average’ clinical practice, given the variation in clinical advice received. These assumptions are tested in scenario analyses.

All patients with a positive result from a 12-lead ECG are correctly diagnosed with AF and begin rate-control treatment and NOACs. A proportion of patients with a negative result from the 12-lead ECG are sent for further testing for paroxysmal AF and a proportion of patients have AF ruled out at this point in the pathway. All patients with a positive result from a paroxysmal test are correctly diagnosed with AF and begin treatment. All patients with a negative result from a paroxysmal test have AF ruled out. A proportion of patients who have AF ruled out after either a 12-lead ECG or a paroxysmal test will have false-negative results owing to patients with paroxysmal AF not being in AF at the time of testing.

Post-diagnostic phase

Once AF has been either diagnosed or ruled out, patients move into a second cohort Markov model that tracks the costs and benefits of these decisions over their lifetime (Figure 11). The second Markov model follows the same structure as the first Markov model after AF has been diagnosed or ruled out. Patients enter the second Markov model in a diagnosed state (AF diagnosed or ruled out) having experienced zero, one or two CVEs. In each cycle, patients with zero or one previous CVE can remain in their current state, move to a worse state following a CVE or move to the death state. Patients with two previous CVEs remain in that state until death. Patients who have incorrectly had AF ruled out and experience a CVE are assumed to have their AF diagnosed as part of the treatment for the CVE. These patients then move on to treatment for AF.

FIGURE 11.

Post-diagnostic phase: Markov model. Note: death is possible from all states.

Model parameters

Uncertainty

Uncertainty in parameter values and the impact this could have on results has been explored both through the scenario and through the sensitivity analyses. Parameters have been varied through probability sensitivity analysis parameters, where probability distributions could be derived from, or were provided in, the literature. Probabilistic sensitivity analysis results have been presented as cost-effectiveness acceptability curves (CEACs) where different willingness to pay (WTP) thresholds for a quality-adjusted life-year (QALY) are used to show which strategy is likely to have the largest net benefit for that threshold.

Interpreting results

Base case 1: 12-lead ECG in primary care, 2 days to 12-lead electrocardiogram

Costs and QALYs generated in base case 1 are shown in Tables 34 and 35, respectively.

Pairwise cost-effectiveness results from the base case 1 analysis for each index test versus the standard diagnostic pathway are presented in Table 36 and incremental analysis results are shown in Table 37.

Summary of base-case cost-effectiveness results

The results of the pairwise analysis show that all lead-I ECG tests lie on the efficiency frontier in each of the four base-case analyses, with ICERs below the £20,000–30,000 threshold usually considered to be cost-effective by NICE. Kardia Mobile is the most cost-effective option in a full incremental analysis, with an ICER no higher than £1060 per QALY gained, compared with the standard pathway and dominates the other lead-I ECG devices (costing less and generating more QALYs).

Lead-I ECG devices are more cost-effective when there is a longer wait to 12-lead ECG (as treatment for AF with a lead-I ECG device is assumed in the model to start earlier than in the standard pathway) and if the 12-lead ECG is performed in hospital. The majority of the patient benefit, however, comes after diagnosis, when a greater proportion of patients are correctly diagnosed with AF and treated for AF than in the standard diagnostic pathway, even if this benefit is slightly offset by an increase in patients incorrectly diagnosed with AF with a lead-I ECG device.

Summary of scenario and sensitivity analyses cost-effectiveness results

The one-way sensitivity analysis showed that the results were sensitive to the assumed prevalence of paroxysmal AF versus persistent and permanent AF. Decreased prevalence of paroxysmal AF increased incremental costs and decreased incremental QALYs for lead-I ECG devices versus the standard pathway. At the extreme, where the prevalence of paroxysmal AF was assumed to be zero, incremental QALYs decreased sufficiently to become negative and resulted in some lead-I ECG devices (imPulse, MyDiagnostick and RhythmPad) being dominated by the standard pathway. The ICERs per QALY gained yielded for other lead-I ECG devices when the prevalence of paroxysmal AF was zero were very large as a result of very small incremental QALYs. When the prevalence of paroxysmal AF was assumed to be 1, incremental costs decreased and incremental QALYs decreased. Increasing the prevalence of paroxysmal AF to 1 resulted in all lead-I ECG devices except imPulse and MyDiagnostick dominating the standard pathway.

The results of the probabilistic sensitivity analysis indicate that, in pairwise comparisons, all lead-I ECG devices included in this assessment were cost-effective in at least 50% of iterations with a WTP threshold of around £15,000 per QALY. When all of the devices were considered together, at a threshold of £20,000 per QALY, just over 80% of iterations showed that Kardia Mobile would be the most cost-effective option, with Zenicor-ECG being the most cost-effective in around 15% of iterations. The standard pathway was found to be the most cost-effective option in zero iterations at a WTP threshold of £20,000 per QALY.

The scenario analysis showed that the results were sensitive to using alternative sensitivity and specificity values for MyDiagnostick. MyDiagnostick yielded the lowest overall costs of all the strategies when sensitivity and specificity estimates from interpretation of the MyDiagnostick lead-I ECG trace by EP2 were used. Kardia Mobile remained the index test with the highest overall QALYs in this scenario, which yielded an ICER per QALY gained of £5503 versus MyDiagnostick (using EP2).

The scenario analysis showed that results were invariant to the following assumptions:

• removing the cost of the lead-I ECG device from the analysis

• patients with AF incorrectly ruled out are not diagnosed with AF prior to a CVE

• removal of 12-lead ECG and Holter monitoring from the lead-I ECG pathway

• shortening the time horizon to 5 years.

The finding that removal of the 12-lead ECG and Holter monitoring from the lead-I ECG pathway did not affect cost-effectiveness results is unsurprising; if a patient had paroxysmal AF, they were assumed to be in AF at the time of lead-I ECG monitoring and therefore the majority of paroxysmal AF would be detected with a lead-I ECG device without the need for 12-lead ECG or Holter monitoring. However, this result should be interpreted with caution as the potential further benefits of a specific diagnosis of paroxysmal AF or of the more detailed diagnosis from 12-lead ECG testing was not considered in the model. Similarly, the extensive scenario analyses on the use of Holter monitoring following 12-lead ECG tests, with or without lead-I ECG testing, showed that, if Holter monitoring is not routinely used for the majority of patients with a negative 12-lead ECG, Kardia Mobile will always have an ICER below £10,000 per QALY gained compared with the standard pathway and, in some circumstances, will be a dominant strategy.

Effect of sensitivity and specificity

High specificity (i.e. high true-negative rate that results in a low false-positive rate) has a greater impact on the model results than high sensitivity (i.e. high true-positive rate), although the impact of high specificity is eroded the lower the sensitivity estimate becomes. For instance, the estimate of specificity for the RhythmPad GP device in the base-case analysis is higher than that for any other device (97.0%, 95% CI 95.5% to 100.0%). However, the benefit of higher specificity for the RhythmPad GP device is eroded by an estimate of sensitivity (67.0%, 95% CI 50.5% to 100.0%) that is substantially lower than the estimate of sensitivity for any of the other devices. In contrast, the Kardia Mobile device has an estimate of specificity (96.8%, 95% CI 88.0% to 99.2%) similar to that of the RhythmPad GP device but a much higher estimate of sensitivity (94.0%, 95% CI 81.5% to 97.7%).

High specificity is important because it reduces the additional treatment costs associated with an incorrect diagnosis of AF. It is assumed in the model that people incorrectly diagnosed with AF will remain misdiagnosed for the rest of their lives, so those who begin treatment with NOACs and rate-control treatment will remain on treatment for their lifetime. No benefit is assumed from treating people without AF with NOACs and rate-control treatment, and a higher risk of bleeding is assumed as a result of treatment with NOACs. Therefore, the higher the false positive rate (i.e. the lower the specificity), the greater the costs that are accrued from the treatment itself and from treating bleeds associated with NOACs without any associated benefit from treatment.

Sensitivity is important because the earlier people with AF are diagnosed, the sooner they can begin treatment and reduce their risk of having a CVE. Low sensitivity (low true positive rate) means that many people with AF may only be identified later and so do not benefit from early NOACs and rate-control treatment. However, the impact of the sensitivity estimate is mitigated in the model by the assumption that people with undiagnosed AF will later have their AF diagnosed (and begin treatment) if they experience a CVE. This means that people with AF that is initially undiagnosed do not accrue the costs of NOACs and rate-control treatment for some months or years, which offsets some of the costs associated with their higher risk of experiencing a CVE.

Search strategy

1. Lead-I ECG.tw.

2. single lead ECG.tw.

3. (lead I or single lead or automated algorithm).tw.

4. Electrocardiography/

5. (electrocardiog* or ECG).tw.

6. 4 or 5

7. 3 and 6

8. lead I electrocardiog*.tw.

9. single lead electrocardiog*.tw.

10. 1 or 2 or 7 or 8 or 9

11. Atrial Fibrillation/

12. AF.tw.

13. (Atr* adj3 Fibrill*).tw.

14. 11 or 12 or 13

15. 10 and 14

16. Kardia Mobile.tw.

17. MyDiagnostick.tw.

18. RhythmPad.tw.

19. Zenicor-ECG.tw.

20. imPulse.tw.

21. 10 and 20

22. 15 or 16 or 17 or 18 or 19 or 21

23. Animals/not Humans/

24. 22 not 23

Crockford 201350

Desteghe 201738

Doliwa 200943

Haberman 201545

Koltowski 201751

Lau 201347

Reeves (NR)63

Tieleman 201448

Vaes 201449

Williams 201539

Search Strategy

1. Lead-I ECG.tw.

2. single lead ECG.tw.

3. (lead I or single lead or automated algorithm).tw.

4. Electrocardiography/

5. (electrocardiog* or ECG).tw.

6. 4 or 5

7. 3 and 6

8. lead I electrocardiog*.tw.

9. single lead electrocardiog*.tw.

10. 1 or 2 or 7 or 8 or 9

11. Kardia Mobile.tw.

12. MyDiagnostick.tw.

13. RhythmPad.tw.

14. Zenicor-ECG.tw.

15. imPulse.tw.

16. 10 or 11 or 12 or 13 or 14

17. 10 and 15

18. 16 or 17

19. Economics/

20. ‘costs and cost analysis’/

21. Cost allocation/

22. Cost-benefit analysis/

23. Cost control/

24. Cost savings/

25. Cost of illness/

26. Cost sharing/

27. ‘deductibles and coinsurance’/

28. Medical savings accounts/

29. Health care costs/

30. Direct service costs/

31. Drug costs/

32. Employer health costs/

33. Hospital costs/

34. Health expenditures/

35. Capital expenditures/

36. Value of life/

37. exp economics, hospital/

38. exp economics, medical/

39. Economics, nursing/

40. Economics, pharmaceutical/

41. exp ‘fees and charges’/

42. exp budgets/

43. (low adj cost).mp.

44. (high adj cost).mp.

45. (health?care adj cost$).mp.

46. (fiscal or funding or financial or finance).tw.

47. (cost adj estimate$).mp.

48. (cost adj variable).mp.

49. (unit adj cost$).mp.

50. (economic$ or pharmacoeconomic$ or price$ or pricing).tw.

51. or/19-50

52. 18 and 51