Individualised variable-interval risk-based screening in diabetic retinopathy: the ISDR research programme including RCT

Article history

The research reported in this issue of the journal was funded by PGfAR as project number RP-PG-1210-12016. The contractual start date was in February 2013. The final report began editorial review in March 2020 and was accepted for publication in June 2022. As the funder, the PGfAR programme agreed the research questions and study designs in advance with the investigators. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PGfAR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2023 Harding et al. This work was produced by Harding et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2023 Harding et al.

Background

A recent report by the World Health Organization (WHO) shows that diabetic retinopathy (DR) remains one of the most common causes of visual loss in adults worldwide. 1 Screening for DR aims to detect sight-threatening DR (STDR) and refer people with diabetes (PWD) to the hospital eye service (HES) for timely treatment. Systematic programmes of screening are universally recognised to be important in preventing visual impairment (VI). 2

In England, annual eye screening for STDR for PWD over the age of 12 years commenced in the early 1990s and developed into the National Diabetic Eye Screening Programme (NDESP), with complete coverage achieved by 2008. 3

As in many other countries, the programme screening interval was set at 12 months in England and Wales for all PWD,4 while in the HES follow-up intervals are variable. Evidence to support the delivery of the pathway is limited to screening research cohorts and with minimal input from users.

There has been a rapid increase in the prevalence of diabetes, increasing faster in low- and middle-income countries than high-income countries,5 and resources are stretched. Much evidence supporting extended intervals suggests that it is safe to screen low-risk people at longer intervals6–8 and this has been introduced in some countries. However, the underpinning evidence for this change comes from observational studies in areas with low incidence rates6,9–11 and from modelling studies,12–14 and is not conclusive. In England and Wales, extended intervals have not been adopted, largely because of safety concerns highlighted by a recent systematic review calling for a randomised controlled trial (RCT) and cost-effectiveness data,8 and recent problems in cancer screening programmes. 15

The emerging technologies of digital data linkage and risk prediction offer opportunities to personalise the approach to screening. By varying the frequency of screening depending on a person’s own risk of progression, an individualised approach can be developed with further potential improvements in effectiveness and cost-effectiveness.

There are potential opportunities for reallocation of resources within the NHS by varying the screen interval, but no cost-effectiveness research data other than from limited modelling. The safety and acceptability of extending the screen interval in low-risk groups have not been investigated.

Estimates of the incidence and prevalence of DR are important in designing screening programmes and clinical services. The landmark epidemiological studies on DR that have provided these data are over 30 years old. 16,17 Since then, there have been changes in diagnostic criteria for diabetes,18,19 and diabetes and blood pressure (BP) control have improved in some populations. 20 At the time of the design of this programme, there were few data on incidence and prevalence in screening populations, mainly those from our own work in the 1990s. 6 As systematic programmes have been introduced and detected early disease, incidence in screened populations has changed, characterised as the ‘first pass effect’. Clinical studies of interventions aimed at early disease utilise cohorts, such as those undergoing systematic screening, and require prevalence and incidence data for study design and comparison.

Data warehouse (work package B1)

The aim of work package B1 was to collate, link and store patient data from routinely collected sources in a study data warehouse (DW) to support the development of a risk-calculation engine (RCE), the RCT and epidemiological, qualitative and cost-effectiveness studies.

The Liverpool Risk Calculation Engine (work package C)

The aim of work package C was to develop an individualised (personalised) method of calculating screen intervals using a RCE based on individual risk factors (see Eleuteri et al. 21).

RCT to evaluate the safety, efficacy and cost-effectiveness of individualised variable-interval risk-based screening (work package E)

The aim of work package E was to investigate the safety, efficacy and cost-effectiveness of longer screening intervals in low-risk PWD and shorter intervals in high-risk PWD (see Broadbent et al. 22,23).

Health economics (work package D)

The aim of work package D was to estimate the likelihood that individualised screening is more cost-effective than current practice (see Broadbent et al. 23).

Qualitative study with patients and healthcare professionals on changing screening intervals (work package F)

The aim of work package F was to explore the underlying perceptions of screening and early detection, and to test the acceptability of our new method of screening to PWD and healthcare professionals (HCPs) (see Byrne et al. 24).

Observational cohort study in people attending DR screening (work package B2)

The aim of work package B2 was to conduct a whole-population longitudinal observational cohort study to estimate up-to-date rates of incidence of DR and STDR in PWD in Liverpool (see Cheyne et al. 25).

Knowledge transfer and preparation for implementation (work package G)

To disseminate the research findings to clinical and academic beneficiaries, develop an NHS implementation plan through a dissemination event and develop any intellectual property (IP).

Figure 1 shows the stages and development of the work packages, how the work packages interconnect and the contribution of each work package (WP) to the whole programme.

FIGURE 1.

Pathway diagram showing the work flow in the ISDR Programme Grant for Applied Research, the work packages and their interrelationships. WP B1 DW development; WP C RCE; PPI (involvement shown in dashed boxes); WP E RCT; WP D Health economics; WP B2 Observational cohort study; WP F Qualitative study; WP G Dissemination. Resources were reallocated to the RCT from the cohort study in year 4 to support delayed recruitment (white hexagon and arrow). Arrows indicate input into specific item(s) of a WP. ADA, American Diabetes Association; ARVO, Association for Research in Vision and Ophthalmology; EASD, European Association for the Study of Diabetes; EASDec, European Association for the Study of Diabetes eye complications study group; LCCG, Liverpool Clinical Commissioning Group; NSC, National Screening Committee.

Aims

We set out to involve members of the public in all aspects of our programme, the RCT in particular, and to support the research team to make the research relevant, attainable and applicable to patients and HCPs.

Methods

Seven PWD were recruited to the programme with the help of local and national service-user groups. Some members were recruited during the development phase of the programme, with all of the current group retaining membership during the course of the research grant.

The methods developed for the PPI group within ISDR are listed below:

• We held regular meetings preceded by lunch in a venue located within the university campus. The location helped to create a relaxed and social atmosphere.

• We sent the agenda prior to meetings, and any information related to agenda items with items signposted for discussion with and decisions by the group.

• We made significant efforts to ‘translate’ any scientific or medical information.

• Our discussions of items often created more questions from our PPI members and a need for other information. We realised early on in the meetings that we often needed at least two (and sometimes three) sessions to discuss agenda items and arrive at a considered position. Much of the information that we gave was technical, scientific or medical and there was often an associated high amount of material. We were asking our PPI group to make significant research decisions, such as the level of risk for the RCE, that would affect other PWD.

• We worked with the PPI group in their decision-making to ensure that they were able to arrive at decisions without any doubts around their understanding or favouring a decision. This involved the following:

  • The chief investigator (CI) of the programme attended every PPI meeting and gave an update on the RCT.

  • We created a ‘research translator’, a member of the team who checked on the PPI group’s understanding and expectations.

  • We invited experts from the ISDR research team to give presentations and answer any queries from members of the PPI group.

  • We fed back at every meeting the progress of the PPI group’s decisions on every aspect of the RCT.

• Members of the PPI group were also on the Programme Steering Committee, Programme Investigators Committee and Trial Steering Committee and had a macro view of the research process. Before each of these meetings, a pre meeting took place between the PPI member and one of the investigators.

• Operational, logistical or political issues were discussed with the group for potential strategies.

Results: summary of patient and public involvement contributions to the ISDR programme

We met with our PPI group 19 times between 2013 and 2019, as detailed in Supplementary Material 1. In summary, they:

1. contributed to the programme development grant and research and programme grant application

2. developed with the research team candidate risk factors associated with progression of DR (WP C)

3. prioritised research questions by identifying the most important patient-centred outcomes (WP B2)

4. were heavily involved in setting the acceptable risk threshold in the RCE to be tested in the RCT (WPs C and E)

5. assisted in the preparation of patient-related materials, such as consent forms and patient information booklets (WP E)

6. suggested measures to improve study recruitment by observing screening clinics and the ISDR RCT consent process (WP E)

7. made suggestions to the clinical director of the Liverpool Diabetic Eye Screening Programme (LDESP) to improve screening attendance

8. served on oversight committees and helped to resolve issues during the lifetime of the grant

9. participated in cost-effectiveness analyses and suggested ways in which savings could be used to improve future services (WP D)

10. commented on this report and draft papers from the RCT and a paper on the acceptability of variable-interval risk-based screening to HCPs and PWD (WP F)

11. were involved in a PPI session in the dissemination conference (WP G).

Discussion: reflections/critical perspective

Members of our PPI group had a very positive impact on all aspects of the programme and strongly supported the research team. Their activities have been wide-ranging, not only in the technical details of the RCT, but also in their lobbying for variable-interval screening. As PWD, they were primarily concerned about the safety of the trial and improving the health of others, and mindful of the potential consequences of making the wrong decision and its impact on others’ lives. Our key lessons on successful PPI are set out in Box 1.

Over the extended duration of our research programme, our PPI members developed wide expertise and became ‘patient experts’. Further work to develop systems to retain and build on this expertise could provide a useful resource for funding bodies and policy-makers.

Interrelation to other work packages and overall aims of the programme

The PPI group members supported most work packages of the programme, including B2 (observational cohort study), C (RCE), D (health economics), E (RCT) and F (qualitative study).

Research aims

To design and develop a purpose-built DW to support the development of the RCE, the delivery of the RCT, and the economic, qualitative and cohort studies.

Methods

Using Structured System Analysis and Design Methodology,27 the ISDR business intelligence (BI) team designed and built the ISDR DW and supporting architecture. The DW centralises data into a single Microsoft SQL (structured query language) server data repository hosted on a secure private local area network within the Department of Medical Physics and Clinical Engineering at the Royal Liverpool and Broadgreen University Hospital Trust (RLBUHT). All incoming and outgoing file transfers were carried out using secure networks and encryption methods, and received local information governance approvals (Privacy Impact Assessment, Data Governance Committee, Caldicott Guardian).

The following data sources interacted with the ISDR DW:

• Demographic and systemic risk factor data from general practices (GPs) via the Liverpool Clinical Commissioning Group (LCCG) (Egton Medical Information Systems, EMIS Web, EMIS Health; www.emishealth.com).

• Demographic, retinopathy grading and visual acuity (VA) data from –

  • the LDESP, collected in the Orion/Digital healthcare database between 1995 and 2013 and in OptoMize from 2013

  • HES-based screen-positive assessment clinics, collected in a bespoke Microsoft Access database (Diabolos) (1991–2015) and the hospital patient electronic notes system (PENS) (from 2016).

• Appointment and treatment records from the hospital’s integrated patient-management system.

• An ‘opt-out register’. PWD who were registered with GPs in Liverpool were informed of the project in a letter addressed from the screening programme. Their consent for inclusion in the DW was obtained via an ‘opt-out’ method and a record of those patients who opted out was kept in a register.

The flow of data in the programme is illustrated in Figure 2.

FIGURE 2.

Diagrammatic representation of ISDR DW data flows.

The DW received all relevant data from PWD streamed from primary and secondary care. The RCE was fed directly from the DW. Risk factors included were demographic (e.g. age, deprivation and ethnicity), ocular (e.g. previous and current retinopathy severity, previous treatment and VA) and systemic (e.g. diabetes control and duration, BP, and lipids). The data fields are listed in more detail in Supplementary Material 2, Table 1.

To ensure the credibility of data coming from multiple sources, which contained data with varying quality, covariates in the source data were examined for outliers and distribution inconsistencies. Credibility limits were developed using a statistical methodology. This set of scripts cleansed and validated incoming data for the DW. Input and output schemas (see Supplementary Material 2, Table 2) were developed for each data transfer element, as indicated by the arrows in Figure 2.

Results

The ISDR DW held data from 2009 on 28,556 participants as of January 2020 (end of the programme) over 11 screening cycles. Data linkage was achieved for all participants. The ISDR DW received test and preliminary data sets from early 2013. One of the main data sets was the ‘ISDR Cohort Study Dataset’, which supported WP B2. This contained data for 28,384 participants in October 2017, equivalent to 318,053,075 data points; a data point is a discrete unit of information,28 an example being a single recording of weight for a patient.

Discussion: challenges in the DW

The ISDR programme collated and linked large-scale routinely collected clinical data across different domains in a single repository. It successfully supported all aspects of the programme and routine patient management in the local screening programme. The DW is dynamic in that it updates on a regular basis. It demonstrated resilience and the potential to be developed into a routine health informatics tool in the NHS for personalised medicine.

We faced a number of challenges. No existing ‘off the shelf’ databases existed. The data were complex and came from multiple data sources. The Liverpool CCG had to set up and run a set of manual processes throughout the lifetime of the project to link the various domains within primary care. There was variability in data quality, which required bespoke processing and the development and application of credibility limits before the data could be considered adequate for a clinical application. Systems were of widely differing design: some were commercial (OptoMize) and some were developed in house (Diabolos, PENS). Systems had to be ‘cracked’ before the relevant data could be extracted. Data labels had to be investigated and tested. Several clinical system upgrades took place during the lifetime of the programme, which tended to occur unannounced, resulting in some lengthy delays and wasted effort.

The multidisciplinary nature of the programme meant that several members of the team had very little understanding of the requirements of database development and engineering, and similarly the database team struggled with adequate engagement with the clinical teams. There appears to be a lack of bioinformatic expertise to act as a bridge between the technical BI and computer science teams on the one hand and the clinical/research end-users on the other. This needs urgently addressing.

We learnt several important lessons. Establishing a real-time data repository within an NHS environment to underpin personalised medicine requires:

• sufficient time for interdisciplinary team working on data sanity checks, outlier/credibility handling protocols, logic rule development and imputation strategies

• investment in developing cross-disciplinary bioinformatics expertise to link technical teams and clinical/research end-users.

Interrelation to other work packages and overall aims of the programme

In addition to the underpinning role of the DW across much of the programme, it has led to development of new IP covered in WP G. Imputation strategies needed to be developed in several WPs to deal with missing data.

Research aims

To develop and internally validate a RCE to estimate the risk of progression to screen-positive or referable DR and assign individualised screening intervals (WP C).

Methods

Model

We selected a continuous-time Markov process to allow for a set of individuals to transition among states over time. 29 The state at each time point was defined by the level of retinopathy, including separation by one or both eyes (Figure 3):30 state 1 – no DR detected; state 2 – non-referable DR in one eye only; state 3 – non-referable DR in both eyes; and state 4 – referable DR (screen-positive for at least one eye).

FIGURE 3.

States and transitions in the continuous-time Markov process in the Liverpool Risk-Calculation Engine.

Only one baseline screening event was used. The risks or intensities for each of six transitions between these states were entered into the model within a probability matrix. The interval censoring seen in screening data required special methodology. Missing clinical data were handled using multiple imputation.

Ten covariates met the entry criteria and were ranked using Wald statistics. A set of nested models were built to estimate corrected Akaike’s information criterion (AICc). The model with the smallest AICc was chosen to give the best fit to the data.

We checked the RCE development data set using random samples of event vectors. The model was checked for the influence of outliers, regression and distributional assumptions; Pearson-type goodness of fit and corrected C-index were calculated. Bootstrapping and fourfold cross-validation were used for internal validation. Further internal validation was conducted using a geographical split based on deprivation index. 31

Results

Data extracted into the RCE development data set were from 11,806 PWD attending the LDESP between 20 February 2009 and 4 February 2014.

Covariates meeting the selection criteria are listed in Table 1. Those that gave the best fit and were included in the final model were disease duration, glycated haemoglobin (HbA_1c), age at diagnosis, systolic BP and total cholesterol. Although the retinopathy stage is not technically a covariate, it is included in the table to show the improvement in predictive power when covariates are added to the model.

The risk model is summarised in three equations. The first gives the hazard rates (or intensities or ‘risks’) of going from one state to another for each transition:

where i, j = {1, 2, 3, 4} and β^C_ij is the model parameter for covariate C (or baseline intensity when C = 0); AgeD is age at diagnosis and DiseaseD is disease duration. From this equation, a transition intensity matrix is derived for the four states described above:

Probabilities of transition occurring at a specific time are obtained by using the third equation:

Tables 2 and 3 show the estimated baseline hazard ratios and probabilities of each transition state. Further details are available elsewhere. 30

Discussion

We have developed and tested a RCE in which an individual’s risk can be predicted from routinely collected clinical data, referenced to the clinical histories of the local population, and using covariates of local relevance. The risk can be reassessed at each screening episode as new clinical information is acquired.

Strengths of our model include the strongly embedded PPI group, which allowed us to develop an appropriate preliminary covariate list, acceptable screen intervals and risk threshold. The model internally handles interval censoring, inherent to retinopathy data in screening. 29 It predicts the probabilities of transition for all patient states and embeds a model for multiple imputation of missing covariates.

Potential limitations include not adjusting for misclassification of retinopathy during grading. This could be addressed by adding a misclassification model but at the cost of substantially more observations and computational complexity. Some potentially useful covariates were not informative in the Liverpool setting: ethnic diversity and prevalence of abnormal estimated glomerular filtration rate levels were low, and ‘type of diabetes’ may not have been accurately recorded. We used the date of the first HbA_1c test to improve data on ‘duration of diabetes’. This was helpful in people with long durations but less reliable since the introduction of HbA_1c as a primary screening test.

The model consistently showed good levels of prediction for the 2.5% risk threshold. The numbers of screen-positive cases with overestimated screening dates and screen-negative cases with underestimated screening dates were smaller. The majority of people were correctly allocated (78% of screen-positives, 80% of screen-negatives), with a reasonable allocation across the 6-, 12- and 24-month intervals (approximately 10% : 10% : 80%). The number of patients who had a screen-positive event before the allocated screening date reduced by > 50% and the overall number of screening episodes by 30%.

Our RCE development process is suitable for a wide range of geographical locations and populations, with a minimum prerequisite of a disease register with adequate historical data. Revision/addition of covariates can be accommodated based on the strength that they add to a locally developed model. We give the key steps to developing and building such a system in Box 2.

The use of near real-time data and a model developed from local data in our approach is novel. Aspelund et al. 13 reported on a risk-estimating model using retinopathy data collected in Iceland between 1994 and 1997, and risks for covariates estimated from data published in the 1980s. This was tested in a prospective cohort of people with type 2 diabetes. 32 Discriminatory ability was good (C-statistic 0.83) but 67 out of 76 people (88.2%) who developed STDR developed it after the time predicted by the model. This overestimation of risk highlights the weakness of using historical data. In 2020, van der Heijden et al. 33 reported a systematic review of risk-prediction models in DR screening. Discrimination was reported in seven studies with C-statistics ranging from 0.55 to 0.84. Our study21 was published subsequently and, therefore, was not included. The corrected C-index in the Liverpool RCE was 0.687.

Access to clinical information is not routinely available. We had to overcome significant challenges in developing a near real-time data flow; this may be too difficult in some populations. However, we determined that including clinical data would aid acceptance among the professional community, offer better prospects for generalisability and allow inclusion of more frequent screening for high-risk individuals. Our view is supported by our own data,6 those of others34 and our PPI group. We recognise that estimates of resource requirements for introduction of our type of RCE are not available.

External validation of models is required before general implementation. 32,35 An implementation phase will include model updating (temporal validation and model tuning) and the opportunity for comparative cross-population (external) validation to correct for potential over-performance. 36

Interrelation with other parts of the programme and overall aims of the programme

The RCE was developed using data from primary and secondary care linked through the ISDR DW developed specifically for this programme. It was required for the development of the RCT and is part of the developed IP. It will form part of any potential future benefit from the programme if deployed within the NHS and other countries. The health economics team worked with the RCE team ensuring that the economic model followed the structure and content of the RCE and could be fully integrated alongside it.

Research aims

We designed a RCT to investigate the safety, feasibility, efficacy and cost-effectiveness of extending screening intervals in low-risk PWD and reducing screening intervals in high-risk PWD (WP E). We utilised the emerging methodology and technologies of personalised risk prediction37,38 and an innovative data linkage system to develop an individualised variable-interval risk-based screening approach. Individualised clinical care offers opportunities for improved patient engagement.

We tested the hypothesis of equivalence between the attendance rates as a primary measure of safety, for individualised and annual screening. An equivalence design was selected, instead of a superiority trial, because the aim was to demonstrate equivalence between attendance rates rather than difference. Our PPI group was involved in all aspects of design, delivery and interpretation (see Patient and public involvement in the ISDR programme).

Methods

This was a single site, two-arm, parallel assignment, equivalence RCT conducted in all screening clinics in the LDESP, part of the NDESP. The trial was conducted with the Liverpool Clinical Trials Centre.

The main inclusion and exclusion criteria were: registered with a GP whose postcode was within the city boundaries of Liverpool, aged ≥ 12 years, attending screening for DR and had not opted out of data-sharing. Age-appropriate patient information leaflets were sent to eligible patients with their screening appointment. Participants provided written informed consent. For children aged 12–15 years, proxy consent by the parent/guardian was provided and, where appropriate, assent from the child. Preston Research Ethics Committee approved the trial (14/NW/0034). The trial registry number is ISRCTN87561257.

Allocation was randomly 1 : 1 to individualised variable-interval risk-based screening recall at 6, 12 or 24 months (intervention arm, high-, medium-, low-risk respectively), or annual screening (control arm, current routine care).

The ISDR DW automatically populated the majority of the fields in the baseline and follow-up electronic CRFs, including data for randomisation. Screening staff and clinical assessors were masked to intervention arm, risk-calculation and interval.

Results

The trial opened on 1 May 2014, randomisation took place from 12 November 2014 to 14 June 2016 and last follow-up was 16 August 2018. In total, 4069 out of 8607 eligible people were excluded, which meant that there were 4538 individuals who were randomised and allocated to a trial arm (2269 fixed interval, 2265 individualised, 4 withdrawals). In the individualised arm, at baseline 198 participants were allocated their first screening recall at 6 months, 211 at 12 months and 1856 at 24 months, which reflects the allocation to high, medium and low risk by the RCE. The trial profile is shown in Figure 4.

FIGURE 4.

CONSORT 2010 flow diagram showing the numbers for eligibility, allocation, withdrawals in the PP and ITT data sets for the primary analysis.

Baseline characteristics in the PP data set are shown in Table 4; the ITT data set had similar characteristics. Participants had a median age of 63 years (range 14–100 years), 60.4% were male, 94.6% were white and 88.5% had type 2 diabetes. Those in the high-risk group were more likely to have type 1 diabetes, a longer diabetes diagnosis and higher HbA_1c, and less likely to have ever smoked, when compared to those in the medium- and low-risk groups.

Primary safety findings

Attendance rates at the first follow-up visit for the control and individualised arms (primary outcome) were 84.7% (1883/2224) and 83.6% (1754/2097), respectively (difference –1.0%, 95% CI –3.2% to 1.2%, PP analysis). Against an acceptability range of 0.05, these attendance rates were equivalent (Figure 5 upper panel, Table 5). PP and ITT analyses with multiple and simple imputation confirmed equivalence in attendance rates between the two arms.

FIGURE 5.

Plot showing point estimates and 90% CIs for participants attending the first follow-up visit in the two arms. Upper panel: primary analysis. Lower panel: high-, medium- and low-risk groups in the individualised arm.

TABLE 5 - Results of the test for equivalence in attendance rate at first follow-up visit and of the non-inferiority in STDR detection within 24 months, based on the PP, ITT and multiple imputation data sets

Approach	Control arm				Individualised arm			Difference in proportions (95% CI)
	N		Attended/STDR detected (n)	Proportion	N	Attended/STDR detected (n)	Proportion
Primary outcome: attendance at first follow-up
PP
Overall		2224	1883	0.847	2097	1754	0.836	–0.010 (–0.032 to 0.012)
High risk		203	157	0.773	195	141	0.723	–0.050 (–0.136 to 0.035)
Medium risk		169	138	0.817	208	171	0.822	0.006 (–0.073 to 0.084)
Low risk		1852	1588	0.857	1694	1442	0.851	–0.006 (–0.029 to 0.017)
Multiple imputation		2269	1910	0.842	2234	1870	0.837	–0.005 (–0.026 to 0.017)
ITT
Overall		2224	1883	0.847	2143	1798	0.839	–0.008 (–0.029 to 0.014)
Multiple imputation		2269	1913	0.843	2265	1903	0.840	–0.004 (–0.025 to 0.018)
Secondary outcome: STDR within 24 months
PP
Overall		2042	35	0.017	1956	28	0.014	–0.003 (–0.011 to 0.005)
High risk		176	20	0.114	127	17	0.134	0.020 (–0.053 to 0.100)
Medium risk		157	5	0.032	179	7	0.039	0.007 (–0.038 to 0.051)
Low risk		1709	10	0.006	1650	4	0.002	–0.003 (–0.009 to 0.001)
Multiple imputation		2269	39	0.017	2234	36	0.016	–0.001 (–0.009 to 0.007)
ITT
Overall		2042	35	0.017	2056	32	0.016	–0.002 (–0.010 to 0.006)
Multiple imputation		2269	39	0.017	2265	39	0.017	–0.001 (–0.008 to 0.008)

Equivalence in the analysis of the three risk groups in the individualised arm (see Figure 5, lower panel, and Table 5) was found for the low-risk group (control 85.7%, individualised 85.1%, difference –0.6%, 95% CI –2.9% to 1.7%). For the medium-risk group, equivalence was not confirmed (control 81.7%, individualised 82.2%, difference 0.6%, 95% CI –7.3% to 8.4%); differences were very small but equivalence was not confirmed because of the relatively wide CIs. Attendance rates were lower in the high-risk group (control 77.3%, individualised 72.3%, difference 5.0%, 95% CI –13.6% to 3.5%) than in the medium- and low-risk groups. Although the hypothesis of equivalence was not supported in this group, the attendance rates observed over 12 months (percentage with at least one attended appointment) were higher in the individualised arm (89.1%) than in the control arm (77.3%).

Discussion

Our study, which, to the best of our knowledge, is the largest ophthalmic RCT performed in individualised DR screening to date, shows that all parties involved in diabetes care can be reassured that extended and individualised (personalised) interval screening can safely be introduced in an established screening programme. We have shown that variable-interval risk-based screening, as developed by us in Liverpool, is equivalent to the current standard of annual screening, in our setting. Our secondary safety findings support this primary result. In the individualised arm, we did not detect any drop in rates of detection of STDR, worse VA or higher rates of VI, or worsening glycaemic control. Attendance levels were maintained throughout the study.

We have also shown improved efficacy for our personalised approach, reducing the number of appointments by 43.2%. In the individualised arm, there were higher proportions of screening episodes that were screen-positive and screening episodes that detected STDR were earlier than in the control arm. In the setting of a RCT, the RCE showed feasibility and reliability with good discrimination.

A move to longer screening intervals for people at low risk of DR has been suggested in the literature prior to39 and during40–42 our programme, but without convincing evidence on safety. 8 Overall, 19.0% of people invited to take part in our study explicitly stated that they wished to remain on annual screening or did not want a change of interval. We did not detect a worsening in glycaemic control. Our findings give substantial reassurance that a 24-month interval for low-risk individuals with diabetes is safe in a setting such as ours. However, for resource-poor or rural settings in low- and middle-income countries, further research is required before longer intervals can be contemplated.

Our findings have some limitations when considering wider application. We developed and applied our approach to screening in a single centre with relatively low rates of baseline DR and low rates of progression to sight-threatening disease. Our programme has been running for over 30 years. Participants’ glycaemia and BP control were relatively good. Our results should be generalised with caution to other settings with higher prevalence, poorer control of diabetes, wider ethnic group representation, or in programmes in set-up. Our study has important strengths including the RCT design, its size, independent oversight, and involvement of an expert patient group.

Our approach allowed us to target high-risk people using a 6-month screening interval. Attendance rates at first visit in this group were lower in the individualised arm (72.3% vs. 77.3% in the control group) but disease was detected earlier through more frequent screening (1.7 visits, 89.1% attendance over the first 12 months vs. 77.3% estimated in controls) than in the control arm. The rates of detection of STDR were higher in the high- and medium-risk individualised groups (13.4% and 3.9% vs. 1.7% in controls). The proportion of screening episodes that were positive was higher in the individualised arm at 5.1% than in the control arm at 4.5%. By contrast, there very low rates of screen-positive episodes in the low-risk group (0.2%). These rates are similar if slightly lower than the 0.3–1.0% reported over 2 years in people with no retinopathy in either eye in a study of seven screening programmes in England. 41

The value of adding in clinical data is a matter of debate. Our PPI group advised strongly during the design phase of the study that including clinical data was important to them. It allows the introduction of a high-risk group and better targeting than stratification (see Further analysis for post-RCT analysis). 30 We would suggest that the advantages for patient and clinician engagement outweigh the added cost and complexity. Around 15% of people had at least one change in risk-based interval: 59% people allocated to the medium (annual) group experienced a change of interval, adding further evidence to move away from a fixed interval for all.

Interrelation to other work packages and overall aims of the programme

Data collated in the ISDR DW (WP B1) informed the RCE (WP C), which in turn supported the RCT. The PPI group developed the risk criteria, actively supported with the production of patient-related materials and observed clinics to improve recruitment.

Research aims

We conducted an economic analysis within our RCT to investigate the cost-effectiveness of individualised variable-interval risk-based screening from an NHS and a societal perspective (WP D).

Our review of the literature showed limited evidence on cost-effectiveness of screening. Study designs have been heterogeneous and relied on modelling rather than direct observation. One systematic review of screening called specifically for cost-effectiveness studies. 8

Methods

Results

The costs of screening are shown in Table 6. Costs per attendance (n = 16,736) were £11.73 for programme costs, £10.00 for photography and £7.00 for grading, with a total NHS cost of £28.73. The cost for non-attendance at £12.73 was principally the programme cost, with a small contribution for lost photographer time. Additional productivity losses and out-of-pocket payments by the patient accounted for £9.00.

The estimated cost of running the RCE was £40,000 per annum for a total screen population of 22,099 in Liverpool allocated to the intervention arm as £1.81 per person.

Table 7 shows summary health economic and cost-effectiveness data. There was no statistically significant difference in QALY scores between the trial arms (EQ-5D 0.006, 95% CI −0.039 to 0.06; EuroQol Visual Analogue Score 0.004, 95% CI −0.049 to 0.052; and HUI3 −0.017, 95% CI −0.083 to 0.04). We observed incremental cost savings per participant of £17.34 (£17.02 to £17.67) from an NHS perspective (not including treatment costs), corresponding to a potential reduction in total programme costs of 20% (95% CI from £193,983 to £154,386), and £23.11 (95% CI £22.73 to £23.53) from a societal perspective, and an INMB for the EQ-5D-5L of £736 (95% CI –£239 to £1718) and £178 (95% CI –£919 to £1338) for the HUI3 £26.19 (95% CI £24.41 to £27.87).

Results following multiple imputation indicated the dominance of the individualised arm in both QALY gain (EQ-5D-5L and HUI3) and cost savings (screening and societal) (Figure 6). All ICER point estimates fell in the south-east and south-west quadrants of the cost-effectiveness plane, signifying the dominance in cost savings, with mean differences in QALYs clustered around the zero threshold, EQ-5D-5L reporting a marginal benefit, and the HUI3 reporting an incremental distribution of near equivalence between the arms.

FIGURE 6.

Baseline adjusted EQ-5D, HUI3, and NHS perspective incremental cost-effectiveness of individualised vs. annual screening from 1000 iteration bootstraps.

Economic model

Model structure

The disease states used in the model were defined in the terms used by the NDESP, namely ‘R0M0’ gradings. Retinopathy is graded as R0 (no retinopathy), R1 (background), R2 (pre proliferative) or R3 (proliferative). Maculopathy is graded as either M0 (absent) or M1 (present).

All individuals were in the screening programme, except for those who had experienced a screen-positive event and were, therefore, in follow-up and may have been receiving treatment. However, those who receive treatment that was successful may have been referred back to the screening programme. Therefore, the model allows for individuals to be in either screening or follow-up, whether or not they received treatment.

Individuals who went on to receive treatment were subject to different progression rates. Therefore, it was necessary to divide the model further based on the stages of the treatment pathway. The model allowed for four separate groups of individuals: (1) those in screening who had not received treatment, (2) those in screening who had previously received treatment, (3) those in follow-up who had not yet received treatment and (4) those in follow-up who had previously received treatment. This increases the number of disease-specific states fourfold, giving 38 states.

The comparators for analysis (described above) are similarly included as an additional event process, applied to all people whose screening outcome is negative and who are referred back to screening. These event pathways are outlined in Figure 7.

FIGURE 7.

Event pathways in the economic model developed within the ISDR programme of applied research. Upper panel: pathway within screening programme; middle panel: pathway within HES for screen-positive individuals; lower panel: pathway showing comparison between three screening models.

The model was developed using Microsoft Excel 2013. The intent was to run the model from NHS and Personal Social Services perspectives over a life-time horizon with cycles occurring monthly, and transitions and events occurring at the end of each cycle. The computational and mathematical requirements of such a large-scale model proved non-operational by the end of the programme, requiring around 4 days of processing. Work continued to produce a final version of the model based on 10,000 people.

Discussion

Our data provide evidence that an individualised screening approach may provide cost savings within the programme. We feel that the assumptions and costs applied throughout were conservative and took care not to overstate the benefits of this approach or underplay any of the costs, for example providing a realistic estimate of replicating the cost of a risk engine elsewhere.

By moving to variable-interval risk-based screening, patients were not compromised on safety or quality of life. We calculated potential incremental cost savings over the 2-year time horizon within the trial of £17.34 NHS costs, rising to £23.11 per patient from a societal perspective. In a population such as in Liverpool, this may amount to potential annual savings in the region of £199,000 in NHS screening programme costs. For England [screening population 2.76M (2018–19)51], this could amount to around £23.9M in the NHS, not including treatment costs, rising to £31.9M from a societal perspective. Such resources could be used to target groups that are hard to reach and those at high risk of VI, and more cost-efficiently screen the expanding population of PWD. Individualised screening (on average) reduces the number of attendances required, given that patients in low-risk groups would be spared the inconvenience and additional personal cost of attending non-essential appointments. In brief, fewer visits reduce both patient and hospital costs and the trial shows that this can happen without negative clinical consequences.

Strengths of this within-trial cost-effectiveness study are the detailed characterisation of the costs to the health service and society of a person attending screening and the large number of observations. Our work is limited by the 2-year time horizon, which is short for a long-term disease, such as diabetes. Our work could have been further strengthened by taking a long-term time horizon to capture lifetime costing differentials and years of sight loss averted. Treatment costs were excluded as per the within-trial analysis and the study objectives of comparing screening options rather than of a lifetime analysis of screening. We did explore adding a post-screening analysis to include treatment costs but considered it very unlikely to be informative because the numbers progressing to treatment were very small (two in each group).

It may have been useful to collect utility data for the entire cohort. We elected not to undertake this to minimise disruption at screening clinics. Multiple imputation indicated that quality-of-life (QoL) data were robust. At a 2-year time horizon, the effects of screening intervals on QoL appear to be close to negligible, where our between-arm utilities and incremental QALYs demonstrated near-equivalence.

Although the intention had been to report cost-effectiveness acceptability curves, the dominance in cost reduction of variable-interval risk-based screening and little fluctuation in QALYs across all instruments rendered this metric uninformative, as the proportion cost-effective was inelastic to varying thresholds. See Strengths and limitations for a further discussion of strengths and limitations.

In 2015, Scanlon et al. 42 reported a cost-effectiveness model developed in a historical data set of 10,942 people in Gloucestershire with screening and clinical data over at least 3 years and validated it in three other English programmes. They reported a 3-yearly screening interval to be the most cost-effective in the absence of personalised risk-based intervals. Using their risk-based strategy, the most cost-effective options were to screen those at low risk every 5 years and those at medium and high risk every 3 and every 2 years, respectively. We now add robust RCT evidence to support the introduction of variable-interval risk-based screening.

Interrelation to other work packages and overall aims of the programme

The health economics and RCE teams (WS C, WS D) worked closely together. The economic model was designed to be fully integrated and usable with the RCE. This is an important element should the Liverpool approach be rolled out into wider practice. Co-ordination occurred with WP F to enable a more inclusive and comprehensive understanding of valuation in QoL in DR screening. This was important as this group was under-represented in the trial itself.

Research aims

We aimed to explore perceptions of changing from annual to individualised variable-interval risk-based screening and to gain wider insights into users’ perceptions to enable successful implementation (WP F). We also wanted to explore the wider aspects of DR screening in general.

Methods

Results

Here we summarise key findings from interviews with PWD and HCPs. Further details included patient narratives, in line with standards for reporting qualitative research,55 and are reported elsewhere. 24 The narratives give a compelling insight into the perceptions and views of users of screening services.

Discussion

To the best of our knowledge, our qualitative study was the first to explore perceptions of changing screening intervals, with our findings showing general support from PWD and HCPs for the introduction of variable-interval risk-based screening. This is reassuring for policy-makers and service providers who are considering introducing variable-interval screening, stratified based on either retinal grading or variable intervals based on risk estimation. Support was more clearly expressed by HCPs who had a better understanding of the aims and current pressures in screening. Our findings also have relevance to other screening programmes with fixed intervals and help to mitigate in part the concerns raised recently in cancer screening. 15,56

For HCPs, and to a lesser extent PWD, the safety of the RCE was crucial to it being accepted into clinical practice. Safeguards for the RCE had to be made visible and obvious to HCPs and PWD, including the opportunity for HCPs or PWD to refer or self-refer back into the eye screening programme.

In terms of variable screening intervals, there were some misunderstandings among PWD participants about the 6-month interval, whereas HCPs welcomed the opportunity for increased surveillance for high-risk groups. PWD and HCPs were supportive of the current 12-month interval, while recognising that NHS resources are finite.

There were widely differing views on extending screening intervals to 24 months, with both groups of participants having significant concerns about safety. There were worries that any changes in a person’s diabetes eye health would be missed, which could lead to developing STDR in this time, leading to poor quality of life for a patient and increased costs to the NHS.

There was a recognition by both groups of participants that against a backdrop of increasing numbers of PWD, current screening intervals are unsustainable and resources could be better used to target high-risk groups. The considerable conflation and misunderstanding about different eye-related appointments has been reported elsewhere in the literature. 57,58 Changing the message to PWD from regular annual check-ups to extended eye screening will be a challenging message to convey positively.

We attempted to recruit PWD who had a history of multiple non-attendance for screening. Our research team tried contacting over 30 people in this group without success. Poor attendance, known to be a risk factor for STDR,59,60 is a major issue for all stakeholders involved in screening. Novel ways need to be developed to approach these individuals so that the reasons for non-attendance can be investigated and addressed.

We had intended to investigate changes in attitudes through sampling prior to and during the RCT but follow-up interviews proved to be difficult to obtain. However, we did not identify any themes suggesting differences in perceptions in those who had already been enrolled in the RCT compared with the pre-trial group.

Among the multiple healthcare-related appointments for diabetes, there are misunderstandings by PWD of NHS systems and other appointments with HCPs. Understanding the frequency of screening required to prevent the development of DR61 was another barrier for PWD and was echoed by HCPs. Allied to appointments related to eye care, there can often be misinterpretations by PWD about the relationship between diabetes and DR, and differences between DR screening and routine eye tests, which in turn led to beliefs that screening appointments had been fulfilled when they had not,62 as well as related anxiety about screening. 63 A study by Strutton et al. 64 concluded that screening attendance could be improved by the sharing of pertinent information between healthcare providers and a greater understanding of patients’ circumstances and barriers.

Interrelation with other parts of the programme and overall aims of the programme

The implication of our evidence is that a clear and sustained communication strategy around eye screening and intervals is needed both in the current screening programme and in any future revision. These were considered in WP G.

Research aims

We conducted an observational cohort study to estimate the prevalence and incidence of DR and STDR in the established screening programme in Liverpool (WP B). We wanted to provide up-to-date data to inform clinical services and screening programmes, and for clinical research programmes, especially in early disease.

Up-to-date epidemiological data on DR are limited. Much of the population data comes from landmark epidemiological studies in the 1970s and 1980s16,17 before systematic screening was introduced. Data from screening programmes are limited and reported inconsistently. When programmes are introduced, those in the community with long-standing disease are detected for the first time and referred into the HES; as this ‘first pass effect’ wanes, the prevalence of referable DR and STDR in the screening population falls. Populations attending for systematic screening are dynamic and subject to several long-term influences. In recent years, there have been changes in the diagnosis of diabetes,18 and improvements in some populations in glycaemic and BP control. On the other hand, there has been a long-term increase in prevalence of diabetes.

Methods

The cohort study was approved separately by the Preston North West NHS Ethics Committee (13/NW/0196), the local governance committees and Caldicott Guardian. The Liverpool Local Medical Committee (LMC) and Primary Care Research Group and LCCG supported Liverpool general practices who were invited to participate in the study. Data-sharing agreements were prepared with each practice, the RLBUHT and LCCG to include all retrospective and prospective demographic and clinical records collected by practice electronic records systems (EMIS web, EMIS Health, www.emishealth.com).

Data for the cohort study were held in the DW described in The ISDR study data warehouse. Individuals were identified from the patient register of all PWD in Liverpool held by the LDESP. Consent for data collection and analysis was through an ‘opt-out’ process (see page 13). Eligible participants comprised all PWD aged ≥ 12 years diagnosed with diabetes who were registered with a practice in Liverpool. PWD aged 12–15 years received a letter and booklet written specifically for young people. Recruitment of general practices commenced in 2013; by 2016, all had agreed to participate. Once a practice had been recruited, newly diagnosed PWD and those moving into the area continued to be invited to the programme.

The following data sets were available: Diabolos (from 1995), a bespoke screening-management software developed in Liverpool; Orion (2005–13), the first iteration of the English NDESP standardised screening-management software; and OptoMize (from 2013), the current NDESP system.

Data underwent credibility checks using MATLAB (version 8.3, Mathworks, Inc., Natick, Massachusetts, USA) before being merged for analysis. The values of the time-dependent clinical variables closest to the time of the screen episodes were used.

Data from the HES were used to investigate the outcomes of a positive screen event. Data from the first biomicroscopy recorded within 1 year of the positive screen event were used to provide the final outcome for that event. The first recorded attended screening appointment was defined as the first recorded attended appointment at which there were no earlier recorded attended screening appointments or biomicroscopies for that individual.

Results

Data set

Data collection commenced on 31 October 2013. Data presented here come from an analysis data set including all relevant data extracted up to at least 31 March 2017. Data were extracted on 25 September 2017.

For the primary analysis, we censored the data until 1 April 2006. EMIS web systems started to be introduced into GPs only in 2009, and the data quality was very variable in the first 12 months.

The size of the data set varied throughout the 11 years studied, with year-end effects and variation owing to sampling time. On 25 September 2017, 42,334 people were registered as having diabetes with a Liverpool general practice. After exclusions/ineligibility, the number of people eligible to be invited to join the ISDR cohort study was 30,771. Data distribution is shown in Figure 8; data from 28,384 PWD were available for analysis.

FIGURE 8.

Data flows in the ISDR cohort study with figures from an illustrative timepoint close to the mid-point of the last full year (25 September 2017).

Incidence

Data on incidences are shown in Table 10. There was some variation of rates of screen-positive across the 11 years, with an overall reduction from 7.9–10.6% in the early years to 4.4–6.7% in the later years.

TABLE 10 - Overall screen-positive/negative/STDR (STR/STM) annual incidences: 2006/07 to 2016/17 (out of the number of people with at least one attended screening appointment)

Screening year is from 1 April to 31 March.

Based on assumption that if category is unknown, then assume category is OED.

Biomicroscopy must be recorded within 1 year of screening appointment when positive screen result occurred.

Note that all percentages are calculated out of the number of individuals with at least one attended screening appointment.

Adjusted for those in the RCT and assigned to 24 m who under normal circumstances would have had a screening appointment in these years. In 2015/16 this amounts to 226 individuals (266*0.85 which is the approximate assumed attendance percentage) and in 2016/17 it affected 1232 individuals (1449*0.85). All are assumed to be screen-negative.

Wilson score confidence intervals.

Note that these are not mutually exclusive categories.

Notes

In 2006/07 there were 16 (0.2%) cases with both STR and STM. Additionally, 1 (< 0.1%) STR case had unknown STM and 1 (< 0.1%) STM case had unknown STR.

In 2007/08 there were 18 (0.2%) cases with both STR and STM.

In 2008/09 there were 48 (0.6%) cases with both STR and STM.

In 2009/10 there were 24 (0.2%) cases with both STR and STM. Additionally, 1 (< 0.1%) STR case had unknown STM and 3 (< 0.1%) STM cases had unknown STR.

In 2010/11 there were 30 (0.3%) cases with both STR and STM. Additionally, 1 (< 0.1%) STR case had unknown STM and 1 (< 0.1%) STM case had unknown STR.

In 2011/12 there were 49 (0.4%) cases with both STR and STM. Additionally, 1 (< 0.1%) STR case had unknown STM.

In 2012/13 there were 31 (0.2%) cases with both STR and STM. Additionally, 2 (< 0.1%) STR cases had unknown STM.

In 2013/14 there were 37 (0.2%) cases with both STR and STM. Additionally, 2 (< 0.1%) STR cases had unknown STM and 3 (< 0.1%) STM cases had unknown STR.

In 2014/15 there were 44 (0.3%) cases with both STR and STM. Additionally, 3 (< 0.1%) STR cases had unknown STM and 3 (< 0.1%) STM cases had unknown STR.

In 2015/16 there were 50 (0.3%) cases with both STR and STM. Additionally, 1 (< 0.1%) STR case had unknown STM.

In 2016/17 there were 57 (0.4%) cases with both STR and STM. Additionally, 5 (< 0.1%) STR cases had unknown STM and 9 (0.1%) STM cases had unknown STR

This table does not include any HES pathway appointments (screening or biomicroscopies).

Variable	Screening year*, n (%^)											Overall, n (%^; 95% CI^^^)
	2006/07	2007/08	2008/09	2009/10	2010/11	2011/12	2012/13	2013/14	2014/15	2015/16	2016/17
Number of individuals with at least one attended screening appointment	6637 (100.0)	8088 (100.0)	8664 (100.0)	10266 (100.0)	11214 (100.0)	13124 (100.0)	13518 (100.0)	15115 (100.0)	15552 (100.0)	15280^^ (100.0)	16096^^ (100.0)	133554 (100.0)
Screen-positive	527 (7.9)	714 (8.8)	922 (10.6)	854 (8.3)	695 (6.2)	921 (7.0)	774 (5.7)	669 (4.4)	805 (5.2)	1031 (6.7)	994 (6.2)	8906 (6.7; 6.5 to 6.8)
Screen-positive for DR	271 (4.1)	355 (4.4)	401 (4.6)	360 (3.5)	327 (2.9)	419 (3.2)	330 (2.4)	345 (2.3)	387 (2.5)	441 (2.9)	437 (2.7)	4073 (3.0; 3.0 to 3.1)
Screen-positive for unassessable images	201 (3.0)	255 (3.2)	358 (4.1)	372 (3.6)	283 (2.5)	425 (3.2)	322 (2.4)	179 (1.2)	258 (1.7)	408 (2.7)	383 (2.4)	3444 (2.6; 2.5 to 2.7)
Screen-positive for other eye disease requiring HES**	55 (0.8)	104 (1.3)	163 (1.9)	122 (1.2)	85 (0.8)	77 (0.6)	122 (0.9)	145 (1.0)	160 (1.0)	182 (1.2)	174 (1.1)	1389 (1.0; 1.0 to 1.1)
Biomicroscopy recorded***	446 (6.7)	618 (7.6)	794 (9.2)	712 (6.9)	612 (5.5)	774 (5.9)	650 (4.8)	621 (4.1)	742 (4.8)	965 (6.3)	868 (5.4)	7802 (5.8; 5.7 to 6.0)
STDR	116 (1.7)	145 (1.8)	192 (2.2)	178 (1.7)	160 (1.4)	223 (1.7)	179 (1.3)	212 (1.4)	246 (1.6)	251 (1.6)	290 (1.8)	2192 (1.6; 1.6 to 1.7)
STR$	41 (0.6)	44 (0.5)	81 (0.9)	65 (0.6)	56 (0.5)	75 (0.6)	57 (0.4)	69 (0.5)	98 (0.6)	85 (0.6)	110 (0.7)	781 (0.6; 0.5 to 0.6)
STM$	91 (1.4)	119 (1.5)	157 (1.8)	135 (1.3)	130 (1.2)	197 (1.5)	149 (1.1)	179 (1.2)	191 (1.2)	216 (1.4)	237 (1.5)	1801 (1.3; 1.3 to 1.4)
Unknown STR & unknown STM	0 (0.0)	0 (0.0)	2 (< 0.1)	2 (< 0.1)	4 (< 0.1)	0 (0.0)	4 (< 0.1)	1 (< 0.1)	1 (< 0.1)	0 (0.0)	0 (0.0)	14 (< 0.1; 0.0 to 0.0)
Not STDR	330 (5.0)	473 (5.8)	602 (6.9)	534 (5.2)	452 (4.0)	551 (4.2)	471 (3.5)	406 (2.7)	496 (3.2)	714 (4.7)	555 (3.4)	5584 (4.2; 4.1 to 4.3)
Unknown STDR status	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	3 (< 0.1)	0 (0.0)	0 (0.0)	23 (0.1)	26 (< 0.1; 0.0 to 0.0)
No biomicroscopy recorded***	81 (1.2)	96 (1.2)	128 (1.5)	142 (1.7)	83 (0.7)	147 (1.1)	124 (0.9)	48 (0.3)	63 (0.4)	66 (0.4)	126 (0.8)	1104 (0.8; 0.8 to 0.9)
Screen-negative	6110 (92.1)	7374 (91.2)	7742 (89.4)	9412 (91.7)	10,519 (93.8)	12,203 (93.0)	12,744 (94.3)	14,446 (95.6)	14,747 (94.8)	14,249 (93.3)	15,102 (93.8)	12,4648 (93.3; 93.2 to 93.5)

The annual incidence of screen-positive owing to DR appeared to decrease from 4–5% (2006/07 to 2009/10) to 2–3% (2010/11 to 2016/17). The rates of screen-positive for unassessable images appeared to fluctuate by around 2–3% (highest proportion 4.1% in 2008/09, lowest proportion 1.2% in 2013/14) and for other eye disease was stable (around 1%) apart from a peak in 2008/9 (1.9%). Rates of STDR (true-positive, examined by a medical retina specialist using biomicroscopy) were relatively stable at around 1.5–2% [sight-threatening retinopathy (STR) around 0.6%, sight-threatening maculopathy (STM) 1.0–1.5%]. Figure 9 illustrates these trends.

FIGURE 9.

Plot of overall screen-positive/STDR/STR annual incidences 2006/07–2016/17 (out of number of people with at least one attended screening appointment) by screening year (with 95% wilson score cis). STM not shown.

Of the 133,554 screening episodes across the 11 years studied, 6.7% (8906) were screen-positive, 3.0% (4073) were screen-positive for DR, 2.6% (3444) due to ungradeable images and 1.0% (1389) due to other significant eye disease.

Of the 7802 screen-positive individuals who attended the HES and were examined by a medical retina specialist, only 28.1% (2192) were confirmed as having STDR [overall incidence 1.6% (2192/133,554)], with 71.6% (5584/7802) having unassessable images, other disease or a false positive. A total of 53.8% (2192/4073) of the people referred with screen-positive disease due to DR were true-positive. In total, 82.2% (1801/2192) of individuals had STM and 35.6% (781/2192) had STR.

Discussion

Our data from an established screening programme give up-to-date estimates of current incidences of key stages in the diabetes care pathway of importance to patients, namely progression to screen-positive, screen-positive because of DR and STDR. The annual incidence of screen-positive cases showed a consistent fall over a decade from 8–10% to around 5–6%. The rates of screen-positive attributed to DR also fell over time to around 3%, slightly lower but consistent with a referable retinopathy rate (equivalent to our screen-positive because of DR) of 4.3% reported in 2013 from Scotland. 9 Both of these trends occurred against a background of stable rates of true-positive STDR. These reductions require further investigation in other screening programmes. They may represent improvements in grading.

The rates of STDR were stable and consistently under 2.0% apart from 1 year early in the study. These low rates have also been reported in 2015 from Gloucestershire, a programme similar to ours in Liverpool. 42

The strengths of our study include the size of the data set, the duration of data collection and the robust analysis. Previous studies were from several decades ago, when levels of diabetes control were poorer. HbA_1c levels were generally well controlled in our population.

The main limitation of this study is the nature of data availability, which meant that the data set was partly retrospective. Around 5% of the population died or moved away each year and without consent we were unable to access their historical data. This censoring could have affected our estimates of DR and STDR between 2006/7 and 2012/13.

The ISDR RCT commenced in 2014, resulting in a proportion of patients moving to 2-yearly screening. We corrected for this effect in the last two years of our analysis.

In Liverpool in 2017/18, 9.4% (2054/21853) people were attending the hospital for management of their diabetes or slit-lamp-based screening. Nonetheless we are able to estimate a rate of STDR in people attending screening of 164/10,000 PWD/annum (2192/133,554*10,000), useful data for people running established screening programmes and in designing future RCTs. Around 82% of these people had STM and 36% STR.

We detected a shift in ethnicity over the 11 years of the study but the data were difficult to interpret due to the increase in unknown data on ethnicity caused by changes in the methods of collection.

Fewer than half of people graded as screen-positive were considered to have DR by the grading teams and around half of these were judged as true-positive in medical retina clinics. A debate needs to take place about these levels of positivity, preferably with patient groups and other stakeholders. This is especially clear in new screenees, where only a 1/5th (21.7%) of people identified as screen-positive had true-positive disease. Much of the screen-positive disease is due to unassessable images, most likely due to cataract, and other significant eye disease, none of which are the objectives of DR screening.

In spite of the high levels of previously undetected cataract and other eye disease (80% of screen-positives due to unassessable images or other significant eye disease), people undergoing their first screening visit did have higher rates (37.5%) of STDR, suggesting that these individuals might be targeted specifically in screening programmes. We detected screen-positive rates in these new screenees of 9.9%. In Hong Kong, which introduced screening in 2015, a similar 10% of screen-positives were detected in their first year of screening a population that had not been screened previously. 65

Interrelation to other work packages and overall aims of the programme

This work package was supported by work from the PPI group around patient-centred outcomes as well as data collected in the ISDR DW (WP B1).

Research outputs

Programme outputs (work package G) as of 30 September 2021 are listed in Peer reviewed, full publications and Other publications.

Dissemination activities

Simon Harding, programme CI, is a member of the Research Advisory Committee, English NDESP and reports on findings and policy implications of ISDR. He has attended UK and international conferences 2017–2020 raising awareness of individualised screening.

We held a dissemination event (25 September 2019) for the screening community and commissioners from the home nations and outside the UK and to involve patient groups.

There was general support on safety of switching to extended intervals in stratified and/or personalised/individualised screening. Scale-up through a whole population ‘pilot’ was considered reasonable. Failsafe and communication needed revising.

Facilitated discussion identified a number of issues in the ISDR data set to be addressed:

• 3503 people did not consent for the RCT during recruitment

• external validation was needed in another UK programme

• further analysis was recommended: non-consented group, model cohort data set through RCE, compare with the stratification model to be used in the NDESP.

Further analysis

The team conducted further analyses within the programme in response to the symposium (see Supplementary Material 4).

People who did not consent to recruitment in the RCT were more likely to be female, older, have longer disease duration, be smokers (all p < 0.001) and had slightly higher screen-positive rates (p = 0.11). They had similar other clinical risk factors. This is likely to reflect older people being less likely to agree to participate in research.

Allocations by the RCE to 6-, 12- and 24-month intervals in the non-consenting group in the RCT (n = 3180) were similar to those who consented. For the ISDR cohort study data set there was a slightly higher (4.5%) allocation to 6 months (high-risk) and a corresponding reduction in the 24-month (low-risk) group. In scaling-up to whole populations there may be a small reduction in resources being freed up and a small increase in the high-risk group.

We compared IDSR variable-interval risk-based screening with the planned English NDESP stratified screening30 by applying the proposed rules to the RCT data set. We estimated that the NDESP stratification will allocate 33.1% of the RCT group to 12-months rescreening and 66.9% to 24-months (ISDR 82.3%), a 26% reduction in the move to an extended interval.

We modelled a ‘virtual RCE annual review’ as a failsafe for people being allocated to extended interval to run at 12 months using updated clinical data and baseline grading. We ran the RCE in the 24-month RCT group (n = 1776) at 12 months; 97.1% (1725/1776) remained in the low-risk group. Two people (0.1%) were reallocated to 6 months and 49 (2.8%) to 12 months.

The Liverpool RCE was ‘re-run’ at the end of the study to act as an internal validation and check of robustness of the model, which showed the RCE model is ‘stable’ in the Liverpool population.

An executable RCE package has been produced. Independent external validation will be completed (at the time of writing, under way in partnership with the Diabetes Research Unit Cymru). We have designed a pilot phase IV whole-population scale-up in Liverpool to start in 2022 (at the time of writing under review by the UK National Screening Committee).

Intellectual property

The team has worked with the RLBUHT Innovation Team’s partner, 2Bio Ltd, throughout the programme. Two components have been assessed as having potential for IP, the RCE and the DW.

RCE

• mathematics, covariate selection, import scripts, imputation protocols

• potential to function as a stand-alone clinical tool.

DW

• SQL server integration service protocols, real-time DW, imputation and credibility systems

• potential to be developed as a NHS clinical tool.

In any future commercialisation or deployment of these systems re-writing in a more portable and commercially attractive programming language will be required. The original code base built in MATLAB and MatSoap for the RCE has been ported into Python and validated using risk-calculations from the RCT.

The RLBUHT Innovation team will support the process of commercialisation, including discussions and negotiations with potential partners and licensees.

Reflections on the programme

A PGfAR offers long-term funding for a large piece of work. Much can be achieved. We completed a large RCT and produced evidence of global significance which will underpin the next 10–15 years of development of DR screening. We completed a large qualitative study that will underpin the future implementation of screening and diabetes services, with relevance outside ophthalmology. We undertook micro-costing and economic analyses using real-world resource data rather than modelling. We overcame the challenges of data access and variability to develop a data linkage and integration system implemented into NHS clinical care. We built and tested a RCE for ophthalmology, tuneable for local populations.

Our findings have given new impetus to the introduction of extended screening in England, due to be introduced in 2022/23. Our RCT has given impetus to the development of variable-interval risk-based screening which is being rolled out in other countries, including Hong Kong, Denmark and Singapore. Our team are engaged in global DR screening research in Malawi, Malaysia and China. Our expertise has been utilised in 2020 by WHO Europe in developing guidance for policy-makers on DR screening. 66

We were not able to deliver on all aspects of the planned work. The recruitment of such a large sample required screening of over 8000 individuals, each requiring identification, information and consenting. It took time to mobilise recruitment teams and we were about 12 months late in delivering the recruitment milestone. The teams that we had originally planned for proved unavailable at a key stage. Accessing the data proved more challenging than expected. The multiple data platforms and the poor or absent documentation prevented us from accessing all the planned data types. We received substantial additional research capacity and sustainability funds from our sponsor at two key stages, to expand the recruitment teams and for the data engineering, both of which we had underestimated in our funding model. We also needed to reallocate resources from our observational cohort study to support the RCT. This meant that we cut the section in WP B on collection of VA and grading data from the HES, to the significant disappointment of the PPI team. Nonetheless we did complete a longitudinal cohort study on over 28,000 individuals attending the screening programme.

Our development grant phase was essential. It allowed us to develop a comprehensive and effective team of co-investigators including a patient representative and a local manager and develop relationships with key stakeholders especially in primary care. We addressed key areas of uncertainty:

• We were able to identify and extract sample data from the widely varying domains in primary care, the screening programme and the hospital.

• We explored and made progress with the challenging governance and data-protection environments in the NHS.

• We built a demonstration version of the RCE.

Managing a long and complex programme of work must not be underestimated. It requires substantial commitment from the CI and an effective and able programme manager. The role of the WP leads is critical. Regular investigator meetings are essential to maintain enthusiasm and engagement. Inevitably co-investigators’ priorities can change or there can be extended periods of leave. Effective support for the CI from senior members of research departments can help maintain momentum or input additional resources and this needs to happen early. An effective Programme Steering Committee is a ‘critical friend’ at challenging times.

Strengths and limitations

Our programme provides for the first time RCT evidence on individualised DR screening with a full economic evaluation with the quality assured by a clinical trials centre. Recommendations are supported by qualitative evidence, not previously reported, and an active PPI group was embedded throughout. Our findings do come with a number of limitations, which have been described above. Here we briefly summarise the most important of these.

Participants were enrolled from a long-established programme with low rates of DR and progression to STDR. Good glycaemic and BP control and a relatively low proportion of type 1 diabetes (4.0%) might have biased the sample. Findings may not be generaliseable across other programme in the UK with differing ethnicity. Low rates of DR have been reported in other similar settings, but our results should be treated with caution in areas with a higher prevalence, poorer control of diabetes or wider ethnic mix, or in programmes during the set-up process.

Our trial had only a 2-year time horizon, which is short in the context of a life-long condition. With a move to extended-interval screening in several countries, we wanted to provide high-quality RCT evidence on how people act when given variable-interval risk-based screening. The study was designed to compare safety and effectiveness of two screening regimens, not the cost-effectiveness of screening versus no screening. To allow for two cycles for the low-risk group would have extended the study duration from 4 to 6 years. The low rates of retinopathy and STDR in the low-risk group suggest that our findings on effectiveness would be unlikely to change; most disease was detected in the high- and medium-risk groups. Our relatively short time horizon could have been mitigated by an extensive risk-based economic model based on the RCE (described in Economic model) but the model ran into computational difficulties. It will be important in future testing to recognise this limitation and ensure robust monitoring of attendance rates. It may also be informative to extend the cost-effectiveness analysis to include the whole care pathway, including downstream costs and outcomes.

Some of our secondary analyses within the individualised group relied on small numbers, especially in the high- and medium-risk groups. Conclusions based on these analyses need to be treated with some caution.

We were unable to recruit a cohort of regular non-attenders so cannot address this important issue. We were unable to complete the component of the cohort study in the HES, so were only able to provide data on incidences in the screening population.

Conclusions

Here we summarise the key findings:

• Evidence from the ISDR RCT shows that extended and personalised interval screening can safely be introduced in established screening programmes.

• Our data provide evidence that an individualised variable-interval risk-based approach has the potential to give improvements in cost-effectiveness without compromising safety or quality of life, but we did not include treatment costs in our economic analyses.

• Our qualitative evidence shows that PWD and HCPs appear to be supportive of variable intervals in screening but with important caveats. For successful implementation interpretable and clear safeguards against increasing non-attendance, loss of diabetes control and system failures are required. Alternative referral pathways are required for those lost to follow-up or whose risk factors change substantially over longer intervals.

• Variable-interval screening supported by clinical data allows 6-monthly targeting of a high-risk group, detecting disease earlier, while substantially reducing the number of appointments for a low-risk group.

• The rates of STDR in our long-established screening programme have dropped to a very low level, supported by other limited UK data. Higher rates are seen in those attending screening for the first time and with type 1 diabetes. The majority of cases classed as screen-positive do not have sight-threatening disease.

• Robust monitoring of attendance and retinopathy rates should be included in any wider testing. Our trial was over a 2-year time horizon when the disease has a long time-frame, and our findings come from a single centre with a relatively stable mainly white population.

• Findings from this research show that PPI can produce key decisions to underpin critical parts of the design and delivery of applied research. Strong commitment of the senior investigators is essential.

• The evidence from our programme and elsewhere shows that RCEs can be feasible, reliable, safe and acceptable to patients with the caveat that reliable monitoring and clear communication to stakeholders are necessary.

• Evidence from our programme has demonstrated that real-time integration of data from primary and secondary care sectors can be achieved, and indeed is necessary for the RCE, but relies on successful engagement of key stakeholders. This may not be feasible in some settings.

Recommendations for future research

Our recommendations are based on the evidence from the ISDR programme and elsewhere.

1. Individualised variable-interval risk-based screening is ready to be upscaled in a phase IV whole-population pilot. Evidence of repeatable results as seen in ISDR should be collected before considering wider dissemination.

2. Further health economics research is required to support potential adoption of individualised/personalised screening, including modelling beyond the 2-year time horizon and extending to include lifetime costs of treatment and care, and blindness averted.

3. The Liverpool RCE requires external validation with at least one other screening programme before it could be considered for adoption.

4. Further qualitative research is required around the potential adoption of extended and variable-interval screening to develop and implement safeguards and monitoring systems interpretable by end-users.

5. Evidence on efficacy in populations with higher rates of diabetes and different mixes of ethnicity, in newly commencing programmes and in low/middle-income countries is required.

6. New research is needed on methods to address the misunderstandings and conflation in PWD around their diabetes and eye care.

7. Evidence of current rates of VI should be collected in line with this being a key patient-centred outcome.

8. Novel approaches to recruit non-attenders to future qualitative research need to be developed for research into non-attendance.

9. Research is needed to establish more appropriate referral thresholds in DR screening.

The knowledge base in the field of screening for DR is changing in some areas. Technology is developing around automated grading and artificial intelligence. Some work is under way to integrate within screening but progress is not rapid. Digital healthcare is being developed with initiatives on big data and data integration which will support future personalisation. However, these approaches are still yet to be embedded in large health systems such as the NHS. Large epidemiological studies especially on important functional patient-centred outcomes are few, with much of the data being outdated and with limited current and future work planned. Pilot studies on introduction of new technologies are being considered and may move the field forwards in the medium term. The qualitative evidence base is small with little of relevance to screening and is unlikely to move forward rapidly.

Contributions of authors

Simon Harding (https://orcid.org/0000-0003-4676-1158) (Chair Professor, Clinical Ophthalmology) was CI on the programme and drafted and produced the final report including revisions.

Duncan Appelbe (https://orcid.org/0000-0003-1493-0391) (Senior Research Information Specialist) was the information-systems manager for the CTRC.

Ayesh Alshukri (https://orcid.org/0000-0003-0852-2068) (Research Associate, Computer Science) was part of the observational cohort, RCE and RCT teams.

Deborah Broadbent (https://orcid.org/0000-0002-3195-5024) (Honorary Senior Lecturer, Ophthalmology) was an applicant on the grant and led the RCT.

Philip Burgess (https://orcid.org/0000-0002-3959-2299) (Clinical Lecturer, Ophthalmology) was part of the observational cohort team.

Paula Byrne (https://orcid.org/0000-0001-8090-6184) (Senior Lecturer, Sociology of Health and Illness) led the qualitative team.

Christopher Cheyne (https://orcid.org/0000-0002-2330-9559) (Research Associate, Biostatistics) was part of the team that conducted the statistical analysis.

Antonio Eleuteri (https://orcid.org/0000-0003-0718-6672) (Honorary Lecturer, Medical Physics and Clinical Engineering) built the RCE.

Anthony Fisher (https://orcid.org/0000-0001-9086-8812) (Consultant Clinical Scientist, Medical Physics and Clinical Engineering) was an applicant on the grant and led the RCE team.

Marta García-Fiñana (https://orcid.org/0000-0003-4939-0575) (Professor of Biostatistics) was an applicant on the grant and led the statistics team.

Mark Gabbay (https://orcid.org/0000-0002-0126-8485) (Professor of General Practice, Health Services Research) was an applicant on the grant and contributed to the qualitative work package.

Marilyn James (https://orcid.org/0000-0003-0408-7898) (Professor of Health Economics) was an applicant on the grant and led the health economics team.

James Lathe (https://orcid.org/0000-0002-5416-0873) (Health Economist) was part of the team that conducted the cost-effectiveness analysis.

Tracy Moitt (https://orcid.org/0000-0002-5579-996X) (Senior Trials Manager) was an applicant on the grant and led the trial management.

Mehrdad Mobayen Rahni (https://orcid.org/0000-0002-1278-8558) (Lead Database Analyst, Business Intelligence) was the data warehouse designer and manager.

John Roberts (https://orcid.org/0000-0003-3392-7243) (Patient representative) was an applicant on the grant and led the involvement of the PPI group.

Christopher Sampson (https://orcid.org/0000-0001-9470-2369) (Health Economist) was part of the team that conducted the cost-effectiveness analysis.

Daniel Seddon (https://orcid.org/0000-0002-0296-6234) (GP, NHS England) was an applicant on the grant, represented the end-user and contributed to several work packages

Irene Stratton (https://orcid.org/0000-0003-1172-7865) (Senior Statistician, Medical Statistics) was an applicant on the grant and provided senior advice and input to the statistical team and all aspects of the programme.

Clare Thetford (https://orcid.org/0000-0003-2188-3052) (Research Fellow) was part of the qualitative team.

Pilar Vazquez-Arango (https://orcid.org/0000-0002-4117-7676) (Research Coordinator, Ophthalmology) was responsible for project management (2018–20) and with Simon Harding produced the final report.

Jiten Vora (https://orcid.org/0000-0002-3488-5287) (Professor/Consultant Endocrinologist) was an applicant on the grant and led the observational cohort study.

Amu Wang (https://orcid.org/0000-0002-1764-6524) (Honorary Research Associate) was responsible for project management (2013–18).

Paula Williamson (https://orcid.org/0000-0001-9802-6636) (Professor of Medical Statistics) was an applicant on the grant and Director of the Liverpool CTRC.

Publications

Cheyne CP, Burgess PI, Broadbent DM, García-Fiñana M, Stratton IM, Criddle T, et al. Incidence of sight-threatening diabetic retinopathy in an established urban screening programme: an 11-year cohort study. Diabet Med 2021;38:e14583.

Eleuteri A, Fisher AC, Broadbent DM, García-Fiñana M, Cheyne CP, Wang A, et al. Individualised variable-interval risk-based screening for sight-threatening diabetic retinopathy: the Liverpool Risk Calculation Engine. Diabetologia 2017;60:2174–82.

Byrne P, Thetford C, Gabbay M, Clarke P, Doncaster E, Harding SP, ISDR Study Group. Personalising screening of sight-threatening diabetic retinopathy – qualitative evidence to inform effective implementation. BMC Public Health 2020;20:881.

Broadbent DM, Wang A, Cheyne CP, James M, Lathe J, Stratton IM, et al. Safety and cost-effectiveness of individualised screening for diabetic retinopathy: the ISDR open-label, equivalence RCT. Diabetologia 2021;64:56–69.

Peer-reviewed, full publications

Other publications and outputs

Data-sharing statement

All requests for data should be sent to the corresponding author. Access to available anonymised data may be granted following review.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that they are stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, CCF, PGfAR or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the PGfAR programme or the Department of Health and Social Care.