Surveillance for ocular hypertension: an evidence synthesis and economic evaluation

Moorfields Eye Hospital

The Moorfields Eye Hospital data set comprises data from a RCT of medical treatment with betaxolol eye drops (FDC International Ltd, Fareham, UK) or placebo in OHT. 32 In the original study 356 patients with OHT were randomised during the period 1992–6.

Eligible participants for the trial included those:

• aged > 35 years with an IOP, by GAT, between 22 mmHg and 35 mmHg and

• with no significant ocular or neurological pathology that would affect visual field performance and

• with visual acuity of 6/12 or better and

• with normal visual fields on Humphrey SAP defined as a mean Advanced Glaucoma Intervention Study (AGIS)33 score on two baseline fields of 0.

There were criteria violations: three participants had an IOP > 35 mmHg on the day of treatment commencement (36, 37 and 40 mmHg) and there were two participants aged < 35 years. Data from these participants were included in the analysis.

Participants were followed up every 4 months with Humphrey SAP, IOP measurement and optic nerve imaging in the study eye for a minimum of 2 years. Treatment status was noted at each visit. Conversion to glaucoma was based on predefined criteria (Box 1). Any participants reaching the conversion end point had IOP lowered by a topical ocular hypotensive other than betaxolol.

For the purpose of the secondary analysis of the trial data set for this study, the variables required were not available for 56 of the 356 trial participants and thus 300 participants are included in the analysis. The Moorfields Eye Hospital data set provides valuable RCT data and, additionally, post-trial surveillance with repeated measures of visual field using SAP. A flow diagram to illustrate the selection of patients to be included in the analysis for the Moorfields cohort is provided in Figure 5.

Rotterdam Eye Hospital

The Rotterdam Eye Hospital data set comprises data from a placebo-controlled, double-masked, prospective, three-arm RCT (placebo, timolol 0.5% twice a day, betaxolol 0.25% twice a day) that was completed on 31 August 2008. This study randomised 412 participants with OHT between November 1997 and March 2001, including both eyes, with raised IOP ≥ 22 mmHg and ≤ 32 mmHg by GAT, and normal visual fields on Humphrey automated perimetry. Participants were deemed eligible if they were of white ethnic origin and had a best corrected Snellen visual acuity of at least 20/40. Participants with any significant coexisting ocular or systemic disease or any use of ocular hypotensives in the preceding 3 months were excluded. The follow-up entailed 6-monthly IOP measurement, visual field analysis and imaging of the optic disc using automated technology. 34 Conversion to glaucoma was based on predefined criteria (Box 2).

For the secondary analysis of the data for the purpose of this study, participants who had used eye drops for at least 6 months or who had not had baseline visual field testing on at least three occasions were not included in the data set. A total of 16 participants were excluded on this basis and thus data on 396 participants were potentially available for analysis from the study start date in November 1997.

On 31 August 2008, the end date for the study, data on 393 participants were available for analysis: two participants did not have follow-up data and were excluded and on reviewing the baseline visual field data one participant was deemed to have early glaucomatous visual field defect at baseline and was excluded from the risk prediction analysis (see Chapter 4). A flow diagram to illustrate the selection of patients to be included in the analysis for the Rotterdam cohort is provided in Figure 6.

Dunfermline

The Dunfermline data set was collated on an electronic patient record of all referrals with confirmed OHT from the year 2000 up until the end of December 2010. A flow diagram to illustrate the selection of patients to be included in the analysis for the Dunfermline cohort is provided in Figure 7.

The Dunfermline guidelines for the management of their ocular hypertensive patients recommend that low-risk cases (see Box 3 for definition) are discharged after two visits (at 1 year); those who develop glaucoma (Box 4) and those deemed as being at high risk of developing glaucoma remain under review.

Visual field data using Humphrey SAP were available for both eyes of all patients (except those unable to complete the test).

Nottingham

The Nottingham data set includes data from a cohort of 185 patients with OHT followed in a community scheme run by locally trained optometrists. The scheme started in 2003 and we included those who had at least 1 year of follow-up at the time of ascertaining the data set for this study in January 2010. Of the 185 patients recorded, 26 did not have follow-up data. Thus, data on 159 participants were analysed. Measurements on both eyes were available. A flow diagram to illustrate the selection of patients to be included in the analysis for the Nottingham cohort is shown in Figure 8.

Patients were eligible to enter the scheme if they were being followed up for OHT in hospital-based clinics and had normal visual fields and normal optic discs and an IOP recorded as > 21 mmHg on at least one occasion. Patients had to be able to undergo reliable visual field testing and have a visual acuity better than 6/12, and patients with both treated (single drop only) and untreated OHT were eligible. At baseline a visual field test was performed (24-2 SITA Fast) and stereoscopic optic disc imaging was performed as a baseline assessment of optic disc appearance/morphology. The community optometrists were provided with stereo viewers and stereo photographs of the patients’ optic disc appearance at entry for comparison throughout follow-up. In addition, the referral guidelines to the community-based service indicated what cut-off of IOP should prompt referral back to the hospital eye service. Patients were reviewed once a year by visual field tests, GAT and clinical disc examination. Treatment state was recorded at each visit. Patients were referred back to the hospital eye service if the optometrist undertaking the annual assessment judged the optic disc appearance to have changed (comparing it with the baseline appearance) or noted a visual field defect on visual field testing or if the patient was unable to undergo reliable visual field testing or the IOP had become elevated above the cut-off indicated in the referral template. All patients were referred back to the hospital eye service at 6 years for assessment by a consultant ophthalmologist. Patients were then returned to the community-based service to continue their annual follow-up.

Criteria for discharge from the monitoring scheme and the definition for conversion to glaucoma are described in Boxes 5 and 6.

Data synthesis

Meta-analysis of each predictor was not performed because the effects of other predictors in a multivariate model are not accounted for and so this would be misleading. Each model was reviewed and summarised separately.

Model validation

Baseline data from the four cohorts described in Chapter 3 were used to validate each included prediction model. However, for the Rotterdam cohort, with the exception of VCD ratio measurements, which were taken 2–3 years after randomisation, we used ocular measurements taken at the first follow-up visit (usually 1 month after randomisation) because of learning curve errors in the visual fields taken at baseline. Because the vertical VCD ratio is used in the assessment of OAG, if the time interval between measurement of the vertical VCD ratio and conversion to OAG was < 12 months, the measurements were not used and were considered missing. Sensitivity analysis was performed to assess the effect of using a time interval of < 24 months. The four data sets were analysed individually to enable assessment of the transferability of models between different geographical locations and settings.

The missing data mechanism was investigated in each cohort. To make the best use of the available data and to avoid bias, missing values were imputed using multivariate imputation by chained equations. 56 By replacing missing values with plausible values based on the distribution of the observed data, 10 imputed data sets were created based on the predictors and outcome variables (Nelson–Aalen estimator for time to glaucoma and the censoring indicator). To avoid excluding a completely missing predictor from the risk estimation, we imputed the average value from a similar data set. Visual inspection of the distribution of imputed values was used to assess comparability with the observed data. For comparison with analyses based on imputed data, complete case analyses were also performed.

Ideally, model validation should be performed using an untreated population but this approach would have more than halved the sample size of some cohorts. Thus, we performed Cox regression analysis with and without stratification by treatment on each imputed data set. Estimates of the coefficient of each predictor were averaged across the 10 data sets to obtain a single estimate. Standard errors were computed based on Rubin’s rules,57 which take into account the between- and within-imputation components of variation in the parameter estimates.

The 5-year risk of developing glaucoma for each patient was calculated using each included prediction equation and the patient’s observed or imputed value of the predictors. The predicted 5-year risk of glaucoma was grouped into three categories, low (< 6%), intermediate (6–13%) and high (> 13%), as defined by the OHTS. 13 Five-year Kaplan–Meier failure curves for each cohort were plotted according to this risk classification and log-rank tests performed to assess equality of the survivor function across groups (note that each plot and the associated log-rank test depicts a single imputed data set as an example).

The predictive ability of a model is frequently assessed in terms of discrimination and calibration. 58 The ability of the model to discriminate between patients who did or did not develop OAG was assessed using Harrell’s c-index. This measure is similar to the area under the receiver characteristic operating curve and is more appropriate for survival analysis. 59 A c-index of 1.0 indicates perfect discrimination (e.g. predicted risks for those with OAG are all greater than for those without OAG, with perfect separation of both groups) whereas a c-index of 0.5 indicates random discrimination. Approximate 95% CIs were computed for the c-index by applying Rubin’s rules to jackknife standard errors obtained for the c-indexes of the 10 imputed data sets. Where the upper limit of the 95% CI exceeded 1.0 the value was truncated at 1.0.

Calibration refers to the agreement between predicted and observed risk. Model overfitting (where predictions in the model are more certain than justified because of the play of chance in the derivation data set) can be identified by assessing model calibration in independent validation data sets. Patients were divided into quintiles according to their predicted risk. Within each quintile, the average predicted risk was compared with the corresponding Kaplan–Meier estimate of the observed risk. Calibration plots were used to illustrate the fit of the models; for a perfectly calibrated model, all points will lie on the 45° line. Additionally, we used the slope of the prognostic index (linear predictor), known as the calibration slope, to quantify calibration. The slope of the prognostic index is the regression coefficient β in a Cox regression model with the prognostic index as the only covariate. The calibration slope should ideally be 1 when predicted risks agree completely with observed risks,60 which will be the case if the effects of predictors in the validation data set are on average similar to those in the development data set (note that, although the tests and summary statistics are based on appropriate analyses of all 10 imputed data sets, the plot depicts a single imputed data set and thus may not directly depict the final results). Because pretreatment values of the baseline predictors were known in the trial-based cohorts, we produced calibration plots for treated and untreated patients separately to assess the extent to which calibration of the model was affected by treatment. Calibration and Kaplan–Meier plots were based on one of the imputed data sets for each cohort. All analyses were performed using Stata 11.1 (StataCorp LP, College Station, TX, USA).

Study selection

The titles and abstracts of 565 articles were screened to identify eligible studies. Full-text papers of 54 of these were retrieved for more detailed evaluation of eligibility (Figure 9). Forty-nine papers were excluded and the reasons are provided in Appendix 2. Of the five included papers, four were based on the results of two RCTs, the OHTS61 and the EGPS. 14,41 These provided three models for which prediction equations were available (full and reduced OHTS models42,62,63 and the pooled OHTS-EGPS means model64,65). The fifth paper reported the independent validation study of the OHTS model in the Diagnostic Innovations in Glaucoma Study (DIGS) cohort. 63

FIGURE 9.

Flow chart of the study selection process.

Assessment of study quality

Both the OHTS and EGPS were large prospective studies that included patients with OHT aged ≥ 30 years who had no evidence of glaucomatous damage at baseline. The OHTS randomised 1636 individuals, with an IOP between 24 mmHg and 32 mmHg in one eye and between 21 mmHg and 32 mmHg in the other eye, to treatment or observation. The EGPS randomised 1081 individuals with IOP ≥ 22 mmHg in at least one eye to treatment or placebo. The inclusion and exclusion criteria used in the DIGS cohort were very similar to those used by the OHTS. For a patient to be included, his or her baseline IOP had to be ≥ 24 mmHg in one eye and ≥ 21 mmHg in the other eye, on at least two occasions.

In the OHTS, all visual fields were assessed using full-threshold white-on-white Humphrey program 30-2 perimetry, whereas in the EGPS this was used for 79.6% of the participants and Octopus 32-2 visual fields for the rest. Visual field assessment was based on three consecutive abnormal and reliable tests with defect in the same location and index. In the DIGS, visual field examination was performed with SAP. At baseline, SAP testing was performed using the program 24-2 full-threshold strategy, but during follow-up, SAP testing was performed using either the full-threshold or the Swedish Interactive Threshold Algorithm strategy. 63 Using different visual field programs based on different algorithms is not ideal, but in a clinical context of determining whether or not glaucoma has developed it is acceptable. In the OHTS, optic discs were assessed using two consecutive sets of photographs judged to have a clinically significant change, whereas the EGPS used a set of photographs judged to have changed by at least two of three masked readers. In both studies, detection of OAG was by a masked end-point committee and was based on reproducible visual field abnormalities or optic disc deterioration. In the DIGS, structural damage to the optic disc at baseline was based on assessment of simultaneous stereoscopic optic disc photographs by two masked readers; conversion from OHT to OAG was based on the development of a reproducible visual field defect or glaucomatous change in the appearance of the optic disc in at least one eye. The original protocols of the OHTS and EGPS did not include CCT and measurements were taken later, 2–3 years after randomisation of the last patient enrolled in the studies. All patients in the DIGS had CCT measurements taken during follow-up.

Cox regression analysis was used to obtain all of the prediction models and all continuous predictors were modelled without categorisation. The OHTS identified candidate variables during the planning phase of the study. For the pooled OHTS-EGPS model, variables in univariate models in the OHTS or the EGPS with p < 0.10 were the candidate variables. Statistically significant variables in the pooled univariate analyses were included in the pooled multivariate analyses. Measures of model performance in the development set were not reported for the OHTS models.

Assessment of a patient’s risk requires the baseline hazard/survivor function in addition to the prognostic index. In the validation of the OHTS models, the baseline survivor function was approximated by the average survival probability at 5 years obtained from the Kaplan–Meier estimates because the function was not published by the OHTS. 63 The OHTS reduced model was published in a brief report,62 and the same population and analyses described for the full model applies. The methodological quality of the two studies42,64 that fully described the development of the OHTS full model and the OHTS-EGPS means model, respectively, and the validation study in the DIGS cohort63 is summarised in Table 2. Unlike the OHTS models, the OHTS-EGPS model has not been independently validated. The model derived from the observation group of the OHTS was validated in the EGPS placebo group, but the pooled OHTS-EGPS model derived from both populations was not externally validated.

Ocular Hypertension Treatment Study prediction model

The OHTS multivariate Cox regression analysis used the means of both eyes for eye-specific variables and identified six factors that were significantly associated with the risk of conversion from OHT to OAG: age, IOP, CCT, VCD ratio, PSD and diabetes mellitus. 42 Horizontal C/D ratio was also significantly associated with glaucoma development in multivariate analysis but, because of its high correlation with the VCD ratio, only the VCD ratio was included in the final multivariate model. Also, the VCD ratio had slightly better predictive ability. Analyses were performed using only the untreated group as well as the combined treated and untreated groups. Similar results were obtained and only the model based on the entire sample was reported.

Changes in PSD and VCD ratio may be indicative of early OAG damage and their inclusion in the model may result in an overestimation of risk. Thus, to avoid inclusion of variables used in the definition of the disease, another model, the reduced OHTS model, was developed. 62 HRs from the multivariate Cox models with and without PSD and VCD ratio were found to be similar and in the reduced model all factors remained statistically significant.

Using 252 eyes from 126 untreated OHT patients retrospectively selected from DIGS, a prospective longitudinal study, Medeiros and colleagues63 validated the OHTS prediction model. Fifteen (12%) of the 126 patients received treatment during follow-up, and for these patients only the pretreatment period was evaluated. To derive the OHTS prediction equation, the average survival probability at 5 years was approximated using the Kaplan–Meier estimates because the baseline survivorship function was not included in the published OHTS results. For the full OHTS model (containing all six predictor variables), the risk estimate for glaucoma development in 5 years was calculated as:

where 0.906 is the average survival probability at 5 years and

where β_i is the regression coefficient in the Cox proportional hazards model for the ith predictor and is the log HR, X_i is an individual’s value for one of the k predictors and x¯i to x¯k are the mean values from the OHTS cohort for the k predictors. Four of the five predictors were transformed as follows:

The ‘DIABETES’ variable takes the value 1 if the patient has a history of diabetes mellitus and 0 otherwise. VCD is the vertical C/D ratio. A risk scoring system based on this model was also developed.

For the reduced model, excluding the VCD ratio and PSD, the risk estimate was calculated as:

where

The full model containing the VCD ratio and PSD performed similarly to the reduced model that excluded these variables. In updated univariate and multivariate analyses of the OHTS model, the protective effect of a history of diabetes mellitus was no longer statistically significant. 66 The authors suggest that the difference from the 2002 article was a reflection of more complete ascertainment of diabetes mellitus. Also, individuals with diabetic retinopathy were excluded from the OHTS and so the patients in the study may not be representative of patients with diabetes.

Ocular Hypertension Treatment Study-European Glaucoma Prevention Study prediction model

A prediction model was developed from the OHTS observation group and validated in the EGPS placebo group. 64 Unlike the OHTS, the EGPS included patients with pigment dispersion syndrome or pseudoexfoliation syndrome (PEX), conditions that may lead to secondary OAG. However, the 19 patients with either condition were excluded from the analyses. In both the OHTS and the EGPS, interactions were not detected between predictors. 64 The final model was derived from both groups combined. The OHTS-EGPS model estimates the 5-year risk of developing glaucoma based on age, IOP, CCT, VCD ratio and PSD. This model uses the means of the right and left eyes of each participant to calculate eye-specific predictors. Similar results were obtained when the pooled analysis was performed with and without the EGPS participants with only one eye eligible for the study and so they were all included in the analyses.

The OHTS and EGPS groups further assessed whether the accuracy and clinical application of the OHTS-EGPS model based on the means of the right and left eyes, the ‘means model’, are equal, superior or inferior to those of prediction models that include eye-specific information. 65 These models include the ‘means plus asymmetry’ model, which includes age and the means of the right and left eyes as well as the absolute difference between eyes for eye-specific variables, and the ‘worse’ eye model, which includes age and values from the eye at higher risk for developing OAG. The ‘worse’ eye was selected using the OHTS-EGPS means model, which was developed from the same sample as the ‘worse’ eye model, thus introducing statistical circularity and increasing the predictive accuracy of the ‘worse’ eye model. The prediction model that uses the means of both eyes for eye-specific variables was reported as the simplest to use and the most robust to measurement variability and error. 65 Using the same transformation as in the OHTS model for age, CCT, PSD and VCD, the risk estimate for glaucoma development in 5 years was calculated as:

where

A calculator to estimate the 5-year risk of developing OAG, based on the pooled OHTS–EGPS means model, is available online at http://ohts.wustl.edu/risk/calculator.html. The calculator truncates predictor values at the bounds of the values recorded for each predictor in the OHTS-EGPS cohort (see Table 8 for the lower and upper limit values of each predictor). To simplify risk assessment, a points-based system was also developed by the OHTS-EGPS group.

Missing predictors

All cohorts provided complete data on age. IOP was missing for a few patients, especially for those recently enrolled in the Dunfermline cohort. The frequency of missing data for each predictor in each cohort is shown in Table 4. The proportion of missing values for CCT was high in all cohorts (between 22.6% and 100%). CCT was sporadically collected for both the Moorfields and Dunfermline cohorts. However, in the Dunfermline cohort, CCT was not recorded in the electronic patient record system. To retrieve this information would have required manual retrieval of all case notes, an immense task with possibly very little yield. Therefore, we imputed the average value, 556 µm, from the Nottingham cohort to allow for calculation of each patient’s risk.

For the Rotterdam cohort, VCD ratio was measured from November 2000 onwards only and CCT from September 2003 onwards. Patients who dropped out or were excluded prior to these dates did not have either a VCD ratio (32.8%) or a CCT measurement (22.6%). For two patients, the time interval between measurement of VCD ratio and conversion to OAG was < 1 year and so their measurements were regarded as missing. Of the 393 patients in the cohort, there were 64 patients (16%) with at least two missing predictors and in all 64 patients both CCT and VCD ratio were missing. For the Moorfields cohort, there were 87 patients (29.2%) with multiple missing predictors, of whom both CCT and PSD were missing in 74. In more than half of the Moorfields cohort, PSD was missing because corrected PSD (CPSD) was recorded more frequently, with this being missing in only 24 (8.1%) patients. In contrast to the Rotterdam and Moorfields cohorts, the Dunfermline and Nottingham cohorts included very few patients with multiple missing predictors – 14 (7.4%) and eight (5.0%), respectively.

Validation sample

The median follow-up times of our observational cohorts (2.7 years for Dunfermline and 4.3 years for Nottingham) were much shorter than those of our trial-based cohorts (8.2 years for Rotterdam and 9.3 years for Moorfields). The median follow-up time was 4.8 years and 6.6 years for the EGPS and the OHTS, respectively. The Kaplan–Meier estimate of the 5-year cumulative probability of developing OAG was 9.3% in the OHTS observation group and 16.8% in the EGPS placebo group. These values were 4.0% (95% CI 2.3% to 6.9%) for Rotterdam, 11.3% (95% CI 7.8% to 16.4%) for Moorfields, 23.7% (95% CI 14.1% to 39.9%) for Dunfermline and 5.1% (95% CI 2.0% to 12.7%) for Nottingham. The baseline characteristics of the four cohorts were generally similar to those of the OHTS-EGPS cohort, although patients in our observational cohorts were on average older than those in the randomised cohorts (Table 5). On average, patients in the Moorfields cohort had a lower CCT than those in the OHTS-EGPS, Rotterdam or Nottingham cohorts. Average PSD in the group that converted to OAG and the group that did not were lower in the Rotterdam cohort than in the corresponding groups in other cohorts.

In univariate analysis, a history of cardiovascular disease was associated with OAG in the OHTS, the EGPS and the pooled OHTS-EGPS. This information was not recorded for the Rotterdam and Moorfields cohorts. Only a few of the Dunfermline cohort had a history of heart disease (10/188, 5.3%), and 5.0% of those who did not convert to OAG had heart disease, compared with 7.1% of those who converted. In the OHTS-EGPS, 7.8% of those who did not convert to OAG had heart disease, compared with 11.5% of those who converted. The proportions of patients for whom predictor values were outside the bounds of the OHTS-EGPS cohort is shown in Table 6. In 22.1% (66/298) of the Moorfields cohort, IOP was < 20 mmHg, and 12.0% (17/142) of the Dunfermline cohort had a PSD value > 3.

Model performance

Examination of the distribution of imputed values showed similarity with observed values and the imputed data were used to compute predicted risks. Using one of the 10 imputed data sets, an example of the distribution of predicted risks for those who developed and those who did not develop OAG is shown in Figure 10. Stratified (treated vs untreated) and unstratified analyses gave similar results, and so only the unstratified analysis is reported. Figures 11–14 show the Kaplan–Meier curves for the risk of conversion from OHT to glaucoma by risk group from the same imputed data sets used for Figure 10. Based on analysis of all 10 imputed data sets for the trial-based cohorts, there was a statistically significant difference in survival across risk groups. Multivariate HRs (and 95% CIs) for the risk factors and the performance measures of the model applied to each cohort using imputed data and complete cases are shown in Tables 7 and 8, respectively. For the Rotterdam and Dunfermline cohorts, there was little or no difference in model performance using multiple imputed data instead of complete case analyses. The converse was the case for the Moorfields and Nottingham cohorts. Data were complete for only 78 patients (26%) in the Moorfields cohort.

FIGURE 10.

Illustrative distribution of predicted 5-year risk in those who developed OAG and those who did not develop OAG during the follow-up period in each of the cohorts for the first imputed data set.

FIGURE 11.

Rotterdam cohort: illustrative Kaplan–Meier curve for the risk of conversion from OHT to glaucoma by risk category for the first imputed data set. Log-rank test: p < 0.001. The number at risk in each risk category – low (< 6%), intermediate (6–13%) and high (> 13%) – is shown at 2-yearly intervals. The numbers who converted to OAG during each time interval are shown in parentheses.

FIGURE 12.

Moorfields cohort: illustrative Kaplan–Meier curve for the risk of conversion from OHT to glaucoma by risk category for the first imputed data set. Log-rank test: p = 0.001. The number at risk in each risk category – low (< 6%), intermediate (6–13%) and high (> 13%) – is shown at 2-yearly intervals. The numbers who converted to OAG during each time interval are shown in parentheses.

FIGURE 13.

Dunfermline cohort: illustrative Kaplan–Meier curve for the risk of conversion from OHT to glaucoma by risk category for the first imputed data set. Log-rank test: p = 0.6. The number at risk in each risk category – low (< 6%), intermediate (6–13%) and high (> 13%) – is shown at 2-yearly intervals. The numbers who converted to OAG during each time interval are shown in parentheses.

FIGURE 14.

Nottingham cohort: illustrative Kaplan–Meier curve for the risk of conversion from OHT to glaucoma by risk category for the first imputed data set. Log-rank test: p = 0.08. The number at risk in each risk category – low (< 6%), intermediate (6–13%) and high (> 13%) – is shown at 2-yearly intervals. The numbers who converted to OAG during each time interval are shown in parentheses.

The c-indexes were between 0.69 and 0.83; the Rotterdam cohort had the best discriminatory ability (0.83, 95% CI 0.75 to 0.91), whereas the Moorfields cohort yielded the lowest c-index (0.69, 95% CI 0.59 to 0.78). Figures 15–22 illustrate calibration of the model in the four cohorts based on the same imputed data sets as in Figures 10–14. For the Rotterdam and Moorfields cohorts, the calibration plots are shown separately for the treated, untreated and combined data to highlight differences in calibration that may exist as a result of the effect of IOP-lowering treatment. The effect of treatment on calibration is not clear from Figures 15, 16, 18 and 19, although calibration appears to be better in the treated groups. However, these plots should be viewed cautiously because they were based on small study sizes and few conversions to OAG, and they depict only a single imputation, which may be atypical. The calibration slopes presented in Table 7 were computed based on the 10 imputed data sets. There was little variation in the estimated calibration slope between imputations for all data sets except the Moorfields cohort, and this may be owing to sensitivity to missing data. For the Rotterdam cohort, the calibration slope was close to 1 (1.09, 95% CI 0.72 to 1.46), the ideal value when there is complete agreement between predicted and observed risks, although the example calibration plot presented in Figure 17, based on a single imputation, shows an atypical relationship. The calibration slopes were 0.59 (95% CI 0.19 to 0.99), 0.21 (95% CI –0.33 to 0.74) and 0.24 (95% CI –0.17 to 0.65) for the Moorfields, Dunfermline and Nottingham cohorts, respectively. Values of the calibration slope that are < 1 represent overfitting, suggesting that, on average, the effects of the predictors were smaller in these cohorts than in the OHTS-EGPS cohort.

FIGURE 15.

Example calibration plot for the Rotterdam cohort for the first imputed data set: untreated group. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 16.

Example calibration plot for the Rotterdam cohort for the first imputed data set: treated group. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 17.

Example calibration plot for the Rotterdam cohort for the first imputed data set: all participants. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 18.

Example calibration plot for the Moorfields cohort for the first imputed data set: untreated group. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 19.

Example calibration plot for the Moorfields cohort for the first imputed data set: treated group. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 20.

Example calibration plot for the Moorfields cohort for the first imputed data set: all participants. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 21.

Example calibration plot for the Dunfermline cohort for the first imputed data set: all participants. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

FIGURE 22.

Example calibration plot for the Nottingham cohort for the first imputed data set: all participants. For each quintile, the average observed risk is plotted with its 95% CI against the predicted 5-year risk of developing OAG. The dotted straight line through the origin represents perfect calibration. The histogram above the horizontal axis represents the distribution of predicted risks.

Sensitivity analysis

Vertical cup-to-disc ratio was missing in > 30% of the Rotterdam cohort because the measurements were not taken at baseline. The time interval between VCD ratio measurement and conversion to OAG was < 24 months for five patients in the Rotterdam cohort. Sensitivity analysis using imputed values for these patients did not affect performance of the model (c-index: 0.83, 95% CI 0.75 to 0.91; calibration slope: 1.06, 95% CI 0.64 to 1.47).

Intraocular pressure and central corneal thickness

Intraocular pressure and age are important and well-established risk factors, but CCT was a major predictor for the development of OAG over 5 years. For the OHTS-EGPS model, there was an increase in risk per 40-µm decrease in corneal thickness (HR 2.04, 95% CI 1.70 to 2.45). The log-HR standardised according to the SD of the measurements of each predictor in the OHTS-EGPS cohort provides a measure of the contribution of each predictor in the model; CCT was the most important predictor. It is unclear to what extent lower CCT is responsible for the increased risk of OAG and previous studies have shown that CCT tends to decrease with increasing age. 71,72 It is also well known that corneal thickness influences the measurement of IOP because GAT has an inherent measurement error because of CCT variations; IOP is overestimated in thicker corneas and underestimated in thinner ones. 73–75 Nevertheless, there is no consensus on the clinical significance of the effect of CCT on IOP measurements. 76 No correlation was found between CCT and IOP (or any of the other predictors) in either the OHTS or the EGPS. This may be owing to exclusion of patients with normal or very low IOPs and also those with very high IOPs. 77 In the Early Manifest Glaucoma Trial, CCT was identified as a significant predictive factor for glaucoma progression in patients with higher baseline IOP but not in those with lower baseline IOP. 78 Racial differences in CCT have been reported in several population studies, with individuals of African ancestry having thinner corneas, on average, than Caucasians, Hispanics or Asians. 79–85 It has been suggested that black race may not be an independent risk factor because black patients tend to have higher IOP, thinner corneas and higher C/D ratios than other patients with OHT and are therefore generally at a higher risk than white patients. 86 In the OHTS and OHTS-EGPS, when either VCD ratio or CCT was included in the multivariate model, race was no longer statistically significant. 42,64 CCT has also been shown to be a highly heritable trait. 87–89 Because thinner CCT has also been found to predict progression of visual loss in patients with OAG, it is plausible that a biological link exists between aspects of the cornea that regulate its thickness and the physical and structural properties of tissues involved in glaucoma pathogenesis. 76

Choice of one eye, two eyes or worse eye

Glaucoma can affect either eye or both eyes but it is often asymmetrical, occurring first in one eye only. The prediction models use the mean of the measurements taken from both eyes for all of the eye-specific predictors. Analysis based on a single eye per individual or on the average of both eyes is convenient but may be suboptimal because of intereye differences. Using a single eye could introduce bias if, for example, data on both eyes are available in some individuals, with one eye randomly selected for analysis, but in other individuals are available for only one eye and so are included by default. 90 With respect to IOP, 21% (105/500) of the EGPS participants randomised to the placebo group and included in the OHTS-EGPS analyses had only one eligible eye. Only the data from the included eye were used. 64 Nevertheless, unpublished data from the EGPS showed that conversion to OAG was almost the same in the group of patients with both eyes included as in those with only one eye included in the trial. 91 When both eyes are available, an alternative approach is to use the ‘worse’ eye, the eye at higher risk of developing OAG; however, this can be problematic when a single variable cannot be used to identify the ‘worse’ eye. Identification of the worse eye in the OHTS-EGPS ‘worse’ eye model was based on IOP, CCT, VCD ratio and PSD. Selection of the ‘worse’ eye was challenging because the eyes of many participants had nearly identical risk estimates at baseline. 65 Using the means of right and left eyes enables the inclusion of relevant data from both eyes of each patient, but potentially at the expense of predictive accuracy. Nevertheless, the performance of the OHTS-EGPS prediction model based on the means of right and left eyes for eye-specific variables was compatible with prediction models that included additional information on baseline differences between eyes. 65

Limitations of the prediction models

The models are limited because they do not include other major risk factors such as race and family history of glaucoma. Race was not associated with the risk of developing OAG in univariate or multivariate analyses of the OHTS or the pooled OHTS-EGPS data. All of the 439 patients from the EGPS included in the pooled data were Caucasians, and the lack of association may be owing to limited power in the OHTS and the combined data set. In the OHTS, data on family history of glaucoma (parent or sibling) were collected by patient recall, without verification, and so may be subject to recall bias. This information was not collected in the EGPS. A positive family history of glaucoma was not significantly associated with an increased risk of OAG in the OHTS analyses. Conversely, the Barbados Eye Studies found more than a twofold increase in risk (relative risk 2.4, 95% CI 1.3 to 4.6) after adjustment for age, gender, IOP, and IOP and blood pressure treatment. 38 Other ocular conditions, such as PEX36,37,41,92 and optic disc haemorrhage,78,91,93 have been found to be predictive for the development of OAG. The OHTS excluded individuals with PEX and pigment dispersion whereas the EGPS included only 19 individuals with these conditions and they were excluded from the pooled analyses.

Another limitation of the models is the use of only baseline measurements of predictors – except for CCT, which was measured later during follow-up – without accounting for fluctuations and changes over time in any of the predictors, especially IOP. IOP fluctuation has been suggested as an independent risk factor for glaucomatous optic nerve damage, but reports are conflicting. 94 In the EGPS, the area under the curve of IOP, a ‘time-dependent’ evaluation of IOP, taking into account the IOP behaviour for each 6-month period, was associated with the development of OAG yet IOP fluctuation during follow-up was not. 91 Medeiros and colleagues95 found that long-term IOP fluctuations were not a significant risk factor for glaucoma development in untreated patients with OHT observed over time, but higher mean IOP levels were consistently associated with increased likelihood of developing glaucoma. Similarly, in the Malmö Ocular Hypertension Study, IOP fluctuations were not an independent risk factor for the incidence of glaucomatous visual field loss in patients with OHT, and higher IOP fluctuation was found in eyes with higher IOP levels. 96

Predictor effects

The HRs obtained when the OHTS-EGPS model was fitted to our cohorts were similar to those obtained in the development cohort, although they were not always statistically significant. Our cohorts were much smaller than the OHTS-EGPS cohort. CCT was the only statistically significant predictor in the Moorfields cohort (HR 2.76, 95% CI 1.44 to 5.26), but was not significant in the Rotterdam cohort (HR 1.31, 95% CI 0.79 to 2.17). This may be because average CCT was lower in the Moorfields cohort than in the other cohorts. Seventeen patients (12%) in the Dunfermline cohort had an average PSD value > 3, the upper limit allowed in the OHTS-EGPS model, and the HR was 1.06 (95% CI 0.96 to 1.16) per 0.2-dB increase in PSD. A large PSD could be a sign of early optic nerve damage and unconfirmed glaucoma but can also reflect an unreliable field, for example because of incorrect patient positioning when undertaking the perimetry. In the observational cohorts patients are likely to have had only one baseline visual field examination, unlike the trial cohorts for which data from at least two reliable baseline visual fields would have been required.

Generalisability

The trial-based cohorts were a highly selected population. The OHTS and EGPS excluded pregnant or nursing women, those with best corrected visual acuity worse than 20/40 in either eye, a history of previous intraocular surgery, life-threatening or debilitating disease, secondary causes of elevated IOP, systemic or ocular conditions capable of causing visual field loss or optic disc abnormalities, patients with diabetes who had evidence of diabetic retinopathy and those on medication. In addition, in the OHTS, patients with pigment dispersion or exfoliation syndrome were excluded. Most of these criteria were applied in the selection of the Rotterdam and Moorfields cohorts (see Chapter 3). The OHTS and DIGS used similar patient selection criteria and the calibration slope (1.086) of the OHTS model in the DIGS cohort was similar to that of the OHTS-EGPS model in the Rotterdam cohort. In both cases, the values were not statistically different from 1, the ideal value. Selection criteria were more flexible in the two observational cohorts, yet both cohorts may not be representative of the general OHT population. Low-risk cases in the Dunfermline hospital-based registry were discharged after two visits (at 1 year), and in the Nottingham community-based registry they were discharged if they had untreated IOP consistently < 21 mmHg as well as normal visual fields and optic discs. Patients in the Nottingham scheme suspected of conversion or with IOP above recommended predefined guidelines (see Chapter 3) were referred back to secondary care. If the secondary-care clinician confirmed conversion, the patient was treated and retained in the secondary-care system. However, if no conversion was judged to have occurred, the patient was returned to the community scheme for annual review. The Dunfermline and the Nottingham data sets represent routinely collected data in clinical practice. A risk prediction tool is potentially most valuable in this context and used as an aid to define criteria for monitoring and treatment guidelines for the management of OHT. Because the discriminatory ability of the model was generally good, the model can be adjusted or recalibrated to correct for model optimism; calibration errors can be eliminated without affecting the discriminatory ability of the model. However, given the limitations of our data, we did not attempt to recalibrate the model to correct for the overestimation of risk.

Variable and outcome definition

Differences in the methods used for data collection or in the definitions of the predictors or the outcome variable can affect model performance. 97 Variability of IOP measurements is a well-known phenomenon, and in both the OHTS and the EGPS between two and nine measurements of IOP were taken in each eye to obtain a stable estimate of baseline IOP. The variability of IOP measurements in the same eye between consecutive visits has been shown to be moderate and is greater than the variability of IOP measurements between right and left eyes at the same visit. 99 IOP of ≥ 22 mmHg and < 35 mmHg on two or more occasions measured within a 2-week period was the entry criterion for the Moorfields cohort. 32 Yet 22% of the Moorfields cohort had an IOP of < 20 mmHg at baseline, a likely reflection of the regression to the mean. One IOP measurement was recorded in each eye, which is typically the case in clinical practice.

In the OHTS and the EGPS, CCT was measured using ultrasonic pachymeters and an average of five measurements per eye was carried out; this was also the case in the Nottingham scheme. Intereye difference in CCT tends to be small but, to ensure data quality and reproducibility, in the OHTS, repeat measurements were made when the intereye difference was ≥ 40 µm. 71 In clinical practice, only one CCT measurement, based on an average of five readings at one visit, is undertaken.

Pattern standard deviation is an objective index of severity of visual field loss. However, the evaluation of the VCD ratio (especially in clinical practice) is subjective and is expected to have a large intra- and interobserver variability.

For risk estimation, the online calculator requires three IOP measurements per eye taken using GAT, three CCT measurements per eye using an ultrasound pachymeter and two PSD measurements per eye using any of the following: Humphrey full threshold 30-2 or 24-2; SITA standard 30-2 or 24-2; or loss variance from Octopus 32-2. Only one measurement of each of these variables was recorded per eye in our cohorts, except for the visual field parameter (PSD), for which at least two baseline visual fields were required.

Events and conversion criteria

With the exception of Dunfermline, there were few incident cases of OAG and the rate of conversion varied between cohorts. Substantial variation in the incidence of OAG was also observed between clinics in the OHTS (3.7–42.9%) and the EGPS (0–25%). The variation was attributed to differences in characteristics of the participants enrolled at the various clinics as well as the small number of participants in some clinics. 64 The rate of conversion from OHT to OAG may also depend on the conversion criteria used. Glaucoma is diagnosed on the basis of a characteristic pattern of damage to the optic nerve with a corresponding and reproducible visual field defect. The conversion criteria used in the observational cohorts were not as rigid and well defined as those used in the trial-based cohorts; conversion was generally based on the development of a visual field defect. This is likely to be a better reflection of clinical practice than the criteria and processes adopted in the trials, but is unlikely to be consistent across the validation data sets as there is no ‘gold’ standard case definition of early glaucoma. The OHTS and EGPS also had a mixed population comprising newly diagnosed and existing patients with OHT attending clinics. Including the latter group could have introduced bias because those with OHT who converted to OAG before recruitment started were excluded from the study, and those who did not convert may be less likely to convert than the whole population with OHT.

Limitations of the model validation

One of the limitations of this study was the inclusion of treated patients in all cohorts, although the only predictor affected by treatment is IOP. This may be important for those who were untreated at the time of recruitment into the study, but is less important for those who were already on treatment when entered into the study cohort, as may be the case with the observational (Dunfermline and Nottingham) cohorts. Although the OHTS-EGPS model was developed to estimate the risk for a patient left untreated for 5 years, the HRs were similar to those of the OHTS model based on treated and untreated patients. This OHTS model was validated in an untreated population and similar HRs were obtained for the predictors. 63

A second limitation was the number of patients and OAG cases in some cohorts. In the UK, patients considered to have a moderate or high risk are often kept in secondary care whereas those deemed to be very low risk are discharged (i.e. no follow-up). This may explain the higher incidence of OAG in the Dunfermline cohort (28/188, 14.9%) than in the other cohorts, and the very low incidence in the Nottingham cohort (5/159, 3.1%) compared with the DIGS cohort (31/126, 24.6%),63 although it exceeds that of the Visual Impairment Project (66/2415, 2.7%). 37 Some patients in the Nottingham cohort received treatment for OHT prior to entry into the scheme, and this could account for the low incidence of OAG, as well as the strict entry criterion (see Chapter 3, Data from observational data sets). Thus, the Dunfermline (secondary care) and Nottingham (community care) cohorts may be unrepresentative of the wider OHT population.

A third limitation was the large proportion of missing values for some predictors in some cohorts. CCT was missing in > 20% of patients in each cohort (100% in the Dunfermline cohort), and was more likely to be missing in those who were enrolled earlier or dropped out/discharged early in the study. The OHTS was the first study to prospectively document that a lower CCT measurement predicts the development of OAG and recommended its measurement in the clinical evaluation of patients with OHT. 42 CCT measurements were not included originally in the protocols of OHTS and EGPS, and 184 patients without CCT measurements were excluded from their analyses. The Moorfields study commenced in 1992, at a time when CCT was not measured routinely, and CCT data were collected during follow-up. It is possible that only those at higher risk of developing glaucoma had a CCT measurement.

A fourth limitation was the enrolment of only Caucasians in the Rotterdam and Dunfermline cohorts, and the fact that race was not documented in the Nottingham cohort (it is likely that there were few non-Caucasians because black patients were considered to be at higher risk of OAG and so were unlikely to be referred to the scheme). Although race was not a predictor in the OHTS-EGPS model, the OHTS population was racially diverse, and approximately 25% of patients enrolled in the study were African Americans. Because there is evidence to suggest that blacks and whites differ with respect to several risk factors for OAG,100 the poor calibration in some cohorts may be partly because of racial differences.

A risk prediction model cannot replace clinical decision-making but is a tool that can facilitate more objective assessment and management of patients. It has been shown that ophthalmologists underestimated the risk of glaucoma in patients with OHT compared with the risk obtained using a risk calculator. 101 Furthermore, the use of a risk calculator changed treatment recommendations made by glaucoma specialists. 102

Contact tonometers

This term refers to tonometers that have direct contact with the cornea.

Goldmann applanation tonometer

The GAT, a contact tonometer, is currently the instrument most commonly used by ophthalmologists to estimate the IOP. It is a slit lamp-mounted device. However, there are some limitations associated with GAT, including the influence of corneal thickness and corneal biomechanical properties, the potential for transmitting infections or causing corneal abrasion and its relative difficulty of use. The GAT calibration assumes that the cornea has a central thickness of between 530 and 560 µm and IOP is likely to be an under- or overestimated when using GAT in people with thinner or thicker corneas, respectively. 103 The operating manual recommends three measurements, but this is often not undertaken in clinical practice.

Dynamic contour tonometer

The dynamic contour tonometer (DCT) is a slit lamp-mounted, contact, digital, non-applanation tonometer and is operated in a fashion similar to GAT. It is commercially available as the PASCAL^® DCT (SMT Swiss Microtechnology, Port, Switzerland). Corneal anaesthesia is required and there is no need for fluorescein. As corneal thickness is an important factor influencing the IOP measurement, the PASCAL DCT minimises the effect of the architecture of the cornea by using a built-in ‘sensor tip’ with solid-state ‘pressure sensor’, which matches corneal curvature. The concave contour surface approximates the corneal shape when the pressures on both sides of it are equal, with minimum distortion, and directs all forces acting within the cornea to the pressure sensor surface, providing an IOP measure. The pressure in the eye is detected 100 times per second and presented as the diastolic IOP (mmHg) in the LCD (liquid crystal display) screen together with the ocular pulse amplitude (OPA) and a quality score, ‘Q’. OPA is the difference between the systolic and diastolic IOP. The quality score of the data and results can be interpreted as follows: ‘Q1’ is optimum; ‘Q2’ and ‘Q3’ are acceptable; and ‘Q4’ and ‘Q5’ are poor and indicate that the measurement should be repeated. The PASCAL DCT self-calibrates at the beginning of every measurement although a performance test can also be performed. 104 A measurement takes about 5 seconds to achieve. Good cooperation is necessary to maintain a steady eye and head position, which are required during the measurement.

Ocuton S

The Ocuton S is a self-measurement tonometer that calculates and displays the IOP value automatically through direct contact of the measuring prism with the cornea. The use of topical anaesthetic is required. 105

The Perkins applanation tonometer

The Perkins applanation tonometer is a hand-held device but otherwise uses the same principles as GAT and requires topical anaesthesia and fluorescein instillation. All hand-held applanation tonometers were included under this heading.

Rebound tonometer

The rebound tonometer (RT) is a simple portable device, commercially available as the Icare^® tonometer (Tiolat, Helsinki, Finland). Although it is a contact tonometer, topical anaesthetic drops are not required and the tonometer has a disposable tip to minimise the risk of cross-infection. The device processes the rebound movement of a rod probe resulting from its interaction with the eye; there is a shorter duration of impact as the IOP increases. The rebound is influenced by corneal thickness, and for this reason measurement of IOP by this tonometer is prone to measurement error as a result of corneal properties. 106,107 Like GAT, it is calibrated for a typical CCT value. Six measurements are recommended to provide accurate results, and the average of six IOP measurements is displayed on the LCD. 108 Regular calibration is required.

TonoPen

The TonoPen^® (Mentor O&O Inc., Santa Barbara, CA, USA; Reichert Inc., Depew, NY, USA) is a hand-held, self-contained portable tonometer that determines IOP by making contact with the cornea (central contact is recommended) through a probe tip, causing applanation/indentation of a small area. The tip contains a transducer that measures the applied force on the cornea. Topical anaesthetic eye drops are used. After four valid readings are obtained, the averaged measurement will appear on the LCD screen. Up to 10 measurements can be performed according to the manufacturer recommendations. It is recommended that calibration is performed daily before instrument use, when indicated by the LCD screen or whenever batteries are replaced.

Non-contact tonometers

This term refers to tonometers that do not have direct contact with the cornea.

Non-contact tonometer

The air-puff tonometer uses a rapid air pulse to applanate (flatten) the cornea, thus working on the same basic principle as the Goldmann tonometer. The force of the air stream increases linearly over milliseconds, progressively flattening a known area of the cornea. The moment of applanation is determined by an optical sensor and the air pulse is then interrupted. The advantages of the non-contact tonometer (NCT) include speed, the fact that there is no need for topical anaesthesia, and thus a low risk of corneal abrasion (hence its use requires minimal training), and, because there is no direct contact with the eye, any infection issues are avoided. IOP measurement by NCT is affected by the corneal thickness. There are several models available in the market; calibration and the number of recommended measurements may vary accordingly. Some models seek to correct the measurement for the CCT. Some patients find the air puff uncomfortable.

Ocular response analyser

The Ocular response analyser^® (ORA) (Reichert Inc., Depew, NY, USA) utilises air-puff technology and an electro-optical system to record two applanation measurements of IOP, one while the cornea is moving inward and the other as the cornea returns. Because of its viscoelastic property, the cornea resists the dynamic air puff, delaying the inward and outward applanation process and resulting in two different pressure values; the average of these two IOP values provides a repeatable, Goldmann-correlated IOP measurement (IOP_G). The difference between these two IOP readings is corneal hysteresis (CH), a new measurement related to corneal tissue properties that is a result of viscous damping in the corneal tissue. 109 The CH measurement provides a basis for two additional new parameters: corneal-compensated intraocular pressure (IOP_CC) and corneal resistance factor. 110,111 The IOP_CC is an IOP measurement that is less affected by the corneal properties. Four good-quality readings per eye are recommended. 112

Transpalpebral tonometer

This type of tonometry includes devices that measure IOP through the eyelid avoiding direct corneal contact. Topical anaesthesia is not required. The Diaton^® tonometer (BiCOM Inc., Long Beach, NY, USA; previously commercialised as TGDc-01, Ryazan State Instrument-Making Enterprise, Ryazan, Russia) is a hand-held, pen-like portable device applying this principle. The pressure phosphene tonometer (PPT; Proview^® Eye Pressure Monitor, Bausch & Lomb Inc., Rochester, NY, USA) has been developed as a self-measurement tonometer. The PPT is a spring compression device calibrated in mmHg that consists of a probe with a flat applicator of the same diameter (3.06 mm) as the area applanated by the GAT. This instrument delivers a phosphene spot when pressure is applied through the closed eyelid in the superior nasal portion of the eye, a self-perceptible visual phenomenon. The threshold pressure for creating a phosphene spot is the estimated IOP. 113

Types of study

Direct comparative studies that assessed the agreement of one or more tonometers with the reference standard tonometer (GAT) in the same group of people were included. Non-English-language studies and conference abstracts were excluded.

Types of participant

Adults aged > 16 years, including those with a diagnosis of OHT or glaucoma, representative of the general population, were included. When the age range was not reported, confirmation from the authors was sought. If no response was received and a mean age and SD or median age and interquartile range (IQR) were provided, a formula was applied (mean – 3SD ≥ 16 or median – IQR/1.35  × 3 ≥ 16) to assess inclusion. This was carried out to prevent exclusion purely on the failure to report the age range of the participants when it was very unlikely that any of the participants were under 16 based on data on the age distribution. Participants with corneal abnormalities were excluded (corneal pathology, including keratoconus, bullous keratopathy or post-corneal grafts).

Types of technology

Types of examiner

Tonometry performed by any type of examiner including optometrists, ophthalmologists, nurses, technicians and patients was included.

Primary outcome

The primary outcome was the agreement [mean difference and limits of agreement (LoA)] between a tonometer and the reference standard.

Secondary outcomes

• Interobserver reliability for two observations taken by different observers with the same tonometer.

• Intraobserver reliability for two observations taken by the same observer with the same tonometer.

• Practicality for ‘doers’ using the technologies.

• Acceptability of the tonometers to users and providers.

• Proportion of participants with a recordable IOP (recordability).

General approach

The primary outcome, agreement, was assessed by calculating summary LoA. 118 Secondary outcomes were tabulated with no quantitative analysis conducted. The 95% LoA interval was calculated for each candidate tonometer from pooled estimates of the mean difference (systematic difference) between a tonometer and the reference standard and of the corresponding variability of agreement (random error). Pooled estimates of mean difference and random error were calculated using the DerSimonian and Laird random-effects method. 119 Measures of agreement variability were based on reported (or derived) SDs of within-participant differences. Imputation of within-participant correlation coefficients to allow calculation of the SD of differences was employed, if required, when a correlation estimate was available from other studies of the same type of tonometer. The imputed value was the arithmetic mean of available correlation estimates when more than one study estimate was available. Sensitivity analyses included a fixed-effect analysis and/or imputation of correlations using the minimum correlation coefficient reported from the studies comparing the same tonometer. Additionally, an approximate 95% prediction interval was calculated for both parameters using the estimated τ-value (±1.96τ) from the random-effects analysis to quantify the impact of between-study heterogeneity on the systematic difference and the random error. The prediction interval provides a range of values that could plausibly be observed if a new study was undertaken, based on the observed between-study heterogeneity. Finally, the proportion of studies within 2 and 3 mmHg of GAT was estimated from the pooled difference and SD using the cumulative distribution function for the standard normal deviates for values within ±2/3 mmHg.

Additional sensitivity analyses

Further sensitivity analyses looked at the impact of excluding studies that used suboptimal methods according to our quality assessment tool (i.e. studies in which at least one of the requirements is clearly not met), excluding studies that reported data clustered within persons (i.e. studies in which some or all of the participants contributed data on more than one eye but in which data for one eye only was unavailable) and using the standardised mean difference (SMD) metric, that is, mean difference divided by pooled SD, to address variation in the number of measurements (systematic difference only). An additional analysis was conducted to correct for the underestimated variation in studies with repeated measurements by using reported estimates of within-participant variation to adjust the results of such studies to reflect the variation if only a single measurement had been taken. 120

Clinical factor analyses

Heterogeneity between the study estimates in the meta-analyses was explored by visual inspection of forest plots and calculation of I² statistics. Possible reasons for heterogeneity were explored through prespecified clinical factor analyses. Where possible, studies were categorised according to CCT, previous corneal refractory surgery, type of examiner and IOP level, with corresponding meta-analyses being conducted. For some studies, data relating to a subset of the main cohort were used for the factor analyses if sufficient data were given for the subset but not the full cohort. For the IOP analyses, studies in which the estimated proportion of the sample with OHT (i.e. > 21 mmHg) was > 33% were compared with studies in which this proportion was estimated to be < 33%. The proportion was estimated using the mean and SD for GAT, assuming a normal distribution (studies in which this was not reported were excluded from the subgroup analysis). A similar approach was taken for the CCT analysis with one of the two subgroups being defined as studies in which 33% of patients in the sample had CCT of < 555 µm (i.e. patients with the highest risk of glaucoma conversion). It was not possible to carry out analyses on the use of disposable tonometer heads as none of the included studies reported such data.

A clinical factor analysis was conducted to compare studies with no previous refractive surgery in their sample with studies that had postoperative subsamples. Another analysis compared studies in which the comparator examiner(s) were known to be solely ophthalmologists with studies in which the comparator examiner(s) were all known to be non-ophthalmologists. For self-tonometers, studies in which patients were the sole examiner type were included as a third category in this subgroup analysis. Because of the observed level of heterogeneity, a further clinical factor analysis investigated the impact of manufacturers in studies in which multiple manufacturers produced the same type of tonometer. Formal comparison between factor subgroups was not conducted because of the high level of heterogeneity in the main analyses. When individual studies reported on the impact of the clinical factors considered in the analyses the results of the individual studies were summarised in narrative form without any statistical analysis.

Characteristics of included and excluded studies

Details of the characteristics of the included studies are provided in Appendix 3. A total of 102 studies reporting 130 comparisons involving 11,582 participants (15,525 eyes) were included. The earliest studies took place in 1988128 and the latest in 2010. 129,130 Fifteen studies took place in the USA,111,124,127,131–142 nine in Italy,143–151 nine in the UK,110,128,129,152–157 seven in Germany,158–164 six in each of Australia,165–170 China (excluding Taiwan)171–176 and Japan,107,130,177–180 five in each of Belgium122,181–184 and Switzerland,104,185–188 four in Spain,189–192 three in each of Saudi Arabia,193–195 India,196–198 Portugal106,199,200 and Greece201–203 and two in Taiwan,125,204 Israel,205,206 the Netherlands,207,208 Sweden209,210 and Turkey. 211,212 One study took place in each of Austria,126 Brazil,213 France,123 Ireland,214 New Zealand,215 Norway216 and Denmark. 217 One additional study was reported as a multicentre study, taking place in Italy and Spain. 218

After contacting authors to clarify the age of study participants, eight authors from nine studies135,140,141,148,189,190,192,201,212 confirmed eligibility for inclusion. One study included only participants who were aged ≥ 50 years but did not give the age range. 166

Seventy-three studies (72%) provided information on gender of the participants, with 3337 women and 2900 men. Fourteen studies provided data on participants’ race. Almost all studies reported the diagnosis; populations varied: healthy volunteers,158 those diagnosed with OHT155 or glaucoma196 or a mixed population. 131

Included studies compared the reference standard tonometer GAT (Haag Streit, Koeniz, Switzerland) with eight different types of tonometer: DCT (PASCAL); RT (Icare); TonoPen; Medtronic Solan, Jacksonville, FL, USA (incorporating Xomed); or Intermedics Intraocular Inc., Pasadena, CA, USA]; Ocuton S; Perkins (Kowa HA-2, Kowa, Japan), NCT (Canon USA Inc., Lake Success, NY, USA; Keeler Ltd., Windsor, UK; NIDEK Co. Ltd., Gamagori, Japan; Reichert Ophthalmic Instruments, Buffalo, NY, USA; or Topcon Corporation, Tokyo, Japan); ORA; transpalpebral tonometer, including the PPT (Proview Eye Pressure Monitor) and the TGDc-01, also known as the Diaton tonometer.

All but three104,179,187 of the included studies reported sufficient data (or data were provided by the authors when contacted) to be included in the agreement meta-analysis. A total of 27, 20, 17 and 37 studies provided data on recordability, acceptability, practicality and reliability, respectively.

The 87 reports that were excluded at the full-text assessment stage as they failed to meet one or more of the inclusion criteria in terms of study participants, study design, candidate tonometers and reference standard are listed in Appendix 2. The tonometers that were not included because they were not commercially available and/or were not suitable for clinical practice were the applanation resonance tonometer,219 ocular blood flow instrument,115 Schiotz tonometer,220 SmartLens (Ophthalmic Development Company, Zurich, Switzerland) tonometer221 and pneumotonometer. 222

Agreement of tonometers with Goldmann applanation tonometer

In total, 99 studies (125 paired comparisons) provided enough data on agreement to be included in the meta-analysis. Comparison across tonometers is difficult given the indirect nature of the analysis. A summary of the main analyses for all candidate tonometers is provided in Tables 9 and 10. Full results of all of the meta-analyses for the candidate tonometers are given in Appendix 4. The percentage of results that would be within 2 mmHg based on the main analysis mean difference and random error (assuming a normal distribution) is also presented. Based on the analysed studies the expected difference did appear to vary across tonometers, with NCT and Tonopen having the smallest estimated difference and Ocuton S the largest. There was substantial uncertainty for most of the tonometers. In terms of the estimated random error, results varied, with Perkins having marginally (over NCT) the smallest expected random error and Ocuton S the largest. For all tonometers, the 95% LoA stretched from at least 3 mmHg less to 3 units higher with Ocuton S and transpalpebral with the widest intervals. With regard to the percentage of measurements that were within 2 mmHg of the GAT value, for most tonometers this value was approximately 50%; Ocuton S at 33% had the lowest value and NCT and Perkins with 66% and 59%, respectively, had the highest. Corresponding values for the percentage within 3 mmHg of the GAT value were 48%, 85% and 79%, respectively, for Ocuton S, NCT and Perkins.

Prediction intervals

Substantial heterogeneity was observed in estimates between studies for most tonometers. The 95% prediction intervals for the mean difference and random error are shown in Table 10. The values illustrate the impact of the heterogeneity between individual study estimates of the mean difference: –4.0 to 9.4 mmHg for Ocuton S, whereas for NCT the range of values was only from –1.4 to 1.9 mmHg. For most tonometers except NCT and Perkins, a difference of > 2mmHg bar was observed. Similarly, the random error 95% prediction intervals illustrate the difference in the level of variability between studies.

Dynamic contour tonometer

Quality assessment

Thirty-four studies representing 3726 participants (4933 eyes) compared DCT with GAT. Figure 24 summarises the quality assessment for these studies. All 34 studies (100%) specified the selection criteria. In nine studies (26%), cases were selected consecutively. A total of 21 studies (62%) reported individual measures taken within 1 hour. In 24 studies (71%) the same clinical data were available for interpretation as would be in clinical practice. In total, 12 studies (35%) reported whether the examiner(s) were masked to the results. In 13 studies (38%), the tonometers were reported as calibrated. Almost all of the studies (31; 91%) included all participants approached in the analysis or stated a reason why not. Only three studies144,148,218 met all of the quality criteria.

FIGURE 24.

Dynamic contour tonometer: summary of quality assessment.

Agreement between dynamic contour tonometer and Goldmann applanation tonometer

Thirty-two studies provided sufficient data to include in the meta-analysis. The full results of the agreement analyses of DCT and GAT are given in Appendix 4. Under the main analysis, the pooled mean difference was 1.8 mmHg (95% CI 1.4 mmHg to 2.2 mmHg) with a corresponding random error of 2.4 mmHg (95% CI 2.1 mmHg to 2.6 mmHg). For both analyses there was evidence of a large amount of heterogeneity, with very large I² values (97% and 95%, respectively), and this can be seen in Figures 25 and 26, in which the forest plots are presented. Based on the main analysis the expected mean difference is 1.8 mmHg (95% LoA –2.9 mmHg to 6.5 mmHg).

FIGURE 25.

Meta-analysis of mean difference between DCT and GAT (main analysis – random effects).

FIGURE 26.

Meta-analysis of random error between DCT and GAT (main analysis – random effects).

Non-contact tonometer

Twenty-eight studies enrolling 2222 participants (2868 eyes) compared NCT with GAT. Figure 27 summarises the quality assessment for these studies.

FIGURE 27.

Non-contact tonometer: summary of quality assessment.

Agreement between non-contact tonometer and Goldmann applanation tonometer

Twenty-six studies provide sufficient data to be included in the meta-analysis. The full results of the agreement analyses of NCT and GAT are given in Appendix 4. Under the main analysis, the pooled mean difference was 0.2 mmHg (95% CI –0.1 mmHg to 0.6 mmHg) with a corresponding random error of 2.1 mmHg (95% CI 1.8 mmHg to 2.3 mmHg). For both analyses, there was evidence of a large amount of heterogeneity with very large I² values (95% for both), and this can be seen in Figures 28 and 29, in which the forest plots are presented. Based on the main analysis, the expected mean difference is 0.2 mmHg (95% LoA –3.8 mmHg to 4.3 mmHg).

FIGURE 28.

Meta-analysis of mean difference between NCT and GAT (main analysis – random effects).

FIGURE 29.

Meta-analysis of random error between NCT and GAT (main analysis – random effects).

Ocuton S

Three studies enrolling 173 participants (258 eyes) compared Ocuton S with GAT. Figure 30 summarises the quality assessment for these studies.

FIGURE 30.

Ocuton S: summary of quality assessment.

Agreement between Ocuton S and Goldmann applanation tonometer

All three studies126,146,170 provided sufficient data to be included in the meta-analysis. The full results of the agreement analyses of Ocuton S and GAT are given in Appendix 4. Under the main analysis the pooled mean difference was 2.7 mmHg (95% CI –1.2 mmHg to 6.6 mmHg) with a corresponding random error of 3.5 mmHg (95% CI 2.4 mmHg to 4.6 mmHg). For both analyses, there was evidence of a large amount of heterogeneity with very large I² values (96% and 88%, respectively), and this can be seen in Figures 31 and 32, in which the forest plots are presented. Based on the main analysis the expected mean difference is 2.7 mmHg (95% LoA –4.1 mmHg to 9.6 mmHg).

FIGURE 31.

Meta-analysis of mean difference between Ocuton S and GAT (main analysis – random effects).

FIGURE 32.

Meta-analysis of random error between Ocuton S and GAT (main analysis – random effects).

Ocular response analyser

Quality assessment

Twelve studies enrolling 867 participants (1147 eyes) compared ORA with GAT. Figure 33 summarises the quality assessment for these studies.

FIGURE 33.

Ocular response analyser: summary of quality assessment.

All 12 studies (100%) specified the selection criteria and included all participants approached in the analysis or stated a reason why not. None of the studies selected cases consecutively; it was unclear in most of the cases (92%). Half of the studies (50%) reported details on individual measures were taken within 1 hour. In nine studies (75%) the same clinical data were available for interpretation as would be in clinical practice. Five studies (42%) reported whether the examiner(s) were masked to the results. Only two studies (17%) reported calibration of the tonometers. No studies met all of the specified criteria.

Agreement between ocular response analyser and Goldmann applanation tonometer

Twelve studies110,111,129,130,135,136,142,174,183,184,212,214 provide sufficient data to include in the meta-analysis. The full results of the agreement analyses of ORA and GAT are given in Appendix 4. Under the main analysis, the pooled mean difference was 1.5 mmHg (95% CI 0.9 mmHg to 2.2 mmHg), with a corresponding random error of 2.8 mmHg (95% CI 2.5 mmHg to 3.1 mmHg). For both analyses, there was evidence of a large amount of heterogeneity with very large I² values (93% and 89%), and this can be seen in Figures 34 and 35, in which the forest plots are presented. Based on the main analysis the expected mean difference is 1.5 mmHg (95% LoA –3.9 mmHg to 7.0 mmHg).

FIGURE 34.

Meta-analysis of mean difference between ORA and GAT (main analysis – random effects).

FIGURE 35.

Meta-analysis of random error between ORA and GAT (main analysis – random effects).

Perkins

Four studies132,166,191,200 enrolling 433 participants (506 eyes) compared Perkins with GAT. Figure 36 summarises the quality assessment for these studies.

FIGURE 36.

Perkins tonometer: summary of quality assessment.

Agreement between Perkins and Goldmann applanation tonometer

Four132,166,191,200 studies provide sufficient data to include in the meta-analysis. The full results of the agreement analyses of Perkins and GAT are given in Appendix 4. Under the main analysis, the pooled mean difference was –1.2 mmHg (95% CI –2.8 mmHg to 0.4 mmHg) with a corresponding random error of 2.1 mmHg (95% CI 1.3 mmHg to 2.8 mmHg). For both analyses, there was evidence of a large amount of heterogeneity, with large I² values (99% and 97%), and this can be seen in Figures 37 and 38, in which the forest plots are presented. This finding is driven by one study166 that had both a much larger mean difference and a much larger random error than the other three studies (see Appendices 3 and 4 for more information on this study). Based on the main analysis the expected mean difference is –1.2 mmHg (95% LoA –5.2 mmHg to 2.8 mmHg).

FIGURE 37.

Meta-analysis of mean difference between Perkins and GAT (main analysis – random effects).

FIGURE 38.

Meta-analysis of random error between Perkins and GAT (main analysis – random effects).

Rebound tomometer

Fourteen studies106,107,145,154,160,165,167,184,189,190,207,209–211 representing 1239 participants (1792 eyes) compared RT with GAT. Figure 39 summarises the quality assessment for these studies.

FIGURE 39.

Rebound tonometer: summary of quality assessment.

Agreement between rebound tonometer and Goldmann applanation tonometer

Fourteen studies106,107,145,154,160,165,167,184,189,190,207,209–211 provided sufficient data to be included in the meta-analysis. The full results of the agreement analyses of RT and GAT are given in Appendix 4. Under the main analysis, the pooled mean difference was 0.9 mmHg (95% CI 0.4 mmHg to 1.4 mmHg), with a corresponding random error of 2.6 mmHg (95% CI 2.1 mmHg to 3.2 mmHg). For both analyses, there was evidence of a large amount of heterogeneity, with very large I² values (94% and 98%), and this can be seen in Figures 40 and 41, in which the forest plots are presented. Based on the main analysis the expected mean difference is 0.9 mmHg (95% LoA –4.3 mmHg to 6.1 mmHg).

FIGURE 40.

Meta-analysis of mean difference between RT and GAT (main analysis – random effects).

FIGURE 41.

Meta-analysis of random error between RT and GAT (main analysis – random effects).

TonoPen

Fifteen studies75,107,138,140,144,153,157,166,169,188,191,197,206,207,216 enrolling 1413 participants (1950 eyes) compared TonoPen with GAT. Figure 42 summarises the quality assessment for these studies.

FIGURE 42.

TonoPen: summary of quality assessment.

Meta-analysis of agreement between TonoPen and Goldmann applanation tonometer

Fourteen studies76,138,140,144,153,157,166,169,188,191,197,206,207,216 provided sufficient data to be included in the meta-analysis. The full results of the agreement analyses of TonoPen and GAT are given in Appendix 4. Under the main analysis, the pooled mean difference was –0.2 mmHg (95% CI –1.0 mmHg to 0.5 mmHg), with a corresponding random error of 3.1 mmHg (95% CI 2.5 mmHg to 3.7 mmHg). For both analyses, there was evidence of a large amount of heterogeneity, with very large I² values (97% and 98%), and this can be seen in Figures 43 and 44, in which the forest plots are presented. One study166 had a much larger mean difference than the other studies, as was the case for the Perkins mean difference for this study. Based on the main analysis the expected mean difference is –0.2 mmHg (95% LoA –6.2 mmHg to 5.8 mmHg).

FIGURE 43.

Meta-analysis of mean difference between TonoPen and GAT (main analysis – random effects).

FIGURE 44.

Meta-analysis of random error between TonoPen and GAT (main analysis – random effects).

Transpalpebral tonometer

Twenty studies124,125,127,133,136–138,158,162,164,168,172,176–178,191,198,205,207,208 enrolling 1509 participants (2091 eyes) compared transpalpebral tonometer with GAT.

Quality assessment

Figure 45 summarises the quality assessment for these studies. A total of 19 studies124,125,127,133,136–138,158,164,168,172,176–178,191,198,205,207,208 (95%) specified the selection criteria and in six studies124,125,172,176,178,205 (30%) cases were selected consecutively. In total, 10 studies124,127,137,138,168,176,178,191,198,208 (50%) reported individual measures taken within 1 hour. In 12 studies (60%) the same clinical data were available for interpretation as would be available in clinical practice124,127,133,137,138,158,162,168,172,177,205,207 and it was reported that the examiner(s) were masked to the results. 124,125,158,162,168,172,176–178,205,207,208 In seven studies125,127,172,177,178,205,208 (35%) the tonometers used were calibrated. Almost all studies124,127,133,136–138,158,164,168,172,176–178,191,198,205,207,208 (18, 90%) included all participants approached in the analysis or stated a reason why not. No studies met all of the specified criteria.

FIGURE 45.

Transpalpebral tonometer: summary of quality assessment.

Meta-analysis of agreement between transpalpebral tonometers and Goldmann applanation tonometer

Twenty studies124,125,127,133,136–138,158,162,164,168,172,176–178,191,198,205,207,208 provide sufficient data to be included in the meta-analysis. The full results of the agreement analyses of transpalpebral and GAT are given in Appendix 4. Under the main analysis the pooled mean difference was –0.5 mmHg (95% CI –1.3 mmHg to 0.3 mmHg) with a corresponding random error of 3.3 mmHg (95% CI 2.8 mmHg to 3.7 mmHg). For both analyses, there was evidence of a large amount of heterogeneity, with very large I² values (98% and 97%), and this can be seen in Figures 46 and 47, in which the forest plots are presented. Based on the main analysis, the expected mean difference is –0.5 mmHg (95% LoA –6.9 mmHg to 5.9 mmHg).

FIGURE 46.

Meta-analysis of mean difference between transpalpebral tonometer and GAT (main analysis – random effects).

FIGURE 47.

Meta-analysis of random error between transpalpebral tonometer and GAT (main analysis – random effects).

Moorfields data set

The Moorfields cohort included data from participants randomised between 1992 and 1996. The following variables were available for each participant and used in this analysis:

1. Baseline characteristics: participant identifier (ID), observer (clinician measuring the IOP), age, gender, date after randomisation, eye (under study), IOP, MD, PSD and medication.

2. Conversion to glaucoma: eye and visit date of conversion to glaucoma.

3. Repeated measures at each 4-monthly visit (1–14): visit date (recoded as days from baseline visit), IOP, MD, PSD and medication.

Rotterdam data set

The Rotterdam cohort included data on participants randomised between 1997 and 2001. The following variables were available for each participant and used in this analysis:

1. Baseline characteristics: participant ID, date after randomisation, IOP and medication.

2. Conversion to glaucoma: repeated measures at each 6-monthly visit (0–16).

3. Visit number, IOP and medication: as the number of days between visits (or dates) was not recorded, this was estimated by determining the total number of days from baseline to final follow-up for each participant divided by the total number of visits. This defined the average days between visits per participant, which was used for the analysis.

Both data sets included data on both eyes. In the Moorfields data set, one eye per patient was selected for analysis; this was based on the eye that was chosen in the original study (indicated in the data set). In the Rotterdam data set, one eye per participant was selected randomly for analysis. In both data sets, only IOP measurements obtained with GAT were included in the analysis. The analyses use information from the placebo group only, which was formed by participants who had been allocated to receive either placebo or no medication within the trial period. Participants in the placebo group who were put on treatment during the study period (crossovers) were treated as censored and contributed information only during the non-medicated period with their data treated as missing for the rest of the study (see Imputation assumptions).

Direct method

The average change of the group can be estimated from the group average at each time point. Therefore, a direct method for estimating the long-term variability uses the variance of the differences between the baseline value and each subsequent time point i. Taking IOP as an example, this can be calculated as:

The time i corresponds to the different follow-up times at which data were recorded – 4-monthly for the Moorfields RCT and 6-monthly for the Rotterdam RCT – with a maximum of 10 years’ follow-up for either trial. The baseline IOP was taken as the first visit measurement for the Moorfields data set and the second visit for the Rotterdam data set, to reduce the potential noise created by the initial response to therapy as well as any potential ‘regression to the mean’ effect. The variability of change is used as a proxy for variation from the true mean at different time points. Therefore, using linear extrapolation on the variation of change it is possible to estimate short-term variation in IOP (noise), which would be proportional to the variance at time point 0. 228 The long-term variability at different time points (signal) can be estimated by subtracting the short-term variability from the variability of the change.

The direct estimation method was used to explore the variability of IOP and visual field variables, specifically MD and PSD.

Linear random-effects model

In addition, the three variability components of IOP were estimated using a linear random-effects model. The main assumption of this model is that the IOP (or some transformation of IOP) for each individual can be adequately described as having a linear change (increase/decrease) over time; importantly, it allows this rate of change to vary between individuals. This model can be written as:

for each individual, where α is the baseline level (in the transformed IOP scale) and β is the change in IOP for every day that passes from the first measurement (days₀ = number of days since first visit). The observed IOP will have a measurement error (ε) and this model can be extended to the whole population using a linear random-effects model on all relevant parameters so that for an individual j at a given time i:

where ∼N(x,y) refers to a normal distribution with a mean x and a variance y. From this model, the average true long-term change is equal to the mean of β = b; the short-term variability or ‘noise’ is equal to the variance of the ε = σ²_w, whereas the long-term variability is equal to the variance of the βs = σ²_β. The two main assumptions in this model are the linearity of the association and the normality in the distributions of α, β and ε. To satisfy these assumptions, the raw IOP data were transformed, using a Box–Cox or power transformation to normalise the complete set of IOP measurements, at each time point and over all the assessment points combined.

Descriptive statistics were used to identify both the degree of normality of the IOP data as well as the presence of possible outliers. Shapiro–Wilk normality tests were carried out to further test the assumption of normality at each assessment point. Allowing for multiple tests adjustment, a 10% significance level was chosen, below which normality was rejected.

Variability of estimates to baseline intraocular pressure and other covariates

Bland–Altman plots were used to investigate the relationship between variability and mean IOP values. First, the mean versus SD was plotted for each subject to check for differences in variance based on an overall mean (as the measures for IOP seemed to be very stable over time for the majority). Second, the mean of the baseline and first visit measurements was plotted against the difference between baseline and first visit to check for differences in variance based on initial IOP measure.

Estimates of the variability components were obtained after stratification by IOP baseline level (threshold at 26 mmHg) to explore potential differences. Relevant covariates were included as part of the linear random-effects model to determine potential association with IOP.

External validation

Having estimated the parameters using the Moorfields data set, the Rotterdam data set was used for external validation of these models. For the Rotterdam data, the eyes were selected for inclusion randomly. For values of IOP entered as a range, the mid-point in the range was used. After running descriptive statistics for IOP, the extreme values found were replaced with the maximum IOP observation for the particular patient, excluding the outlier. Up to 16 visits within the trial period were considered for this analysis. The same imputation methods and Box–Cox transformation as for the Moorfields data were used. The validation process consisted of predicting the transformed Rotterdam IOP values using each estimated Moorfields model. The proportion of times that the observed value fell in the estimation interval given by:

was calculated, where PTT is the predicted transformed Rotterdam IOP, TT represents the transformed IOP and σ²_TT is equivalent to σ²_α in Equation 10. High values indicate greater accuracy of the corresponding model.

Moorfields data set

The placebo group in the Moorfields data set includes data on 153 participants (61% male) with a mean (SD) age of 58.8 (10.9) years and an age range of 29–90 years. The majority were of white ethnicity (76%). The mean (SD) baseline IOP was 24.5 (3.5) mmHg with a range of 16–36 mmHg.

A total of 14 approximately 4-monthly visits in the period 1992–8 were included in this analysis. As the visit number (i.e. follow-up) increased, the number of participants with data steadily decreased. This ranged from 4.6% missing values at visit 2 to 63.4% missing values at visit 14. The mean IOP measurement over time for the placebo group is shown in Figure 48. A slight reduction can be observed over time, contrary to initial expectations.

FIGURE 48.

Mean IOP values over follow-up period (Moorfields data – placebo group only).

Figure 49 shows the distribution of IOP at each of the 14 visits. The normality test results are also shown and normality was rejected at four time points (visits 4, 5, 6 and 9). A single outlier was identified (IOP = 48 mmHg, visit 4). This outlier was not included in further analyses.

FIGURE 49.

Intraocular pressure measurements over time (Moorfields placebo group – original IOP data).

Figure 50 shows the IOP measurements after the data were transformed (Box–Cox transformation with λ = 0.26) and corresponding normality test results. Normality was rejected for only one visit (visit 2, p = 0.002) suggesting that the transformation resulted in data that were more suitable for methods (such as the linear random-effects model) that assumed normality.

FIGURE 50.

Intraocular pressure measurements over time (Moorfields placebo group – transformed incomplete IOP data).

Handling of missing data

Intraocular pressure

The change in IOP from baseline at each visit (visits 2–14) was calculated for the raw and imputed data sets. Graphs of the mean and variance of the change in IOP from baseline as a function of time are shown in Appendix 8. Bland–Altman plots (Figure 51) show that the variability decreases with high mean IOP. This could be owing to censoring of those with higher IOP.

FIGURE 51.

Bland–Altman plots using transformed Moorfields IOP data: (a) mean vs SD of transformed IOP per subject; (b) mean vs difference of transformed IOP at baseline and visit 1.

A total of 50 out of the 153 patients in the data set had crossed over from placebo to treatment, representing 33% of this group. Once a patient crossed over to treatment, his or her measured IOP was not included in the analysis. In these patients, the data points after crossover were replaced using LVCF. A single case presented several crossovers during the trial period, starting as a placebo case. This patient was included in the analysis and a sensitivity analysis run later to investigate the effect of the presence of this patient on the estimated parameters, with their inclusion having no significant impact. Stratification of the placebo group by baseline IOP resulted in 59 (39%) patients with IOP higher than the risk value (26 mmHg), of whom 30 (51%) were included among the crossover group mentioned above.

Rotterdam data set

The Rotterdam placebo data set includes data on 132 participants (51% male). The age range was 24–81 years, with a mean (SD) of 56.1 (11.0) years. Only those of white ethnicity were eligible to participate in the RCT. The mean (SD) baseline IOP was 25.7 (2.5) mmHg with a range of 18–35 mmHg.

A total of 23 visits were available; however, the second visit was taken as the baseline visit to account for regression to the mean, leaving only 21 visits for the initial analysis. The percentage of missing data at each visit, illustrated in Appendix 9, increased over time, as expected, with a peak at visit 10 of 38%, and then a slow increase from 25% to 36% between visits 10 and 15, to be followed by a sharp increase to a final 85%. We used data up to visit 15.

The IOP values in the placebo group from baseline (equivalent to visit 2 in the original data set) show a steady increase up to visit 7 and then a decline in mean IOP (Figure 52). Bland–Altman plots showed no association between variability and mean IOP level (Figure 53).

FIGURE 52.

Mean IOP values over follow-up period (Rotterdam data – placebo group only).

FIGURE 53.

Bland–Altman plots using transformed Rotterdam IOP data: (a) mean vs SD of transformed IOP per subject; (b) mean vs difference of transformed IOP at baseline and visit 1.

Signal-to-noise ratio

Table 12 presents the signal-to-noise ratio (σ_β²t²/σ_w²) at 3 and 6 months, and 1, 2 and 3 years obtained under the models for IOP in the placebo group. According to these ratios, the LVCF linear random-effects model estimates a greater signal than noise after 3 years. To explore the impact of having multiple measurements of IOP at a single visit and using the average as an estimate for the true IOP, we generated the equivalent signal-to-noise ratios based on the average IOP of two and three observations (this assumes conditional independence between the IOP measures of an individual in one visit). These values are presented in Table 12 and show that, for the average of two measures, the signal-to-noise ratio would reach a value of 1 within 2 years, while for the average of three measures the signal-to-noise ratio reaches a value of 1 between 1 and 2 years from baseline.

True- and false-positives

We explored the impact of the noise in the observed IOP measurement given a defined threshold of IOP (26 mmHg) for different baseline levels (21, 25 and 28 mmHg). The calculation of true and false-positives indicates that measuring IOP in shorter than 1-year periods results in identifying only noise. However, for the purposes of clinical management it is more relevant to explore the potential changes as defined by the model, that is, the expected change in IOP (both true and observed) for an average individual in the population and for one having an extreme change in IOP. Table 13 presents these IOP values for different baseline levels.

Based on the results obtained from the LVCF linear random-effects model (see Table 11), we can estimate the number of observed IOP measures that differ from their true IOP by ≥ 2 mmHg and explore how different measurement regimes, using the mean of two, three, four or five measures (assuming independence), improve on these percentages. These estimates are presented in Table 14.

For comparison, linear random-effects models were fitted over the treatment group using the transformed not imputed data. Model 1T in Table 15 was obtained by regression of the transformed IOP on time. In this case, the distribution of the random slope was similar to the one obtained with model 1 in the placebo group. The mean of the baseline transformed IOP was slightly lower but its variance was much larger than the one estimated by any of the three models in the placebo group. The same was true for the noise estimate.

Internal validation – Moorfields data

The purpose of these models is to predict the optimal monitoring frequency, particularly for inclusion into the economic model. For this reason it is necessary to explore the accuracy of these models on the prediction of change at a population level. To do this, we simulated the percentage of patients with a given predicted change from baseline at different time points using data from the LVCF model and compared these proportions against the observed change from baseline IOP for the Moorfields data set (placebo group only). These are presented in Tables 16 and 17 and show that the model underestimates large changes in IOP (≥ 5mmHg) for all time points.

The histograms in Figure 54 present the empirical (histogram) and estimated (dotted line) distributions of the residuals and slopes obtained from the (LVCF) transformed IOP Moorfields data. They show that the normality assumption is adequate for the residuals, but the corresponding slopes-estimated model underestimates the proportion of extreme values observed. Appendix 9 gives a table comparing the cumulative distributions from the model (normal distribution) and the empirical distribution for the slopes.

FIGURE 54.

Histograms presenting the distributions of residuals and slopes from the raw data and simulated data (Moorfields data): (a) histogram of the mixed-model residuals; (b) histogram of slopes from LVCF (transformed) IOP data.

Converting these estimates to the original IOP scale (Figure 55a) shows that the noise term for a single measure is likely to be in the region of 0.6–1.4 mmHg (normal range). Therefore, assuming two measurements (e.g. baseline and follow-up) with additive errors and taking the upper level of the normal range for the error (1.4 mmHg), a difference of 3 mmHg between two readings can be due only to noise (no real change). Figure 55b presents a histogram of a series of simulated differences between two values with no real change (difference owing to noise only). The probability of having a difference of ≤ 3 mmHg is 73% (dotted lines in figure).

FIGURE 55.

Histograms presenting (a) the distribution of the noise in the IOP scale and (b) the difference between two measurements attributed to noise only (no real change).

External validation – Rotterdam data

The distribution of the placebo IOP data from the Rotterdam data set after applying the same Box–Cox transformation as for the Moorfields data set (λ = 0.26 for all visits) resulted in a normal distribution in 12 out of the 16 visits. Normality was rejected at visits 1 (p = 0.002), 2 (p = 0.003), 4 (p = 0.004) and 13 (p = 0.004).

For each model (i.e. LVCF, reduced median imputation, no imputed data), 98% of the time the observed TT_it in the Rotterdam file fell within the 95% CI defined by Equation 11. This would suggest that, in terms of predictive ability, the model would accurately predict the observed variability in the Rotterdam data.

As with the internal validation with the Moorfields data set, we calculated the proportions of observed change from baseline IOP for selected differences. These are presented in Table 18. Reassuringly, these percentages appear to be similar to those obtained in the observed Moorfields data set and suggest that the variance is similar across populations. As for the Moorfields data, the model consistently underestimates the proportion of individuals with large observed changes.

Impact of baseline intraocular pressure

We explored the variability of estimates according to different IOP baseline levels. Two separate models were obtained based on IOP levels at baseline. A low IOP level group was defined as those individuals with observed baseline IOP < 26 mmHg whereas a high IOP level was defined as ≥ 26 mmHg. All estimates were obtained using a linear random-effects model and LVCF as the imputation method. Table 19 shows that the estimates are very similar for both groups with a smaller noise estimate in the high IOP level. The impact of this is that the signal-to-noise ratio reaches a value of 1 earlier in the high IOP level group (Table 20).

As with the model for the full data, we explored whether the assumption of normality on the errors and the slopes was appropriate. The histograms in Figure 56 present the empirical and estimated distributions of the slopes for the two strata (IOP < 26 mmHg and IOP ≥ 26 mmHg) obtained from the (LVCF) transformed IOP Moorfields data. They show that the corresponding slopes estimated by the model underestimate the proportion of extreme values observed (heavier tails), as was observed in the full model.

FIGURE 56.

Histograms for the empirical and estimated distributions of slopes based on the Moorfields’ (LVCF) transformed IOP data for two strata: (a) IOP_baseline ≥ 26 mmHg; (b) IOP_baseline < 26 mmHg.

The histograms in Figure 57 present the empirical and estimated distributions of the residuals for the two strata (IOP < 26 mmHg and IOP ≥ 26 mmHg) obtained from the (LVCF) transformed IOP Moorfields data. They show that the assumption of normality appears adequate.

FIGURE 57.

Histograms for the empirical and estimated distributions of residuals based on the Moorfields’ (LVCF) transformed IOP data for two strata: (a) IOP_baseline ≥ 26 mmHg; (b) IOP_baseline < 26 mmHg.

Impact of important covariates

The covariates age and observer are regarded clinically as potential modifiers of IOP measurement. The effect of these clinically significant covariates was analysed by fitting a separate model for each covariate. None of these covariates was statistically significant for the models fitted over the placebo group. In the treatment group, the coefficient for age was not significantly different from 0, whereas investigator had a significant effect (Table 21). Also, this model returned a smaller estimated variability [i.e. Var(TT_i), Var(β) and σ_w² decreased].

Advisory panel

An advisory panel was convened to identify potential ‘attributes’ (characteristics influencing preferences for alternative monitoring services) and thus inform the framework (topic guide) for the planned focus group discussion with service users. Participants for the advisory panel were purposively selected to provide a broad UK perspective on managing OHT, but were not directly involved in the project. The panel included two service users [people under surveillance for OHT (SOH), Grampian, Scotland], one optometrist (academic optometrist, England), one non-clinical health service manager (ophthalmology service manager, secondary care, England), one specialist glaucoma nurse (secondary care, Scotland), one ophthalmologist (glaucoma specialist, England) and one community optometrist (Scotland). Six members of the local study team attended the advisory panel meeting (chief investigator, ophthalmologist, three economists and study research fellow). The meeting was recorded and transcribed. Minutes of the meeting with key points of agreement were circulated to participants following the meeting.

Discussion was driven by the participants on the desired elements of a monitoring service. Key themes (attributes) are summarised in Table 23.

Focus group

The findings from the advisory panel (see Table 23) were used to develop the topic guide for the focus group discussion with users of a service, the Grampian Glaucoma Referral and Monitoring Scheme Service. This service started in June 2004 and was developed to reduce unnecessary referrals to a hospital glaucoma clinic and to develop a community monitoring scheme for people at risk of developing glaucoma but not requiring treatment. Potential participants were identified from this scheme. Both men and women with OHT (who may or may not be undergoing treatment) or suspected of having OHT, of different ages and geographical locations within the Grampian area, were purposely sampled.

Participants were approached by post with an invitation letter from the consultant ophthalmologist and an information leaflet explaining the purpose of the study. In total, 93 people were invited and 11 agreed to participate in the study. Of these 11, six were able to attend on the suggested date for a focus group. In the event, five attended on the specified date, in September 2009. Summary statistics for the participants are presented in Table 24.

Pictorial cards were used to represent the most common tests for monitoring OHT (measuring IOP, looking at the back of the eye, measuring the field of vision and photographing the back of the eye) and to describe the different stages of progression of OHT and the likelihood of progression. This was based on a 5-year risk of conversion from OHT to glaucoma of 10%64 and a 0.65 treatment effect237 (i.e. treatment reduced the risk of conversion to 65% of the risk faced by those who were not treated) for those opportunistically detected with OHT.

Before the focus group, the topic guide and the stimulus material (pictorial cards) were piloted among six members of the Health Services Research Unit and Health Economics Research Unit at the University of Aberdeen. In general, the content of the topic guide was easily understood. Feedback regarding the content and format of the pictorial cards was incorporated to improve their value in promoting discussion among the group.

The discussion in the focus group was facilitated by the main researcher (ABP, who also acted as moderator), an economist (RH) and an ophthalmologist (AA) to explore participants’ views on how a monitoring regime could be organised. Following initial introduction, the moderator (ABP) explored participants’ experiences of how they were monitored and used the pictorial cards to introduce information about different characteristics that a monitoring service might have. Participants were encouraged to discuss their views and express their preferences about different characteristics of a monitoring programme, as well as discussing possible trading between attributes (i.e. how much of one attribute would they be willing to give up to get more of another). Throughout the discussion, the facilitators answered questions on current monitoring services in the UK, risk of progression of the disease and cost, as required.

The discussion was audiotaped and subsequently transcribed and analysed independently by two researchers (ABP, RH), using a modified framework approach. 238 After initial familiarisation with the transcripts, the data were grouped according to common themes that reflected the main research questions and key issues which transpired from the discussion. These themes emerging are described below.

Summary of attributes

The final attributes for the DCE are shown in Table 25. They include four attributes of health outcome from the Glaucoma Utility Index and two attributes of the monitoring process (communication and understanding; location). When a price proxy (cost attribute) is included in the DCE, WTP, a monetary measure of benefit, can be calculated for changes in individual attributes as well as for different configurations of a monitoring programme. 235,239,240 This is useful from a policy perspective, allowing estimation of the maximum amount of money individuals are willing to pay to participate in different monitoring programmes. We therefore also included a price proxy.

The next step was to define levels for each of these seven attributes.

Attributes 1, 2 and 3: risk of developing glaucoma, severe glaucoma and visual impairment

Based on the preliminary advisory panel and focus group discussions, a description of glaucoma, with a definition of each disease stage, was developed. The risk of developing glaucoma was also illustrated. This is shown in Figure 59. The levels for the risk of developing glaucoma were determined using an existing economic model, tailored for individuals with OHT241 progressing to the different stages of glaucoma (glaucoma, severe glaucoma and visual impairment). All individuals under a monitoring regime were assumed to receive treatment. Those under no monitoring may be opportunistically diagnosed and treated. A 5-year risk of conversion to glaucoma was used for untreated individuals with OHT. 12 We assumed a treatment effect on risk of conversion, with treatment lowering the IOP for those with OHT and this reducing the risk of conversion to glaucoma compared with no treatment (HR 0.56, 95% CI 0.39 to 0.81). The risk of progression of glaucoma was also reduced by treatment237 (HR 0.65, 95% CI 0.49 to 0.87). The economic model estimated the proportion of individuals with OHT likely to develop glaucoma, severe glaucoma and visual impairment each year and cumulatively over a 10-year follow-up. We used 95% CIs and two mid-points between these to obtain final levels. 4,237 The estimates produced in the economic model were reported as the number of people experiencing a particular level of severity over a 10-year period. Figures were developed for a hypothetical cohort of 10,000 individuals who started in the model with OHT. The values presented to DCE participants were rounded to improve understanding of risk by respondents whilst still maintaining representation of the model estimates (see Table 26).

FIGURE 59.

Describing the clinical condition. Images reproduced with permission from the Belgian Glaucoma Society (http://www.glaucoma.be/en/index.html).

Because individuals are known to have problems with understanding risk,242 visual aids were used to help explain the concept of risk both in the focus group and subsequently in the DCE. Following Ancker and colleagues,243 a graphical presentation was used in the DCE (Figure 60).

FIGURE 60.

Graphical explanation of risk of developing glaucoma, severe glaucoma and visual impairment given to DCE participants.

Attribute 4: unwanted effects of treatment

Levels were defined from the glaucoma-specific utility measure4 and are detailed in Table 26.

Attribute 5: communication and understanding

There was some indication from the focus group that understanding the purpose of monitoring and feeling at ease were key components for monitoring. Therefore, following reflection on the findings of the focus groups and discussion within the study team four levels were developed for this characteristic:

• made me feel at ease and made sure I understood the purpose of monitoring

• made me feel at ease but did not make sure I understood the purpose of monitoring

• did not make me feel at ease but made sure I understood the purpose of monitoring and

• did not make me feel at ease and did not make sure I understood the purpose of monitoring.

Attribute 6: location

Defined as hospital eye clinic or local optician.

Attribute 7: cost per year (value)

To identify levels for the cost attribute a direct WTP question was included at the focus group stage. Here, participants were asked to consider the value that they would put on a monitoring service for people with raised eye pressure. Participants were presented with an example relating to the purchase of an article of clothing to help in understanding the concept of monetary value.

Pre-piloting

The questionnaire was pre-piloted opportunistically among 10 members of staff from the University of Aberdeen. Comments on the format of the questionnaire were given in terms of the ordering of the different sections. One participant argued that having the ‘easier’ questions concerned with personal characteristics at the beginning of the questionnaire would help to ease respondents into the rest of the survey. Other useful comments related to the format were: ‘useful to have an access link to the explanation of each characteristic and its levels throughout the choice section in the same screen shot’, ‘provide a hyperlink to the example choice as a guide for answering the choices’, ‘highlight key words to emphasise meaning and capture attention’. Most people agreed that pictorial representation of disease stages together with the graphical risk explanation made information more understandable. Changes were made to the format of the survey accordingly.

Developing the surveillance care pathways

The clinical care pathways for surveillance of patients diagnosed with OHT were developed by the Aberdeen project management group. The group included three ophthalmologists (AAB, AT and JB), three health economists (LV, RH and MR), a statistician (JC) and two project research fellows (APB and KB). Over a number of meetings the group reviewed national guidelines and used information from a review of economic evaluations of monitoring for OHT (Systematic review of economic evaluations of ocular hypertension surveillance programmes), a review of risk prediction tools (see Chapter 4), the optimal criteria for monitoring (see Chapters 5 and 6) and public preferences for monitoring (see Chapter 7) to determine the most appropriate care pathways to be compared in the model.

Five pathways were developed:

1. SOH in a primary care (community) setting

2. SOH in a hospital eye service setting

3. ‘treat all’ pathway

4. NICE guidelines (intensive)

5. NICE guidelines (conservative).

Public preferences for monitoring (see Chapter 7) and the evaluation of optimal monitoring criteria (see Chapters 5 and 6) informed a new pathway of care defined as the SOH pathway. This pathway was developed for both a community eye care setting (optometry) and a hospital eye service setting.

In the ‘treat all’ pathway, all individuals with an IOP > 21 mmHg would be treated with a PGA with advice to have their IOP checked annually at their community optometrist. This pathway was informed by a modelling evaluation by van Gestel and colleagues,30 which suggests that this pathway is cost-effective in a Dutch context.

The comparator pathways were defined by the NICE guidance. 1 Detailed specification of these pathways can be found in the next section. In brief, the NICE guidelines present a range of recommended monitoring intervals for patients dependent on both the initial risk stratification and treatment recommendation. We developed two pathways for current care from the NICE guidelines: ‘NICE intensive’, representing monitoring on the more intensive side of the guidelines’ recommended monitoring frequency (4-monthly to annually depending on initial risk stratification), or ‘NICE conservative’, representing the more conservative end of the recommended frequency of monitoring (6-monthly to biennially).

Description of the surveillance pathways compared and outline of the model

The SOH pathways were defined for a primary care (community) setting and a hospital eye service setting. In both of these pathways baseline IOP would be measured twice and a mean value calculated. Also, the frequency of the surveillance visits was taken to be 2 years. This interval was based on the work conducted to estimate the signal-to-noise ratio for measurement of IOP, reported in detail in Chapter 6. Other features of the pathways, such as the level of communication and understanding between the health-care professional and the patient or the formula for the estimation of the risk level that would trigger treatment, were informed by professional experts and public preferences for surveillance (see Chapter 7) and the comparison of IOP measurement agreement between alternative tonometers (see Chapter 5), respectively.

The ‘treat all’ pathway reflects a minimum level of care of people with OHT (i.e. no explicit or organised surveillance). In this pathway, all individuals with OHT receive treatment and these individuals are advised to seek, and assumed to take up, an annual IOP check-up from a provider based in the community (e.g. community optometrist). If there is a < 15% reduction in IOP from baseline (model entry) individuals are referred to secondary care.

The NICE guidelines1 provide a broad description of the range of management options but do not provide a sufficiently precise definition of the management that would be provided to an individual with a given set of risk characteristics; such detail is required to construct a model. Nevertheless, using the descriptions provided within the guidelines we developed two pathways for current care: ‘NICE intensive’ or ‘NICE conservative’. These pathways differed in terms of the time intervals between surveillance visits. For instance, with a range of 6–12 months stated within the NICE guidelines, the ‘NICE intensive’ pathway used a 6-month interval between surveillance visits whereas the ‘NICE conservative’ pathway used a 12-month interval between visits. For both of these surveillance pathways the location of surveillance was taken to be a hospital setting. Treatment was recommended as detailed in the NICE guidance.

These pathways are summarised in Table 33.

Characterisation of the risk profile of the simulated cohort and surveillance and treatment criteria for each pathway

Table 34 gives a detailed description of the risk stratification rules and surveillance and treatment criteria for each pathway. Every simulated individual entering the model has confirmed OHT based on an IOP of > 21mmHg and no ocular comorbidity. The pathways described in the first two columns of Table 34 are founded on the premise that an individual’s risk of converting to OAG is based on a set of definable characteristics. In contrast to this the ‘treat all’ pathway, described in the third column of Table 34, is implicitly based on the premise that individuals, regardless of their underlying characteristics, face the same risk of converting to OAG.

The sequence of treatment

Within the model the initial treatment choice for each individual is defined by the individual’s characteristics at the time when surveillance is initiated. Although the initial treatment may vary between individuals, based on a judgement of their risk of conversion to OAG, a common sequence of treatments was defined following discussions within the project team and with expert clinicians. Individuals will vary according to the point at which they enter this sequence (Figure 63). An individual with OHT would progress through these treatments if a judgement is made that IOP is not controlled. IOP is considered uncontrolled if the reduction from the IOP level at the start of the model is < 15% of the initial IOP [e.g. if an individual’s IOP level at the start of the model is 26 mmHg, then an IOP level > 22 mmHg (26 – 26 × 0.15 = 22.1) after treatment has been initiated would be regarded as uncontrolled]. Individuals who convert to OAG would receive PGAs if they had previously received either no treatment or BBs. However, a proportion of individuals will receive surgery (trabeculectomy) if prior to conversion they had received treatment with PGAs or combination therapy. The sequence of treatments allows for a maximum of two trabeculectomies, but does not allow those who are not diagnosed with OAG to receive surgery.

FIGURE 63.

Treatment sequence assumed for the model.

A further restriction on use of treatment is that we assume that individuals can progress through the treatment sequence but if they are initiated on a treatment that is part way through the sequence then they cannot subsequently go back to a treatment earlier in the sequence. The possible changes in treatment for those who have uncontrolled OHT are described in Table 36. Table 37 describes the possible changes in treatment for those who convert to OAG and Table 38 outlines the possible sequence of treatments for those with OAG.

Description of the discrete event simulation economic model

As already noted a discrete event simulation model was used to compare the pathways outlined in Table 33. On entry each individual has a predefined risk of developing OAG. This risk is based on their characteristics (e.g. age, IOP, etc., as described in Estimation of parameters for the natural history of ocular hypertension and open-angle glaucoma) defined above and the risk prediction algorithm reported in detail in Chapter 4. The model then compares the consequences for an individual, depending on their risk characteristics, for each of the five surveillance pathways described in Developing the surveillance care pathways. As described in Characterisation of the risk profile of the simulated cohort and surveillance and treatment criteria for each pathway, these surveillance pathways differ in terms of the process of surveillance, but following a correct diagnosis of OAG the progression of disease and the process of care are assumed to be identical. Thus, the model for these elements for the five surveillance pathways compared is also identical.

In discrete event simulation models a process or system is represented as a chronological sequence of events. The situation of an individual being modelled changes as a result of an event happening. For instance, in a model for the surveillance of individuals with OHT, the event can be a visit to the ophthalmologist where a particular set of tests is conducted. The change that might occur as a result of this event would be the diagnosis (or not) of OAG at that time.

Discrete event simulation models usually have five elements:

1. Entities: in this evaluation the entities are the individuals modelled within the discrete event simulation.

2. Characteristics: a set of characteristics for those individuals, which for this model are age and other risk predictors for OAG. These were defined based on the OHTS-EGPS risk prediction algorithm presented in Chapter 4.

3. Events: events occur at a particular point in time. The main events in this model are surveillance visits where the levels of defined characteristics are checked and the disease condition assessed. Other events include glaucoma surgery (e.g. trabeculectomy) after which individuals might enjoy a length of time with no progression without the need for further treatment.

4. Outcomes: these include mean cost, clinical outcomes (e.g. number of individuals converted to OAG), mean QALYs and mean WTP (as described in Chapter 7).

5. Relationships: mathematical or logical relationships that link the elements noted above together. An example of relationships in the model would be the way that the characteristics of an individual would link to the outcome of converting to OAG.

The important feature of a discrete event simulation is that an individual’s pathway through the model can change only if an event occurs. The likelihood of these events occurring is governed by the characteristics of the individual (i.e. the entity) and the relationships we define for the disease process and hence pathways of care. As events occur at particular moments in time during the individual’s pathway of care, the model includes a ‘clock’ to register an event.

Figure 64 illustrates how this process is operationalised to model the sequence of events involved in surveillance. This figure is based on the visual representation of the model in Simul8 version 17.0 (Simul8 Corporation, Boston, MA, USA), the software package used to construct the model.

FIGURE 64.

Model structure: OHT process. BB, beta-blockers; Q, queues (waiting for further action); WC, work centres; WEP, work entry point.

In Figure 64 we have marked five parts of the sequence of events involved as A, B, C, D and E. These points represent (A) the starting point, described in modelling terminology as the work entry point (WEP); (B) ‘queues’, where individuals wait until a further component in the model is free; (C) work centres (WC), where an event happens and some action is taken (e.g. a surveillance visit at which IOP measurement or perhaps a full assessment will take place); (D) end points, from which individuals cannot move elsewhere (i.e. death); and (E) the point at which, as a result of the findings of surveillance tests, IOP is judged to be off target or conversion to OAG is observed.

At the start of the model (marked as A in Figure 64) and within the WEP, the level for the characteristics (age, IOP, etc.) of each individual are defined, which determines the risk of progression to OAG. The value for each characteristic is based on data on the distribution of these characteristics in the population; we used the data from the Rotterdam study,34 the details of which are described in Chapter 3. As described in Table 34, for each surveillance pathway a set of criteria are used to decide whether treatment for OHT is prescribed and, if so, which treatment. This is reflected in Figure 64 by allocating individuals to no treatment, BBs or PGAs at the point at which they wait to start regular surveillance visits, illustrated by the queues marked as points B in the ‘monitoring visit loop’). The queues are an important aspect of a discrete event simulation as they are used to capture the potentially variable length of time that might exist between events, which in the context of Figure 64 are the surveillance visits.

When individuals enter the monitoring visit loop they go through a succession of surveillance visits (these are marked with a C in Figure 64). Before each visit the model re-estimates the level of IOP that the individual has at that time and whether or not they have truly converted to OAG. During each visit, IOP and disease stage are assessed and, based on the findings of the tests, any change in management is made (e.g. remain with current therapy and follow-up; change of therapy if IOP is ‘off target’; or referral to OAG follow-up if conversion to OAG is detected). This is marked with an E.

Figure 65 provides a more detailed illustration of this sequence and shows the model schematic for an individual who is currently receiving BBs. If at a surveillance visit IOP is off target or conversion to OAG is detected then the individual is sent to a queue (identified by point F) where they await assignment to the next treatment in the sequence of treatments available to them (see The sequence of treatment).

FIGURE 65.

Surveillance visit loop. Q represents queues for specific surveillance visits defined as WC at which either IOP measurements only are taken (WC_IOP only) or there is a full assessment (WC_ongoing monitoring).

If an individual converts to OAG then subsequent management will follow the NICE guidelines1 for individuals with diagnosed OAG. The model structure describing the sequence of events following conversion is shown in Figure 66 and is identical for all individuals who convert to OAG regardless of the surveillance pathway that they were on. As described in Table 34 the model assumed that all surveillance and care once an individual has converted to OAG would be provided by an ophthalmologist in a secondary care setting.

FIGURE 66.

Model structure: surveillance and treatment sequence following conversion to OAG. assess, assessment. Q, queues (waiting for further action).

The first stage of the treatment sequence (not shown in Figure 66) is when the ophthalmologist decides what treatment should be given (see Characterisation of the risk profile of the simulated cohort and surveillance and treatment criteria for each pathway). Within the model this occurs in the working centre ‘WC_ophthalmologist decision node’ (marked as G in Figure 66).

Within the model there are three possible therapeutic options: (1) medical management (PGAs only or in combination with BBs); (2) surgery; and (3) PGA only or in combination with BBs prescribed after failed surgery. Once an individual is prescribed a medical treatment, he or she will undergo a programme of surveillance that will continue until the individual exhausts the available medical treatment options (because medical treatments fail to keep IOP on target or the OAG continues to progress despite medical treatment and potentially escalation of medical treatment). This option is represented by the rectangle to the left-hand side of Figure 66.

Once medical treatment options have been exhausted then surgery will be considered as the second therapeutic option (represented by the rectangle in the centre of Figure 66). Surgery may also be ineffective at keeping IOP on target or preventing OAG progressing. In such a situation, individuals might return to medical therapy (e.g. PGAs only or in combination with BBs), the third therapeutic option. This option is represented by the rectangle on the right-hand side of Figure 66.

Estimation of parameters for the natural history of ocular hypertension and open-angle glaucoma

In this section we describe how we define several key sets of parameters used to populate the model structure described in Characterisation of the risk profile of the simulated cohort and surveillance and treatment criteria for each pathway. The first set of parameters relates to how we simulated the characteristics of the individuals who enter into the model. The second and third sets of parameters relate to assessment of IOP and changes in OAG status. Within the model both IOP and OAG status are assessed as these determine changes in management. The final set of parameters relates to the natural history of OHT and OAG and how treatments may alter the natural history.

Parameters used to modify natural history of ocular hypertension and open-angle glaucoma

Within the model there are two sets of parameters that have the effect of modifying the natural history of OHT and OAG. The first set relates to how accurate the measurement of IOP is in determining whether or not there is a clinically important change in IOP. This set of parameters determines whether or not the treatment of an individual should change. The second set of parameters relates to the effectiveness of treatments to control IOP and slow the rate of conversion to and progression of OAG.

Parameters used to describe the costs of surveillance, diagnosis and treatment

When surveillance was performed in a secondary-care setting (i.e. the NICE pathways and the SOH hospital eye service pathway, which were defined in Description of the surveillance pathways compared and outline of the model), we used the unit cost for an ophthalmology service outpatient visit from the Scotland National Statistics Information Services Division272 to cost a visit to measure IOP only. This cost was doubled to reflect the cost of a full assessment (i.e. IOP measurement together with visual field measurement and assessment of the optic nerve) conducted in a secondary-care setting.

For the ‘SOH primary care’ pathway the unit cost for a non-ophthalmologist full assessment conducted in a primary care or community setting was assumed to be that of the NHS sight fee. 273 Likewise, we assumed that the cost of an appointment at which only an IOP measurement was taken would be half of that of a full assessment (this latter value was also used as the cost of the annual measurement of IOP by a community optometrist recommended as part of the ‘treat all’ pathway).

Table 45 reports the unit costs for full assessment and IOP measurement only visits.

Health state utilities and benefit valuations

Model calibration and validation

In the development of any simulation model, care should be taken throughout the process so that credible and eventually useful results are produced. Parts of the development process are the verification, validation and eventually calibration of the model. Verification of the model refers to the steps taken to ensure that the model of an individual’s pathway, as well as all mathematical calculations that govern the journey through the model, are correct. Model validation refers to the actions taken to verify that the model fits with empirical data. If the model predictions do not appear to tally with the available empirical data, calibration is needed (i.e. change of model parameter values in order to correct these discrepancies).

To calibrate the model we first simulated individual characteristics for a population of 1000 people at the point at which they enter into the model. The data produced were then presented to the project management group and clinical experts and interpreted alongside data from the literature to form a judgement as to whether they had face validity. We did this by comparing the model simulated values with those of the Rotterdam data set (see Table 5).

Data analysis

The model compared alternative surveillance regimes by simulating for each pathway the cost per individual, clinical outcomes (e.g. number of individuals converted to OAG, number of individuals who progressed to moderate OAG, severe OAG and visual impairment), mean QALYs (e.g. EQ-5D-based QALYs) and mean WTP (e.g. monetary measure of benefits) for 10,000 individuals with OHT. A cohort of this size was chosen to enable the variation that would be expected between individuals to be represented in the estimates of costs and outcomes. Within the base-case analysis we adopted an NHS perspective and discounted costs and QALYs at the recommended 3.5% discount rate. WTP estimates were not discounted in the base-case analysis as these data implicitly included the respondents’ rates of time preference as respondents to the DCE were asked to consider trade-offs between short- and long-term outcomes. The model itself estimated costs and outcomes over a 20-year time horizon and all monetary values are expressed in 2009/10 prices.

From the data produced by the model for each surveillance pathway the following comparative results between surveillance pathways were estimated:

• incremental cost per number of OAG cases avoided, and incremental cost per case of each level of severity of OAG avoided

• incremental cost per QALY gained and

• incremental net benefit.

We addressed uncertainty by conducting deterministic (e.g. one-way) sensitivity analyses. The following sensitivity analyses were considered.

Model calibration

The simulated individual characteristics for a population of 10,000 ocular hypertensive individuals at model entry are described in Table 50.

We consulted the project management group and clinical experts about whether or not these data had face validity. It was agreed that, in comparison with available data detailed in Table 5 (the Rotterdam and Moorfields data sets), these estimates were appropriate; in Table 5 the mean age ranged from 56 to 64 years, while IOP ranged from 21 to 25 mmHg for those without OAG. Similar assessments can be drawn for CCT, VCD ratio and PSD. Finally, the predicted average 5-year risk of conversion for the (untreated) simulated individual group was 10%.

Estimates of effectiveness and mean cost per pathway

Table 51 shows the discounted cost and effectiveness analysis results. Effectiveness of SOH was expressed in several different ways: the number of individuals who converted to OHT; the number of individuals who progressed to moderate or severe OAG; and the number who progressed to visual impairment. Although not a prespecified form of analysis, the presentation of cost and clinical effect data for each pathway is equivalent to presenting the analysis as a cost–consequence analysis.