Home environmental assessments and modification delivered by occupational therapists to reduce falls in people aged 65 years and over: the OTIS RCT

Trial registration

The trial was assigned the International Standard Randomised Controlled Trial Number (ISRCTN) ISRCTN22202133 on 21 June 2016.

Inclusion criteria for OTIS cohort

• Aged ≥ 65 years.

• Willing to receive a home assessment from an OT.

• Community-dwelling.

• Had had at least one fall in the past 12 months or reported a fear of falling on their screening questionnaire (for at least some of the time).

Exclusion criteria for OTIS cohort

• Unable to walk 10 feet (3.05 m), even with the use of a walking aid.

• Unable to give informed consent, for example due to dementia.

• Lived in a residential or nursing home.

• Unable to read or speak English and had no friend or relative to translate/interpret for them.

• Had had an OT assessment for falls prevention in the previous 12 months or were on the waiting list for an OT assessment.

Participants who met any of the exclusion criteria for the study or did not meet the first three inclusion criteria were notified in writing that they were ineligible and no further correspondence was sent. Participants deemed eligible (i.e. those who met all of the inclusion criteria and none of the exclusion criteria) were sent a baseline questionnaire (see Report Supplementary Material 5) and a pack of monthly falls calendars (18 months’ worth; see Report Supplementary Material 6). Participants were eligible to be randomised once they had returned a completed baseline questionnaire and at least one falls calendar in the preceding 3 months. Participants who had neither had a fall in the past 12 months nor reported a fear of falling but were otherwise eligible for the study were deemed as ‘pending’ in terms of their eligibility and were followed up every 3 months for falls data. If individuals reported a fall, and were still willing to be in the study, they became eligible and were sent a pack of falls calendars and a baseline questionnaire to complete. If individuals did not report a fall by the end of the recruitment period, they were sent a letter to inform them that recruitment to the study had closed and that they were ineligible for the study.

Usual-care group

Participants allocated to the usual-care group continued to receive usual care from their GP and other health-care professionals, which may have included referrals to falls clinics. They were sent a falls prevention leaflet produced by Age UK27 with their baseline questionnaire in the post.

Intervention group

In addition to the usual care and the falls prevention leaflet given to the usual-care group, the participants in the intervention group were offered home environmental assessment and modification to identify personal fall-related hazards. The assessment was undertaken by an OT registered with the Health and Care Professions Council (HCPC). The OT used the Westmead Home Safety Assessment (WeHSA) tool28 to structure their assessment visit. The WeHSA was developed in Australia in 1997 for older adults and is a validated tool. It comprises a 57-item functional assessment organised into 15 domains: internal/external traffic ways, general indoors, living area, seating, bedroom, toilet area, bathroom, kitchen, laundry, mobility aid, footwear, pets, medication management and safety call systems. The assessment consisted of an initial discussion about the participant’s history of falling, lifestyle, patterns of usage of areas in the home, risk-taking behaviour, strategies already adopted to reduce falls, environmental changes made and functional vision. The participant and the OT then moved through the home and identified potential falls hazards. The OT encouraged the participant to identify potential strategies to mitigate any hazards, but offered suggestions if needed. A list of recommendations was agreed and, if needed, the OT would either refer on to other agencies or liaise with family members regarding the provision of equipment and modifications or refer on to other health-care services as deemed clinically appropriate. The Timed Up and Go test29 was also conducted. The Timed Up and Go test is a standardised tool used as a falls risk indicator, with scores of > 14 seconds indicating a high risk of falling. These data were collected to allow secondary analysis to be undertaken once the main findings of the study are published. A second visit by the OT could have been undertaken if deemed clinically necessary, but, to our knowledge, none was. The OT contacted the participant 4–6 weeks after the home visit to collect data on whether or not the recommendations had been acted on. To document the activities undertaken at the visit, the OT completed an OT booklet (see Report Supplementary Material 7).

Occupational therapist training to deliver the intervention

The OTs delivering the intervention attended a 1-day face-to-face training session on how to conduct the assessment before conducting the home visits with participants. A standardised training package was developed by the co-applicant who had undertaken the pilot study16 (Alison Pighills) in collaboration with Professor Lindy Clemson, who originally developed the WeHSA. A standardised approach was used to ensure that the OTs who delivered the intervention did so consistently across all of the trial sites. The training materials included a presentation with detailed notes, a training manual and videos of older people undertaking activities of daily living, which the OTs assessed using the WeHSA. The training covered the following 10 domains: prevalence of falls; evidence underpinning environmental assessment and modification; falls risk factors; the person–environment–occupation conceptual model of practice and occupational performance;30 background on falls – categories, types and locations; environmental assessment; equipment and ideas for falls prevention; adherence to recommendations; action-planning; and scoring a video of an older person carrying out functional tasks at home using the WeHSA. Self-efficacy was addressed in the OT training in relation to its influence on participants’ confidence in their ability to engage in activities without falling, and the potential for low self-efficacy to lead to activity avoidance and deconditioning. OTs were encouraged to address both falls self-efficacy and fear of falling with participants as a component of the visit. The OT training addressed grading the demands that activities placed on the individual, particularly in the context of ‘environmental press’, which is the demand that the environment places on the individual, and reducing that demand for frail older people to enhance their self-efficacy. The initial training session was led by the co-applicant (Alison Pighills) who conducted the pilot trial. Two other HCPC-registered OTs and study co-applicants (Shelley Crossland and Avril Drummond) were also trained to facilitate the training at subsequent training sessions using a ‘train the trainer’ approach. 31,32 These face-to-face training sessions were audio-recorded for the purpose of evaluating training fidelity. The face-to-face training was supplemented by an online training module. 33 Those who could not attend the face-to-face training undertook the online training course in addition to attending ‘cascade’ training delivered by one of the OTs who had attended the face-to-face training sessions. These ‘cascade’ trainers were provided with the same training package used in the face-to-face sessions, which included extensive notes. All OTs trained to deliver the intervention were supported by Shelley Crossland, Avril Drummond and Alison Pighills, who addressed any queries or points for clarification raised during the training or throughout the trial.

Primary outcome

The primary outcome of the trial was the number of falls per participant over the 12 months from randomisation. A fall was defined as ‘an unexpected event in which the participant comes to rest on the ground, floor or lower level’. 34 Data were collected from self-reported monthly falls calendars, on which participants were asked to mark the number of falls they had on each day or to indicate that they had not fallen that month. An explanation of what the researchers considered to be a fall was included in the participant information sheet and on the falls calendars. If a participant was uncertain whether an event would be classed as a fall, they were encouraged to ring the research team at YTU to discuss it. Participants who indicated on their falls calendar that they had a fall were contacted by the research team for further information. Information collected about the falls included the date and number of falls, the location of the fall, what the participant was doing when they fell (i.e. the cause/reason for the fall), injuries from the fall (e.g. superficial wounds, such as bruising, sprain, cuts, abrasions or fractures, including type of fracture) and any hospital admissions (see Report Supplementary Material 8).

Data collected on the 4-, 8- and 12-month post-randomisation follow-up participant questionnaires included the number of falls in the previous 4 months and were used to calculate the falls rates for participants who did not return any monthly falls calendars.

Secondary outcomes

All secondary outcomes were self-reported by the participant and collected using questionnaires at baseline and at 4, 8 and 12 months post randomisation, or on monthly falls calendars. The secondary outcomes were:

• proportion of participants reporting at least one fall in the 12 months from randomisation

• proportion of participants reporting multiple (two or more) falls in the 12 months from randomisation

• time to first fall from date of randomisation and between subsequent falls

• fracture rate

• fear of falling

• health-related quality of life, as measured using the EuroQol-5 Dimensions, five-level version (EQ-5D-5L)

• health service utilisation.

Scoring of instruments

Other data collected

Primary analysis

The number of falls per person was analysed using mixed-effects negative binomial regression adjusting for sex (male/female), age at randomisation (continuous), history of falling and the allocation ratio used to randomise the participant as fixed effects, and centre as a random effect. Participants were classified into two groups for the history of falling covariate: (1) one or no falls in the 12 months prior to completion of the screening questionnaire; and (2) two or more falls reported in the 12 months prior to completion of the screening questionnaire. The model included an exposure variable for the number of months that the participant returned a monthly falls calendar (i.e. the number of months’ worth of falls they report).

The point estimate for the treatment effect in the form of an adjusted incidence rate ratio (IRR) is provided, along with its associated 95% CI and p-value.

This analysis primarily included falls data from the falls calendars, but, where no post-randomisation calendars were returned, data from the 4-, 8- and 12-month participant questionnaires were used for a participant, where available. When no falls data were provided at all, an assumption of zero falls over a negligible follow-up time of 0.1 months was made for the participant in the analysis.

Analysis of the primary outcome was checked and verified by a second statistician.

Sensitivity analyses

Therapist effects

In some centres, more than one therapist delivered the intervention visits to the participants. We therefore have clustering by therapist in the intervention group. The value of the intervention may depend on the skill/experience of the therapist and their relationship with the participant. Therefore, to account for this potential variation between therapists, a sensitivity analysis was conducted using an artificial cluster method. With this method, every participant (whether allocated to the intervention group or to the usual-care group) was associated with a therapist. For intervention participants, their trial therapist was known; however, for usual-care participants, we assigned them a counterfactual therapist, that is, one they could have seen had they been randomised to the intervention group. For centres where only one therapist delivered the visits, all of the usual-care participants were assigned this therapist. For centres that had more than one trial therapist, the usual-care participants were randomly assigned one of these therapists in the proportion that the therapists saw intervention participants. Each therapist then had their own cluster of usual-care and intervention participants. Therapist (rather than centre) was then included as a random effect in the primary analysis model.

Subgroup analysis

The primary analysis was repeated including an interaction term between the treatment allocation and whether or not a participant received care in a hospital (outpatient appointment, day case, accident and emergency (A&E) presentation, or hospital admission) as a result of a fall in the 4 months prior to completion of the baseline questionnaire.

Secondary analyses

The following secondary outcomes were analysed by mixed-effects logistic regression adjusting for sex, age, history of falls and allocation ratio as in the primary analysis, and centre as a random effect:

• proportion of participants who fell at least once over the 12 months from the date of randomisation*

• proportion of multiple fallers (participants who had two or more falls in the 12 months from randomisation)*

• proportion of participants who reported that they worried about falling at least some of the time at 12 months post randomisation

*using data from monthly falls calendars except where no post-randomisation calendars were returned, in which case data from the 4-, 8- and 12-month participant questionnaires were used, where available.

Adjusted odds ratios (OR) for the intervention effect and their associated 95% CIs and p-values are provided.

The primary and secondary analyses relating to falls (number of falls and proportion of single and multiple fallers) were also repeated using data only from participant follow-up questionnaires.

The proportion of participants having at least one fracture or multiple fractures resulting from a fall are reported but were not formally analysed owing to the rarity of these events.

Fear of falling was also analysed in its continuous form using a covariance pattern model incorporating all post-randomisation time points and adjusting for baseline fear of falling, sex, age, history of falling, allocation ratio, treatment group, time, and a treatment group-by-time interaction. The correlation of observations within participants over time was modelled using participant and centre as random effects. The Akaike information criterion was used to compare models specifying different correlation structures (smaller values were preferred). 39 An unstructured covariance structure was used in the final model. Model assumptions were checked visually. The normality of the standardised residuals was checked using a Q–Q plot, and homoscedasticity was assessed using a scatterplot of the standardised residuals against fitted values. There was no evidence to suggest a violation of the underlying assumptions, so data were not transformed. Adjusted mean differences in fear of falling between the two groups at 4, 8 and 12 months are provided, with their 95% CIs and p-values.

The time to the first fall was derived as the number of days from randomisation to the first fall reported on the monthly falls calendars. The time between any subsequent falls was also calculated. Participants who did not have a fall were treated as censored at their date of trial exit, or the date of their last available assessment, or 365 days or trial cessation, as appropriate. For months for which no calendar was returned, it is assumed, by default, that no falls were experienced in these months. The proportion of participants yet to experience a fall was summarised using a Kaplan–Meier survival curve for each group. Time to fall was analysed using the Andersen and Gill method40 for analysing time to event data when the event can be repeated. The analysis treats each time to event or censoring as a separate observation. The data were analysed by Cox proportional hazards regression, using robust standard errors to account for dependent observations by participant and adjusting for the same covariates as in the primary analysis model. Adjusted hazard ratios and their associated 95% CIs and p-values are provided.

Recruitment

The original funding application stated that we planned to randomise 1299 participants to OTIS over a 10-month period. Participants would be identified from either cohorts of participants held at the University of York and the University of Sheffield or direct mail-out to patients on GP lists. However, study commencement was delayed by approximately 4 months because of contractual issues and delays in obtaining regulatory approvals. As the trial progressed, recruitment fell below the expected level, due, in the main, to the delay in setting up sites. Approval was obtained from the funder to extend the study by 12 months to a total of 43 months (June 2016 to December 2019). This permitted the set-up of additional sites and mail-out of recruitment packs to potential participants from GP surgeries. Details of the recruiting sites and the dates that research and development departments confirmed their capacity and capability to undertake the study can be found in Appendix 1. To facilitate recruitment, approval to use additional recruitment strategies, namely opportunistic screening by OTs and other health-care professionals, media advertising for participants and rescreening participants, was received in January 2017, May 2017 and March 2017, respectively.

Treatment fidelity

In July 2017, additional strategies were included in the protocol to assess treatment fidelity. Discussions about fidelity strategies were guided by the TSC/DMEC and agreed by the funder. Additional strategies included some observational work of the OTs delivering the intervention and a review of the OT booklets to ensure that key elements of consultations had been included.

Intervention: undertaking the follow-up telephone call

In October 2017, a change to the protocol allowed members of the research team as well as OTs to undertake the follow-up telephone call to check participants’ adherence to the recommendations suggested in the home visit. This was intended to relieve some of the burden on the OTs; however, in the end the OTs had capacity to undertake all of the follow-up calls.

Raw data

In total, 1303 (97.9%) trial participants returned at least one falls calendar following randomisation (intervention, n = 419, 97.4%; usual care, n = 884, 98.1%), with 1204 (90.5%) returning a complete 12 months’ worth of calendars post randomisation (intervention, n = 377, 87.7%; usual care, n = 827, 91.8%). Of the 28 participants who did not return any falls calendars, five (four usual care, one intervention) provided falls data on at least one of the 4-, 8- and 12-month participant questionnaires. These data were used in the analysis for these participants. In total, 2260 falls were reported: 826 in the intervention group (mean 1.9 falls, SD 5.5 falls; median 1 fall, range 0–94 falls) over an average of 338 days (median 365 days), and 1434 in the usual-care group (mean 1.6 falls, SD 3.0 falls; median 1 fall, range 0–41 falls) over a mean of 345 days (median 365 days).

At least some information, such as the location and perceived cause, was available for 2037 (90.1%) falls (intervention, n = 700, 84.7%; usual care, n = 1337, 93.2%; Table 6). Just over half of falls for which there was available location information occurred indoors (53.4%), with the majority (85.7%) of these occurring inside the participant’s own home rather than inside another premises. About half of the falls resulted in a superficial injury or worse, with 2.8% of the falls resulting in a broken bone (from 16 falls in the intervention group and 41 falls in the usual-care group), most frequently in the wrist or the hand (17/57, 29.8%).

Covariates

The primary analysis model controlled for sex, age at randomisation, history of falling and allocation ratio, and centre. Full covariate data were available for all randomised participants so the primary model was based on 1331 participants.

Post hoc adjustment for Parkinson’s disease

One participant, in the intervention group, reported 94 falls over the 12 months from randomisation. This number of falls was significantly greater than the next largest of 41 (in the usual-care group). We observed that this participant had Parkinson’s disease, and also that there was a chance imbalance in the proportion of participants in the two groups who had Parkinson’s disease. We therefore decided, as a post hoc sensitivity analysis, to repeat the primary analysis, including whether or not the participant had Parkinson’s disease as an additional fixed effect. The resulting IRR was decreased to 1.11 (95% CI 0.94 to 1.31) and there was no evidence of a difference between the two groups (p = 0.23).

Non-compliance

When non-compliance with the intervention was accounted for using an instrumental variable CACE analysis approach, defining compliance as receipt of an OT-delivered environmental assessment within 12 months of randomisation, the CACE estimate of the intervention effect was very similar to that from the ITT analysis (IRR 1.18, 95% CI 0.98 to 1.43; p = 0.08).

Missing data

Ten (2.3%) participants in the intervention group and 13 (1.4%) participants in the usual-care group did not provide any falls data post randomisation. These participants were still included in the model as zero falls over a negligible time frame of 0.1 months was imputed for them. However, we conducted a logistic regression to assess whether or not any baseline factors were associated with missing outcome data and whether or not a participant reporting a fall in the 12 months prior to completing the screening questionnaire was the only statistically significant predictor. Participants who had had at least one previous fall were more likely to have missing outcome data (OR 5.84, 95% CI 1.13 to 30.21; p = 0.04). When this was added as a covariate in the primary analysis, the intervention effect estimate was virtually unchanged (IRR 0.17, 95% CI 0.99 to 1.38; p = 0.07).

Number of falls as reported for the previous 4 months on the 4-, 8- and 12-month participant questionnaires

In the intervention group, 404 participants (94.0%) provided a valid response to the number of falls they had experienced over the previous 4 months on at least one of the 4-, 8- and 12-month questionnaires. An average of 2.0 falls was reported (range 0–67 falls) over an average of 11.4 months. In the usual-care group, 870 participants (96.6%) provided a valid response to the number of falls they had experienced over the previous 4 months on at least one of the 4-, 8- and 12-month questionnaires. An average of 1.8 falls was reported (range 0–43 falls) over an average of 11.5 months. After imputing zero falls over a negligible time period of 0.1 months for those who did not provide any data, the adjusted IRR obtained from the negative regression model was 1.05 (95% CI 0.89 to 1.25; p = 0.56).

Comparison of the number of falls reported on falls calendars and in participant questionnaires

We compared the number of falls reported by participants in the questionnaires and on falls calendars. Of the 1331 randomised participants, 1132 (85.0%) returned 12 months’ worth of falls calendars and provided data on retrospective falls in all three participant questionnaires (4, 8 and 12 months). Among these, in the intervention group, the average number of falls reported on calendars was 1.9 (SD 5.7, median 1, range 0–94 falls) and in questionnaires was 2.0 (SD 5.0, median 1, range 0–67 falls). The correlation between the two counts was high, at 0.93 (95% CI 0.91 to 0.94). In the usual-care group, the average number of falls reported on calendars was 1.6 (SD 2.9, median 1, range 0–41 falls) and in questionnaires was 1.8 (SD 3.2, median 1, range 0–43), with correlation at 0.89 (95% CI 0.88 to 0.90). Figure 6 depicts a Bland–Altman plot of the agreement between falls reported on the calendars and falls reported in the questionnaires. There is strong agreement; the mean difference is –0.16 (participants tended to report marginally more falls in the questionnaires than on the calendars). The 95% limits of agreement are –3.5 to 3.2. There is a hint of a trend that the higher the average count of falls (between the two measurements), the larger the difference between the two tends to be.

FIGURE 6.

Bland–Altman plot of the agreement of falls reported in questionnaires and on calendars.

Proportion of fallers and multiple fallers

In total, 245 out of 430 (57.0%) intervention group participants and 506 out of 901 (56.2%) usual-care group participants reported at least one fall on their monthly falls calendars or in participant questionnaires (adjusted OR 1.06, 95% CI 0.83 to 1.34; p = 0.65).

Using only data from the participant questionnaires, 230 out of 430 (53.5%) randomised participants in the intervention group and 506 out of 901 (56.2%) participants in the usual-care group reported at least one fall. There was no evidence of a difference in the likelihood of reporting a fall between the two groups (adjusted OR 0.91, 95% CI 0.72 to 1.15; p = 0.42).

These analyses, by default, assume that (1) participants who did not return any falls data did not fall and (2) partial responders had no falls in the months for which no data were provided. In terms of point 1, 57 (4.3%) participants failed to provide any falls data in the participant questionnaires, and only 23 (1.7%) participants failed to provide any falls data either on the monthly calendars or in the participant questionnaires. At the extremes, we could assume that all 57 participants with missing questionnaire falls data in each group, in turn, did fall at least once in the 12 months following randomisation. Assuming that the 26 participants in the intervention group fell (but leaving the usual-care group unchanged) the adjusted OR favours usual care, but the result is not statistically significant (1.17, 95% CI 0.92 to 1.49; p = 0.19). Conversely, assuming that the 31 participants in the usual-care group fell (but leaving the intervention group unchanged), there is weak evidence of a reduction in the proportion of participants in the intervention group who experienced at least one fall (adjusted OR 0.79, 95% CI 0.63 to 1.01; p = 0.06). The conclusion that there is no difference in the proportion of fallers in the two groups is, therefore, reasonably robust.

Proportion of multiple fallers (those having two or more falls in the 12 months from randomisation)

The proportion of participants who reported two or more falls on their falls calendars (or participant questionnaires, where no falls calendars were returned) following randomisation was also slightly higher in the intervention group than in the usual-care group (148/430, 34.4%, vs. 298/901, 33.1%; adjusted OR 1.11, 95% CI 0.86 to 1.43; p = 0.42).

Using only data from the participant questionnaires, 149 out of 430 (34.7%) randomised participants in the intervention group, and 323 out of 901 (35.9%) randomised participants in the usual-care group reported at least two falls. There was no evidence of a difference in the likelihood of reporting multiple falls between the two groups (adjusted OR 0.96, 95% CI 0.75 to 1.24; p = 0.78).

Similarly, these analyses must be interpreted with the following in mind. By default, they assume that (1) participants who did not return any falls data actually fell no more than once in the 12 months from randomisation, (2) partial responders who reported one fall experienced no falls in all the months for which no data were provided; and (3) partial responders who reported no falls experienced no more than one fall in all the months for which no data were provided. No further analyses to test the robustness of these results to these assumptions were deemed warranted, as such a large proportion of participants (90%) returned 12 months’ worth of falls calendars.

Proportion of participants having at least one fracture over the 12 months’ follow-up

A total of 54 participants reported a fracture from a fall in the 12 months from randomisation (intervention group, 16/430, 3.7%; usual-care group, 38/901, 4.2%).

Proportion of participants obtaining multiple fractures

Only two participants, both in the usual-care group, reported more than one fracture resulting from a fall, from separate events, in the 12 months from randomisation. One reported three fractures (wrist, ribs and knee) and the other reported two fractures (wrist and knee).

Proportion of participants who reported that they were worried about falling at 12 months post randomisation (reported worrying about falling at least some of the time)

Fear of falling was measured at screening and in the 4-, 8- and 12-month questionnaires using the question ‘During the past 4 weeks how often have you worried about having a fall?’, with the following response categories:

• 1 – all of the time

• 2 – most of the time

• 3 – a good bit of the time

• 4 – some of the time

• 5 – a little of the time

• 6 – none of the time.

At 12 months, 197 out of 375 (52.5%) participants in the intervention group and 440 out of 818 (53.8%) participants in the usual-care group (where the denominator represents those providing a valid response to this question) reported that they worried about falling at least some of the time (some, a good bit, most or all of the time). There was no evidence of a difference in the likelihood of participants in the intervention group reporting a fear of falling relative to the usual-care group (adjusted OR 1.00, 95% CI 0.78 to 1.29; p = 1.00).

Fear of falling in its continuous form

Raw scores for fear of falling are summarised in Table 7 and were fairly consistent between groups and over time. At screening, the average score was 4.3 (SD 1.3) in the intervention group and 4.2 (SD 1.3) in the usual-care group. At 12 months, the average score was 4.2 (SD 1.3) in the intervention group and 4.1 (SD 1.3) in the usual-care group. There was no evidence of a difference in fear of falling between the two groups at any post-randomisation time point (see Table 7; Figure 7).

FIGURE 7.

Adjusted means for fear of falling outcome by randomised group.

Time to fall

The median time to fall was 119 days in the intervention group (95% CI 105 to 133 days) and 144 days in the usual-care group (95% CI 132 to 155 days). Kaplan–Meier survival curves are presented for each group in Figure 8. The adjusted hazard ratio from the Cox proportional hazards model for the intervention effect was 1.24 (95% CI 0.94 to 1.63; p = 0.12), indicating that the hazard or chance of falling at any particular time was higher in the intervention group than in the usual-care group, but this ratio is not statistically significant. This analysis assumes that no falls were experienced in the months for which no falls calendar was returned; no further investigation or consideration of imputation of data was deemed necessary to evaluate the impact of this assumption, as such a high proportion of randomised participants (90%) returned a full 12 months’ worth of falls calendars.

FIGURE 8.

Kaplan–Meier curves by randomised group for time to fall.

Costs of health-care services

Unit costs for health-care resource use items were obtained from established costing sources such as NHS Reference Costs 2017/1851 and the Unit Costs of Health and Social Care 2018 from the Personal Social Services Research Unit (PSSRU). 52 The unit cost items used in the analysis are summarised and presented in Table 8. The year of pricing was set as the mid-year of the trial (i.e. 2017–18), based on the trial period of June 2016–19. All costs were evaluated in Great British pounds for 2017–18. Unit costs were multiplied by the corresponding resource use data to give an estimation of the total cost per participant.

The total cost for each participant comprised the following elements: health-care resource utilisation (i.e. primary, community and secondary care visits and attendances) and the cost of the OT intervention (for those in the intervention group only). Further details of the intervention cost are provided in Costing the occupational therapist intervention section. Participants in the usual-care group received usual care from their GP and health-care professionals, and hence the cost of usual care incorporates the cost of this resource use. As the falls prevention advice leaflet produced by Age UK27 was provided to participants in both groups, it has been excluded from the analysis.

Costing the occupational therapist intervention

The cost of implementing and delivering the intervention was estimated based on information obtained from the OT home assessment visit that participants in the intervention group received, plus information regarding the cost of training the OTs to undertake the home visits. The intervention cost therefore comprised (1) the cost of OTs’ time spent on the home visit, including travel for the visit, and follow-up telephone call(s); (2) the cost of training the OTs; and (3) the cost of the equipment provided to the participant following recommendations from the OTs at the home visit. The three elements of the intervention cost are described in detail in the sections below, and the corresponding unit costs are summarised in Table 9.

Multiple imputation

Missing data are likely to arise in economic studies that involve the collection of patient-level data. 57 Reasons for missing data include participants failing to complete certain items in the questionnaire, and participants failing to return the questionnaire at all. The total mean cost generated as part of an economic analysis often involves the summation of the costs of several resource use items. If responses to any of the items are missing, however, the overall cost would be considered missing if an approach such as complete-case analysis was utilised. This form of analysis would lead to meaningful data being excluded;58 there might be only a small proportion of missing resource use items, but a high proportion of individuals with missing total costs. 57 Complete-case analysis is considered a useful starting point, although it is not recommended for use in the base case of a within-trial economic analysis. 59 Available case analysis uses data more efficiently, although it also has limitations, for instance relating to the use of different samples for the costs and outcomes. 59

Multiple imputation (MI) is an alternative approach to deal with missing data that relies on the assumption that data are missing at random; that is, the probability that data are missing is not dependent on the unobserved data, conditional on the observed data. 60 MI estimates a set of plausible values for the missing data using the distribution of the observed data,60 with multiple ‘complete’ data sets created. Owing to the creation of multiple predictions for each missing value, MI incorporates the uncertainty in the imputations and provides accurate standard errors. 61

Implementing MI with chained equations when data are not missing at random could result in biased estimates. 61 Descriptive analysis of missing economic data was undertaken to determine whether or not the missing at random assumption underpinning MI was plausible. Missing economic data at all follow-up points for each study group and missing data patterns were explored. Logistic regression was used to assess the association between missingness and baseline variables, and the association between missingness and observed costs and EQ-5D-5L scores.

The findings of our investigations into missing economic data indicated that the missing at random assumption is plausible, as presented in more detail in Results. Therefore, the base-case analysis was conducted using MI for the handling of missing data. The model used MI with chained equations, with predicted mean matching on utilities at 4, 8 and 12 months and the cost estimates, and hence ensured that only plausible values were imputed. The imputation model included age, sex, centre, history of falling at baseline, utilities (at baseline and at 4, 8 and 12 months) and total costs at the resource use level (e.g. total cost of GP visits over 12 months). Three imputations are suggested to be sufficient for a data set that has 20% of total data missing. 62 Owing to the extent of missing data in the present study, MI by chained equations was undertaken for a total of 10 imputations, with graphical plots used to compare the distributions of the observed data with the imputed data, thereby depicting whether or not the imputed data resemble the observed data. Mean cost and QALY estimates were generated using the combined imputed data sets following Rubin’s rules. 63 The multiply imputed data set was used for the base-case economic analysis, with incremental cost-effectiveness ratios (ICERs) generated as described in Incremental analysis.

For comparison purposes, we conducted a complete-case analysis as a separate sensitivity analysis, in which only participants with observed data for all costs and utilities were included. As complete-case analysis relies on the data being not missing at random, the sensitivity analysis allows us to explore the impact of this alternative assumption around missing data. Available case analysis has been used for initial exploration of the data.

For participants who died during the study, usual imputation methods were applied when these participants had missing data before their death. For resource use and QALY data that would have been obtained via questionnaires received at any follow-up time point after their death, zero resource use and zero QALYs were assumed.

Incremental analysis

Costs and health outcomes (i.e. QALYs and falls) are summarised for both groups in terms of mean value and SD, and mean difference (with 95% CIs) between the groups. The analyses used conventional decision rules when comparing the mean costs and outcomes for both groups, with the results presented in terms of ICERs (i.e. the additional cost per extra unit of benefit), where appropriate. The ICERs of (1) the cost per fall averted and (2) the cost per QALY gained were estimated based on Equation 1:

where incremental costs and incremental effects are shown by ΔC and ΔE, respectively.

The ICER is compared with the willingness-to-pay threshold, i.e. the amount that a decision-maker is willing to pay for an additional QALY. A willingness-to-pay threshold of £20,000–30,000 has been used for the analyses, in line with NICE recommendations;42 that is, if the estimated cost per QALY is below the threshold, the intervention will be considered cost-effective. Interventions with an ICER < £20,000 would generally be considered cost-effective.

The findings are also presented in terms of the net monetary benefit (NMB),64 which involves translating the health benefits associated with the intervention into monetary terms and use of the cost-effectiveness threshold λ. If the mean incremental NMB (i.e. the difference in NMB between the intervention and usual care) is greater than zero, then the intervention is considered to be cost-effective at the threshold under consideration:

The NMB was estimated at a range of willingness-to-pay thresholds, in addition to the uncertainty around the net benefit estimates that is presented graphically using cost-effectiveness acceptability curves (CEACs), as detailed in Analysis of uncertainty.

QALY data were adjusted for baseline EQ-5D-5L scores48 to allow for any differences at baseline between the two groups. Adjustment was made for covariates consistent with those used in the primary statistical analysis. Differences between the groups were found to be statistically significant if the p-value was < 0.05 and are presented alongside CIs around the differences in costs and outcomes.

The cost-effectiveness analysis produced estimates of the mean differences in costs and outcomes using seemingly unrelated regression equations, with 95% CIs estimated using bias-corrected and accelerated bootstrap methods. Bootstrapping was undertaken for 10,000 replications, thereby generating 10,000 estimates of incremental costs and incremental effects, and presented on the cost-effectiveness plane. Regression methods have been used to take into account differences in stratification or prognostic variables, and other sources of heterogeneity. Analyses were conducted using Stata release 16 (StataCorp LP, College Station, TX, USA).

Analysis of uncertainty

The decision uncertainty around the cost-effectiveness estimates has been explored by investigating the probability that the OT intervention is cost-effective relative to usual care at a range of values that decision-makers are willing to pay to gain one additional QALY. As mentioned above, CEACs have been used to illustrate this uncertainty. 65 Sensitivity analyses have been undertaken to explore the variability in estimating cost-effectiveness, as follows:

1. complete-case analysis as an alternative to the use of MI for dealing with missing data

2. inclusion of non-falls-related health-care resource use in addition to the falls-related resource use

3. inpatient stay data from falls data sheets, rather than from participant-completed questionnaires

4. exploration of the assumption that all equipment provided as part of the intervention is funded by the NHS and PSS (rather than in the base case, which attaches costs only to the items that were paid for by the NHS and PSS in the study and not to the items that were reported as funded by participants themselves, i.e. out-of-pocket expenditure)

5. paid care worker visits being paid for by the NHS and PSS (rather than by the participant/relative as in the base case).

Extrapolation to the longer term

A further analysis using an exploratory decision model was also due to be conducted to explore the possible long-term impact of the trial assuming that a falls reduction leads to a reduction in fractures. However, owing to the study findings regarding falls (i.e. that the intervention was associated with an increased falls rate), it was not possible to extrapolate to the future in terms of a fracture reduction, and hence a long-term model has not been included in the economic evaluation.

Participant population and missing data

A total of 35 participants died during the study period: 13 out of 430 (3.0%) in the intervention group and 22 out of 901 (2.4%) in the usual-care group.

The complete-case analysis (sensitivity analysis 1) included participants who had complete data for all costs and utilities over the entire study duration. A total of 412 participants had complete economic data: 121 intervention participants and 291 usual-care participants. The proportion of participants who had complete economic data available remained at similar levels over the course of the study; 54.4% and 55.5% of participants had complete data for the intervention and usual-care groups at baseline, respectively, compared to 55.6% and 56.8% at 12-month follow-up. Further details on the remaining time points are available in Appendix 5. For both groups, there was a higher number of complete cases at 4 months than at any of the other time points. The missing data patterns displayed for EQ-5D-5L and cost data are not monotonic; there are participants with intermittent missing data. For example, some participants were lost to follow-up at 4 months (i.e. had missing data), but had observed data at 8 months and/or 12 months. The use of complete-case analysis would therefore be inefficient due to discarding data from these individuals altogether, on the basis of having one or more missing data items.

The findings of the logistic regressions of indicators of missing cost and QALY data on study group allocation, several baseline variables and observed outcomes are presented in Appendix 6. Age, sex and lower EQ-5D-5L scores at baseline were significantly associated with missing cost data at 12 months. Baseline EQ-5D-5L score, age and history of falling were found to be significant predictors of missing QALY data at 12 months. Observed values for QALYs at both 4 months and 8 months were significantly associated with missingness of QALY data at 12 months, and also with missingness of cost data at 12 months. Observed costs at 4 months and 8 months were not found to be significant predictors of missing QALY data at 12 months or of missing cost data at 8 months. Observed costs at 4 months were, however, significantly associated with missingness of QALY data at 12 months. These findings suggest that the data are unlikely to be missing completely at random and support our assumption that data are missing at random.

Economic data were, therefore, assumed to be missing at random, with MI by chained equations used to deal with the missing data for the health economic analysis. Missingness was assumed to depend on baseline covariates (sex, age, history of falling and EQ-5D-5L at baseline) and observed costs and QALYs. The distributions of the imputed data were compared with the distributions of the observed data, as a means of checking the fit of the imputation model, with the distributions found to be similar for both costs and utilities (see Appendix 6).

Health-related quality of life

The EQ-5D-5L is categorised as complete if each of its five dimensions contains a response. The completion rates for the EQ-5D-5L were 93% for both groups at baseline (see Appendix 7), and, although the completion rates decreased by 12 months, they remained high, at 91% and 86% for the usual-care and intervention groups, respectively. Full details of the missing dimensions for the incomplete EQ-5D-5L questionnaires are provided in Appendix 8.

A breakdown of the different EQ-5D-5L levels (1–5) reported by participants, according to dimension, time point and group, can be seen in Appendix 9, for all available cases. Over 80% of participants reported having problems in terms of pain/discomfort at all time points in both groups, and problems with mobility and usual activities were experienced by at least 62% of participants. Fewer participants reported problems with the dimensions of self-care and anxiety/depression, with over 25% and 37% experiencing these at the study follow-up points.

The mean EQ-5D-5L visual analogue scale scores were found to be similar at baseline: 73.3 in the intervention group and 73.7 in the usual-care group (see Appendix 10). Scores reduced slightly in both groups over time, with 12-month scores being 71.5 for the intervention group and 72.1 for the usual-care group. Median visual analogue scale scores remained at 75 throughout the study for both groups at all time points.

Participants’ mean utility scores at each time point are summarised in Table 10 for all available cases, alongside the mean difference between the groups, both unadjusted and adjusted for baseline utility. At baseline, utilities were higher, on average, for usual-care participants (0.69) than for participants in the intervention group (0.68). After adjusting the mean difference for baseline utility, a marginally greater health-related quality of life gain (of 0.0003) was seen in the intervention group than in the usual-care group at 12 months. However, the difference in mean utility score was not statistically significant at any time point, whether unadjusted or adjusted for baseline utility (see Table 10).

In terms of the total QALYs gained by participants over the trial duration for all available cases (Table 11), no statistically significant differences were found between the groups, when adjusting for baseline utility only or when adjusting for baseline utility and all covariates. QALYs were found to be marginally lower for the intervention group (reduction of 0.0015 QALYs; p = 0.814) when adjusting for baseline utility and all covariates.

Falls

At the primary end point of 12 months, participants in the intervention group experienced additional falls compared with those in the usual-care group (IRR 1.17, 95% CI 0.99 to 1.38; p = 0.07), as summarised in Chapter 3. As the intervention was associated with a higher falls rate (i.e. no falls were averted) and an additional cost, it was not clinically meaningful to calculate the ICER in terms of the cost per fall averted.

Health-care resource use and costs

The falls-related health-care services utilised by participants during the study period are summarised in Table 12. The resources participants used most commonly in relation to falls were visits to the GP, nurse and physiotherapist, in addition to hospital outpatient visits and inpatient overnight stays. The most notable difference between the groups was seen for inpatient hospital stays; participants in the intervention group stayed for an average of 0.05 nights over the past 4 months, when asked in the 12-month questionnaire, whereas participants in the usual-care group stayed for an average of 0.28 nights. Further resource use due to ‘other’ reasons is provided in Appendix 11. A greater proportion of missing data occurred for resource use related to falls than for ‘other’ resource use (i.e. non-falls-related). This applied for each resource use item, that is, for GP visits, nurse visits and all remaining resource use items, as shown in Table 12 and Appendix 11.

The proportion who attended a falls service or clinic over the 12 months was slightly higher in the intervention group than in the usual-care group, at 6% and 5%, respectively. A higher proportion of participants in the intervention group (19%) received a (non-study) falls prevention visit over the 12 months of the study duration, than in the usual-care group (9%). A possible reason for this could be participants misunderstanding the question and instead recording their OTIS visit. Based on the possible confusion by participants, we explored the impact of excluding the non-study falls prevention visits from the analysis, as summarised in Cost–utility analysis and uncertainty.

Unit costs have been attached to the resource utilisation from Table 12 to indicate the cost differences for each resource. The largest differences can be seen in the costs of inpatient hospital stays and day-case hospital visits, with the mean costs being lower for both of these for participants in the intervention group. The cost of physiotherapist and OT visits was higher for the intervention group, with only small differences seen for the remaining items. However, it is worth pointing out the high proportion of missing data for the majority of these items. Table 13 indicates the total mean costs for each resource use item over the 12-month follow-up period for all available cases. The corresponding mean costs for the multiply imputed data set is provided in Table 14. Total out-of-pocket expenditure that features in the societal perspective analysis can be seen in Appendix 12.

Costing the occupational therapist intervention

A total of 382 participants received the home visit as part of the intervention; the visit duration was recorded for 374 of these participants, with an average duration applied for the remaining eight participants for whom duration was not recorded. The mean intervention cost was found to be £137 for the 382 participants who received the home visit, which comprised the costs outlined in Table 15.

The training cost per participant was estimated by dividing the total cost of training the OTs involved in the study by the number of participants who received the intervention. Of the 27 trained OTs, 23 conducted the 382 visits, seeing 16 or 17 participants on average. However, if the intervention were to be rolled out in practice, an OT is likely to see more people than this, and hence the cost of their training will be spread over more people.

The average equipment cost was relatively low; a high proportion of 71% (273/382) of participants did not receive any equipment/adaptations. Participants had a mean of 0.48 (SD 0.94) items installed, with this ranging from 0 to a maximum of 6 items installed per participant (and the median being 0). A breakdown of the mean number of the 31 possible equipment items installed is provided in Chapter 5 (see Table 22). In addition to these items, OTs had space to enter five ‘other’ items. An average cost for ‘other’ equipment was applied based on the five most commonly occurring equipment items (full details are in Appendix 4).

Cost–utility analysis and uncertainty

The OT intervention was associated with an additional cost for the base-case analysis, shown in Table 16, of approximately £19 per participant, on average. In terms of the effect of the intervention on health-related quality of life, only marginal differences in QALYs were demonstrated. The base-case analysis found 0.004 fewer QALYs for participants in the intervention group than for those in the usual-care group, on average, over the 12 months. It was not appropriate to report the base-case results in terms of the cost per QALY gained owing to the finding that usual care was the dominant option; that is, it was less expensive than and generated additional QALYs to the intervention, although this QALY gain was marginal.

The NMB at £30,000 per QALY was –£145.31 (95% CI –£149.90 to –£140.72), which also indicates that the intervention is not a cost-effective option when compared with usual care. The NMB at £20,000 per QALY of –£103.13 (95% CI –£106.73 to –£99.53) similarly shows that the intervention is not cost-effective when this lower threshold is considered.

Figure 9 illustrates the 10,000 bootstrap sample estimates, which are spread across the four quadrants of the cost-effectiveness plane, although the north-west and south-west quadrants are more heavily populated than the remaining two quadrants. The probability of the intervention being cost-effective at different willingness-to-pay thresholds is shown in Figure 10. At the lower NICE threshold of £20,000 per QALY, the probability of the intervention being cost-effective is 29.1%, and is similar at 26.9% for a threshold of £30,000 per QALY (under the base-case scenario).

FIGURE 9.

Cost-effectiveness plane for the base-case analysis: NHS perspective (MI adjusted for all covariates).

FIGURE 10.

Cost-effectiveness acceptability curve for the OT intervention relative to usual care, for the base-case analysis: NHS perspective (MI adjusted for all covariates).

The secondary analysis undertaken from the societal perspective, which also included indirect costs rather than just considering the costs borne by the NHS and PSS under the base case, found an incremental mean cost per participant of £73.15 (95% CI £68.41 to £77.89). A reduction in QALYs (–0.0025, 95% CI –0.0026 to –0.0023) was again demonstrated for the intervention group versus the usual-care group. The cost-effectiveness plane displaying the bootstrap sample estimates for the secondary analysis is provided in Appendix 13. The corresponding CEAC (see Appendix 13) depicts a 34% chance of the intervention being cost-effective at the £30,000 per QALY threshold.

Sensitivity analysis

The findings for each of the sensitivity analyses are shown in Table 16 in terms of the incremental mean costs, the incremental mean QALYs, the ICER and the probability of the intervention being cost-effective. For all sensitivity analyses, the QALYs remained relatively similar, with the exception of sensitivity analysis 1, which comprised the complete-case analysis. We therefore focus on the cost findings here. When only complete cases were included (sensitivity analysis 1), the intervention was found to be the dominant option compared with usual care. Additional QALYs were experienced by the intervention group, on average, although this QALY difference was found to be small (0.0076) and not statistically significant. This sensitivity analysis found a cost saving of £69 for the intervention group, in contrast to the additional costs demonstrated in the base case. However, the complete-case analysis comprised only 412 participants. The corresponding cost-effectiveness plane (Figure 11) demonstrates the QALY gains, with more of the point estimates being situated in the north-east and south-east quadrants. The CEAC for the complete-case analysis can be seen in Figure 12. The cost-effectiveness planes and CEACs for sensitivity analyses 2–5 are provided in Appendices 14 and 15, respectively.

FIGURE 11.

Cost-effectiveness plane for the complete-case analysis (NHS perspective, adjusted for all covariates).

FIGURE 12.

Cost-effectiveness acceptability curve for the complete-case analysis (NHS perspective, adjusted for all covariates).

In the base-case analysis, the intervention was dominated by usual care, as the intervention generated additional costs and fewer QALYs. The intervention remained dominated for both sensitivity analysis 3 (alternative source of inpatient stay data) and sensitivity analysis 4 (all equipment funded by NHS and PSS). Sensitivity analysis 3 explored the use of an alternative source of data for the number of nights spent in hospital due to falls. When the falls data sheets were used instead of the follow-up questionnaires to obtain data on falls-related nights spent in hospital, additional costs were found for the intervention group, which in turn led to an additional mean cost of £120 per participant. Sensitivity analysis 4 investigated the findings when all the equipment installed following the OTIS home visit was funded by the NHS and PSS. As expected, because the equipment component of the intervention cost increased, an overall increase in the mean cost of £40 was found. For the analysis exploring the inclusion of non-falls-related resource use (sensitivity analysis 2) and the analysis of paid care worker costs being included (sensitivity analysis 5), the intervention group experienced an average cost saving of £204 and £55, respectively. The corresponding ICER results were presented in terms of the cost per QALY lost, whereby QALYs are being lost but costs are being saved. The ICERs here indicate how much would need to be saved to justify a loss of one QALY.

The effect of removing the non-study falls prevention visits from the analysis, owing to participants’ potential misunderstanding of these, was also explored; the overall cost saving was £68.28 per participant (as opposed to an additional cost of £18.78 in the base case).

Training of providers

We used a standardised training package in order to maximise consistency of delivery of the intervention across all of the trial sites. The training materials included a presentation with detailed notes, a training manual and videos of older people undertaking activities of daily living, which the OTs assessed using the WeHSA. 28 The training was delivered by three modalities (face-to-face, ‘cascade’ and online training).

The initial training session for OTs was held face to face on 11 January 2016 (with eight attendees), led by Alison Pighills and observed by two other HCPC-registered OT co-applicants (Shelley Crossland and Avril Drummond). Three subsequent face-to-face training sessions were facilitated by Shelley Crossland and Avril Drummond (on 9 and 10 January 2017, with a total of 13 attendees) or Shelley Crossland alone (on 10 November 2017, with five attendees). Alison Pighills and Shelley Crossland each subsequently conducted a one-off training session for a single OT. All training sessions were audio-recorded. The audio-recordings of all but the one-off training delivered by Shelley Crossland were reviewed, and the training sessions were rated by Alison Pighills using the OTIS training workshop checklist (see Report Supplementary Material 14) for eight key elements that the trainer should have covered (prevalence of falls; evidence base underpinning environmental assessment and modification; falls risk factors; the person; environment; occupation conceptual model of practice and occupational performance; background on falls; environmental assessment; equipment and ideas for falls prevention; and adherence). These key elements were rated as N/A, not applicable; 0, omitted; 1, observed – some elements included; 2, observed – most elements included; or 3, observed – all elements included. Additionally, five sessions of cascade training were assessed by a single assessor (Alison Pighills) using the same checklist, as was the online training resource to ensure that it aligned with the face-to-face trial intervention training provided by the three facilitators. The scores for each of the key elements, and an overall mean score (calculated using the scores for each completed element, i.e. all for which N/A was not recorded), are summarised.

Delivery of treatment

To assess fidelity of treatment delivery, we used three strategies. First, one of the original training facilitators (SCr) directly observed the OTs delivering the intervention. Second, an audit of the WeHSA form and subsequent recommendations was conducted for all trial assessment visits to evaluate whether or not the delivery of the trial intervention was consistent with the training received. Finally, qualitative interviews were conducted with OTs who delivered the intervention and with the original training facilitators.

Facilitation strategies

For four of the five training sessions, a perfect score of 3 was awarded for all eight elements (resulting in an overall mean score of 3). For one session (the one-off session delivered to a single OT by AP), a perfect score of 3 was awarded for seven elements, with ‘Equipment and ideas for falls prevention’ awarded one (resulting in an overall mean score of 2.75). This was because the training was done via videoconference and the trainee had only a set amount of time available, so the information on equipment and ideas was provided to the OT to review in their own time.

Additionally, five sessions of cascade training were assessed by a single assessor. Scores for this training are provided in Table 18.

Alison Pighills completed the OTIS WeHSA online training checklist, and all domains were awarded a perfect score of 3.

Overall, the training was well received by OTs. The content was considered to be balanced between evidence and practice and OTs liked how the online training built on the content of that received during the face-to-face workshop, particularly as it provided a ‘refresher’ between the face-to-face workshop and the first OTIS home visit. OTs reported that the training prepared them well for delivering the intervention in the context of the trial. Although the majority of OTs felt that the training did not provide them with any new information about falls prevention education, this was not viewed negatively. While some OTs implemented the Home Falls and Accidents Screening Tool (Home Fast) in current routine practice, the WeHSA was a tool that the OTs had not used previously:

I would say that the training was incredibly comprehensive. It was very relevant. It was well facilitated and delivered. I wouldn’t have any kind of suggestions in terms of improving it. The resources that they provided were excellent. Yeah it was very good.

OT 06, 267–270

The training was really very thorough and the presenters were extremely approachable and although they were working from an academic point of view, we felt that they were really quite in touch with the real world as well.

OT 01, 98–101

Adherence/exposure

Occupational therapists’ adherence to the intervention delivery and dose (i.e. whether or not all elements of the intervention were delivered as per protocol) was assessed as described in Box 1.

A total of 382 participants allocated to the intervention group received an environmental assessment and modification visit from an OT (Table 19). The assessments were conducted by 23 OTs (median 16 visits per OT, range 1–54 visits). Nineteen of the OTs attended a face-to-face training session, and four were ‘cascade’ trained by another OT who had attended face-to-face training. For all visits a WeHSA form was completed. The visits took place between 1 day and 411 days after randomisation (median 27 days) and lasted a median of 90 minutes (range 25–180 minutes). Nearly two-thirds of the intervention group (277/430, 64.4%) had received the visit within 6 weeks of being randomised, and 381 out of 430 (88.6%) had received it within 12 months. One participant received the visit more than 12 months after they had been randomised, and 48 did not receive a visit.

It was not always possible to ascertain whether or not these 382 participants received a follow-up telephone call 4–6 weeks after their initial assessment. We know this call was conducted for 324 participants and definitely was not conducted for 28 participants (either because the participant could not be contacted even after several attempts or because it was agreed that no follow-up was necessary as no recommendations had been made at the initial visits). Therefore, we conservatively assume that 324 of the 430 (75.3%) participants randomised to the intervention group received all elements of the intervention.

The vast majority of OTs questioned the criteria used to recruit participants into the trial (having either one previous fall in the past 12 months or a fear of falling), as trial participants were often ‘poles apart’ (OT 11) in terms of functioning and demographics from patients/clients that the OTs would have on their ‘normal’ caseloads. OTs described such trial participants as active and ‘very able-bodied’ (OT 08, 62–63) with high standards of living in their home environments:

In my usual job I would probably see a little bit more of, you know, homes that were maybe not so well looked after or, you know, carpets that were a bit more worn and that sort of thing. I don’t know if it was just . . . the luck of the draw as it were but I did seem to get quite a lot of . . . well-kept houses, maybe more middle-class people.

OT 07, 249–254

Occupational therapists considered that some participants were participating in the trial only because they felt that they should ‘give something back to the community’ (OT 5) rather than because they felt that they would personally benefit from receiving a falls prevention service. A significant frustration for OTs was that some of their OTIS recipients had never experienced a fall or that the fall that had occurred was in an external environment and/or was an accidental ‘one-off’. This made it more difficult for OTs to deliver the intervention. Some participants questioned their own suitability for the trial as ‘quite a few of them felt that they were too good for the study and they were a bit embarrassed about having volunteered’ (OT 15, 277–280) and, as a result, some declined a home visit during the initial telephone contact with the OT:

A few people when I phoned them up and said, ‘is it OK if I come?’ They said, ‘oh no I think you’re wasting your time, I’ve never had a fall. I’m absolutely fine and don’t see the point of you coming to the house’. I would have thought that would have been made clear when they signed up to the trial that it’s all part of the process. Some people were quite defensive and didn’t want [me] going into their homes.

OT 14, 244–250

Some OTs chose to discontinue the home visit after learning the cause of the fall:

There was one lady I refused to do the assessment with . . . she had said on the phone ‘I don’t want you to waste your time, I’m not sure really I’m the sort of person you want to be speaking to’ and I said ‘well I’ve got the time I’ll come out and go through things’. She said it wasn’t a fall . . . She’d hit a pothole and caught her bike and had a fractured shoulder . . . So when I got there I thought this is totally inappropriate for me to do with somebody who’s fitter than most people . . . I actually thought I’m not going to do this because it’s not appropriate.

OT 5, 199–210

Active participants spent less time in the house, which made it more difficult for OTs to arrange the home visit and to contact them to complete the follow-up 4–6 weeks later:

I think I had a better response . . . from people that lived on their own, you know, maybe quite enjoy the company of like someone coming in and having a chat and taking a real interest in like their background and their sort of lifestyle and maybe a bit more vulnerable so quite keen to have help. The people that I found who weren’t so keen were some of the more sort of [more active], a couple of people I struggled to get appointment with because they play golf and yoga and gym, so they say ‘I’ve got an hour, you can come on this day’.

OT 14, 289–27

In addition, OTs participating in the trial found that delivering the OTIS intervention alongside their normal workload was sometimes problematic, with some OTs expressing the challenges they experienced in transitioning between their existing caseload and the OTIS trial. Despite having ‘allocated’ time to complete the OTIS intervention, some OTs were told by senior staff members to prioritise caseload work over OTIS. This was particularly the case for OTs whose casework regularly involved patients who required emergency/fast response or palliative care. Participation in the trial also presented tensions for some part-time OTs whose employers were explicit that they were to complete OTIS during non-working days only. As a result, there were instances where this tension affected the delivery of the OTIS intervention, as OTs were unable to deliver the follow-up telephone call to participants within the intended time scale of 4–6 weeks (post home visit):

In terms of follow-up, that was a big failing on my part I’ll be honest in that, particularly when I was transitioning between the two job roles and to be honest anyway when I was in palliative care service, because obviously we had to stick to the . . . the guidance was that we had to do a review in 6 weeks. I failed that quite significantly just through the pressure of my other caseload.

OT 12, 218–266

Quality of delivery

Findings from the observations of the nine OTs using the OTIS observational checklist are presented in Table 20, suggesting that OTs were delivering the intervention to a high standard. There was very little variation in scores across the domains, with a median score of 3 across all OTs observed for all 12 components.

Additionally, six OTs across six centres completed the OTIS observational checklist about a visit they had conducted (see Table 20). The visits took place between 22 November 2017 and 3 May 2018.

The documentation audit of the WeHSA booklets were assessed for all 382 visits (Table 21). On average, the overall mean score for the 12 domains assessed was 2.6 out of 3. The three most poorly completed domains appear to be (1) WeHSA used to assess functional capacity while participant doing tasks within the context of their environment; (2) participant engaged in devising possible solutions to minimise identified hazards; and (3) action plan left with participant, as it was not always possible to tell from study documentation if this had taken place.

However, as described above, there was a general sense of disengagement among OTs towards completing the intervention with participants whom they considered unsuitable for the trial, and this influenced how they delivered it. For example, with participants who reported that falls were one-offs or had happened in external environments, such as while hiking, OTs reported ‘going through the motions’ (OT 12) of competing the trial documentation rather than identifying falls hazards, as they did not perceive the participant to be at any risk. This also meant that the OTIS intervention was modified by OTs, as they skipped questions they thought were irrelevant or had already made assumptions about the participant’s home environment:

Most of the participants, probably 90% of the participants we saw were very active and to go and ask them in detail about everything on the list was too much. Yeah so I wouldn’t ask them about every question.

OT 15, 125–129

Sometimes it didn’t seem appropriate, you know, sometimes I went to see people and they had no mobility problems at all and it wasn’t that they refused to do it but I got the feeling that they didn’t want to do it. So you know I used my judgement there as to whether or not it was appropriate to complete.

OT 02, 179–184

Although OTIS was designed to be delivered by OTs, qualitative data suggest that wider team members (e.g. physiotherapists) may have been involved in intervention delivery at one site. However, all intervention booklets indicated that home assessments were completed by a trained OT, so we are unable to provide characteristics of other team members or an estimate of how frequently this may have occurred:

We picked up some [participants] as our cases and carried out the intervention as a team, so it wasn’t necessarily that an OT that did it [delivered the OTIS intervention], you know, it could have been a physio or someone else.

OT 17, 53–55

Intervention complexity

The OTIS intervention was considered a very detailed tool that provided a very thorough assessment of a participant’s home environment. None of the OTs reported that the forms missed any key features of an environmental falls prevention assessment. For some participants, the OTIS intervention was too complex and identified areas for change over which they (the OT) felt they had no control, such as the participant’s external environment (e.g. path, driveway, gardens):

It’s almost too comprehensive . . . areas which I can’t do anything about . . . I can’t do anything with a garden if it’s got a slope on it . . . so then why are you kind of assessing and recording it?

OT 10, 251–257

The reporting form itself was not considered to be user-friendly and this may have had an impact on the quality of delivery. OTs raised issues with the format of the form (e.g. not providing enough space for free text), and the ordering and repetitiveness of questions. These issues meant that the form was complicated to complete, especially while also trying to establish and maintain a conversation with the participant. This resulted in some OTs completing the form after the home visit and relying on notes and memory to do so, or missing out parts of the form that they believed to be irrelevant or unnecessary:

A lot of it was repetitive and you’re having a conversation with somebody trying to work out where to write things or where things go, what you need to know, some of that I would write up when I got back to the office. In the end I just didn’t follow it . . . I wouldn’t ask them [the participant] about every single bit because I just look at what was relevant.

OT 15, 242–244, 267–271

I think probably generally speaking I filled the paperwork out once I got home. I usually came out and made come quick notes about things so that I didn’t forget things and then came back and sit and trawl through it all and if I was stuck on a point I would just ring the patient back. But I think I only did this one or two times.

OT 11, 384–387

Occupational therapists reported that, owing to the complexity of the form, it took on average 1.5 hours to deliver (ranging from 25 minutes to 3 hours). A number of factors had an impact on delivery time, including the size of the participant’s house, their current health condition(s) and the number of falls they had previously experienced. Some OTs felt that they took more time completing the form because it was for research, and if the form were used in routine practice they would be more inclined to skip over parts or leave some parts incomplete. The quality of delivery of the OTIS intervention improved the more frequently OTs completed the form. OTs reported that they became more familiar with the form the more often they used it, which enabled them to complete the form on-site during the home visits, rather than at a later time, and also led to shorter delivery times. For a number of OTs, the time it took to deliver the OTIS intervention made them uncertain about the feasibility of delivering it during routine practice:

In my everyday role . . . I would not have time to go through the whole of that.

Community OT 15, 369–370

I think the reason it took longer than perhaps one of our normal falls assessment is that you’re just trying . . . because it’s a research project, you are trying to really make sure that you have covered all the elements for the form as well. Whereas in normal practice, a little bit like I said before, that I might make certain assumptions and miss bits because I can see it’s not really relevant.

OT 01, 398–403

Participant responsiveness

The number of recommendations made to intervention participants following the OT home visit ranged from 0 to 11 [mean 2.46 (SD 2.08), median 2 recommendations]. Following the follow-up telephone call, OTs were asked to judge ‘Taking into consideration all of the recommendations made and advice given, how many of these have been implemented?’. Responses were as follows: most (49.5%), some (34.8%) and none (15.7%).

Table 22 presents data on items of equipment that were recommended following the home visit, whether or not these were installed and whether or not the installation was carried out by either health services or social services. We have calculated the proportion of participants who followed the recommendation for each item of equipment and also the proportion of participants who had the item installed by health or social care services.

The mean (SD) number of items installed that were recommended per participant was 0.47 (0.92) (minimum 0, maximum 6; median 0). The most commonly recommended equipment was a non-slip mat (161/382, 42%); 58% of participants who were recommended this implemented it, 26 of whom were supplied it by formal services. Installation of recommended items varied from 0% to 89%; however, the numbers of some items recommended were very small. It was clear from the data that some items were more readily available from health and social care than others, for example the majority of grab rails were provided to participants. It is not clear from the data available the extent of local variation in the provision of equipment by formal services. However, even when equipment was available from formal services, uptake was not universal. It is also important to note that even in the absence of formal provision of recommended items/alterations, some participants were motivated to follow recommendations independently, indicating that support from services was not always essential for equipment to be purchased and fitted for those who had the resources to do so. Overall adherence to the recommendations was relatively low; on average, participants installed 17% of the items that they were recommended.

Occupational therapists reported that the majority of participants were accommodating during the OTIS visits and willingly participated in completing the activities that OTs requested they do around the home to enable the assessment to be completed. However, there were a couple of instances during home visits where participants considered the level of questioning to be intrusive, which prevented OTs from delivering the intervention as intended:

I phone up [the participant] and she was aware that I was coming. But when I got there she was quite defensive . . . she lived in the really, really big house and you could see that there was a lot of things everywhere. I think she might have been a bit of a hoarder. So when I explained for the purpose of my visit and, you know to ask about the environmental, she said I don’t mind you asking me the questions but I don’t want you to look around my house.

OT 14, 192–198

Completing the full environmental assessment enabled OTs to give participants informed recommendations about how to improve their home environments to minimise their risk of future falls, for example by installing of a piece of equipment. Participants who responded well to this advice completed the suggested modifications to their environment and/or purchased the equipment. OTs considered responsive participants to be older and less active and have lower functioning:

The ones who were really struggling with their mobility and things they were really grateful of the advice and so yeah they definitely took it and were happy to go ahead with any equipment and recommendations like referrals to social services and things.

OT 07, 161–166

Where recommendations had not been actioned, this was sometimes because the participants needed assistance to make the modifications or because they did not consider themselves at risk of falling and/or did not highly prioritise falls prevention. OTs reported the latter group of participants to be less receptive to their advice, and described them as being younger and more active and high-functioning (i.e. more able to undertake activities of daily living within their environment):

I would say probably the more high-functioning patients were the ones who weren’t quite ready to accept the sort of equipment side of things. So I was telling them maybe about the rails that they could get or bits of equipment maybe to help them around the house and they’d be like, ‘oh yes I’ll bear that in mind but I don’t think I’m quite ready for that yet’.

OT 07, 161–166

The well-functioning sort of very middle-class, yeah, reasonably well-off sort of groups of people that were definitely involved and they then struggled to make that transition into seeing themselves on being the receiving end of a treatment.

OT 5, 256–259

Although ‘high-functioning’ participants were less receptive to acting on the OTs’ recommendations, the home visit presented OTs with the opportunity to provide participants with advice and signpost them to useful services they may need in the future. OTs reported that this information was positively received by many high-functioning participants.

Occupational therapists felt that the functioning and the demographic characteristics of some trial participants prevented the uptake of advice when OTs identified areas of modification, as the participants did not deem themselves to be at risk of falls:

[Some] people just saw it for what it was, just an assessment of them and I think some of that is about the cohort of people I felt we were seeing . . . I think sometimes why people were reluctant to accept the advice because they didn’t see it as something relevant to them because they were doing this for the good of others and yet actually when the advice was being passed to them, they weren’t able to make that link . . . At that point they didn’t see themselves as . . . requiring the feedback.

OT 5, 228–236

For some participants (those with greatest need), support from health and social services in implementing recommendations will most likely have influenced their adherence to some recommendations. However, the qualitative data do suggest that for a large proportion of the participants the most likely reason for the relatively low uptake of many of the OT recommendations was that they did not feel that they would benefit from these suggested changes.

Programme differentiation

Occupational therapists confirmed that an environmental falls assessment was implemented as part of their routine practice when conducting falls-related home visits. Some reported using the Home Fast to carry out the assessment. Overall, there was no tool that was standardised between trusts. Many OTs spoke of how the tools they used had been developed ‘in house’ and how the delivery time was relatively short because environmental checks formed only a small part of a broad, multifactorial assessment [e.g. including checks on medication(s), osteoporosis, alcohol, footwear, memory]; this theme was particularly apparent among OTs whose roles were more community service based as opposed to those whose roles were falls-specific. The fact that the OTIS intervention was not a holistic assessment was considered a disadvantage by some OTs:

In our normal practice we would also be looking a little more at the psychological elements of patients, so how their mood is, perhaps their memory is, what sort of participation level they have in these activities . . . are they actually engaged in functional tasks? So that element isn’t really captured in OTIS.

OT 01, 326–333

Some OTs also felt that the key areas of the OTIS intervention were not too dissimilar to those of their routine environmental assessments. However, most agreed that the OTIS intervention was far more comprehensive, and this was positively received, particularly by OTs who had fewer years of experience and/or among those who did not specialise in falls prevention:

It [the OTIS intervention] was really comprehension . . . our assessment is nowhere near as in-depth as that, so it was really good. It was a great prompt for certain things . . . I liked that it made you really think about every little detail that maybe you might miss sometimes.

OT 14

The multifactorial assessment doesn’t have a particularly brilliant environmental assessment . . . so this was an added extra which they [OTs] liked.

OT 17, 115–117

Occupational therapists who were highly experienced in falls management did not believe that their practice would change as a result of using the OTIS intervention, as they felt that the tools they already used in their routine practice, coupled with their experience, were just as effective. Some experienced OTs suggested the WeHSA may be a useful tool to use in certain circumstances, but not during every falls-related home visit. However, they recognised that the comprehensive nature of the OTIS intervention would be of particular benefit to OTs who did not work specifically in falls, OTs who were newly qualified or those who were training to be OTs:

I would see it as a great tool for say newly qualified OTs who are not familiar with doing this particular, you know, type of assessment, so it gives them more structure . . . but [for more experienced OTs] I think a tool in a toolbox.

OT 17, 294–295

I can’t say there’s been anything else that it’s changed my practice as such. I guess from just so many years from doing similar or elements of it, you know, I think it helps to have, as an OT, to have the experience of doing falls assessments and environmental assessment . . . so for another OT it might actually give them other ‘oh I hadn’t really thought about that before’, you know, in the detail.

OT 01, 366–376

A number of OTs were willing to incorporate the OTIS intervention into routine practice should the trial prove it to be effective. However, the main concerns were in relation to (1) the time the intervention took to deliver, which was twice as long as routine appointments, and (2) the cost to the trust purchasing it:

Cost is always something that the trust considers and if it’s an expensive tool the trust will quite often look for a cheaper alternative.

OT 14, 165–167

A number of OTs described how they had begun incorporating some elements of the OTIS intervention into their routine practice, particularly where they had learned something new. The visual and sensory elements of the OTIS intervention were particularly praised:

From using the Westmead I’ve . . . started veering off . . . from my original form . . . and thinking more about the Westmead to ask more questions that are in that I wouldn’t normally ask because I’ve found it really useful for kind of identifying different information.

OT 14, 109–113

I’ve become a lot more aware of generally sensory alterations, obviously can come within ageing. So I think I’ve incorporated a lot more of that into my current practice . . . I’m a lot better at looking at some of the elements of people’s environments to do with lighting and flooring than I probably used to be.

OT 5, 347–354

Context

It is clear from the data collected that the participating OTs were committed to delivering the OTIS intervention thoroughly and to as high a standard as possible. All of the OTs who participated in the interviews had volunteered to take part in OTIS. OTs saw the trial as an opportunity for professional development and were keen to learn something new that they could apply to their current practice to improve patient experience. Many OTs were keen to be involved in research that was different from their daily job role, acknowledging both that research on the topic of falls prevention was very limited and the gap in evidence that OTIS was aiming to address:

I was interested just to see if there was anything really that I could learn that I could then apply to my practice. But also I was just really interested to get involved in some research and to try something new and different. Certainly when I spoke to [the developer of the OTIS intervention] . . . she was just so passionate about this falls intervention and she really kind of sparked my interest and really made me keen to be involved.

OT 07, 22–29

However, as the data suggest, despite this commitment, the most significant barrier to delivering the intervention was the relatively high well-being of the trial participants. In the context of sustained national government austerity measures across England and Wales, OTs observed that the participants they were being asked to provide the intervention to were ‘poles apart’ (OT 11) in terms of functioning and demographics from patients/clients they would see on their ‘normal’ caseloads. In current clinical practice, OTs see only the more severe cases who, as well as being in poorer health, are also less likely to have other resources (e.g. financial) to compensate for any lack in functional impairment. This difference in the sociodemographic profiles of people who are generally in receipt of OT services and those of the trial participants may have been exacerbated by the nature of recruitment strategy for the trial; responding to a mail-shot favours those with lesser morbidity and deprivation:77

I did seem to get quite a lot of . . . well-kept houses, maybe more middle-class people.

OT 07, 249–254

Trial population

This large, multicentre modified cRCT comprised 1331 community-dwelling men and women aged ≥ 65 years: 430 were randomised to the intervention group and 901 were randomised to the usual-care group. We exceeded our original sample size target of 1299 by 32 participants. Participants were recruited primarily from mail-outs from general practice surgeries, from the Yorkshire Health Study, or from YTU cohorts of participants from previous trials in similar populations who had agreed to be contacted about future research projects. Recruitment was more successful (based on the conversion rate from mail-out to randomisation) from the YTU previous trial cohorts than from the general practice surgeries or Yorkshire Health Study, that is, it was more successful among people with prior experience of, and engagement in, similar research.

Two-thirds of OTIS participants were women, and participants’ average age was 80 years. Three-quarters of participants had reported a fall within the past 12 months at baseline, and one-fifth of these had attended hospital because of a fall. However, in the qualitative interviews, several of the OTs commented that they considered several of the intervention participants to be less frail and more active than the population of patients who would be referred in clinical practice.

Primary outcome

In total, 2260 falls were reported. The average number of falls per participant was slightly higher in the intervention group (1.9, range 0–94 falls) than in the usual-care group (1.6, range 0–41 falls). The two groups provided data for a similar amount of time on average (338 days in the intervention group and 345 days in the usual-care group). Details of each reported fall were sought over the telephone, and at least some information was obtained for about 90% of the falls. Just over half the falls occurred indoors (53%), with the majority (86%) of these occurring inside the participant’s own home rather than on another premises. Superficial injuries or worse were sustained in about half of the falls, with 3% of the falls resulting in a broken bone, most predominantly in the wrist or hand. There were nine reported hip fractures. This fracture rate is a lot lower than that reported in the SCOOP trial,21 where 16.0% of participants reported a clinical fracture and 3.5% reported a hip fracture. However, the population, although recruited from GP surgeries and of a similar age (70–85 years), comprised women who were not taking an antiosteoporotic drug.

We found that the environmental assessment did not lead to a reduction in falls over 12 months of follow-up. In fact, we found weak evidence of a difference in falls, with the intervention group reporting about 17% more falls than the usual-care group (p = 0.07). History of falling was seen to be significantly associated with a threefold increase in subsequent falls.

When compliance with the intervention was accounted for using an instrumental variable CACE analysis approach, the estimate of the intervention effect was very similar to the ITT analysis, likely to be because the vast majority of intervention participants (nearly 90%) received an assessment. The findings were robust to other sensitivity analyses exploring missing data and therapist effects.

There are some possible contributing factors to the increased reporting of falls in the intervention group, relative to the usual-care group. First, following their home assessment visits, participants in the intervention group may have been more likely to be mindful of falls, to consider an event as a fall and to recall and report a fall on their falls calendars. Second, there may be some risk compensation, whereby the intervention participants experienced a heightened sense of confidence and immunity from falls after the OT visit and so, perhaps counterintuitively, increased their risk-taking behaviour and were actually more likely to fall. These are reasonable theories and may have played a role; however, we believe it is likely that the weak evidence of a difference in falls rate is a chance finding and that the intervention neither significantly increases nor decreases the chance of falls. This is supported by the sensitivity analysis in which Parkinson’s disease is adjusted for, a known predictor of falls. We observed a chance imbalance in the proportion of participants with Parkinson’s disease in the two groups, with participants in the intervention group more likely to have Parkinson’s disease than those in the usual-care group. Although only 23 participants in the randomised population had Parkinson’s disease, adjusting for this reduced the difference in falls rate between the groups so there was little evidence of a difference (p = 0.23).

The intervention may be more effective in a ‘higher-risk’ population. In hypothesis-generating subgroup analyses, a qualitative difference was observed in the treatment effect estimate within those participants who reported having had hospital treatment due to a fall in the 4 months prior to enrolling in the trial compared with those who had not. In the former subgroup, considered at ‘higher risk’ of falls, the intervention participants reported 14% fewer falls than usual-care participants. However, the trial was not powered to detect subgroup effects and the interaction was not statistically significant.

Secondary outcomes

There were no differences in any of the secondary outcomes. Just over half (56%) of all participants reported at least one fall and one-third reported more than one fall. At 12 months, approximately half of the trial participants, in both groups, reported that they worried about falling at least some of the time.

Sample size

Our original sample size calculation was based on a binary outcome rather than a count (number of falls). We have considered, post hoc, a sample size calculation based on negative binomial regression. In REFORM, our earlier trial, we observed a mean predicted falls rate, over 12 months, of 1.66 in the usual-care group and 1.45 in the intervention group (IRR 0.88; p = 0.16). The dispersion parameter was estimated at 1.34. In OTIS, we observed a mean predicted falls rate, over 12 months, of 1.76 in the usual-care group and 2.05 in the intervention group (IRR 1.17; p = 0.07). The dispersion parameter was estimated at 1.29. We estimate that, assuming a falls rate in the usual-care group of 1.7 and a dispersion parameter of 1.3, with 1331 participants (2 : 1 allocation) we had 90% power to detect a 23% decrease in falls (two-sided 5% significance). 79 Ultimately, given the parameters observed in OTIS, we estimate that we were powered at about 70% to detect the 17% increase in falls we detected.