Individual health trainers to support health and well-being for people under community supervision in the criminal justice system: the STRENGTHEN pilot RCT

Aims and objectives of the pilot trial

The aim of this pilot randomised controlled trial (RCT) was to develop and implement a health trainer-led intervention to support health and well-being improvements for those under community supervision in the CJS. Furthermore, the pilot study sought to explore uncertainties about the acceptability and feasibility of the trial methods and intervention to inform the design of a full RCT.

The specific objectives were to:

1. assess the acceptability and feasibility of the STRENGTHEN intervention, alongside routine engagement with community supervision services, for the key stakeholders including participants receiving community supervision, CRCs, the NPS and health trainers themselves

2. assess the acceptability of recruitment, randomisation and assessment procedures in a pilot pragmatic RCT

3. determine, from the pilot RCT, completion rates for proposed outcome measurements to assess well-being (i.e. the WEMWBS), behavioural measures (e.g. self-reported alcohol consumption, smoking status, diet, physical activity, substance use) and quality of life [measured using the Short Form questionnaire-36 items (SF-36) and EuroQol-5 Dimensions, five-level version (EQ-5D-5L)] at baseline and at the 3- and 6-month follow-ups

4. provide data to contribute to sample size calculations for a fully powered RCT to primarily assess subjective well-being (measured using the WEMWBS) and to ensure that the effect size (intervention vs. usual care) chosen for powering the definitive trial is plausible

5. use a mixed-methods process evaluation to further refine and understand the acceptability and feasibility of the intervention, its delivery and the trial procedures

6. estimate the resource use and costs associated with delivery of the intervention, and to pilot methods for the cost-effectiveness framework in a full trial.

Incorporating the Five Ways to Well-being

The framework chosen for promoting mental well-being was the Five Ways to Well-being (5WWB) [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/phr/145419/#/ (accessed 30 August 2019)]. The 5WWB were developed as an accessible public health message based on evidence-based practices people can perform easily every day, which could lead to improvements in mental well-being. 23 PPI work supported the 5WWB as being an acceptable and useful framework that could be applied to the target population.

To incorporate the 5WWB, members of the research team took part in a 1-hour training session during which they were trained to understand and focus on their own well-being to ensure familiarity and understanding of the framework. Following this, the 5WWB were incorporated into the training manual as a stand-alone section for supporting people who want to improve their well-being. A section was also developed that embedded ways to promote the four health behaviours of the original health trainer manual in ways that would maximise their impact on well-being. For example, supporting a reduction in alcohol use could also link to exploring how this might help a client to connect with others (who may be trying to do likewise), learn about the physical and mental health consequences of alcohol use and guidance on safer levels of use, discover how physical activity can help deal with alcohol cravings, notice the effects of alcohol on financial, social, emotional and cognitive functioning, and give support to others to manage their alcohol consumption. A similar set of examples can be developed for each of the health behaviours.

Adapting the health trainer role and intervention

Content from the original health trainer manual that was considered appropriate was adapted for the STRENGTHEN intervention; the central ethos of being client-centred and embedded in the community was carried forward into the STRENGTHEN intervention, as were components such as action-planning, problem-solving, self-monitoring and signposting. The intervention included:

• a heavy focus on engagement, trust/rapport building

• a focus on reduction rather than stopping smoking, or pushing guidelines (five a day, 14 units, etc.), as this would be seen as threatening

• flexibility of timing, frequency and duration.

The core competencies

As with our previous work adapting the health trainer role,35 a set of six core competencies were developed, which were designed to underpin the work of the health trainer (see Appendix 1). They reflected elements that were considered to be crucial to successful delivery of the intervention, and were reinforced throughout the manual, health trainer training and the supervision process. They were (1) active participant involvement, (2) motivation-building for changing a behaviour and improving well-being, (3) set goals and discuss strategies to make changes, (4) review efforts to make changes/problem-solving, (5) integration of concepts: building an association between well-being and behaviour and (6) engage social support and manage social influence. These competencies not only served as a guide for what the health trainers should be mindful of in their delivery, but also for assessing intervention delivery fidelity (as discussed in Chapter 6).

Patient and public involvement and stakeholder input to intervention development

In order to ensure that the intervention was acceptable and tailored to the needs of the target population, intervention development work was undertaken in the form of the establishment of PPI groups and a stakeholder consultation. A summary of the findings that were used to shape the intervention manual and health trainer training is provided in the following sections.

The STRENGTHEN peer researcher input to intervention development

The research team have established a collaborative relationship with a local day service for substance use and alcohol rehabilitation that supports people with multiple and complex needs. The service had recently collaborated in extensive PPI activities for a trial of an intervention to support prison-leavers with common mental health problems to achieve their goals (ENGAGER 2). 36 The research team attended the regular Monday morning group session in order to provide service users with information about the STRENGTHEN pilot trial and invite them to an introductory session to help them decide if they would like to be involved in advising about the development of both the intervention and the trial. It was at this stage that service users advised the research team that, due to the potentially sensitive nature of the topic, there should be separate groups for men and women. It was also seen as beneficial to the development of the intervention, as potential gender-specific aspects of content, implementation and delivery could be teased out in order to maximise acceptability to both women and men.

Group members were keen to adopt the title of ‘peer researcher’ that was used in the ENGAGER 2 PPI groups. 37 This both helped them to define their role in the project and put them on an equal footing in the team, with their expertise being their lived experience and understanding of the context in which the intervention would be delivered. The groups met on a bi-weekly basis for 4 months (with two or three missed sessions to take account of school holidays as a result of parenting responsibilities of some group members). Although there was some fluctuation in the attendance of both the men’s and women’s groups, a ‘core group’ of attendees emerged who attended the majority of peer researcher meetings (approximately five in the men’s group and six in the women’s group). This continuity allowed peer researchers to follow the development of the study and to witness how the outcomes of the previous meeting were implemented. Each meeting was 2 hours long, with a 15-minute mid-point break, and was facilitated by two members of the research team. Each meeting followed a schedule of activities to address issues regarding the design of the intervention and/or the research, with flexibility to discuss other topics that peer researchers raised as relevant to the intervention/research. The start of each group involved a catch-up on progress with the pilot trial and, as the groups progressed, how the advice that the group had provided during the previous meeting had been used and implemented. It was clear that these updates of how the work of the group had been utilised were key to maintaining engagement by showing the changes to and progress with the study to which the peer researchers contributed.

The PPI groups contributed to the intervention in terms of its conceptualisation, content and practicalities of delivery. Each of these will be dealt with in turn, with reference to the contribution and changes made by the groups and illustrative quotes from group meetings, when appropriate.

Stakeholder input to intervention development

Lynne Callaghan interviewed eight stakeholders from a range of related health trainers and CJSs in order to identify any changes/adaptations that needed to be made to the intervention to meet the needs of the population and deliver the intervention in the current context. Roles included management and delivery of a similar health trainer intervention in probation services; practitioners with a remit of providing well-being and housing services, with a particular focus on women; a court advice and support service; a community-based support worker working alongside custody liaison and diversion workers; and a signposting and support service that worked in collaboration with the CRC and other key services. Most of these services were located in the south-west of England, with two participants located in the south central region. As the second site had not yet been identified and secured at this stage, it was not possible to include services from this area. Interviews had a focus on understanding the facilitators of and barriers to working with men and women in the CJS, in particular those under community supervision; the experiences of supporting clients to change health behaviours and mental well-being; the process of goal-setting used with clients; the mode and frequency of contact and how they worked with clients to support initial and ongoing engagement; and what works well and what does not work so well in supporting clients to change their health behaviours and improve mental well-being.

Interviews were transcribed verbatim and analysed using thematic analysis. A summary of emergent themes is included in the following sections.

Eligibility criteria

Participants had to satisfy the following criteria to be enrolled in the trial:

• male or female and aged ≥ 18 years

• currently receiving community supervision

• having a minimum of 7 months left of community sentence/supervision

• having been in the community for at least 2 months following any custodial sentence

• willing and able to receive support to improve one or more of the four target health behaviours and/or mental well-being

• willing and able to take part in a pilot randomised controlled trial with follow-up assessments at 3 and 6 months

• residing in the geographical areas of the study.

Exclusion criteria were as follows:

• presenting a serious risk of harm to the researchers or health trainers

• unable to provide informed consent

• having disrupted/chaotic lifestyles that may have made engagement in the intervention too difficult.

Sample size

A recruitment target of 120 participants was set across the two geographical regions (with the aim of recruiting 60 participants per region). Following consultation with the Trial Steering Committee, the decision was made for a 60 : 40 men-to-women purposive sample to inform understanding of the experience of women in the CJS. Women make up a smaller proportion of those under community supervision, and represent a small number of the total prison population. 41 However, the aim was to over-recruit for women in an attempt to avoid losing that understanding of experience through proportional sampling.

This pilot trial was not powered to detect between-group clinically meaningful differences in the proposed primary outcome. Therefore, the target sample size was primarily set to assess the feasibility objectives of the trial and to inform sample size calculations for a planned definitive trial. When data from a pilot trial are required to estimate the SD of a continuous outcome, to maximise efficiency in terms of the total sample size across pilot and main trials, the recommendation is that a two-group pilot trial should have follow-up data from at least 70 participants (i.e. 35 per group). 42 As most participants would remain engaged with the probation service for the length of the trial, it was anticipated that retention would be reasonably high. A recruitment target of 120 participants, based on an assumed non-differential retention rate of 75% at 6 months, in an aim to obtain follow-up outcome data on a minimum of 45 participants in each of the allocated groups, across both regions. A retention rate of 60% would still provide sufficient data for planning the future trial. 42 Local services suggested that, over a 3-month window, there may be 20–30 ex-offenders entering each of the two local community supervision systems per week. It was estimated that around 10% would decline to participate in a baseline assessment11,17,43 and a further 20% would be found to be ineligible following the baseline assessment. Based on recruitment rates from other probation trials,11 it was estimated that around 50% of eligible subjects would consent to participate.

Recruitment

Recruitment for the trial was over a 14-month period between October 2016 and December 2017; initially, the planned recruitment period was a 3-month period from October to December 2016. This is discussed in more depth in Chapter 4. There were two pathways to participant recruitment: (1) via the CRCs and NPS and (2) via community organisations, including drug and alcohol rehabilitation centres, homeless hostels and day centres (in the Plymouth site only) (Figure 1). Recruitment via community organisations was introduced as an attempt to reach those not engaging regularly with the CRC or NPS services.

FIGURE 1.

Participant pathway to recruitment. CRF, case report form.

Initially, a single point of access administrator was identified for both the CRC and NPS. The single point of access administrator identified potential participants using the nDelius record system (Beaumont Colson Ltd, Newcastle upon Tyne, UK) for both services. The offender managers of identified individuals were then consulted by the researchers for screening for inclusion/exclusion criteria and assessment of risk. Further into the trial, a decision was made to alter this process, with researchers helping the offender managers to screen caseloads (i.e. sitting alongside them, but without visibility of personal information) to maximise efficiency and reduce overall staff demand [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/phr/145419/#/ (accessed 30 August 2019) for documents related to the screening process].

Those individuals who were assessed as eligible for participation in the research were initially approached by their offender manager, who explained the trial and asked if clients would be interested in speaking to the researcher directly after their appointment (if researchers were available), at their next scheduled appointment or via the telephone. On receiving verbal agreement to approach the client, the offender manager facilitated this meeting, providing an introduction. All participants were given the opportunity to meet the researchers for the initial appointment at the CRC/NPS offices.

Randomisation and concealment

Allocation to the intervention or control group was 1 : 1 and used a minimisation algorithm with a random element, to ensure balance between allocated groups with respect to age, gender and recruitment region.

On completion of the screening interview and baseline data collection, the researcher entered the participant details into a password-protected web-based randomisation system set up and managed by the Peninsula Clinical Trials Unit (PenCTU), and confirmed that the participant had completed the baseline case report form (CRF). The participant would then be allocated a unique randomisation number, and the participant’s allocated group (STRENGTHEN intervention or control) was then sent to the trial administrator via e-mail. To maintain blinding of the research assistants (RAs), the website would confirm that the allocation process had been successful, but would not display the participant’s allocated group. Health trainers would contact participants (via telephone) who had been allocated to the intervention arm of the trial and arrange an initial date/time to meet. Participants allocated to the control group were contacted by telephone or in person at CRC/NPS offices by either a health trainer or a research administrator, to maintain RA blinding. The conversation included a discussion of the randomisation process, to ensure that the participant had understood which group they had been allocated to.

Blinding of the researchers was tested for feasibility, to see whether it would be possible in a definitive trial. Researchers were asked to record instances when they believed they had been unblinded in the baseline, 3- and 6-month CRFs. We recognised from pilot work in our ENGAGER37 trial that concealment of trial arm may be very difficult because the RA mostly conducted the follow-up assessments in the offender management service, and offender managers or participants themselves may mention their involvement in the intervention in passing.

Data collection

Proposed outcome measures were collected at baseline (at or shortly following recruitment) and 3 and 6 months post baseline. Six months is the proposed primary assessment point for the future definitive trial [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/phr/145419/#/ (accessed 30 August 2019) for documents related to data collection].

Feasibility and acceptability questions

A key aim of this trial was to collect data on the following acceptability and feasibility outcomes:

• proportion of eligible participants

• recruitment rates

• rates of attrition and loss to follow-up

• completion and completeness of data collection

• estimates of the distribution of outcome measures

• acceptability of intervention to participants

• acceptability of trial procedures (e.g. blinding, randomisation) to participants.

Proposed primary and secondary outcomes

The proposed primary outcome for the definitive trial was the WEMWBS, to measure subjective mental well-being, which has good psychometric properties. 27,44 The short Warwick–Edinburgh Mental Well-being Scale (SWEMWBS) was subsequently calculated for the purposes of possible future interest. 44

Secondary outcomes were:

• self-reported smoking (number of cigarettes smoked per day)

• Fagerström Test for Nicotine Dependence

• alcohol use (measured using the Alcohol Use Disorders Identification Test)

• diet (measured using the Dietary Instrument for Nutrition Education)

• physical activity (measured using the 7-day Physical Activity Recall questionnaire)

• substance use (measured using the Treatment Outcomes Profile)

• confidence, importance (i.e. an individual’s perception of the importance of changing the target behaviour), access to social support, action-planning and self-monitoring measures relating to health behaviours

• health-related quality-of-life [measured using the EQ-5D-5L and Short Form questionnaire-6 Dimensions (SF-6D) (derived from the SF-36)]

• intervention costs (related to health trainer time, training, supervision, travel, consumables)

• health care use, social care use and other resource use data were collected using a participant self-report resource use questionnaire.

The secondary outcome measures were selected as they link on to the four health behaviours and were rated as acceptable to participants during the PPI consultation stage.

Summary of process evaluation methods

The aims of the process evaluation were to:

• assess whether or not the intervention was delivered as per the manual and training

• ascertain components of the intervention that are critical to delivery

• explore reasons for divergence from delivery of the intervention as manualised

• understand when context is moderating delivery

• understand the experience and motivation of participants in the control arm of the pilot trial to maximise retention in a full trial

• explore reasons for declining to participate in the trial

• explore reasons for disengaging in the intervention before an agreed end

• understand, from a participant perspective, the benefits and disadvantages of taking part in the intervention.

Data collection

Analysis

Intervention fidelity was assessed through the scoring of audio-recordings of health trainer sessions against a developed list of key intervention processes, or the six core competencies detailed in Chapter 1 [(1) active participant involvement, (2) motivation-building, (3) goal-setting, (4) reviewing efforts to make changes and problem-solving, (5) integration of concepts and (6) engaging social support and managing social influences]. These were scored on two domains: (1) practitioner adherence to the core competencies outlined in the intervention manual and (2) competence of delivery. Recordings were scored independently by two researchers.

Quantitative data were summarised descriptively, with confidence intervals (CIs) as appropriate.

Any factors that were identified as possibly contributing to participants’ intervention engagement, and trial recruitment and retention will be explored in more detail in Chapter 6. All data were organised using NVivo version 11 (QSR International, Warrington, UK). 45 Data related to feasibility and acceptability of trial method and the intervention were analysed using thematic analysis. Interview data and session notes were synthesised into a framework analysis grid to understand the experience of participants in receiving the intervention in order to understand how the intervention works in practice and the components of the intervention that are critical to delivery. This allowed the feasibility and acceptability of the intervention, the intervention delivery and the research data collection to be assessed. Any procedures that needed to be adapted were identified and potential improvements and solutions were suggested.

Statistical analysis

A detailed statistical analysis plan was written by the trial statisticians and approved by the chairperson/independent statistician on the Trial Steering Committee (statistical analysis plan version 1.0, dated 8 June 2018) prior to trial database lock.

Cost-effectiveness and data collection

The pilot trial aimed to estimate the resource use and costs associated with the delivery of the intervention, and develop a framework for estimating the cost-effectiveness of the STRENGTHEN intervention plus usual care, versus usual care alone, in a future economic evaluation alongside a fully powered RCT. We aimed to develop and test economic evaluation methods for the collection of resource use data, for estimating related costs, and also the collection of outcome data appropriate for economic evaluation. Full details of the methods used are presented in Chapter 5.

Patient and public input to trial methods

The peer researcher groups introduced in Chapter 2 reviewed and discussed trial methods, including the following:

• participant information sheets and consent forms

• semistructured interview schedules

• CRFs and associated data collection.

Both peer researcher groups reviewed the information sheets and consent forms for the pilot trial and the process evaluation. The first drafts reviewed were adapted from ENGAGER 2 information sheets and consent forms that themselves had been reviewed by the ENGAGER 2 peer researchers. Peer researchers provided guidance to adapt wording and order of text to aid comprehension and to ensure that the main points of the trial and requirements for participation were clear.

The first draft of the CRF was given to each of the peer researchers (literacy levels had already been assessed by researchers). Over 3 weeks, peer and trial researchers role-played each of the measures in pairs or threes, as appropriate. It was made clear that peer researchers did not need to answer honestly but could play a role in order that they did not have to disclose any confidential or sensitive information. Peer researchers discussed the measures in turn and/or annotated drafts and gave them to the trial researchers. Each of the issues raised by the peer researchers in relation to the CRF are detailed in the following list, with reference to the changes made, when applicable:

• Removal of the Patient Health Questionnaire-9 items (PHQ-9)

  The original STRENGTHEN CRF contained the PHQ-9 (a measure of depression). Peer researchers questioned the use of the PHQ-9 for a trial in which the primary outcome was mental well-being, as opposed to mental illness. They were particularly concerned about the impact that some of the items may have on participants who may be experiencing challenging life circumstances, and potentially have mental health needs, when it was not in a therapeutic context. As one of the peer researchers stated, ‘if someone was just starting a recovery journey, they may not be in a stable headspace and question 6 could be triggering.’ The trial researchers took this issue back to the project team, which supported the PHQ-9’s removal from the CRF as the collection of this outcome was not critical to the aims of the pilot trial.

• Simplification of response format

  There was some concern among peer researchers that some of the formats by which participants could respond to items were complex and in some instances provided too nuanced a set of choices. It was explained to the peer researchers that, in the case of validated measures, the responses could not be amended. However, for the measures included to capture confidence, control and connectedness in relation to changing each of the target health behaviours, where there was originally a 9-point scale, this was amended to a 7-point scale on the advice of the peer researchers. Furthermore, also on the advice of the peer researchers, to assist participants in making the appropriate response to the items contained in the validated measures, the researchers produced laminated A4 ‘answer cards’ that contained the options required for each item.

• Understanding of item choice

  Peer researchers noted that some of the items could be perceived as sensitive by participants and that it was not always clear as to the rationale for including all of the measures. Therefore, it was agreed that the research team would add a short paragraph or script for researchers at the start of each measure to explain why they were asking the items contained in each measure.

• Wording of items

  Similar to response format, peer researchers were aware that the wording of validated measures could not be amended. Concerns were raised about what were considered ‘Americanisms’ in the SF-36, for example ‘blocks’ and ‘pep’. For items such as this, the peer researchers provided alternatives that researchers could use to aid participants’ comprehension.

• Order of measures

  In the first draft of the CRF, the WEMWBS and the items related to offence history were at the start of the booklet. Peer researchers felt that it would be best to start with generic questions before asking those that could be considered more personal. They understood the need to order the WEMWBS near to the start of the CRF to ensure that this was collected if a follow-up appointment was unexpectedly cut short. It was therefore agreed between the peer researchers and the research team that this would be presented after demographic measures and that offence data (considered particularly sensitive) would be collected after measures of target health behaviours.

Retention of participants

Overall, as Figure 2 shows, of the 120 participants recruited and randomised, 60% (n = 72) completed the follow-up assessment at 3 months and 60% completed the 6-month follow-up assessment.

As Table 5 shows, there was no consistent difference in retention at the 3- and 6-month follow-ups between the intervention and control groups across all sites. Retention varied across recruitment site from 48% to 70% at 3 months, and from 48% to 88% at 6 months. Observed differences in retention rates between allocated treatment groups were not consistent at each follow-up assessment, suggesting no systematic bias in retention, but it is something to be wary of in a larger trial. Retention rates were higher among those recruited through the NPS than the CRCs at both the 3- and 6-month follow-ups; again, this is something to be wary of in a full trial. The process evaluation (see Chapter 6) further considers this.

Further qualitative information about the factors influencing retention is reported in Chapter 6.

Warwick–Edinburgh Mental Well-being Scale descriptive data by time, allocated arm and Community Rehabilitation Company/National Probation Service

Based on fairly small numbers of participants, we explored if there were noticeable differences in WEMWBS scores at each time point by trial arm and overall, for participants recruited via the CRCs and NPS. As shown in Table 11, baseline scores were slightly lower for the CRC participants. Differences in WEMWBS scores between trial arms were most noticeable at follow-up among the NPS participants.

Association between intervention engagement and change in Warwick–Edinburgh Mental Well-being Scale score

Table 13 shows the WEMWBS score over time for those who had low (up to one session), moderate (two to five sessions) and high (six or more sessions) intervention engagement, among those completing the WEMWBS assessments. It is noticeable that a smaller proportion of those assessed at baseline in the ‘low engagement’ group provided data at 3 and 6 months, compared with in the moderate and high engagement groups. It would appear that those with moderate engagement had higher WEMWBS scores at the 3- and 6-month follow-ups than those with the lowest and highest engagement.

Factors associated with intervention engagement

Table 13 shows the WEMWBS scores at baseline for those with different levels of intervention engagement. The scores were lowest for those who went on to have no more than one session with the health trainer.

Recruitment issues

Table 14 presents the planned and actual recruitment to the trial. We originally anticipated, based on expected numbers of probable eligible participants coming through the OMSs, that we would recruit approximately 10 participants per month (for 4 months) per OMS from September 2016. In the first 7 months after the first participant was recruited, we had recruited only 22 participants. Once recruitment processes were established across the three OMSs, it took 9 months to recruit 90 participants (i.e. 3.3 per OMS per month).

Given that the CRCs were still being established at the time of the original bid, the target was somewhat speculative. Our original bid included plans to recruit from offender services in Plymouth and Southampton. During trial set-up in Southampton, we discovered that health trainers had been commissioned to support offenders in the community, and we decided against comparing the STRENGTHEN health trainer intervention with the local health trainer support. Plymouth and Manchester were already collaborating on another trial involving support for offenders leaving prison, so we set Manchester up as a second site. This caused a delay in recruitment until contracts and excess treatment costs were confirmed, and researchers and health trainers were appointed. The first recruitment took place in April 2017 in Manchester. In Plymouth, the first recruitment in the CRC took place in October 2016, just a few weeks after intended. Recruitment via the Plymouth NPS eventually began in April 2017.

The conversion of potential participants from being approached and then randomised was more efficient in the NPS. One explanation for the NPS service having such efficient recruitment processes is that clients had less chaotic lives and once a step had been completed, there were fewer interruptions. It is not clear why the Plymouth CRC was more efficient at recruiting than the Manchester CRC, but we will seek to better understand this ahead of a full trial to maximise our resource efficiency in all CRCs.

The importance of the co-operation of the OMS for recruitment was very evident in December 2016 and January 2017 when the Plymouth CRC was unable to support us as a result of corporate and staffing difficulties. Several staff were on sick leave and others resigned, which meant searching through service records of potential participants stopped. Later dips in recruitment were due to researcher sick leave.

Retention issues

We originally defined our longest follow-up assessment to coincide with when we expected participants to have a 6-month follow-up with their offender manager, and, in many cases, to coincide with the end of their community sentence. As the study evolved, it became clear that the OMSs had not found it easy to ensure that a formal (face-to-face) 6-month session took place; therefore, we attempted to remain in contact with our sample and complete follow-up assessments around 6 months whenever they could be arranged. In three cases, this extended to 6 weeks after the assessment was scheduled to take place. Given that it was our aim to demonstrate that we could assess participants after at least 6 months, this is not seen as a major issue, but it did delay database cleaning and lockdown. No participant who withdrew from the trial requested that their previously collected data be removed from the trial database.

The slightly higher 6-month follow-up rate for the intervention group for the whole sample was particularly influenced by the large difference between the two trial arms among participants recruited in the south-west CRC, as shown in Appendix 2 (see Figure 5). Among the 47 south-west CRC participants randomised, 65% and 38% were followed up in the intervention and control groups, respectively. It is not clear why there was a between-trial arm difference in the south-west CRC participants, but not in the north-west CRC. As Figures 5–7 in Appendix 2 show, follow-up rates were, overall, considerably lower in participants from the two CRCs (51% in the south-west, and 48% in the north-west) than in the NPS participants (88%).

Data completeness

The completeness of data collection for all variables at baseline and follow-up assessments is shown in Appendix 6. Researchers reported few difficulties with the data collections that were mostly administered by the RA in face-to-face format. For the administration of the WEMWBS, almost all participants completed the measure themselves. As one might expect, our researchers reported that sensitive questions about substance use were probably incomplete. Throughout the rest of this chapter, summary statistics (including percentages) are based on the number of participants responding to the given questions; therefore, the number of respondents is also given.

Blinding of researchers at follow-up assessments

Table 15 shows the extent of possible bias in capturing follow-up data. As is common in studies involving complex interventions such as a health trainer intervention, participants may mention something associated with the intervention they have or have not received to a researcher when collecting follow-up data. Researchers were more likely to become aware of the assigned trial arm if a participant had been allocated to the intervention group than if a participant had been allocated to the control group. The data indicate that it will be very challenging to ensure that outcome measures are assessed by blinded researchers in a full definitive trial.

Data were analysed to determine how well the protocol was followed in terms of the screening and randomisation processes and in completing follow-up assessments in a timely manner, given the challenges in working with a considerable proportion of the sample. Table 16 shows the number of days it took between screening and completing the baseline assessment, and between completing the baseline assessment and randomisation. From the point of satisfactory screening and gaining informed consent to being assigned to the trial group was approximately 9 days.

The mean time between completing the baseline and 3- and 6-month follow-ups was 14 and 28 weeks, respectively, suggesting some slippage, but this may not be surprising given the challenges of keeping in contact with the participants in this trial. There was no noteworthy difference between trial groups in terms of the times between key events (data are not shown).

Indicative sample size for a definitive trial of the STRENGTHEN intervention

As specified in the agreed statistical analysis plan, indicative sample size calculations are based on estimates from the STRENGTHEN pilot data of the required parameters. The data in Table 18 have been used to produce potential target sample sizes for a definitive trial of the STRENGTHEN intervention, to detect a between-group difference of three units44,49,50 for the proposed primary outcome of WEMWBS score at the primary end point of 6 months post baseline, under a range of plausible assumptions.

The base case for the sample size calculation conservatively assumes a SD of 14.8 units (i.e. the upper one-sided confidence bound from the pilot data), with calculations also shown for SDs ranging from 12 to 16 units.

As the planned analyses would include adjustment for baseline WEMWBS, the effect of allowing for the correlation between the baseline and 6-month WEMWBS scores has also been considered. 51 From the STRENGTHEN pilot trial, the point estimate of the correlation between baseline and 6 months across all participants was 0.68, with the one-sided 80% lower bound being 0.63. The base case for the sample size calculation assumes a correlation of 0.63, with calculations also shown for correlations of 0.5 to 0.7.

Finally, an allowance is made for the estimated follow-up rate at 6 months. In the STRENGTHEN pilot trial, the overall retention rate at 6 months was 60%. However, a recent trial,43 funded by the National Institute for Health Research Programme Grants for Applied Research programme, in the CJS observed an increase in retention rate from 50% to 67% at the 3-month follow-up following the introduction of a small financial incentive for completing the trial. Therefore, the target sample size in the base case assumes a retention rate of 70%.

In summary, the base case assumes detecting a between-group difference of 3 units (SD of 14.8 units), a correlation between the baseline and 6-month WEMWBS scores of 0.63 and a follow-up rate of 70%; with two-sided 5% alpha and 90% power.

Based on Table 18, the number of participants required to be followed up at the 6-month primary end point ranges from ≈970 to ≈2060, before allowing for the correlation between baseline and follow-up. After including adjustment for this correlation, the number of participants required to be recruited reduces to a range of 580–1240.

Design

The economic analysis was conducted concurrently with the pilot randomised controlled trial, using within-trial intervention, resource use and outcome data, over a 6-month time horizon.

The incremental cost for the delivery of the STRENGTHEN intervention was considered when provided in addition to usual care, and costs associated with health/social care service use and quality-adjusted life-years (QALYs) were estimated for the intervention and control groups.

Intervention and comparator

The intervention arm (see Chapter 2) received the STRENGTHEN intervention plus usual care and the control arm received usual care alone.

Perspective

We adopted the primary perspectives of the UK NHS and Personal Social Services (PSS), with broader aspects of care also reported from a societal perspective.

Time horizon

The pilot trial time horizon of 6 months’ follow-up was employed in the economic analysis, with outcome assessments at both the 3- and 6-month follow-ups.

Intervention resource use and costs

The additional (incremental) costs associated with delivery of the STRENGTHEN intervention, when added to usual care, were estimated using resource use data collected within-trial.

The health trainers who delivered the intervention kept a record of participant contact on the intervention section of the data management system on which they collated details of face-to-face, telephone, text, e-mail and letter contacts. These were grouped according to whether the contact was a predesignated intervention session, a planned contact or another form of contact. The health trainer also recorded the time spent travelling in relation to these contacts (whether or not the participant attended the appointment) and documented any additional time spent unsuccessfully attempting to contact participants.

Other resources required for the delivery of the intervention were identified via a questionnaire and discussion with the intervention developers after the intervention had been delivered. Such resources included the health trainers’ handbook, worksheets and folders for participants, training and supervision of the health trainers, and additional administrative and management activities undertaken by the health trainers and the health trainer co-ordinator. The health trainer co-ordinator prospectively recorded their time involved in intervention-related activities.

Costs were applied to the intervention resources in Great British pounds at 2017 costs. Unit cost estimates were drawn from nationally recognised, published sources (Unit Costs of Health and Social Care52,53 and NHS reference costs54,55), with the project-specific information about costs provided by those who delivered the intervention. These unit costs are set out in Table 19.

Health, social care and broader societal resource use and costs

Self-report resource use was collected via interviewer-administered questionnaires at baseline and at the 3- and 6-month follow-ups (covering the prior 2-, 3- and 3-month periods, respectively). (The first reporting time period was 2 months as, in line with the trial inclusion criteria, participants may have been in the community for only 2 months following a custodial sentence.) The questions enquired about health care, social care and other services that participants may have used, such as criminal justice and education resources, in addition to assistance provided by relatives or friends [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/phr/145419/#/ (accessed 30 August 2019)].

We combined health and social care follow-up resource use data with nationally recognised, published unit costs52–55 to estimate the mean (SD) resource costs per participant. Costs were in Great British pounds at 2017 costs or adjusted for inflation if costs for 2017 were not available (Table 20).

Outcomes

Our primary economic end points were costs and QALYs at the 6-month follow-up. QALYs are a commonly used summary measure of health-related quality-of-life, taking account of both quality and quantity of life,57,58 and are the metric used by the National Institute for Health and Care Excellence (NICE) when considering the cost-effectiveness of interventions across a broad range of health and social care contexts. 59 We estimated QALYs over the 6-month trial follow-up using the EQ-5D-5L trial data and applying the internationally recognised ‘cross-walk’ algorithm60 to provide QALY weights from a UK general population survey to value the EQ-5D-3L (EuroQol-5 Dimensions, three-level version) health states. 61 This methodological approach adheres to the current ‘position statement’ of NICE regarding use of the EQ-5D-5L. 62

Given the uncertainty of the appropriateness of the EQ-5D-5L for this population, we also used trial data from the SF-36 to estimate QALYs using the SF-6D. 63

Analysis

Resource requirements and cost of the STRENGTHEN intervention

Sixty-one people were provided with the STRENGTHEN intervention. This included 20 from the centre in Manchester, 23 from the Plymouth CRC and 18 from the Plymouth NPS. The resources used to provide the intervention, and their quantities and costs, are detailed in Tables 21–24.

Health and social care resource use and costs

Table 25 presents quantities of resources used from the primary economic perspective of the NHS/PSS, across the 6-month follow-up, described by treatment arm. These resources are disaggregated by item and grouped as primary care, secondary care and social care services. Table 26 presents the costs associated with this disaggregated resource use. Total NHS/PSS costs were £773 in the control group and £585 in the intervention group. The largest contributors to overall costs were general practitioner (GP) surgery appointments, counselling sessions, community drug or alcohol services, hospital day cases and social worker contacts. Tables 36–38 in Appendix 4 present the health and social care resource use of the intervention and control groups.

Details of NHS and social care resource use and costs for the 2 months prior to baseline, and separately at the 3- and 6-month follow-up assessments, for the intervention and control groups, are given in Appendix 5 (see Tables 39–41) and Appendix 6 (see Tables 42–44).

Broader societal resource use

Table 27 gives details of broader societal resources that intervention and control group participants reported using over the 6 months of follow-up. These items are disaggregated and grouped by education services, other services and informal care. Appendix 6 provides additional details of use of these resources in the 2 months prior to the baseline assessment, and separately for the 3- and 6-month follow-ups. Similar patterns of resource use were apparent in the intervention and control groups at baseline and at follow-up.

Outcomes

Data completeness

The cost of the STRENGTHEN intervention

We estimated the mean cost of the STRENGTHEN intervention to be approximately £348 (SD £128) per participant. The main cost drivers for the intervention, determined by data prospectively collected using health trainer/participant contact sheets, activity logs of the health trainer co-ordinator and a questionnaire for completion by the intervention providers, were: (1) staff time of the health trainers and the health trainer co-ordinator and (2) supervision of the health trainers.

Supervision of the health trainers cost approximately £94 per participant, and involved approximately 1 hour of contact between the health trainer co-ordinator and each health trainer per week. This degree of supervision and support is considered imperative for the nature of the intervention, working with this population under community supervision, and would probably be replicated in a definitive trial and if, longer-term, the intervention is implemented more widely. In other work involving health trainer support for reducing smoking, we have managed to create a more cost-effective supervision process with shared virtual sessions involving up to eight health trainers across four sites. In a full STRENGTHEN study, we would therefore expect to reduce supervision costs with an economy of scale.

Costs for training the health trainers amounted to approximately £13 per participant, but there were significant uncertainties regarding the resource requirements for training and the allocation of costs across caseload, given the high staff turnover in the pilot. It is anticipated that the cost of training would be reduced in a larger trial, and if the intervention is shown to be effective and cost-effective and is put into practice across the UK. Economies of scale might mean that more health trainers could be trained simultaneously. With implementation, a more formalised training programme might be established, for example in relation to a City and Guilds qualification. A further developed training approach would affect resource requirements and costs of the intervention, and would warrant additional consideration and investigation in the context of a definitive trial.

It is notable that it cost approximately £12 per participant in health trainers’ time travelling to meet participants who did not attend appointments. It may be that this is a realistic component of providing an intervention to this population. The tenacity of health trainers in repeated attempts to contact and support participants may be an important feature in contributing to the potential effectiveness and cost-effectiveness of the intervention.

The estimated cost of the STRENGTHEN intervention is greater than that of the health trainer intervention provided in the Exercise Assisted Reduction then Stop (EARS) pilot study (£192). 35 This difference in cost appears largely attributable to more time spent training, supervising and supporting health trainers in the STRENGTHEN trial. This is a necessary requirement for the health trainers to be appropriately skilled and supported given the focus in the STRENGTHEN trial on a broader range of health behaviours and well-being and the greater need to build trust and rapport in the first session before moving on to behaviour change.

A future full trial and cost-effectiveness analysis could usefully consider other aspects of the structure of the intervention that might alter if it were to be implemented in practice, and how this might affect resource requirements and their costs. Such differences could be planned for in sensitivity analyses in a definitive cost-effectiveness analysis. For example, in examining the association between intervention engagement and WEMWBS scores at follow-up (see Chapter 4), it appeared that the scores were greater for those who attended 2–5 sessions rather than more (or fewer). Limiting the number of sessions to no more than five may help reduce the intervention cost.

Health/social care resource use and costs

The pilot found a lesser cost of health/social care resource use (excluding the intervention) in the intervention group over the 6 months of follow-up. When the cost of the intervention was also considered, the incremental cost in the intervention group was greater than that of the control group (over the 6-month follow-up period). However, differences in health/social care resource use were largely accounted for by a small number of costly drivers (e.g. hospital day cases) and the groups also differed in the cost of their resource use in the 2 months prior to baseline. The groups may, therefore, have been imbalanced at baseline, and the small sample size susceptible to the effect of a limited number of outliers and costly resource events.

Outcomes

At the 6-month follow-up, the intervention group had higher WEMWBS scores, more QALYs based on the SF-6D and marginally more QALYs based on the EQ-5D-5L; however, the pattern of scores at baseline indicated that the intervention group had higher SF-6D health state values, but poorer EQ-5D-5L and WEMWBS scores.

Learning from the pilot for a full cost-effectiveness analysis alongside a definitive trial

There are many learning points from this pilot for a future cost-effectiveness analysis, the most important of which are as follows.

Development of an economic evaluation framework for a definitive trial

In addition to our learning described above, for a future full economic evaluation and cost-effectiveness analysis we would follow good practice guidelines67 and national policy guidance. 59 Our base-case approach of an NHS/PSS perspective follows the methodological guidance for cost-effectiveness analyses set out by the NICE,59 and a broader societal perspective will be further explored in future sensitivity analyses, as recommended by the second panel on cost-effectiveness. 68

For the full trial, we will assess the cost-effectiveness of the STRENGTHEN intervention in relation to the threshold of £20,000 to £30,000 per QALY used by NICE for recommending treatments or interventions for funding in the NHS. 59 Our primary economic analysis will estimate mean costs and mean QALYs by treatment allocation, and estimate differences between groups over the follow-up period. We will calculate incremental costs and incremental effects, and combine these to present incremental cost-effectiveness ratios (ICERs), to enable decision-makers to assess value for money using cost-per-QALY estimates [ICER = (cost_STRENGTHEN – cost_{usual care})/(QALY_STRENGTHEN – QALY_{usual care})].

To present the level of uncertainty regarding the cost-effectiveness estimates, we will use the cost-effectiveness plane to present combinations of incremental costs and incremental QALYs from bootstrap replicates. We will also use the cost-effectiveness acceptability curve (CEAC),69 with the net benefit statistic [(incremental QALYs × willingness to pay per QALY) – incremental cost],58 to present the probability that the STRENGTHEN intervention is cost-effective (i.e. incremental net benefit statistic is > 0), against a range of potential cost-effectiveness thresholds.

We will analyse the data on an intention-to-treat basis and, as the follow-up period will not be > 12 months, no discounting of future costs or outcomes will be undertaken. We will explore the extent of missing data, with the intention of using regression-based multiple imputation as required in sensitivity analyses.

Aims

The aims of the process evaluation were to:

• assess whether or not the intervention was being delivered as per the manual and training

• ascertain components of the intervention that are critical to delivery

• explore reasons for divergence from delivery of the intervention as manualised

• understand when context is moderating delivery

• understand the experience and motivation of participants in the control arm of the pilot in order to maximise retention in a full trial

• explore reasons for declining to participate in the trial

• explore reasons for disengaging in the intervention before an agreed end

• understand, from a participant perspective, the benefits and disadvantages of taking part in the intervention.

Methods

Audio-recordings of health trainer-delivered sessions (n = 18) were purposively sampled across participants who had also taken part in a process evaluation interview to allow for the potential to triangulate process evaluation data. Methods employed were those used in previous research conducted by members of the research team for assessing treatment fidelity,71 albeit to assess the competence of three health trainers in supporting change in two health behaviours: physical activity and smoking. The health trainers in the present study had the opportunity to support change in four health behaviours and well-being, but we were interested in the generic competencies, irrespective of the behaviour(s) that the participant wished to work on. In this pilot trial, we also aimed to test the methods for assessing intervention fidelity, given the difference in aims of the STRENGTHEN intervention.

The participants sampled (n = 9) had a minimum of a first session and a subsequent audio-recorded session. The first session and a later session for each participant were listened to and scored as a whole, independently, by two researchers, in order to capture competencies that may not have been present in the first session (e.g. core competency 4). The Dreyfus system for skill acquisition72 was used to assign a score for each of the six core competencies on a 7-point Likert scale (0–6) reflecting six levels of competence (from incompetent to expert). Written guidance was provided to the researchers to inform their scoring (see Appendix 1, Figure 4, for rating criteria for delivery fidelity). A score for each participant was generated based on the health trainer’s performance across two sessions to attempt to reflect their experience of the intervention delivery as designed around the six core competencies. Owing to the comparatively large number of health trainers (n = 6) across the two sites, no comparison between individual health trainers was planned.

As part of an initial standardisation procedure, a recorded session was listened to simultaneously by both researchers who independently scored the health trainers’ levels of competence. The scores were then discussed and agreement reached over interpretation of the scoring scales. A further three sessions were scored independently and subsequently discussed between researchers to ensure that both researchers employed similar interpretations, before independently reviewing and scoring the remaining sessions.

Scores classed as ‘competent delivery’ were considered acceptable delivery, those as ‘advanced beginner’ may need further development in future training and any below ‘novice’ may represent problematic performance and warrant further investigation.

Results

Table 31 shows the mean scores for each of the six core competencies, overall and for each coder.

Inter-rater reliability was assessed across all items using a two-way mixed, consistency, average measures intraclass correlation coefficient (ICC). The resulting ICC was in the excellent range (ICC = 0.84), suggesting that coders had a high level of agreement, which indicated that the ratings scales were employed consistently. Overall, the total mean scores for intervention delivery differed by 0.07 between coders, and the mean for overall delivery was scored at 2.99 (SD 0.84), suggesting overall competent delivery.

Active participant involvement (core competency 1) scored highest, approaching the proficient level of delivery, whereas engaging social support and managing social influence (core competency 6) scored the lowest. All other items were rated as approaching the mid-point of the scale for competent delivery (Figure 3).

FIGURE 3.

Overall mean (SD) fidelity scores by core competency.

Discussion

Overall intervention delivery fidelity was judged to be acceptable, with some areas for improvement. The overall score for competence of delivery was judged to sit between ‘advanced beginner’ and ‘competent’. Given that this was a novel intervention being delivered with a complex target population, delivery can be deemed to be acceptable within the context of a pilot trial, albeit limited by the small number of sessions analysed.

Active participant involvement scored notably higher than the other competencies; this is probably due to the importance placed on it during intervention development and health trainer training. The population was acknowledged as being potentially very distrustful of services; therefore, rapport- and trust-building (a key component in core competency 1) were key aims emphasised throughout training and supervision. They were particularly evident as aims of the first session, and through a function of the sampling procedure, more opportunity to demonstrate core competency 1 was observed because 50% of the sessions sampled being a first session.

Engaging social support and managing social influence (core competency 6) showed a trend of being the least well-delivered competence. Anecdotally, this may be due to the difficulties this population faces in feeling trapped by their social circles and influences, unable to relocate from a community of offending, and perceived barriers to what are considered new socially acceptable activities owing to their being labelled as an offender with the attached stigma. Conversely, some participants had consciously isolated themselves in attempt to move away from a culture of offending. Future training and intervention development would benefit from increased understanding of how the population perceive their social influences and identify acceptable ways to help participants to positively manage negative social influence and engage positive social support.

Although the other four competencies scored reasonably well, they were all slightly below the threshold for competent delivery. It is possible that this occurred as a result of over-emphasis on actively engaging the participant, which occurred at the expense of the other competencies; it also possibly occurred as a result of the sampling procedure. Additional scoring of sessions other than the first sessions may have uncovered more examples of proficient delivery of the other competencies. These issues should be considered in more detail in future research and health trainer training.

Methods

Findings

Suggested adaptations

Participants in both arms of the pilot trial found their participation in the research and, where relevant, the intervention, to be acceptable. Participants suggested potential improvements including:

• Researchers should explain potentially hard to understand words within the CRF without the participant having to ask or show that they did not understand.

• Worksheets could be used more systematically to review overall progress.

• A participant with a diagnosis of attention deficit hyperactivity disorder suggested that the intervention could be offered alongside activities that could facilitate people with a similar diagnosis to make the most of the support being offered.

• Signposting to other services.

• Providing literature and information about events linked to target behaviours.

• Offering more sessions to follow up their progress or extend the intervention.

Findings

Key themes from health trainer interviews are presented in the following sections, with example quotations to support each theme.

Suggested adaptations

Health trainers suggested adaptations to enhance their delivery of the intervention, including the following:

• training –

  • more time to practice MI techniques and receive feedback

  • wider range of modes of delivery throughout the training sessions to break up time with activities

• manual –

  • add inclusion of a clear definition of the role and remit of the health trainer

  • provide guidance regarding the use of the data management system for reference post training

  • provide guidelines and suggested wording for telephone calls and text messages to participants to support effective communication

• supervision –

  • allocation of more time for discussion of core competencies and how they are applied in practice to enhance learning.

Two proposed additions to supporting materials were actioned in the early stages of delivery:

1. Development of a working document of local services and schemes to which participants could be signposted. The original health trainers in both sites were able to use their time as their caseloads built up to identify local services to collate a list of signposting options.

2. It was suggested that the participants should be provided with some paperwork to support their engagement with the intervention. This led to the development of a participant pack containing information about the intervention remit, the health trainer and local services. This was presented in a folder in which any further information and worksheets used by participants during the intervention could be stored.

Findings

Key themes from offender manager interviews are presented in the following sections with quotes to support each theme.

Case study: Jack’s story

This case study presents the experience of an individual who engaged well with the intervention and who focused on and changed multiple behaviours. Jack’s focus was on healthy eating, physical activity and mental well-being. All personal identifiers, including distinguishing features of his presentation, have been carefully modified or removed to ensure anonymity. The case study was selected as an example of multiple health behaviour change alongside change in mental well-being. The case study makes use of transcribed health trainer session recordings, health trainer session notes and participant interview to describe Jack’s journey through the intervention.

Recruitment

We eventually recruited the planned 120 participants, albeit over a longer time scale owing to a number of delays beyond our control. The pilot trial has helped us to better understand the recruitment constraints experienced in three different offender services in two cities as well as the resources needed to recruit participants in a timely manner, from both quantitative and qualitative data. Given that the CRCs in both Plymouth and Manchester were only just finding their feet at the beginning of the study and at times were under extreme operating pressures, it was an extra challenge to build vital relationships with the offender managers and the service leads. We are indebted to those working in the services who ran searches for potentially suitable participants, informed us of when appointments were taking place to allow our researchers to make contact, and for their support in many other ways. We have used all our experience of working in the CJS to make it work; should further changes occur to services, we feel we will be even more equipped to minimise barriers to recruitment, across different services and recruitment sites, to a larger trial.

At the start of the trial, there were uncertainties about the conversion rates within the recruitment processes and we largely addressed these. Early experiences of working with the OMS led to more consistent and informative understanding of the numbers of potentially eligible participants and the probable recruitment rates. The main reasons for not including potential participants in the pool to be screened was that that they did not fit in a predefined window of having lived in the community for at least 2 months (following release from prison, if relevant) or they had < 7 months of community supervision remaining within their court order ‘typical’ lifestyles and levels of well-being. The CRC and NPS data management systems were mostly resource efficient for completing this prescreening, although our researchers still had to remove people from our pool of potential participants because of lapses in time between the initial search by the service and further screening.

Many potential participants were also not included in the pool of potential participants or were subsequently excluded owing to the risk they posed to the trial researchers and health trainers in the community. Making such assessments was understandably resource intensive, involving both objective and subjective criteria for the OMS, and delayed the recruitment process at times of high pressure on staffing in the services. Owing to this screening and subsequent lone-worker policy procedures, there were only a few minor incidents in which researchers and health trainers felt uncomfortable and these were efficiently resolved.

We also wanted to ensure a balance between including participants who had appeared to have a chaotic life (e.g. were hard to establish contact with and perhaps often missed OMS appointments) and low levels of well-being, who may have the most to gain from the health trainer intervention, and excluding those who were likely to be lost to follow-up, given that we had set a challenging progression criterion for the pilot trial. We also wanted to include only those who were likely to remain in the same geographic area to maximise follow-up rates and intervention engagement. The results provisionally indicate a possible association between baseline WEMWBS scores and intervention engagement: those with the lowest WEMWBS scores at baseline tended to have the lowest levels of intervention engagement, and they tended to be more likely to be lost to follow-up. Leading into a full trial, we would continue to review our processes for including and excluding those with a chaotic life, and ways to maintain intervention engagement and completion of follow-up assessments.

The process evaluation also tried to identify why, through interviews, some participants had chosen not to take part in the study. The reasons largely mirrored those recorded by researchers during screening, and it was not always easy to differentiate between whether they simply did not want to take part and did not value what was on offer, or had other conflicts and chaotic lives meaning that even a brief follow-up meeting or intervention session would have been difficult. Interviews with staff in the OMS also identified a tension between the objectives of the pilot trial to assess client-centred health trainer support, and the need for the OMS to provide opportunities for their clients to gain credits for community engagement. The research team agreed to resist this as it would have changed the dynamics of the intervention. We could probably have increased recruitment had the intervention been badged as something to provide ‘credits’ but we resisted this.

The trial and intervention procedures were generally well understood, based on our process evaluation. Participants in the control group were occasionally disappointed at not receiving the intervention but, in general, they were happy to feel that they were contributing to the research evidence. We had developed participant information sheets with our PPI groups and these seemed to generally be well understood. The idea of being in a RCT and having a chance of receiving (or not) a complex intervention was not always easy to convey, but there were few examples of misunderstanding. Describing a complex intervention, such as health trainer support to explore change in four health behaviours and well-being, with a largely open-ended format in terms of intervention aims, frequency and location of sessions and intervention duration, is not easy. The overall levels of engagement suggest that we had considerable success in recruitment and intervention engagement. Ahead of a full trial, we would seek to explore, with our PPI group, further ways to improve understanding of the trial methods and nature of the intervention, but it is anticipated that only minor changes would be necessary.

Study attrition and follow-up data completion

As we indicate above, a full trial could have improved follow-up rates by including only those who have less chaotic lives but we have learnt a lot about getting the right balance between selective recruitment and maximising the potential for the intervention to support those with the greatest need. A full trial will be about producing effectiveness and cost-effectiveness information on the health trainer intervention and also understanding for whom it is beneficial. It will be important to maximise recruitment and retention from the range of people with community sentences.

Over the course of the pilot trial, we did improve follow-up completion rates in a number of ways, such as making contact directly with participants rather than waiting for planned service offender meetings (which did not always take place), using social media to stay in touch and by putting more focused staffing resource into the process across sites. We will further explore the socioeconomic, demographic and other factors influencing study attrition, but propose that, in a full trial, we will include a financial incentive to improve follow-up rates.

We found only limited evidence that the trial assessment procedures were overly burdensome, and thereby contributed to lower follow-up rates. All follow-up assessments were completed in face-to-face format with our researchers and, once a meeting was under way, the level of data completion was very good for most outcomes.

Participant characteristics and reach

We cited information in Chapter 1 from other sources that show that the offender population have typically lower levels of well-being and poorer lifestyles than the general population. The demographic characteristics, lifestyle behaviours and measures of well-being in this pilot study mirror those in the literature, and therefore indicate good reach and generalisability for the findings. For example, original data obtained from the Scottish Prison Service in 2014 (personal communication) showed a mean WEMWBS score of 43.2 (SD 12.3) (range 14–70), compared with a general population score of 51.6 (SD 8.71) for England31 and 49.9 (SD 8.5) for Scotland. 34 Our overall sample had a mean WEMWBS score of 44.2 (SD 11.8). Lower WEMWBS scores are typically associated with smoking, lower consumption of fruit and vegetables, high alcohol intake and lower socioeconomic status. The data in this pilot trial indicate similarly unhealthy lifestyle behaviours, low levels of educational attainment and other sociodemographic indices.

Quality-of-life scores (EQ-5D-5L and SF-6D) for the present sample are also very low compared with the general population. 75–78

The sample, overall, was 91% male, and this reflects the overall low proportion of females in the UK CJS. In the process evaluation, we interviewed offender managers who spoke about the gender balance among the clients they worked with, and the focus of the OMS on risk level, with NPS participants being mainly males who had committed sexual offences or serious assault.

Recruitment

• We will seek further PPI input into how to disproportionally over-recruit females to our sample to maximise the generalisability of the findings.

• We have already approached and will work closely with all 16 OMSs to identify trial procedures including recruitment processes. The OMSs that we have already engaged with are very keen to be involved again, and we are aware that they can be powerful advocates for bringing new OMSs into the study. We propose a full trial to recruit from eight cities, paired into four regions, with a NPS and CRC in each city, to maximise researcher efficiency.

• We envisage more efficient recruitment from our NPS partners, and will therefore consider the balance of research resources needed to recruit similar numbers of participants from each OMS.

• We have found from initial RA training that maintaining RA engagement is essential for recruitment in the challenging OMS environments. We will establish bi-weekly virtual RA meetings to share learning about recruitment processes and opportunities, as in the pilot trial and in another NIHR health trainer trial. 80

• The addition of a £10 shopping voucher as a reward for completing the 3-month follow-up assessment and £20 at 6 months may also provide an extra incentive for potential participants, as identified by our PPI group.

• There is a risk of bias in terms of which participants will be identified as being suitable for the trial by offender managers. RAs at each site will be trained to provide each OMS with a clear brief on our inclusion/exclusion criteria and how health trainers work with those with more chaotic lives, to try to ensure that only the most high-risk clients are excluded. We will build on the procedures used in the pilot trial to monitor and report on risk of bias in the recruitment process. For example, the reasons for exclusion were sometimes reported retrospectively in the pilot trial, which was resource intensive; with revised procedures and RA training, this could be done proactively, and linked to the PenCTU database system.

• Given the range of recruitment efficiencies (i.e. conversion rates from approach to randomisation) between Manchester and Plymouth and NPS/CRC, we will review processes at all sites and carefully monitor recruitment rates, with the addition of support as needed.

• We recruited just six participants via the community (i.e. without searching NPS and CRC databases) at a time when we were struggling to recruit to the trial. These few participants were not subjectively different from other participants in the trial but required additional resources. We would not plan to use such an approach in a full trial.

• Finally, there are proposed changes to the organisation of OMSs; NPS will take over offender management from CRCs in England when their contracts expire in spring 2021. There will be 10 regions, each led by a ‘Regional Probation Director’. Other probation services including unpaid work, accredited programmes and resettlement/rehabilitation interventions will be commissioned from private and voluntary sector providers. CRCs in Wales will be taken over by NPS from 1 December 2019.

Trial retention

• To improve trial retention (i.e. increase follow-up assessments to at least 70%), we would introduce the following changes to a full trial protocol: (1) the attendance at follow-up assessments would be incentivised, with participants receiving a shopping voucher on completion of 3-month (£10) and 6-month (£20) visits; (2) we would maintain close working relationships with offender managers to facilitate the need for RAs to conduct follow-up assessments in conjunction with the 3- and 6-month supervisory visits to the OMS, or elsewhere if needed; (3) we would reflect on pilot trial processes and those in another trial (ENGAGER)37 to optimise ways to stay in touch with participants outside the OMS; and (4) special attention would be given by RAs to improve retention rates for those participants under supervision in the CRCs, to limit bias in the findings.

• In a full trial, we would focus process evaluation resources on trying to determine the reasons for loss to follow-up using OMS information and participant interviews.

• We will conduct further exploratory analysis from the pilot trial data to describe the characteristics of participants who did/did not complete follow-up assessments to inform strategies to further boost trial retention.

Intervention engagement

• We will conduct further exploratory analyses from the pilot trial data to describe the characteristics of participants who completed more/less/no intervention sessions to inform strategies to further boost intervention engagement, especially among CRC participants.

• We will further explore the association between the number of intervention sessions attended and WEMWBS scores at follow-up.

• We will further use the data captured in the process evaluation of the intervention in the pilot trial to identify what the health trainers found difficult to do and what was well received by participants, and how the intervention affected generating trust and rapport, and changes in well-being and lifestyle. This would be used to inform PPI discussion groups and make what we anticipate to be relatively minor changes to the intervention and training materials ahead of a full trial.

• We would also use ongoing data gathered from the delivery of a health trainer intervention in our four-site RCT (TARS)80 with a focus on changing smoking and physical activity, involving 450 participants who are being offered health trainer support. In this trial, we have developed training materials for a single training event for eight health trainers over 3 days, with subsequent regular virtual training. All health trainers have remained in post over 12 months to date, and ongoing process evaluation is helping us to reflect on training and supervision processes and intervention content. Leading into a full STRENGTHEN trial, this experience will be added to the learning from the pilot trial to produce more efficient and effective training and delivery of the health trainer intervention and assessment of intervention fidelity.

Intervention delivery

The training for the practitioners should be adapted to incorporate more focus on delivering the competencies that were not so well delivered in the pilot trial (e.g. engaging social support and managing social influence). Although an active participant is central to the intervention, such a strong focus on training, including throughout ongoing supervision, possibly came at the expense of the other competencies. This should be accounted for through extra time for training and supervision, particularly early on.

Blinding

In response to the finding that RAs had become unblinded in assessing 60% and 75% of participants at 3 and 6 months, respectively, with a higher proportion of intervention participants inadvertently indicating their trial arm to RAs, we would explore, with PPI input, ways to best minimise unblinding ahead of a full trial. Unblinding in studies of this kind are notoriously difficult to maintain, as in another NIHR trial involving offenders (ENGAGER). 37 It is possible that blinding would have been even more unlikely without the steps that we put into place, namely using an administrator to notify participants as to which arm they had been allocated to, instead of using the RAs. So we would propose to use a similar system to that in the pilot trial, with some minor changes. The process evaluation revealed that unblinding resulted from participants mentioning if they had been involved in the intervention, and sometimes the offender manager (often present at follow-up assessments) mentioned it. We would train RAs to reinforce to participants and offender managers the need to not discuss intervention involvement (or not) until after any assessment is completed. We will also conduct sensitivity analyses in the main analysis to determine the possible effects of unblinding.

Key features

The health trainer should encourage the client to be actively involved in the consultation. The idea is to maximise the client’s autonomy as the main agent of change, developing intrinsic rather than extrinsic motivation, and encouraging her/him to be the person coming up with ideas for improving the situation. However, the client should not be allowed to ramble excessively in an unstructured way and the consultation should be guided through skilful use of MI techniques. A collaborative/shared decision-making style is appropriate and the health trainer may share his/her own expertise and ideas, using techniques such as elicit–provide–elicit. Overall, the client should be increasingly empowered to take control of her/his behaviours and decisions. Interactions should be encouraging, respectful and non-judgemental (the opposite of a didactic, telling or persuading style of interaction). The client should ideally talk for at least half of the time. The interaction should also be individually tailored to the participant’s specific information needs, beliefs, motivations and barriers. The health trainer should engender a clear sense of warmth, genuineness and empathy (within professional boundaries) to develop trust.

Intervention techniques

The intervention techniques used were the following: open questions, affirmation, reflective listening and summaries (OARS). Reflective listening may include simple reflections of content but may also be more sophisticated (e.g. amplified reflection, reflection with a twist) and used to direct the conversation or highlight key strengths or barriers. The elicit–provide–elicit technique should be used to exchange information (e.g. to address misconceptions or offer helpful new information, and to provide vicarious experience). Empathy-building techniques and individual tailoring should be used throughout the consultations: from the initial consultation through to action-planning through to review/maintenance sessions.

Key features

The health trainer should work with the client to explore initial beliefs and motivations about why they want to make any changes. The client’s motivation for making change is built up/enhanced through the exchange of information and techniques to assess and enhance motivation, that is to enhance the perceived benefits (importance) of making a change and confidence (self-efficacy) to take the actions needed.

Intervention techniques

Open questions, affirmation, reflective listening and summaries should be used specifically to explore current behaviour and the pros and cons of changing, and to develop discrepancies between current behaviour and desired behaviour (or outcomes). The decisional balance technique or 0–10 questions may be used to explore importance and confidence. Information should be exchanged on the pros and cons of changing and this and other techniques (exploring possible futures, discussing past attempts) should be used to explore barriers and possible solutions to increase confidence about cutting down/quitting. Motivation building should ideally happen around the start of the intervention process, although it can be further explored and reinforced at later (action-planning, review and maintenance) stages. Establishing self-rewards or incentives (e.g. saving money in a jar, planning rewards) may be part of the process for maintaining motivation.

Key features

The health trainer should work with the client to discuss a range of strategies for making the desired changes. They should agree a verbal plan of action, seeking to make this as specific as possible. They should discuss the use of self-monitoring to keep track of progress. Written goals and self-monitoring can be implemented if appropriate.

Intervention techniques

The intervention techniques used were the following: goal-setting (with gradual/graded progression), action planning, self-monitoring and deconditioning strategies. Any or all of the strategies presented in the manual may be presented and discussed. The action plan should normally be made verbally, but the health trainer should seek to make this as specific as possible in terms of ‘what, where, when and who with’ and making the goal as specific, measurable, achievable, relevant and time-related (SMART) as possible. The health trainer should introduce and discuss with the client the usefulness of self-monitoring of behaviours (level of behaviour, pattern of behaviour, how behaviours link). A specific plan for self-monitoring should be included in the action plan. The health trainer may also encourage self-monitoring of the contexts (social or environmental or emotional circumstances) in which problems/relapses might occur. Pre-empting and thinking of solutions for possible problems (making a coping plan, if/then plans, barriers/facilitators) is also appropriate here and may involve the use of other recognised behaviour change techniques (e.g. engaging social support, stress management).

Key features

The health trainer should work with the client to reflect on progress with goals. The health trainer should affirm/reinforce any successes. The client and health trainer should discuss any setbacks (reframing to normalise them, identifying barriers and exploring ways to overcome them). The health trainer and client should then set new targets, either progressing from the old one or establishing new ones that help avoid successive failure.

Intervention techniques

Use of OARS specifically to reinforce successes, to discuss setbacks, to identify barriers (including social or environmental contexts that hinder progress) and explore ways to overcome them (problem-solving). Reframing should be used to normalise setbacks. Goals/action plans should then be reviewed. There may also be some reflection on, and reinforcement of, the client’s skills in avoiding or managing relapse to undesired behaviour (building skills and self-efficacy). Problem-solving may involve the use of other recognised behaviour change techniques (e.g. engaging social support, stress management).

Key features

The health trainer should work with the client specifically to help her/him gain an appreciation of the relationship between behaviours and well-being. A clear rationale should be presented for how behaviours and feelings/mood are linked and can influence one another. However, both explicit processes (encouraging the client to complete activities to specifically enhance their well-being) and implicit processes (encouraging clients to embrace approaches to changing their behaviour that incorporate the 5WWB) should be facilitated by the health trainer.

Intervention techniques

Use of OARS, goal-setting (with gradual/graded progression), action planning, self-monitoring and deconditioning strategies. Health trainers should present the rationale to the client in an appropriate way and time (after the client is well engaged) and invite clients to reflect on if/how it relates to their own behaviour and well-being. Health trainers could encourage clients to use self-monitoring approaches and treat it as an experiment, to see how their well-being is on a day-to-day basis in relation to how their other behaviours are. Using MI principles and guiding clients towards plans that incorporate the 5WWB (e.g. connecting with non-drinkers to help reduce their alcohol intake).

Key features

The health trainer should encourage the client to engage social support (to assist on making or carrying out plans) or manage social influences on their behaviour. Social support can be informational (helping to make plans, providing ideas), emotional (not putting pressure on the person to perform unwanted behaviour/accepting their decision to change) or practical (e.g. helping to monitor progress). They should also look to support people to engage social support as a way to connect with others wherever possible.

Intervention techniques

Open questions, affirmation, reflective listening and summaries may be used to explore social influences and to identify possible problems and solutions relating to social influences.

Below is some guidance on how these core competencies may be scored as part of the research process when session recordings are being reviewed. If it is helpful, a scoring approach could also be used in supervision sessions.

Adjusting for the presence of participant difficulties

Adjustments may be needed when participant difficulties are evident (e.g. excessive avoidance or resistance). In such circumstances, the rater needs to assess the health trainer’s therapeutic skills in the application of the methods. Even though the health trainer may not facilitate change, credit should be given for demonstrating appropriate skillful interaction.

When rating the item, you should first identify whether some of the ‘key features’ are present. If the health trainer includes most of the key features and uses them appropriately (i.e. misses few relevant opportunities to use them and delivers them well), the health trainer should be rated very highly. It is also possible that not every item will be applicable in every consultation. It is important to remember that the scoring profile for this scale should approximate to a normal distribution (i.e. a mid-point of 3), with relatively few scoring at the extremes.