A multicomponent psychosocial intervention to reduce substance use by adolescents involved in the criminal justice system: the RISKIT-CJS RCT

Adolescence as a critical developmental stage

Adolescence is a critical developmental stage when young people make behavioural and lifestyle choices that have the potential to affect their health and well-being into adulthood. Although risk-taking is important for healthy psychological development for many, inappropriate risk-taking is significantly associated with health and social harms during adolescence, and these harms can persist well into adulthood. 13 Young people are much more vulnerable than adults to the adverse effects of substance use because of a range of physical and psychological factors that often interact and because of the differential impact of substances on the developing brain. 14–16 In addition to an increased risk of accidents and injury,17 substance use in adolescence is also associated with poor educational performance and exclusion from education. Over the academic year 2015–16, almost 10% of permanent school exclusions in state secondary schools were because of alcohol and substance use. 18 In the longer term, substance use is also associated with an increased prevalence of non-communicable diseases, such as cancer, cardiovascular disease and gastrointestinal disorders. 19,20

Prevalence of substance use among young people in the UK

The number of young people who consume alcohol is declining, although those who do drink tend to drink more. In 2018, 54% of youths aged 14 years and 69% of youths aged 15 years consumed alcohol, with 23% having consumed alcohol in the past week. 21 The mean weekly alcohol consumption for males at age 14 years was 5 units and for those aged 15 it was 7 units, where one unit equals 10 ml or 8 g of pure ethanol. For females, the mean weekly alcohol consumption was 5.5 units and 5 units, respectively. Six per cent of youths aged 14 years and 11% of youths aged 15 years reported having used cannabis in last month. In addition, 2% of 14-year-olds and 4% of 15-year-olds reported having used a class A substance at least once. 21

Young people in the criminal justice system

Although the relationship between criminal activity and substance use is complex, there is clear evidence that the prevalence of substance use is far higher in the youth offending population than in the general youth population. Data derived from the YOT AssetPlus3 indicate that most (76%) young people in the CJS use substances and 72% have a mental health need.

Data from the Juvenile Cohort Study22 show that 32% of young offenders score ≥ 2 on the ASSET tool for substance use, indicating that substance use is at least, in part, a reason for them associating in criminal activity, and 12% of young offenders score ≥ 3. Substance use is defined as the use of alcohol, traditional illicit substances or legal highs, as well as inappropriate use of prescribed medication. Although the relationship between substance use and criminal activity is complex, it is clearly a major issue in the youth offending population.

In the CJS, substance use and offending are related to other forms of disinhibitory behaviour, such as aggression and risk-taking. Young people involved in the CJS are a particularly vulnerable group, with a greater propensity to take risks that are likely to have a long-term impact on their future health and well-being. This is because young offenders often lead chaotic lives and face complex problems, including substance use, unsuitable accommodation and emotional or mental health issues. 23 Literacy levels among young people involved in the CJS are unacceptably low, and the vast majority of young people involved in the CJS have, in the past, been excluded from school. 24 As a result of the above risk factors, the list of negative consequences that result from substance use by young people is extensive and includes physical, psychological and social problems in both the short and the long term. In 2020, the number of young people receiving education in a PRU was just over 15,000,25 of whom 10% had been excluded from mainstream education because of substance use. Being excluded from school and educated in a PRU increases the propensity for young people to engage in criminal activity and 23% of new entrants to the CJS are educated in a PRU. 26

Young people who offend often experience a range of complex multiple risks and vulnerabilities, including neglect and abuse,27 substance use and related problems,28 and exclusion from school. 29 Research has shown that young people who offend are more likely to experience a range of inequalities in later life, for example worse physical health,28 early pregnancy in females,30 and higher rates of tobacco use and drug and alcohol dependence,29,31,32 reduced employment opportunities and economic hardship. 33 Indeed, there is widespread agreement that young people who offend are at an increased risk of health and social problems, making them one of the most vulnerable populations in the UK. 34 Furthermore, the UK has one of the highest youth custody populations in western Europe. 35 Epidemiological studies have shown that, in common with other vulnerable groups of young people, such as the homeless and those in care, young offenders are a hard-to-reach group from a health needs perspective, accessing physical and mental health services only in times of crisis, and that accessing these services is often associated with involvement with other agencies. 32,36,37

The Youth Justice System in England and Wales works to prevent offending and re-offending by those aged < 18 years. The latest available data indicate that there were 19,000 arrests of young people in 2019, which is an 82% drop from 2009. 38 Boys accounted for 83% of these arrests, and the average age of offenders was 15.3 years. Over the same period, there were 11,000 first-time entrants, that is, those receiving first reprimand or warning of community conviction, to the Youth Justice System, which is a reduction of 84% since 2009. 38 It is estimated that 38.5% of new offenders go on to re-offend after serving their initial sentence. 38

The ASSET tool is a standardised assessment tool that was developed within the CJS in England and Wales. The ASSET tool aims to identify the underlying causes of a young person’s offending behaviour to plan appropriate interventions. 3 The ASSET tool is often used on multiple occasions to help measure change in young offenders’ health and social needs and the risk of re-offending over time, and has been used with all young offenders in England and Wales since 2000. The ASSET tool examines 12 dynamic risk factors: (1) living arrangements, (2) family and personal relationships, (3) education, (4) neighbourhood, (5) lifestyle, (6) substance use, (7) physical health, (8) emotional health, (9) perception of self and others, (10) thinking and behaviour, (11) attitudes to offending and (12) motivation to change. The severity of each domain is rated on a scale of 0–4, with 0 being the least severe and 4 being the most severe, and a score of ≥ 2 is indicative that the domain contributes to the individual’s offending behaviour. 3

Previous research

To date, systematic reviews of interventions for substance-using offenders in CJS environments have not identified a clear evidence-based intervention strategy,39–41 but they have highlighted the paucity of good-quality research in the area and a lack of UK-based studies, with no scientifically rigorous studies focusing on young offenders. A recent meta-analysis of 22 studies42 synthesised the evidence regarding the use of motivational interventions for adolescents (aged 12–20 years) who engage in substance use. The results showed that, compared with TAU, the use of motivational interventions reduces the number of heavy alcohol use days by 0.7 days per month [95% confidence interval (CI) –1.6 to –0.02 days], substance use days by 1.1 days per month (95% CI –2.2 to –0.3 days), and overall substance-related problems by a standardised net mean difference of 0.5 (95% CI –1.0 to 0).

Developing and evaluating the RISKIT multicomponent intervention

The development of the RISKIT multicomponent intervention43 was based on two streams of work: (1) a participatory consultation with young people and (2) a review of the current research evidence. The theoretical perspective was informed by the social development model. 44–46 This approach suggests that the distal influences of socioeconomic status, biology, normative regulation and discipline are mediated through proximal influences on behaviour, which are identified as perceived opportunities for poor antisocial behaviour and perceived rewards for this behaviour. The social development model marries the ecological context of young people’s behaviour to an explanation of how this ecology influences their behaviour. It suggests that, even in the absence of a structural change to their health ecology, the provision of socioemotional and cognitive skills can help young people prevent or reduce risk-taking behaviour and suggests that the building of bonds with organisations promoting prosocial learning and opportunities is important in the reduction of risk-taking. The model provides a coherent and empirically validated approach that suggests that intervention approaches should be multicomponent and encompass knowledge and education, cognitive and learning skills, self-efficacy and motivation.

The participatory consultation was adapted from participatory action research47,48 and was carried out with a number of groups of young people. The aim of the exercise was to establish, with young people, what they perceived as risk-taking behaviour, why they took risks, the consequences of taking risks and how they perceived the problems could be addressed. The main themes in terms of risk-taking behaviour centred around criminal activity, substance use and sexual activity, and these activities were considered as being linked. The participants thought that prevention programmes that focused on the negative outcomes of risk failed to appreciate that risk-taking can be positive and can lead to positive outcomes, an issue highlighted by other research exploring the processes associated with risk-taking. 45,49 The young people identified the need for education regarding risks and consequences, but particularly highlighted the preference for interventions that provided skills and strategies to manage risk and the opportunity to discuss these skills with peers and to learn how to implement them. Interestingly, parental influences were not considered critical to any intervention, and many young people considered parental involvement to be inappropriate and unacceptable. The primary focus for the young people was not on eradication of risk-taking, but rather a focus on how risk could be reduced and how the negative consequences could be minimised.

Our second stream of work focused on the current evidence base that could inform the development of a multicomponent intervention. We consulted a number of existing reviews and research studies50–55 and found that, although there is a growing body of research in the field, there is a paucity of rigorously evaluated interventions, with the majority of research arising from the USA, with limited applicability to the UK. Of importance was what has been proven not to work, and this includes focusing on negative aspects of risk and abstinence from risk-taking behaviours. Promising intervention approaches include motivational interviewing and cognitive and socioemotional life skills training. In addition, there is emerging recognition of the importance of providing interventions in a structured manner and, with the young people’s preference for peer group interventions, the importance of managing the potentially negative effects of labelling and peer influence.

A synthesis of the participatory group views, the theoretical underpinnings and a review of the evidence was undertaken, and the RISKIT intervention model was developed as an approach that focuses on those who are vulnerable to the negative consequences of their risk-taking behaviour. The RISKIT intervention combines individual motivational interviewing sessions to target motivation and behaviour change with eight group-orientated life skills sessions that cover a variety of areas, including identifying and managing risk, communication skills, assertiveness training, anger management, preparing for behaviour change and sexual health. In addition, the group sessions focus on identifying resources within the community that could be of benefit for the young people and provide opportunities to access these resources.

An initial feasibility study was undertaken followed by a larger quasi-experimental study43 in which the RISKIT intervention was delivered across schools in Kent to 226 adolescents who were identified as engaging in excessive risk behaviour. Consent rates in the eligible population were high (80%), with almost all adolescents attending at least part of the intervention and 74% of adolescents attending all the intervention sessions. Follow-up rates were high, with 82% of adolescents being followed up at 6 months. At this point, 32% of the intervention group had reduced their risk-taking behaviour to a point where it was of no further concern and significant improvements were observed in terms of number of days abstinent from alcohol and other substances, indicating a positive effect on this domain. Participant views were positive, with high levels of engagement and satisfaction, and a general view that the intervention had been useful in developing new skills, was informative and could lead to changes in behaviour. Delivery of the model was sustainable, but required the input of specialist, rather than generic, staff, and a full economic evaluation of cost-effectiveness was not undertaken. The most recent outcomes for the RISKIT intervention, delivered in school settings over 2014 and 2015, indicated a 92.9% completion rate for those youths identified as being eligible, a 37.8% reduction in any alcohol consumption, a 24.6% reduction in using other substances and, for those who continued using illicit drugs, a reduction of 27.2% in drug-using days. In addition, improvements were observed in terms of mental health and well-being. Further to our pilot study, the RISKIT intervention was tested for feasibility in both custodial and community CJSs, with high levels of satisfaction on the part of the participants. A recent study in the community setting,56 commissioned by Kent Police as an element of their diversionary strategy, targeting young offenders committing a drug-related offence (n = 175), suggested that consent and engagement was high, with 90% of young offenders consenting, 100% of young offenders attending at least one session and 95% of young offenders attending all sessions. Further analysis at 6 months post intervention indicated a 50% reduction in re-offending rates, that is, a significantly greater reduction than that observed in a similar population who received youth justice interventions only. As part of this work, we adapted the intervention in terms of delivery by providing it over two half-day sessions over consecutive weeks on weekends, rather than the 8-weekly 1-hour sessions provided in the school-based study. 1

Aims

Our aim was to evaluate the clinical effectiveness and cost-effectiveness of the RISKIT-CJS intervention, compared with TAU, in reducing substance use for adolescents engaged with the CJS.

Primary objective

• To conduct a prospective pragmatic RCT to evaluate the effectiveness of the RISKIT-CJS intervention, compared with TAU, for substance-using adolescents involved in the CJS.

Secondary objectives

• To evaluate the cost-effectiveness of the intervention compared with TAU.

• To explore participants’ and criminal justice staff’s experience of the intervention and the acceptability of the methods employed.

• To assess the fidelity with which the intervention was conducted and to explore the role of fidelity, therapeutic alliance and baseline psychological factors on the outcomes observed.

• If the intervention was shown to be effective within the parameters set, to develop a protocol for dissemination and integration of the intervention in current practice.

Primary outcome

Our primary outcome measure was per cent days abstinent (PDA) from substance use in the 28 days prior to the 12-month follow-up and this was measured using the Timeline Followback 28 (TLFB28),57 which is a valid and reliable tool for assessing the quantity and frequency of substance use over time periods ranging from 1 to 365 days. The outcome has been validated for use in adolescent populations58 and recent pilot work has indicated high levels of agreement between the shorter, 28-day, and longer, 90-day, reference period.

Secondary outcomes

In addition to PDA, the TLFB28 allows for the derivation of a number of secondary outcomes over the period, including quantity and type of substances consumed. The TLFB28 is completed by a trained member of research staff and takes approximately 20 minutes. The outcome was measured at baseline and at 6 and 12 months. Mental health and well-being was assessed using the WEMWBS. The WEMWBS is a 14-item self-completed scale that addresses different aspects of eudemonic and hedonic mental health and well-being. The scale has established valid reliable psychometric properties in adolescent populations5 and is sensitive to change. 59 The WEMWBS instrument is highly correlated with other measures of psychological health and well-being, including the Strengths and Difficulties Questionnaire and the General Health Questionnaire. The WEMWBS was administered at baseline and then again at 6 and 12 months.

Motivational state and readiness to change substance use behaviour was measured using SOCRATES-7DS. The using SOCRATES-7DS contains 20 items and has four items for each stage of precontemplation, contemplation, preparation, action and maintenance. When an individual item is missing, then this results in a missing score for the relevant domain of behaviour change.

The Brief Situational Confidence Questionnaire (BSCQ) was used to assess self-efficacy. 60 The BSCQ comprises eight visual analogue scales to record confidence (on a scale of 0–100) in resisting the temptation to use drugs in a variety of situations.

Health-related quality of life was measured using the KIDSCREEN-10 Index. 61 This unidimensional measure represents a global score incorporating elements of physical and psychological well-being, autonomy and parent relations, peer and social support, and school environment. The KIDSCREEN-10 Index comprises 10 items rated on a Likert scale from 1 to 5, which are summed to give a total score. In addition, the KIDSCREEN-10 Index includes an overall self-reported assessment of general health (i.e. poor, fair, good, very good or excellent). The total score for the 10 items is calculated when there are no missing values for any individual item.

Process outcomes

Therapeutic alliance was measured at the end of the intervention using the Therapeutic Alliance Scale for Children: Youth version (TASC-r). 62 The TASC-r distinguishes between the affective bond and client–therapist collaboration. Items are rated on a scale from 1 (not true) to 4 (very much true) and are summed to give a total score. When fewer than four items are missing, the total is estimated using mean substitution for missing values.

All individual motivational interventions were recorded and a random sample of 20% (stratified by region, age and therapist) were selected and assessed by independent raters using the Behaviour Change Counselling Index (BECCI)63 to assess fidelity and quality. Overall scores were calculated by summing the checklist item scores and by dividing by the number of items to give a mean score. Mean substitution was used when an item on the checklist was not applicable.

Attendance at each of steps 1 to 4 of the intervention was recorded for each participant. Each interventionist delivering the RISKIT-CJS intervention was assigned a unique identifier and this was recorded for each step of the intervention for each individual.

Economic outcomes

The economic outcome measures addressed the costs of delivering the interventions, changes in health utility in the 12 months after randomisation and the costs associated with participants in the 12 months after randomisation. Costs associated with delivering the intervention were derived using a micro-costing approach, accounting for the actual costs, including associated training, facilities and overheads, and management costs. Health utility was assessed using the self-completed EuroQol-5 Dimensions, five-level version (EQ-5D-5L), and the Child Health Utility – 9 dimensions (CHU-9D),64 which were assessed at baseline and at 6 and 12 months. Service utilisation on the part of the participant was assessed using a specifically designed Client Receipt Service Inventory65 previously used with the adolescent population. Service use was assessed from a wide public sector perspective, encompassing health and social care, criminal justice, education and employment service utilisation.

Qualitative outcomes

Twelve RISKIT-CJS groups were purposefully selected according to geographical region and group dynamics, and a group discussion was conducted at the end of step 3 of the intervention, that is, the last group session attended. The group discussions were facilitated by experienced qualitative researchers, trained in both qualitative methods and the participatory rapid appraisal approach. 66 The researchers had experience of the RISKIT-CJS intervention but had not been part of the session delivery, enabling young people a space to share both positive and negative responses. Two researchers joined the selected groups at the close of the last group session and this created a distinction between the role of the researchers and interventionists. The approach employed was a participatory rapid appraisal66 to elicit in-depth exploration of the acceptability and perceived effectiveness of the programme and of the different elements within it.

To explore the RISKIT-CJS intervention from the perspective of the practitioners, 23 semistructured telephone interviews were conducted 2 weeks after the final motivational interviewing intervention. The semistructured interviews and field notes maintained by interventionists were used to explore a number of key objectives, feasibility, acceptability and perceived effectiveness of the programme.

Telephone interviews were carried out with a purposively selected sample of practitioners, including staff working in YOTs, PRUs and SMTs, who were involved with RISKIT-CJS programme and were chosen according to their profession and region. The aim was to explore the impact of the RISKIT-CJS intervention from the perspective of the YOT staff who worked with the target population. It was proposed that there would be 24 semistructured telephone interviews with staff across the participating teams and that these should be conducted 4 weeks after the final step 4 of the motivational interviewing intervention.

Study setting and population

Young people aged 13–17 years (inclusive) were targeted. Young people were recruited from a mixture of YOTs, PRUs and SMTs between September 2017 and June 2020. Recruitment was conducted in four geographical areas of England (i.e. North East, South East, London and North West).

In the YOTs, all young people are routinely screened using the ASSET tool, and research staff approached any young person who scored ≥ 2 on the substance use domain, indicating a relationship between criminal behaviour and substance use. In PRUs, school staff identified potential participants who were subsequently assessed by research staff to ensure that they had been involved with CJS and were engaged in substance use. The SMT cohort included young people who had recently come to the attention of the police for being involved in criminal activity and who were also found to be in possession of illegal substances. As part of a scheme to divert these young people from criminal justice services, they were referred to a young person’s SMT for assessment and, when necessary, intervention.

Eligibility checks were conducted by a researcher trained in the process and who had previously worked with adolescent populations. Young people who were eligible were provided with detailed trial information (see Report Supplementary Material 2) both verbally and in writing and were asked to consider participating. It was made clear that a participant could withdraw from the intervention or the trial completely at any stage and withdrawing would have no effect on any intervention they would receive as TAU. All young people, irrespective of eligibility and consent, were recompensed for their time with a £10 voucher. If a participant was willing to consent, then the researcher decided whether or not the young person aged ≥ 16 years met the criteria for being Gillick competent and, if they were, then consent was taken. For young people aged < 16 years and young people aged ≥ 16 years and considered not to be Gillick competent, assent was taken from the young person and formal consent taken by contacting the primary caregiver, who was provided with a copy of the trial information sheet. The young person was asked to consent both to the trial and for access to offence records held on the Police National Computer. After consent had been taken, the researcher conducted the TLFB28 on paper. The young person completed the rest of the baseline assessment on a tablet device using specifically designed software. The researcher was available to aid any young person who had difficulty in understanding the questions. After the completion of the baseline assessment, randomisation was conducted automatically through the tablet device and the young person was informed of their allocation. Once sufficient numbers of young people had been allocated to the intervention to form a group, usually six or more, but not exceeding eight, then the list of group members was reviewed by staff within centres to identify any potential risks, such as a lone female in a dominant male group, group members known to be engaged in violent conflict or associated with different gangs and very vulnerable participants. If any risks did arise, then the young person was considered for another group at the same setting. If no other group was available, then the young person received the individual components of the intervention. Follow-up was conducted as close to the 6- and 12-month follow-up as possible. Young people had a choice in how they were followed up: in person with a researcher, by telephone with a researcher or by completing a paper-based questionnaire by e-mail or post. Once follow-up was completed at 6 and 12 months, participants were recompensed for their time with a £10 voucher at both time points. This both recognised the contribution the participant made to the research and was designed to reduce attrition over time. 67

Inclusion and exclusion criteria

To maximise the generalisability of the trial population, inclusion and exclusion criteria were kept to a minimum. Participants were included if they were aged 13–17 years (inclusive), if they had recently engaged in substance use and had been involved in the CJS, and if they were engaged in a participating YOT, PRU or SMT.

Participants were excluded if the severity of their substance use was such that a referral to specialist services for detoxification was required, if the severity of criminal involvement was such that the young person was likely to be incarcerated during the intervention or follow-up period, or if the young person was currently on a court-mandated order with alcohol or substance use abstinence as a prerequisite.

Randomisation

Randomisation was conducted at the level of the participant by an independent secure randomisation service, using random permuted blocks of random length. Strings developed independently were encrypted and stored, with the tablet devices and allocation revealed only when a participant had been judged eligible, provided written consent and completed the baseline assessment. Copies of the strings for audit and quality assurance purposes were held in an encrypted database by an organisation not associated with the research. Randomisation was stratified by site, sex and age group [i.e. 13–15 years (inclusive) vs. 16–17 years (inclusive)].

Staff identified to deliver the RISKIT-CJS intervention

We worked in collaboration with youth substance misuse services provided by Addaction, now renamed We Are With You (London, UK). Staff delivering the interventions were paid, experienced youth workers and youth substance misuse workers who had been trained and accredited, and had experience of delivering motivational interviewing. Many of the staff had experience of delivering the schools version of the RISKIT intervention, which is similar to the RISKIT-CJS intervention but is delivered for 1 hour per week over an 8-week period.

RISKIT-CJS intervention training and support

Staff were invited to participate in a 2-day training course as part of their continuous professional development. In addition to providing an overview of their roles and responsibilities relating to the trial, the 2-day course explored the theoretical underpinning of the RISKIT-CJS intervention, detailing delivery of each programme element, including managing group dynamics, managing ambivalence, identifying risks and safeguarding. Staff engaged in written coursework and role play, and had an opportunity to conduct motivational interviews and manage groups while being observed by an experienced senior practitioner. Only when a staff member had been deemed competent by the senior practitioner were they allowed to deliver the RISKIT-CJS intervention. Intervention delivery was monitored on an ongoing basis by senior staff who observed practice in individual and group sessions, providing ongoing guidance and supervision.

The RISKIT-CJS intervention

The RISKIT-CJS intervention was based on a similar individual and group intervention developed in collaboration with young people and delivered in school settings. 43 The intervention delivered in the trial consisted of four distinct steps. The first step, delivered immediately after randomisation, was an individual face-to-face session with a trained interventionist, lasting approximately 40 minutes. This session used techniques associated with motivational interviewing to discuss substance use, to explore the role substance use plays in risk-taking behaviour or emotional dysregulation, to explore and support behaviour change, and to enhance motivation to engage with the intervention. Step 2 was a group session delivered about 1 week after step 1. The group session, delivered over half a day at a convenient location for the participants, employed a group cognitive–behavioural therapy approach, employing group discussion and interactive whiteboards. The aim of the session was to provide both psychoeducation and skills development, encompassing understanding substance use and associated harms, understanding triggers to substance use, developing strategies for harm minimisation and reducing consequential risks, exploring techniques to divert or distract from substance use, and exploring substance use and sexual health risks. The actual topics and depth covered were decided at the beginning of the session to meet the needs of the group. Step 3 occurred usually 1 week after step 2 and took a similar group approach, but with a greater emphasis on skill development. Topics included communication strategies and assertiveness training, managing anger and anxiety, mindfulness and planning for the future. Two interventionists covered each group, with one interventionist delivering the content and the other encouraging engagement within the group. Step 4 was conducted about 1 week after step 3 and comprised a 40-minute face-to-face session conducted in a mutually convenient location. This session, using a similar motivational interviewing approach as step 1, explored outstanding barriers to change, managing expectancy and enhancing self-efficacy. In addition, each step 4 intervention was tailored individually to consolidate what had been learned from the group sessions and to explore opportunities to engage in local services and provision to encourage pro-social behaviour. Participants were not paid to attend any of the sessions, although they were reimbursed for any travel expenses and provided with refreshments.

Assessing intervention fidelity

Intervention fidelity relates to the extent to which an intervention is true to the underlying therapeutic theory. Researchers need to be able to determine whether or not the intervention is delivered as intended and, although complete manualisation of an intervention goes against many therapeutic principles, it is important to be able to assess the content of an intervention. High-fidelity delivery of an intervention makes it more appropriate to draw causal conclusions regarding any intervention and effect observed; it also makes any intervention easier to replicate in practice. In addition to keeping a log of what subjects were covered in the group sessions, interventionists sought the consent of young people to audio-record their individual sessions. From the sessions recorded, a 20% random sample was drawn, stratified by sessions (1 or 4), interventionist and site. These sessions were listened to by two independent raters. The raters scored the sessions using BECCI,68 that is, a tool specifically designed to measure the micro-skills of behaviour change counselling and motivational interviewing.

The BECCI focusses on the interventionist’s consulting behaviour and attitude rather than a self-report. Scores are provided on a scale of 0–4 (where 0 = not at all, 1 = minimally, 2 = to some extent, 3 = a good deal and 4 = a great extent). Ratings consider 15 domains of practitioner skill and are completed in accordance with the rater manual. Two raters considered each recording and made decisions regarding the overall score. When significant deviation regarding the allocated rating occurred, the two raters listened to the recording together and came to a consensus on the score. Ratings were incorporated into the assessment of fidelity reported in Chapter 3.

Changes to the original protocol

As discussed in Chapter 1, data on the criminal justice activity of participants had not been made available to the research team, and this meant that the economic analysis could not be conducted from the public sector and societal perspective and, therefore, analysis conducted from the health and social care perspective is presented.

Quality of life

The primary outcome measure in the economic evaluation is QALYs derived from the EQ-5D-5L. The descriptive system, which comprises five dimensions81 (i.e. mobility, self-care, usual activities, pain/discomfort and anxiety/depression) was converted into utility scores at baseline and at the 6- and 12-month follow-up interviews, using the EQ-5D-5L value set developed for the general UK population. 82,83 Assuming that health status changes gradually over time between the follow-up points, QALYs were calculated from the utility scores using the area under the curve method. 84

Costs of using health and social care

The services used by participants were informed by other studies with similar populations. 85,86 Each participant was asked, at baseline and at 6 and 12 months, to indicate how often they had used services in the 6-month period prior. Each contact with services was allocated a unit cost. All costs were measured in 2019–20 prices either by directly relying on unit costs measures in the NHS reference costs or via established sources of unit costs of health and social care87,88 and, when necessary, including appropriate adjustment for inflation. Table 15 shows the costs associated with each resource item, and Table 5 in Report Supplementary Material 5 provides corresponding references and explanations for these costs. The cost of service per participant was calculated as the product of units used and cost per unit, and this cost was summed across all service categories.

Cost of the intervention

The resources assessed for the delivery of the intervention included those costs that would be incurred if the intervention was delivered in practice. Therefore, the resources associated with the scientific evaluation were excluded. In a similar manner, resources that were common across both arms of the study (i.e. TAU) were also excluded.

The costs of the RISKIT-CJS intervention have been calculated based on the actual resource use and the corresponding national unit costs for individuals with qualifications required to deliver the intervention (e.g. substance misuse/youth workers). These workers were invited to training sessions to obtain the necessary skills to deliver the intervention. The cost of the training component involved the opportunity costs of time devoted to the training by all the participants, trainers and the venue hire costs. It was calculated based on the actual costs that were incurred and then divided over the number of participants allocated to the RISKIT-CJS intervention arm at the start of the trial.

The intervention took place over 2 consecutive weeks and included the four steps described in detail in Chapter 2, Trial processes, that is, (1) an individual face-to-face 40-minute session with the interventionist; (2) a group session over half a day (4 hours) at a convenient location (by two interventionists); (3) a group session over half a day (4 hours) conducted a week after step 2, with the same participants, same staff and at the same location; and (4) a concluding individual face-to-face 40-minute session with an interventionist.

Total cost

The total cost for each participant in the RISKIT-CJS intervention arm was estimated as the cost of delivering the intervention, including costs allocated for training and supervision, divided by the number of participants receiving it and the individual-level resource use costs at follow-up across health and social care. In the control arm, the costs were estimated as individual-level resource use costs at follow-up across health and social care.

Data

Data included in the analysis relied on responses from participants who were eligible and consented to participate in the trial. Primary analysis was conducted using a per allocation approach, with participants’ outcomes analysed according to the group they were allocated to rather than the treatment they received.

For the incremental cost-effectiveness analysis, the differential in the arithmetic means of individual annual costs89 and the differential in the adjusted means of annual cost have been estimated using the multiple regression analysis, controlling for sex, site and baseline costs, and this concurs with the approach taken for the clinical analysis described.

Managing uncertainty

As the costs and outcomes data are usually not normally distributed, we employed a non-parametric bootstrap to determine the level of sampling uncertainty in incremental costs and benefits and, hence, cost-effectiveness/cost–utility. 90 Non-parametric bootstrapping is a sampling approach that generates multiple replications of ICERs by sampling with replacement data from the original data set. The bootstrap method is preferable for the skewed distributions, as it does not rely on the parametric assumptions about the underlying data. 89 One thousand bootstrap replications91 produced ICERs for the CEP and CEAC to explore the uncertainty. 92 The latter allowed the graphical representation of the probability that the RISKIT-CJS intervention is preferred to TAU at different values of WTP. 93

Sensitivity analysis and missing data

The cost-effectiveness analysis could be sensitive to the quality of data, predominantly when the distribution of the cost data is affected by outliers, various assumptions that are necessary to construct the measure of the total cost and the measures from which the QALYs are derived. Therefore, the primary analysis will be subject to sensitivity checks along several dimensions, including (1) exclusion of outliers in terms of costs; (2) replacement of individually varying intervention costs with fixed costs based on the total cost of intervention spread across all of the participants in the treated group; and (3) replacement of actual training costs incurred within the trial, with the training costs spread over a larger number of participants and assuming that, in real life, one trained interventionist would be able to deliver the intervention to six cohorts of participants on average. Each of these checks has been undertaken as a one-way analysis, followed by the corresponding analysis taking into account sampling uncertainty.

The clinical analysis showed a significant proportion of missing data for the primary and secondary outcomes. For the PDA from all substances and WEMWBS score, the percentage of data missing at 12 months was 45% and 55%, respectively, and, consequently, the multiple imputations were not performed. The situation is even more problematic with the EQ-5D-5L combined with the cost measures, as only 70 (27%) individuals in the control group and 62 (25%) individuals in the treatment group provided all of the data needed for the cost-effectiveness analysis. As a result, the primary economic analysis is based on complete cases without imputations. However, multiple imputations have been performed for costs and outcomes (following Faria et al. 94) and within the scope of the sensitivity analysis of uncertainty. Eight scenarios are explored,95 with the health-related quality of life and the total costs being 10% lower or higher in one or both arms as compared with the MAR setting.

Costs

Table 16 summarises the intervention costs. As the screening and recruitment costs and the TAU costs were the same for both the intervention and the control groups, they are omitted from the incremental cost-effectiveness analysis.

The RISKIT-CJS intervention consisted of two individual sessions, which each lasted 40 minutes each and cost £32.67 per participant, and two group sessions, each lasting for 4 hours and costing £428 per session. The number of participants in the group sessions varied from one to nine, with an average of 4.64 (SD 2.29) for the first group session and an average of 4.16 (SD 2.28) for the second group session. The training cost was divided by the total number of participants allocated to the RISKIT-CJS intervention, and this was £49.24 per participant. Therefore, the overall cost of delivering the RISKIT-CJS intervention amounts to £267.28 (SD £162.37) per participant.

Service use from the health and social care perspective, at baseline and at 6 and 12 months by allocated group, is presented in Table 17. Respondents did not report any family and individual therapist visits and so these categories are excluded.

Among participants in education, 33% in the control group compared with 16% in the RISKIT-CJS intervention received at least one exclusion, with the difference being statistically significant at a 5% level. Furthermore, at the 6-month follow-up, participants in the RISKIT-CJS intervention group who were working reported more working hours than participants in the control group [25.27 (SD 14.23) hours vs. 12.15 (SD 11.49) hours, respectively], with the difference being statistically significant at the 5% level.

The costs of health and social care service use were estimated as the products of each resource use category in Table 17 and the corresponding unit costs reported in Table 15. These costs were then added to the cost of intervention from Table 16 to obtain the total cost for each participant in both the treatment group and the control group over the year (Table 18). These calculations show that the resources used in the health and social care sectors are the most significant contributor to the total cost. The mean health and social care cost for the RISKIT-CJS intervention group over the first 6 months was £91.96 (SD £332.25), compared with the £901.32 (SD £6246.40) in the control group, resulting in a difference of –£809.36 (95% CI –£2381.45 to £762.73). At the 12-month follow-up, the health and social care cost was £201.12 (SD £1034.58) for the RISKIT-CJS intervention group and £394.69 (SD £2643.03) for the control group, a difference of –£193.56 (95% CI –£901.50 to £514.37). Therefore, the total cost over the 12 months was £560.36 (SD £140.26) for the RISKIT-CJS intervention group and £1296 (SD £1055.95) for the control group. The residual 12-month costs after adjustment for the RISKIT-CJS intervention group were £279.11 (SD £266.38) and £848.47 (SD £993.36) for the control group, resulting in a difference favouring the RISKIT-CJS intervention of –£569.37 (95% CI –£2716.48 to £1577.74). This difference indicates that the RISKIT-CJS intervention resulted in cost savings from the health and social care perspective, compared with the TAU. However, the CI is wide and includes zero, and so the difference is not statistically significant at the 5% level.

Outcomes

Table 19 shows the differences in the EQ-5D-5L-based utility scores at baseline and at 6 and 12 months between the RISKIT-CJS intervention group and the control group, as well as the calculated QALYs. The utility scores at the three time periods are used to calculate the QALYs using the area under the curve method. The residual mean changes in QALYs after adjustment were 0.3756 (SD 0.0949) and 0.3842 (SD 0.0928) for the RISKIT-CJS intervention group and the control group, respectively. The resulting incremental changes in QALYs between the RISKIT-CJS intervention group and the control group were –0.0427 (95% CI –0.0958 to 0.0105) without adjustment and –0.0086 (95% CI –0.0409 to 0.0238) after adjustment.

Results of the cost-effectiveness analysis

Table 20 brings together the incremental changes in QALYs with the incremental changes in costs to produce the ICERs. In both cases, within and without adjustment, both mean costs and mean QALY gains were smaller in the RISKIT-CJS intervention group than in the control group. However, the Fieller 95% CIs are wide, meaning that we cannot be 95% confident that the two approaches (i.e. the RISKIT-CJS intervention and control) differ in value.

We used a bootstrap resampling method, consisting of 1000 replications, to assess the uncertainty surrounding the cost-effectiveness estimates to construct the CEP and the CEAC. 96 The adjusted CEAC is presented in Figure 8.

FIGURE 8.

(a) Cost-effectiveness plane; and (b) CEAC, adjusted for baseline utility and costs, age, sex and site at 12 months (completed cases).

The CEP is divided into four quadrants and allows us to visualise the variation within the resulting bootstrapped sample of ICERs. The horizontal axis represents incremental effects with positive ones, indicating that the RISKIT-CJS intervention is more clinically effective than the control. The adjusted mean effect becomes smaller in absolute value, with the combinations of incremental costs and effects distributed across all four quadrants and the majority in the south-west and south-east quadrants, suggesting cost saving with no effect on QALYs. The CEAC shows a 69% probability that the RISKIT-CJS intervention is cost saving when the WTP is equal to zero. At the NICE-recommended thresholds, the probability that the RISKIT-CJS intervention is more cost-effective than control is 60–63%.

Sensitivity analysis

Sensitivity analyses were conducted to evaluate the robustness of the findings in the primary analysis.

Table 6 in Report Supplementary Material 5 provides the results of the incremental cost-effectiveness analyses for all three sensitivity tests.

Results of the secondary economic analysis

The secondary economic analysis involves two modifications. The first modification is related to the compliance of the participants in the treated group with the treatment indicated in the research protocol. The second modification is related to the construction of the QALY gain based on the CHU-9D measure rather than the EQ-5D-5L. The results are summarised in Report Supplementary Material 5, Table 7, with the graphical representation of the CEPs and CEACs in Report Supplementary Material 6, Figure 30.

Cost–consequence analysis

As the completed cases for the cost data are available for only 132 participants, the estimation of the effect of the intervention on outcomes is repeated for the sample with cost data to enable meaningful comparison between the costs and consequences. However, complete cost data for the primary and secondary clinical outcomes were available for only 128 participants. Table 21 presents the total costs and the primary and secondary clinical outcomes for the RISKIT-CJS intervention group and the control group, as well as the difference between them.

Cost-effectiveness analysis for other outcomes

Incremental cost-effectiveness ratios were estimated for the primary outcome of PDA from substances in the last 28 days at the 12-month follow-up and the secondary outcome of WEMWBS score for 128 complete cases, after adjusting for a set of covariates (i.e. sex, age, site, and baseline cost and outcome). Table 22 shows ICERs and Fieller’s 95% CIs. As can be seen, the latter are unbounded from the right and, therefore, for the PDA all substances, for WTP > £971, there is no 95% CI for the RISKIT-CJS intervention and TAU to differ in cost-effectiveness. Similarly, for the WEMWBS score, for WTP > £80, the two interventions do not differ in cost-effectiveness. It is not common in the literature to construct CEPs and CEACs for outcomes other than health economic ones. In addition, these measures do not have recommended WTP thresholds. However, exploration of these outcomes, given their clinical importance, may present additional evidence to the primary health economic analysis findings.

Handling missing data

Given the proportion of missing data and the low likelihood that data are MAR, a decision was made to build the primary economic analysis on the available data, limiting it to those participants who responded to all the questions needed to construct cost and outcome variables. Unfortunately, it cannot be considered an intention-to-treat analysis because randomised patients with follow-up data were excluded if they did not provide information at any one time point. 97 However, as a very high proportion of the data needed for the cost-effectiveness analysis in the RISKIT-CJS trial were missing, it is very difficult to rely confidently on imputation methods. Therefore, imputations are employed in the sensitivity analysis only. Furthermore, multiple imputations have been chosen over single imputation with mean values because the former underestimate the uncertainty. 94

Baseline values were available for 495 participants and were not imputed for this analysis.

Multiple imputations by chained equations were performed for QALYs and health and social care costs at the 6- and 12-month follow-ups, using predictive means matching and baseline variables of sex, age, type of service, utility score and primary outcome at baseline. The number of imputations was set to 50, stratified by arm. Using imputed variables, the QALY gains were calculated using the area under the curve, and the total costs were calculated as the sums of the health and social care costs over the previous 6 months (reported at the 6- and 12-month follow-ups).

Figure 9 offers CEPs (see Figures 9a–h) and CEACs (see Figure 9i) for eight scenarios, each of which is described by the multiplicative factor (i.e. QALYTAU, QALY_RISKIT, CostTAU, Cost_RISKIT). Therefore, the first scenario (see Figure 9a) is the base-case scenario with imputations performed under the MAR assumption. The rest of the scenarios represent various combinations to reflect possible patterns of MNAR. For example, scenario 5 differs from scenario 1 (MAR) only in terms of the QALYs for the treated group, assuming that those of the treated participants who are missing have 10% lower QALY gain over the year, as compared with the MAR assumption. Scenario 6 represents the reverse situation: participants in the control group for whom the information is missing have a 10% lower QALY gain. The CEACs and CEPs are constructed from a bootstrapped data with 1000 replications.

FIGURE 9.

Cost-effectiveness planes and CEACs, all adjusted for baseline utility and costs, at 12 months, MAR multiple imputations (scenario 1) and sensitivity for MNAR (scenarios 2–7). MNAR scenarios are defined by the four parameters, which represent multiplicative factors to change the imputed parameter to reflect MNAR possibility. The outcomes in the order of appearance are QALYs for the control group, QALYs for the treatment group, total cost for the control group and total cost for the treatment group. (a) CEP, scenario 1; (b) CEP, scenario 2; (c) CEP, scenario 3; (d) CEP, scenario 4; (e) CEP, scenario 5; (f) CEP, scenario 6; (g) CEP, scenario 7; (h) CEP, scenario 8; and (i) CEACs.

Figures 9a–h shows the CEPs for all eight scenarios. At first glance, it seems that the CEP based on the MAR imputation (see Figure 9a) provides similar evidence to the primary analysis based on the complete cases shown in Figure 8. However, the corresponding CEAC (see Figure 9i) shows a different picture, with a much higher probability of the RISKIT-CJS intervention being more cost-effective than TAU within the NICE-recommended threshold range of £20,000–30,000, although still not statistically significant. However, the value of the MNAR analysis lies in the comparison between the scenarios. Note that six out of eight scenarios produce a very similar result.

Nevertheless, two of the scenarios stand out, both of which refer to the pattern in the missing outcome measure. First, if participants who dropped out from the treatment group or chose not to answer EQ-5D-5L questions had, on average, a 10% lower QALY gain over 1 year (see Figure 9e), then the RISKIT-CJS intervention has a very low probability of being cost-effective. However, with the same costs, if the participants in the control group who dropped out or chose not to answer the EQ-5D-5L questions had, on average, a 10% lower QALY gain over 1 year (see Figure 9f), then the RISKIT-CJS intervention would have been cost-effective.

As the results are sensitive to the exclusion of outliers, the exploration of uncertainty was also performed for the sample without the outliers. The results for this analysis are presented in Report Supplementary Material 6, Figure 32. For all the scenarios but one, the majority of the incremental costs–incremental outcome pairs are located in the north-west quadrant, which means that the RISKIT-CJS intervention is associated with a lower effect at an increased cost. The CEACs show close to zero probability of the RISKTIT-CJS intervention being cost-effective. Only scenario 6 (see Figure 9f), leads to a probability of 50–70% that the intervention is cost-effective within the NICE-recommended threshold range.

Data sources

The qualitative study used four core data sources: (1) records of meetings with a purposive sample of participants, (2) focus groups with participants, (3) interviews with practitioners and (4) field notes taken by interventionists. Table 23 shows the sources of qualitative data by type and site.

The first data source was notes taken in meetings between the practitioner and young person at baseline and at each follow-up point. These notes recorded the participant’s attitudes towards, and views of, the RISKIT-CJS programme. We purposively sampled the notes to achieve data saturation for each of the following criteria: setting, site and sex. The number of participants sampled was 59 (PRUs, n = 43; YOTs, n = 16), mean age was 15 years (range 13–17 years) and 32% of participants identified as female.

The second data source was participatory research focus groups that ran at the end of the second group session. We employed participatory rapid appraisal tools. 98 Across the sites, 14 focus groups were conducted, involving 40 young people. Three groups were conducted in London, five in the South East and six in the North East, with 10 focus groups taking place in a PRU and four within a YOT.

The data were collected through verbal group discussions, while being aided and recorded using written materials, such as flip charts and Post-it^® Notes (3M, Bracknell, UK). The data included short comments about the ‘good’ and ‘bad’ bits of the programme; ranking the programme elements using different coloured dots on a flip chart; rating programme elements as a timeline, with participants drawing a line across a flip chart on which a horizontal axis was pre-drawn (their line showed how their experience of the programme in terms of how useful or not it was over time); and review of risks using a ‘ladder of risk’ of young people’s behaviour (arrow-shaped Post-it Notes were placed on the ladder to show the extent to which the programme had, in the young people's view, influenced their risk behaviours). To address potential literacy issues within the groups, research staff provided support to enable all participants to take part in the discussion; this included taking notes of comments and verbally reading the comments to the group.

A third source of data was face-to-face or telephone interviews with practitioners. These semistructured interviews were conducted with purposively selected informants from the CJS, YOTs, PRUs and early help, as well as interventionists, headteachers, administrative staff and social workers. All of these informants worked with young people within the participating settings and played a role in referring or facilitating the RISKIT-CJS programme in their setting. Overall, 23 interviews were conducted: 10 with interventionists, seven in PRUs and six in YOTs. Each interview was recorded and lasted between 45 minutes and 1 hour.

The fourth source of data was field notes taken by research staff. These included observations made while facilitating the focus groups (e.g. elaborating on brainstorming and discussions of the strengths and weaknesses of the programme elements, and the association with risk behaviour). Field notes were also taken on contextual issues at the intervention sites that were observed in setting up and implementing the project. In total, 18 such field notes were compiled and analysed. All data collected were anonymised, transcribed verbatim or photographed and stored in NVivo (QSR International, Warrington, UK) for analysis.

Data analysis

To analyse the data, we used the approach of ‘adaptive coding’. 99 NVivo11 software acted as a data management tool, which enabled us to analyse various types of data in one place. Four researchers completed the coding of all the data. The perceived contexts and mechanism of action were explored by starting with a list of provisional orienting codes, which were developed before data coding began. Our approach of ‘adaptive coding’ was based on both our pre-existing understanding of the underlying theory, that is, the social development model,43,45 which informed the three qualitative research questions, and researchers’ experience of being involved in the implementation and the delivery of the programme. We also identified new ‘grounded’ codes based on relevant items found in the data that were not included in the provisional codes.

In answering the third research question on implementation, we combined our adaptive coding of the data with normalisation process theory (NPT)100 as a sensitising tool to consider issues of translation of the RISKIT-CJS intervention into routine practice. NPT is an implementation theory, developed within the context of health-care innovations, which largely focuses on understanding the implementation, embedding and integration of new technologies and organisational innovations. 101 In this context, it was used to provide a theoretical basis for exploring the implementation of the RISKIT-CJS intervention into routine practice in a YOT or PRU setting, and this demonstrated the adaptability of NPT for use in different settings.

Using the RISKIT-CJS theoretical model in conjunction with the NPT framework enables analysis of how the likely presence of CMO configurations could either facilitate or hinder implementation into routine practice. NPT challenges researchers to consider how their intervention might become normalised in routine practice (Table 24). 102

Question 1: what are the perceived strengths and weaknesses of the programme with respect to what seems to work and what does not work?

Question 2: what is the perceived usefulness and acceptability of the intervention?

Question 3: what is the potential for implementing the intervention in routine practice?

Area

Rurality/urbanisation of the area of implementation affected the operation of the programme by determining how easy it was to get participants to the places where the intervention was being delivered at the right time and how easy it was to connect them to other local services for additional support. In more rural areas, the greater distances between programme participants and services made both of these aspects of the programme more difficult.

The levels of deprivation or affluence of the areas in which the programme operated also affected how it worked. More affluent areas tended to offer more opportunities for support and onward progression for young people than did the more deprived areas, and this is partly because there were more employment opportunities for older programme participants in the more affluent areas. However, there has also been an effect of the politics of austerity in reducing funding for council services most severely in the most deprived communities,111 and this means that councils in the most deprived areas have been the least able to protect provision of non-statutory support services for the types of young people with complex needs who were engaged in the RISKIT-CJS programme. This is important for the operation of the RISKIT-CJS programme because it is a multicomponent, but relatively brief, programme and it partly relies on connecting participants to other services in the area. If these services are absent or underfunded, then this component of the programme is less able to succeed in supporting young people towards positive outcomes.

Setting

As the RISKIT-CJS programme evolved, it was implemented in different types of setting. We observed systematic differences between YOTs and PRUs that affected implementation. A combination of these features meant that the programme was generally easier to implement in PRUs than in YOTs. Three features of settings that were likely to affect implementation and outcomes were management, staffing and space. In general, it was easier to implement the programme where there was strong commitment to its implementation by senior management and where the host agency was well managed (i.e. where there were clear lines of responsibility and communication in the organisation). There were also problems in delivering the programme at sites that were short of staff or that did not have much space to use for private and confidential discussions or group work.

Individual

As discussed below, many of the participants in the programme had issues with mental health and family estrangement that both deepened their need for support and complicated the provision of such support. These are important contexts for the operation of the programme, as early problems in parental relationships and self-regulation have been identified as part of the ‘developmental cascade’ for early-onset and problematic substance use and other risk behaviours. 44,112,113

Target population

The age of the target population (i.e. 13–17 years) led to the inclusion of participants who displayed a range of complex needs, including early-onset and escalation of risky substance use, as well as offending and antisocial behaviour, educational drop-out or exclusion with high levels of truancy and poor attainment of key stage learning objectives, mental health problems (e.g. anxiety and depression), and exposure to early or unwanted sex and parenthood. In the data, we saw several examples of individuals having mild to moderate mental health problems (including previous diagnoses of depression and anxiety) and a few participants whose mental health problems were severe enough for them to be already engaged with child and adolescent mental health services.

Patterns of early illicit drug use tend to cluster with other risk factors, including risky sexual behaviours,114 poor educational attainment115 and mental health problems, both during adolescence and in early adulthood. 116 These associations heighten the need to work effectively with these young people, to counter the combined impact of the various risk factors they experience which increase their propensity for escalating and persistent negative behaviours and outcomes. Our data suggested that the programme was effective in initially reaching people who shared these vulnerabilities.

Intervention

The RISKIT-CJS intervention manual describes the components of the programme to be delivered to the recruited participants. In practice, programme staff had to be flexible in delivering these components, depending on the needs and resources of the host sites and the recruited participants.

Each component listed in the logic model was mentioned as being important by at least some staff and participants. There were some participants who did not value any aspect of the programme, as demonstrated by their unwillingness to engage and their early drop out. However, for other participants, the combination of individual and group support provided a range of opportunities that they experienced as helpful, and this points to the value of a multicomponent programme providing a range of different opportunities, but also to the danger of overloading some participants who are less willing or able to engage.

A potentially limiting feature was that we observed some problems in using group work with mixed sex groups, which tended to be dominated by male participants, as this tended to leave less space for female participants to discuss issues that were relevant to them, especially around sexual health.

Positive mechanisms

Participants entered the programme with a range of types and levels of motivations, and this was an explicit aim of including motivational interviewing in the one-to-one aspects of the intervention, as this could work with ambivalence: ‘. . . some of them . . . wanted to change. They just didn’t know how, they didn’t know where to go’ (INT10).

The programme gave such participants a place and tools to use. For example, the enhanced motivation to change was combined, in some cases, with learning consequences of risk behaviours.

For some participants, the programme provided an opportunity to improve their educational engagement, as well as to have better communication with parents and teachers. Several participants reported that they found the programme sessions to be ‘better than school’ (PRU11, PRU12 and PRU54), as they were more active, relevant and engaging. Some participants increased their attendance and achievement in other educational provision. Communication is one of the life skills targeted by the programme.

We included improved self-efficacy in the logic model, as the term brings together various reports of participants’ improved ability to regulate their own behaviours, identify consequences and act towards desired goals. Self-efficacy would be expected to result from the intention of the programme to improve the information that participants have about risks, their skills in putting this information into practice and the opportunity to practise these skills in individual and group sessions.

Positive peer effects relate to the beneficial effects that some staff and young people reported from being able to share experiences with other young people who were experiencing similar trials and tribulations. Some of the group sessions worked as intended in enabling young people to share and reflect on experiences, as triggered by the planned elements of the group sessions and the skilled intervention of programme practitioners.

Negative mechanisms

Data analysis did reveal some mechanisms that would be expected to reduce or even counter the positive effects of the programme, including treatment fatigue and potential for negative peer effects.

Moderators

The data included information on moderators that could affect the pathway between contexts, intervention components, mechanisms and outcomes. These moderators can be seen as ways in which contexts and mechanisms of the programme interact.

Negative moderators

Labelling, otherwise known as ‘secondary deviance’,117 refers to the possibility that placing the label of ‘offender’ on a young person may have the effect of confirming for them that they are ‘deviant’ and so lead them to increase or prolong negative behaviours, as has been observed in longitudinal studies of young people who offend,118 and this was a possibility that was stated by managers of sites who were reluctant to take part in the programme. Others told us that their young people were already too entrenched in offending behaviour to make it worthwhile to trial the programme with them, and this could itself be seen as a form of labelling.

In some interviews, including follow-up interviews with participants, it was noted that participants lived in places with active drug markets and ‘gang’ or ‘on-road’ activity, which is particularly prevalent in deprived urban areas,119 and this provides opportunities and temptations for young people, which may reduce or overwhelm the prospects of the programme helping them towards reduced substance use and offending.

Some practitioners noted the wide range of age ranges targeted by the programme, which brackets the peak in the classical ‘age–crime curve’, which tends to occur in the mid-teens. 120 Substance use also tends to increase from childhood through adolescence. 121 Participants at the younger end of the targeted age range may have been recruited into the programme while they were on the ‘natural’ progression to more substance use and offending, that is, as they were ageing into increased risks.

Positive moderators

Practitioners involved in the programme observed and reported that it was more likely to succeed where there was strong commitment of host sites and strong local champions for the programme at these sites.

Staff training was identified as being crucial to the success of the programme. The RISKIT-CJS intervention manual describes the aims and framework of the programme, but it cannot, on its own, give staff the skills they need to provide a variety of programme components effectively with a group of young people who experience a range of risk factors and who often display behaviours that are difficult to manage.

In some cases, the presence of engaged parents was mentioned as being supportive of the programme’s effect in producing good outcomes for young people. When young people reported that their parents were active and supportive in discussing the problems they faced, they were more likely also to report engaging with the programme and achieving good outcomes.

For participants at the older end of the targeted age range, the downwards slope of the age–crime curve after the mid-teens may have helped them in using the programme to age out of risks, especially for those who entered employment or became a parent.