A cluster randomised controlled trial of a manualised cognitive behavioural anger management intervention delivered by supervised lay therapists to people with intellectual disabilities

Randomisation

Centres were recruited, and baseline data collected on all participating service users and their key workers, before randomisation of the centre took place. Randomisation was performed using the method of minimisation. 75,76 Centres were balanced on (1) their service users' average baseline self-reported Provocation Index (PI) score (see Primary outcome measure, below); (2) the number of service users recruited; and (3) the average number of hours per week spent by the service user with at least one trainer outside of sessions. A random component, set at 80%, was used alongside the minimisation procedure to ensure that the process was not completely determined.

Changes from the published protocol

There are no major changes from the published protocol74 (see Appendix 1).

Services and service users

Thirty services providing day activities for people with mild to moderate learning disabilities were recruited, on the basis that they reported significant anger control problems among some of their service users. Services were run by statutory or independent sector providers, and varied in their mode of operation from traditional day centres to individualised community-based activity programmes, including residential care providers. The study was implemented in three regions: one in Wales, one in England and one in Scotland. In each region, 10 services were identified, of which five were randomly allocated to the intervention group and five to the control group. Between four and eight service users were recruited to each group.

Participating centres received funding (£1000) to cover the costs of replacing the staff who acted as ‘lay therapists’. In order to maintain engagement in services randomised to the control group, training and supervision to run a group was offered at the end of the controlled comparison phase of the study.

Other participants

For each service user, a key worker and, where applicable, a home carer were also recruited (not all service users had a home carer depending on their residential setting). In each participating centre, between two and four staff were recruited to act as ‘lay therapists’. Staff were nominated by their managers and selected, without reference to formal qualifications, on the basis of their motivation to take on this role and their openness to using a cognitive–behavioural approach. Service managers were also recruited from each participating centre.

Inclusion and exclusion criteria

Participating services were required to meet all of the following inclusion criteria and neither of the exclusion criteria (Box 1).

Service users were eligible for the trial if they met all of the following inclusion criteria and none of the exclusion criteria (Box 2).

Consent

A contractual agreement was negotiated with participating services. Consent was sought from five types of participants: the service users themselves, their key workers, their home carers, lay therapists and service managers. Written consent was taken from the service managers, lay therapists, key workers and home carers, using consent forms and procedures that complied with standard Research Ethics Committee guidelines.

For service users, a more accessible consent procedure was used:

1. The trial was explained verbally in simple terms, using a standard script written in accessible language, with frequent checks for understanding.

2. In addition to the general information sheet that was provided to all participants, service users were also given a simplified accessible pictorial information sheet to take home and read in their own time and at their own speed, with support from carers.

3. At least 2 days were given to consider and ask questions of researchers or carers.

4. The explanation was repeated in a second meeting.

5. Consent was recorded by the service user, checking and initialling a set of tick boxes and signing the consent form.

6. In order to assure that the service user had been properly informed, without coercion, the whole process was witnessed and signed off by a staff member who was independent of the research team.

For therapists and service users selected for interview after the end of the intervention (see below), a separate consent was taken at the time, using the same procedures as above.

Baseline measures

After a service user had provided written consent, the first assessment administered was the Comprehensive Quality of Life Scale – Intellectual Disability (ComQoL-ID). 77 This instrument assesses quality of life via ratings of the importance of and satisfaction with different life domains. However, these ratings are preceded by a set of exercises to assess the respondent's ability to use simple rating scales and the ability to order quantities using a Likert-type scale. This part of the ComQoL was used to assess the inclusion criterion ‘able to complete the assessments’.

For service users who were assessed as eligible, a battery of psychometric and health economic assessments (described below) was administered prior to randomisation. Service users put forward for inclusion who were able to provide consent but unable to complete the assessments were allowed to attend the anger management group but were excluded from the trial.

Intervention

Participants in services randomised to the intervention group of the trial received a manualised CBT intervention,48 consisting of 12 weekly psychoeducational group sessions supplemented by ‘homework’. The intervention was delivered by day service staff acting as ‘lay therapists’, under the supervision of one of three clinical psychologists, each of whom was herself supervised by a consultant psychologist. After randomisation and before the start of the intervention, the clinical psychologist provided the lay therapists with three training sessions, delivered over the course of a single day, covering the principles of anger management and use of the therapy manual. The clinical psychologist subsequently provided fortnightly supervision to the lay therapists during the intervention. Staff training followed a training manual developed within the project for this purpose.

Topics addressed over sessions included the triggers that evoke anger, physiological and behavioural components of anger, behavioural and cognitive strategies to avoid the build-up of anger and for coping with anger-provoking situations, and acceptable ways of displaying anger (assertiveness). Presentation relied heavily on brainstorming (e.g. ‘What makes us angry?’) and role play. After the first session, about one-third of each session was devoted to discussion by facilitators and group members of participants' experiences, focusing primarily on problem-solving around ways in which situations might have been handled differently to produce a better outcome. In addition to simplifying the language used in sessions, wherever possible the use of written materials was avoided in favour of pictorial representations. Towards the end of every session, participants were asked to undertake a homework assignment, which consisted of working with a staff member to complete a functional analysis (‘hassle log’) of a situation in which they had been angered that week, which was described, analysed and evaluated, using a pictorial workbook. At the end of the intervention, reports were provided on each of the participants and recommendations were made for further input by staff to maintain and increase treatment gains.

Participants in services randomised to the control group of the trial had no therapeutic contact with the research team until the final post-intervention follow-up assessments had been completed, at which point the same training, intervention and supervision procedures were implemented.

Follow-up assessments

Quantitative outcome assessments were administered before randomisation and in a 2-week window beginning 16 weeks after the date the group was randomised, and again in a 4-week window beginning 6 months later (i.e. 10 months after randomisation). The 16-week time point was chosen to allow 2 weeks before the start of the 12-week intervention for staff training, and a further 2 weeks to take account of likely delays due to centre closures or staff absences. The researchers undertaking the outcome assessments did not have any involvement in training or supervision of the therapists, and were in principle blinded to the group allocation of the service, although the group allocation sometimes became apparent during direct interaction with the service user. After the 10-month follow-up assessments were completed, the lay therapists in the control services were trained and the intervention was delivered to the control groups, followed by further assessments in a 2-week window beginning 16 weeks after the training date.

In addition to the quantitative assessments, a sample of service users and lay therapists was interviewed within 2 weeks (service users) or 4 weeks (lay therapists) after the end of the group, and service managers were interviewed at baseline and at the 10-month follow-up (see Qualitative assessments, below). Health-economic data, covering the preceding 12-week period, were collected at baseline and at the 10-month follow-up.

Primary outcome measure

The main outcome measure was the PI as completed by the service user, at the 10-month follow-up. The PI is a direct measure of felt response to defined situations that may provoke anger, which has frequently been used with this service user group. 28,81 It consists of a list of 25 different situations that can evoke anger, each of which is rated on a four-point scale (0–3) for the amount of anger that it would evoke. Scores on this measure have been shown to correlate with staff-reported levels of aggression. 28

Secondary outcome measures

Assessment also involved completion of the PI by a key worker. 11,48 For this and other measures, in the event that a service user's key worker was involved in the trial as a lay therapist, then, wherever possible, the measure was completed by another staff member.

The PACS11,48 was completed by both service user and key worker to assess the development of alternative, more functional coping skills. The PACS first identifies three situations that reliably trigger anger in the individual service user, then asks for ratings of the usage of each of eight coping skills in each of those three situations. The PACS has previously been used only with key workers and carers. For the purposes of the trial, a modified service user version was produced, with two changes: a more detailed description of each coping skill was provided, and a three-point rating scale, rather than a four-point rating scale, was used. For comparability across respondents, PACS scores were expressed as a percentage of the maximum possible score.

The service user version of the PACS also includes a rating (on a three-point scale) of the anger evoked by each situation, which is elicited prior to the ratings of skill use. This represents a form of imaginal provocation test (IPT)47 and provides a further measure of the proclivity to experience anger. These Profile of Anger Coping Skills – imaginal provocation test (PACS-IPT) ratings differ from the PI in relating to actual, rather than hypothetical, situations.

Mental health was assessed using the Glasgow Depression Scale (GDS) and the Glasgow Anxiety Scale (GAS), which are established measures of depression and anxiety among people with an intellectual disability,82,83 and an adaptation of the Rosenberg Self-Esteem Scale (RSES) for people with an intellectual disability. 84 Scores of ≥ 13 on the Glasgow scales are indicative of clinical levels of depression or anxiety. Self-reported quality of life was assessed using the ComQoL-ID;77 the scaling exercises that were administered at the baseline assessment (see Baseline measures, above) were omitted from follow-up assessments.

Aggressive challenging behaviour was assessed by key worker report using the Hyperactivity and Irritability domain items of the Aberrant Behaviour Checklist (ABC)85 and the Modified Overt Aggression Scale (MOAS). 86 In previous work, severe challenging behaviour has been defined by a criterion of either a combined score of > 30 on the Aberrant Behaviour Checklist – Hyperactivity (ABC-H) and Aberrant Behaviour Checklist – Irritability (ABC-I), or > 4 behaviours rated as a severe problem (i.e. at level 3). 87 Key workers' attributions with respect to challenging behaviour were measured by the Controllability Beliefs Scale (CBS). 88

In order to assess generalisation across settings, the anger, coping skills and aggression measures (PI, PACS, ABC-H, ABC-I, MOAS) were also completed with the service users' home carers.

For all instruments, the total score was pro-rated to take account of missing items (or, in the case of the PI, ‘do not know’ responses), provided that a minimum of 80% of items were completed (excluding ‘do not know’ responses).

Reliability of the instruments used

The internal consistency (Cronbach's alpha) and test–retest reliability of the instruments used are summarised in Table 4.

For comparison with the published literature, Cronbach's alpha was calculated from the baseline data (all participants) for all measures except the PACS, ComQoL-ID and MOAS, which do not lend themselves to this assessment because they do not involve computation of summated scales.

Fidelity monitoring

Service users

We aimed to interview 8–12 service users after the intervention to gain an understanding of their experiences of participating in CBT. Interviews were conducted within 2 weeks of the end of the group by the clinical psychologist from the local region, who had previously met the service user when taking consent but had not attended any of the group sessions. Service users were randomly selected from a ‘short list’ of those participants in each group who were considered to have sufficient expressive language ability to be interviewed and to be willing and able to talk about their experiences of participation. The interviews were conducted according to a semistructured interview schedule, containing questions that encouraged the participants to focus on ‘personal meaning’ and make sense of their experience of the therapeutic process. Participants were asked about their involvement in the group (including their expectations and the perceived value of the group) and the impact of the intervention on them. The interview was designed to cater for potential cognitive deficits, such as limited language comprehension and expression, and short attention span. Visual prompts of group activities (taken from the intervention manual) and of people and places were used during the interview for the interviewer and participant to consult.

Lay therapists

We aimed to interview 8–12 lay therapists post intervention in order to investigate their personal experiences of learning and applying new therapeutic skills as cognitive–behavioural therapists, as well as their impressions of the ‘climate’ within the group and the impact of the group on the wider service. They were selected purposively as those who had been most active in terms of running each group. Semistructured interviews were conducted, within 4 weeks of the end of the group, by a clinical psychologist from outside the region who had not had previous involvement with that group. The interviewees were asked to talk freely about their initial hopes and fears training and working with the manual and the CBT model, the role of ‘therapist’ and support needs in running the group, perceived impact of the intervention, and future developments.

Service managers

A related but separate part of the qualitative evaluation aimed to gain an understanding of service policies and practices for service users who express anger inappropriately, and the effects on services as a whole of hosting a CBT group intervention. This was accomplished by interviewing the managers of the services in the intervention group at the start of the study (before any input to the service), and at the 10-month follow-up. These were structured telephone interviews conducted by a clinical psychologist from outside the region.

Clinical psychologists

At the end of the study, but before having access to any of the group-level data, the clinical psychologists were each asked to predict which of the groups that they had supervised (in both the intervention and control arms) would have the three best and three worst clinical outcomes. (By exclusion, there was also a third set of predicted intermediate groups.) From these predictions, the psychologists were asked to reflect on their experience of supervision and provide the basis for their predictions.

Feedback to participants

After completing the data collection and analysis, the findings were fed back to the research participants and significant others in their lives. The aim was to find out whether or not these individuals considered the results to be credible. 92 This was part of the effort to ensure that the qualitative research was carried out to a high standard and with rigour.

Intervention costs

All resources used in delivering the intervention were recorded prospectively. These included:

1. the time input of:

   1. the consultant psychologist in each region who trained and supervised the clinical psychologists who in turn trained and supervised the lay therapists

   2. the clinical psychologists in each region who trained and supervised the lay therapists running the groups

   3. the lay therapists who ran the groups

   4. administrative/secretarial staff attributable to the intervention

2. travel

3. the costs of other consumables attributable to the intervention (e.g. production of manuals, telephone calls, photocopying and materials used in group sessions).

Resource use was recorded on logs developed for the study. Three logs, one for each category of staff, recorded the time input and related travel costs of each consultant psychologist, clinical psychologist and lay therapist week by week during the intervention period. A fourth log recorded administrative time input, length of telephone calls, numbers of photocopies and the costs of other consumables used, again on a weekly basis. A fifth log was used to record details of each group treatment session, again on a weekly basis. This log included specification of the number of service users attending the session, for how many the session was additional to treatment as usual, the number of lay therapists running the session and its length.

Impact on service use

Information on health and social care resource was collected from service users' key workers and/or home carers for a 12-week period immediately preceding the baseline and follow-up data collections.

Total resource use by each participant was measured using the Client Service Receipt Inventory (CSRI). 93,94 The CSRI is a validated tool that has been previously used in evaluations involving service users with intellectual disabilities. 87,95,96 It is amended for each study to record relevant resource inputs. The CSRI for this study collected data on:

1. accommodation arrangements

2. employment and income

3. use of day services (e.g. attendance at day centres, sheltered work, adult education, etc.)

4. use of psychotropic medication for the control of aggression or related challenging behaviour

5. number and length of multidisciplinary meetings held to discuss their care and the staff and professionals attending each

6. receipt of other community-based professional services

7. receipt of hospital-based services

8. provision of domiciliary support or respite care for participants living independently, in the family home or in minimally staffed residential arrangements

9. staffing levels for participants living in partially to fully staffed residential arrangements

10. hours per week at work lost to informal carers due to the need to provide support because of the participant's anger or aggression.

In order to attribute staffing costs in group residential services to individual participants, the CSRI also recorded:

1. the number of people living in the residential setting

2. typical numbers of staff in the morning, afternoon, evening and night

3. whether night staff were waking, sleeping in or on call

4. whether the participant in question required a similar staff input to other residents and, if not,

5. how much more or less.

Quantitative outcomes

Process evaluation

Health economic outcomes

Resources used were valued using unit costs for the financial year 2010–11 from a variety of sources specified in Appendix 2. Costs are expressed per participant week.

Service users

Participants' demographic characteristics are shown in Tables 10 and 11. As in previous studies of this kind, participants were mainly middle-aged (median age = 38 years) and male (71%), with a median FSIQ of 57. In terms of adaptive behaviour, the mean short-ABS score of 86 converts to a full ABS score of 223, which is equivalent to the 75th percentile of people with intellectual disabilities living in residential or community care. 80 The majority of participants lived in the family home (41%) or in staffed/supported living arrangements (38%). The two groups were well matched on all demographic variables (see Table 10).

Participants in the three regions of the trial were well matched on age, gender and indicators of level of intellectual disability (Table 11). However, they differed markedly in their accommodation arrangements. In Scotland and Wales, participants lived predominantly in the family home (61% and 54%, respectively). However, in England only 12% of participants lived with families, with 60% in group homes or other supported living arrangements, and 20% in secure (hospital) settings. This reflects the fact that traditional local authority day services still exist in Scotland and Wales, but have been disbanded in many parts of England, leading to recruitment, in this trial group, of residential services that provided their own day activities (see Table 7).

Other participants

The median age of key workers (Table 12) was 46 years and they were predominantly female (61%). Unlike other references to key workers elsewhere in this report, Table 12 refers to the head count rather than the number of reports the key workers provided. It describes the 124 key workers recruited for the baseline assessment. During the course of the study a further 37 key workers were recruited as replacements following staffing changes. Their demographic characteristics were similar to those of the original cohort.

The ns in this table refer to the actual numbers of key workers. Elsewhere, key worker numbers refer to their reports on service users, which are higher because some key workers reported on more than one service user.

Home carers (Table 13) were of a similar age (median = 50 years) and gender balance (65% female) to key workers. Family carers were somewhat under-represented relative to staff carers; excluding service users who lived independently, 45% of service users lived in the family home, but only 37% of home carers were family members (see Table 13). This difference reflected the experience that family carers were more difficult to recruit than staff carers. Service users in the intervention group were more likely to have a female home carer but the groups were otherwise very well matched.

Lay therapists (Table 14) had a median age of 42 years and 63% were female. They had worked with people with learning disabilities for a median of 9 years, with 6 years in their current organisation and 5 years in their current role. Just over 40% had been educated to higher school-leaving level [National Vocational Qualification (NVQ) level 3, A-level or similar qualifications], a similar proportion had some university or equivalent-level education, and 18% of lay therapists had low [NVQ level 2, General Certificate of Secondary Education (GCSE) or similar] or no qualifications. The lay therapists in the intervention group were more likely to be male (49% vs 27%), and were better educated (32% vs 11% had a relevant degree or professional qualification). However, these differences do not influence the outcome of the RCT, because the lay therapists in the control group occupied this role only after the end of the trial. None of the lay therapists had received any prior formal training in CBT.

Service managers (Table 15) were similar in age to the other two groups of staff (median = 46 years), with an approximately equal gender split (46% female). They were comparable with the lay therapists in their occupational history, and all had served in their present role for at least a year. Over three-quarters (77%) had NVQ level 4 or university-level qualifications, and almost half (46%) had a relevant degree or professional qualification. Managers in the intervention group tended to have been with their organisation for longer than managers in the control group, but the two groups were similar in all other respects, including their experience in the management role.

Anger

Provocation Index scores were similar to those observed in earlier studies on a comparable population,11,48 with service users rating themselves higher than they were rated by their key workers or home carers [F(2,487) = 38.05; p < 0.001]. Key worker and home carer PACS scores were also similar to those reported previously. 11,48 The service user PACS was developed for the purpose of the present study and has not been previously reported. Service users rated themselves as coping better than did their carers [F(2,475) = 61.3; p < 0.001].

Mental health

A relatively high proportion of service users displayed significant levels of depression, with 34% of them scoring at or above the cut-off value for clinical depression of 13 on the GDS. 83 An even higher proportion of service users displayed significant anxiety symptoms, with 73% scoring at or above the cut-off value for clinical anxiety of 13 on the GAS. 82 However, their satisfaction with life, as assessed by the ComQoL, was somewhat higher than the population average of 75. 77

The incidence of psychopathology in the present sample – particularly anxiety – was substantially higher than population estimates of the prevalence of depression and anxiety among people with intellectual disabilities, which are, respectively, > 4% and 5–10%. 108–110 The association between problem anger and high levels of psychopathology is consistent with previous reports in people with intellectual disabilities111 and in the general population. 112–114

Challenging behaviour

Forty-seven service users (26% of the sample) were rated by key workers as meeting the criterion for severe challenging behaviour (either a combined score of > 30 on the ABC-H and ABC-I, or more than four behaviours rated at level 3 on these scales). 87 Median MOAS scores were a little lower than those recorded in a trial of neuroleptic drug treatment for aggressive behaviour in people with intellectual disabilities, which reported a median [interquartile range (IQR)] MOAS score of 12 (8–25) at baseline in the placebo-treated group, and a dramatic fall (to a median score of ∼3: considerably lower than the present participants) during subsequent weekly assessments under placebo treatment. 115

There is a limit to the extent of challenging behaviour that could in principle be observed in this study because most of the services from which participants were recruited would often exclude individuals who are overtly challenging. Nevertheless, the 26% of participants who met the criterion for severe challenging behaviour is higher than the proportion among the general population supported by the participating services. Mean ABC scores were about 30% higher than those reported in an unselected sample of individuals attending comparable services. 116 Using the same criterion, rates of severe challenging behaviour have been reported to be 15% in the general population of people with intellectual disabilities117 and 22% in residential services. 118 These figures include individuals with severe and profound disabilities and people who challenge at a level that cannot be managed within many services providing day activities. Neither of these groups would have been eligible for the present study: the former group because they would not have been able to consent and the latter because they would have been excluded, which the present participants, by definition, were not. The high level of challenging behaviour among the present sample is not surprisingly, given the well-described relationship between anger and challenging behaviour among people with intellectual disabilities. 15,17,28

Regional differences

Baseline scores differed somewhat between regions (Tables 19 and 20). Service users in Scotland rated themselves, and were rated by both groups of carers, as less angry than service users in England or Wales, and they were rated by themselves and their home carers as having fewer anger coping skills. Service users in England had higher levels of psychopathology, and were significantly more likely to meet criteria for a diagnosis of depression or anxiety. They were more challenging than Scottish participants on all three challenging behaviour measures and Welsh participants on two of the three measures. This reflects the differences between regions in the types of services recruited to the trial, which, in England, included some secure settings and only a single local authority day service.

Correlations between clinical variables

Inter-rater correlations for PI values are shown below the diagonal in Table 21. Service user PI ratings were only marginally correlated with carer ratings, but the correlation between the two carer ratings was much larger, suggesting that key workers and home carers see service user anger similarly to one another, but differently from the service user's self-perception.

Inter-rater correlations for the PACS are shown above the diagonal in Table 21. All correlations were significant but small (0.01 < p < 0.05). PI and PACS ratings were significantly correlated for service users (r = 0.217, p = 0.005) but not for key workers (r = -0.076) or home carers (r = 0.122), again suggesting a difference between carer perceptions of service users and service user self-perceptions.

Service user ratings on the three mental health measures were significantly intercorrelated (all p < 0.001). The highest correlation was between the depression and anxiety scores (GDS vs GAS: r = 0.697); correlations with self-esteem (RSES) were somewhat lower (r = -0.384 and -0.302, respectively).

Similarly, both sets of carer ratings on the three challenging behaviour measures were all strongly intercorrelated, as shown in Table 22 (all r > 0.446, p < 0.001). The correlations between key worker and home carer ratings were highly significant for the MOAS (r = 0.486, p < 0.001), less so for the ABC-I (r = 0.243, p < 0.01) and not quite significant for the ABC-H (r = 0.176, p = 0.059).

All three key worker ratings of challenging behaviour were significantly correlated with their ratings on the CBS (ABC-H: r = 0.173, p < 0.02; ABC-I: r = 0.188, p < 0.02; MOAS: r = 0.148, p < 0.05), indicating that service users who were seen as more able to control their challenging behaviour were rated as more challenging. Key workers' CBS ratings were not significantly related to age (r = 0.044), IQ (r = 0.133) or gender (t = 1.1).

Regression analyses

The regression analyses for the three sets of PI ratings are summarised in Table 23a and b, with the significant predictors that are discussed below highlighted. The regression analyses for the PACS did not identify any significant predictors and these data are not reported.

For service users, demographic variables were a marginally significant predictor of PI ratings (block 1, p = 0.047), and the model improved very significantly when mental health variables were added (block 2, p < 0.001); adding challenging behaviour variables (block 3) did not significantly improve the model. The significant predictors were age (older service users rated themselves as less angry) and anxiety. As anxiety and depression were highly intercorrelated, the latter may be taken as a proxy for poor mental health.

For both key workers and home carers, IQ was a significant predictor of their PI ratings, with more able service users rated as more angry. For key workers, the addition of mental health variables significantly improved the model (block 2, p = 0.012), reflecting a significant effect of depression (p = 0.007). Depression was also almost significant in the home carer model (p = 0.053). However, for both sets of carers, the effect of depression disappeared when challenging behaviour was added to the model (block 3, p < 0.001), with significant prediction of PI by ABC-I scores (key workers) and MOAS scores (home carers). Again, given the high intercorrelations between challenging behaviour measures (see Table 23), we interpret these results to mean that challenging behaviour was a more important predictor of PI ratings than their perceptions of the service user's mental health for both sets of carers. The significant effect of GDS score in block 2 but not block 3 of the key worker model probably reflects the fact that these ratings were significantly correlated with key worker ABC-I scores (r = 0.348, p < 0.001).

A further predictor in the home carer model was age, which was significant in all three stages of the analysis: older service users were rated as less angry, consistent with service users' self-ratings. However, this relationship was not present in key worker ratings.

The outcomes of these analyses are summarised in Figure 2. Older service users were rated as less angry by service users and home carers but not by key workers; more intellectually able service users were rated as more angry by both sets of carers but not by the service users themselves; and, as predicted, the clinical factor most strongly associated with PI ratings was mental health status for service users and challenging behaviour for carers. The regression models account for a relatively small proportion of the overall variance (12–31%), but this is not surprising considering that the models did not include any variables directly related to the service users' personal history or environment.

FIGURE 2.

Predictors of PI scores. Within the circles are shown the intercorrelations between measures of mental health (MH, above) and challenging behaviour (CB, below). The arrows to the right of the circles show the individual measures (above the arrow) and blocks of data (below the arrow) that significantly predict PI scores. Other significant predictors (age and IQ) are shown to the right.

Consistency

The consistency of the scale was calculated based on agreed ratings where available (n = 49) or single ratings where not (n = 3). ‘Not applicable’ scores were treated as missing values and the value was set to the mean of all present/absent ratings for that item. Consistency was high (Cronbach's alpha = 0.864). Five items (2.i, 5.ii, 7.iii, 8.ii and 20.iii) had item total correlations of < 0.2, but the effect on Cronbach's alpha of removing individual items was minimal (range 0.854–0.872).

Factor structure

For the factor analysis, ‘not applicable’ ratings were permitted for the seven items (each marked with an asterisk) in Table 5, and treated as missing values, as above.

Principal components analysis identified 10 factors with eigenvalues of > 1, accounting for 73% of the variance in MAGIC scores. The scree plot indicated a single major factor that accounted for 25% of the variance, with the remaining nine factors accounting for 3–9% each. Table 33 shows the rotated component matrix, with item loadings > 0.5 shown in bold text and loadings of 0.4–0.5 in bold italic text. It is apparent that with the exception of item 10.iii (which was the least-reported and among the least-reliable items) all of the items contributed to at least one factor, there was relatively little cross-loading, and there were no single-item factors. Taking into account the high level of consistency of the overall scale, the single factor solution identified using the scree plot criterion, and the complexity of the factor structure implied by the eigenvalue criterion, the total MAGIC score, rather than factor scores, was used to evaluate the fidelity of delivery of the intervention.

Factor content was examined for its descriptive value. Table 34 lists the highest-loading items for the first five factors, which together accounted for 51% of the variance. The table shows the items with loadings of > 0.5 on the first five factors, which account for just over 50% of the variance. The entries in italic text represent an interpretation of each factor. The main factor, labelled ‘communication’ as well as factors 2 (‘respect’) and 4 (‘empathy’) relate to general group work skills, whereas factors 3 (‘management of the session’) and 5 (‘use of materials’) relate to the mechanics of running the group.

Reliability and validity

Because the MAGIC was developed within the context of a relatively large cluster RCT, the development is based on the observation of two sessions from each of a large number of therapeutic groups (n = 27), by a large number of observers (n = 16), working in different paired combinations. The results indicate that the inter-rater reliability of the MAGIC is acceptable, in terms of both the overall score and all of the individual scale items.

In common with the Cognitive Therapy Scale65 and the CTS-Psy,66 the MAGIC includes items that address general therapeutic processes (e.g. interpersonal effectiveness), as well as specific CBT skills (e.g. homework issues). Non-specific factors, in particular the therapeutic alliance, are increasingly highlighted in the therapeutic literature as contributing to therapeutic change. 139–142 Therefore, in addition to rating how far therapists abide to the manual, it is also helpful to obtain reliable data about the therapists' overall success in implementing the intervention. Haddock et al. 66 presented two subscales for the CTS-Psy, for general skills and technical skills, corresponding roughly to the first and second halves of the instrument. As our factor analysis identified a single-factor solution, we did not compute subscale scores for the MAGIC. However, participants made global ratings of overall quality and of the three areas that the MAGIC was designed to assess: delivery of the manual, attention to group process, and the principles of CBT. Excellent inter-rater reliability was seen on all four of these global ratings. It appears, therefore, that even without specific subscales, the MAGIC can provide reliable estimates of both general psychotherapeutic and specific CBT skills. The global ratings were made after live scoring of the individual items, and on to the same scoring sheet, so we assume that the global ratings were informed by the detailed information acquired during the session. (That is, the same level of inter-rater agreement may not have been reached if observers were making their own notes rather than using the same standard instrument.)

The data collected using the MAGIC demonstrate that the processes that underpin the adaptation of group therapy for people with intellectual disabilities can be reliably recorded. Although constructs to emerge most strongly from the factor analysis, such as ‘communication’, ‘respect’, and ‘empathy’, are clearly relevant to psychotherapeutic interventions with any group, issues such as the pacing of sessions and pitching communication at the right level are particularly pertinent therapeutic judgements when working with people who have intellectual disabilities. 67,143 Thus, the analysis seems to provide some support for the face validity of the instrument with this client group. The validity of the MAGIC is supported by its ability to predict two key outcomes of the intervention: anger ratings and the acquisition of anger coping skills. (We note, however, that higher fidelity was associated with both better acquisition of coping skills, as predicted, and higher anger ratings, which is the inverse of the predicted relationship. This has implications for the interpretation of PI scores, which will be discussed in Chapter 10.) The validity of the MAGIC is also supported by the significant relationship between fidelity and predictions of clinical outcome, which were made on the basis of knowledge acquired through supervision of the lay therapists but without any direct contact with the groups.

Utility

With regard to the delivery of the anger management manual by the lay therapists, it is noteworthy that the levels of fidelity recorded were reasonably good, and consistent across two separate waves of data collection. Indeed, our ratings of the lay therapists, whose training comprised a single day of instruction followed up by fortnightly supervision, compared favourably with those reported by Haddock et al. 66 for the CTS-Psy. In their study, only 8 of 14 psychotherapy trainees scored above 50% compared with our mean overall score of 64%. The difference could result either from the fewer items in the MAGIC (n = 30) as compared with the CTS-Psy (n = 60), with the possibility that we selected items that are easier to achieve, or more likely, from the use in the present study of a prescriptive manual containing detailed session plans. In a classic study, Strupp and Hadley144 reported that there was no difference between psychotherapeutic outcomes between experienced psychotherapists and therapeutically naive ‘college professors chosen for their ability to form understanding relationships’. Although our ‘lay therapists’ had no prior training or experience in CBT, the ability to form understanding relationships is the principal requirement for selection to their employment as support workers for people with intellectual disabilities, and this ability is likely to contribute to their high scores on the fidelity monitoring checklist.

As might be expected, groups varied in the fidelity ratings achieved. However, there did appear to be particular strengths in the delivery of the manual, with activities usually carried out and materials used appropriately. Moreover, therapists also demonstrated the desired communication and group skills in the majority of instances, suggesting that most lay therapists were able to communicate the ideas contained in the manual effectively. The weakest components were in relation to specific CBT skills. These included a number of specific techniques that were not delivered because they had not been explicitly emphasised in the manual, but, also, most of the more abstract CBT elements, concerned with the service users' thoughts and feelings. It might be argued that delivering CBT requires therapists to have grounding in relevant theory and a particular set of therapeutic skills,145 making it unsurprising that these elements were largely absent. However, a further reason for the lower scores regarding these CBT components of the intervention might have been that service users struggled to grasp the relevant concepts. Clearly, fidelity instruments such as MAGIC are meant to assess how faithful the therapists remain to the content and spirit of the approach set out in the manual. However, it may also be necessary to take account of the service users' intellectual difficulties when interpreting the fidelity ratings.

A number of limitations of this study can be identified. First, some minor changes were made to the guidance provided to monitors during the course of the study. However, results obtained in the second wave of data collection were very similar to those obtained previously, so it appears that this had little effect on the overall picture. Second, these changes arose out of difficulties identified in identifying some items as ‘absent’. Considering the high overall fidelity scores recorded, it must be considered that the criteria used to decide that an item was present may have been too generous. Against this, we note that there was considerable variability in overall scores across groups, with one set of ‘lay therapists’ achieving a very poor score. Finally, the MAGIC has not yet been used outside the context of the present manualised intervention for anger. Although we consider that the reliability of the instrument would probably be retained in other therapeutic contexts, it might be lower for interventions with a less prescriptive manual.

One of the reasons that it has been difficult to carry out intervention studies with people who have intellectual disabilities has been a lack of reliable and validated assessments. This applies as much to fidelity measures as to other domains, such as self-report measures of distress. 146 The heterogeneity of people with intellectual disabilities, in terms of communicative and intellectual abilities, is also thought to be a barrier to the development of manualised interventions. However, the present use of the MAGIC has shown that it is possible not only to develop a manual that can be used flexibly to address the particular needs of group members but also to record reliably whether or not the intervention is delivered in a sensitive fashion and whether or not therapists can successfully engage group members in the sessions, as well as identifying individual strengths and weaknesses. The MAGIC items were deliberately worded in a non-specific manner, so that the instrument could be used in other therapeutic contexts, and we would hope that the degree of reliability that we observed would be retained. Therefore, the development of this fidelity measure may play a role in facilitating intervention studies with people who have intellectual disabilities, and increasing the evidence base in relation to group-based CBT interventions.

Self-reported Provocation Index at 16-week follow-up

At the 16-week follow-up, the difference in PI scores was larger than at 10 months and approached, but again did not reach, statistical significance (p = 0.083) (Table 39). These results reflect a tendency for PI scores to increase in control subjects and to decrease in the intervention group (Figure 4a).

FIGURE 4.

Comparison of self-reported (a), key worker-reported (b) and home carer-reported (c) PI scores between control and intervention at baseline, 16 weeks and 10 months. HC, home carer; KW, key worker. *p < 0.05; **p < 0.01; ***p < 0.001.

As with the 10-month outcome, when the analysis was adjusted to include only intervention-compliant participants (CACE analysis) the intervention effect increased but still failed to reach significance (p = 0.063).

Key worker-reported Provocation Index

Key workers reported larger decreases in service users' anger, relative to the control group, than was reported by service users (Table 40). The decrease in PI scores, as reported by key workers, was not quite significant at the 16 week follow-up but was significant at the 10-month follow-up. Again, these results reflect both increased scores in the control subjects and decreased scores in the participants receiving the intervention (Figure 4b). The adjustment of the analysis for those participants who were compliant with the intervention (CACE analysis) increased the effect sizes, which were significant at both time points (16 weeks, p < 0.05; 10 months, p < 0.01).

It had been intended that, if key workers also acted as lay therapists, another staff member who knew the service user well would provide the key worker information. However, in some cases this was not possible, particularly for the follow-up assessments. In these cases, lay therapists provided key worker information. This represents a potential source of bias, as lay therapists might exaggerate the gains made by service users in their group. In order to evaluate this potential source of bias, the key worker analyses were repeated after excluding any reports from key workers who were also lay therapists. A total of 14 key worker reports in the intervention group were excluded for this reason. Contrary to expectations, the effects of the intervention actually increased slightly after excluding these reports (16 weeks, p < 0.05; 10 months, p < 0.025) (Table 41).

The results for individual groups are shown in Figure 5. The control groups showed an increase or a very small decrease in key worker-rated PI scores. In three earlier studies there was a mean 13-point decrease in PI scores from baseline to follow-up 3–6-months post intervention. 11,39,48 The intervention groups showed decreases or very small increases in key worker-rated PI scores, with three groups (#207, #308 and #209) showing decreases that were similar to those previously reported.

FIGURE 5.

Centre differences in key worker-reported PI at 10 months. Difference between 10-month follow-up and baseline key worker PI scores in control group centres (a) and intervention group centres (b).

Home carer-reported Provocation Index

Home carers did not report significant decreases in service users' anger (Table 42 and Figure 4c). The effects of adjusting the analysis to exclude non-compliant participants were negligible.

Self-reported Profile of Anger Coping Skills anger ratings

A further source of information about service users' anger comes from the PACS-IPT. An exploratory post hoc analysis of these data was undertaken as a potential means of understanding better the unexpected lack of a significant effect of the intervention on service user PI scores, which was discrepant from both key worker PI ratings (see Figure 4) and the published literature (see Table 2).

Service users' mean ratings on the PI were just under 60% of maximum at baseline, and, as reported earlier, little change was seen at follow-up. PACS-IPT ratings were much higher at baseline (85% maximum), which was expected, given that the scenarios included in the PACS represented situations that triggered anger for the individual participant. PI and PACS-IPT ratings are compared in Figure 6. In contrast with the lack of change in PI scores, PACS-IPT ratings decreased significantly in the intervention group at the 16-week follow-up (p < 0.01), and the difference between intervention and control groups was maintained at the 10-month follow-up (p < 0.05) (Table 43).

FIGURE 6.

A comparison of PI (a) and PACS-IPT (b).*p < 0.05; **p < 0.01; ***p < 0.001, based on ranked analysis of covariance, controlling for baseline PACS-IPT.

Self-reported Profile of Anger Coping Skills

In contrast to the small changes in self-reported anger, service users reported large and highly significant increases in use of anger coping skills (Table 44). The intervention and control groups differed significantly at 16 weeks (p < 0.001), and this difference was maintained at the 10-month follow-up (p < 0.01; see Figure 7a). At the 16-week follow-up, PACS ratings of service users in the intervention group were significantly aligned to key worker reports (r = 0.27, p < 0.05), but remained non-significant in the control group (r = 0.12, non-significant).

FIGURE 7.

Comparison of self-reported (a), key worker-reported (b) and home carer-reported (c) PACS scores between control and intervention at baseline, 16 weeks and 10 months. HC, home carer; KW, key worker. *p < 0.05; **p < 0.01; ***p < 0.001.

The results for individual groups are shown in Figure 8. Most of the control groups reported poorer coping at the 10-month follow-up compared with baseline. Six of the intervention groups (#104, #206, #103, #205, #301, #311) had a large increases in PACS scores that were similar to or greater than the effects reported by key workers in earlier studies,11,48 with four others showing more modest effects and minimal change in the other four groups.

FIGURE 8.

Centre differences in self-reported PACS at 10 months. Difference between 10-month follow-up and baseline PACS scores in control group centres (a) and intervention group centres (b).

Key worker-reported Profile of Anger Coping Skills

Key workers also reported large improvements in the use of anger coping skills, at both the 16-week (p < 0.002) and the 10-month (p < 0.01) follow-up assessments (Table 45 and Figure 7b). The proportional increase in PACS scores as reported by key workers was substantially larger than service users' self-reports. This reflects the fact that key worker ratings of anger coping were lower at baseline than service users' self-reports (as reported in Chapter 3): the effect of the intervention in absolute terms was similar in size in self-reports and key worker reports. The estimates in Table 45 correspond to adjusted mean differences on the original scale of 8 at 16 weeks and 9 at 10 months.

In order to evaluate the potential bias arising from the inclusion of key workers who were also lay therapists, the analyses were repeated after excluding reports from these participants. As with the PI, the changes in key worker PACS ratings remained significant in both the 16-week (p < 0.005) and the 10-month (p < 0.025) follow-up assessments (Table 46).

The results for individual groups are shown in Figure 9. Most of the control groups reported poorer coping at the 10-month follow-up than at baseline. In two previous studies there was a mean 13-point increase in PACS scores from baseline to follow-up 3–6 months post intervention. 11,48 The intervention groups all showed an increase in key worker-rated PACS scores, with the increases in the best six groups (#311, #103, #107, #104, #209, #205) similar to those reported earlier. The rank ordering of groups was similar for service user and key worker PACS ratings (r = 0.54, p = 0.05).

FIGURE 9.

Centre differences in key worker-reported PACS at 10 months. Difference between 10-month follow-up and baseline key worker PACS scores in control group centres (a) and intervention group centres (b).

Home carer-reported Profile of Anger Coping Skills

The PACS scores as reported by home carers improved in the intervention group in both follow-up assessments, but the differences from controls were much smaller than reported by service users and key workers and non-significant (Table 47 and Figure 7c).

Anger coping in treatment-compliant participants

The exclusion of non-compliant participants (CACE analyses) increased the effect size for anger coping in all analyses. This had the effect of increasing the significance level for service users at 10 months (p < 0.005) and for key workers at both time points (p < 0.001). Home carer ratings remained non-significant.

Key worker reports

Key workers reported significant improvements, relative to control subjects, at the 16-week follow-up, on two measures of challenging behaviour: the ABC-H (p < 0.001) and ABC-I (p < 0.005) (Table 49 and Figure 10a and c). As with PI scores (see Figure 4), the changes reflect a tendency for ABC scores to decrease in the intervention group and increase in the control group, and as with PI scores, these changes increased slightly after excluding reports from key workers who also acted as lay therapists (ABC-H, p < 0.002; ABC-I, p < 0.005). As with other measures, the exclusion of non-compliant participants (CACE analysis) slightly increased the effect size (p < 0.001 for both measures).

FIGURE 10.

Comparison of key worker-reported (a, c and e) and home carer-reported (b, d and f) ABC scores between control and intervention at baseline, 16 weeks and 10 months. (a) Key worker-reported ABC-H; (b) home carer-reported ABC-H; (c) key worker-reported ABC-I; (d) home carer-reported ABC-1; (e) key worker % severe challenging behaviour; (f) home carer % severe challenging behaviour. HC, home carer; KW, key worker. *p < 0.05; **p < 0.01; ***p < 0.001.

The improvements in challenging behaviour reported at 16 weeks were not maintained at the 10-month follow-up. However, this largely reflects changes in ABC scores in the control group, rather than an increase in challenging behaviour by participants who received the intervention.

Figure 10e shows the proportion of service users in each group who met the criterion for severe challenging behaviour, which decreased by 23% in the intervention group at 16 weeks and remained at a similar level at 10 months. There was no change in the control group at 16 weeks, followed by a spontaneous decrease at 10 months.

No significant effects were reported for the MOAS. Neither did key workers' attributions of challenging behaviour, as reported by the CBS, change significantly (see Table 50).

Home carer reports

Home carers also reported improvements in challenging behaviour, relative to the control group, at the 16-week follow-up, as measured by the ABC-H (p < 0.05). A similar pattern of results was seen on the ABC-I, albeit these changes were not statistically significant (Table 50 and Figure 10b and d). The significance of effect of the intervention on the ABC-H was lost at the 10-month follow-up, but challenging behaviour continued to decrease on both measures. Effect sizes increased marginally when non-compliant participants were excluded (CACE analysis). No change was seen on the MOAS.

Figure 10f shows the proportion of service users in each group who met the criterion for severe challenging behaviour, which decreased by 25% at 16 weeks, with a further decrease to 32% of the baseline value at 10 months. In the control group the proportion showing severe challenging behaviour increased at 16 weeks, followed by a return to 87% of the baseline value at 10 months.

Complier-adjusted causal effect analysis

The compliance ICC (the level of cluster within centres in whether or not a service user was compliant) was calculated as 0.38. However, in terms of outcomes, compliers and non-compliers were not substantially different from each other. We would, therefore, not expect the clustering of compliance to contribute to the misestimation of variance in our CACE estimates, despite this clustering being considerable.

Secondary analyses of primary outcome

Secondary analyses of Profile of Anger Coping Skills outcome

Some secondary analyses were also conducted on the key secondary variable of anger coping at the 10-month follow-up assessment.

Influence of baseline mental health status

An exploratory analysis was conducted to examine whether or not the outcome at the 10-month follow-up was influenced by participants' baseline mental health status. As was done for PI scores, the primary analysis was extended by including baseline mental health scores (GDS, GAS and RSES) as covariates and interacting them with trial group, based on 128 service users (complete cases) nested within 28 centres. Both the main effect and interaction term were non-significant for RSES scores (p ≥ 0.67). For GDS scores, the main effect was significant (p < 0.01) but the interaction was not (p = 0.46), indicating that participants who were more depressed at baseline had a poorer prognosis at the 10-month follow-up, irrespective of trial group. However, for GAS scores the main effect and the interaction were both significant (p < 0.01 and p < 0.05, respectively). These effects are illustrated in Figure 11. For participants in the control group, higher levels of anxiety at baseline were associated with a poorer prognosis at the 10-month follow-up.

FIGURE 11.

The GAS by trial arm interaction.

However, for participants in the intervention group, higher levels of anxiety at baseline were associated with greater benefit from the intervention.

Theme 1: What we did in the group

Most service users described the overall purpose of the group:

Talking about your feelings, talking about things that you can do to control your anger.

The group tried to help me calm myself down...not to get angry.

as well as specific group activities, such as relaxation exercises:

You use your ... put your toes up in the air, do your breathing, your arm, then go off into a happy place then.

Techniques, like, you could count to 10 (...) and take deep breaths, and, if you get angry – walk away (...) and ... just relax. (...) Just do something else, take your mind off the situation and do something, do something else.

and role play:

We had a line (...) somebody pushed in and then somebody shouted at them and told them to get back in the queue (...) which I thought was funny.

I was on the computer (...) doing some work, and M... came up to me and was like, ‘J ..., I want to talk to you,’ and I was like, ‘Wait, I'm busy,’ ... I was acting.

All service users' responses indicated that they had clear memories of the group sessions.

Theme 2: What it was like to take part

A number of service users reported to have been nervous before they started the group:

I was nervous (...) because it was a new thing for me.

I just felt shy about it.

but that this was short-lived and the overwhelming majority of accounts were positive evaluations:

... just find it interesting and fun.

It was good. I enjoyed everything.

I love that group.

Although the interviewer repeatedly prompted each of the service users to suggest improvements for future groups or state what they had found least useful/positive, negative evaluations were reported by only two participants: one who commented on his dislike of another group member and another who found one part of the relaxation exercises difficult.

Most of the participants emphasised the therapeutic value of sharing experiences and the opportunity to talk about problems, as well as their recollections of having experienced a sense of fun and enjoyment:

We couldn't stop laughing.

I liked people talking to me.

I worked out that if you're swapping stories it helps each other out.

Theme 3: What difference the group made to my life

The service users were able to identify a number of positive changes in their lives which they attributed to the CBT group intervention and spoke of using the newly acquired strategies in their everyday lives. Most focused on the use of ‘walking away’, ‘counting to 10’ and ‘asking for help’. Several participants described incidents in which they had used strategies successfully:

That (relaxation exercises) is what kept me from going overboard today.

They described improved relationships with peers:

Me and my housemate used to argue and I used to go off on one, but since this course I haven't done that.

and improved mood and the ability to regulate their arousal state:

I'm just the same person, but ... if I get angry I talk about what's annoying me ... makes me feel much, what's the word, makes me feel much better ... with myself.

Cause I'm a different person now, I used to be all boisterous but I'm not no more, I've calmed right down.

Not in a bad mood in the house now. (...) Eh ... up in the morning. Not mad. Not mad. Not else. Stop the crying ...

One service user described how these improvements were limited to the settings in which he felt comfortable and were not apparent in a club he attended where he was not happy.

Theme 4: Presenting a positive self

Participants frequently expressed pride in what they had learned in the group and achieved in real life, such as the ability to control anger. They also mentioned the praise they had received from others:

Just go for a walk on your own and count to 10 in your head. (...) That's what I done good with that.

Happy that (therapist) told (key worker) that I worked hard.

Quantitative outcomes

The uniformly positive feedback from the service users interviewed raises the question of whether they were typical of their group or had experienced particularly positive clinical outcomes. Table 54 compares the interviewees with the other service users in the intervention group of the trial who were not interviewed. The interviewees had higher PI scores and slightly lower PACS scores at baseline, with larger changes in both their PI and their PACS scores at the 16-week follow-up assessment.

Theme 1: Getting started; hopes, fears and readiness

Participants felt that taking on the role of therapist was an opportunity with a number of potential benefits, such as the chance to develop their professional knowledge and skills, enhance their support of service users, and enable their service organisation to develop current models of proactive support for service users with anger issues and related behavioural difficulties:

We thought it would be a good thing, one for our service users but secondly for ourselves, perhaps to get some ideas to use as a tool for our own working practices ... I'd perhaps get some more insight into one or two of the clients and there might be a general thing that we could use in other ways with other clients.

However, despite recognising their involvement as an opportunity, all participants described feeling initially apprehensive about running the anger groups. Reasons given related to personal doubts regarding how ‘qualified’ they were to run a CBT group and feeling daunted by taking on what they considered to be a specialist role. Participants expressed their uncertainty about how effective and credible they would be as therapists:

The only fear I had was would they be able to listen to me, would they think wait a minute you're just a support worker, why should I listen to you?

Other concerns related to more practical issues, such as whether or not they would be able to protect the time within their working week to run the group. Some felt concerned about how service users would respond, for example querying how well they would engage or form a cohesive group. Notwithstanding their initial feelings of doubt and apprehension, participants were unanimous that the training they received from the clinical psychologist provided sufficient information and guidance to feel prepared and confident to run the group and to understand the principles of group work and the CBT model. The assurance of regular supervision from a clinical psychologist throughout the intervention and provision of a manual as a guide was mentioned as helpful in allaying their fears.

Theme 2: Using the manualised approach

Generally, participants provided positive feedback about the manualised approach. They reported that the manual provided a useful framework and that the session plans outlined were practical and had saved them time in preparation. The manual was described as user-friendly and jargon free, and the fact that all the relevant resources and materials were provided was considered helpful:

The manual was absolutely first class because it told almost verbatim what you were going to be doing and that instilled you with confidence because you knew what was coming next. The other worker and I would meet before each group and say ‘this is what we're doing, this is how we're going to approach it’.

However, two of the participants commented that they had experienced the manualised approach as somewhat constraining and they would have liked more flexibility to modify the session content and how long to spend on different activities. A difficulty expressed in this respect was feeling under pressure to bring discussions that were in full flow to a halt to fit everything in and finish on time.

The overall feedback about manual content was positive regarding the activities and taught techniques. The material was felt to be accessible to service users and the 12-week period for running the groups was thought to be about right. However, limitations were also highlighted. One participant mentioned that the technique ‘count to 10’ was perceived as unhelpful by group members. Two participants commented that role play in their groups had limited success (this was a principal technique used to help group members to increase their awareness of triggers to anger and to practise coping strategies). Group members were reported as either reluctant to take part in role play or, when they did, struggling to make meaning of it:

They did enjoy the role plays and they took part themselves, but it just seemed to be lost once the role play was finished. We tried different things and different strategies, there were three of us and we mixed them up and shuffled the pack ... but everyone sort of agreed that the role plays were definitely the weakest part.

Moreover, complex coping strategies that involved cognitive concepts such as ‘thinking differently’ and using assertiveness were considered to be too complicated for some group members:

You know, the idea of looking at it from someone else's perspective, she [group member] had the words but she only had half the idea if you see what I mean and her capacity is quite high so I think others were struggling more than that.

Theme 3: Making the group work

Participants described factors that were essential to the groups running successfully. Skills of facilitation were considered as fundamental to the role of therapist and included working well with a broad ability range in the group. Communicating effectively, pitching the information at the right level and working at the right pace to aid understanding and help maintain engagement were considered important. Having people with differing levels of ability working together in the same group was mentioned as a difficulty. Two participants commented that tensions could arise when more able group members were quick to understand topics, whereas others required repeated explanations. However, having more able group members who helped to explain things could be positive, as it promoted supportive and cohesive group dynamics.

Other aspects of managing group dynamics included recognising individual differences within the group, such as encouraging quieter group members to develop their confidence and ‘find their voice’, avoiding more talkative people from dominating, and being able to manage the flow of the group discussions:

Everybody gets a chance to speak and everybody gets a chance to listen ... some people would go off on a slight tangent but we would guide them back and I think that's basically through eye contact, your general style, your proximity to them and your empathy, or redirecting questions if someone kinda wanders slightly you can redirect a question to bring them back to where they should have been.

Creating a therapeutic environment through the development of supportive relationships and trust within the group, while setting appropriate boundaries, was described as paramount, for example by keeping confidentiality and adhering to group rules:

We had rules and we posted them up on the wall every week and we would always go back if someone was butting in ... and at the very top is privacy and what's said in the room stays in the room and the privacy thing was a big issue. But I think because the group trusted each other and they were discussing things and things were getting said and brought up, under normal circumstances people would be running about telling everyone, but with the group it seemed to stay private and things were never discussed out in the open.

Although being fully aware of their remit to deliver a manualised group intervention, the participants recognised a need for group members to raise things that were important to them and, in so doing, demonstrated a flexibility to respond to individual needs. Participants encouraged the group members to share ‘ownership’ of the group and to steer the direction of the discussions where appropriate.

A number of participants felt that it was important to avoid establishing themselves as group leaders but to convey a sense of ‘all learning together’. They did this by sharing their own experiences of anger, and some even completed the homework activity (although they had not been asked to do this). They felt that this not only helped to reinforce key messages of the group work, such as ‘normalising’ anger as one of many basic human emotions, but also provided opportunities to ‘model’ coping with anger. Participants who adopted this approach felt that it served a crucial function in helping service users to feel more ready to discuss their own experiences, particularly those who were initially reluctant to admit to anger difficulties:

It took them a while initially and I think bringing in scenarios where we'd get really, really cross and did something that we weren't proud of and us being to say ‘No, I felt ashamed of what I did’ ... and because we were able to bring humour into it as well, but not laughing at what we'd done but being able to say ‘I felt really ashamed and I had to go and sit in my bedroom on my own’ and that sort of thing, being able to say it's okay to feel ashamed and it's ok to feel cross, upset, or crying, and that was good.

Certain aspects of group dynamics were more challenging to manage. These included handling the level of openness within the group and disclosure of information of a highly sensitive or distressing nature. Engaging with the service users at a level different from what was normally expected and having to manage the personal impact of hearing emotive issues were highlighted as important. Supervision was identified as a mechanism for obtaining support:

Some things came out like really private you know about family and things. At first it was uncomfortable and it was giving me cause for concern, but we then had supervision ourselves so we were able to talk about it and discuss it ... From a therapist point of view it was difficult because some of the things you are hearing aren't pleasant. You know what goes on and think you can't do anything about it, you can do as much as you can but you can't stop it all can you?

Theme 4: Observing progress

Many accounts were given of service users making considerable progress in terms of increased anger control and personal development in other areas:

Self-confidence, communication skills improved as well, it improved understanding of emotions, it improved a lot of things and I think it improved anger levels as well which I think it should have done!

It was recognised that better anger control led to systemic benefits, such as a sense of improved relationships and a more relaxed social environment at the day centre overall:

I think it's a good tool, I think the majority of it has worked for those who attended and I think because of that things here are a lot better ... if we get another group in for some that haven't done it we might help those people, which then again has that knock on effect of making life that little bit easier for the rest of us.

However, a few group members were described as having difficulty applying the skills taught in the group:

I'd like to say it was totally rewarding but it was also very frustrating because we'd go through things in the group and it was clear that people had an understanding ... the interesting thing and very frustrating thing was that they could apply that knowledge to other people but they didn't always apply it to themselves. They talk about something here about how not to do it and they go straight back home and do it!

Theme 5: The ingredients of success; factors influencing group outcomes

Participants commented on aspects of the group process that they thought contributed to service user progress. The development of self-awareness and control was a central factor, including developing awareness of triggers, recognising feeling angry, and making connections between thoughts, feelings and behaviour:

The main outcomes would be recognising what anger is, recognising the physical effects, recognising triggers and understanding techniques for dealing with these emotions, physical feelings and triggers and applying them and thinking about it afterwards. I think reflection is a big part of it as well, I think the hassle logs, they did their part because they allowed people to reflect.

Offering alternatives through psychoeducation was considered as a key contributor to progress, as it increased group members' awareness of different strategies and offered them positive suggestions about what they could do rather than stressing what they should not do:

I think it's the feeling that people could choose, make choices about what they did, whether they could ignore something. It was the strategies I think that it gave, things you could do because quite often we say ‘Don't get angry’, but that's not very helpful, but you know do go and listen to some nice music, do walk away, do think that person's got out of the wrong side of bed today ... these were useful things to offer people’.

Finally, although two of the groups reportedly struggled with the role play, for the most part it was considered to be a crucial learning tool whereby group members could observe angry situations acted out, which helped them to gain insight and practise coping techniques:

Creating examples that people can relate to I think was a big factor you know, you could sit there for two hours and read something and people are like ‘Well I sort of understand what you are talking about but where are you going with this?’ and when you introduce a role play exercise it keeps people focused, you could find that people drift off and get a bit detached from what is being delivered. But when you're delivering information and you role play it, you demonstrate it and it brings it all together and that was good to see that work.

Theme 6: Taking on the ‘therapist’ role

Participants recognised a contrast and continuity with their usual role. They experienced a contrast in terms of working in a more focused and at times more intense way, but they also identified a crossover of skills between the two roles:

But then as a day service worker or a support worker you deal with emotions every day and you have to do some kind of counselling role, sometimes you don't realise it but you do quite a lot of counselling roles in there. I think if you've got a good relationship with who's in the group to start with then you're kinda sorted when it starts.

Several of the participants asked ‘Was I really a therapist?’ and preferred to think of themselves as a group leader or facilitator:

I think facilitator, yeah facilitator is a better word because the service users are really taking you where you are going, although we know we're guiding, the book's very good at guiding you where, but what comes out of the service users' mouths is very interesting. I do think it's based on relationships and bonding and trust, and to go with the flow. I enjoyed going with the full flow of it and I really didn't think it was a difficult task to do at all.

In whatever way they interpreted the role, participants described feeling effective and confident and felt that their confidence grew over time. Some said that taking on the role had enabled them to develop closer relationships with service users, characterised by greater trust and a stronger bond than existed before:

It's as if we have all been through something together and we all seem to have like a bond that was formed, not that you treat anyone different but it just feels different with that group ... I think it's because we shared so we're more open and honest with each other, I think it's just made us closer like groups become.

Participants considered that the therapist role had helped them to develop their knowledge and skills by gaining a better understanding of anger issues and developing a different understanding of group members' support needs. Use of supervision was also regarded as a useful further way to develop skills and confidence in the role. Participants' enjoyment and satisfaction in running the groups was clear. Without exception, their comments expressed a sense of truly valuing the experience and the opportunity it provided.

Theme 7: The future; taking the group work forward

A number of participants had plans to run another group, albeit that finding the time and additional staff cover to run the group were potential barriers to doing so. Having the skills to run a group ‘in house’, especially in the current financial climate, was recognised as beneficial:

We're also at times asked by statutory agencies if we know can anybody run an anger management course so we say yes we can do that, because there aren't many. I mean one of the organisations that we did use has lost their funding and so that has been a loss to us ... it's a matter of not accessing things very easily.

Participants recognised that service users who had attended the groups would continue to require support to maintain their progress and that staff would need to take a proactive role in this. It was also noted that sharing information with other staff could enable knowledge and skills to be disseminated and increase the staff team's awareness of anger support strategies, which potentially could help to maintain gains that were made in the CBT group.

What made your service decide to join the research trial?

What are the current service policies and practices?

What are the causes of the behaviours that challenge your services?

What are the consequences of behaviour that challenge services?

What works well?

What does not work well?

What is the service like now?

Any plans for further group work?

The participants were asked whether or not they intended to offer group work in the future. All of the managers reported that they were planning future groups.

Any suggestions for improvements?

Finally, managers were asked to think of suggestions for improving the group intervention.

Theme 1: Benefits of group work

Both service users and therapists experienced the group as an environment where it had been possible to get to know each other better and in a different way, developing relationships and trust by sharing experiences and having fun.

All three groups (service users, therapists and managers) commented on the benefits of the group ‘spilling out’ into the wider service, changing the organisational culture for the better and, in some cases, having a ‘knock-on’ effect where the ability to control anger on the part of some of the service users reduced the number of incidents and, therefore, improved relationships, the environmental groups, and the emotional well-being of others attending and working in the service.

All three participant groups also noted that a number of service users' self-awareness and self-control had noticeably improved as a result of group attendance.

Managers and therapists observed that staff came to work better with each other and in a more consistent manner with service users. They also saw the group intervention and the staff training involved as a springboard to further opportunities for service development, and appeared motivated to run more groups in the future, albeit in some cases refresher sessions rather than providing the whole course to a new set of service users.

Theme 2: Using ‘in-house’ therapists

Managers, service users and therapists all noted the significant advantages of having ‘in-house’ therapists who are present and available to help with ‘homework’ and encourage the use of skills learned in the group in other contexts, thereby enabling generalisation and maintenance of skills over time.

Therapists emphasised the positive impact of facilitating the group (and receiving training and clinical supervision from psychologists) on their confidence and job satisfaction, despite initial anxieties about taking on the role. They also reported that the experiences gained during the training and intervention phases of the project had increased their insight into service users' problems, which allowed them to respond in more psychologically informed and skilled ways. These positive impacts were echoed by their managers.

Theme 3: Using the manual

Most of the participants in all three groups regarded the manual as a useful tool, providing structure and thus reassurance to the therapists. The specific exercises contained in the manual were considered to have been effective and enjoyable. Role play was remembered more clearly and fondly by service users and (with two exceptions) therapists than any of the other exercises, indicating that these ‘in vivo’ experiences not only provided insight into specific behavioural and cognitive response modes and their functionality, but also promoted memory retention.

Service users did not suggest changes to the manual (despite being prompted and encouraged in a number of ways to critique the course and the materials used). However, managers and therapists proposed a number of changes which mostly concerned lack of flexibility, accessibility and inclusivity of the manual, and reducing the number of sessions needed to complete the course.

1. Intellectual ability

There is some evidence that intellectual ability influences the outcome of anger management interventions for people with intellectual disabilities but the data are variable. Three studies have reported that outcomes were better in more intellectually able participants, but inconsistently:12 one study41 found a relationship with receptive language ability as assessed by the BPVS, a second39 found a relationship with Full-Scale or Verbal IQ but not Performance IQ, whereas a third42 found a relationship with non-verbal ability but not receptive language ability. A fourth study found no significant relationship between intellectual ability and outcome. 49 Nevertheless, on the balance of this evidence, we predicted that higher levels of intellectual ability (IQ and/or BPVS scores) would be associated with better outcomes for anger management.

2. Mental health

It is well established that depression (but not anxiety) is one of the strongest risk factors for non-compliance with medical treatment. 148 Depression has also been reported to predict a poor outcome for treatment of externalising psychological disorders, such as substance abuse149 or bulimia. 150 Consistent with this literature, high levels of depression and low self-esteem (but not anxiety) were associated with a poorer outcome for anger management. 151 We therefore predicted that high levels of depression and/or low self-esteem would be associated with poorer outcomes in the present study.

3. Compliance

There is a well-established dose–response relationship in psychotherapy, with the greatest gains occurring in early sessions and diminishing returns thereafter. 152,153 A recent meta-analysis of anger studies in the general population confirmed that effect sizes increased significantly with more sessions, although the differences were relatively small: the effect size for 12 sessions was roughly 10% larger than that seen with four sessions. 64 The mean number of anger treatment sessions delivered was 8.5. It was suggested that eight sessions should be adequate to demonstrate decreases in anger, and that treating angry clients beyond eight sessions provides limited further benefit. 64 However, psychological interventions for people with intellectual disabilities typically progress more slowly than with more intellectually able clients. 154 On the basis of this literature, we predicted that outcomes would improve with the number of sessions attended.

4. Fidelity

There is evidence that the fidelity of delivery of CBT makes an important contribution to the effectiveness of the intervention. 62,63 The above-mentioned meta-analysis of 96 studies of anger treatment in the general population reported that manualised interventions achieved better outcomes than non-manualised interventions (effect size: 0.81 vs 0.76) and studies that used fidelity checks achieved better outcomes than those that did not (effect size: 0.85 vs 0.73), albeit that neither effect achieved statistical significance (p = 0.059 and 0.090, respectively). 64 We predicted that higher fidelity would be associated with better outcomes.

Service user demographics

The only significant demographic effect was an association between higher IQ and larger decreases in key worker PI ratings. There was no significant effect of receptive language ability. The service users' age and gender were not significant predictors of any of the outcome measures.

Service user pre-intervention mental health

Higher pre-intervention levels of depression and lower self-esteem were associated with smaller decreases in key worker-rated PI scores.

Higher baseline anxiety levels predicted less improvement in coping skills (service user PACS). This effect is in the opposite direction to that described in the main trial but as both of these results were of marginal significance (p = 0.048 and p = 0.039, respectively), and considering the large number of analyses that have been conducted, they may represent chance findings.

Some non-significant trends in relation to depression and anxiety scores were also noted (see Table 58). No significant effects were associated with baseline measures of quality of life (ComQoL) or key workers' attributions of the service users' control over challenging behaviour (CBS).

Lay therapist characteristics

There was no effect of experience or educational qualifications but better outcomes were achieved by younger lay therapists (larger decreases in service user PI scores) and by female lay therapists (larger increases in service user PACS scores).

Implementation quality

The number of sessions attended and fidelity of delivery were associated with reduced and increased service user PI scores, respectively.

Tests of a priori predictions