Assessing the risk of self-harm in an adult offender population

Self-harm definition

Self-harm is known by many different names and has been defined in a number of different ways. These include the following definitions:

The term self-harm covers a spectrum of behaviour. The most serious forms relate closely to suicide, while behaviours at the milder end of the spectrum merge with other reactions to emotional pain.

Skegg, 20051

. . . the deliberate destruction or alteration of body tissue without conscious suicidal intent.

Favazza, 19892

Self-injury is a behaviour that involves deliberately injuring one’s own body, without suicidal intent and with or without pain.

Duffy, 20063

. . . self-poisoning or self-injury, irrespective of the apparent purpose of the act.

NICE, 20044

From these differing definitions, it is apparent that there is a lack of consistency in how self-harm is defined. This confusion increases with the introduction of the multiple names by which the concept of self-harm is known. The phenomenon of self-harm is also known as self-injury, self-injurious behaviour, self-mutilation, deliberate self-harm, deliberate self-injury, non-suicidal self-injury, self-cutting, self-mutilative behaviour and parasuicide. Some of these names refer to a narrower definition than others, but generally they all refer to the notion of a self-harm event, regardless of the intent and motivation. However, self-harm is also often associated with suicide, and the following names are often used in situations in which the final outcome of death is seen as the primary motivating factor: suicide attempt, suicidal behaviour, suicidal gesture and suicide ideation (parasuicide may also be included in this list).

It has been recognised5 that the terms used to describe self-harm could be harmonised, as the variety of different names and terminology creates confusion regarding which specific construct is under investigation. 6,7 It has also been stated that part of the difficulty in understanding self-harm is the result of the multiple terms used to describe the behaviour,8 and the confusion surrounding whether or not self-harm represents a suicide attempt. 9 Nock and Prinstein10 stated that a broad classification of self-harming behaviours includes actions ranging from stereotypic skin-rubbing to completed suicide. This corresponds with the view of Skegg,1 who contended that the term ‘self-harm’ covers a spectrum of behaviour. It has, however, been suggested that attempted and completed suicides should be treated as aetiologically distinct from self-harm. 11,12 Messer and Fremouw5 have pointed out that the lack of distinction between those who are attempting suicide and those who are mutilating with no intent to die is particularly concerning. It is suggested that differentiating between these two groups is key when examining functions or explanations of the behaviour. This may improve how research is interpreted and prevent confounding results and obscuring relevant findings. 5,7

In contrast to the above perspective, Lohner and Konrad12 reported that some consider both of these phenomena to be on a continuum of lethality, and they consider any differentiation to be irrelevant, confusing and possibly even dangerous. 13,14 Messer and Fremouw5 recognised that matters are further complicated by the findings that self-harmers are at greater risk of attempted suicide and suicidal thoughts and are more likely to have a history of suicide attempts. 8,15 This supports the previous finding that approximately 55–85% of self-mutilators have a history of at least one suicide attempt. 16 A strong statistical connection between self-harm and subsequent suicide has also been reported, and it has been estimated that around one-quarter of suicides are preceded by self-harm in the previous year. 17,18

Although self-harm and suicide attempt may be separated by the motivational intent, this may be irrelevant to the primary care teams and authorities that are charged with dealing with any sort of self-harming behaviour, regardless of the prior motivating factor. This view is supported by Lanes,19 who stated that it is important to note that self-harmers generally distinguish between self-harm and genuine suicidal intent, but this does not qualify as a basis for judging the potential outcome of threatened or enacted self-harm. Despite the motivational and aetiological differences between self-harming and suicide attempts, as the final outcome is likely to be similar in terms of treatment cost and impact, it may make sense, from a public health-care commissioning perspective, to group all self-harm behaviours together, regardless of the intent.

Considering the public health implications that are present in the prison setting of this study, the definition of self-harm provided by the National Institute for Health and Care Excellence (NICE)4 may potentially be the most appropriate; here, it is described as self-poisoning or self-injury, irrespective of the apparent purpose of the act.

This definition is all inclusive and, thus, relates more closely to epidemiological outcome events. However, given the strength of the arguments for separation of the phenomena of self-harm and suicide attempt, research in this area may be problematic, as epidemiological outcome event statistics may not distinguish between the two without a degree of more in-depth information being available.

Self-harm in the community

The best current UK estimate of hospital attendance as a result of self-harm is 400 per 100,000 hospital attendances (0.4% of all hospital attendance). 20 The current incidence of self-harm is estimated at between 300 and 600 cases per 100,000 per year. 21,22 Despite difficulties in diagnostic classification, self-harm is one of the commonest reasons for admission to a medical ward, with around 200,000 hospital attendances per year in the UK, with the majority of these cases (80%) involving self-poisoning. 23 However, it is widely recognised that prevalence rates of self-harm behaviour in the general population are difficult to estimate given that the self-harm may go unreported and not result in a hospital attendance. 17,23,24 Among the general population (who do not routinely present at accident and emergency), physical self-harm is more common, with cutting being the most common form. 25

Prevalence and incidence estimates are likely to be affected by the different classifications and terminology used when quantifying self-harm, along with what is judged to be a meaningful history of self-harm. Depending on classifications, self-harm behaviours may range from lip chewing or lightly biting the inside of the mouth, right through to a genuine suicide attempt. These behaviours are difficult to quantify, and a direct comparison of estimates would also require the definitions of self-harm to be explicitly stated and to remain consistent between studies. With this in mind, the prevalence of reported self-harm is highly variable. Jacobson and Gould26 reviewed eight studies, two involving adults and six involving adolescents (broadly defined as ‘mainly high school students’), and reported varying 12-month prevalence rates of 2.5–12.5% and lifetime prevalence rates of 13.0–23.2%. Muehlenkamp and Gutierrez8 reported that estimates of self-injurious behaviour among adolescents range from 5.1% to over 40%, and Skegg1 stated that 5–9% of adolescents in western countries report having self-harmed within the previous year, with lifetime prevalence ranging from 13% to 30%. It has also been reported27 that self-harm occurs in 4% of the general population28 and 14% of college students. 29 Furthermore, Gratz7 reported that 35% of college students have carried out at least one self-harm behaviour in their lifetime. Along with the issue of differing self-harm definitions, limitations may also be present in these estimates because of sampling biases and interview methods.

Large surveys suggest that 4.6% of the population in the USA and 4.4% in the UK have self-harmed. 23 These results are similar to those of Meltzer et al. ,30 who reported that 14.9% of respondents in a national survey had contemplated suicide at some point in their life and that 4.4% of respondents had actually attempted suicide at some point in their life. In all, 2% of all respondents stated that they had deliberately harmed themselves without suicidal intent. This was a large, national (UK) study involving a representative sample (n = 8450) and should, therefore, provide a fair representation of the adult population (aged 16–74 years). It should be noted, however, that these results are based on a single self-harm question; therefore, an element of subjective judgement may be present, along with the recall bias limitations of retrospective studies.

Characteristics of self-harmers

While self-harm can be found across the entire population, it is more common among those who are socioeconomically disadvantaged and those who have limited social support. 30 Those with mental health disorders are 20 times more likely to report having harmed themselves. 30 Among respondents who had reported a lifetime prevalence of self-harm, 57% were categorised as having a neurotic disorder, 6% as having a psychotic disorder, 24% as alcohol dependent and 16% as drug dependent. 30

Self-harm in prisons

Given the increased prevalence of self-harm in those from socioeconomically disadvantaged areas, and in those with mental health problems, it is not surprising that self-harm presents a significant problem within prisons. 31 Self-harm in prison custody is defined as ‘any act where a prisoner deliberately harms themselves irrespective of the method, intent or severity of any injury’. 32 This definition corresponds to the NICE4 definition mentioned previously. The use of this definition in the prison setting is supported by Lanes. 19 He points out, with reference to the different perspectives on self-harm described above, that in the prison setting the distinction between self-harm and suicide attempt is unlikely to be useful in terms of overall management of the prisoner given that prison authorities are ultimately concerned with preventing both suicides and self-harm events. 33

Within offender populations, certain groups are recognised to be at greater risk of self-harm, including those who are psychiatrically ill, those with long sentences and ‘poor copers’, who are defined as acutely vulnerable prisoners whose major problems are unrelated to psychiatric illness or the nature of their offence. 34 ‘Poor copers’ tend to be young offenders (under 26 years) who have committed acquisitive crimes and have a poor ability to cope with being in prison. 35 Even controlling for the characteristics of a prison sample, rates of self-harm in prisons seem to be much higher than they are in the general population. 36

Self-harm incidence in prisons

There are differing estimates of self-harm incidence within offender populations and corrective institutions. Again, these differing estimates are possibly a result of the different definitions of a ‘self-harm event’. Appelbaum et al. 37 identified that published research has estimated that 30% of prisoners engage in self-harming behaviour. 38 In addition, 50% of female prisoners are stated to have a history of self-harm. 39 The proportion of prisoners engaging in self-harm in American prison systems during 2008 varied from 0.03% to 8.93% across prison systems, with an overall rate of 0.71%. 37 In marked contrast, the prevalence of self-harm behaviour among Greek male prisoners was reported to be 49.4%. 40 Potential reasons for this discrepancy include differing classifications of self-harm, differences in the samples (cultural, diagnostic, offender demographic, etc.) and differing modes of data collection. It may be worth noting that the Greek data40 were derived from face-to-face prisoner interviews, whereas the American prison system data37 were derived from recorded events within prison institutions.

Given this discrepancy in reported prevalence rates, it is important to note how self-harm data are gathered. In the UK, the most complete data are likely to come directly from the offender management statistics. 41 These statistics are published quarterly and are therefore likely to be the most up-to-date estimates that are available. Although there may be some deviation between individual institutions, these statistics relate to actual recorded self-harm events, so the classification of a ‘self-harm event’ is likely to be broadly consistent across all institutions. However, it should be noted that unreported and untreated self-harm events will not be accounted for.

The number of incidents of self-harm in UK prisons rose rapidly between 2003 and 2005. By 2005, there were 23,781 incidents of self-injury in UK prisons, rising from 16,393 incidents in 2003. This rise of 45% was over 11 times the rise in the overall UK prison population for the same period, which was just over 4%. Between 2005 and 2011, the incidence of self-harm in prisons seems to have largely stabilised (Figure 1). This stabilisation could be a result of the prison response to the previously observed rise.

FIGURE 1.

Changes in the overall UK prison population and the number of reported self-harm incidents 2003–11.

According to the Ministry of Justice,32 there were 24,648 incidents of self-harm reported in 2011, with roughly two-thirds of these attributed to the male inmate population. These self-harm events were carried out by 6854 individuals, with 82% of these being males.

An overall incidence rate cannot be accurately calculated because of the transient nature of prisoners within the system and the lack of statistics regarding the turnover of prisoners. However, using the average number of prisoners within the system in 2011 (85,951), the overall approximate yearly incidence of self-harm within prisons is 8%, with a rate of 6.9% for males and 29.4% for females. This equates to 194 self-harm incidents and 69 self-harming individuals per 1000 male prisoners, and 2104 self-harm incidents and 294 self-harming individuals per 1000 female prisoners. Among the individuals who self-harm, males report an average of 2.8 self-harm incidents per individual and females report an average of 7.1 self-harm incidents. Although prison turnover has not been taken into account, these values are approximately twice those reported in the Corston Report,42 in which it was stated that 16% of women self-harm in prison, compared with 3% of men.

Implications for the prison system

Self-harm can present a major challenge and place considerable demands on prison health-care systems,19 the responsibility for which resides with primary care trusts. In 2007, the prison service introduced a care-planning system called ACCT (Assessment, Care in Custody, and Teamwork)43 to improve care for prisoners at risk of suicide or self-harm. The ACCT process effectively establishes an assessment and care pathway system (CAREMAP) for those deemed to be at risk; however, it does not incorporate a standardised diagnostic test to estimate the risk of future self-harm.

There is some evidence to suggest that screening for psychiatric illness upon entry to prison can help to identify true cases of psychiatric illness. 44 This early indication of psychiatric illness is beneficial to prison staff in terms of prisoner management and, therefore, suggests that a screening process can be useful. However, the evidence to support the routine use of any screening instrument for self-harm in offender populations is limited. A recent review article45 assessed screening tools that have been used to assess the risk of suicide and self-harm in adult offenders. This review identified four screening instruments across five studies. Three of these instruments were specifically aimed at screening for suicide (or suicide risk) rather than self-harm (or risk of self-harm). Furthermore, two of the studies used retrospective methodology, which may result in non-comparable information between study participants. Limited evidence suggests that the Beck Hopelessness Scale (BHS)46 was predictive of self-harm among offenders with mental disorders. 47 Several other scales are available for assessing the risk of self-harm, for example the Self-Harm Inventory (SHI),48 but few have been validated for routine use in offender populations. A newer scale, Suicide Concerns for Offenders in Prison Environment (SCOPE),49 has been specifically developed to assess vulnerability to risk of suicide and non-fatal self-harm behaviour in young adult offenders but, again, has not been tested with regard to routine implementation in prisons, for those of older ages or for prospective predictive validity.

The limited evidence for the use of screening instruments for self-harm in prisons led Perry et al. 45 to conclude that ‘There is a clear need for additional psychometric research on the validity of suicide and self-harm behaviour screening tools in offender populations.’

Scoping method

There are many questionnaires available to assess and/or screen for self-harm, some of which relate specifically to self-harm behaviours (e.g. the SHI48) and some of which relate to other underlying correlates of self-harm such as depression [e.g. the Patient Health Questionnaire (PHQ)50]. Perry et al. 45 recognised that there are problems with the transferability of existing screening and assessment instruments to a prisoner population as a result of the unique environment in which prisoners are accommodated. Some instruments, however, have been explicitly designed for, or validated within, specific offender populations. 47–49

The first stage of the project involved a scoping exercise to systematically identify available instruments that could be used to screen for self-harm. A search was carried out with the Scopus database [encompassing MEDLINE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature (CINAHL) and EMBASE], using appropriate search terms such as ‘self-harm’, ‘self-injury’, ‘suicide ideation’, ‘prison’, ‘jail’, ‘risk’, ‘questionnaire’ and ‘screen’. All journal article titles and abstracts were read for any mention of self-harm measurements or scales. This was followed up with a search of the grey literature (e.g. university theses, commissioning reports, etc.) and a related internet search.

Once the instruments were identified, a range of practical inclusion criteria had to be fulfilled prior to assessing the psychometric properties of the applicable scales according to a standardised protocol.

The practical inclusion criteria included the following:

• The instrument must be able to be administered by generic primary care/prison/research staff who may not have had mental health or clinical training.

• The instrument must be able to be administered orally by staff rather than self-administered (because of low literacy levels).

• The instrument must be able to be administered without specialist training specific to the instrument, in line with the circumstances in which it would be administered on prison reception. This is also a practical point with regard to the implementation of the research project.

• The instrument must not be specifically designed for administration after a self-harm event (people at risk may or may not have actually carried out an act of self-harm).

• The instrument must comprise closed questions with a discrete response format to allow for objectively measured responses and consistency among respondents. This response format also allows for direct psychometric analysis of individual questions and their corresponding response format.

• The instrument must be brief, in line with the circumstances in which it would be administered in a prison environment. Any instrument containing more than 50 individual questions was excluded as inappropriate.

• The instrument must be available for use within the study.

The psychometric criteria that were assessed included:

• Has the instrument been used to directly screen for self-harm?

• Is the instrument directly related to self-harm (or a self-harm correlate)?

• Has the instrument been validated for an offender population?

• Have the psychometric properties of the instrument been assessed?

Each instrument was rated in terms of its practical application and psychometric properties and then a set of potential instruments was taken forward to an expert panel meeting (consisting of two psychometricians, two prison-based clinicians/researchers, a forensic psychologist, a psychological medicine and health-care researcher, and a service user, all with relevant experience), in order to reach a consensus on the instruments to be used in the pilot study.

Within the expert panel discussions, the same practical and psychometric criteria were applied to the instruments, along with any further practical information relating to prison policy or existing implementation processes. All comparative strengths and weaknesses of the instruments were considered. The aim was to select an array of scales from the potential set that might have moderately different focuses, thus maintaining a range of different screening criteria that could be tested. Where unanimous consensus could not be reached, disagreements were resolved by majority vote among panel members.

Scoping results

Once duplicates were removed, the initial search yielded 955 unique journal article records. Following the title and abstract screening, along with the grey literature and related internet search, 130 unique potential self-harm or suicide screening measurement instruments remained. Following the application of the practical and psychometric inclusion criteria, 13 potential screening instruments remained. The majority of these potential scales were removed as a result of inappropriate administration constraints (i.e. clinician-rated scales) or inappropriate or unspecific scale content (i.e. a scale specifically focused on anger or suicide rather than self-harm, without any self-harm component). Potential scales were also removed if they were specifically to be administered only after a self-harm event had occurred, if they were deemed to be too long or if no further information could be found on the identified scales.

The initial 13 potential screening instruments were as follows:

• Prison Screening Questionnaire (PriSnQuest)51

• SHI48

• Borderline Symptom List-23 (BSL-23)52

• SCOPE49

• BHS46

• Clinical Outcomes in Routine Evaluation – Outcome Measure (CORE-OM)53

• Depression Anxiety and Stress Scales (DASS-21)54

• PHQ-950

• The Referral Decision Scale (RDS)55

• Functional Assessment of Self-Mutilation (FASM)10,56

• Deliberate Self-Harm Inventory (DSHI)7

• Beck Depression Inventory (BDI)57

• Hospital Anxiety and Depression Scale (HADS). 58

Following the discussions of the expert panel, eight instruments remained. The instruments removed at this stage were the RDS, the FASM, the DSHI, the BDI and the HADS.

The RDS is primarily a screening tool for mental health disorders, which was developed for use within the US criminal justice system. This was discarded in favour of the PriSnQuest, which was developed to perform a similar role within the UK criminal justice system.

The HADS and BDI are both measures of depression, which is a correlate of self-harm. These measures were left out in favour of the PHQ-9, which contains similar content but is a shorter scale and is already used within UK primary health-care services.

The DSHI and the FASM are both measures relating to previous self-harm behaviours. These were left out in favour of the SHI, which covers similar content but has favourable psychometric properties. 59

The eight remaining instruments (PriSnQuest, SHI, BSL-23, SCOPE, BHS, CORE-OM, DASS-21, PHQ-9) went forward for use in the pilot study. The results of the scoping exercise are summarised in Figure 2.

FIGURE 2.

Summary of scoping process, from initial search to final instrument selection.

Pilot study methods

Following the identification of candidate screening instruments, a pilot study was undertaken in three prisons in northern England which were collaborating with the Prison and Offender Research in Social Care and Health (PORSCH) network: two male institutions (prisons A and C) and one female institution (prison B). The pilot study was undertaken over 6 weeks to determine several operational aspects of the screening process:

• the operational and safety requirements for introducing a screening procedure, identifying the most appropriate times and locations and the implications for staffing (e.g. prison officers’ time for escorting prisoners)

• evaluating the face validity and acceptability of the chosen screening instruments to prisoners, to assess for problems in their application

• discussions with ACCT assessors to see if they foresee and/or have observed any problems in the administration, reliability or validity of the chosen instruments

• evaluating the time taken to administer the questionnaire packs and gauging the opinion of the respondents regarding the burden of responding.

Furthermore, the pilot study also served the functions of providing:

• a sample on which to test the follow-up process

• an estimate of the incidence of self-harm during follow-up for main study power calculations.

The information gained from the pilot study was to have a direct impact on the final set of instruments selected for inclusion in the main study.

To limit the burden of the respondents in the pilot study, a block design was used, meaning that everyone taking part in the study was asked to respond to four scales (Table 1). Everyone responded to the DASS-21 and the PHQ-9, along with two of the other six instruments.

Pilot study results

Overall, 75 people were recruited to the pilot study: 50 (66.7%) were male, and 22 (29.3%) were female, with data missing for three (4%). Age ranged from 18 to 62 years [interquartile range (IQR) 23–39 years] and the median age was 28 years. Once a routine had been established, there were no problems reported with the process or logistics of running the pilot study.

Follow-up

At follow-up, 25 (33.3%) of the prisoners were still housed in the original prison, 28 (37.3%) had been released, 20 (26.7%) had been transferred and the status of two (2.7%) was not known (Table 2).

TABLE 2 - Status of prisoners at follow-up

Follow-up status	n (%)
Still in original prison	25 (33.3)
Released	28 (37.3)
Transferred	20 (26.7)
Missing status at follow-up	2 (2.7)
Total	75 (100)

The mean valid follow-up time was 172 days (SD 100 days). During the follow-up period, 30 (40%) prisoners performed a self-harm event (Table 3); however, the rate of self-harm varied by prison (Table 4). The number of self-harm events carried out by each individual during follow-up is shown in Figure 3.

TABLE 3 - Summary statistics for follow-up time, and time to first self-harm event

Max., maximum; min., minimum.

Statistic		Follow-up time	Time to first self-harm event
Study population
Valid, n		72	30
Missing, n		3	45
Number of days
Mean		171.65	64.80
Median		216.50	45.00
Range (min.–max.)		306 (1–307)	233 (1–234)
Percentile	25	73.25	18.75
	50	216.50	45.00
	75	253.00	106.75

TABLE 4 - Self-harm rate by prison

95% confidence interval (CI) for overall self-harm rate: 28.9% to 51.1%.

Self-harm	Prison			Total
	A	B	C
No, n (%)	10 (47.6)	8 (34.8)	24 (77.4)	42 (56.0)
Yes, n (%)	11 (52.4)	12 (52.2)	7 (22.6)	30 (40.0)a
Missing, n (%)	0 (0.0)	3 (13.0)	0 (0.0)	3 (4.0)
Total, n (%)	21 (100)	23 (100)	31 (100)	75 (100)

FIGURE 3.

The number of self-harm events carried out by each individual during follow-up, presented as a percentage of the full pilot sample (n = 75).

Of those who self-harmed, the median time to the first self-harm event (after the administration of the questionnaires) was 45 days. Importantly, in only one case was the first self-harm event after 6 months (Figure 4) and the rate of self-harm did not increase substantially as the follow-up time increased (Table 5). Table 5 also shows the cumulative self-harm rate and the number of prisoners lost to full follow-up via release and transfer for various follow-up periods. Pilot data suggest a loss to follow-up rate of 18.7% at 6 months (11 transferred without data available after transfer and three missing all follow-up data) and 22.6% at 9 months (14 transferred without data available after transfer and three missing all follow-up data).

FIGURE 4.

Time (in days) to first self-harm event (of those who self-harmed).

TABLE 5 - Cumulative self-harm rate and loss to follow-up rate as a result of release and transfer by follow-up time point

Includes the additional three prisoners missing all follow-up data.

Follow-up period	Self-harm rate, n (%)	Released/transferred with no further follow-up, n (%)	Loss to follow-up: transferred with no further follow-up,a n (%)
5 months	28 (37.3)	28 (37.3)	13 (17.3)
6 months	29 (38.7)	31 (41.3)	14 (18.7)
7 months	29 (38.7)	36 (48.0)	16 (21.3)
8 months	30 (40.0)	39 (52.0)	17 (22.6)
9 months	30 (40.0)	42 (56.0)	17 (22.6)

Initial sample size calculations

The sample size was primarily determined by the need to compare the areas under the curve (AUCs) between each pair of self-harm screening instruments. A secondary requirement was to achieve the relevant degree of precision required by the psychometric analysis (Mokken scale and Rasch analyses) and the Cox proportional hazards regression model.

An audit revealed that approximately 20% of inmates are assigned to an ACCT in any given year. Other work has shown that up to one-quarter of women could self-harm during their current term. 4,65

Thus, assuming a prevalence of self-harm of 20%, it was estimated that a sample of 405 prisoners would be required to achieve 80% power to detect a difference of 0.1 between a diagnostic test with an area under the receiver operating characteristic (ROC) curve of 0.8 and another diagnostic test with an AUC of 0.9 using a two-sided z-test with a 5% significance level. This calculation was based on discrete (rating scale) responses and assumed similar levels of variation for responses in prisoners with and without self-harm for both diagnostic tests, i.e. the ratio of the SD of responses of prisoners with self-harm to those without was 1.0 for both diagnostic tests; and a correlation between the two diagnostic tests for both the prisoners with and without self-harm of 0.6 [PASS 2008 (NCSS, LLC, Kaysville, UT, USA)].

Given that an ACCT is an indicator itself of potential risk of self-harm, it was thought that the prevalence in this group might be substantially higher than the general estimated level of 20%. Thus, the sample size above would have sufficient power to detect smaller differences between the AUC of any two diagnostic tests. Consequently, for the comparison of each pair of scales, a sample size of 405 was required. Given the block design above, a sample of 840 would provide 420 prisoners who could be compared on any pair of screening instruments (Table 10). With a degree of uncertainty surrounding the follow-up rate that would be achieved, a conservative estimate led to deliberate oversampling of approximately 70%, meaning that the initial aim was to assess approximately 1400 prisoners. This would allow for recruitment of 840 subjects with sufficient follow-up information available for a reliable AUC analysis.

For the Rasch analysis, sample size is primarily concerned with the degree of precision of the estimate of items for any given scale. A sample size of 400 respondents for any given screening instrument would estimate the item difficulty within a scale, with significance level of 0.01, to within ± 0.3 logits. This is the minimum practical level of stability expected for most variables. 66

Finally, for the Cox proportional hazards regression analysis it was originally estimated that 400 prisoners would provide > 99% power to detect a hazard ratio for self-harm of 2.72 between two a priori risk groups with a SD of 10.0. This corresponded to an assumption that the risk of self-harm in a prisoner identified as at risk from a diagnostic test was 2.72 times that of a prisoner identified as a non-risk case from the same diagnostic test, and was selected to represent the smallest hazard ratio which could be detected given the available sample size and assumptions. The calculation also assumed a self-harm prevalence of 20%, and was adjusted to account for correlation between the risk factor (identified within a diagnostic test) and other covariates (such as prisoner characteristics) assuming that a multiple regression of the risk factor on the other covariates in the Cox proportional hazards regression model was expected to have an R² of 0.1 (PASS 2008). It was thought that this would allow for a model using individual scale items should the necessity arise (where there are a minimum of 420 responses on any item in any scale). It was, however, recognised that the estimate of a SD of 10 (given a dichotomous risk factor) and > 99% power was implausible, and the power estimate for the Cox proportional hazards regression analysis was therefore re-estimated at the same time as the re-estimation using the results of the pilot study.

Sample size re-estimates

The pilot study brought about several changes to the protocol, including the estimated sample size required for the study.

As shown in Table 5 and Figure 5, the rate of self-harm did not increase substantially as the follow-up time increased beyond 6 months, suggesting that follow-up time could be restricted to a 6-month period in order to maximise the recruitment period in the main study.

FIGURE 5.

Time to first self-harm event during pilot study follow-up.

The original sample size was inflated by approximately 80% to allow for a final sample with sufficient follow-up time for a reliable AUC analysis. However, after further consideration, it was agreed that, as the focus of the study was self-harm during the follow-up period post ACCT or the time to release, whichever was sooner, prisoners who were released prior to the end of the follow-up period would not be considered lost to follow-up, assuming that full data would be available for them during their time in prison post ACCT. A prisoner would, therefore, be considered lost to follow-up if he or she were transferred prior to the end of the follow-up period with no available follow-up after their transfer date or if no follow-up data were available at all. Given the loss to follow-up rates observed in the pilot study (see Table 5), in which a loss to follow-up rate of 18.7% was observed at 6 months and of 22.6% at 9 months, it was agreed that a loss to follow-up rate of 20% at 6 months could be assumed for the main study.

The original sample size estimates assumed a self-harm rate of 20%; however, the overall self-harm rate observed during the pilot study was 40%, with an overall 95% confidence interval (CI) of 28.9% to 51.1%. The proportion of prisoners recruited from each prison in the main study was expected to be similar to that in the pilot; however, considerably lower rates were observed in prison C than in prisons A and B. As described above, it was also planned that the follow-up period in the main study would be reduced from 9 to 6 months. Thus, when considering the sample size re-estimates, an expected self-harm rate of ≈ 30% was considered appropriate, based on the lower limit of the 95% CI, in order to limit the deviation from the prior assumption of 20%.

Given the results of the pilot study, the sample size for the AUC analysis and secondary Cox proportional hazards regression analysis were re-estimated assuming a self-harm prevalence rate of 30% and loss to follow-up rate of 20% by 6 months. The power calculations for the Cox proportional hazards regression analysis were also re-estimated for the comparison of a priori risk groups with appropriate estimates of SD.

Given an estimated self-harm prevalence rate of 30% and loss to follow-up rate of 20%, a sample size of 359 prisoners would provide 80% power to detect a difference of 0.1 between the AUC for two diagnostic tests at the 5% significance level. Similarly, 475 prisoners would provide 90% power to detect such a difference (Table 11). As per the original sample size assumptions, it was assumed that the detection of a difference of 0.1 between the AUC for two diagnostic tests would involve one test with an AUC of 0.8 and the other with an AUC of 0.9; similar levels of variation for responses in prisoners with and without self-harm for both diagnostic tests (i.e. the ratio of the SD of responses of prisoners with self-harm to those without was 1.0 for both diagnostic tests); and the correlation between the two diagnostic tests for both the prisoners with and without self-harm being 0.6.

For the Cox proportional hazards analysis, given an estimated self-harm prevalence of 30% and loss to follow-up rate of 20%, Table 12 presents the sample size requirements under different power and hazard ratio requirements. To detect a hazard ratio for self-harm as small as 1.75 between two a priori risk groups with 80% power, 464 prisoners would be required. This corresponds to an assumption that the risk of self-harm in a prisoner identified as at risk from a diagnostic test is 1.75 times that of a prisoner identified as a non-risk case from the same diagnostic test. It was assumed that the proportion of prisoners belonging to a risk group from any diagnostic test would be 0.5, thus yielding a SD of 0.5. As per the original sample size assumption, it was assumed that the correlation between risk group and other covariates (such as prisoner characteristics) would be 0.1. Detection of a hazard ratio smaller than 1.75 would have required substantially more prisoners, and this sample size was considered sufficient given that the Cox proportional hazards analysis forms a secondary analysis. If in fact the hazard ratio for self-harm between two a priori risk groups is larger than 1.75, fewer prisoners are required to yield similar power (see Table 12).

Summary of pilot study and implications for main study

The pilot study showed that it was possible to administer a set of screening instruments in a prison setting and that the prisoners themselves were happy to spend time in an interview setting, and were able to answer questions from a broad range of instruments. Just over three in five were still within the prison system at the time of follow-up, and the loss to follow-up rate at 9 months was found to be 22.6%. The self-harm rate was found to be 40%, with the majority of events occurring within 6 months.

Given these findings, the block randomisation of instruments was abandoned, and it was decided that all prisoners would be administered all of the chosen instruments at the same time, combined into a single questionnaire pack (see Appendix 1). Using a conservative rate of 30% for self-harm, and a 6-month follow-up period with a 20% loss to follow-up rate, it was calculated that 359 and 475 cases would be sufficient to give 80% and 90% power, respectively, for the AUC analysis and Cox proportional hazards regression analysis. This sample size would also, as before, be sufficient for the Rasch analysis. The same prisons involved in the pilot study would be used for the main study.

Confirmatory factor analysis

A fundamental assumption of test theory is that a set of items should measure just one attribute or dimension; otherwise the score is not interpretable. 68,70 This unidimensionality is an assumption in which a set of items are to be summated to give a total score. CFA makes it possible to test whether or not such a hypothesised factor structure of a questionnaire (based either on empirical data or on theory) is supported by actual data. 71 This may take the form of a single set of items (questions) measuring a single domain, or confirming that a larger set of items map onto many pre-specified domains. Consequently, analysis of the dimensional structure of the candidate screening tools chosen for the current study represents the foundation of the psychometric analysis, as all further stages have the assumption of unidimensionality. CFA is undertaken with the MPlus package (Muthen & Muthen, Los Angeles, CA, USA ) and is based on a polychoric correlation matrix. The polychoric correlation coefficient is a measure of association for ordinal variables which rests upon an assumption of an underlying joint continuous distribution. Although strict CFA interpretation would require uncorrelated errors between indicators (items) of a scale, it is quite common in health-related scales (e.g. depression) to find items which are linked in some fashion such that errors should be correlated. Sometimes these items reflect nuances of the construct that are important for clinical management (e.g. dressing upper body and dressing lower body) and, thus, discarding such items because they breach the assumption of local independence would be inappropriate. Thus, the correlation of errors will be allowed within the CFA.

Several fit statistics will be used to determine if the CFA is satisfactory. The primary measure is the chi-squared statistic, where a non-significant value indicates that the data conform to expectations. 72 Supplementary fit statistics include the root-mean-square error of approximation (RMSEA), where a value of < 0.08 would be considered sufficient. A Tucker–Lewis index (TLI) and comparative fit index (CFI) value of > 0.95 would also support the proposed data structure.

Given these fit parameters, scales can be graded indicating the degree of support for unidimensionality (Table 13).

Mokken scaling

Mokken scale analysis is used for scaling items and measuring respondents on an ordinal scale. 73,74 It is a non-parametric probabilistic version of Guttman scaling,75 and it is used similarly to other techniques for data reduction that allow for the unidimensional measurement of latent variables. The stochastic cumulative scaling model offered by this approach is ideally suited when the intention is to score an underlying latent trait by simple addition of the item response values. 76 It has been shown to have a number of advantages over some other measurement models; for example, it includes an item parameter that shows how items differ in their distribution, it is probabilistic rather than deterministic and it can be applied in situations in which latent variables must be operationalised with only a small number of indicators. 77

The process has a number of assumptions which are to be found in most non-parametric and parametric (e.g. Rasch model) IRT models. These are unidimensionality, local dependence and monotonicity [the probability of affirming an item increases as the underlying level of the construct (theta) increases]. As with Guttman scaling, model violation is crucial to interpretation, and this revolves around a triple of objects consisting of one subject and two items. The number of model violations in a data set is defined as the number of transitivity relations (e.g. if a > b and b > c, then it always follows that a > c) among all such triples that are violated. 77 Homogeneity, whether of items or subjects, is defined by relating the number of model violations observed to the number of violations that can be expected under the model of stochastic independence. This provides the item coefficient of stability, operationalised as the Loevinger’s H. In practice, this reflects the amount of discrimination of an item where, for example, very low values of H would indicate poor discrimination (a flat item response function). Consequently, many computer programs adopt a minimum requirement of H > 0.3 for item selection. Levels of scaling based on H have been reported as:

H_ij < 0.3 indicates poor/no scalability

0.3 ≤ H_ij < 0.4 indicates useful but weak scalability

0.4 ≤ H_ij < 0.5 indicates medium scalability

H_ij ≥ 0.5 indicates good scalability.

The use of Mokken scaling in the current study is designed to provide information to support the summation of a set of items to provide an ordinal scale. Given the double monotone homogeneity of the procedure, which orders both persons and items, it can also be considered a prelude to Rasch analysis. Thus, failure to satisfy Mokken scaling criteria would indicate that a scale would be unlikely to satisfy Rasch model assumptions. Furthermore, given adequate scaling, cut points, which are simply a magnitude on an ordinal scale, would be valid and more than adequate to identify ‘caseness’ (e.g. for depression). Thus, Mokken scaling confirms the validity of cut-point analysis using AUC. As it has the assumption of unidimensionality, this analysis follows the CFA of the candidate scales.

However, some concerns have been expressed about the merits of the Mokken scale. 78 The first concerns monotone homogeneity and sample independence, and the other concerns the meaning and usefulness of the H coefficient. It has been argued that H is not a measure of monotone homogeneity, and that it is not sample independent. In practice, these two aspects are satisfied by only the Rasch model.

Rasch analysis

While Mokken scaling offers a test to see if a set of items forms an ordinal scale, fit of the data to the Rasch measurement model tests to see if the data satisfy the requirements of a quantitative structure, so providing interval scale measurement. 69,79 Briefly, the objective is to determine if data from the scale satisfy a parametric probabilistic version of Guttman scaling. 75 The process involves a number of activities, which include testing to see if the data meet Rasch model expectations; information on the quality of individual items, including individual item fit; testing the assumption of unidimensionality; checking to see if the scale works in the same way across groups (invariance as determined by DIF); and examining the reliability and targeting of the scale to the sample.

The distinct advantage of scales which satisfy Rasch model assumptions is that the items will make a unidimensional scale where, as with the Mokken scale, the raw score is a sufficient statistic (that is that the raw score gives an estimate of the person’s ability at the ordinal level, and does do not require any additional information). 80 Furthermore, the raw score can be transformed to interval scaling such that change scores and other appropriate mathematical calculations can be performed. Given appropriate distributional properties, the transformed score can then be used in parametric statistical procedures. If the distribution of this is non-normal, further transformations could be applied. As items (as well as persons) are calibrated on a metric, the approach lends itself to establishing unidimensional ‘item banks’, where items (questions) from different instruments can be calibrated together on the same metric. Thus, the operational ranges of instruments can be compared and the items can be made available to Computer Adaptive Testing which can minimise respondent burden. 81,82 For the current study, an item bank may offer an alternative source of items for predictive purposes, as opposed to the standardised scales themselves.

In the current study, data are fitted to the Rasch model through the RUMM2030 software (RUMM Laboratory, Perth, WA, Australia). An iterative process tests if polytomous items are properly ordered; if the assumption of local response independence holds;83 if the assumption of unidimensionality holds; if the scales are invariant across key groups such as gender or sentence status; and if the items follow the stochastic ordering as required by the model. For testing the stochastic ordering requirements, a range of fit statistics are available, including chi-squared fit where a non-significant (Bonferroni-adjusted) deviation from model expectation would be required, and where individual item-person residuals would be within standardised range of ± 2.5 (99% CI). 84 In addition, a person separation reliability is reported, consistent with Cronbach’s alpha when persons have a normal distribution, but less so when data are skewed or where there are floor and ceiling effects.

For those scales where there are more than two response options for an item (i.e. polytomous items), it is possible to evaluate whether or not the categories are working as expected [i.e. a monotonic increase in category transition (threshold) across the trait being measured]. Where response options were found to be not working as intended across the whole item set, a generic rescore was considered. This is a post hoc adjustment of the original response categories which treats two (or more) adjacent response categories as equivalent. It is necessary to do this as the disordering of the original response categories implies that the respondents (i.e. the prisoners in this case) do not distinguish between the presented response categories, meaning that the intended discrete, ordered response category structure is not working in the way that it was originally designed. When rescoring, it is logical for this to be guided by the content and wording of each response category. It is often possible to see where the confusion may arise (where response options are similar or overlap) and linking these response options back to the observed threshold patterns helps to inform rescore options.

For DIF, prison, gender, age group (≤ 30 vs. ≥ 30 years), remand status (on remand vs. sentenced), age left full-time education (< 16 vs. 16 + years) and religion (whether or not prisoner stated that he or she practised a religion) were tested for invariance. Where the unidimensionality assumption is questioned by post hoc tests, a bi-factor solution is also available within the approach, where all items are considered to load on one dominant factor, as well as unique factors. 85,86 The amount of unique variance which is removed from the latent estimate to achieve this solution is reported. A post hoc test of unidimensionality is also available, following the recommendations by Smith. 87 Independent sets of items are used to generate two estimates for every individual, which are then compared by a t-test. The lower bound of the binomial CI for proportions should be less than 5% when comparing these estimates, given that the items belong to a unidimensional construct. Further details of the process of Rasch analysis are given elsewhere. 88–90

In the current study, the initial fit statistics for each scale are summarised within corresponding tables. The Rasch analysis was also progressed in alternative ways.

Follow-up time

The time included in the follow-up period was variable, with the aim being to complete a 6-month follow-up period. In some cases this was not possible as the prisoner had been released, but in some cases the records allowed for a longer follow-up time. Where a longer follow-up was possible, the information for the full follow-up period has been included. However, for the predictive element of the study, the follow-up period was restricted to 198 days (6.5 months). Only one person reported their first self-harm event after this cut-off point. During follow-up, 126 people actually carried out a self-harm event, but only 125 of these were within the valid time frame.

Incidence of self-harm

During the follow-up period, a total of 423 self-harm events were reported, based on 126 individuals followed up for 66,789 prisoner-days. This gives an ‘event incidence’ of 6.33 per 1000 prisoner-days among those who had been placed on an ACCT, or ‘prisoner incidence’ of 1.84 per 1000 days. For example, if 20% of the current prison establishment had previously been on an ACCT, then, in a prison housing 1000 inmates, one self-harm act per day could be expected. However, this is only the average from the current study, and it is notable that this varies considerably by gender (Table 17) and, to a lesser extent, between prisons. Thus, the event incidence in the female prison is much higher, at 15.83 per 1000 prisoner-days, as opposed to the male event average of 4.02 per 1000. Looking at persons, rather than events, there is a clear gradient across prisons, with a low person incidence in the male prison A, rising through 1.79 in the male prison C to the much higher incidence in the female prison B.

It becomes obvious that the ratio of persons to events is different across prisons, with the male prisons having a ratio between 2 and 3, whereas the female ratio is above 5. The frequency of events is shown in more detail in Figure 7.

FIGURE 7.

The number of self-harm events carried out by each individual during follow-up, presented as a percentage of the full main study sample (n = 450).

The median time to a first self-harm event during follow-up was 37 days, with a range of 0–190 days (Figure 8). The conditional probability of an ACCT Index self-harm event, given previously reported self-harm, was 0.33; of subsequent self-harm (i.e. during follow-up), given reported previous self-harm, was 0.28; and of subsequent self-harm, given a known self-harm ACCT Index event, was 0.47. See Table 60 for additional detail about the nature of these self-harm events.

FIGURE 8.

Time (in days) to first self-harm event (of those who self-harmed).

Associations with self-harm

Various characteristics may be considered a potential risk or mediating factor for self-harm. Just over two in five (42.2%) reported that they practised a religion, the rate being much higher in one of the male prisons than elsewhere (Table 18). Over one-third of subjects reported being homeless in the 12 months prior to prison, and almost three in five (57.9%) reported seeing a psychiatrist outside prison. Almost three-quarters (74.4%) reported receiving medication for mental health problems. Almost one-third of the subjects (32.4%) considered themselves to be dependent on alcohol and one-third (33%) considered themselves to be dependent on drugs. Almost four in five (78%) reported that they had self-harmed outside prison and over three in five (61.7%) that they had done so within prison. Females were much more likely to carry out self-harm in prison, but not so outside prison, where one of the male prisons reported a lower rate of self-harm but the other male prison reported a rate equivalent to that reported by females. Just over four in five (82.1%) were recruited from their first ACCT during their current stay in prison, but females were much less likely than males to be on their first ACCT.

Given that the frequency of reported previous self-harm was so high, it is instructive to examine the behaviours engaged in. Taken from the SHI, given a total history of self-harm (have you ever), behaviours range from ‘tortured self with self-defeating thoughts’, reported by four in five (79.7%) of those who have self-harmed, through to ‘abused laxatives to hurt self’, reported by just 5.2%, mostly female (Table 19). Over three-quarters (77.9%) reported that they had attempted suicide at some time in the past, which showed a significant difference across prisons. One in five reported a suicide attempt within the last week (BSL-23-F supplementary items), but this did not show any difference across prisons. More than half of the behaviours showed a significant difference in reported frequency across prisons, many of which (e.g. engaged in sexually abusive relationships), but not all of which, related to gender differences. On average, subjects who had self-harmed reported nine behaviours, but there were significant differences in the numbers of behaviours reported and the patterns of those behaviours.

A two-step cluster analysis with binary variables revealed four clusters of behaviours, their number being significantly different across clusters (Table 20). Cluster 1 is characterised by an average of 12.5 reported behaviours out of a possible 22 (from the SHI). All behaviours are extremely common, but it is within this cluster that rejected or sexually abusive relationships are to be found. Given that numbers are similar in clusters 1 and 2, the difference is more marked by the absence of certain behaviours in cluster 2. For example, ‘burning self’ and ‘hitting self’ are much less common in cluster 2. ‘Scratching self on purpose’ is almost absent, whereas it is very common in cluster 1. Cluster 3 is characterised by a low average number of behaviours. In practice, there is a significant difference [χ² = 22.1; degrees of freedom (df) 3; p < 0.001] by gender across cluster membership. Over half of females (50.4%) are to be found in cluster 1, whereas over two in five males (40.1%) are to be found in cluster 2, compared with just over one in five females (20.9%). Proportions of males and females are similar in cluster 3, suggesting that about one in five of each gender who report previous self-harm had engaged in relatively few behaviours. A fourth cluster, identified as an ‘outlying cluster’, consisted of just 32 prisoners and had equal representation across prisons. It had some similarities to cluster 1, with emphasis upon relational matters, but, because of the numerical difference between the two male prisons, prison A had a similar proportion of prisoners in this cluster as did the female prison (prison B), whereas it was less common in prison C.

The Borderline Symptom List-23 (frequency-based responses)

Initial analysis of the BSL-23-F revealed that the items in the scale failed to meet Rasch model expectations (see Table 26). Individual item fit revealed evidence of a number of problematic items displaying fit parameters outside the normally expected and accepted range. Additionally, all items displayed disordered thresholds, meaning that the response categories were not functioning as intended. At this initial stage, only two items displayed DIF at the Bonferroni-adjusted level. Item 13 (‘I suffered from shame’) displayed DIF by age group and item 16 (‘criticism had a devastating effect on me’) displayed DIF by both prison and gender, although the prison DIF is likely to be just an interactive manifestation of the gender DIF that is present.

Rescore

As the response options were not working as intended across the whole item set, and the observed response patterns were similar for most items, a generic rescore was implemented.

The generic rescore of all of the BSL-23-F items was as shown in Table 25.

TABLE 25 - Rescoring of BSL-23-F response categories

Original response code	Response wording	Rescored response code
0	Not at all	0
1	Only occasionally	1
2	Sometimes	1
3	Often	2
4	Most or all the time	2

This rescore also has the follow-on effect of reducing the total scale score. Originally, the scale would be scored 0 to 92, but with the rescore in place the total scale score is contracted to 0–46.

Following the generic recode, all items displayed ordered categories except item 15 (‘I suffered from voices and noises from inside or outside my head’).

The summary fit statistics at this stage are presented in Table 26, along with the plot of relative item threshold difficulties and person abilities (the targeting plot, Figure 9).

TABLE 26 - Summary Rasch fit statistics for the BSL-23-F

PSI, person separation index.

Insufficient power in t-test procedure (i.e. < 10 thresholds used to generate comparative estimates).

	Item location		Person location		Item fit residual		Person fit residual		Chi-squared interaction			PSI		Alpha	Unidimensionality t-tests
Analysis	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Value	df	p	With extremes	Without extremes		Number of significant tests	Total number of tests	%	Lower bound 95% CI
Initial	0	0.39	0.15	0.9	0.77	3	–0.12	1.7	479	138	0	0.918	0.921	0.932	37	436	8.49	6.40%
Rescored	0	0.69	0.65	1.29	0.05	2.7	–0.13	1.3	416.7	138	0	0.895	0.893	0.918	39	431	9.05	7.00%
Resolution A	0	0.77	0.56	1.19	0.25	1.3	–0.18	1.1	116.9	84	0.01	0.852	0.843	0.853	22	424	5.19	3.10%
Resolution B	0	0.73	0.9	1.38	0.33	1.4	–0.18	1.1	99.07	78	0.05	0.823	0.807	0.875	18	414	4.35	–
Resolution B2	0	0.75	0.83	1.38	0.33	1.3	–0.18	1.1	96.08	84	0.17	0.838	0.823	0.882	19	417	4.56	–
Supplementary items rescored	0	1.36	–1.62	1.04	0.09	2.0	–0.10	0.4	67.11	24	0	0.018	–0.27	0.486	0	336	0a	–

FIGURE 9.

Targeting plot for the BSL-23-F following rescoring (n = 443, mean = 0.652, SD = 1.294).

Sources of individual item misfit at this stage are summarised in Table 27.

TABLE 27 - Summary of individual sources of misfit within the BSL-23-F item set, following a generic recode

✗ indicates misfit sources.

Item	Disordered thresholds	Fit residual> 2.5	Fit residual< –2.5	Misfitting chi-squared statistic	Misfitting F-statistic	Prison DIF	Gender DIF	Age DIF	Religion DIF	Response dependence (residual correlation > 0.2)
1
2
3		✗		✗	✗
4										✗
5
6		✗		✗	✗
7										✗
8
9
10
11			✗	✗	✗					✗
12			✗	✗	✗					✗
13										✗
14
15	✗	✗		✗	✗
16						✗	✗
17
18
19
20
21			✗	✗	✗					✗
22										✗
23			✗	✗	✗			✗		✗

Even when item 15 is rescored in an alternative rescore pattern to resolve the disordered thresholds, the reported misfit is still present.

Borderline Symptom List-23 (frequency-based responses) supplementary items

The eight items of the supplement were also looked at as a separate scale.

All thresholds were disordered with category probability response patterns tending towards a dichotomous structure. All items were, therefore, dichotomised, which resulted in an extremely low person separation index (0.02), along with other unfavourable fit statistics. See Table 26 for the BSL-23-F supplementary items summary fit statistics at this stage. This analysis was not progressed because of the lack of power in the tests of fit, as indicated by the low person separation index.

The Clinical Outcomes in Routine Evaluation – Outcome Measure

The CORE-OM can be assessed in various different ways. The 34-item scale can be assessed in its entirety, or broken down into its separate domains of well-being (four items), problems/symptoms (12 items), functioning (12 items) and risk (six items). The CORE-OM is also commonly summed with the risk domain excluded (CORE minus risk). Additionally, the short-form 10-item screening tool, the CORE-10, is embedded within the larger 34-item scale.

It is postulated that the four domains all contribute to a higher-order construct, but, prior to this being formed, it holds that each individual domain should function independently. First, the results of the complete CORE-OM will be presented, followed by, second, the independent domains and the CORE-10.

The Clinical Outcomes in Routine Evaluation – Outcome Measure complete scale

Initial analysis of the CORE-OM revealed the scale to be problematic in terms of fit to the Rasch model. The summary fit statistics at this stage are presented in Table 28. Individual item fit revealed evidence of a number of problematic items displaying fit parameters outside the normally expected and accepted range.

TABLE 28 - Summary Rasch fit statistics for the CORE-OM, domains and CORE-10

Insufficient power in t-test procedure (i.e. < 10 thresholds used to generate comparative estimates).

Analysis	Item location		Person location		Item fit residual		Person fit residual		Chi-squared interaction			PSI		Alpha	Unidimensionality t-tests
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Value	df	p-value	With extremes	Without extremes		Number of significant tests	Total number of tests	%	Lower bound 95% CI (%)
CORE-OM initial	0	0.48	0.09	0.48	0.83	3.47	–0.02	1.43	1074	238	0	0.9	0.9	0.9	77	448	17.19	15.20
CORE-OM rescored	0	0.84	0.34	0.83	0.38	3.26	–0.07	1.23	849.7	238	0	0.89	0.89	0.9	59	448	13.17	11.20
Bifactor resolution	0	0.69	0.15	0.71	0.41	0.91	–0.32	0.97	27.22	28	0.507	0.856	0.856	–	20	445	4.49	–
CORE-OM resolution B	0	0.83	0.32	0.94	0.28	1.11	–0.15	0.98	104.04	102	0.425	0.815	0.81	0.83	25	448	5.58	3.60
Well-being initial	0	0.18	0.45	0.71	0.55	1.7	–0.31	1.2	80.3	28	0	0.43	0.3	0.58	0	448	0.00a	–
Well-being rescored	0	0.26	0.68	1.02	0.99	2.03	–0.29	1.36	63.2	16	0	0.26	0.03	0.54	0	383	0.00a	–
Problems/symptoms initial	0	0.33	0.37	0.76	0.82	2.64	–0.17	1.36	221.8	84	0	0.82	0.81	0.85	26	441	5.90	3.90
Problems/symptoms rescored	0	0.59	1.01	1.17	0.02	2.19	–0.12	0.95	154.1	72	0	0.75	0.73	0.83	12	425	2.82	–
Problems/symptoms resolution A	0	0.47	0.96	1.21	0.03	1.48	–0.23	1.11	44.7	32	0.067	0.686	0.64	0.8	7	410	1.71a	–
Problems/symptoms resolution B	0	0.62	1.06	1.21	0.02	1.15	–0.21	1.1	43.03	24	0.0099	0.652	0.594	0.78	9	407	2.21a	–
Functioning initial	0	0.38	0.07	0.5	1.05	2.59	–0.08	1.34	201	72	0	0.75	0.74	0.74	37	448	8.26	6.20
Functioning rescored	0	0.43	0.12	0.86	0.86	2.71	–0.07	1.34	223.95	72	0	72	69	0.78	36	444	8.11	6.10
Functioning resolution A	0	0.47	0.23	0.99	0.52	1.17	–0.23	1.52	61.94	50	0.1198	0.72	0.671	0.74	19	437	4.35	–
Functioning resolution B	0	0.44	0.05	1.03	0.72	1.26	–0.19	1.31	62.96	45	0.0396	0.708	0.638	0.74	12	431	2.78a	–
Risk 1 initial	0	0.68	–0.61	0.86	–0.38	1.08	–0.32	0.65	129.2	36	0	0.65	0.6	0.73	23	404	5.69a	3.60
Risk 2 rescored	0	1.34	–0.7	1.38	–0.4	1.11	–0.34	0.69	94.8	36	0	0.65	0.53	0.72	27	401	6.73a	4.60
CORE-10 initial	0	0.4	0.3	0.65	0.71	2.61	–0.16	1.16	195.97	50	0	0.764	0.751	0.79	22	446	4.93	–
CORE-10 rescored	0	0.74	0.73	1.04	0.12	2.6	–0.14	0.92	170.75	60	0	0.71	0.693	0.77	11	442	2.49	–
CORE-10 resolution	0	0.83	0.78	1.1	0.11	1.24	–0.18	0.85	69.2	48	0.024	0.659	0.612	0.73	5	434	1.15a	–

Additionally, the observed response patterns for the items were very similar to those observed for the BSL-23-F, as all items displayed disordered thresholds, meaning that the response categories were not functioning as intended.

Rescore

As the response options were not working as intended across the whole item set and the observed response patterns were similar for most items, a generic rescore was implemented, although this rescore was different for regular scored items and reverse scored items.

The generic rescore of all of the CORE items was as in Table 29.

TABLE 29 - Rescoring of CORE-OM response categories

Original response code	Original reversed response code	Response wording	Rescored response code	Rescored reversed response code
0	4	Not at all	0	2
1	3	Only occasionally	1	1
2	2	Sometimes	1	1
3	1	Often	2	0
4	0	Most or all the time	2	0

This rescore also has the follow-on effect of reducing the total scale score. Originally, the scale would be scored 0 to 136, but with the rescore in place the total scale score is contracted to 0 to 68.

Following the generic recode, 28 items displayed ordered thresholds, but six items still displayed disordered thresholds. Despite the remaining disorder, this response structure was maintained across the item set.

The summary fit statistics at this stage are presented in Table 28, along with the targeting plot (Figure 10).

Sources of individual item misfit at this stage are summarised in Table 30.

FIGURE 10.

Targeting plot for the CORE-OM following rescoring (n = 449, mean = 0.340, SD = 0.829).

TABLE 30 - Summary of individual sources of misfit within the CORE-OM item set, following a generic recode

✗ indicates misfit sources.

Item	Disordered thresholds	Fit residual> 2.5	Fit residual< –2.5	Misfitting chi-squared statistic	Misfitting F-statistic	Prison DIF	Gender DIF	Age DIF	Religion DIF	Response dependence (residual correlation > 0.2)
1							✗
2					✗					✗
3		✗		✗	✗					✗
4
5										✗
6	✗									✗
7
8	✗	✗		✗	✗					✗
9										✗
10
11						✗
12										✗
13										✗
14										✗
15										✗
16										✗
17			✗	✗	✗
18	✗
19	✗	✗		✗	✗	✗	✗			✗
20
21
22	✗									✗
23			✗	✗	✗					✗
24			✗	✗	✗					✗
25										✗
26
27			✗	✗	✗					✗
28										✗
29
30		✗
31		✗		✗	✗					✗
32										✗
33										✗
34	✗							✗		✗

Although the items with disordered thresholds can be recoded in an alternative rescore pattern to resolve the disordered thresholds, the reported misfit is still present.

The Prison Screening Questionnaire

Initial analysis of the PriSnQuest showed the scale to be problematic in terms of fit to the Rasch model. The summary fit statistics at this stage are presented in Table 34, along with the initial targeting plot (Figure 11). Individual item fit revealed evidence of some items displaying fit parameters outside the normally expected and accepted range, but the individual item misfit did not suggest the same level of misfit as was found in the overall scale fit statistics.

FIGURE 11.

Targeting plot for the PriSnQuest (n = 447, mean = 0.512, SD = 1.213).

As the PriSnQuest items are all dichotomously scored, there is no opportunity for item thresholds to be disordered, as each item has only a single-measurement threshold. Therefore, no rescoring is necessary, or possible, among the PriSnQuest items. The sources of individual item misfit at this stage are summarised in Table 35.

At this initial stage, the main anomaly seemed to be the sizeable response dependency that was apparent between item 4 (‘have you recently felt that life isn’t worth living?’) and item 5 (‘have you recently found yourself wishing you were dead and away from it all?’) (residual correlation = 0.505). This apparent dependency was accounted for through subtesting the affected items, and the summary statistics following this amendment are presented in Table 34.

Despite accounting for this item dependency, the PriSnQuest appeared similar to the CORE well-being and risk domains, in that the PriSnQuest has a limited number of items within the scale, and, even after accounting for various forms of apparent misfit, neither resolution A nor resolution B was applicable to this set of items. Again, this means this scale did not conform to the strict requirements of Rasch scaling, but this does not preclude it from conforming to ordinal scale requirements, or having use as a screening tool or a series of single-indicator items. However, this item set fails to conform to the expectations of Rasch analysis, with the main individual item problems highlighted in Table 35.

The PriSnQuest appeared to function differently in male and female populations, suggested by the gender DIF that is apparent in the initial analysis. Consequently, it may be useful to treat the PriSnQuest as a different scale among male and female ACCT populations. The summary statistics of the initial PriSnQuest for the separate male and female samples are presented in Table 34. Although the fit of the scale to the model is weak for males, it does appear that a separate gender-based solution is more appropriate.

The Patient Health Questionnaire-9

Initial analysis of the PHQ-9 showed that the scale failed to satisfy Rasch model expectations. The summary fit statistics at this stage are presented in Table 36. Individual item fit revealed evidence of relatively few problematic items displaying fit parameters outside the normally expected and accepted range.

However, all items except one (item 4: ‘feeling tired or having little energy’) displayed disordered thresholds, meaning that the response categories were not functioning as intended.

Rescore

As the response options were not working as intended across almost the whole item set, and the observed response patterns were similar for most items, a generic rescore was implemented.

The generic rescore of all of the PHQ-9 items is shown in Table 37.

TABLE 37 - Rescoring of PHQ-9 response categories

Original response code	Response wording	Rescored response code
0	Not at all	0
1	Several days	1
2	More than half the days	1
3	Nearly every day	2

This rescore also has the follow-on effect of reducing the total scale score. Originally, the scale would be scored 0 to 27, but with the rescore in place the total scale score is contracted to 0 to 18.

Following the generic recode, all items displayed ordered categories.

The summary fit statistics at this stage are presented in Table 36, along with the targeting plot (Figure 12).

Sources of individual item misfit at this stage are summarised in Table 38.

FIGURE 12.

Targeting plot for the PHQ-9 following rescoring (n = 442, mean = 0.878, SD = 1.317).

TABLE 38 - Summary of individual sources of misfit within the PHQ-9 item set, following a generic recode

✗ indicates misfit sources.

Item	Disordered thresholds	Fit residual> 2.5	Fit residual< –2.5	Misfitting chi-squared statistic	Misfitting F-statistic	Prison DIF	Gender DIF	Age DIF	Religion DIF	Response dependence (residual correlation > 0.2)
1
2				✗	✗
3
4
5
6
7
8
9

The Self-Harm Inventory

Initial analysis of the SHI revealed a few individual elements of misfit, but the overall scale did not appear to be too problematic in terms of fit to the Rasch model. Individual item analysis, however, revealed evidence of some items displaying fit parameters outside the normally expected and accepted range. The majority of this misfit was attributable to DIF parameters mainly in the form of gender DIF, although prison DIF (unrelated to the gender DIF) was also present.

As the SHI items are all dichotomously scored, there is no opportunity for item thresholds to be disordered, as each item has only a single-measurement threshold. Therefore, no rescoring is necessary, or possible, among the SHI items.

The initial summary statistics for the SHI can be found in Table 39, along with the targeting plot (Figure 13). The sources of individual item misfit at this stage are summarised in Table 40.

FIGURE 13.

Targeting plot for the SHI (n = 441, mean = −0.610, SD = 1.157).

At this initial stage, there was response dependency that was apparent between item 1 (‘overdosed?’) and item 18 (‘attempted suicide?’) (residual correlation = 0.347), along with a lower level dependency apparent between item 11 (‘been promiscuous?’) and item 12 (‘set yourself up in a relationship to be rejected?’).

However, the majority of the misfit was attributable to DIF parameters, mainly in the form of gender DIF, although prison DIF (unrelated to the gender DIF) was also present. Religion DIF was present for item 14 (‘distanced yourself from God as punishment?’). This was the only religion DIF present across any of the scales.

Gender separation

The large amount of gender DIF that is apparent in the initial SHI analysis suggests that the SHI is functioning differently for males and females. It may therefore be beneficial to treat the SHI as a different scale among male and female ACCT populations. The summary statistics of the initial SHI analysis for the separate male and female samples are presented in Table 39. An example of an item displaying a clear gender DIF is presented in Figure 14.

The sources of individual item misfit at this stage are also summarised in Tables 41 and 42.

FIGURE 14.

An example of an item (‘emotionally abusive relationships’) displaying gender DIF, with females obtaining a higher affirmation rate at all levels of the underlying trait. ExpV, expected value.

TABLE 41 - Summary of individual sources of misfit within the SHI: male sample onlya

✗ indicates misfit sources.

Disordered thresholds are not possible as the response structure is dichotomous.

Item	Disordered thresholdsa	Fit residual> 2.5	Fit residual< –2.5	Misfitting chi-squared statistic	Misfitting F-statistic	Prison DIF	Gender DIF	Age DIF	Religion DIF	Response dependence (residual correlation > 0.2)
1	–						–			✗
2	–						–
3	–						–
4	–						–			✗
5	–						–			✗
6	–						–
7	–						–
8	–						–
9	–					✗	–
10	–					✗	–
11	–						–			✗
12	–						–			✗
13	–						–
14	–						–		✗
15	–						–
16	–						–
17	–						–
18	–						–			✗
19	–						–
20	–					✗	–
21	–						–
22	–						–

TABLE 42 - Summary of individual sources of misfit within the SHI: female sample only

✗ indicates misfit sources.

Disordered thresholds are not possible as the response structure is dichotomous.

Item	Disordered thresholdsa	Fit residual> 2.5	Fit residual< –2.5	Misfitting chi-squared statistic	Misfitting F-statistic	Prison DIF	Gender DIF	Age DIF	Religion DIF	Response dependence (residual correlation > 0.2)
1	–					–	–			✗
2	–					–	–
3	–					–	–
4	–					–	–
5	–					–	–
6	–					–	–
7	–					–	–			✗
8	–					–	–
9	–					–	–
10	–					–	–
11	–					–	–
12	–					–	–
13	–					–	–
14	–					–	–			✗
15	–					–	–
16	–					–	–
17	–					–	–
18	–					–	–			✗
19	–					–	–
20	–					–	–
21	–					–	–
22	–					–	–

Gender-specific area under the curve

Based on the indications in the literature,42,93 along with the indications provided within the Rasch analysis, the AUC analysis was repeated on a gender-specific basis to assess whether or not results differed from the collated analysis.

The male-specific AUC results for all scales are summarised in Tables 45 and 46.

The only scale score which offered a significant predictive value among the male sample was the PriSnQuest. The corresponding ROC curve is presented in Figure 20.

FIGURE 20.

Receiver operating characteristic curve of the PriSnQuest for males. Diagonal segments were produced by ties.

The only scale scores which offered a significant predictive value for males were the two alternative conversions of the PriSnQuest (initial and male-specific resolutions). The corresponding ROC curves are presented in Figures 21 and 22.

FIGURE 21.

Receiver operating characteristic curve of the PriSnQuest initial for males, converted from Rasch estimates. Diagonal segments were produced by ties.

FIGURE 22.

Receiver operating characteristic curve of the PriSnQuest (male-specific conversion) for males, converted from Rasch estimates. Diagonal segments were produced by ties.

The female-specific AUC results for all scales are summarised in Tables 47 and 48.

The only scale score that offered a significant predictive value among the female sample was the SHI. The corresponding ROC curve is presented in Figure 23.

FIGURE 23.

Receiver operating characteristic curve of the SHI for females. Diagonal segments were produced by ties.

Consequently, the only scale score that offered a significant predictive value for females was the conversion of the SHI resolution B. The corresponding ROC curve is presented in Figure 24.

FIGURE 24.

Receiver operating characteristic curve of the SHI resolution B for females, converted from Rasch estimates.

In summary, while two scales demonstrated an AUC significantly different from 0.5, all scales failed to have any meaningful predictive value, with only the SHI showing a ‘poor’ level of discrimination for females. Given this, it is not surprising that there was no significant difference between the AUCs of the various instruments. For example, with a pairwise comparison of ROC curves, the level of significance of the difference between the SHI and PriSnQuest was 0.9761, and between the SHI and PHQ-9 was 0.6253. Thus, from a predictive perspective, all scales were as poor as one another.

Cox proportional hazards regression modelling

Cox proportional hazards regression modelling was used to investigate the hazard rates for different a priori determined risk groups while adjusting for important baseline factors (see Chapter 2, Cox proportional hazards regression modelling).

The populations included in this analysis are described in Chapter 3, Populations.

Chapter 3, Cox proportional hazards regression modelling: baseline model presents the results of the univariate analysis used to determine which baseline factors to include in the Cox proportional hazard regression model; Cox proportional hazards regression modelling: Rasch-scored questionnaires presents the multivariate Cox proportional hazards regression modelling used to test for differences in the time to first self-harm event for risk groups and continuous scores based on prisoners’ converted Rasch scores, adjusting for important baseline factors identified. Rasch-based scores are available for all prisoners with a response to at least one item within each questionnaire or subscale.

Populations

Various analysis populations are considered, and the number of prisoners belonging to each population, and reasons for exclusion from populations, are summarised in Table 49.