Impact and cost-effectiveness of care farms on health and well-being of offenders on probation: a pilot study

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 11/3050/08. The contractual start date was in July 2013. The final report began editorial review in August 2016 and was accepted for publication in March 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Disclaimer

This report contains transcripts of interviews conducted in the course of the research, or similar, and may contain language which offends some readers.

Copyright statement

© Queen’s Printer and Controller of HMSO 2018. This work was produced by Elsey et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2018 Queen’s Printer and Controller of HMSO

Defining care farming

Care farming (also known as social farming) has been defined as the use of commercial farms and agricultural landscapes as a base for promoting mental and physical health through normal farming activity. 1–3 A care farm (CF) utilises the whole or part of a farm to provide health, social or educational care services for different groups of people, providing a supervised, structured programme of farming-related activities rather than occasional one-off visits (www.carefarming.org.uk; accessed January 2014). 4

Care farming is a truly complex intervention. 5 Farms differ in the type of farming activities (e.g. horticulture and livestock farming), type of other activities (e.g. gardening, conservation and woodwork), the level of support provided (e.g. health promotion, counselling and skills qualifications) and the range of service user groups. A diverse range of activities can be offered to service users at a CF. Activities selected are primarily determined by the particular needs and capabilities of the service user and the type of farm. Likewise, depending on the service user group, they may or may not be involved in agricultural production. The main defining feature of a CF is the involvement in farm activities for a therapeutic purpose. It is not a horticultural- or animal-based therapy per se.

Individuals supported through care farming include those with learning difficulties, those with autism spectrum disorder, those with mental ill health, disaffected youth, people with physical disabilities, older people, people with drugs and alcohol problems, adult probation service users, people with dementia and ex-service personnel. 6 In the UK, the largest service user groups are those with learning difficulties, those with autism spectrum disorder, those with mental ill health and disaffected youth (Table 1).

Green care

Care farming sits within a broader framework that describes our interactions with nature. These interactions can be viewed as a continuum that ranges from general everyday contact, such as viewing, working or undertaking recreational activities, through to using nature deliberately as a therapeutic or treatment resource (i.e. green care) involving activities, such as wilderness therapy, social and therapeutic horticulture, animal-assisted therapy and care farming (Figure 1).

FIGURE 1.

Continuum of interaction with nature according to human needs. Reproduced from Bragg R, Atkins G. A Review of Nature-Based Interventions for Mental Health Care. Natural England Commissioned Reports, Number 204. York: Natural England; 2016. 7 © Natural England and other parties 2016. Contains public sector information licensed under the Open Government Licence v3.0 (www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). This report is published by Natural England under the Open Government Licence – OGLv3.0 for public sector information. You are encouraged to use, and reuse, information subject to certain conditions. For details of the licence visit http://publications.naturalengland.org.uk/copyright.

Reproduced from Bragg R, Atkins G. A Review of Nature-Based Interventions for Mental Health Care. Natural England Commissioned Reports, Number 204. York: Natural England; 2016. 7 © Natural England and other parties 2016. Contains public sector information licensed under the Open Government Licence v3.0 (www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). This report is published by Natural England under the Open Government Licence – OGLv3.0 for public sector information. You are encouraged to use, and reuse, information subject to certain conditions. For details of the licence visit http://publications.naturalengland.org.uk/copyright

Green care has been defined as ‘nature-based therapy or treatment interventions which are specifically designed, structured and facilitated for individuals with a defined need’. 8 Green care utilises plants, animals and landscapes to create interventions to improve health and well-being (i.e. it does not represent a casual encounter with nature). Green care also provides care and support to enable people to reach their true potential; that is, although many of the approaches are termed ‘therapies’ or ‘therapeutic’, they are not necessarily directed at treating or curing conditions and diseases. In the case of people with learning difficulties, for example, green care provides care, support, training and other opportunities to enable those individuals to develop. Such opportunities are often not available elsewhere. 8

How care farming works

As a highly complex intervention comprising multiple activities and involving many service user groups with differing needs, it is likely that multiple mechanisms and interactions will be at work to bring about changes in individuals. At the core of the intervention is the connection with nature, which has value in its own right but also provides the platform for the range of activities. Studies have also shown that connectedness to nature has a positive effect on people’s mental, physical and psychological well-being and spiritual beliefs9,10 and that engaging in nature-based activities, such as farming or gardening, enables people to find solace. 10 As a result, CFs may be beneficial for a wide range of service users.

A number of theories have been mentioned within the care farming literature and some of these speak specifically to the nature element, such as the attention restoration theory11 and biophylia hypothesis. 12 Other theories relate specifically to the service user groups that attend CFs, for example desistence theory for probation service users13 and the recovery model for people with mental ill health. 14 Within these theories are the proposed mechanisms5 that explain how any effective intervention would be expected to bring about change. Within the review we followed the Medical Research Council (MRC)’s definition of a ‘mechanism of impact’, that is, ‘how intervention activities, and participants’ interactions with them, trigger change’ (p. 8). 5 Therefore, identifying these mechanisms within the care farming interventions will provide the links required to understand outcomes. These mechanisms are detailed in the theories we assessed. For example, desistence theory suggests that interventions that lead to a reduction in recidivism involve building human relationships, opportunities for reflection and change,15,16 developing self-efficacy17–19 and social capital by learning and applying new skills to develop a new, more positive identity. 17,20–23 A sense of community and the development of friendships are indeed valued aspects of a CF. 24 Furthermore, farmers are perceived as positive role models with a strong sense of identity, thus offering an essential positive figure that can be emulated through forming a new identity. 25 Both the concepts of building human relationships and creating a new identity are clearly present within care farming interventions.

The need for a study

Green care plays a particularly important role in modern society. Sitting within the third sector, they provide a type of support beyond that which can be provided by statutory health and social care organisations. Utilisation of the third sector by statutory services appears to be increasing, as demonstrated by recent social prescribing initiatives. 26 This has been driven partly by a capacity issue within health services relating to increasing life expectancies over the 20th century27 and an increasing prevalence of long-term conditions. However, it also relates to changing needs and demands of populations within modern societies. Many of the problems presenting to health-care providers (often as the first point of contact) are complex and often are underpinned or exacerbated by social problems (such as poor education, poor housing, unemployment and social isolation) and the skills within health services to address these issues do not exist within this sector. Likewise, among local authorities and probation there appears to be a recognised value in attending for social reasons, such as educational and employment needs and social interactions. Thus, there is the potential for care farming to improve the health and well-being of different population groups. The number of CFs has been growing, particularly in Europe, with an estimated 1000 CFs in the Netherlands,28 over 200 in the UK,4 900 in France, 300 in Belgium, 160 in Germany, 675 in Italy and 100 in Ireland. 29 Despite this growth, however, sustainability is a major challenge. Funding comes from a variety of sources: charitable donations, private sector funding schemes, charity funding schemes, contracts with local authorities, clinical commissioning groups, probation services and through income generated from the sale of farm produce. A key part of the process of securing funding is providing evidence of benefit for health and social outcomes. Although there is a growing body of evidence pointing to the benefits of nature-based interventions for a range of health-based outcomes,30–33 characteristically small sample sizes and a range of methodological limitations allow only tentative conclusions to be drawn. Within care farming research specifically, there have been a number of observational studies published across Europe and a small number of reviews7,34,35 and these studies document the extent and range of care farming initiatives and summarise the evidence for benefits. However, evidence gaps remain and these need to be addressed to ensure that commissioners and policy-makers understand how, and if, care farming is of benefit, and to which service user groups.

Rationale for the study

Health-care commissioners, in general, lack awareness and understanding about care farming. Furthermore, they consider the lack of evidence on the clinical effectiveness and cost-effectiveness to be a barrier to commissioning. 36 Although CFs cater for a wide range of vulnerable population groups and the evidence on all service user groups is of interest, the main focus of this research is one service user group: probation service users undertaking community orders (COs). By removing the wide diversity in need, the activities and support provided on the farm would be considered less complex. Furthermore, a single service user group enables an appropriate comparator group to be selected for the purposes of quantitative evaluation. Given the limited quantitative studies with CF participants and with probation service users, there were many methodological questions that required answers before a larger study could be considered, thus a pilot study was proposed. The key questions posed were:

1. How can recruitment of probation service users undertaking COs on CFs and in comparator settings be maximised?

2. What are the optimum ways to collect baseline and follow-up data (while minimising dropout), cost data from CFs and probation trusts and individual reoffending data from the Police National Computer (PNC)?

3. What are the impacts of CFs on probation service users’ lives and how appropriate are the measures used in the pilot study for identifying changes in quality of life, health and well-being?

4. What is the extent of variation between the activities and approaches used on different CFs and the variation in outcomes for probation service users at different types of CF?

5. What is the influence of seasonality on decisions concerning the allocation of setting to serve a CO, the activities on the CFs and the health and well-being of participants?

6. What factors may lead to selection bias and what are the potential confounders, particularly in terms of factors influencing allocation to CFs and other CO settings, and how can these best be measured?

7. What is the feasibility of measuring key parameters of costs and quality-adjusted life-years (QALYs) to conduct a future cost-effectiveness analysis of CFs in comparison with other CO settings for probation service users?

We formulated these questions into a series of objectives requiring a number of different methodological approaches within one study. In order to address the wider question regarding the evidence across all service user groups, we considered that a systematic literature review would clarify the extent of the evidence and highlight where key gaps remained.

Aims and objectives

The aim of this pilot study was to build knowledge on the mechanisms through which CFs improve the health and well-being of adult probation service users undertaking COs and to explore the feasibility of assessing the cost-effectiveness of CFs in achieving these outcomes in a subsequent study.

Specific objectives were to:

1. Conduct a systematic review of published and grey literature evaluating the impacts and mechanisms within CFs and green care in improving the health and well-being of disadvantaged populations and the impacts of COs and costs of offending to society and health services.

2. Identify factors that drive the decision of CO allocation in order to identify potential selection bias and confounders as well as the most appropriate ways to collect data on these factors.

3. Identify the most appropriate ways to gain informed consent, maximise recruitment and follow-up while minimising dropout and deliver questionnaires effectively among probation service users undertaking COs in CFs and comparator CO settings.

4. Identify the most appropriate ways to collect cost data, including health-care resource use by probation service users, cost of reoffending, costs of crime and gains of employability of probation service users in CFs and comparator CO settings and explore the feasibility of measuring cost–utility.

5. Identify differences in effectiveness in terms of quality of life, mental health, lifestyle behaviours and reoffending rates between the three CFs and between CFs and comparator settings in order to estimate variation and thus determine the intracluster correlation coefficient and sample size required in the follow-on study.

6. To develop a logic model based on probation service users’ experiences of serving their CO on a CF and their perceptions of the impact the CF has on their lives and well-being, taking into consideration seasonal changes, to illustrate the possible mechanisms that lead to changes in health and well-being among probation service users undertaking COs on CFs.

Public involvement

Our aim was to involve a small number of ex-probation service users in the design and interpretation of the study, particularly in relation to the pilot study (see Chapters 3 and 4) and the qualitative interviews (see Chapter 6). At the start of the study we were invited to an established service user board (SUB) group comprising, and facilitated by, ex-probation service users with some staff also present. It was suggested that a researcher should attend the 6-monthly meetings so that a small part of the meeting could be dedicated to the study. During the meetings, there were between 3 and 6 service users in attendance. In total, we were invited to three meetings. Early in the study, the service users contributed to the design of the questionnaire booklet containing the outcome measures, the information sheet and the consent form. They also advised us on what to expect when approaching some probation service users. We were cognisant that the SUB ex-probation service users had previously spent time in prison and that they may not reflect those who had received a CO. As explained in Chapter 3, coincidental with the privatisation of probation services, the SUB facilitator left and meetings stopped. During this time, we located and attended a separate SUB meeting in a different town within the same probation service. The service users’ (three attended the meeting) and facilitators’ expectations of the research were quite different from its actual purpose and because we were not offering a new ‘service’, the group felt that they had other priorities for their meetings and we did not attend further sessions. Service user engagement was therefore interrupted as we tried to identify who was responsible for the SUB group in the original location. On establishing this, we found a new SUB board in place comprising mostly staff and only two service users, which was a change in the dynamics of the group. Interest and knowledge about the study had receded and we were required to outline the purposes of the study again. Ultimately, little progress had been made from the very first visit and our early plans to involve service users in the qualitative work were not realised. On reflection, establishing our own group rather than tapping into an existing group may have, to some extent, militated against the impact that the privatisation of probation services had on the SUB. Our contacts within probation at the start of the study had advised us that volunteers were sparse and that attendance at an additional meeting (to that of the SUB group) may be poor.

About this report

This report comprises four main sections representing the four main workstreams that address the objectives (see Aims and objectives) outlined.

• Chapter 2 presents the methods and findings from the systematic literature review on the effectiveness of care farming on quality of life for a range of vulnerable groups (objective 1). One of the service user groups within the review is that of probation service users. The review does not include our own pilot study results, as these were not available at the time of conducting the review. We also present a number of developed logic models for particular service user groups but, as a result of the lack of studies, not one for probation service users. Within objective 1, we also included a review of evidence guiding cost-effectiveness analysis. This covered a review of the costs of offending and a review of the impacts of care farming and green care on health-related quality of life (HRQoL). These reviews are included in Chapter 5.

• Chapter 3 presents the methods for the pilot study, but also includes a summary of probation services and COs and how care farming fits within these. There is a further and important explanation of the major organisational changes that occurred in probation during the period of research and how these impacted on the study. As a consequence of these changes, some of our objectives could not be met in full. Chapter 3 addresses objectives 2, 3 and 4, which are essentially about processes.

• Chapter 4 presents the quantitative results for the pilot study, specifically addressing objective 5.

• Chapter 5 presents the methods and findings of the reviews guiding cost-effectiveness analysis (objective 1) and then reports the analysis of the key parameters of costs and utilities collected as part of the feasibility data that would be used for a larger study of cost-effectiveness of CFs in improving the quality of life of probation service users undertaking COs (objective 4).

• Chapter 6 presents the qualitative study, which involved interviewing probation service user managers regarding allocation decisions (objective 2) and probation service users (objective 6) as well as care farmers. Owing to changes in probation services during the study and other issues that arose in relation to care farming, we were unable to obtain sufficient data to enable us to fully address objective 6. These factors will be explained in Chapter 3. We considered our options and, based on our garnered knowledge regarding probation, we opted to interview a broader range of study participants to explore the role of care farming in probation. We considered that this would be most valuable to probation services going forward.

• Chapter 7 presents a discussion chapter that brings together the findings from each of the previous chapters and outlines the implications for research and practice.

Introduction, aims and objectives

The protocol for this review was registered with the Campbell Collaboration (SW2013–04). The primary objective was to systematically review the available evidence of the effects of CFs on quality of life, health and social well-being on a wide range of service user groups.

We aimed to synthesise the evidence in order to:

1. understand the size of the effect of CFs on the health, well-being or social outcomes of different population groups

2. examine whether or not effects differ depending on the activities and characteristics of the farm or farmer, the duration of time that participants spend at the farm, the number and diversity of the participants on the farm and whether or not the farm is the only intervention

3. understand the mechanisms of change for different population groups attending CFs using a range of study methodologies, including qualitative studies.

The evidence for the effectiveness of care farming is relatively recent (within the last 10 years). The complexities and multifaceted nature of CFs means that randomised controlled trial (RCT) study designs are challenging. In the light of this, our study synthesised the published and unpublished literature using a mixed-methods systematic review design. The review is registered as PROSPERO CRD42014013892 and SW2013–04 (the Campbell Collaboration). The full protocol can be found on the Campbell Collaboration website (www.campbellcollaboration.org/library/impact-of-care-farms-on-quality-of-life-different-population-groups.html; accessed 1 October 2014).

Methods

We conducted a mixed-methods synthesis using a sequential explanatory approach37 that involved the development of an intervention framework based on the theory and qualitative data with subsequent testing using the quantitative evidence. This approach is valuable in identifying possible mechanisms of change to inform the development of a logic model for care farming. We based the design of our logic models on the description and definitions provided by the MRC’s guidance for process evaluation of complex interventions. 5 Here, a logic model is defined as:

A diagrammatic representation of an intervention, describing anticipated delivery mechanisms (e.g. how resources will be applied to ensure implementation), intervention components (what is to be implemented), mechanisms of impact (the mechanisms through which an intervention will work) and intended outcomes.

Reproduced from Moore et al. (p. 8). 5 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

Given the nature of the evidence base in this area, and the limited studies assessing health outcomes, in our logic models we split outcomes into ‘process outcomes’ and ‘health outcomes’. Examples of process outcomes are those impacts that may well be precursors to health outcomes, for example self-esteem or confidence, and examples of health outcomes include anxiety, depression and quality of life. A narrative approach incorporating a wide range of study designs was planned based on early scoping of the literature that indicated a dearth of RCTs evaluating the effectiveness of CFs.

Findings

We found 2176 articles through searching of electronic databases and 125 via grey literature retrieval methods (Figure 2). We obtained full copies of 215 articles and, of these, 38 papers (representing 27 studies) met the inclusion criteria. We retained 26 studies that provided theoretical (to support the development of the logic model) and contextual information (to shed light on activities relating to specific service user groups) about the care farming intervention, despite the study’s exclusion from the main review.

FIGURE 2.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.

Fourteen qualitative studies reported in 17 papers (Table 2), 12 quantitative studies reported in 20 papers (Table 3) and one mixed-methods study28 met the inclusion criteria. From 20 publications, we identified 17 theoretical or philosophical concepts quoted in connection to care farming. From these, we retained 15 that offered a potential explanation for how CFs may work to bring about change in various service user groups.

Quality assessments

Six of the qualitative studies24,52–56,58,59 (40%) met more than 50% of the 37 quality assessment criteria (Table 4). Only one study59 met more than 60% of the criteria. One study met less than 20% of the criteria. 60 Openness about the researcher’s bias and assumptions and evidence of a pre-existing or newly established relationships were each addressed by only one study. Two criteria fundamental to all research practice are evidence of ethics approval and evidence of informed consent. These were not reported in nine (60%) studies. 24,28,45–51,53,59,60 All studies that met more than 50% of the quality criteria used a theoretical framework.

All quantitative studies had many limitations and were rated as being at a high risk of bias (Tables 5 and 6). For the two RCTs,61–64 allocation concealment was clearly described in one study. 64 Baseline assessments reported in both studies revealed differences between the intervention and the control groups in one study,25 which were not adjusted for in analysis. Differences in baseline characteristics were also reported in one study61–63 but unclear in the other. 64

The logic models

The MRC’s guidance on the evaluation of complex interventions5 provides the basis for the design of our logic models and the definitions of the terms used within them. For clarity, the MRC’s definition of a logic model is given as:

A diagrammatic representation of an intervention, describing anticipated delivery mechanisms (e.g. how resources will be applied to ensure implementation), intervention components (what is to be implemented), mechanisms of impact (the mechanisms through which an intervention will work) and intended outcomes.

Reproduced from Moore et al. (p. 8). 5 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

Theoretical concepts

Exploration of the identified theories revealed five theoretical concepts: restorative effects of nature, being socially connected, personal growth, physical well-being and mental well-being. The primary outcomes explicitly suggested by theories were related to depression and anxiety, and quality of life. The process outcomes suggested by theories were related to confidence, stress, coping and self-efficacy, and pro-social behaviours.

Intervention components, mechanisms and process outcomes

Through the process of deconstruction of reported themes, we identified 85 intervention components, 164 mechanisms and 24 process outcomes. Intervention components5 were organised into five categories: being in a group, the farmer, the work, the animals and the setting (Table 7). Overall, care farming intervention components relating to the farmer and the work appeared to be prominent features in the findings. Despite the fact that data were infrequently reported for single service user groups, we did observe some differences in the types of intervention components mentioned that may indicate differences in either the types of activities made available to disparate service user groups or the level of importance of those activities to types of service users.

TABLE 7 - Intervention categories according to service user group

Twenty-two out of 64 findings also included disaffected youth and service users with learning disabilities and older people.

Only 5 out of 25 findings were solely disaffected youth and 7 out of 21 findings included service users from other groups.

Seven out of 21 findings included service users from other groups.

Intervention category	Number of findings in each category (%)
	All client groups	Mental ill health and substance misusea	Disaffected youthb	Learning difficultiesc
Being in a group	13 (15)	10 (16)	4 (16)	4 (19)
The farmer	23 (27)	17 (27)	5 (20)	6 (29)
The work	29 (34)	20 (31)	7 (28)	10 (48)
The animals	11 (13)	10 (16)	3 (12)	0
The setting	9 (11)	7 (11)	6 (24)	1 (5)
All	85	64	25	21

We used the MRC’s definition of mechanisms to guide our identification of mechanisms within CF studies:

The intermediate mechanisms through which intervention activities produce intended (or unintended) effects. The study of mechanisms may include: participant responses – how participants interact with a complex intervention; mediators – intermediate processes which explain subsequent changes in outcomes; unintended pathways and consequences.

Reproduced from Moore et al. (p. 8). 5 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

Through the iterative clustering exercise, mechanism-based findings were organised into 15 categories of mechanisms (Table 8). Although we identified the frequency that the mechanisms occurred, we do not suggest that any one mechanism is more important than another at an individual level. In terms of frequency and spread of findings, the categories ‘understanding the self’, ‘social relationships’ and ‘belonging and non-judgement’ represented the most common categories across all studies. ‘Creating a new identity’ and the farm as a ‘distraction’ were least often observed across the studies.

TABLE 8 - Categories of mechanism with descriptions

Mechanisms	Description	Frequency of findings in each category (all groups)
Achievement and satisfaction	Working at the farm gives service users a sense of satisfaction. At the farm, participants learn to perform activities; hence, they spent their days being constructive. There is satisfaction with using their bodies and spending time outdoors	15
Belonging and non-judgement	The CF is seen by service users as a place of belonging and mutual acceptance. Feelings of solidarity are created through shared experiences. Service users enjoy working with the animals, which are perceived to be non-judgemental	22
Creating a new identity	Service users view themselves in a new light as a worker, principally as a farmer	3
Distraction	The farm creates physical work, which offers both a practical and mental distraction from the service users’ own negative thoughts. Conversations centre on work, which offers further distraction	6
Feeling valued and respected	Service users feel valued, appreciated and needed by the farmer (and the animals) and consider that they are respected ‘for who they are’	12
Feeling safe	The atmosphere at the farm creates a feeling of safety and security, providing a mental shield between illnesses and addictions. For some service users this experience is enhanced through physical contact with the animals, but for others there is a need to overcome fear of animals, which can then lead on to a feeling of safety	12
Learning skills	CFs give service users the opportunity to learn new skills ranging from growing crops to looking after animals, which enables some to gain qualifications, enabling then to (re)enter the work place	11
Meaningfulness	Service users perceive tasks as meaningful because they are judged to be useful to others and are ‘needed to conduct day-to-day activities at the farm’. Service users also see their role as personally meaningful, contributing to society giving them a sense of purpose, happiness and fulfilment	11
Nurturing	Through helping each other and caring for the animals/plants, service users become considerate of other people’s needs and recognise that they are doing good for other living creatures	4
Physical well-being	Through physical activity on the farm, service users improve their physical strength. There is a sense of ‘good’ tiredness from physical work. Service users start to feel more independent and healthier	10
Reflection	The CF environment is quiet and peaceful, allowing service users to stop and reflect on their problems, their social influences and also the progress they have made. For young people, working at the farm gives space and time away from their family and friends	3
Social relationships	CFs provide opportunities for participants to interact with the farmer and other service users. For instance, often service users were working together in groups which helped them to develop their communication skills. As the intervention progressed, the service users deepened their relationships with the farmer and considered him or her as a role model. Once service users gained social confidence, their social networks grew. In particular, they found that talking about their farm work was more interesting than talking about their illness in social functions. However, a few service users did not want to interact with others and found it difficult to deal with the diverse range of service users at the farm	16
Stimulation	Service users find tasks stimulating, giving them more energy and encouraging a mindful approach to work, especially around animals, which are unpredictable. Working with animals offers a sensory experience and the energy derived from the work enables them to work through their own problems better. The experience of being in nature is energising	7
Structure	The daily farming activities provided a predictable work environment to the service users. This consistency helped the service users to gain a normal rhythm. Moreover, the farmers also allowed service users to work at their own pace as they understood that the service users can have a ‘bad day’ and may not be able to work at full capacity. Similarly, farmers involved participants in deciding tasks for the day	8
Understanding the self	The CF environment has allowed service users to better understand themselves. Participant’s self-awareness grew while at the CF. For example, learning to master an activity at the farm increased their self-respect and positive self-image. At the farm, participants were free to be themselves, and they also had the opportunity to learn and when they made mistakes they were given time and guidance to learn from their mistakes. This gave them the understanding that tasks at the farm are manageable, which enhanced their self-efficacy and self-confidence. Some found that caring for and cuddling animals helped them to deal with problems	24

In terms of service user groups, findings from the substance misuse and mental ill health service users were reported together and could not be separated. This service user group, as a combined group, was the largest, with 105 findings from 10 studies. No findings relating to ‘reflection’ or ‘creating a new identity’ were found in this combined service user group. For disaffected youth, ‘feeling safe’ was more frequently reported than ‘belonging and non-judgement’. ‘Achievement and satisfaction’ was frequently mentioned in both the combined mental health and substance misuse group and the learning disabilities group, but was reported less often in the disaffected youth group. ‘Reflection’ was also reported more often in the disaffected youth group than in the others. In the learning disability service user group, ‘understanding the self’ was reported less frequently than ‘social relationships’, ‘belonging and non-judgement’, ‘social relationships’ and ‘meaningfulness’.

Of the 24 process outcomes reported in the qualitative studies (Table 9), 11 related to emotions, such as increased confidence and self-esteem. Improved coping and feelings of well-being were also mentioned in numerous studies, as was independence.

TABLE 9 - Qualitative studies with categories of mechanisms and process outcomes

ART, attention restoration theory; PET, psycho-evolutionary stress reduction theory.

Category with most findings.

Philosophical concept rather than theory.

Studies scoring higher on quality assessment.

References	Number of mechanism findings	Categories of mechanisms	Target groups	Outcomes reported by participants	Theoretical basis
Baars et al.45	16	Achievement and satisfaction; belonging/non-judgement; distraction; feeling valued/respected; learning skills;a nurturing; physical well-being; social relationships; structure; and understanding the selfa	Mental ill health	Less medication	None stated
Bjørgen and Johansen46	5	Distraction;a feeling safe; and social relationshipsa	Mental ill health	Increased confidence, physical health and vocational rehabilitation	None stated
Elings47	20	Achievement and satisfaction; belonging/non-judgement; creating a new identity; feeling safe; feeling valued/respected;a learning skills; meaningfulness; stimulation; structure; and understanding the self	Mental ill health: service users with intellectual disabilities	Improved self-esteem	None stated
Elings and Beerens;48 and Elings and Hassink49,50	16	Achievement and satisfaction; belonging/non-judgement; distraction; learning skills; meaningfulness; physical well-being; stimulation; and understanding the selfa	Mental ill health: psychiatric; substance misuse	Increased confidence	Anthroposophyb
Elings et al.28	3	Feeling safe; meaningfulness; and structure	Mental ill health; substance misuse	–	None stated
Ferwerda-van Zonneveld et al.51	5	Feeling safe; reflection; stimulation; structure; and understanding the self	Children with autism spectrum disorder	–	None stated
cGranerud and Eriksson52	13	Achievement and satisfaction; belonging/non-judgement;a feeling safe; feeling valued/respected; meaningfulness; physical well-being; structure; and understanding the self	Mental ill health: long-standing severe psychotic disorders, personality disorders; and substance misuse	Improved well-being, self-perception; social life and confidence	Recovery model
cHassink;53 and Hassink et al.24	10	Belonging/non-judgement;a feeling safe; feeling valued/respected; learning skills; social relationships; structure; and understanding the self	Mental ill health; disaffected/excluded youth; and older people	–	Recovery model
cIancu et al.54	7	Belonging/non-judgement; learning skills;a meaningfulness; physical health; social relationships; and understanding the self	Mental ill health	Increased confidence, mood and less tiredness	Recovery model
cKaley55	22	Achievement and satisfaction; belonging/non-judgement;a creating a new identity; feeling valued/respected; meaningfulness; nurturing; social relationships; stimulation; and understanding the self	Learning difficulties	Increased independence, well-being and reduced anxiety and healthy lifestyle	Therapeutic landscape concept
cKogstad et al.56	6	Feeling safe; meaningfulness; physical well-being; reflection; social relationships; and understanding the self	Disaffected/excluded youth	–	Recovery theory
Leck et al.57	16	Achievement and satisfaction; belonging/non-judgement; feeling safe; learning skills; meaningfulness; nurturing; physical well-being; social relationships;a and structure	Mental ill health; substance misuse; disaffected/excluded youth; and learning difficulties	Increased happiness and improved pro-social behaviours	ART, biophilia, PET
cPedersen et al.58	16	Achievement and satisfaction;a belonging/non-judgement; distraction; feeling safe; feeling valued/respected; learning skills; meaningfulness; nurturing; physical well-being; stimulation; understanding the self	Mental ill health: people with depression	Increased confidence and independence	Biophilia; self-efficacy theory;
cSchreuder et al.59	5	Feeling valued/respected;a reflection; understanding the selfa	Disaffected/excluded youth	–	Salutogenic theory
North Essex Research Network, South Essex Service User Research Group60	4	Achievement and satisfaction; social relationships; stimulation; understanding the self	Mental ill health	Increased independence	None stated

Mapping mechanisms from qualitative studies to the theoretical concepts

Across the three main service user groups (combined mental health and substance misuse, disaffected youth and learning difficulties), there were some differences in how mechanisms mapped to theoretical concepts. In the combined mental ill health and substance misuse group, the number of mechanism findings that mapped to ‘mental well-being’ was almost double that of any other theoretical concept. In the other service user groups, ‘being socially connected’ and ‘mental well-being’ were similarly represented by the mechanisms. Only four single findings within the mechanism categories of ‘reflection’, ‘stimulation’ and ‘feeling safe’ appeared to map on to the theoretical concept of ‘restorative effects of nature’, but these mapped more appropriately to ‘mental well-being’. The categories of mechanisms were then combined with the intervention components and process outcomes to create a logic model for the following service user groups: all service user groups; combined mental ill health and substance misuse group; disaffected youth group; and learning disabilities group (Figures 3–6). There were insufficient data to develop models for older people and for people with autism spectrum disorder.

FIGURE 3.

Logic model for all service user groups. a, Mechanisms that were most frequently found and with greatest spread across studies. Grey and black symbols show quantitative evidence for which – means no significant difference and + means significant difference; grey represents RCT evidence; two symbols beside each other show different time points within the same study; and shaded process outcomes equate to evidence from qualitative literature. t, theory based.

FIGURE 4.

Logic model for combined mental ill health and substance misuse group. a, Mechanisms that were most frequently found and with greatest spread across studies. Grey and black symbols show quantitative evidence for which – means no significant difference and + means significant difference; grey represents RCT evidence; two symbols beside each other show different time points within the same study; and shaded process outcomes equate to evidence from qualitative literature. t, theory based.

FIGURE 5.

Logic model for disaffected youth group. a, Mechanisms that were most frequently found and with greatest spread across studies. Black symbols show quantitative evidence for which – is no significant difference and + is significant difference; and shaded process outcomes equate to evidence from qualitative literature. t, theory based.

FIGURE 6.

Logic model for learning disabilities group. a, Mechanisms that were most frequently found and with greatest spread across studies. Shaded process outcomes equate to evidence from qualitative literature. t, theory based.

Summary

There is a general trend of evidence to suggest that care farming may be effective for some service user groups; however, this evidence is limited. Small study sizes, evaluations involving mixed service user groups, the use of multiple and sometimes unvalidated outcome measures, short follow-up periods and absence of missing key outcomes that fit with theory have all hampered the development of a more robust evidence base. However, we now have a set of theory-based logic models that offer a framework for research evaluations, and with recommendations in place to address the current research inadequacies there is an opportunity to vastly improve the evidence base for care farming.

Despite the current lack of conclusive evidence from RCTs and robust effectiveness studies, the indications from the available evidence suggest that CFs are a promising alternative or adjunct to mainstream approaches for mental ill health. Lack of choice, gender inequalities and overburdened statutory services indicate the need for a credible alternative treatment option. A concerted effort to increase awareness among commissioners of health care, front-line service providers and potential service users about care farming, and how and for whom it may work, is needed. Models across Europe that offer a more integrated approach between green care and statutory services could provide the learning of this. The evidence for care farming for other service user groups is not as well developed as for mental ill health but that is not to say there is not a need. Disaffected youth, adult probation service users and people with dementia represent significantly large vulnerable population groups for whom current service provisions struggles to meet demand. The need to continue to improve and provide high-quality research in these areas is therefore pressing.

Objectives

1. Identify factors that drive the decision of CO allocation in order to identify potential selection bias and confounders as well as the most appropriate ways to collect data on these factors.

2. Identify the most appropriate ways to gain informed consent, maximise recruitment and follow-up while minimising dropout and deliver questionnaires effectively among probation service users undertaking COs in CFs and comparator CO settings.

3. Identify the most appropriate ways to collect cost data, including health-care resource use by probation service users, cost of reoffending, costs of crime and gains of employability of probation service users in CFs and comparator CO settings and explore the feasibility of measuring cost–utility.

4. Identify differences in effectiveness in terms of quality of life, mental health, lifestyle behaviours and reoffending rates between the three CFs and between CFs and comparator settings in order to estimate variation and thus determine the intracluster correlation coefficient and sample size required in the follow-on study.

5. To develop a logic model based on probation service users’ experiences of serving their CO on a CF and their perceptions of the impact that the CF has on their lives and well-being, taking into consideration seasonal changes, to illustrate the possible mechanisms that lead to changes in health and well-being among probation service users undertaking COs on CFs.

Care farming and probation service users

Care farming is one possible location where probation service users may be allocated to serve their CO in England. A survey conducted in 2013 estimated that approximately one-quarter of CFs provided support for probation service users. 6 However, more recent work by the study team suggests that at the time of this study, the number of CFs taking service users was much smaller. Our investigations identified only three CFs in England holding contracts with probation services. Our study period coincided with the official disbanding of probation services in May 2014 and subsequent establishment of new organisations as community rehabilitation companies (CRCs) and the National Probation Service (NPS). This organisation change may well have had an impact on the services commissioned by probation.

Community orders and the role of care farming

The most serious crimes and those committed by serial offenders tend to result in a custodial sentence. Offenders whose crime is considered to be less serious in nature can be given a CO, which is completed in the local community under the supervision of probation services. The ultimate purpose of COs is to reduce the risk of reoffending through ‘requirements’ – these are interventions that the probation service user is ‘required’ to comply with to complete the order successfully. There are many types of requirements (Figure 7) and within these there may be multiple programmes, activities and projects. Probation service users can be given one requirement or more depending on the nature of their offence, their offending history and their lifestyle behaviours. COs vary in the degree to which they aim to rehabilitate or punish the probation service user. Probation service users with the greatest needs, for example those with drug or alcohol problems, will be offered a CO with a substantial rehabilitative requirement such as alcohol, drug or mental health treatment. Irrespective of the level of need, all orders, however, must include a punitive element. In predominantly punitive orders, the main requirement might involve a form of community payback (CP) that includes, for example, litter picking or working in a charity shop.

FIGURE 7.

Types of requirements within COs. Adapted from Type of Community Order. 80,81 Crown copyright. Contains public sector information licensed under the Open Government Licence v3.0 (www.nationalarchives.gov.uk/doc/open-government-licence/version/3/).

Adapted from Type of Community Order. 80,81 Crown copyright. Contains public sector information licensed under the Open Government Licence v3.0 (www.nationalarchives.gov.uk/doc/open-government-licence/version/3/)

The exact process for allocation of a service user to different CO sites is not clear and differs between probation regions. One of the aims of this study was to understand this process better, particularly to identify any potential confounders, that is, factors that were associated with the allocation decision and with the outcome (i.e. health outcomes and reconvictions)

Care farming as part of a CO can be either rehabilitative or punitive depending on how the probation service views the role of care farming in probation and, in addition, how well the local CF itself has communicated its intentions, services and desired outputs.

Changes to probation services and impact on the research

The privatisation of the NPS was announced as the current research study began. The process lasted for the duration of the study. From the outset, this affected our capacity to carry out our research and the effects of this will be described throughout this report, particularly in relation to the pilot study. Over and above the changes described throughout this chapter, there were three key time points when our study was most affected. The first key time point was June 2014, when the official split in probation services took place, forming a NPS (for the management of higher-risk offenders) and interim CRCs. At this time, the CRCs were not, in fact, new organisations but a temporary reorganisation of existing probation staff. At this time, information technology (IT) services were split, staff were moved and new procedures were put in place. The second major event was the announcement of the preferred bidders in October 2014. Up to this point, various consortia had been preparing their bids, and some of these involved senior staff from existing probation services as well as organisations that were contracted to probation. This included one of the CFs within our pilot study. The third and final key event was in February 2015, when the final contracts were awarded and changes in administrative processes were announced alongside changes in IT services in response to this.

Throughout the study, and particularly at these key time points, staff were leaving or had changing roles, new contracts were being issued and IT staff were under pressure to deliver new IT systems in response to changes in assessments. All of these factors meant that establishing, and maintaining, reliable contacts who could respond to our requests and support our recruitment processes was particularly challenging.

Target population, sample sizes and timelines

This was a pilot study aiming to assess feasibility, recruitment and follow-up rates, clarify selection biases and effects of confounding. As a pilot, the study was not powered to assess effectiveness, so a conventional sample size calculation was not considered appropriate. We estimated that, in order to determine sample size for a follow-on study that took account of between-CF effects and the possible effects of bias (i.e. response rates and dropout), a pilot sample of 300 probation service users would be sufficient.

With an expected loss to follow-up of 40%, this allowed a final sample size of 180 participants (90 CF attendees and 90 comparator location attendees) with both baseline and follow-up data.

We planned to recruit over three centres as follows:

• Centre 1 – n = 120 (n = 60 from CF; n = 60 from comparator site) over a 12-month period.

• Centre 2 – n = 90 (n = 45 from CF; n = 45 from comparator site) over a 9-month period.

• Centre 3 – n = 90 (n = 45 from CF; n = 45 from comparator site) over a 9-month period.

Our monthly targets were therefore set at 10 participants in each centre. Owing to delays in setting up and recruitment, which are explained in the remaining sections of Chapter 3, we made adjustments to these monthly targets. Our initial 12-month recruitment period for centre 1 fell to 10 months, and thus our monthly target became 12 participants. In the other two centres we set a monthly recruitment target of 15 participants, with an expected shortened recruitment period of 6 months.

Outcome measures

The primary outcome of the study was quality of life and well-being, which was derived from the Clinical Outcome in Routine Evaluation–Outcome Measure (CORE-OM). 82 CORE-OM has previously been used to evaluate quality of life of probation service users. In particular, it has been used to assess the risk of self-harm among prisoners83 and among mentally disordered offenders in a high-security hospital. 84 The CORE-OM is also commonly used to measure psychological distress in evaluations of mental health interventions85 and it can be used to derive utilities that can then be transformed into QALYs. 86 The 34 items cover four dimensions: subjective well-being, problems/symptoms, life functioning and risk/harm. 82 The full version of the questionnaire can be found on the CORM-IMS website (www.coreims.co.uk; accessed 1 October 2013).

The secondary outcomes were:

• individual-level data on reconvictions over a maximum 18-month period obtained from individual-level data from the PNC

• mental well-being derived from Warwick–Edinburgh Mental Health and Well-Being Scale (WEMWBS)87

• measures of smoking, alcohol, drug use, diet and physical activity adapted from the General Lifestyle Survey88 and Health Survey of England89

• measures of the connectedness to nature90,91

• exploration of health utility, as derived from the CORE-6D based on the CORE-OM,82 and QALYs permitting a cost–utility analysis86

• exploration of the health and social care use cost and the cost of a CO as an intervention.

Recruitment and follow-up plan

The logistics of the recruitment and follow-up processes were bespoke to each site; however, the overall plan was to perform recruitment and follow-up face to face by a research assistant (RA). Informed consent was obtained to take part in the study and also, independently, to access personal information from the probation and police services. When follow-up could not be performed face to face because of changes in a probation order, we used postal questionnaires. The overall aim was to complete follow-ups at the end of the CO or if the completion of the order went beyond the 1-year recruitment period participants were followed up for 6 months from the start of their order, regardless of whether or not their order had been completed. For probation service users who were ‘breached’, that is failed to comply with their order, we monitored their return back into the system and followed them up at the end of their subsequent CO or at the end of the follow-up period. For those probation service users not returning back into the system, we offered a financial incentive to complete a postal questionnaire to maximise response rates. If they were given a prison sentence, they were noted as ‘lost to follow-up’ for the quality of life, health and well-being measures; however, their reconviction outcome was still assessed.

Approvals

A critical initial step in gaining access to probation services was gaining both ethics and National Offender Management Service (NOMS) approvals. Ethics approval from the Leeds School of Medicine Research Ethics Committee (SoMREC/13/014) was obtained promptly (December 2013) and approval from the NOMS given in February 2014 (NOMS; 2013–257). The delay in obtaining NOMS approvals impacted on the start date for recruitment in our first site (see Appendix 3, Figure 13, for the impact on recruitment timeline).

Bespoke study documentation

So as to distance ourselves from the probation service and the MoJ, we did not include their logos on any study documentation. Probation staff considered this important because of the possibility that probation service users may not be honest in their questionnaire responses if they thought it would influence their CO. In addition to standard research documentation (information sheet and consent forms), we developed a separate consent form granting permission to access individual data held on probation IT systems and reconvictions data held by the police and MoJ on the PNC. The permission to access personal data form was developed with senior staff in the probation services and was a requirement of the information sharing agreements (ISAs) that were developed later on in the study. In addition, the signed forms were provided (as proof that permission had been granted to us) to the police in one centre in which an ISA was not required under the Freedom of Information Act. 92

We also developed a service user assistance form and a service user details form. The service user assistance form enabled us to assess any difficulty with the outcome measures. The service user details form recorded (in addition to name, address, etc.) information on the type of requirement [i.e. unpaid hours or specified activity (SA)], number of unpaid hours/session, planned attendance days, number of hours completed at baseline and allocated project. This assisted with monitoring progression and changes through the probation order.

Development of the study procedures within the participating centres

Based on information provided by Care Farming UK, we contacted CFs that indicated that they worked with probation service users. Farms that routinely took in probation service users through their local probation service in sufficient numbers to meet our recruitment targets were asked to take part in the pilot study through a telephone call and face-to-face visit. Three CFs, (CF1, CF2 and CF3) agreed to take part. We subsequently sought verbal agreement from the three local probation services (PS1, PS2 and PS3) to conduct the study. Collectively, we refer to these as centres 1, 2 and 3, respectively. Having ethics and NOMS approvals in place was key to progressing the study at each of the three centres.

Although at the time of study set-up probation was a nationalised organisation, there were key organisational differences between each of the probation services that led to the development of bespoke research approaches to probation service user recruitment and data acquisition. At each service, contact points varied from a single individual senior staff member (e.g. head of operations) at one site, to a mix of different levels of staff, both management and front line at the other services (Table 10). These staff were fundamental to establishing key information items that would enable identification, recruitment and follow-up of probation service users from both CF and comparator sites into the study. The key information items included:

• the type of order given to the probation service users, who were subsequently allocated to CFs

• the characteristics of probation service users allocated to CFs including, for example, risk of harm, risk of reoffending [Offender Group Reconviction Scale (OGRS)], age range and gender distribution, plus any restrictions (e.g. no one with unstable substance misuse or who was currently in employment)

• when during the CO the probation service users were allocated to the various requirements and projects

• who decided on allocation and where this was done

• when the probation service user commenced their requirement/project during the CO.

Having established the type of order and the characteristics of the probation service users who attended CFs, we were then able to establish the characteristics of the comparator group.

In two of the probation services, front-line staff, that is CP supervisors, project officers and some offender managers, were key project contacts. In the third probation service, the head of operations was the key person for liaising and information. Each probation service used a different approach to allocating probation service users to various projects within COs. This meant that the way in which CFs were utilised within COs differed. Therefore, the process of identification and recruitment of service users differed in each centre. The following section describes local service configurations, including the process of probation service user allocation and the approach taken to identification and recruitment at each centre.

Centre 1

Centre 2

Centre 3

Data acquisition across all centres

Statistical methods

All analysis was completed as ‘intention to treat’, that is, it was analysed on the initial allocation of the service user for their CO (i.e. CF: yes or no), ignoring non-compliance, deviations from the CO and anything that happened after CO allocation. Although intention-to-treat analysis can be criticised for being too cautious, and thus being more susceptible to type II error, this approach is recommended in the Consolidated Standards of Reporting Trials (CONSORT) guidelines on the reporting of RCTs,94 and it was felt that, as relatively fewer service users were recruited than originally expected, a comprehensive analysis of non-compliance could not be undertaken.

Differences in the service users’ characteristics at baseline between CO allocation, arrival on site and completion of follow-up questionnaire were summarised and tested using regression models. The service users’ characteristics were placed in four categories: demographics, outcomes from questionnaires, health and lifestyle questions and health and social care use. Depending on the characteristic being tested, the following summary statistics and models were used: median with interquartile range (IQR) and median regression for continuous non-normal characteristics; numbers with percentages and Fisher’s exact test for categorical characteristics; and median IQR and negative binomial regression for counts. The same approach was used to show the differences in follow-up characteristics between CO allocation and nDelius and offending characteristics by CO allocation.

Differences in the changes in outcomes at follow-up between CO allocations were summarised and explored using median (IQR) and median regression for continuous non-normal outcomes and numbers with percentages and multinomial logistic regression for categorical outcomes. The differences were shown as unadjusted and adjusted for baseline.

As the follow-up outcomes were collected in only a fairly small number of service users, no further adjustment for differences in CO allocation was possible. However, the reconviction status at 18 months was available for 120 service users; therefore, further adjustment for this outcome was possible.

The differences in reconviction at 18 months by CO allocation were explored using Cox proportional hazards regression models, for which the time to event was either the first reconviction date since recruitment for those with a reconviction or 18 months if no reconviction occurred within that time. The unadjusted hazard ratio provides an estimate of the risk of 18-month reconviction for those allocated to a CF compared with those with other CO allocations. However, this estimate of the risk is unadjusted for other differences in characteristics between the two CO allocation groups that may also account for any differences in the risk of reconviction. Adjustment for ‘confounding by indication’, as it is known, was carried out by applying the stabilised inverse probability treatment weights (IPTW) to the Cox proportional hazards regression model. The IPTW produces a pseudo-population in which CO allocation is independent of any of the variables considered to be potential confounders, enabling an unbiased estimate of the impact of CFs on 18-month reconviction rates compared with other COs.

To calculate the IPTW, a logistic regression model was used to estimate the probability that the service user was allocated to a CF. Baseline and offending characteristics that showed differences between CO allocation (see Table 30, Appendix 4) were explored further to assess if they were potential confounders and, if so, if they should be included in the model to predict the probability that the service user was allocated to a CF. The associations between the potential confounders and the reconviction rate and CO allocation were tested using Cox proportional hazards regression models and logistic regression models, respectively. Confounders were those characteristics associated with both reconviction rate and CO allocation. Furthermore, those characteristics associated with reconviction rate, but not CO allocation, were also included in the model to predict the probability that the service user was allocated to a CF. However, those characteristics associated with CO allocation, but not reconviction rate, were not included. This method aims to include as many potential predictors as possible, even if their contribution is small, but is balanced in this case with the relatively small sample size (usually recommended to be at least 200 service users). 94 The inverse of the probability (IPTW) was then used to adjust each participant’s risk of reconviction for the confounders, in which the reciprocal of the probability was used if the participant was not allocated to a CF. However, without a numerator this inverse weight is likely to be highly variable and so the inverse probability of treatment weight is stabilised. The numerator was either the proportion of participants allocated to a CF or its reciprocal for those participants allocated to other COs.

The logistic regression models to predict if a participant was allocated to a CF can include only participants with complete data. In total, 600 data items were included in the models used to produce the weights. Of these, 14 (2.3%) were missing. To impute these missing data, we used switching regression, an iterative multivariable regression technique that retains an element of random variation in the estimates. Within Stata^® (StataCorp LP, College Station, TX, USA), these methods are incorporated within the mice and mi estimate programs. For each variable, the distributions were compared with and without the imputed data to confirm that the imputed data did not alter the distribution of any of the variables. The imputed data were used only in the prediction of CO allocation and were not included in the reconviction rate models, as the IPTW-adjusted Cox proportional hazards model accounts for these confounders.

A number of different models were employed to explore the differences in the seasonal indicators (season, temperature, rainfall and sunshine) dependent on the outcome. To assess if there any seasonal differences in CO allocation, median regression models were used for temperature, rainfall and sunshine and Fisher’s exact test for season at time of allocation. To explore the association between temperature, rainfall, sunshine and season with CORE-OM and WEMWBS scores at both baseline and follow-up, median regression models were used.

Description of sample: characteristics of respondents

Of the 136 respondents initially recruited to the study, two subsequently withdrew from the study and so were removed from all subsequent analysis (Figure 8). For the 134 respondents, 50 (37%) were allocated to a CF for their CO. At baseline, the only factors that were significantly different (at the 5% level) between those allocated or not to care farms were gender, the number of missing CORE-OM questions, whether or not any substances or cannabis were used during the past 4 weeks, smoking status and agreement with the statement ‘Healthy foods are enjoyable’.

FIGURE 8.

The Consolidated Standards of Reporting Trials Diagram (CONSORT) flow diagram: recruitment and follow-up pathways in pilot study. 94 a, For the first 3 months of the study, exclusion criteria included in breach/custody, not allocated to CF or comparator setting, no risk data (not allocated), out of area and CO with other requirement (n = 32). However, after review of the early recruitment period, these criteria were removed. b, The CF closed to probation at the end of September. Prior to this 15 out of 19 (76%) service users completed their CO at their allocated placement, and the closure lead to 11 participants being moved to another placement. c, Lost to follow-up includes being in breach, in custody, not returning the questionnaires, not followed up and dead/terminally ill.

Table 11 shows only those with significant differences. Table 30, Appendix 4, shows all characteristics, whether or not significant. OGRS scores (risk of reoffending) were significantly different (at the 5% level) between those allocated or not to CFs (Table 12).

• Only 2 out of the 50 (4%) service users allocated to CFs were women, compared with 37 out of the 84 (44%) allocated to other COs (p < 0.001 from Fisher’s exact test).

• Although the median number of missing CORE-OM questions for those allocated to CFs and to other COs were both zero, there are differences in the variability between the two groups (IQR CF 0–24 and non-CF 0–1). The difference in variability is reflected in the difference in the number of missing CORE-OM questions between the two groups. On average, the CF group had a percentage change of 139% [95% confidence interval (CI) 21% to 370%] more missing questions than those allocated to other COs.

• Overall, 47% of those allocated to CFs reported that they had used substances in the past 4 weeks compared with 24% of those allocated to other COs (p = 0.01).

• In total, 34% of those service users allocated to CFs reported that they had used cannabis in the past 4 weeks compared with 16% of those allocated to other COs (p = 0.04).

• Overall, 78% of those service users allocated to CFs were current smokers compared with 57% of those allocated to other COs (p = 0.02).

• Proportionally more respondents agreed than disagreed with the statement ‘Healthy foods are enjoyable, of those service users allocated to other COs compared with those allocated to CFs, although more respondents allocated to the CFs neither agreed nor disagreed with the statement (p = 0.003).

• Mean OGRS scores at the time of disposal of the CO were 26 points higher (95% CI 6.86 to 45.14 points) among CF users than those at comparator sites.

None of the questionnaire outcome measures was significantly different between the two groups at baseline (Table 30, Appendix 4). For example, the median CORE-OM score was 7.4 in the CF group and 7.1 in those allocated to other COs. Although the mean difference between the medians was 0.3, the 95% CI of this estimate is –3.95 to 4.55, which includes zero; therefore, there is no significant difference in the CORE-OM score between the two groups at the 5% significance level. The 95% CI is dependent on the sample size and, as only 93 out of the 134 respondents completed the CORE-OM to derive scores, missing data will have contributed to a wider estimate of the CI than if all respondents had completed the CORE-OM. Further breakdown of which particular questions were not completed on the CORE-OM, WEMWBS and Connecting with Nature questionnaires can be found in Table 31, Appendix 4.

Using the respondents’ postcode we were able to assign the index of multiple deprivation (IMD) ranking via lower super output area. These rankings were then placed into quintiles: first, for quintiles within the cohort and, second, matched to quintiles for England. The second of these shows that the Evaluating COs (ECO) study cohort was from more deprived quintiles than the English population in both CO allocation groups (see Table 30, Appendix 4). For example, 46% of respondents were from the most deprived quintile in England in both CO allocation groups. Comparing the IMD quintiles of the two CO allocation groups within the cohort shows some differences in the deprivation profile. Proportionally more of those allocated to other COs were in the most deprived quintile than those who had been allocated to a CF (26.5% vs. 10.2%, respectively) and in the least deprived quintile (21.7% vs. 16.3%, respectively). Conversely, proportionally more of those allocated to a CF were in the middle quintile than those allocated to other COs. Although these differences are not significant (p = 0.08), there is some indication that this may be a potential confounder when looking at the difference in the change in outcomes between the two groups.

Table 13 explores whether or not the baseline characteristics of the respondents were different between the three sites. (Table 13 only shows those with significant differences, see Table 32, Appendix 4, for all characteristics.) Although summary statistics for each of the three sites are shown, testing for whether or not there were significant differences in the characteristics between both site and CO allocations could be carried out with only site 1 and 3 because of the small number of service users recruited at site 2. The interaction between site and CO allocations for each characteristic was used to test these differences. Owing to the small numbers in the categorical characteristics within each sites, testing of the interaction was carried out on only a limited number of characteristics.

There are significant differences in the ages of the recruits between sites and between CO allocations within sites (p = 0.02). In site 1, those participants allocated to the CF had a median age of 33.5 years and those allocated to other COs had a median age of 31 years. In site 3, those participants allocated to the CF had a median age of 29 years and those allocated to other COs had a median age of 42 years.

There were significant differences in the CORE-OM scores between sites and between CO allocations within sites but not for the WEMWBS and Connected to Nature scores (CORE-OM, p = 0.04; WEMWBS, p = 0.07; and Connected to Nature, p = 0.16).

There were significant differences in the characteristic of ‘Number of days of ≥ 30 minutes’ physical activity in the last week’ between sites and between CO allocation within sites (p = 0.02). No other baseline characteristics appeared to show differences between sites and between CO allocations within sites, although the comparisons are limited as a result of the relatively small numbers within sites.

Loss to follow-up with reasons

Of the 134 respondents, 70 (52.2%) completed the follow-up (Table 14). Those respondents followed up, through the completion of the questionnaire, were older, more likely to be in the NPS rather than the CRC, did not smoke, used fewer substances in the last week and used fewer health services in the past month than those not followed up. The mean difference in the median age between those followed up and those who were not was 6 years (95% CI 2.59 to 9.41 years). There are no other differences in baseline characteristics between those respondents followed up or not including the proportion allocated to the CFs (p = 1.00) (see Table 33, Appendix 4).

Follow-up and outcomes

For the 70 respondents at follow-up, those that were allocated to a CF (n = 27) were more likely to have changed their CO during the follow-up period, have a higher CORE-OM score, have used substances and used more of them during the past 4 weeks and were less likely to agree with the ‘Healthy foods are enjoyable’ statement (Table 15). In total, 49% of those respondents allocated to a CF had changed their CO, compared with only 28.9% of those allocated to other COs (p = 0.01). There was an indication that those respondents allocated to a CF had, on average, fewer hours to complete of their CO at follow-up than those allocated to other COs (mean difference –7.55 points, 95% CI –16.03 to 0.93 points), although this was not significant (Table 33, Appendix 4).

Those service users allocated to a CF had a higher CORE-OM score at follow-up, on average 4.1 points higher (95% CI 0.63 to 7.57 points), than those allocated to other COs. The other questionnaire outcomes showed no significant difference between groups. Further breakdown of particular questions on the CORE-OM, WEMWBS and Connecting with Nature questionnaires at follow-up can be found in Table 35, Appendix 4.

For the 134 respondents, it was possible to link 120 with nDelius and reconviction information. Three respondents did not consent to access of their nDelius information and it was not possible to link 11 respondents to their reconviction data. Of the 120 allocated to a CF (n = 40), the respondents had a higher OGRS score at disposal (a term used in the criminal justice system when an order is given by the court once proceedings are complete), were more likely to be white British and more likely to report having a disability than those allocated to other COs (Table 16).

The median OGRS score at disposal was, on average, 26 points higher (95% CI 6.86 to 45.14 points) in those service users allocated to a CF than those allocated to other COs. Furthermore, there are significant differences in the OGRS scores between sites and between CO allocations within sites (p = 0.02). In site 1, those service users allocated to the CF had a median OGRS score of 32.5 points (IQR 5–56 points) and those allocated to other COs had a median score of 25 points (IQR 11–49 points). In site 2, those service users allocated to other COs had a median score of 28.5 points (IQR 17–40 points), although only one respondent allocated to a CF had an OGRS score (10 points). In site 3, those service users allocated to the CF had a median OGRS score of 87 points (IQR 70–90 points), although those allocated to other COs had a median OGRS score of 21 points (IQR 7.5–54 points).

The ethnic group stated by the respondents at the time of the disposal shows that 95% of those service users allocated to a CF identified as white British compared with 72% of those allocated to other COs (p = 0.02). At disposal, 32% of those service users allocated to a CF identified as having a disability compared with 15% of those allocated to other COs (p = 0.04).

From the reconviction information, 53% of those service users allocated to a CF had a reconviction within 18 months, compared with 21% of those allocated to other COs (p< 0.001). There were no significant differences in the length of time and the number of convictions for the first reconviction within 18 months between the COs allocation groups. This comparison of the reconvictions rates does not take into account the length of follow-up for each respondent, which is a more appropriate analysis and is outlined in Reconvictions.

The questionnaire outcomes scores at follow-up, outlined in Table 15, are limited in being able to show differences between the CO allocation groups, as it does not account for each respondent’s starting point. The change from baseline to follow-up in the questionnaire outcomes is a better assessment on the impact of CFs compared with other COs (Table 17). In general, there were no differences in the change in the questionnaire outcomes between the CO allocations, even with adjustment for baseline.

Further details of the results for specific outcomes are outlined in the following sections.

Reconvictions

The potential confounders for inclusion in the weight to adjust for differences in the CO allocation groups, so as to provide an unbiased estimate of the difference in the risk of reconviction within 18 months, are provided in Table 12. On assessing whether or not these characteristics were associated with reconviction within 18 months or allocation to a CF, five characteristics were identified for inclusion in the IPTW: probation type at baseline; used substances during the past 4 weeks before baseline; smoker at baseline; OGRS score at baseline; and agreement with the ‘Healthy foods are enjoyable’ statement. Apart from probation type, these characteristics were associated with both reconviction within 18 months or allocation to a CF, as indicated by bold text in Table 12. Probation type was found to be associated only with reconviction within 18 months and not allocation to a CF, and so was identified for inclusion in the IPTW.

The unadjusted risk of reconviction within 18 months is over three times higher (95% CI 1.58 to 5.96) in those allocated to CFs than those allocated to other COs (Table 18). With adjustment for differences in the CO allocation groups, so as to provide an unbiased estimate of the difference in the risk of reconviction within 18 months, the risk of reconviction within 18 months is 50% higher in the CF group. However, there is no significant difference in the risk of reconviction between the CO allocation groups (HR 1.50, 95% CI 0.64 to 3.53). This would indicate that any difference in reconviction between the two CO allocation groups results from differences in the characteristics of those service users allocated to a CF and those allocated to other COs.

TABLE 18 - Reoffending rate within 18 months by CO allocation: unadjusted and adjusted by IPTW

Intervention	Association with reconviction within 18 months, HR (95% CI)
	Unadjusted	Adjusted by IPTW
CF
No (n = 80)	1.0	1.0
Yes (n = 40)	3.07 (1.58 to 5.96)	1.5 (0.64 to 3.53)

Inverse probability treatment weight diagnostics and sensitivity analysis

To assess the performance of the IPTW to adjust for differences in the CO allocation groups, it is recommended to review the IPTWs between the two groups, including an assessment of the balance of confounders (Table 19). 97 As expected, the unstabilised IPTW is more variable than the stabilised; therefore, the latter is used in the Cox proportional IPTW-adjusted model. The balance of the stabilised IPTWs between the two groups is appropriate (standardised difference < 0.10) in all variables, expect for probation type. This is confirmed when we review the standardised differences for each variable before and after adjustment by IPTWs (Figure 9). The standardised differences are greatly reduced by the IPTW, except for probation type. If we exclude this variable from the IPTW, the adjusted HR would be 1.6 (95% CI 0.71 to 3.64), which is similar to the IPTW-adjusted HR including probation type (HR 1.50, 95% CI 0.64 to 3.53). As exclusion of this variable does not change the overall results of there being no difference in the 18-month reconviction rate between the two CO allocation groups (with adjustment for the difference between groups), it was decided to leave probation type in the IPTW, as it does make some adjustment for the differences between the two groups.

FIGURE 9.

Standardised difference in IPTW of variables included.

TABLE 19 - Inverse probability treatment weight diagnostics

Variable included in IPTW	CF (all sites, N = 120)		Standardised difference
	No (n = 80)	Yes (n = 40)
Overall unstabilised IPTW
Mean (95% CI)	1.45 (1.34 to 1.57)	3.17 (2.29 to 4.05)
Median (IQR)	1.3 (1.21–1.43)	2.24 (1.5–3.87)
Minimum, maximum	1.05, 4.11	1.05, 16.53
Overall stabilised IPTW
Mean (95% CI)	0.97 (0.89 to 1.04)	1.06 (0.76 to 1.35)
Median (IQR)	0.87 (0.81–0.95)	0.75 (0.5–1.29)
Minimum, maximum	0.7, 2.74	0.35, 5.51
	Mean stabilised IPTW		Standardised difference
Probation type: NPS vs. CRC	0.19	0.04	–0.459
Used substances during the past 4 weeks: yes vs. no	0.34	0.33	–0.025
Smoker: yes vs. no	0.64	0.66	0.032
OGRS score at disposal: point increase	38.07	38.94	0.029
Healthy foods are enjoyable’: compared with strongly agree/agree
Neither agree nor disagree	0.2	0.19	–0.005
Disagree/strongly disagree	0.02	0.03	0.071

Seasonality

There are no differences in the seasonal indicators between the CO allocation groups, at the time of disposal (Table 20), indicating that there were no seasonal differences in allocation. There was no difference in the median temperatures, mean percentage difference from average centimetres of rain and mean percentage difference from average hours of sunshine between those allocated to a CF and other COs. There were no significant differences in CO allocation by season (p = 0.85).

At baseline, the only seasonal indicator that showed a significant association was between the WEMWBS score and rainfall (Table 21). With each point increase in WEMWBS score, the percentage difference from average rainfall is increased by almost 5% (median 4.93%, 95% CI 0.45% to 9.41%). There were no significant differences in the CORE-OM and WEMWBS scores at baseline based on which season the baseline questionnaire was completed. No associations were found between CORE-OM score at baseline and temperature, rainfall or sunshine and between WEMWBS score with temperature or sunshine. At follow-up, there are no seasonal indicators associated with the CORE-OM or WEMWBS scores. Furthermore, our findings show that collection and linkage of seasonality data would be feasible in a larger study.

Discussion of statistical methods

The relatively small number of service users, particularly those at follow-up, should be considered when we review the conclusions of the results from the pilot study. In that context, we should also consider the number of statistical tests and models included in the analysis, that is, the potential problem of multiple testing. To negate multiple testing we have focused the analysis on the estimates of differences and changes (and corresponding 95% CI) rather than relying solely on p-values. This does not fully invalidate the problem, as the 95% CI indicates whether or not there are significant differences or changes at the 5% level and so correspond to the p-values. However, by estimating the differences and changes, we can better understand the impact of any change or difference between the CO allocation groups.

To assist the consideration of the sample sizes involved and their impact on what differences or changes can be estimated, Table 22 shows a post hoc analysis of numbers involved in the four main outcomes. For the three questionnaire outcome measures (CORE-OM, WEMWBS and connectedness to nature scale) we used the mean change score at follow-up [and standard deviation (SD)] by CO allocation group with the number of respondents to calculate the corresponding effect size that this difference in change represents. From this, we can then calculate the power (given α = 0.05) for that effect size and number of respondents, that is, the probability that the test correctly rejects the null hypothesis (no difference) when the alternative hypothesis (difference in groups) is true.

As can be seen, there is insufficient power to estimate a significant difference (at the 5% level) between the CO allocation groups in the change in scores at follow-up for the three questionnaire outcomes.

For the reconviction within 18 months outcome, we used the hazard ratio (both unadjusted and adjusted by IPTW) for the risk of reconviction in the CF group compared with the other CO allocation along with the number of respondents to each to, again, estimate the power. Here we see that we have sufficient respondents (n = 120) to estimate the unadjusted differences in reconviction risk between the two groups (i.e. power > 0.90), but not the adjusted differences in reconviction risk.

Although these calculations indicate that we do not have the number of respondents required to estimate significant differences, they do not take into account that there may not be differences in these outcomes between the CO allocation groups in the first place.

The other major statistical consideration is the adjustment for the decision to allocate to a CF rather than other COs and other differences between these two groups. A variety of methods have been suggested for this adjustment, including propensity scores. 98 Four methods have been outlined in the statistical literature for how to use propensity scores: covariate adjustment, stratification or subclassification, propensity score-matching and the IPTW. We have used IPTW as it has been found that, even in small study samples, propensity score-matching and the IPTW can yield correct estimations of treatment effect (in this study the effect of CFs). 99 However, this recommendation applies if only true confounders are included and those variables related only to the outcome and not the exposure are not included in the propensity score model, which we followed when developing the logistic model to predict the propensity scores.

Consideration is also required in regard to the number of variables included in the models used to produce the propensity weights. The recommended sample size is between 200 and 400 service users or 10–20 cases for each variable included. 100 Therefore, we had adequate power to use propensity score models in this context, that is, 120 service users with five variables.

There is a continuing debate as to which variables should be included in the models to estimate the propensity weights. Results from simulation experiments101 suggest that variables related to outcome but not to treatment should always be included because they decrease the variance of estimates without increasing bias, that is, overfitting is a positive aspect of these models. As part of this process, the diagnostics of the IPTW are important to determine whether or not conditioning on the estimated propensity score has removed observed systematic differences between CF and other CO allocation groups, which we followed as recommended for IPTWs. 97 However, the possibility of residual confounding remains because of unmeasured variables that are associated either with CO allocation decisions or with reconviction rates. Although we included variables that cover the spectrum of characteristics, unmeasured confounders may include service user preferences and compliance with COs.

Summary

Taking into consideration the statistical methods employed, this analysis was able to illustrate the feasibility of comparing outcomes between CO allocation groups with robust and valid adjustment for factors that drive the decision of CO allocation. The use of propensity scores provides a viable method for comparison despite differences in participant characteristics at CFs and comparator sites. These methods are becoming more common, particularly in the evaluation of complex interventions in which it is not, as in probation, possible to randomise.

As discussed in Chapter 3 and highlighted from these results, retention for follow-up was problematic. However, using reconviction data linked from police data without the need to directly follow-up subjects was shown to be a robust outcome with the appropriate adjustment. The weights used in the adjustment were calculated based on relatively few variables (the largest being OGRS score), highlighting that the data required to adjust appropriately for differences between CO allocation and sites are not extensive. Furthermore, the methods for adjustment for CO decision can also be employed to adjust for those lost to follow-up and so could be employed in future studies with a primary outcome of reconviction based on linked records.

Economic evaluation of care farms: a review of the evidence

The impact of care farms on health-related quality of life: a systematic review

Results

The literature search identified 1132 potentially relevant studies. After the removal of duplicates, 948 titles and abstracts were screened, of which 937 studies were excluded. Eleven possibly relevant studies were retrieved for full-text assessment. 106 After full-text review, five studies were included in the analysis. 106 A flow diagram can be found in Figure 10.

FIGURE 10.

Flow diagram of excluded and included studies in HRQoL review.

Cost of crime: a systematic review

This review was published during the duration of the project; please refer to Wright et al. 106 for details.

Results

The literature search yielded 8520 potentially relevant studies. After removal of duplicates, 6265 were identified for screening. In addition, twelve studies were identified through hand-searching of references and in total 6277 titles and abstracts were screened. During full-text review, 21 studies were included in the analysis. 109–129 A PRISMA chart describing the inclusion and exclusion processes can be found in Figure 11.

FIGURE 11.

Flow diagram of excluded and included studies cost of crime review.

Analysis of key parameters for a future cost-effectiveness analysis

Summary

The health economics component of this project aimed to (1) conduct a review of published and grey literature evaluating the impacts of COs and costs of offending to society and health services and (2) identify the most appropriate ways to collect cost data, including health-care resource use by probation service users, cost of reoffending, costs of crime and gains of employability of probation service users in CFs and comparator CO settings and explore the feasibility of measuring cost–utility.

The first review objective was completed with three useful reviews. The review of the economic evaluation of CFs concluded that most cost analyses were rudimentary and, based on only estimated cost savings, there was no study using a cost-effectiveness analysis. The second review on the impact of CFs or green care on HRQoL measures showed the paucity of research that used utility and preference-based instruments to measure benefits, despite the fact that HRQoL is increasingly used as an outcome to measure effectiveness in both medical and social interventions. 107 Finally, the third review of the cost of crime was substantial and gathered evidence on cost of crime that could be easily integrated in a future decision-analysis model evaluating interventions in crime area.

Regarding the second objective, we were able to collect health-care resource used by probation service users in self-reported questionnaires completed at baseline and at the 6-month follow-up; we also surveyed and summarised costs of crime via the third literature review.

We found that the CORE-OM is a suitable outcome measure to generate utilities for probation service users undertaking COs. We found that only 13.9% of the participants requested help or clarification from the RA. The results indicate a large number of missing data at baseline; however, this was mainly attributable to human error as the questionnaire package was missing two pages of the CORE-OM. At follow-up, the number of true missing data were small. Although it is possible that the participants found the questionnaire suitable, several items stood out as being particularly challenging for some participants. The participants requested help from the RA to seek clarification of words, such as overwhelming, warmth and affection, and optimistic. This suggests that if the CORE-OM is administered, it is better to employ a RA so that the participant can ask for any clarifications, which will reduce the rate of missing values. Using the CORE-6D we were able to generate utilities from which QALYs could be generated to undertake a cost–utility analysis.

The total annual cost per site was estimated to be £51,234 in the CF and £65,466 in the comparator site. A limit to the costing of the intervention we faced in the study is that, although we were able to appropriately cost CFs as an intervention from the perspective of the probation trust, the cost of the comparator intervention was based on only one type of CO within centre 1 and we had to assume that all other COs would cost the same amount to implement. It is likely that this assumption may be inappropriate and some COs may be cheaper or more expensive.

Considering the small complete-case data set (n = 53), it is likely that the exploratory cost–utility analysis would lead to incremental cost-effectiveness ratios with a high level of uncertainty and it would therefore be difficult to generalise the results of this study.

An interesting complementary analysis would be a cost-effectiveness analysis using reconviction rates as the outcome of benefit; the sample for this secondary analysis would be larger than the study sample using QALYs as reconviction rates are available for 120 participants (vs. only 53 participants for QALYs). However, we collected health and social services usage costs over only the baseline and 6-month time points and we would have to assume that probation service users pursue the same use of health and social care over 18 consecutive months.

Furthermore, if the type of reoffence had been collected we could have used the results of the systematic cost review to extrapolate the incremental costs of reconviction in the analysis and construct a full decision-analysis model.

Introduction

As outlined in Chapter 1, objective 6, we aimed to develop a logic model based on the experiences of probation service users of serving their CO at a CF. In Chapter 2, we reported the findings from our systematic literature review, in which we developed a theoretically based logic model to explain the mechanisms for how care farming might work for different groups of vulnerable people. Briefly, the development of the logic model involved identifying theoretical concepts as a basis for the model and exploring the qualitative literature to unearth the potential intervention components, mechanisms and process outcomes that were of importance to the various groups.

The plan for this part of the pilot study was to conduct 24 in-depth interviews with probation service users and six with CF staff to further contribute to the development the logic model. However, we took the decision to refocus our qualitative work to explore the relationship between probation services and CFs in depth. One of our early observations during the study was the different approaches taken by the three probation services in their approach to the use of CFs for service users on COs. Two services had positioned CFs within CP requirements (centres 1 and 2) and the third placed the CF as a SA rehabilitation requirement (centre 3). CP is regarded as primarily punitive and is offered to those with limited or no identified rehabilitative needs. Punitive CP orders can include activities such as litter picking, park maintenance and warehouse work. In contrast, SA requirements aim to rehabilitate service users with an identified need to reduce the risk of reoffending. Given the therapeutic aims of care farming to use agricultural and farming practices to support vulnerable groups,4 its approach is more closely aligned to the rehabilitation rather than the punishment of probation service users. We wanted to understand, therefore, if, and how, care farming could support probation service user rehabilitation if it was viewed by probation as part of a punitive order. We considered that exploring this in more depth would be of value to probation services and also to care farming. We therefore aimed to conduct a qualitative study to understand the role of care farming in COs.

Methods

In the light of the changed focus of this qualitative study, we addressed the following specific objectives:

1. to understand the extent to which probation services in other regions in England utilise CFs

2. to understand the perspective of probation service users on serving their CO on a CF

3. to explore, from the perspective of the probation services, the use of CFs within COs, including the allocation process

4. to explore, from the perspective of care farmers, the relationship of the care farmer with probation services and their experiences of supporting probation service users on their farms.

Findings

Development of a logic model on care farming for probation service users

By combining the findings from the current qualitative study with the existing theory on care farming and desistance151 and the logic model developed from the literature review (see Chapter 2) we were able to construct a logic model that was specific for probation service users (Figure 12). This involved fitting the themes from the qualitative findings to the mechanisms identified in all service user logic model (see Figure 3). The new logic model for probation service users differs from those that were constructed as part of the literature review, as it included the views of care farmers rather than just service users. As explained in Chapter 7, we consider this to strengthen the logic model. Only the process and final outcomes measured in the published studies included in the systematic review are shown in Figure 12. Other process and final outcomes displayed in the other logic models are likely to be relevant here, but as yet these have not been measured for probation service users.

FIGURE 12.

Logic model on care farming for probation service users. t, theory based; coloured process outcomes equates to evidence from qualitative literature; black + symbols = qualitative evidence where – is no significant difference and + is significant difference.

Summary

This qualitative study provides contextual insights into probation services and the use of CFs that can support the interpretation of key findings in the pilot study. It also suggests which mechanisms appear to be operating to further aid the development of the logic model for this particular service user group.

Probation officers confirmed a range of factors that influence allocation decisions for service users that are not always recorded in the probation system. These added a layer of complexity in establishing the influence of various confounders on health outcomes. In centre 3, probation staff confirmed that those attending the CF had the most entrenched behaviours and were the most challenging to support. In centre 1, the probation staff worked only with those on unpaid hours with no specific identified needs; however, within this there was recognition that some service users were more challenging to support than others. Probation staff tended to talk about whether or not staff at the various unpaid work placements (particularly charity shops and the comparator site warehouse) could cope with certain service users but they did not mention staff at the CF in this context. This supports informal discussions that were held at the probation services at the time of the pilot study set-up. Both the manager at the comparator site (warehouse) and staff at probation services talked about the types of individuals who would be appropriate. Specifically, as a business, the comparator site had targets to meet for their employer. It, therefore, expressed a preference for individuals who were willing to work with minimal disruption. The CF, on the other hand, did not rely on production to support itself and, therefore, it was not subject to the daily pressures of the warehouse. This meant that the CF could take service users who had more challenging behaviours. These early discussions also indicated that the CF even took in some high-risk (of harm and reoffending) probation service users, even though officially it was allocated as a placement for medium- to low-risk probation service users.

Our qualitative study included care farmers, service users and probation staff. Many of the studies included in our literature review (see Chapter 2) only included service users and, indeed, in studies that included others we used data only from service users to construct our logic model. The benefit of including care farmers here is twofold. We were able to interview only a small number of service users, and many of them were distrustful of authority figures. Although we attempted to interview people in neutral locations, there was little motivation for people to take part outside their unpaid hours or supervision sessions. Thus, the institutional setting of the probation offices became the setting for interviews. This created a natural divide between the interviewers and the service users, who did appear reluctant at times to reveal personal feelings. By also including care farmers as interview participants, we were able to gain insights from one individual about many of the service users. We were aware that care farmers could have referred to only positive changes within service users, but they did mention that some service users were ultimately not suited to the farm and were asked to leave. This was supported by probation staff, who also talked about service users dropping out and not completing their time at the farm. The additional benefit of including care farmers in the interviews was that they discussed changes in individuals that the individuals might not recognise in themselves. The care farmers suggested that service users were not always aware that care farmers were working on their personal development through specific interactions, such as presenting a sheep to them with the implicit aim of breaking down the bravado or even sitting down to eat a hot meal together with the unspoken aim of creating a social environment. This insight and different perspective have enriched the data collected in this study.

The themes that emerged provide an interesting fit with existing theories on desistance and other theories that have been connected with care farming that contributed to the logic models. The five intervention components represented within the logic model were all represented thematically within the interview data. In particular, the farmer and the animals seemed to be core to many of themes and mechanisms suggested within the logic model. For example, the farmer was undoubtedly responsible for creating and maintaining the easy environment that promoted a sense of non-judgement, value and respect. By creating tasks that were novel and interesting, service users felt motivated and satisfied and they gained more skills. These concepts are clearly present within desistance theory. The role of animals on the farms offered a unique insight into how care farming might work in a different way for probation service users compared with other vulnerable groups. No other studies have talked about the role of farm animals in breaking down macho personas. Here it appeared that working with animals enabled the more vulnerable side of service users to be exposed, thus paving the way for more open and honest interactions. Although these almost seemed like pivotal moments to care farmers they were not mentioned by service users. A number of possible explanations exist: the setting and the lack of established relationships limited the extent to which service users were able to open-up during interviews; those who experienced this chose not to be interviewed, perhaps as part of their continued macho persona; or the service users did not recognise this as a key moment or factor in their changing attitudes.

One theme, in particular, stands out as conflicting with the existing logic model for other service user groups. The importance of developing social relationships as a mechanism within the logic model featured strongly across many of the studies within the literature review. Among the probation service users, however, there was a clear message that social contact was to be avoided. There was a desire to just complete the order with minimal interaction, either to avoid negative influences (as appeared to be the case for centre 3) or to just compartmentalise the experience and banish it from memory (as suggested at centre 2). Interaction between care farmers and service users did appear to be more open and service users seemed to value the contact. Care farmers were more positive about developing relationships between service users and, again, this difference in experiences might partly reflect the interview sample.

In contrast to a number of other qualitative studies about care farming, we have identified service users with negative experiences. Most of these experiences relate to CF 2, where the work was considerably less varied, there was a lack of a therapeutic element and it was without a clear care farmer role. However, this further helps to clarify what it is that a CF needs to provide to benefit probation service users.

It was clear from the interviews at centre 3 that the experiences of service users, the care farmer and the probation staff concurred. There was a collective sense that care farming offered structured rehabilitation, meeting the needs of service users in a unique way. The positioning of care farming as a SA was clearly well-thought through, with probation staff and care farmers closely aligned in their understanding of what care farming could offer and who would be suitable to attend. The views of the care farmer in centre 1 appeared to be misaligned with that of the probation services. Whereas it was offered as unpaid work to service users and, therefore, was viewed as more of a punishment, the care farmer talked about non-judgement, social reintegration, safety and nurturing, which are concepts that feature in desistance theories. It is unclear if service users attending the CF here, or even probation staff, considered a need for rehabilitation. Furthermore, as a punishment it is unclear if service users felt less able to draw on the potential benefits of the CF because of the overriding message that it was there to punish rather than to support. Discussions between the care farmer and the probation staff about the positioning of care farming and its role in supporting targeted rehabilitation could be of value to service users and probation staff. In centre 2, although the CF was officially recognised by Care Farming UK, it was clear that the over-riding intention was to support the CF’s charitable efforts to help the homeless, and the farm was a means to do that with free labour from probation service users. Service users in general found this to be a less therapeutic environment with routine site maintenance being the main activity. Probation staff also did not seem to favour the CF over and above any other unpaid work placement. In this respect, apart from the farm being labelled as a ‘CF’, both the service users and probation were aligned in their views about this particular placement.

Care farms aim to offer therapeutic benefits through farming activities. CFs that can clearly demonstrate this should therefore be used as a rehabilitative requirement. Care farmers need a clear understanding about how they can contribute to the process of rehabilitation so that they can communicate to probation services and appropriately position themselves within the right type of CO.

Systematic review evaluating the impacts of care farms

The studies included approximately 980 participants from a range of service user groups. The largest single service user group (albeit spanning a range of conditions within) was those with mental health problems (11 out of 27 studies).

Based on data from 15 qualitative studies (including the mixed-methods study) and information from 15 theories we were able to develop logic models to explain the mechanisms for change for four service user groups: mental ill health, substance misuse, disaffected youth and learning disabilities. As studies reported combined data for mental ill health and substance misuse, there is one logic model for both groups. Although there were some data on older people and individuals with autistic spectrum disorder, they were insufficient to develop a service user-specific logic model. The developed overall model (for all service user groups) highlights the importance of being in a non-judgemental, structured, stimulating and safe environment that allows for reflection, helping individuals to understand themselves, to feel that they belong, are valued and respected, to support them to develop social relationships, to have a sense of achievement, satisfaction and meaningfulness and to learn new skills, to allow for the development, and nurturing, of a new identity if wanted and to help them become physically healthy. These mechanisms are a good fit with a number of theories that have been, up until now, mentioned only in connection with care farming.

Although we ordered mechanisms based on frequency and spread, we do not suggest that any one mechanism is any more important than another at an individual level. However, as a gauge, and based on available data, we observed potential differences in the way CFs work for particular service user groups. For example, a sense of achievement and satisfaction may be more important to the combined substance misuse and mental ill health group and the learning disabilities service user group than the disaffected youth, for whom feeling safe may be a priority. In this last service user group, having the opportunity to reflect seemed to be valued. Although we do not have sufficient data to be able to robustly link the intervention components to the mechanisms, we do tentatively suggest that in the disaffected youth group the emphasis on reflection appears to fit with the greater focus on the ‘setting’ aspect of the intervention. As with the causal pathway between intervention components and mechanisms, the relationship between many of the mechanisms and process outcomes is unclear. For example, ‘understanding the self’ (a mechanism category), which included findings such as increasing self-respect and understanding of tasks that are manageable, was clearly linked to process outcomes relating to self-efficacy and improved confidence. However, the connection of other, seemingly important, mechanisms such as ‘belonging and non-judgement’ to outcomes is less clear. It is likely that many of these mechanisms interact in a way that is not yet understood to influence outcomes. These hidden features of complex interventions are commonly observed within logic models. 5

A key finding within this aspect of the review was that the theoretical concept ‘restorative effects of nature’ was represented by the intervention components (but to a notably lesser extent than ‘the work’ and ‘the farmer’ components) but was not represented at all in the categories of mechanisms. This was somewhat surprising, given that, informally at least, one of the most lauded attributes of care farming is its nature-based approach. Only four findings of the 164 that mapped to the theoretical concept about mental well-being could potentially relate to nature. We suggest that the absence, or near absence, of ‘the restorative effects of nature’ is not a true absence. Rather, nature is the essential component of the intervention that triggers other, more overt, mechanisms to be acted out. Thus, as individuals recall their experiences on the farm, it is primarily the mechanisms promoted through the work and the interactions with the farmer that are at the forefront in thought. It is not clear exactly what individuals were asked about in the qualitative studies, but given that the aims were primarily about exploring the experience and benefits of care farming, more specific questioning about nature would not have been part of the topic guides.

Despite being able to develop the logic models, we could not test the learning disabilities model because of an absence of quantitative data. Although we could test the combined substance misuse and mental ill health group and the disaffected youth logic models, only very limited testing of secondary outcomes was possible with the disaffected youth group. Based on limited quantitative evidence from only two RCTs we did not find any significant positive effects of CFs in improving quality of life. We did find some limited and inconclusive evidence to suggest that care farming can reduce anxiety. For depression, although there appeared to be significant reductions following the intervention, as assessed in UBA studies in the context of a RCT, no significant differences were found between the intervention and the control groups.

For process and secondary outcomes, there were no significant positive effects for self-efficacy and coping (measured in the RCTs) at the end of the intervention. However, a significant improvement in self-efficacy (but not coping) was reported at follow-up. The possibility that there may be some delayed benefits (as with anxiety) for self-efficacy requires confirmation by future studies. A number of UBA studies reported significant improvements in self-esteem, stress, affect, mood and group cohesion at the end of the intervention. However, only stress and affect were measured at follow-up (3 months after the intervention ended) and improvements were not sustained. Most of the primary and secondary outcomes were limited to immediately post intervention with only three (social functioning, eating and appetite, and mental status) reported beyond 6 months. With respect to disaffected youth, there was some suggestion that coping might be improved, but that it had no impact on self-esteem.

By far the most studied service user group in care farming research is people with mental ill health. Although in the UK currently there are more CFs providing support for people with learning difficulties (93% of farms) and autism spectrum disorder (84% of farms) than there are for those with mental ill health (75%),6 only 3 of the 15 qualitative studies explored the experience of care farming for learning disabilities and autistic spectrum disorder. Similarly, disaffected youth, who as a group are supported by around 64% of UK CFs,6 were again the focus of only four studies,25,56,59,79 with two being quantitative. 25,79

Yet even with this, green-care services are used relatively infrequently when compared with traditional approaches. 7 Given that, in the UK at least, CFs are underutilised,6 lack of capacity across the broader green-care service is not the issue. Lack of access may contribute, specifically within more urban areas, which have fewer green spaces, higher deprivation and lack of transport. Lack of understanding and awareness is, however, likely to be a major factor. In some countries, such as Norway, Sweden and the Netherlands, where care farming is well established and research is most active, there is greater integration with statutory services. In Northern Ireland and Republic of Ireland there has been an active push to market care farming directly to commissioners, combined with the establishment of a network of farms supported by European Union funding,152 and this could be an option in areas where engagement has been low.

Maximising recruitment, data collection and follow-up

Throughout the pilot study we tried several different approaches to recruitment, data collection and follow-up. It must be noted, however, that the dissolving of probation trusts in 2014, subsequent privatisation during our study implementation period and the closure of one of our CF sites had significant impacts on the feasibility of identifying, recruiting and following up participants. These adverse circumstances meant that of the 300 participants that we planned to recruit across the CFs and comparators in all sites, we were able to recruit only 134, 50 of whom were allocated to CFs and 84 to comparator locations. Once approached, however, users were willing to consent to participate in the study, with only 21 not wanting to consent. As described in Chapter 3, the main challenges to recruitment were, above all, found in our interactions with the probation system, and many of these were as a result of the impact of probation changes on staff turn over, redundancies and IT and other systems. A stable probation service could facilitate more successful recruitment and, once identified, the high level of service user willingness to complete the questionnaires indicates that recruitment could be feasible, particularly drawing on the lessons learned from this study to maximise recruitment.

In summary, the lessons learned on maximising recruitment and data collection included having a RA seconded from probation services within the centre (centre 3), having a co-investigator working at a senior level within a probation service (centre 1), incentivising users by allowing time spent with the researchers to count towards their unpaid hours (centre 1), including probation service users with multiple requirement orders and recruiting at weekends as well as during the week. The differences in operations in each probation service required bespoke recruitment strategies; however, our lessons on enablers of recruitment may well be of use to other research teams working with probation services.

Our attempts at follow-up were less successful, with only 52% of our participants successfully followed up to complete questionnaires. We tried several approaches. Linking with scheduled probation meetings proved particularly effective, and contacting users by short messaging service (SMS), telephone and letter were of limited effectiveness, even with high-street vouchers offered as incentives. We found systematic differences between those followed up and those lost to follow-up, with those followed up less likely to smoke, and more likely to have a better quality of life (as measured by the CORE-OM but not by the WEMWBS), to find healthy foods enjoyable and to be employed. This is clearly an area of concern of bias in any future study.

In terms of the feasibility of data collection and the differences between the CORE-OM and WEMWBS questionnaires, our experience and data on the questions answered point to greater challenges with the CORE-OM than with the WEMWBS questionnaire. The positive framing of the WEMWBS has been identified as helping questionnaire completion by other studies. 153 Given the benefit of deriving QALYs through use of the transformation of the CORE-OM into the CORE-6D, which is not currently a possibility with the WEMWBS, the CORE-OM is still considered a valuable measure that can feasibly be used among this population. This is reinforced by the fact that only 14% of the participants requested help or clarification from the RA in completing the CORE-OM.

The challenges of following up probation service users to fill in any of the questionnaire measures still present a limitation to feasibility. However, our use of pre-existing routine data, namely reconviction rates from the PNC, did prove to be a feasible approach to gaining sufficient outcome data for this group. Participants consented to, and we were able to access and link, probation service and reconviction data for 90% of respondents. Given the challenges and potential bias in following up probation service users to fill in questionnaires, the feasibility of using existing PNC data to assess reconvictions among our participants at 6 months (or more) after completing their CO is a valuable finding to inform future studies. It should be noted that reducing reoffending is the main performance indicator for the new CRCs. 154 Using the reconviction data would, therefore, be of clear policy relevance to the new probation services, as they consider the services to commission.

We have shown that users are prepared to consent to providing questionnaire data and the vast majority were willing to consent to access to their probation records (nDelius) and PNC data on reconvictions. Furthermore, that with sufficient time and energy built into the study plan to secure the required data sharing agreements, these routine data can be retrieved from the MoJ and from probation services and then linked to an individual’s questionnaire data. Although most of the data could be linked to PNC and nDelius records, a small proportion (11 out of 134) could not. Failure to access certain records may relate to changes that occurred to information systems as part of the privatisation of probation services during the study or also because special permissions (over and above usual informed consent) to access individual records are needed. This would be the case for those who were publicly known or for whom the release of their data would put them at risk of harm. Although we are only speculating on the former, we know that the latter situation does exist. The possibility that individuals provided incorrect personal details at the time of recruitment into the study has been ruled out because we were able to track the progress of COs with the administrative hub, which needed the name and date of birth of the offender in order to check the records. Ultimately, our study suggests that data linkage is possible.

Any future study must take into consideration the time required to build the necessary relationships and agreements with the MoJ and the required number of probation services, and the time required to recruit sufficient participants into any study and establish the necessary agreements to access the routine probation data and reconviction data from the PNC.

Collecting cost data and feasibility of measuring cost–utility

The collecting of cost data from probation services and CFs proved challenging, particularly as our request came during the process of tendering for the new CRCs. In the light of this, we collected cost data from probation services at only one of the sites. However, we were able to access sufficient data on the costs of sessions and this, in combination with the data found during the systematic reviews of the cost of crime and the results of the health service use questionnaires, provided us with sufficient data to run an exploratory cost-effectiveness analysis.

Feasibility of measuring differences in effectiveness between care farms and comparators

Although our systematic review highlights the need for further sufficiently powered studies to assess the effectiveness of CFs in improving quality of life, mental well-being and other outcomes, the experience of this pilot study highlights the challenges of doing so. We identified significant differences between the CFs and between probation services. Service users had different relationships with CFs and different levels of engagement and interaction with nature. Our qualitative findings show this diversity, as the farms that we studied ranged from having very well-developed rehabilitative elements, able to support clients with challenging behaviour and substance abuse issues, to very little support available and limited opportunities for clients to interact with nature. Similarly, probation services showed considerable difference in how CFs were used, with some using CFs as a punitive intervention and others as rehabilitative or a mixture of the two, and in the type of service user allocated to the CFs in each setting, with one farm being allocated users who had a high risk of reoffending, higher levels of substance misuse and low unemployment, although other CFs took lower-risk users. These differences in the users allocated to CFs have not happened as a result of chance but because of a recognition (even if not formally) within probation services, particularly when the focus is on rehabilitation, that the CF may offer a more suitable and beneficial environment for users with greater needs.

Identifying comparators with ‘comparable’ users was challenging, with the comparator in centre 3 taking users with very different special requirements (e.g. to counteract drink driving) compared with those at the CF. Given the continued, and even increased, diversity of approaches within the new CRCs,154 these differences are unlikely to lessen. Although these issues of comparability clearly undermine a standard controlled study design, our statistical methods, particularly the use of propensity scores, detailed in Chapter 4, present a robust way to deal with these differences, allowing comparison between sites with participants with different characteristics.

In particular, it should be noted that the OGRS score, which is a predictor of reoffending combining age, gender and criminal history,155 proved to be a valuable composite measure for adjusting for the differences between those allocated to CFs and comparators. We were able to collect OGRS scores from 90% of the sample through linkage with the nDelius data held by probation. With a suitable sample size and the use of the propensity score method, we believe that a robust and reliable study of the effectiveness of CFs in reducing recidivism and improving offender health could be conducted.

As a pilot study, the study was not designed to detect a difference in quality of life between users at the CFs and comparator sites. It should be emphasised that the small sample size of this pilot study limits any interpretation of the primary and secondary outcomes and cost-effectiveness results. The differences initially seen in the unadjusted reconviction rates between CF and comparator users disappeared once the differences between the two groups had been adjusted for in the analysis. The analysis of confounders highlighted the value of the OGRS score as a composite measure of the factors associated with reoffending. Utilising this score in any future study is vital in order to adjust for the differences between groups, particularly as, in some probation services, those with high OGRS scores are actively sent to CFs.

As all necessary variables were available from the study for both the exploratory statistical and health economic analysis, a larger study could be conducted. Our study has also demonstrated that it is feasible to include an analysis of the impact of seasonality and, although this was not found to drive allocation decisions, this is another element that may potentially moderate effectiveness so needs to be included in any future study.

However, a major challenge to the conduct of such a definitive study in future is the limited number of CFs that are currently contracted by probation services to receive probation service users on COs. This reduction in the number of CFs working with probation may reflect the instability of this period of transition to CRCs and NPS. Once the new system is firmly established, it may be that more CFs are commissioned to work with probation. This would facilitate the recruitment of sufficient participants for a fully powered study in improvements in quality of life and fewer reconvictions in CFs compared with other CO placements. It is encouraging that recent parliamentary assessment of the Transforming Rehabilitation programme emphasises the need for MoJ to overcome the ‘significant barriers to encouraging the promised innovative practice in rehabilitating offenders’ and to realise the ‘full potential of the third sector’ (p6–7154). These recommendations bode well for CFs, which are clearly third-sector organisations that are able to provide innovative solutions with the potential to reduce recidivism.

Understanding community order allocation

Conducting the study in three different probation service regions, and the in-depth qualitative study, helped to shed light on very different approaches within probation services, in particular how CFs are viewed (as a punitive or rehabilitative intervention) and the different approaches taken to allocation of users to the CFs or other setting. These differences are exemplified by the fact that centre 3 used the CF as a SA, allocating users who were unemployed and who had a high risk of reoffending. This was very different in the other two centres, where probation service users had a lower risk of offending and many were employed. In centre 1, the qualitative findings highlighted that the comparator CO sites were often unwilling to take challenging users with complex needs and who had a high risk of reoffending; this was not the same in the CFs, where all users were accepted. The impacts of this are seen in the quantitative data, in which the CF users had significantly higher OGRS scores on allocation to the CF. CFs are clearly a resource for probation in being able manage and support offenders who had a higher-risk profile.

Our analysis of the characteristics of users allocated to the CFs as opposed to the comparator sites highlights significant gender difference in the allocation (4% of those service users sent to CFs were women compared with 44% sent to the comparator sites). This gender bias is reflected across many of the care farming studies found in our review. 25,28,75,77 This may well reflect the preconceived ideas of probation staff about which locations are appropriate for men and women to serve their CO. Probation staff who were interviewed considered CFs to be male dominated and, therefore, not an appropriate environment for women; how the farms became predominantly male in the first instance is not clear. The availability of independent placements (that require minimal supervision from non-probation staff) for lower-risk probation service users may be more appealing to women. In addition, given that CFs seem to take higher-risk probation service users, these placements have an over-representation of men over and above the preponderance of male probation service users. However, this does not mean that women cannot potentially benefit from nature-based interventions as part of their CO and probation services may want to consider this option. We know that, at the time of the study, one CF was looking into the possibility of supporting women through a residential scheme as part of a suspended sentence order and this will be worth further investigation.

Our qualitative findings highlight how probation officers, responsible for allocation particularly in centres 1 and 2 (West Yorkshire and Hertfordshire), rarely considered the potential of CFs as a rehabilitative location in which the beneficial elements of the CF, so closely linked to desistance theory, could reduce recidivism. Instead, the concern was for the order to be completed as smoothly and efficiently as possible. Such an approach may appear to support short-term benefits for the service; however, it may reduce life improvements for individual users and, ultimately, long-term benefits in reducing recidivism. When the CF was seen as aiding rehabilitation, particularly for those seen as having a higher risk of recidivism and more challenging needs, probation officers were more aware of the potential benefits of the farm environment.

Logic model of mechanisms for change among probation service users

A significant contribution of this study is the development of a logic model that elucidates the mechanisms of change leading to potential improvements in health outcomes. The logic model derived from the mixed-methods review guided the analysis of the qualitative data collected from users, care farmers and probationers as part of this study.

The qualitative themes that emerged were well aligned with the all-service user logic model presented in the review. For example, the easy environment identified by the users in our study fits well with the description of the ‘setting’ provided by other CF service user groups, the themes from our participants highlight the important and supportive role of the farmer or supervisor and the interaction with animals and the work fits well with the logic model from the review. The mechanism of ‘personal growth’ was also clearly identified by our participants and those from the synthesised evidence. One area in which there are differences, however, is in developing relationships; the building of friendships and opportunity to spend time in a group was seen as a positive benefit of the CF for all service user groups in the review; however, for the probation service users in our study, many were keen to keep to themselves, serve their order and not build any new friendships with other probation service users that might get them into trouble again at a later date. It may be that for a probation service user attending CFs, building relationships with the farm staff and probation supervisors may be more acceptable and valuable to aid service users build a new non-offending identity. The idea of ‘looking back’ emerged as a distinctive theme among the probation service users, whereas this concept was seen as integral to the setting among the other service users studied in the review. In fact, probation service users rarely identified the farm as a setting for ‘looking back’, and this was more likely to happen on the journey to and from the farm. It may be that the distractive elements of the farm – which is a very important element for mental health CF users – were sufficient to keep probation service users from looking back on their past. According to attention restoration theory,11 this opportunity for space or time-out from anxiety-inducing thoughts is part of mental health recovery, and it may also be that probation service users are able to benefit from this time-out to gain a new perspective on their offending behaviour and its impact on their lives.

Strengths and limitations of the research

The limitations of each component of the study are presented in each corresponding chapter. Here, the overall limitation of conducting this pilot study during a period of major transformation of probation services should be noted. In particular, the changes in staffing within probation, managing relationships with CFs when they faced contractual uncertainty with probations services, the challenges of co-ordinating researchers’ site visits with the availability of service users and the need to establish multiple data sharing agreements with both the old and new organisations significantly affected our ability to recruit, both to the qualitative and quantitative study, the follow-up of participants and the timely implementation of the study. A strength of this study is that, with all these challenges, we have been able to try a number of different mechanisms for recruiting and following up participants, and the learning gained from these is presented throughout the report, particularly in Chapter 3. However, with changing systems it may be that the process would need further adaptions in future.

A further limitation of the study is the limited engagement of service users. Although at the design phase we had hoped for significant engagement in the design, implementation and analysis of the study, we were successful in gaining inputs only into the early stage of designing information sheets and recruitment processes. In part, this was attributable to our strategy of linking into existing service user structures established by probation services. With hindsight, a more productive and rewarding engagement could have been nurtured if we had established our own service user group with a core group of participants who were interested in our study. We could have supported participants in building their understanding of the concept of the study, and care farming, strengthening their capacity for understanding research. Although this would have engaged only a small, and perhaps unrepresentative group, it would have brought valuable user perspectives to bear on our work.

Recommendations for future research

Our study has highlighted that, although challenging, it is feasible to collect the data required to conduct a cost-effectiveness analysis of CFs compared with other COs. We recommend that future studies consider a primary outcome of reconvictions. Data on this outcome can be acquired, with the correct permissions and arrangements, from the MoJ, and our study shows that participant consent to, and linkage of, individual data is feasible. Furthermore, this outcome is of utmost importance to policy-makers and commissioners within probation services.

Our study has also shown that health and social outcomes for probation service users can be collected. The constraints to following up this participant group to complete outcome questionnaires should not be underestimated. In the light of this, we recommend that health and social outcomes are considered as secondary outcomes in future research.

The lack of CFs currently contracted by probation services would make a larger study unfeasible at the present time; however, it may be that this changes as the restructured probation services re-establish contracts with a wider range of providers, particularly those in the third sector, with a view to encouraging innovation and ultimately reducing recidivism.

Our study highlights the challenges of finding ‘comparable’ comparator sites and participants. We recommend that the propensity score analysis methods used in this study be considered and used more extensively in natural experiments to respond to this inevitable lack of comparability found when assessing cost-effectiveness in complex social and public health interventions.

Methodologically, conducting such a study with probation service users as participants has many advantages, not least because while serving their CO they are continually monitored by probation services, enabling some degree of follow-up and, most encouragingly, reconviction data can be used as provide outcome data for 90% of participants. Furthermore, data on confounders can be gathered from probation services, particularly the OGRS score, which can be used within the study analysis.

However, it should also be noted that with increasing emphasis on third-sector provision of health and social services for mental health (including long-term conditions and dementia) that commissioners across the health, social and criminal justice economy are seeking evidence of the effectiveness of CFs. Further studies to determine effectiveness in improving mental and physical health for these client groups would clearly be of value.

Our review found much similarity across the qualitative studies on the mechanism and process outcomes for a range of service user groups attending CFs. We would therefore recommend that conducting further similar qualitative studies would need a clear justification and explorations in new directions. One group that we identified as consistently absent, both in the qualitative and quantitative studies in the review, was those service users who decided to leave the CF. Although these participants may be more challenging to identify and interview, such work would be valuable in providing greater depth of understanding of who may benefit from CFs and may not. For probation service users, this issue is somewhat different, as they must serve their order where they are allocated or they are in breach of their CO and return to court. However, among service user groups that can leave the farm out of choice, a great exploration of the reasons for this would be valuable. This may be of particular relevance to female service users; as CFs are male dominated it may be that some women are de-incentivised to stay. A further area to be explored, which may lend itself to both quantitative – the connectedness-to-nature scales used in this study are valuable here – and qualitative research, is the link between interaction with nature and health. Our review identified how the relationships were rarely explored explicitly within studies, and yet this could provide more detailed information on this characteristic of CFs that is rarely found in other similar interventions.

Recommendations for practice

Although further research is needed to determine the cost-effectiveness of CFs in reducing recidivism, our review and qualitative study highlight the potential of CFs to provide innovative rehabilitation interventions to benefit the well-being of probation users and, ultimately, to reduce recidivism. In the light of this, we recommend that probation services, both CRCs and NPS, build an understanding of the CFs within their catchment areas, identifying their potential to support the rehabilitation of probation service users. Our study highlights that not all CFs are the same, and an understanding of the ethos and aims of the farm is therefore important before considering allocating service users. CFs that can clearly demonstrate that they offer therapeutic benefits through farming activities should be used as a rehabilitative requirement.

Aligning probation service goals and practice to focus on rehabilitation, rather than a purely punitive focus, is clearly a first step in this process. Considering specific rehabilitation needs of individuals and aligning these with the intervention components of local CFs will help ensure that those most likely to benefit gain access to CFs. Our logic model indicates that probation service users who might benefit from learning new skills and participating in meaningful, structured work with a strong role model figure of the farmer, would be particularly suited to a placement on a CF.

In order to ensure that probation services understand the potential benefits of CF, care farmers need a clear understanding of how they can contribute to the process of rehabilitation and an ability and means to communicate this to probation services. This will allow CFs to appropriately position themselves within the right type of CO.

Despite the turbulence within probation services during the time frame of this study, we have benefited from the willingness of probation services and the MoJ to establish data sharing agreements that have enabled us to use existing probation and PNC data. We strongly recommend that requests to access and link individual data continue to be supported within the new probation structures. This will allow further robust research, which can take into consideration important confounders, to understand the effect of different probation interventions on recidivism.

Contributions of authors

Helen Elsey (Lecturer in Public Health) led the design, overall implementation and synthesis of the project, ensuring that milestones were met and the overall objectives of the project were addressed.

Rachel Bragg (Director of Care Farming UK) provided support and guidance throughout the duration of the study.

Marjolein Elings (Scientist and Researcher in Agriculture and Health) provided support and guidance throughout the duration of the study.

Cathy Brennan (Lecturer in Public Health) was an active member of the steering committee, providing guidance throughout.

Tracey Farragher (Statistician and Public Health Epidemiologist) carried out the statistical analysis for the main pilot study.

Sandy Tubeuf (Associate Professor in Health Economics) designed the health economics analysis, led the health economics systematic reviews and performed the cost-effectiveness analysis.

Rochelle Gold (previously Manager of the Research Department at West Yorkshire Probation Trust) provided access to a number of contacts within probation and provided guidance on a number of procedures within probation.

Darren Shickle (Professor in Public Health) was an active member of the steering committee, providing guidance throughout.

Nyantara Wickramasekera (Research Fellow, Public Health) took a lead role in the systematic review on quality of life and conducted the systematic review on cost of reoffending.

Zoe Richardson (Research Fellow, Public Health) undertook recruitment and follow-up during the pilot study and contributed to screening during the literature review on quality of life.

Janet Cade (Epidemiologist, leading the Nutritional Epidemiology Group) was an active member of the steering committee providing guidance throughout.

Jenni Murray (Senior Research Fellow, Public Health) co-ordinated the project and took lead roles in the systematic review on quality of life, quantitative pilot study and the qualitative study.

Publications and presentations

Data sharing statement

All available data can be obtained by contacting the corresponding author.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health and Social Care.

Information sharing agreements with probation services

Initially, creating ISAs with probation services meant developing three ISAs; however, the split in probation into the NPS and the individual CRCs meant that the process became more complex. The ISAs that had been developed with probation services prior to the split remained with the CRCs. For probation service users who were moved to the NPS, a one-off memorandum of understanding was developed with NOMS. In centre 1, access to NPS offender data was no longer possible; therefore, these data were obtained from NOMS. In centre 2, a lack of progress with recruitment and high staff turnover in probation meant that the ISA was not developed. Instead we sought nDelius data directly from NOMS. In centre 3, the IT service at the CRC was able to access NPS offender data and provided nDelius data for all study participants apart from those they were unable to trace (through name and date of birth). See Figure 13 for the data acquisition timeline.

FIGURE 13.

Timeline for data acquisition. The light blue line denotes the point that probation split into CRCs and NPS. Planned targets were n = 300 recruitment (centre 1: CF, n = 60; comparator, n = 60), (centre 3: CF, n = 45; comparator, n = 45; centre 2: CF, n = 45; comparator, n = 45). n = 180 were followed up (40% loss).

Obtaining data from the Police National Computer

Based on the premise that it was a national system, we approached the MoJ to identify if reconviction data could be obtained from one source. This was requested prior to any recruitment. At the time, the MoJ were piloting the ‘DataLab’, which held aggregated offending history data, and it was suggested that this could be used as a source of outcome data for the comparator group. As the DataLab did not hold WEMWBS outcome data and it did not allow access to individual-level data for the identification of potential confounders, we could not use this source. The MoJ were subsequently willing to provide anonymised individual-level data, but were unable to commit the time required to develop a data sharing agreement and so our request was declined. As a consequence, we approached the police services linked with each of the three probation services.

In centre 1, the local probation service held an existing ISA with the local police force. The request for reconviction data therefore went through the local probation service (with which we held an ISA). At the time of request, the probation service had already become the CRC and it did not have access to NPS offender data. Its request to the police (from us) contained only the PNC numbers for CRC probation service users. Therefore, PNC data from this source did not include NPS reconviction data.

At centre 2, a local police disclosure manager contacted us regarding what he understood to be the development of a nationwide ISA between the Department of Health and the police. After clarification about the nature of our study, the development of an ISA was provisionally agreed. Over a number of months, however, no progress was made despite attempts, and as there was lack of progress in recruiting probation service users at this site the decision was taken not to pursue the ISA further.

At centre 3, we tried a number of routes to access reconviction data. The approach used in centre 1 did not work in centre 3. Our contact in probation identified the business development manager as being the appropriate person to liaise with the PNC manager within the local police force. Unfortunately, our request for access to data was declined. We subsequently approached a contact in the police and crime commissioners office; however, this attempt proved fruitless as well. We finally identified an information request service on the police website and our request was approved. As we were able to provide written proof of consent to access criminal record data, the information service considered that an ISA would not be required. The service was also satisfied that data could be transferred safely, as we had acquired an account with the criminal justice secure e-mail system. Data provided at this centre included both NPS and CRC reconviction data.

Having developed a memorandum of understanding with NOMS for the nDelius data, we enquired about the possibility of obtaining reconviction data on NPS probation service users. This time the MoJ agreed to develop an ISA. Data on all service users that could not be found through other sources were subsequently requested from the MoJ. PNC numbers provided by NOMS were used to request reconviction data from the MoJ. Ten per cent of probation service users could not be found on the MoJ system, despite all unique PNC identifiers being checked. We have, however, clarified that future reconviction data could be obtained from MoJ with the development of a single ISA. The time taken to develop the ISAs for all sites is substantial, so early planning is essential.