Evaluating alcohol intoxication management services: the EDARA mixed-methods study

Aims

The overarching aim of this evaluation was to examine and estimate the effect of AIMSs, primarily on EDs and ambulance services but encompassing front-line staff generally and considering the impact on police services where possible. The evaluation is a controlled mixed-methods longitudinal observational study with parallel ethnographic evaluation, comparing services in cities where AIMSs have been implemented with services in cities where AIMSs have not been implemented.

Exploratory analyses

Additional exploratory work examining police activity, referrals from AIMSs to EDs and assaults on NHS staff was also considered. These areas of work are not central to the main evaluation but could provide additional insights that reveal how AIMSs work in the NTE more generally. The data used for these additional analyses are known to have significant limitations: they are not available in all jurisdictions, there is considerable variability in how these data are recorded and the validity of the measures is uncertain.

Police activity

Police officers do not routinely record the time that they invest in managing AAI. Instead, key performance indicators are outcome-oriented, such as the number of assaults recorded. We hypothesise that AIMSs will release police resources otherwise invested in caring for those who are vulnerable as a consequence of AAI, and that greater police presence, facilitated by the implementation of AIMSs, will prevent violence escalating to the extent that severe injury is sustained, therefore reducing assault-related ED attendance. Conversely, increased police presence will increase police ascertainment of violence and therefore increase levels of police-recorded violence. Although these data are not specific to police activity, we include these analyses to explore possible associations between AIMSs and police activity.

Referral to emergency departments from AIMSs

Where possible, we considered the proportion of patients admitted to AIMSs and who subsequently transit to an ED for treatment. This requires that patient identifiable information is available both from AIMSs and EDs and that records can be linked.

Emergency department assaults on staff

A key outcome of AAI in EDs is the effect on staff. ED staff can be exposed to verbal, physical and sexual assault. Although there is a statutory obligation under Reporting of Injuries, Diseases and Dangerous Occurrences Regulations (RIDDOR) to record such occurrences, including near misses, guidelines vary across England and Wales and there is no agreed template for either recording occurrences or capturing whether or not the perpetrator of the assault was under the influence of alcohol. This matter is currently being addressed. 38 This analysis is therefore exploratory given the uncertainty in the underlying data.

Type of study

A controlled mixed-methods longitudinal observational study with parallel ethnographic evaluation.

Sampling frame

A natural experiment comparing areas in which AIMSs have been implemented (or planned and implemented during the project) against matched areas with no provision (control areas, matched using Home Office iQuanta ‘similar families’). 39

Locations vary by work package. For outcome analyses, economic evaluation and surveys of users’ views, we examine six locations in which AIMSs have been implemented and compare these with six matched control locations. The ethnographic study involved two locations in which a service has been implemented and one control location where a service is not envisaged.

To safeguard the anonymity of places and people, locations are referred to as sites, whereby a site refers to the services in the locale in which the AIMSs are established.

Inclusion and exclusion criteria

Cities included in the study were within the jurisdictions of England and Wales. EDs accepting children only (aged < 18 years) were excluded. Further details on site selection and reasons for some areas dropping out of the study are provided in Appendix 1.

Protocol and amendments

The Dissemination and Impact Strategy document is available on the project webpage (www.journalslibrary.nihr.ac.uk/programmes/hsdr/140425/#/; accessed 14 February 2020).

Amendments made to the protocol were as follows:

• 13 December 2015 – final proposal, following review.

• 1 November 2015 – workstream 1 (iv): specified sampling process; researcher Laura Gray replaced Penny Buykx.

• 30 November 2015 – workstream 1 (iii): clarified all six AIMSs; added intent to seek verbal permission to conduct telephone questionnaire. Workstream 2: removed intention to collect NHS numbers from AIMS and ED users, instead to examine mechanisms for linking AIMS data to routine NHS data. Reformatted proposal with front sheet and table of contents. Changes reported to the National Institute for Health Research (NIHR) via the National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre (NETSCC).

• 15 January 2016 – workstream 1 (iii): changed telephone to postal survey. Workstream 1 (iv): sample size revised; addressed comments from initial PPI Review submission (09 December 2015) regarding sensitive AIMS patient approach. Terminology: ‘alcohol treatment centre’ changed to ‘alcohol intoxication management service’.

• 16 January 2016 – removed section on dissemination.

• 17 January 2016 – minor revisions.

• 18 January 2016 – workstream 1 (iii): added reference for walk-in centre survey process. Clarified PPI roles, steering committee members and separate advisory group. Cross-referenced all numbered references. Logos added.

• 28 January 2016 – workstream 1 (i, ii): added sample size, recruitment information and potential case study sites. Workstream 2: added provision of Crime and Disorder Act 199840 for facilitated data access.

• 23 March 2016 – amendments in response to Research Ethics Committee (REC) comments (10 March 2016). Addition of statement on limited confidentiality.

• 12 July 2016 – substantial amendment to REC; addition of AIMS staff obtaining consent from AIMS users for workstream 1 (iii) interviews to facilitate recruitment.

Intervention

The intervention to be evaluated is the AIMS, defined as follows:

• Initiated by local agencies that identify the problems and impacts that AAI is having on services and the wider community.

• Set up and run by multiple agencies (e.g. ambulance services, local hospital trusts, police, local councils and voluntary sector organisations), so staffing can be a mix of nurses, paramedics, police and volunteers.

• Located in buildings or mobile units stationed in or near to areas with a high density of licensed premises.

• Open regularly and at times when AAI is expected to peak (e.g. Friday and Saturday evenings) but may offer additional cover on other evenings such as student nights, evenings of sporting events and national holidays.

• Users are typically referred in by local ambulance, police or third-sector volunteers (e.g. Street Pastors).

• Users would typically be triaged by a suitably qualified person, given basic first aid and water, monitored while they sober up and referred to ED if required.

• Users may be screened for alcohol use problems and offered some form of advice, brief intervention or referral for specialist alcohol support.

• Users are discharged at the point at which they are deemed safe, usually to the care of friends or family, with plans for safe travel home discussed and arranged.

Procedures

Methods

Semistructured interviews were conducted with 19 AIMS users recruited from four AIMSs that provide different models of service delivery (sites A, B, C and D; see Chapter 3). Our intention had been to interview 16–20 participants from across all six AIMSs to achieve maximum variation of experienced care, age, gender and ethnicity before designing the questionnaire. We reduced this from six to four AIMSs because two AIMSs refused to grant permission for this aspect of the research. Researchers attended the AIMS and approached users to obtain consent to interview towards the end of their stay and when they were sufficiently sober to provide consent. It was anticipated that recruitment from such a participant group would be challenging, which was the case, and we accordingly amended the recruitment process so that AIMS staff could approach users on behalf of the research team at times during which project researchers were unable to attend. Written consent and participant contact details were recorded. Interviews were conducted by telephone within 3 days of consent being obtained.

Researchers made up to three attempts to contact participants by telephone and/or text message. Where contact was established, interviews commenced by reconfirming a participant’s identity and consent to be interviewed. Interviews covered views of using the AIMS in terms of access, treatment, experiences with different staff from different services, discharge, and information and advice about alcohol intoxication.

Interviews were transcribed verbatim and entered into the qualitative data analysis software NVivo version 11 (QSR International, Warrington, UK). Framework analysis was used. 46 The framework approach was appropriate because it allows for the exploration of both a priori and emergent issues. The following steps were taken:

• Familiarisation – the first three interview transcripts were read (by PB, AI and AOC).

• Following initial discussion of emerging themes, a thematic framework was developed (by PB, AI and AOC).

• All transcripts were read and coded according to the framework (by AI).

• Parallel interview data collection led to an evolving framework to account for additional emergent themes.

• Discussion of the themes and the connections between them.

• Short case summaries of individual users were prepared to illustrate the diverse types of AIMS users in the sample.

Data from the first nine interviews, alongside field notes taken by researchers during recruitment and ongoing engagement with the PPI panel, were used to develop a two-sided questionnaire. Initial interviews highlighted as important certain aspects of the care that they had received. These are reflected in the survey service rating scales (e.g. safety, communication, care and compassion). Interviewees talked about being escorted to the AIMS by a variety of associates and services and quantification was therefore sought. Furthermore, commonalities between the AIMS and ED survey were sought so that responses could be compared.

Rationale

Removing alcohol-related ED users may improve views of the ED environment for other ED users.

Evidence base

The CQC undertakes ED user surveys nationally around every 2 years. They obtained a 38% response rate for their 2012 survey. 47 A review49 of 16 articles assessing ED patient satisfaction reported average response rates for in-department questionnaires/interviews of 74% (range 51–84%) and for postal surveys of 18% (range 9–25%).

Design

Comparison of users’ views in EDs in six areas with AIMSs and six control areas.

Methods

We undertook a survey of patients who attended EDs in cities where our six AIMSs were located and six control cities.

We developed the ED questionnaire to address the research question. We constructed a set of questions about the ED environment, focusing on issues potentially related to the presence of alcohol-intoxicated ED users or associated visitors. We included the question ‘did you feel threatened by other patients or visitors?’, which is used in the CQC ED patient survey, to allow us to compare our findings with those of other ED surveys. We also used their 11-point scale to measure perceptions of overall experience of the ED. We included a set of questions about more generic aspects of the ED visit so that the questionnaire was relevant to all users. We did not expect to see any differences in these generic items. Our aim was to keep the questionnaire short to help us obtain higher response rates. We discussed the questionnaire with members of the Sheffield Emergency Care Forum (SECF), who endorsed it as acceptable and easy to complete with minor changes to the wording of questions.

We obtained permission from the CQC to replicate their ED survey processes, including using their patient covering letter, reminder letter and survey administration procedures. We followed their process except for sending one reminder rather than two.

In each ED, patients who registered between the hours of 20.00 and 04.00 on Fridays and Saturdays in the month of March 2017 were sent a survey pack. Owing to the withdrawal of one of our AIMSs from the research, users were sampled in August 2018 with a replacement AIMS site (site F) and its control site (site J). Friday and Saturday evenings were selected to reflect operational hours of AIMSs. After our data collection we learnt that site B AIMS was open on Wednesday and Saturday nights, so had to remove our Friday night data from site B and its control site in our analysis (see Chapter 4).

We planned to survey 360 users in each ED, with an expected response rate of 25%, yielding around 90 responses per ED. This would result in a total of around 500 responses across all AIMS areas and equivalent numbers across all control areas. We expected a lower response rate than the 38% obtained by the 2014 CQC ED survey47 because it was focused on a narrow aspect of ED experience and may have lacked salience for some ED users.

Analytic strategy

Regression analysis was used to compare the responses from intervention and control sites while adjusting for potentially important covariates (fixed effects) and accounting for potential clustering of data by site (random effect). We planned to adjust for age and waiting time. We were unable to record actual waiting time so used the participant’s estimate of how long they were in the ED. In addition to including all variables, regardless of significance level, in each model forward regression was also applied where the initial model included only AIMSs, and significant variables were added one at a time until there was no change in the goodness of fit of the model.

To limit the number of hypothesis tests we undertook four comparisons between intervention and control sites:

1. Overall mean service rating score – general questions rating the service were converted to a score ranging from one (very poor) to five (very good) and the mean score calculated across all eight questions. Cronbach’s alpha for the scale was high (α = 0.912). Linear regression was used to estimate the effect of the intervention on mean service rating score. We did not expect to see any differences in this comparison.

2. Perception of feeling threatened – logistic regression was used to estimate the effect of the intervention on the odds of responding to the question about feeling threatened by other patients or visitors with ‘yes’ (definitely or to some extent) rather than ‘no’. This question was used by the CQC in its ED surveys.

3. Overall mean negative experiences score – questions relating to potential negative experiences from alcohol-intoxicated patients and visitors were converted to a score ranging from one (strongly disagree) to five (strongly agree) and the mean score calculated across all seven questions. Cronbach’s alpha for the scale was high (α = 0.892). Linear regression was used to estimate the effect of the intervention on mean score for negative experiences.

4. Overall experience score – linear regression was used to estimate the effect of the intervention on the overall experience score measured on a scale of 0 to 10.

We included one open-ended question: ‘how could the time you spent in the emergency department be improved?’. We searched for comments that included ‘alc’, ‘alcohol’, ‘alcoholic’, ‘drink’, ‘drunk’ or ‘drinker’ as potentially related to impact of patients who had used alcohol. We read these responses and coded them using content analysis.

Data collection

The survey was subject to several changes over time that were relevant to our analysis. Time of attendance was not recorded prior to 2008 and the 2003–2008 surveys contained the question ‘while you were in the emergency department, did you feel bothered or threatened by other patients?’, whereas from 2012 onwards the word ‘bothered’ was removed. We reasoned that ‘bothered’ was not equivalent to ‘threatened’ and, therefore, the results from 2003–2008 and 2012–2016 surveys were not comparable. Because the perception of ‘feeling threatened’ at key times of attendance (Friday and Saturday nights) were key considerations, pre-2012 survey results were not requested, and analysis was limited to the 2012, 2014 and 2016 surveys.

We requested the following data from the CQC data set: trust and ED site identifier (to determine intervention or control groups), age, sex, time and date of ED attendance, reported time spent in ED, response to the ‘feeling threatened’ question and overall experience (0–10 rating). We then used time and date of ED attendance to select surveys from respondents who attended the ED during the hours of AIMS activity (weekend nights).

Analytic strategy

For both analyses (the four study sites only and all AIMSs in England) we used the date of starting operation (defined as being fully operational, open at regular times and not in a pilot phase) to classify whether the survey was undertaken before or after implementation of the AIMS. The same approach was used for control sites, with the start date of the paired intervention site being used to determine whether the survey was before or after implementation.

Regression analysis was undertaken to estimate the effect of AIMS implementation on ED survey responses while adjusting for differences between respondents at intervention and control sites, changes over time and clustering by site. We included age, duration spent in ED, time of arrival, intervention or control site and pre or post intervention as fixed effects, and site and year of survey as random effects. In addition to including all variables regardless of significance level in each model, forward regression was also applied where the initial model included pre/post intervention on its own and significant variables were added one at a time until there was no change in the goodness of fit of the model.

Data collection

Data were collected from routine administrative data sources. All data were anonymised (i.e. no personal details were collected).

Key performance indicators and ED attendance data were requested from NHS Digital. The acute hospital trust nearest to the main area of night-time activity in the town or city was identified and available data from all ED attendances were requested. We selected attendances for analysis if they occurred at intervention sites during the regular times of AIMS operation, or during equivalent times before the AIMSs were implemented, or at control sites during the regular times of operation of the matched intervention site. Only adult attendances (aged > 14 years) were analysed.

We were provided with ED data from 1 April 2009 to 31 March 2016 for all cities, but we chose to use only the data from 16 November 2010 because the ED at site F relocated to a new facility on 17 November 2010 (from 00.00), rendering prior data inconsistent and unreliable.

Hospital Episode Statistics (HES) data were requested from NHS Digital. We requested data for all admissions with International Statistical Classification of Diseases and Related Health Problems, Tenth Revision50 (ICD-10) codes potentially related to alcohol intoxication, using an existing method,51 based on classifications used by the Centers for Disease Control and Prevention and the Office for National Statistics. 52 We restricted admissions to adults admitted between midnight on the night that the AIMSs were open (or would have been open) and midnight on the following night. This allowed for delays in the admission process (it was considered unlikely that a patient avoiding admission by presenting to a service at 20.00 would have been admitted to hospital before midnight) and imprecision in the recording of time of admission.

Ambulance service response time data were requested from each ambulance service that agreed to participate. Ambulance services cover large geographical areas, often including more than one study site. We therefore requested data from calls originating in the administrative post codes of the respective city so that ambulance data related to a specific study site. Again, analysis was restricted to calls relating to adults and that occurred during times of actual or assumed AIMS activity.

AIMS activity

We asked each service to provide anonymised data describing attendances to the service during hours of activity. There was substantial variation in the methods used to record attendances, so data varied accordingly. We explored whether or not it was possible to identify if people attending AIMSs subsequently attended a hospital ED. We also asked a member of staff at each service to complete a short survey describing how clinical governance issues were managed.

Outcomes

The primary outcome was all ED attendances occurring during times of AIMS activity.

Secondary outcomes were:

• Emergency department attendances stratified by age (14–39 years vs. ≥ 40 years), patient group (assault or deliberate self-harm vs. other), diagnosis (poisoning, contusion, laceration or head injury vs. other) and treatment (no investigation or treatment vs. any investigation or treatment). These analyses were planned to determine whether or not AIMSs reduce attendances that are more likely to be related to alcohol intoxication or attendances that result in only observation and/or advice.

• Emergency department key performance indicators – median and 95th percentile total time in the ED (all attendances and stratified by admission vs. discharge); median and 95th percentile time to treatment (all attendances); median and 95th percentile time to initial assessment (ambulance arrivals only); the proportion leaving the ED before being seen for treatment; and the proportion reattending the ED within 7 days.

• Hospital admissions attributable to alcohol intoxication (admissions with primary ICD-10 codes F100, T510, T511, T519, X45, Y15, R780, Y90 or Y91) and hospital admissions attributable to alcohol-related conditions (admissions with primary ICD-10 codes F100, T510, T511, T519, X45, Y15, R780, Y90, Y91, E244, F101, F102, F103, F104, F105, F106, F107, F108, F109, G312, G621, G721, I426, K292, K70, K860, K852 or Q860).

• Ambulance service key performance indicators – proportion of high-priority (category A) calls responded to within 8 minutes; proportion of high-priority (category A) calls responded to within 19 minutes; median time to arrival of ambulance-dispatched health professional for all calls and high-priority calls; proportion of incidents managed without transport to the ED; and proportion of patients who recontacted within 24 hours following treatment and discharge at the scene.

These outcomes were selected because we anticipated that they would be available and reliably recorded for the study period. However, in practice, some measures were not available or reliably recorded. We were therefore unable to examine several planned outcomes:

1. The ED attendance data did not provide enough detail of patient group or diagnosis to allow these stratified analyses.

2. The ED performance data did not accurately report the proportion leaving before being seen or the proportion reattending within 7 days.

3. Ambulance services did not reliably or consistently record if incidents were managed without transport to the ED and were not able to identify recontacts within 24 hours.

4. Ambulance services did not reliably record if calls were potentially related to alcohol intoxication, so we were unable to determine if implementation of the AIMS changed the incidence of such calls compared with control sites.

5. We were unable to obtain data on staff retention, turnover and sick days as human resource records are protected from freedom of information requests.

Furthermore, on undertaking initial descriptive analysis, we found substantial baseline variation between sites for some outcomes or stratification variables (caused by variation in definition or recording of outcomes) that meant that comparative analysis between intervention and control sites was not appropriate. Details are provided alongside the individual analyses. In general, variables used as key performance indicators appeared to be more consistent (at least in the English or Welsh health services) than variables used for other purposes.

Analysis

Secondary analysis

The secondary analyses described here are used to estimate a range of different outcomes. Different analyses are used depending on the type of outcome variable.

Effects of AIMSs on service resource use related to key performance indicators

We examined the effects of key items of resource use on ED attendance rates by including variables on resources measured in the time series models. Resource variables explored included hospital admissions and length of stay for alcohol-related conditions, staffing changes in ED, total time in ED, number of ambulance journeys made and response times, and AIMS attendances. This analysis explains the effect of AIMSs on NHS resources and ambulance and police services.

Costs of services with and without AIMSs

We set out to quantify estimates of the costs of an AIMSs model versus usual care. We undertook a costing exercise to quantify the main components of costs from the six study sites. Set-up and running costs for an AIMS were examined by a standardised costing exercise (a structured request for information on staffing levels and grades, staff time, equipment, consumables, training and overheads) and by examining commissioning documents, including business cases for proposed service developments and actual commissioning contracts. The resource use related to the cost of the ED services during and after set-up of the associated AIMS were obtained from HES ED data and ambulance service dispatch data.

Unit costs for resource use, such as ED attendance and length of stay, were obtained from NHS reference costs55 for HES data and ambulance service data. Staff grades and times had been recorded in the ethnographic analysis and unit costs were obtained from both Agenda for Change56 and Unit Costs for Health and Social Care. 57 Set-up costs were obtained from commissioning documents and business plans. Costs of AIMSs were compared with usual care and results presented for cost per ED admission avoided, and further comparisons compared the costs of types of AIMSs. We also considered costs per avoided ambulance dispatch and examined the consequence of setting up AIMSs at a local level by looking at the displacement costs to local trusts of setting up AIMSs.

Typically, AIMSs are developed through partnerships that can include local government, police, and health-care and other agencies, and set-up costs are likely to be provided across these sectors. In this study the focus is on costs to the health-care sector, though costs to other sectors are important; the main analysis presented here examines costs incurred by the health-care sector. It is expected that the health-care sector will meet some of the costs of the AIMSs; therefore, the sensitivity analysis presents results for scenarios in which the health-care sector meets all the costs and in which the health-care sector meets none of the costs.

Intervention site A

The AIMS at site A opened in September 2012 as a response to the impact on the local ED of patients presenting with AAI. Site A is located in the city centre, 3.5 miles south of the ED and close to an area characterised by many licensed premises and major sporting venues. The space consists of a small seating area, six stretchers in a central treatment area with access to a single-occupancy treatment room, toilets and an additional waiting area. The small seating area is used as a preliminary triage area. Once triaged, patients may be admitted and seated or laid on a bed for observation and treatment; the preference is to keep patients seated to reduce airway risks. The space can be partitioned with screens for privacy and there is seating for friends and family around the periphery of the room. There is a staff room and kitchen that functions as a hub for clinical staff stationed at the service as well as other staff and volunteers working in the NTE, for example local Street Pastors and visiting police officers. The service has capacity for up to seven patients supine and up to 18 patients seated. The service is open from 20.00 to 06.00 on Friday night/Saturday morning and Saturday night/Sunday morning and is also open on other nights/mornings when it is anticipated the NTE will be busy (e.g. during the local university’s student freshers’ fair, pride parades, international sports events, Christmas and New Year). The service is staffed by an emergency nurse practitioner (ENP), one health-care support worker and a senior nurse practitioner. One police officer is stationed at the service during opening hours. The ambulance service has two double-crewed ambulances (four staff) stationed at the service and responds to calls within the city centre. The health board helped to set up the service and secure initial funding from local government for a 1-year pilot of the service. The service’s success resulted in funding from the local government’s Regional Collaboration Fund and it is currently funded through a collaboration between the health board, police, ambulance service and the Substance Misuse Fund. Based on the operational information provided, it is estimated to cost £2170 per session to operate the service.

Intervention site B

The AIMS was established by the Police and Crime Commissioner’s Office in response to the demand placed on the ED and police and ambulance services as a result of alcohol-related assaults and AAI in the NTE. The intention was to divert intoxicated people away from the care of police officers and reduce the burden on the ED while also contributing to making the NTE safer. Site B is located in the city centre, 6 miles south of the ED in an area characterised by the high density of premises licensed for the sale and on-site consumption of alcohol. The AIMS is housed in static cabins and contains toilets and areas designated for seating, staff and clinical activity. It is supported by St John Cymru (Cardiff, Wales): it has an operations vehicle and a rapid response vehicle (RRV). Patients may wait in the seating area until triaged. Within the cabin are two clinical treatment rooms with seating and stretcher beds in which patients may receive treatment and recover. There is a small staff room with kitchen facilities. The St John Cymru vehicle is stationed outside and serves as an operations centre receiving 999 calls to the site B ambulance service. All calls to the ambulance service originating from the city centre are redirected to the AIMS, where a response is co-ordinated. Police officers also use the AIMS to take witness statements when it is appropriate to do so. The AIMS has capacity for up to three patients supine and a further five patients seated. Staff can provide treatment and support for minor injuries and support those who are vulnerable. Patients are discharged once it is safe to do so and contact has been made with friends or family members to ensure a safe transit home. The AIMS is open on Wednesdays nights/Thursday mornings (owing to local university student drinking events), and Saturdays nights/Sunday mornings, between 22.00 and 06.00. They also occasionally operate additional periods when significant events are expected to bring additional numbers into the city (e.g. sporting events and bank holidays). A slightly larger proportion of patients were female (51.1%). The majority (72%) of patients were aged 18–24 years. Overall, 14.8% of patients were conveyed (transferred by ambulance) to the local ED. The AIMS is staffed by up to three St John Cymru staff, including volunteer first aiders, an advanced first aider, a nurse practitioner and a paramedic with access to a RRV. In addition, three students from a local university work as police volunteers to support the AIMS team. A police officer is stationed at the AIMS during opening hours. The Police and Crime Commissioner, Home Office Police Innovation Fund and health board currently fund the AIMS. Based on the operational information provided, it is estimated to cost £1345 per session to operate the service.

Intervention site C

The AIMS opened in November 2014 in a static ambulance. In July 2015 the service moved to premises that could also provide space for patients to recover. The service operates from a building in the city centre, 2.5 miles north of the main hospital, near an area characterised by a number of restaurants and licenced premises. New patients are registered and triaged at the front desk. An open-plan area provides seating for patients, friends and family and functions as the treatment area. Mats and temporary room dividers can be used to partition areas, giving patients an opportunity to lie down. Two treatment beds are present in separate rooms. The AIMS has capacity for approximately 10 patients supine and 10 patients seated. The service is intended to be an alternative pathway for those who have become vulnerable or injured, including those who are exhibiting AAI, in the NTE, but not an alternative to specialist care in the ED. Street marshals collect and escort users into the service by foot or wheelchair. The service provides a space where patients can be monitored until they recover and given treatment for minor injuries and where arrangements can be made for a safe journey home. Typically, patients are seated and offered water and a sick bowl. The service is usually staffed by up to three substance misuse workers, one nurse, two medics provided by Medics UK Ltd (Redcar, UK) and security staff. Two street marshals employed by the city council also operate from the AIMS. We are unable to report on patient demographics, referral routes or discharge destination owing to limitations on the activity data collected. Only monthly data were provided, for April 2016 to June 2017, so we have no data on specific opening dates.

Intervention site F

The AIMS was established in 2010 by the local police and ambulance services to relieve pressure on the ambulance service from AAI on Friday and Saturday nights and as a place of safety. The service is a collection of vehicles: a large mobile St John Ambulance treatment unit, a St John Ambulance ambulance and a police van. The service (as of 19 May 2017) is situated in the city centre, 1 mile south-east of the ED. The St John Ambulance treatment unit provides a space for initial triage, seating for minor injury treatment/waiting and two stretchers for further treatment and recovery. St John Ambulance typically receives calls via the ambulance service and city-linked radio to attend to intoxicated or injured patients and bring them back to the AIMS or provide direct conveyance to ED. The police van provides a space where vulnerable patients can wait in a safe environment. Patients with uncomplicated acute intoxication (initial triage by St John Ambulance) can sit and recover before safe transport home can be arranged. The service has capacity for approximately three patients supine and eight seated. Furthermore, the AIMS provides a service in which people can seek assistance if they are feeling vulnerable or are lost. In addition to medical assistance, people can speak to a police officer in confidence. Typically, patients will receive minor wound care for a brief period and then be offered a place to sit and recover in the police van.

We are unable to report on patient demographics owing to limitations of the activity data provided.

We are unable to report on patients attending the police van who may not have required any medical assistance and so bypassed the treatment unit. Of all patients attending the police van, 8.3% were taken to ED by ambulance and 23% were advised to go to ED.

Typically, the service would be staffed by two to five paid St John Ambulance staff, one St John Ambulance EMT (who assumes the role of clinical lead and takes responsibility for the facility), an emergency transport assistant and up to four other volunteer first aiders (advanced and non-advanced). The police van is run by two police officers. The ambulance trust pays St John Ambulance £1075 per session for the provision of the resources (including staff and non-staff costs). The Police and Crime Commissioner funds their own vehicle and police officers; the value of this funding is undisclosed.

Intervention site G

The AIMS was opened in December 2014 as an extension of an existing Street Pastor scheme and in response to the perceived burden that AAI patients from the NTE were placing on ED. In addition, local Street Pastors raised concerns that most of their time was spent on waiting for ambulances and trying to meet the needs of ill and injured people on the street, which is not seen as their core purpose. The AIMS was designed to help vulnerable people in the NTE as well as reduce the impact on emergency services of those who have consumed excessive amounts of alcohol or have used illicit substances. The local ambulance service did not, unlike other sites, refer patients to the AIMS. The service is in a building in the city centre, 0.5 miles west of the ED and close to licensed premises and fast-food outlets. The space consists of a main reception area with seating around the periphery. A partitioned area contains the office equipment, closed-circuit television (CCTV) and a viewing window to an adjoining recovery room. The recovery room contains three crash mats on the floor. There is a small single-occupancy treatment room with a stretcher bed, medical supplies and an additional small room. There is a kitchen and a toilet for staff and patient use. A large people-carrier-style vehicle is stationed outside. The main reception area is used to initially seat and assess patients and for friends and relatives who are waiting. After initial triage, patients may be taken to the recovery room and laid on crash mats or taken to the treatment room for first aid. The additional room may be used for people who may require privacy and emotional support. Staff, patients, friends and visiting Street Pastors use the main reception area to congregate and await referrals. The vehicle is used by staff to pick up patients from the city centre and to transport patients home. The AIMS has capacity for approximately five patients supine and a further five seated. It is open every Saturday night/Sunday morning, some Friday nights/Saturday mornings, Sunday nights/Monday mornings and bank holidays from 22.00 to 04.00. There was a slightly higher proportion of females (53.6%) using the service. The average age of patients was 24 years (median 21 years) and ranged between 16 and 61 years. Overall, 10% of patients were conveyed (transferred by ambulance) to the local ED. The service is staffed by between four and 10 volunteers, including a senior shift leader and a team of local college students, and has managerial and administrative support from the lead organisation. There is no clinical input. The service is run by a Christian charity and is funded by the Police and Crime Commissioner’s Office, a local church, the city council and a local foundation. Based on the operational information provided, it is estimated to cost £1186 per session to operate.

Intervention site H

The AIMS was established following the death of three young adults in late 2000: two men drowned in a river during a night out drinking and a third youth was found dead in a city centre nightclub as a result of alcohol intoxication. The service has three vehicles that are intended to keep people safe and reduce unnecessary ambulance call-outs by assisting anyone at risk: a bus, a medical unit and a mobile support vehicle (minibus van). All three vehicles park in a layby in the city centre, 5 miles west of the ED. The bus provides a long, narrow space with bench seating to triage and monitor patients. The medical unit contains two stretcher beds, used when administering treatment, and provides additional space for recovery. The mobile support van usually responds to calls from licensed premises and local street-based volunteers for assistance and returns patients to the bus for assessment. The bus has capacity for two supine and three seated patients but will use public seating outside the bus with appropriate staff support as an additional waiting area. There was a slightly higher proportion of women (51.2%) using the bus. Most patients were aged 18–24 years, with an average age of 24.5 years (median 21 years); 6.9% of patients were conveyed by ambulance to the local ED. The bus team consists of St John Ambulance first-aid volunteers and community first responders, who are supported by an ambulance paramedic and a private security guard. A volunteer shift leader from a charity is supported by other shift support volunteers, including a driver who can operate the mobile unit. The bus receives funds from local CCGs, local businesses and private donations and is run by a registered charity. Based on the operational information provided, it is estimated to cost £1106 per session to operate.

Control site M

Alcohol intoxication management services are not planned in this city and the usual pathway is for patients to be taken to ED by the ambulance service. In addition to usual service there is an initiative run by the ambulance service: a paramedic and a police officer travel together in a paramedic car on Friday, Saturday and Sunday nights for a 12-hour shift, responding to issues in the NTE. The initiative runs intermittently because of a lack of police availability. The initiative aims to reduce unnecessary hospital visits via paramedics treating the injured and police de-escalating the situation. The initiative receives calls from both ambulance and police control centres. There are no plans to introduce an AIMS; this is because the NTE is split into six separate areas [student area, café culture area, real ale trail, party zone, LGBT (lesbian, gay, bisexual and transgender) area and lower-income/non-workers area] and there is not the concentration of drinkers found in other cities in which the NTE is centralised. Other reasons include legality, the responsibility involved in caring for people and lack of funding.

Church-led and similar organisations are prominent. Street Angels provide help for the intoxicated and this includes cycle responders: mobile paramedics on bicycles who carry a defibrillator as well as a larger first-aid kit than those carried by Street Angels. Each shift there are typically six Street Angels and two cycle responders. The cycle responders are a Street Angels initiative with additional support from the NHS. The local police provide additional funding and storage for equipment. In addition, a night church opens once per month and provides space for drinkers to take time out and reflect. Staff there can look after intoxicated people and perform basic first aid, although this is not the night church’s primary function.

Street marshals operated from 2008 to 2015 but were withdrawn because of a lack of funding. They were originally taxi marshals and the role developed; they operated every Friday and Saturday, then Saturdays only (usually the busiest weeknights). Street marshals provided pastoral support in the NTE, collected glass from the streets, managed taxi ranks and performed other activities as appropriate. Their role developed and they became a conduit between the intoxicated and the Street Angels. Over a 4-year period, the Street Angels helped > 3000 people directly, most assistance involving first aid. Street Angels are volunteers who receive training in emergency first aid, provide bottles of water and flip flops and carry first-aid kits. They help drinkers to rehydrate, sober up and get a taxi (if they are sober enough) or locate a partner or friend to take the individual home safely. Street Angels try not to send people away in ambulances if they need only sobering up and have no other health issues. If they do feel that they need to call an ambulance, they will wait with the individual for 2–3 hours or more.

Control site L

There are no plans for AIMSs in this city. There are typically 10 police constables and one police sergeant on patrol on Fridays and Saturdays between 13.00 and 06.00. In addition to the usual services, Street Pastors also offer support. In matters of AAI, police will defer to Street Pastors, who stay with the intoxicated individuals and make certain they are safe and that someone can collect them. This releases police officers back to duty. Police-led AIMSs were trialled twice in a church in the city centre, once during a major sporting event and once over the Christmas period. Numbers attending were approximately two or three patients per night. People were instead routed to the local ED, unless the police directed them or took them to the AIMS. The pilot AIMSs were staffed by the local locality manager, one police officer, one advanced paramedic practitioner, one health-care assistant and one nurse practitioner. There were two RRVs and the AIMSs were run in partnership with St John Ambulance, who provided two staffed support vehicles for transportation.

Control site N

The police are investigating options for an AIMS in the city centre. This would be a multiagency setting with volunteers, capable guardians and some form of health service involvement. Currently, Street Pastors help look after intoxicated persons and an ambulance is called if necessary, but often police officers end up taking people to ED. A service was trialled on Millennium Eve in the town hall, staffed by ED doctors and nurses, but this ended because of low demand. A police–ambulance service was trialled from 2009: an ambulance fast-response car staffed by a paramedic and police officer. The ‘polambs’ (ambulances with police officers on board) ran for 4 years but ended as there was nowhere to privately assess the patients. A Dynamic Emergency Care Centre (DECC) was trialled: a partnership between the local council and the police. This was a static unit with an assessment area inside. Referrals to the DECC were mostly by ambulance. The DECC was funded by the local CCG and staffed by two first aiders from St John Ambulance or the British Red Cross (London, UK), one paramedic and one police community support officer. The DECC operated only on key dates (e.g. New Year’s Eve) because there was no call for it on some weekends and funding was limited. Otherwise, AAI was handled by the usual services with additional support from Street Pastors.

Control site K

A clinically-led AIMS is cost prohibitive, with significant resource implications and uncertainty over CQC compliance. Street Pastors have had a ‘Safe Space’ since 2010, where shelter can be given to vulnerable people (not only intoxicated individuals). In addition, there is a service similar to Street Pastors, currently without a base, that uses non-medical intervention or ‘very light’ first aid. Polambs were used for many years but ceased operation because of budget pressures on the ambulance service and safety concerns about solo staffing. Ambulance crews do not use the Street Pastor Safe Space.

Control site J

Proposals for AIMSs, including a mobile bus that provides support and somewhere for people to rest rather than attend ED, were considered but judged financially inviable, and concerns were raised over the level of medical qualifications required. There is a Street Pastor team that provides coverage. There used to be a city centre triage unit, which was a joint venture between the ambulance service and police: an ambulance with police in attendance. This ceased operation because of lack of demand. An AIMS would be useful because it would take pressure off the police and ED, but funding and resources are an issue. The police have a duty of care to the individuals they engage with, so if those individuals are unable to care for themselves, the police need to find someone who can look after them. Street Pastors can stay with the individual, but this is not ideal as the Street Pastors have no base to go back to, so must stay on the street.

Control site O

A mobile medical support bus was launched in 2012 by the local NHS trust in collaboration with the community safety partnership. The vehicle was provided by St John Ambulance and staffed by a paramedic, St John Ambulance first aiders and trained volunteers from Street Pastors. A similar service, operated by Street Pastors, ran from October 2015 to March 2016. The bus was well used but closed in 2013 because of financial constraints. Located in the city centre, it was accessible and well managed. It was open on Friday and Saturday nights, the two busiest nights in the city centre, and freed up the police and ambulance services. Currently, intoxicated people are helped by the police, ambulance patrols and Street Pastors. Owing to heavy demands on ambulance resources, police officers occasionally transport patients to the ED. The bus was larger than an ambulance, with space for people to sit down, lie down and receive treatment and there was a private space at the back.

Emergency department attendances

Analysis

We planned to fit ARIMA models to daily ED attendances to compare AIMSs with usual care and different levels of AIMSs and their impact on ED attendances. However, initial explorations using these models on daily data revealed large variability in the data and, subsequently, models were unable to identify any meaningful patterns. Alternative ARIMA models were explored, including models fitted to weekly data, monthly data and Saturday data only, but these would not allow identification of when AIMSs were open. It was therefore decided not to use ARIMA owing to limits on interpretability and, instead, generalised linear models were used. To determine the main effect of AIMSs on ED attendances, all intervention cities and all control cities were included in a single model. Data were restricted to only those days of the week when AIMSs were operational and to the hours they were typically open, and controlled for a linear effect of time to adjust for the known increase in ED attendance over time. Data were realised as a panel and a random-intercept fixed-slope multilevel model was applied, drawing the cities together under the assumption that they are each drawn from a larger population of possible cities. Clustering was by city, to adjust for heteroscedasticity and within panel autocorrelation, but sensitivity analyses were also conducted, clustering on pairs of cities (control–intervention) to better realise the design of the evaluation. No notable differences were observed between models. In the interests of transparency, we have reported all analyses undertaken in Appendix 6.

Consistent with the descriptive statistics (see Table 3), which show a relative decrease, or difference-in-difference, in AIMS sites of 3.85 attendances compared with control sites (excluding site H), AIMSs were associated with a relative decrease in attendance in ED [β = –4.99, 95% confidence interval (CI) –9.44 to –0.54; p < 0.05]. Whereas both control sites and intervention sites exhibited an increase in ED attendance, the increase for intervention sites was less than for control sites. An increase in attendance across all hospitals was observed over time (β = 0.005, 95% CI 0.002 to 0.007; p < 0.001), but this trend did not notably differ across control and intervention groups (z = 0.42; p < 0.34). The model was repeated dropping site D and site E. This model yielded a similar main effect of the intervention (β = –5.30, 95% CI –9.62 to –0.62; p < 0.05). However, stating that this effect is common across all AIMSs in the analyses is not appropriate, particularly given the variability in AIMS configurations (see Chapter 3). Referring to Table 3, this effect is likely to be attributable to a subset of AIMSs.

Interrupted time series analyses were therefore used to understand the underlying effect of AIMSs on a city-by-city basis. These models compare linear trends pre and post intervention. They identify any initial change in the outcome at the time of implementation and any net trend subsequent to implementation.

Models were realised in two ways. First, all dates when an AIMS was open were identified (Table 4) and compared with baseline, controlling for notable dates (e.g. News Year’s Eve, Halloween) and days of the week. Second, the models were restricted to the days when AIMSs were typically open (most commonly Friday and Saturday; Table 5), in an intention-to-treat analysis, and without controlling for notable dates and days of the week (see Table 5). The reason for the sensitivity analyses was that the inclusion of day of week and other notable events risked over-identification in the regression models, particularly given the limited pre- or post-AIMSs data available in some models. Comparisons between intervention cities and control cities, and pooled control sites, were planned; however, these models exhibited significant baseline imbalance, a matter that substantially increases the risk of both type I and type II errors. 64,65

TABLE 4 - Interrupted time series analyses: ED attendances, single city, all opening dates, controlling for day of week and notable dates, with total days pre and post implementation

*p < 0.1, ***p < 0.01.

AIMS site	Effect	Trend	Time (days)
			Before	After
H			0	2019
A	–4.89***	–0.014***	728	1290
B	–2.41*	0.008*	1460	557
G	2.22	–0.002	1533	478
F	4.05	0.004	1716	295
C	1.29*	–0.011	1533	485
D	–6.02	–0.016	1680	331
E	–5.24	–0.106	105	1913

TABLE 5 - Interrupted time series analyses: ED attendance, single city, with total days pre and post implementation

*p < 0.1, ***p < 0.01, ****p < 0.001.

AIMS site	Effect	Trend	Time (days)
			Before	After
H			0	2019
A	–4.93***	–0.013****	728	1290
B	–2.63	0.008*	1460	557
G			1533	478
F	3.32	0.008	1716	295
C	1.12	–0.010	1533	485
D			1680	331
E	–0.22	0.006****	105	1913

Interrupted time series analysis models including control cities exhibited imbalance between control and intervention cities at baseline, whether the control was the matched site or pooled control sites, and the days available for analysis were unbalanced. Notably, for sites H, F, D and E, ITSA did not include a full calendar year either pre or post intervention, suggesting that any seasonality would not be balanced. Because the AIMS at site H opened in 2001, earlier than the ED attendance data available, it cannot be included in these models. All models corrected for serial autocorrelation. In paired analyses only, the site B–site L pair did not exhibit imbalance at baseline and this model yielded an insignificant effect of AIMSs and post-implementation trend.

Tables 4 and 5 (full model details are available in Appendix 6) provide mostly consistent results, suggesting that the only city demonstrating an initial effect on ED attendances (as opposed to a trend) was site A (Figure 2). Sites G, H and D do not have enough data for the reduced-form single-city analyses. For site H there is no baseline data and sites G and D opened sporadically over the follow-up period. Site A shows a significant initial reduction in ED attendances and a post-implementation trend suggesting that ED attendances would be expected to fall further (see Figure 2), consistent with the view that AIMS effects take time to bed in. This was the only effect that was consistent across single-city analyses (pre and post implementation with no control). Restricting analyses to the total number of ED patients aged between 14 and 39 years reduced the effect observed at site A (–2.71; p < 0.05) with a non-significant post-implementation trend and raised the effect at site B to significance (–2.64; p < 0.01) with a positive post-implementation trend (0.007; p < 0.05), suggesting that the effect of the AIMS on ED attendance numbers may have decreased over time.

FIGURE 2.

Site A ED attendances (dark blue dots) compared with site N (control) ED attendances (light blue dots). Vertical dotted line denotes service implementation.

Emergency department key performance indicators

Total time in ED (time from initial attendance to discharge) is a key performance indicator for EDs. From the individual-level data available we derived a binary indicator of whether or not each patient was seen, treated and discharged within 4 hours. Table 6 presents the percentage of patients with such episode durations of < 4 hours. Over time, performance in EDs declined.

Pooled analyses were not undertaken because of inconsistencies between English and Welsh data. However, as English data were reported consistently, we undertook a pooled analysis limited to English sites; this showed that intervention sites were more likely, but not significantly, to see patients in the ED within the 4-hour target when an AIMS was open. The marginal effect was 0.015 (95% CI –0.039 to 0.069; p = 0.581) across all sites, 0.005 (95% CI –0.044 to 0.053; p = 0.848) when analysis used the original sites and 0.017 (95% CI –0.062 to 0.095; p = 0.678) when analysis used the replacement sites. In single-city analyses no effect was observed for sites H, A and G. A fall in the number seen within 4 hours was observed for sites F and E. An increase in the number seen within 4 hours was observed for sites B, C and D. Additional analyses were planned for time to treatment and time to initial assessment. However, data were too inconsistent to facilitate analysis, possibly attributable to variable recording practices across EDs.

Ambulance service key performance indicators

We received ambulance data for sites F, C and O from 1 January 2010 to 31 December 2016, for site H from 1 July 2010 to 31 December 2016, for sites K and N from 1 January 2012 to 31 December 2016, for sites A, B and L from 1 January 2012 to 1 December 2016, for sites J and M from 5 March 2012 to 31 March 2017 and for site D from 1 April 2012 to 31 December 2016. The ambulance services covering sites E and G were unable to provide data; we used data from 16 November 2010 to 31 March 2016 where available. As with the other analyses, site H opened before the start of data availability, so there are no pre-intervention data for this site or its matched control.

From 1 October 2015 the Welsh Ambulance Service changed call categorisation to a three-level red–amber–green system. 66 The high-priority (red) calls were more strictly defined and are therefore not comparable to English or previous Welsh high-priority calls. Prior to this, the English and Welsh services used broadly similar categories, although there are inconsistencies in the way that they are/were operationalised. For example, in Wales the clock starts when the chief complaint has been established, whereas in England the clock starts for the highest priority (‘RED1’) calls when the call is answered.

Discussion

Overall, data quality and model specification limit opportunities to report on AIMS effectiveness. Provisional results suggest some support for AIMSs influencing key performance indicators on aggregate ambulance data. Site A provides the strongest evidence for an effect of AIMSs on ED attendances. Pooled analyses suggest that AIMSs affect urgent ambulance calls, which are responded to within 19 minutes. Performance against all key performance indicators deteriorated over the study and sites varied considerably.

Much of the analytic obstacles to reporting on effectiveness can be resolved with enough AIMSs, a longer time series and a greater number in the counterfactual. However, the number of AIMSs in England and Wales, specifically those opening on regular days, is limited and many have been functional only for a short period of time, the consequences of which are compounded by the varying quality of routine data (primarily due to changes in recording standards).

Set-up and running costs

The set-up and running costs were collected from six participating AIMSs from which data were available (sites H, A, B, G, F and C) using a standardised costing exercise. An inventory of resources was designed by the team and piloted using information from site A (see Appendix 7). Managers of AIMSs were contacted and asked to provide details of their budget and running costs. Information requested included details on staffing, premises, consumables, equipment and software licensing. For each item we asked for details of which sector met the costs (e.g. NHS, ambulance service, police, charity, voluntary).

No AIMSs provided details on the cost of setting up the service (see Appendix 8); therefore, the following assumptions were made as to what would happen when an AIMS was set up. The decision to set up an AIMS consisted of an initial meeting lasting 2 hours, attended by a hospital consultant, paramedic, police officer, council representative and NHS employee from finance. After the meeting, a manager would be appointed to put forward the business case for an AIMS and oversee its set-up. Appendix 8 provides details of assumptions related to the initial meeting and unit cost source. It was assumed that the manager would be employed for 6 months at 50% full-time equivalent prior to the opening of the AIMS, and management salary costs were assumed to be band 8a of Agenda for Change56 (£10,107). These assumptions were discussed and agreed by the study team. Total staff costs were estimated to be £588.

Most AIMSs are run by NHS staff; therefore, no additional staff training costs were included at this stage. However, site G’s AIMS is run by volunteers and if most of the staff are volunteers then training costs need to be included. Training given to volunteers at the site G service consists of an initial induction and further training if the volunteer wants to be a medical assessor. The induction for drivers, communicators and café befrienders lasts for 2 hours. Medical assessor volunteers receive 6 days’ accredited first aid training and also attend a safeguarding session. The cost of this training is £720 per person, with the volunteer asked to contribute £95 up front and £15 per calendar month for 1 year. Therefore, the AIMS would pay £445 to train each volunteer and the volunteer would pay £275. Hence, if there were to be 10 volunteers the initial cost of training would be £4450.

Running costs were provided at varying levels of detail from each AIMS; these costs are detailed as staff costs, premises-related costs and other costs (see Appendix 7).

Emergency department attendances

The method for estimating the reduction in ED attendances was described in Effectiveness, Emergency department attendances. The cost of an ED admission was taken from NHS reference costs55,67 averaged across all admissions and is £148.36.

Ambulance costs

The proportion of cases that arrived in ED via ambulance was obtained from the ED data for each city. All these cases are assumed to involve ‘see, treat and convey’, with a unit cost of £247.00. The average cost of ED attendance was estimated as the cost of an ED attendance plus the proportion arriving by ambulance multiplied by the cost of conveyance by ambulance.

Inpatient admissions

The inpatient data analysed were for the period 2009–16. A new set of reference costs are released each year based on the latest definitions of Healthcare Resource Group (HRG) codes. HRG codes evolve over time, with some codes being discontinued as their definition changes and new codes being added. For example, what was AA03Z (‘Intracranial procedures from trauma with minor diagnosis’) in 2009/10 became AA03A and AA03B (‘Intracranial procedures for trauma with diagnosis of head injury/skull fracture with and without complications and comorbidities’) for 2010/11. These definitions were reclassified again in 2012/13. These changes could create additional noise in the cost data because some of the variation in cost could be due to how the HRG code was defined. Therefore, different methods of costing were used to estimate the cost for each episode. The first method costed the HRG code recorded in the data set; the second method estimated a HRG code based on the 2015/16 NHS reference cost definitions. 67 The percentage brought in as inpatients is defined following the HES ED data and is the percentage of those seen when an AIMS was regularly open who were coded as admitted as inpatient (for English data this was those coded 1 or 2 for the ‘aeattenddisp’ variable and for Welsh data coded 01 or 02 for the ‘outcomeofattendancecode’ variable).

Method 1: inflated costs

A data set was constructed that contained all HRG codes between 2008/09 and 2015/16 (the latest reference costs available); reference costs for 2012/13 were excluded because they were split by specialty and not recorded as a combined total. If the cost was not available for the latest NHS reference costs (2015–16)67 then costs were inflated using the Hospital and Community Health Services Index. 57 A separate data set contained the latest trim points for each HRG code. Each unit cost is based on a defined length of stay for each HRG code (the trim point); if the episode length exceeds the trim point then additional days are costed based on the excess bed-day cost for the HRG code. The inpatient data set was then merged with the unit cost data set and trim point data set. From this an episode cost was estimated based on the unit cost of the HRG code and any excess bed-days.

Method 2: 2015/16 Healthcare Resource Group codes

To make HRG codes comparable by year and NHS trust, the 2015/16 reference cost grouper was used to generate a HRG code. The provider spell number was also not provided, so HRG codes generated by the reference cost grouper related only to the episode level, not to the spell level. After the HRG codes had been generated, the unit cost could be assigned to each episode from the NHS Reference Costs 2015–16. 67 Excess bed-days were costed the same way as in method 1.

Creating inpatient spells from episodes

The York Centre for Health Economics method68 was used to create inpatient spells from inpatient episodes. Spells were created based on patient identifier, provider code, admission date, discharge method, episode start date and episode end date. A spell cost was then estimated based on the combined cost of all episodes in the spell. This method can only create spells for patients who remain in the same hospital (or in the same NHS trust for the English data); this is known as a provider spell. Analysis was limited to alcohol-related admissions (primary diagnosis codes: F100, T510, T511, T519, X45, Y15, R780, Y90, Y91, E244, F101, F102, F103, F104, F105, F106, F107, F108, F109, G312, G621, G721, I426, K292, K70, K860, K852 and Q860).

Analysis

Duplicates were removed from both ED and inpatient data prior to analysis. The primary analysis used ED and ambulance costs. A sensitivity analysis included inpatient admission costs for those patients who were admitted with an alcohol-related code. The average cost per ED attendance avoided was estimated for each city and overall. The number of ED attendances avoided per night for an AIMS to be cost-effective was calculated by dividing the average service cost per session by the average cost per ED attendance. If the number is less than the reduction in ED attendances, then the AIMS is cost saving. Unit costs were obtained for the 2016/17 financial year; any unit costs that were based on a financial year prior to 2015 were inflated to the latest price levels using the Hospital and Community Health Services Index. 69 Sites D and E dropped out of the study and did not provide AIMS set-up or running costs so are not included in this analysis.

Results

Sensitivity analysis

Cost of conveyance by ambulance was included in a sensitivity analysis (Table 8). Including ambulance conveyance resulted in the average cost per ED attendance ranging from £256 to £276 per attendance. AIMSs at sites H, A, B and G reduced ED attendances and produced some cost savings to the NHS. However, for AIMSs to break-even they would need to divert between 4.0 and 8.6 ED attendances per session if the only cost savings related to reduced ED attendances.

Costs of admissions were included in a sensitivity analysis (Table 9). The proportion of admissions that were alcohol attributable (from HES data for admissions that were alcohol attributable seen on days that the AIMS was regularly open; there was no time stamp for time of admission) was obtained and ranged from 21% at site A to almost 50% at sites G and C. Between 15% and 27% of admissions were either fully or partially due to alcohol. Including these costs in the cost of attendance increased the ED costs from £420 (site A) to £628 (site G).

Using method 1 (inflate costs) to estimate inpatient costs resulted in an average cost per ED attendance of £521.60, ranging from £419.22 at site A to £621.39 at site G. The number of attendances needed for AIMSs to break-even ranged from 1.79 to 5.40, with only site A observing a greater reduction in ED attendances (n = –6.39) per night. Using method 2 (HRG costs from 2015–16) to estimate inpatient costs resulted in an average cost per ED attendance of £473.10, ranging from £366.27 at site A to £578.74 at site G. The number of attendances needed for AIMSs to break-even ranged from 2.07 to 6.18, with only site A observing a greater reduction in ED attendances (n = –6.39) per night.

Discussion

Alcohol intoxication management services would need to divert an average of 11 attendances per session (ranging from 7.3 to 15.3 attendances per session depending on service configuration) from the ED to break-even from the perspective of the ED. Sensitivity analysis included ambulance and inpatient admission costs and suggested that AIMSs may break-even if 3.5 patients (ranging from 2.1 to 6.2 depending on service configuration) are diverted. It is interesting to note that the site G volunteer-led service was not the least expensive service. The most expensive service, site A’s, was led by nurse practitioners.

Overall, AIMSs received, on average, 7.3 patients per session. Analysis of individual AIMSs found that attendance varied considerably, from an average of 2.7 to 11.8 patients per session. Given evidence that AIMSs attract unmet need that would not otherwise be expected to require attendance in ED (see Chapter 6), it is reasonable to assume that the average attendance at AIMSs does not reflect the total number of patients who are diverted away from ED. Therefore, it is unlikely that AIMSs are cost-effective from the perspective of the ED.

Analyses are hindered in several ways. Key performance indicators are noisy and, therefore, accurately identifying what is likely to be a small effect of AIMSs in highly variable ED and ambulance data is difficult. Key performance indicators are prone to seasonality, for example varying in frequency weekly (e.g. increases at weekends), monthly (e.g. pay day) and annually (e.g. summer compared with winter). Without balance before and after implementation, any observed effect could be due to characteristics of the time series. Underlying trends, if not accounted for, can further obscure estimates. Given that the expected effect of AIMSs is relatively small, models can be subject to overfitting, where noise drives the results more than the effect of AIMSs themselves. These issues are particularly prominent where sample sizes are reduced, such as in the single-city analyses.

Although there is agreement across analyses suggesting that AIMSs significantly affect ED attendance numbers and ambulance response times, it is similarly clear that such effects cannot be attributed to each AIMS. Site G, for example, has no formal relationship with the local ambulance service and it is not feasible to attribute changes in ambulance response times to the effect of site G. The high variability in AIMS design therefore precludes opportunities for a general statement on effectiveness and cost-effectiveness. However, a model led by a nurse practitioner incurs the greater cost but is also more likely to successfully divert patients away from the emergency health-care system.

Assaults on emergency department staff

Freedom of information requests to all participating EDs sought data concerning assaults on staff working in ED. These data were collected in a variety of formats and for a variety of time periods across sites. To make comparisons, all data were aggregated to quarterly time periods and all assault categories combined. The latter was necessary because some hospitals, for example, used several defined categories (e.g. sexual, physical, verbal) to describe events. Mean number of assaults by control site and intervention site is presented in Figure 3. This figure suggests that EDs with a locally implemented AIMS maintained a more modest number of assaults on staff than EDs without a locally implemented AIMS. To assess this relationship statistically, a simple random effects negative binomial regression was used, including time as a covariate, regressing the status of whether or not an AIMS was operational (0 or 1) in the city for all cities supplying data (n = 13) and the sum of all forms of assault as the outcome variable. The model performed better than chance (Wald χ² = 11.77; p < 0.01), with the incident rate ratio (IRR) suggesting that EDs with operational AIMSs have fewer assaults than control sites (IRR 0.86, 95% CI 0.79 to 0.94). These data must be viewed with some scepticism; they are aggregated and, therefore, cannot distinguish between evenings when AIMSs are operational from other times of day, nor whether assaults were alcohol related or due to other reasons.

FIGURE 3.

Mean total (with standard error bars) number of assaults on ED staff, by quarter, year and groups, by cities with AIMSs (intervention) and without (control).

Police-recorded violence

Data concerning police-recorded violence was accessed from the UK Police open data portal (https://data.police.uk; accessed 11 February 2020) and grouped by city: nine cities known to have AIMSs and six control cities. Data recorded total number of sexual assaults, total number of assaults resulting in injury and total number of assaults resulting in no injury, by time (year and quarter). Figure 4 presents the total count of all assaults by time, for cities with AIMSs and cities without AIMSs, and suggests that trends diverge: cities with AIMSs show a greater increase in assaults than control cities. A conditional fixed-effects Poisson regression was used to determine any effect of AIMSs being operational (0 or 1; 0 for control cities) on police-recorded violence over time and for each of the crime categories. In each of the three models there was a significant influence of time on the outcome variable (sexual: IRR 1.05, z = 46.83, p < 0.001; with injury: IRR 1.02, z = 39.49, p < 0.001; with no injury: IRR 1.05, z = 111.40, p < 0.001). For each of the crime types, an operational AIMS was associated with a greater ascertainment of crime (sexual: IRR 1.05, z = 2.52, p < 0.05; with injury: IRR 1.03, z = 3.80, p < 0.001; with no injury: IRR 1.21, z = 23.38, p < 0.001). Possible reasons for this divergence include more police in NTEs able to record incidents that would otherwise have been missed. However, the quarterly nature of these data mean that attribution to AIMSs is not feasible and the effects could be an artefact of other changes locally.

FIGURE 4.

Police-recorded violence^b for control and intervention cities. a, Recording processes and counting rules for ‘violence against the person’ changed substantially from 2012 onwards and accounts for the upward trend evident here; b, violence without injury, violence with injury, and sexual offences.

Case study 1: site A

The site A NTE (Figure 5) has a high density of premises licensed for the on- and off-site sale of alcohol. It has a large student population and is a destination city, attracting people from surrounding areas and across the UK every weekend. The central actor in managing the acutely intoxicated is the AIMS, which is based in a permanent location in the city centre. Established in September 2012 through a partnership between the health board, the police and the ambulance service, the AIMS was a response to the growing challenge of managing acutely intoxicated individuals in the ED and the stress that it was causing staff and patients. Its primary aim was to divert individuals with AAI and/or minor injuries away from the ED on Friday and Saturday nights. Personnel were funded initially by grants from the Substance Misuse Area Fund and the Regional Collaboration Fund and volunteered to work in the AIMS on an overtime basis. Grants reduced and although hospital staff, paramedics and EMTs continue to volunteer to work in the service on an overtime basis, the shift now forms part of the core duties of health-care workers and police officers.

FIGURE 5.

Ethnographic infographic for case study 1.

Case study 2: site B

Site B has a compact, densely populated NTE (Figure 6). It has a large student population and, like site A, it attracts people from the surrounding areas, including for hen and stag parties. The central actor in managing the NTE is the AIMS. Located in two portable buildings in a city centre car park, the service was established in September 2014 through a partnership between the health board, St John Ambulance, ambulance service, local authority, police and a local university. Founded on a number of partnership initiatives that had been operating in the NTE since the early 2000s, the service’s primary aims are to alleviate the demand on emergency services (police and ambulance services and ED) and to create a safer NTE by diverting individuals who may be acutely intoxicated, injured and/or vulnerable to an alternative facility. The AIMS is led by St John Ambulance.

FIGURE 6.

Ethnographic infographic for case study 2.

Case study 3: site J

The site J NTE (Figure 7) is characterised by a large student population and is a destination for drinkers across the UK. Unlike sites A and B, the city does not tend to attract people from surrounding areas, perhaps owing to the number of other large cities nearby, nor does it have a reputation as a destination city attracting stag and hen parties. The NTE does not have an AIMS but operates a number of partnership initiatives designed to improve the safety of its NTE through promoting best practice among bars and clubs. It also runs K-responder, a joint initiative between the police and the ambulance service intended to reduce the demand on these services in the city centre over Friday and Saturday nights, which has been in operation for around 6 years. A paramedic practitioner (on overtime) and one or two special constables (police volunteers with power to arrest) respond to calls predominantly from licensed premises in and around the city centre on Friday and Saturday nights in a RRV.

FIGURE 7.

Ethnographic infographic for case study 3. Resus, resuscitation area.

Police

The primary focus of police work in the NTE is to deal with perpetrators and victims of crime and alcohol-related crime. Police work in the NTE was organised differently than at other times of the day. It aims to be preventative rather than reactive and is designed to ensure high visibility to enhance safety, deter criminal activity and facilitate early intervention to prevent issues escalating. In all sites, police work entailed active and friendly engagement with patrons of the NTE and the use of dispersal notices to manage antisocial behaviour. Arrests were avoided where possible, because this involves additional administrative work and takes officers away from the front line if they are required to place someone in custody. Police work also involved intervening with individuals who were intoxicated and/or vulnerable or injured, although this varied across the sites. In the sites with AIMSs, there was more proactive police engagement with this population and referral of cases to the AIMS either directly or via a third party (e.g. ambulance, Street Pastors). At site J, in the absence of an AIMS and with no referral pathway available to them, officers displayed a disinclination to engage with the acutely intoxicated because their management could be protracted and took them away from their core duties. At sites A and B, a police officer also worked in the respective AIMS, primarily to protect health-care staff, intervene in any incidents and interview victims of crime. Officers also assisted with AIMS work, such as information gathering, handing out vomit bowls, tidying and cleaning up after patients, helping nurses to contact friends or family and interacting with patients and/or their friends or family.

Ambulance crews

Ambulance crews play a key role in managing alcohol-related risk in the NTE, but this varied across sites. At site A, a mobile medics crew operated from the AIMS and responded to all city centre 999 calls plus radio calls generated from street-level agents. At site B, all city centre 999 calls were directed to the AIMS, leaving ambulance crews to deal with other emergency calls outside the city. At both sites, city centre 999 calls were triaged by ambulance operatives. At site A, mobile medics were the resource of choice to respond to city centre calls, but other ambulance resources would be dispatched if the mobile medics were not available. At site B, if an 8-minute response was indicated, an ambulance resource would be dispatched prior to the call being transferred to the AIMS if St John Cymru resources were unavailable, but usually the Welsh Ambulance Services NHS Trust (WAST) resource was ‘stood down’ for AIMS staff to attend. At site J, city centre calls were managed by either the K-responder or regular crews.

The work of ambulance crews is to assess the case and make a disposal decision. In the case studies with AIMSs they could treat at scene, refer to AIMS or refer to ED. At site J, in the absence of an AIMS, the only options were to treat at the scene or refer to ED.

Street Pastors

Street Pastors, one of the many volunteer initiatives run by the Ascension Trust (London, UK), operated in each of the three case study sites. Pastors understood their NTE work to fulfil their responsibilities as good Christians by helping the vulnerable. Pastors tailored their work to each locale and the needs of individuals. In city centres this is generally understood to mean helping individuals who are intoxicated, although their work extends beyond intoxicated individuals to anyone who was vulnerable. Although not clinically qualified, all Street Pastors had undergone drug and alcohol training and, at site A, basic life support training was also provided by one of the ED nurses as a direct result of the Street Pastors’ involvement in the AIMS partnership.

Street Pastors worked at street level and intervened in a number of ways to ensure the safety and well-being of the patrons of the NTE by, for example, interacting with homeless people and providing them with chocolate, water and/or blankets; handing out water and/or flip flops; offering assistance to those who appeared to be intoxicated and/or vulnerable and/or injured; collecting glass bottles and safely disposing of them to prevent glass injuries and their use in potential assaults; and assisting individuals into taxis to get home or those who were lost.

The role of Street Pastors in the NTE varied across the case study sites. In all cases they were actively involved in managing intoxicated and/or injured individuals. At sites A and J they responded to calls for assistance from other agents in the NTE (door staff, police, taxi ambassadors, mobile medics) and at site A they made referrals to the AIMS. At site B, Street Pastors were less integrated into the management of alcohol intoxication and alcohol-related risk in the wider NTE ecology, possibly because of the availability of student volunteers aligned with the AIMS who fulfilled the functions performed by Street Pastors in other cases. At site J, of the four agencies shadowed, the Street Pastors and the K-responder teams were the only agencies that understood dealing with intoxicated individuals to be one of the core purposes of their work and not a distraction from it.

St John Cymru

St John Cymru, the Welsh arm of St John Ambulance, is a third-sector organisation operating across Wales. It provides first aid, training, first aid equipment and resources, patient transport and first aid cover for events. It is the lead organisation in the operation of the AIMS at site B.

Metropolitan Police Service student volunteers

The site B AIMS is supported by two to six Metropolitan Police Service (MPS) student volunteers (male or female) depending on availability and anticipated demand. All student volunteers attend a local university and are given the opportunity to volunteer at the AIMS as part of their undergraduate degree in public services. Students who choose to do this commit for the entire 3 years of their course. All volunteers undertake training in first aid (provided by St John Cymru), manual handling, vulnerability training and violence and aggression training. MPS volunteers retrieved patients, brought them to the AIMS and undertook proactive patrols in the city centre looking for potentially injured, intoxicated and/or vulnerable individuals who needed assistance and/or referral to the AIMS.

Door workers

Door workers, typically employed by licensed premises, also have a role in managing alcohol-related risk in the NTE. They can refuse entry to individuals who appear acutely intoxicated and intervene in cases where an individual’s behaviour suggests intoxication. Sometimes intoxicated individuals were managed inside the venue by being moved into a separate first aid area and their needs (e.g. receiving water, calling friends) met. Door staff also referred cases to other NTE agents as appropriate.

Taxi drivers

Taxi drivers in the NTE often have a role in transporting intoxicated patients to and from the city centre in exchange for a fare. Intoxicated individuals are considered to be risky customers. If someone vomits in the back of a taxi, this costs the driver time and lost earnings because of the need to clean the vehicle. Nonetheless, intoxicated individuals are an important source of income for drivers. Taxi drivers who chose to work these nights mitigated these risks by attempting to judge how intoxicated individuals appeared before allowing them into their cabs. Some male drivers were also wary of accepting lone female passengers because they did not want to be accused of sexual assault.

Site B made extensive use of taxis to transport people to ED and home, although the site B city centre Street Pastors were more cautious about taxi use. Site A staff used taxis to transport individuals with minor injuries to ED and home but relied on known drivers only. At site J, NTE agents used taxis to get people home and to attend ED for minor injuries but reported that drivers often refused to transport an intoxicated passenger. Site J Street Pastors described how they tried to stay out of sight when escorting individuals to a taxi rank to avoid being a ‘red flag’ to taxi drivers, because many taxi drivers refused to take intoxicated individuals. Ambulance crews identified the refusal of taxi drivers to transport intoxicated individuals to ED as a reason for inappropriate use of ambulance resources. At both site A and site B, the city council runs a taxi ambassadors (council employees) scheme to oversee the taxi rank, directing patrons into taxis, ensuring that taxis take patrons or, in cases where drivers refuse to take an intoxicated patron, radioing the AIMS for assistance. At site J there were only taxi drivers, no taxi ambassadors. Taxi drivers sometimes refused to transport intoxicated patrons to ED, leaving the task to other services:

That’s half the battle really; if the taxis would take them we wouldn’t have to take them, but we have no choice but to take them to A&E [accident and emergency department].

M (emergency care assistant)

(Quotation attributions minimally describe respondents’ roles while ensuring anonymity, but with sufficient detail to link data with the original transcripts.)

The challenges of managing alcohol-related risk

Alcohol-related risk and harm in the NTE presents in diverse ways. It includes those who have sustained minor or major injury either through accident or assault, individuals who are ‘worse for wear’ (often unable to control bodily functions) and require support to sober up, individuals who are unresponsive (‘flat out’), the acutely vulnerable (such as intoxicated young women who had become separated from friends) and others with longer-term vulnerabilities such as mental ill health, homelessness and alcoholism and other addictions.

People who are intoxicated present several challenges to workers in the NTE. They are often difficult to reason with and considered to have the potential to become violent and aggressive; they may also be reluctant to wait in ED to be seen. Dealing with alcohol intoxication is also ‘dirty’ work, with acutely intoxicated individuals likely to lose control of their bodily functions. Arguably, the most important challenge is assessing if an individual simply needs time to sober up or there is a more serious reason for their symptoms, such as a head injury requiring specialist care:

It’s too risky to discharge on scene. I don’t doubt some do, but you have to be really careful, they could have a head injury, or they could choke on their vomit.

Ambulance crew member (site B)

Acutely intoxicated people are often unable to offer an account of what has happened to them, with agents having to rely on friends and other observer accounts. At sites A and B, the presence of an AIMS allows such assessments to be undertaken over a longer time frame than at site J, at which such assessments will result in ambulance crews or police being involved for a protracted length of time or, depending on the other demands on services, a referral to ED:

Like with drunks, they don’t need to be in ED, but if we discharge on scene and their friends don’t look after them properly, and they die, then it’s our fault and not theirs for getting so drunk. So I’ll only ever discharge if they actually seem fine, you know, talking and walking, and with people who can look after them, but if they’re out of it, it’s too risky. Plus alcohol can mask stuff like head injury, so it’s better just to be safe.

Paramedic (site B)

Much of the work of managing AAI also involves managing vulnerability, particularly of women. AIMSs operate as places of safety in such cases, but this option is not available at site J.

Street-level work

At all three case study sites the different agents in the NTE worked flexibly and responsively to identify and manage alcohol-related risk. Activity was underpinned by an organising logic that emphasised prioritisation of activity in relation to demand, using resources judiciously given agents’ wider responsibilities, and the avoidance of inappropriate use of emergency services. At site J, in the absence of an alternative to ED, ambulance crews acted as gatekeepers to ED:

I manage it by trying to discharge on scene if I can. Now, that will take more time, but if it saves a bed up there it’s more cost-effective and a better outcome because they [the intoxicated] are hard to manage, aren’t they?

Advanced EMT (site J)

Agents at all three sites were connected by radio; this facilitated reflexive monitoring of the NTE and situational awareness of shifting patterns of demand. Radio can be used to alert agents about incidents where intervention is required and negotiate referrals based on agents’ availability and wider priorities:

The AIMS mobile medics received a call from the Street Pastors to attend to a ‘paralytic female’ who they could not get ‘off the floor’. The mobile medics agreed to attend, leaving the AIMS to respond to the call. However, en route they received what they felt was a higher priority call (an unresponsive female) from door staff so they decided to defer their attendance to the first call, informing the Street Pastors that they were attending a ‘higher priority call’ and advising them to radio police for assistance.

Field notes

Maintaining situational awareness appeared to be more challenging at site J owing to the geographical spread of the NTE. Outside the primary areas of night-time activity the NTE was distributed across many streets, some of which had only one large venue. K-responders fulfilled an important role, linking on-the-ground intelligence of the demands in the NTE to ambulance control. Ambulance dispatch used the K-responder paramedic to ‘quickly eyeball’ cases in the city centre before determining what resource was required, as information from calls was often limited. Furthermore, because calls were often from the same vicinity, and for similar kinds of complaints, dispatch could also make use of the K-responder paramedic to verify how many patients were on scene, to prevent dispatching too many resources unnecessarily.

Referral pathways in the NTE can be complex. A referral might begin with a member of door staff first noticing an intoxicated individual inside or outside the premises and deciding if they should handle the issue themselves, contact the police, Street Pastors or ambulance services or do nothing. To some extent, referral pathways were informed by the availability of other agents. When an agent arrived at the scene, they had to decide on the appropriate action. The police at all sites were encouraged to use the Street Pastors (or, at site B, MPS volunteers) where this was appropriate to avoid being distracted from their core work.

In the presence of an obvious injury, determining the requirement for ED or AIMS referral or treatment on the scene was usually straightforward. In cases of acute intoxication and absence of an injury, or where individuals were ‘flat out’, reaching a disposal decision was more complicated, reflecting the need to manage risk and use resources appropriately. Decision-making was often based on situated judgements taken on the ground, in the light of the perceived state of the individual, whether they were alone or with friends, and at what stage of the night the agent encountered the individual. For example, if an individual appeared very intoxicated but was in the care of friends who were acting in the individual’s ‘best interests’, then assisting the individual and their friends into a taxi home was viewed as a satisfactory outcome. In cases where this was not possible, the AIMS provided a useful facility where they could be kept safe and monitored until sobering up. At site J, in the absence of an AIMS, agents had to remain on the scene until they were able to discharge or had to take the individual to ED. Site J Street Pastors reported that they spent more time dealing with individuals who were vulnerable, particularly lone females, than individuals who were acutely intoxicated. They also reported sometimes using the 24-hour McDonald’s (McDonald's Corporation, San Bernardino, CA, USA) on the edge of the city as a safe space where lone females could wait until they could get a train or bus home in the morning.

AIMSs

Although organised and structured differently, the successful operation of both the AIMSs hinged on negotiating the management of intoxicated individuals to consider the core roles of each of the agents involved while working flexibly to mitigate some of the risks to which individuals in the NTE are exposed. Although there was a strong sense that nurses are the overall authority on patient care, during busy periods roles and responsibilities become more fluid. The need to manage patient flow was an important organising logic in both AIMSs, but this appeared to be driven by the need to maintain capacity rather than meeting performance targets (such as in the EDs).

An individual attending an AIMS was first assessed by a nurse and a decision was made about whether they could be treated or should be referred to ED. Although referral pathways to the site B AIMS were open, the site A service had formal referral criteria. Nevertheless, we observed that NTE agents regularly referred individuals to the site A service who did not meet the referral criteria; ED nurses were happy to assess and make the ED referral if this was indicated.

The site A AIMS was formally aligned with the ED; thus, the management and progress of referred patients reflects the fact that they have been triaged in the AIMS. Minor injury patients were transported to ED by taxis or the urgent care service; in more serious cases (e.g. head injury), ambulance crews were involved. Operating within the same governance structures as the ED also enabled the AIMS nurses to refer patients to mental health or social services when necessary. Unlike ED, the AIMS also had direct access to PARIS (mental health services database) and indirect access to the police database (via the officer in the AIMS). Access to the PARIS database enabled ENPs to establish whether a patient presenting with potential mental health concerns was known to mental health teams or a new referral was required. Irrespective of whether or not the patient was known to mental health services, if they displayed suicidal behaviour or expressed suicidal thoughts then they were referred to the ED for observation.

There was no formal relationship between the AIMS and the ED at site B. In instances where the nurse was unable to fully assess or treat the patient, the patient was advised to attend the ED following their assessment. An informal agreement between St John Cymru and the ED enabled St John Cymru patients to be offloaded quickly to allow the crew to return to the city centre. AIMS staff also made extensive use of taxis to transport less serious cases to ED.

Emergency department

In the absence of AIMSs or when the AIMS was not operating, acutely intoxicated patients were referred to ED. Here, intoxicated patients were cared for in the context of the wider spectrum of ED work and prioritised according to need and clinical risk. All ED work in the case studies was driven by waiting-time targets, where the aim is to see, treat and discharge patients within 4 hours. During the fieldwork, the effects of this were most marked at site J where, unlike in Wales, there are financial penalties for failing to meet these targets. The structure of EDs in all cases included the areas minors, majors and resus (resuscitation area), with additional areas such as a Clinical Decision Unit (site J) or Ambulatory Care Unit where patients could be placed if they did not require a trolley and/or were likely to be discharged after a period of observation and assessment.

At site J, all patients arriving by ambulance were admitted initially into majors or resus but, depending on the demands on the service, could be reallocated to minors by the nurse who triaged patients on arrival. The management of patient flows and the effective utilisation of beds was a nursing responsibility, but pressures on admissions and downstream blockages in the hospital meant that sometimes there was no option but to attend to patients in the corridors and in the middle of ward areas. Fieldwork at site J revealed nurses to be working under considerable pressure and feeling dissatisfied by how the demands on the system compromised their ability to provide dignified care. In this context, nurses tended to perceive intoxicated patients as inappropriate users of the service.

According to the lead nurse at site J, most of the intoxicated patients in the ED were not acutely intoxicated but had sustained a minor injury under the influence of alcohol. These patients would be seen in minors and treated for their presenting complaint. Most sobered up while waiting to be treated and had a degree of sobriety when seen. Acutely intoxicated patients in majors tended to arrive by ambulance and during the fieldwork presented with a combination of intoxication and a head injury or a fall or a mixture of recreational drug and alcohol use. Patients unable to maintain their own airway, presenting a low Glasgow Coma Scale (GCS) score, who had aspirated vomit or who were violent and aggressive were admitted to resus. Patients considered too intoxicated to be discharged were moved to the Clinical Decisions Unit where they could be observed before discharge. This was also a mechanism for managing waiting-time targets.

Street level

At all case study sites, much of the work of managing alcohol intoxication and risk took place at street level. Door workers, police, Street Pastors, MPS volunteers and ambulance crews worked hard to deal with situations on the scene. This might involve assisting someone who is unwell, providing water or helping people to contact friends and/or supporting individuals to get a taxi home. At sites A and B, the AIMS provided an alternative place of safety to enable individuals to recover, whereas at site J intoxication management could occupy street-level agents for prolonged periods of time and/or result in ED admission.

AIMS

Staff at site A and site B AIMS worked actively to manage intoxication. During the fieldwork there were relatively few occasions where a patient’s stay extended beyond 2 hours, and both services worked with an organising logic designed to sober up patients and/or discharge patients into the care of family and friends. Observed strategies for achieving this included:

• putting intoxicated patients on a chair rather than on a stretcher/bed to prevent them from falling asleep

• repeatedly assisting patients to sit upright

• giving intoxicated patients water

• attempting to engage intoxicated patients in conversation to keep them awake

• enlisting friends and family into the care and management of intoxicated patients

• proactively contacting friends and family at the earliest opportunity to expedite discharge.

Emergency department

At all case study sites the active management of intoxicated patients in AIMSs contrasted with the management of intoxicated individuals in ED. On the one hand, intoxicated ED attendees who were not deemed to be at risk tended to be left to sober up in the waiting room, with staff privately hoping that they would self-discharge, particularly when ED was busy. Unlike AIMSs, EDs did not have the resources for staff to proactively contact friends and family to enable discharge. On the other hand, ED staff observed that ‘flat out’ patients tended to be overinvestigated by doctors. Pressures of work on staff, a target-driven culture and the very real challenges of undertaking an assessment on someone who is unconscious or uncooperative meant that intoxicated attendees were often sent for computerised tomography (CT) scans to exclude a possible head injury before a full clinical assessment had been carried out. At site J, patients required a nurse to accompany them to a CT scan and one consequence of this was to deplete ED nurse staffing levels, which undermined the quality of care nurses felt able to provide.

A member of staff from the site J ED highlighted that one advantage of an AIMS would be to improve the quality of care received by intoxicated patients in the ED:

I think people just expect too much. The more you give, the more they’ll take. One patient annoyed me the other day, an intoxicated one – she was asking me if I could get her a mirror so she could look at her hair [pulls a face of disbelief]. I felt so angry with her. As if we have time to do that?! So I just told her she could go and look in the bathroom. But if I was a staff nurse in an AIMS I think I’d feel different because that’s what the service is there for, so I would feel like I could provide better quality care for these intoxicated patients, without feeling bad about it taking me away from giving quality care to my other patients. I think it would be a win-win. Non-intoxicated patients in here would get better care, because we would be able to give them our complete focus, and intoxicated patients would get better care because we wouldn’t have to be worried about caring for our more serious patients.

Staff nurse

Treatment milieu

The organisational culture of the two AIMSs contrasted markedly to the clinical culture of all three EDs and this had important implications for staff interactions with the acutely intoxicated. In both facilities, service users were generally young, out enjoying the NTE and accompanied by friends in the clinical areas. Most staff in the AIMS had chosen to work in the AIMS and saw the management of intoxicated individuals as their core work, whereas in the three EDs such patients tended to be viewed as distractions from the ‘real’ work of ED staff:

If it’s a purely intoxicated patient, they don’t need to be here really, they don’t need urgent emergency care, I mean we don’t really provide any clinical care, and I just end up feeling like I’m their mother, looking after them, like a babysitting service, and I’m not here for that. Imagine when it’s busy and we have an intoxicated patient, an intoxicated patient with injuries, and then all your other urgent patients. It’s an ethical thing too. It costs £250 just for an ambulance. It’s a ridiculous waste of resources.

Staff nurse

Compared with work roles in the ED, work roles in the AIMS were more fluid [‘everyone mucks in’ (staff nurse)], albeit with clear demarcation in relation to clinical decisions. In addition, other NTE agents dropped in to catch up with staff or to deliver food and drinks to staff, particularly if the city centre was quiet. Staff would share stories and discuss the politics of their respective jobs. For the various agents of the NTE, these opportunities for ‘chatter’ were viewed as a valuable resource for learning and better understanding each other’s work, and sharing their knowledge of the NTE in general.

The above qualities had the effect of making the AIMS environment feel more informal than the ED. The revelry of the NTE seemed to seep into the service, and the AIMS staff appeared to enjoy this. For example, at site A, staff collected and kept the various fancy-dress props left behind or given to them by patients, and spent time talking with patients. AIMS staff seemed to enjoy opportunities to exchange stories with their patients. Because there was less concern that an intoxicated patient’s behaviour would have a negative impact on the experiences of their fellow patients, who were also likely to be patrons of the NTE and thus out to have a good time, staff were able to engage with patients ‘on their level’, whereas in the ED intoxicated patients whose behaviour was inappropriate were challenging to manage within the prevailing interactional norms.

Staff orientation to acute alcohol intoxication

At all three case study sites, emergency services staff regarded AAI as normal behaviour and for the most part staff were non-judgemental in their interactions with patients:

I mean we’ve all been there.

Paramedic (site B)

We’ve all been like it.

Police officer (site B)

Nevertheless, although AAI was accepted as normal behaviour, it was not condoned. Although AIMS staff were happy to engage and interact with patients on friendly terms, this did not extend to what, at site A, they termed ‘there, there nursing’, that is, anything that made the patient too comfortable. Indeed, anyone exhibiting such behaviours, for example, by rubbing a person’s back or providing blankets, was reminded that this was not how things should be done.

At street level, the primary complaint was not that people became intoxicated but that responsibility for looking after them fell to the emergency services:

Problem is, people won’t take responsibility for themselves anymore and no one will take responsibility and turn inappropriate people away either.

Paramedic (site B)

We used to get ‘em [drunks] all the time before the AIMS. Trouble is, it’s too easy to call an ambulance and people don’t like to take responsibility. [. . .] People don’t seem to have any common sense anymore. Friends won’t just put their mate in a taxi or look after them. It’s just too convenient to call us and have someone else look after them.

Paramedic (site B)

In the ED there was ambivalence about the care of intoxicated patients. On the one hand, being deemed responsible for their own condition, they were regarded as undeserving and inappropriate users of the service. For example, nurses referred to the care given to those sleeping off the effects of intoxication as ‘babysitting’. On the other hand, it was also recognised that individuals were vulnerable and presented clinical risks. The care of intoxicated patients in majors and resus represented dilemmas for staff; intoxicated patients were regarded as an illegitimate demand on resources that should be left available for other emergencies:

Why are they here if they’re just intoxicated? There’s nothing wrong with them?

As a place of safety, you can’t just leave them on the street, can you? They’re vulnerable.

Right, I get it, but there should be somewhere else you can take them.

I know.

I mean, there’s over a 4-hour wait to see a doctor.

Well, they’re not really here to see a doctor, are they, they’re here to sober up.

I’d say she’s just smashed, and he is too by the look of his eyes. They all say they’ve been spiked but the consultant tells us that that’s pretty much urban myth for the most part. You can tell the difference between someone who has been spiked and someone who is just pissed. I mean, if I’d had that many drinks I’d be on the floor. It’s funny really.

Yeah, spiked on Jägerbombs more like, that’s what I said.

But yeah, they don’t really need to be here, and they’re taking up beds, so it is annoying, but what else can you do with them? There’s nowhere else for them to go. We had one here in resus last week, only 16!

Do you think there’s a need for it [an alcohol intoxication management service]?

Definitely. It’s a joke. Every weekend between 2 and 3 [o’clock] that’s all we get, but it’s all the time really, y’know, we’ll have someone intoxicated in most days.

AIMS

Many staff had self-selected to work in the AIMS. At site B, for the St John Cymru paramedics and nurses whose core day-to-day roles were either working for WAST or the health board, working in the AIMS reportedly offered an easy way to make a ‘bit of extra’ money, providing a break from some of the pressures they experienced day-to-day:

I love it up there [in ED], but it’s so full-on all the time, so this place is a nice way to get a bit of extra, although it can be mental too.

K (St John nurse/senior ED nurse)

The St John first aid volunteers and MPS volunteers, on the other hand, freely offered up their time to work in the AIMS, and reportedly enjoyed their time there. Furthermore, unlike in the ED, where interaction with intoxicated patients was viewed as a distraction from work, interaction with patients was a central component of the work in AIMSs. The volunteers appeared to enjoy engaging with their patients. Police officers, usually based in Community Safety Teams, do one AIMS shift per month. Officers reported to ‘like’ these shifts and appeared to enjoy their time chatting with other staff members and patients.

Similarly, at site A the AIMS shift seemed to provide ED staff with light relief and entertainment after what had usually been an intense, and almost always fast-paced, working week in the ED. Furthermore, unlike the seriousness and complexity of some of the conditions of patients in the ED, intoxication was for the most part viewed as straightforward to treat and the unintended outcome of some fun and frivolity rather than anything more serious. Ambulance crews saw the AIMS shift as an easy way to accrue overtime, and the paramedic and EMT seemed to enjoy the general revelry of the NTE, and the varied pace of the work (some nights it would be quiet; other nights they would be working non-stop), which contrasted with their day-to-day work, where the volume of calls was almost always high, with little time for pause. For the urgent-care assistants (UCAs), the mobile medics shift was part of core duties and this group expressed more mixed perspectives; some found intoxicated patients difficult to manage and did not enjoy dealing with bodily fluids and aggressive behaviour. Police officers’ views of their work in AIMSs also varied. For some, it was ‘a welcome reprieve’ from their normal duties in the NTE, whereas for others it was viewed as dull in comparison. Nevertheless, all officers seemed to understand that police presence in AIMSs was necessary for nurses to feel safe while conducting their work and understood that without their presence the service would no longer function. AIMS shifts were therefore accepted as a necessary part of their work in the NTE.

Street-level workers

At the street level, acutely intoxicated individuals were regarded as just one example of numerous inappropriate demands made on front-line emergency services. Agents in the NTE talked about how the growing volume of their work related to the care of cases that challenged their sense of professional jurisdiction:

Our bread and butter, our top five, I’d say, is drug, alcohol, assaults, chest pain, mental health and falls.

Yep, we get a lot of unnecessary calls, your mental health, your drug and alcohol, they’re just difficult patients because you don’t know what to do with them – they’ll keep coming back again and again. A lot of the young ones don’t like it.

Why?

I don’t know. They don’t want the bread and butter for some reason, but this is the job, this is the world we live in.

I don’t mind working overtime when it’s for someone genuine, but when it’s not [sighs] – I’m losing my patience I think. [. . .] I don’t think I could do this job full-time [having just returned from maternity leave]. There’s just too many social calls.

Paramedic (site B)

Emergency department staff

Emergency department staff recognised that some intoxicated patients had a legitimate need for treatment and/or presented serious clinical risks (e.g. head injury). However, those who attended ED to sober up were generally viewed as inappropriate admissions that used scarce resources (e.g. nursing time, trolleys). Although intoxication was regarded as normal behaviour, those who ended up in the ED were considered to have taken it too far and had failed to take responsibility for themselves (or their family and friends had failed to take responsibility for them). Nurses were observed to work under pressure that caused them to struggle to give the quality of care they aspired to. In this context, care of intoxicated patients was an unwelcome additional demand on an overstretched team.

Interviews

Survey of AIMS users

Controlled cross-sectional emergency department survey

Health and Care Research Wales public involvement community

This network is a group of people interested in being actively involved in health and social care research and it is based in Wales. The network was set up in 2006 as part of the Clinical Research Collaboration Cymru and since 2010 has been incorporated into the National Institute for Social Care and Health Research Clinical Research Centre. The network is made up of over 200 patients, carers, service users and members of the public interested in a range of areas including dementia, children and young people, cancer, diabetes mellitus, arthritis, stroke and public health. We engaged with the Involving People Network in Wales in the early stages of proposal development, which recruited one member from South Wales to act as independent lay advisor to the SSC. The Welsh member had been a victim of an alcohol-related assault and had an interest in health service research.

Sheffield Addiction Recovery Research Panel

The Sheffield Addiction Recovery Research Panel (ShARRP) is a PPI group receiving support from the Clinical Research Office of Sheffield Teaching Hospitals NHS Foundation Trust. The group comprises former and current service users for drug and alcohol treatment and family and carers of people who have had experience of drug and alcohol misuse. The group represents most of the main providers of local drug and alcohol treatment and support groups, including known national agencies such as Alcoholics Anonymous (New York, NY, USA), Turning Point (London, UK), Addaction (London, UK), Phoenix Futures (London, UK) and Change, Grow, Live (formerly Crime Reductions Initiatives) (Brighton, UK). Members are also networked with local and national service users and recovery groups, allowing the facility to demonstrate how research can engage and promote PPI.

Two of the founding members of the ShARRP joined the project lay advisory group (LAG) because they had experience of drug- and alcohol-related harms and were keen to see how AIMSs cared for people affected by alcohol.

Lay advisory group involvement activities

The main aim of LAG involvement was to achieve insight into the methodology most likely to be well received by the intended participants (users of AIMSs and EDs). ShARRP members advised on survey design and provided revisions so that surveys could be read and understood by participants.

The normal mechanism for PPI in the Evaluating the Diversion of Alcohol Related Attendances (EDARA) project has been face-to-face meetings between the LAG and PPI lead. Meetings have been held at regular intervals throughout the project but with flexibility to enable more intensive periods of work and around various PPI-related events. In meetings and by e-mail correspondence, the LAG was invited to review all project documentation, review and comment on patient-friendly recruitment and consent processes, and review and suggest improvements on patient information sheets and consent forms.

The SECF has a long history and expertise in urgent and emergency care research and so was consulted specifically around ED users’ views. 79 Researchers attended regular SECF meetings to update on progress. Researchers also attended regular ShARRP meetings to discuss preliminary findings and to solicit feedback on data on AIMS users’ views.

Study Steering Committee

A fully independent member of the public was invited onto the SSC. Independent of the ‘hands-on’ LAG, the PPI SSC member represented patient and public interests at these meetings, specifically around adherence to the protocol and any issues around participant safety. The PPI SSC member ensured that the safety, well-being and rights of participants were considered. 82 The SSC member was kept informed of LAG activities outside SSC meetings via e-mail to ensure that PPI-related issues could be raised when required.

Specific tasks

Four specific tasks that could usefully include PPI were identified:

1. advise on the AIMS user interviews, recruitment and consent processes, interview question wording and interview format and design

2. advise on the analysis of AIMS user interview data (within the bounds of confidentiality) and the development of the AIMS exit survey

3. input into the design and content of the ED user questionnaire

4. co-production of guidance on AIMS development and implementation.

One of the early project management group and LAG discussions centred on what were to be the inclusion criteria of the AIMSs, what term would be applied and what the definition of these centres should be. Members were keen to avoid the word ‘treatment’ in the name for services, because this was felt to be misleading insofar as no obvious treatment for alcohol use as a disorder is provided. Similarly, ‘recovery’ was highlighted as a term to avoid because no support for recovery is provided and this phrase often connotes long-term support for abstinence in the drug and alcohol service user and research community.

Development of the AIMS exit survey

• Lay advisory group members highlighted the need to ‘ask about comparisons’, for example ‘are you glad you came to AIMS, or prefer to have gone to ED?’.

• Researchers suggested that it would be useful to ask AIMS users ‘in principle, do you think people who use this service should be asked to pay for it?’. The LAG challenged the inclusion of this question, claiming it ‘doesn’t feel fair, appropriate, or relevant’ and that the ‘question could stoke up resentment, which may colour answers to the following questions’. Based on the objections raised, the question was removed. ShARRP LAG members asserted that the questionnaire should not appear judgemental or moralistic.

• Bigger tick boxes were suggested and implemented to make the responses easier for people who may be unwell or intoxicated.

• Easy-to-read, colour-coded faces, happy to sad, were suggested and applied to help people understand the scale ‘very good’ to ‘very poor’.

• Thinking about the sentence in the protocol about ‘concerns that AIMSs may normalise alcohol intoxication’, LAG members asked ‘did they [AIMS users] know their local AIMS existed? If so, did knowing the safety net was there affect their level of drinking?’.

• Incentives to improve survey completion rate: LAG members discussed the initial problems and poor response rate to the AIMS survey and the advantages and disadvantages of using a small monetary incentive. The LAG raised no objection to the use of incentives in principle but felt that it was unnecessary for a short questionnaire and would be practically very difficult to implement. Based on the issues raised, incentives were not implemented.

AIMSs data interpretation

Development of the emergency department users’ questionnaire

• The LAG members agreed that the ED survey should match parts of the AIMS survey for comparability of results.

• The SECF members suggested that we should be interested in hearing from people in our sampling frame who attended an ED but walked out, because this may be because of finding the ED unpleasant in some way. This suggestion was discussed and reflected in the ED survey guidance documents, which directed participating EDs to include all those registered at the ED, including those who subsequently walked out without being seen.

• One SECF member commented that ‘I think your survey needs to be sent out as soon as possible after the visit to A&E; memories are very short!’. The research team considered the current instructions manual issued to hospitals. Prompt mail-out was highlighted.

• A draft version of the ED survey was presented to a SECF meeting for mock completion and comments. One member commented that ‘the questions all seem reasonable and straightforward and should give a good insight into people’s experience of weekend A&E’.

Emergency department survey response and interpretation

Despite having a robust process for checking ED survey patient sample lists for deaths before mailing, a small number were returned and marked as ‘patient deceased’. Our comprehensive review of procedures and report to the REC included feedback from the LAG. In this review, members suggested that text be added to the survey cover letter (and reminder) such as ‘if there has been any change in circumstances in the health of the intended recipient, please accept our apologies for this mailing and disregard this letter and survey’. The REC was satisfied with other corrective and preventative action taken by the research team and so this sentence was not used.

Sheffield Emergency Care Forum and Sheffield Addiction Recovery Research Panel meeting feedback

Project researchers capitalised on good relationships with both PPI groups to attend their regular meetings to provide updates on project progress and garner any additional feedback required. Preliminary findings were presented to both groups. SECF members were surprised by the apparent lack of any impact of AIMSs on ED users’ views. This may be the result of the general assumptions that AIMSs would divert large numbers of intoxicated patients and that the ED environment, especially at weekends, is unpleasant. These thoughts are reflected in the main discussion.

Impact on lay advisory group members

I feel that being involved in the study has been very informative and has hopefully equipped me to be able to contribute to other studies and related projects. Indeed, [En], [Mg] and I recently wrote a letter to [the REC] in support of a related follow-up research application. Looking at consenting people, especially vulnerable people, has been particularly interesting and an experience that will be useful in many future studies. Being involved with EDARA has also dispelled many stereotypes and confirmed that things are not always as reported in the press.

Ln (LAG and SECF member)

Ln’s comments highlight the personal benefits of the involvement activities and in enabling her to positively contribute to further research:

Being involved in the EDARA project has been a positive experience. I have felt at all times our comments have been valued and acted up. For example, the PPI group debated the suitability of including a financial charge question to service users in the AIMS survey: ‘would they be prepared to pay for future services?’. The group thought this was inappropriate. This resulted in it being removed from the questionnaire. The EDARA PPI members regularly feed back to the quarterly ShARRP group to enable us to gain a wider perspective from people who have real-life experiences of problematic alcohol use. I do not have any experience of formal research. My involvement with EDARA has helped my personal development and enabled me to gain an understanding of research methodology. In addition, I have been interested in learning about the work of the AIMS.

Md (LAG and ShARRP member)

Overall, the contribution of the LAG has been valuable not only in the research process but as an enjoyable and useful experience for members working closely with the research team at site J. Managing public involvement across two research centres (sites A and J) created some challenges, which have been reflected on:

I feel I would have benefited from one direct meeting with the site J PPI group as I felt quite isolated in the SSC situation. I did arrange a personal visit to the Site A AIMS which was very informative and I reported on this in the newsletter. I found the protocol of my role within the SSC rather difficult to grasp – I found the meetings worthwhile but was unsure of the value of my contribution.

Hn (SSC lay advisor)

The lay SSC member’s role was to represent patient and public interests at SSC meetings, specifically around any issues of participant safety. The disconnect between the LAG and SSC member was necessary and unavoidable but created challenges for LAG work to inform SSC input. In hindsight, a member of the site A team would have been well placed to offer Hn face-to-face advice and support on his role as SSC member.

The Sheffield Addiction Recovery Research Panel

The ShARRP was formed partly to meet the public involvement needs of the EDARA project and to address the gap in the provision of drug- and alcohol-related PPI. Throughout the EDARA project, ShARRP has grown and is now well recognised as a source of relevant PPI, supporting several other drug- and alcohol-related research projects:

I have been pleased to be part of the LAG. Initially I was concerned that our involvement may have been a tick box exercise, but I am pleased to say that I always felt that our comments and suggestions were valued and acted on. I thought that our input to the format of the questionnaire was important. It ensured the questions and information was explained clearly to service users.

Md (LAG and ShARRP member)

Involvement activities with clear results reassured ShARRP members that their input was worthwhile. Contrary to the often self-stigmatising views of (former or current) drug and alcohol users and their friends and family, this project has reinforced the fact that people with experience of drug and alcohol use are valued contributors.

Workstream 1

1. Ethnographic studies of the impact of AIMSs on the front-line work setting (lead: DA).

2. Context.

3. AIMS users’ views.

4. ED users’ views.

Workstream 2

Outcome evaluation.

Workstream 3

Economic evaluation.

Site I

Despite initial positive responses from the AIMS at site I and its associated ambulance service, support for EDARA was withdrawn because of uncertainties around the operation of the AIMS and concerns over pressures on staff time and capacity to support workstream 1 work and undertake workstream 2 data extraction requests. Site I was therefore unable to take part in any element of the evaluation.

We identified site F as a replacement for site I, which matched site J as a control site. Site F was able to participate in workstream 1 (iii) and (iv), workstream 2 and workstream 3. Administration of the ED survey was delayed at both site F and site J to accommodate the change of site (the paired survey ran in August 2017; all other ED paired surveys ran in March 2017). Delays in setting up site F meant late engagement with the ambulance-service-run AIMS in workstream 1 (iii) and (iv), yielding no interviews and very few survey responses. Site B replaced site I as a workstream 1 (i) case study site.

Site D

Early recruitment visits to site D (May–July 2016) were unsuccessful owing to very low or zero clients that could be recruited. Despite initial approval from the AIMS lead organisation (site D police), site D withdrew their support for workstream 1 (iii) owing to ethical concerns over approaching vulnerable adults for the purposes of research. This notwithstanding, we were able to confirm the operational dates of the AIMS and therefore kept this site in workstream 1 (iv), workstream 2 and workstream 3.

Site E

Despite preliminary approvals from site E ambulance service and AIMS staff, site E withdrew support owing to significant operational demands and concerns over staff capacity to support the study. We were able to confirm the operational dates of the AIMS and therefore kept this site in workstream 1 (iv) and workstream 2. We identified site H and site G as sites to replace site D and site E in workstream 1 (iii), workstream 2 and workstream 3. However, for the reasons outlined above, the ambulance service for site G was unable to provide data, so we could not include this site in the workstream 2 ambulance data analysis and workstream 3.

Descriptive statistics

Emergency department attendances were analysed across all sites from 16 November 2010 to 31 March 2016. Table 12 shows the mean number of ED attendances per day during hours of AIMS activity at each site before and after the service started operating at the intervention site in each matched pair. Pre-intervention data were not available at site H (or site K when matched with site H) because the site H AIMS started operating before 2010. The number of ED attendances per night ranged from 31 at site G pre intervention to 127 per night at site K post intervention. Emergency department attendances decreased after intervention at sites A and C, and at site L post intervention at its paired site (site B), but increased after intervention at all other sites.

Tables 13 and 14 show the percentage of male patients (see Table 13) and mean patient age in years (see Table 14) in the same format. The mean percentage of male patients across all sites was 51.9% pre intervention and 51.1% post intervention, with little evidence of men or women being over-represented at any site before or after implementation. The mean age of patients across all sites was 42.4 years pre intervention and 44.3 years post intervention. All sites except site D showed a small increase in mean age post intervention.

Linear regression analysis (estimated using ordinary least squares)

Tables 18–20 show the results of linear regression analysis using intervention-site-only analysis (see Table 18), paired analysis with a single control (see Table 19) and paired analysis with pooled control sites (see Table 20). Bonferroni correction across eight tests resulted in a p-value of ≤ 0.0125 for statistical significance.

TABLE 18 - Emergency department attendance: linear regression analysis – intervention-site-only analysis

MSE, mean squared error; n/a, not applicable.

The effect is for an AIMS being open compared with no AIMS being open in the period after an AIMS was introduced.

Site	Constanta (p)	Open (p)	Post AIMS (p)	F (p)	R 2	Root MSE
Ha	75.34 (< 0.001)	–4.53 (0.003)	n/a	27.85 (< 0.001)	0.172	10.71
A	74.52 (< 0.001)	–4.04 (< 0.001)	–3.26 (< 0.001)	40.18 (< 0.001)	0.243	9.96
B	50.90 (< 0.001)	–0.90 (0.299)	1.55 (0.001)	38.71 (< 0.001)	0.236	8.26
G	32.84 (< 0.001)	1.06 (0.389)	2.33 (< 0.001)	20.69 (< 0.001)	0.142	6.17
F	80.47 (< 0.001)	0.52 (0.689)	6.16 (< 0.001)	86.88 (< 0.001)	0.409	9.50
C	82.79 (< 0.001)	–2.36 (0.041)	–0.15 (0.809)	27.94 (< 0.001)	0.182	10.33
D	121.52 (< 0.001)	0.67 (0.838)	3.98 (< 0.001)	25.90 (< 0.001)	0.171	16.61
E	124.35 (< 0.001)	1.56 (0.580)	4.61 (0.007)	30.81 (< 0.001)	0.197	14.90

TABLE 19 - Emergency department attendance: linear regression analysis – paired analysis with single control sites

DiD, difference in difference; MSE, mean squared error; n/a, not applicable.

The effect is for an AIMS being open compared with no AIMS being open in the period after an AIMS was introduced.

Site		Constant (p)	Intervention site β (p)	Post AIMS β (p)	DiD (open) (p)	F (p)	R 2	Root MSE
Intervention	Control
Ha	K	121.70 (< 0.001)	–44.21 (< 0.001)	n/a	–2.27 (0.008)	907 (< 0.001)	0.793	11.90
A	N	54.05 (< 0.001)	21.53 (< 0.001)	4.02 (< 0.001)	–1.62 (0.057)	250 (< 0.001)	0.528	10.63
B	L	49.52 (< 0.001)	2.07 (< 0.001)	–0.31 (0.351)	2.07 (< 0.001)	59.48 (< 0.001)	0.210	8.03
G	M	82.95 (< 0.001)	–46.65 (< 0.001)	2.13 (< 0.001)	–0.02 (0.993)	1784 (< 0.001)	0.889	8.39
F	J	102.05 (< 0.001)	–22.85 (< 0.001)	4.93 (< 0.001)	4.30 (0.001)	360 (< 0.001)	0.617	10.39
C	O	69.78 (< 0.001)	10.85 (< 0.001)	2.93 (< 0.001)	–3.99 (< 0.001)	99.6 (< 0.001)	0.308	10.08
D	K	122.36 (< 0.001)	–1.02 (0.028)	6.75 (< 0.001)	–2.95 (0.324)	62.2 (< 0.001)	0.218	14.69
E	M	82.46 (< 0.001)	39.35 (< 0.001)	5.96 (< 0.001)	3.45 (< 0.001)	628 (< 0.001)	0.737	84.70

TABLE 20 - Emergency department attendance: linear regression analysis – pooled control sites analysis

DiD, difference in difference; MSE, mean squared error; n/a, not applicable.

The effect is for an AIMS being open compared with no AIMS being open in the period after an AIMS was introduced.

Site		Constant (p)	Intervention site β (p)	Post AIMS β (p)	DiD (open) (p)	F (p)	R 2	Root MSE
Intervention	Control
Ha	All control sites	82.43 (< 0.001)	–8.466 (< 0.001)	n/a	–0.94 (0.435)	47.11 (< 0.001)	0.054	22.22
A	All control sites	78.03 (< 0.001)	1.57 (0.008)	3.97 (< 0.001)	0.42 (0.756)	28.60 (< 0.001)	0.035	22.22
B	All control sites	79.66 (< 0.001)	–21.08 (< 0.001)	4.09 (< 0.001)	1.10 (0.571)	116 (< 0.001)	0.128	22.09
G	All control sites	80.75 (< 0.001)	–45.37 (< 0.001)	4.01 (< 0.001)	–2.68 (0.538)	444 (< 0.001)	0.361	21.93
F	All control sites	82.07 (< 0.001)	–5.93 (< 0.001)	5.67 (< 0.001)	2.95 (0.245)	48.78 (< 0.001)	0.059	22.10
C	All control sites	81.41 (< 0.001)	–0.18 (0.749)	3.63 (< 0.001)	–5.82 (0.003)	34.16 (< 0.001)	0.042	22.14
D	All control sites	81.98 (< 0.001)	35.88 (< 0.001)	4.19 (< 0.001)	1.86 (0.652)	279 (< 0.001)	0.262	22.58
E	All control sites	77.85 (< 0.001)	41.09 (< 0.001)	5.27 (< 0.001)	–4.84 (< 0.001)	378 (< 0.001)	0.325	22.41

The single-site analysis showed effects ranging from 4.0–5.0 fewer to 1.6 more ED attendances per night. The decrease in ED attendances at site A and increase in ED attendances at site B were statistically significant after Bonferroni correction.

The paired analysis with single control sites showed effects ranging from 4.0 fewer to 4.3 more ED attendances per night. The increases in ED attendances at sites B, E and F and the decreases in ED attendances at sites H and C were statistically significant after Bonferroni correction.

The paired analysis with pooled control sites showed effects ranging from 5.8 fewer ED attendances per night to 3.0 more ED attendances per night. The decreases in ED attendances at sites C and E were statistically significant after Bonferroni correction.

The overall pooled analysis across all sites showed markedly different results depending on whether all sites, the original sites (i.e. including sites D and E) or the replacements sites (i.e. including sites H and G) were used. The substantial variation between sites in the ED attendance rate also meant that the overall estimate across sites was subject to substantial uncertainty, with much less precise estimates than those for the individual sites. The estimated effect of implementing the AIMSs was 8.89 additional ED attendances per night (95% CI –14.44 to 32.22; p = 0.405) when all cities were used, 14.83 additional ED attendances per night (95% CI –6.91 to 36.58; p = 0.146) when the original cities were used and 0.59 additional ED attendances per night (95% CI –13.98 to 15.16; p = 0.924) when the replacement cities were used. Overall, there was no evidence of an effect from AIMSs on ED attendances, but CIs were wide.

Figure 8 shows paired comparisons between intervention and control sites of the rate of ED attendances per night over time. This provides further insight into the findings for individual sites.

FIGURE 8.

Graphs of ED attendance by intervention site and paired control site (dark blue, intervention site; light blue, control site). (a) Site E and control site; (b) site A and control site; (c) site G and control site; (d) site D and control site; (e) site C and control site; (f) site F and control site; (g) site H and control site; and (h) site B and control site.

Individual sites’ results are inconsistent owing to the noise in the data and violation of ITSA assumptions [paired analysis showed that there was a difference in ED attendances between intervention and control sites (Z) and a difference in the change in ED attendances over time between intervention and control sites (Zt)].

Figure 8 suggests that there is a decrease in ED attendances for site A post AIMS, which continues to decrease over time. This decrease was not reflected in the ITSA analysis but was observed in the single-city regression analysis. Other results for site A suggested an increase in ED attendances ranging between 2.5 and 4.0 ED attendances.

The site B AIMS was associated with variable effects across the analysis, ranging from four fewer to six more ED attendances per night, and had no obvious effect in the figure.

The site G AIMS was associated with increased ED attendances on ITSA but had no obvious effect in linear regression or in the figure. This suggests that matching site G with site M was inappropriate, at least in terms of ED attendances, because site G had much fewer ED attendances throughout the study.

The site F AIMS was associated with variable effects across the analyses, ranging from one fewer to four more ED attendances per night, and had no obvious effect in the figure. The site C AIMS also showed variable effects, being associated with one to three additional ED attendances on ITSA but two to seven fewer ED attendances on linear regression. The figure suggests no obvious explanation for this inconsistency.

The site D AIMS was associated with six to nine fewer ED attendances per night on ITSA but had no obvious effect in logistic regression or in the figure. The finding of a reduction in ED attendances on ITSA is inconsistent with the low level of activity we observed in the AIMS.

The site E AIMS had very inconsistent effects across the analysis and no obvious effect in the figure. We were unable to obtain any activity data from site E to determine whether or not activity would be consistent with an effect on ED attendances.

The lack of pre-intervention data for the site H AIMS meant that this could be assessed only in the linear regression, which suggested that it was associated with reduced ED attendances, ranging from four to five per night in the single-site analysis to one per night in the pooled analysis.

In summary, the findings suggest that the effects of AIMSs on ED attendance is variable, inconsistent and uncertain. Given that some of the ITSA assumptions are violated, further analysis using a simplified model is explored in Tables 21 and 22.

TABLE 21 - Emergency department attendance: random-effects generalised least squares regression – all sites

SE, standard error.

Difference: z = 0.42; p = 0.34.

Variable	All sites
Observations	8092
Groups	16
Observations per group
Minimum	289.0
Mean	505.8
Maximum	578.0
R 2
Within	0.046
Between	0.035
Overall	0.005
Variable	Coefficient	SE	z	p-value	95% CI
Attendance
AIMS	–4.991	2.271	–2.200	0.028	–9.443 to –0.540
Time	0.005	0.001	3.820	< 0.001	0.002 to 0.007
Constant	–12.611	23.027	–0.550	0.584	–57.743 to 32.521
sigma_u	25.420
sigma_e	11.777
Rho	0.823
Variable	dy/dx	SE	z	p-value	95% CI
Marginal effects
Control sites	0.005	0.001	3.430	0.001	0.002 to 0.007
Intervention sites	0.006	0.002	2.470	0.014	0.001 to 0.010

TABLE 22 - Emergency department attendance: random-effects generalised least squares regression – all sites, including sites G and H, excluding sites E and D

SE, standard error.

Includes sites G and H, excludes sites E and D.

Difference: z = 0.99; p = 0.16.

Variable	All sitesa
Observations	6358
Groups	12
Observations per group
Minimum	289.0
Mean	529.8
Maximum	578.0
R 2
Within	0.040
Between	0.000
Overall	0.010
Variable	Coefficient	SE	z	p-value	95% CI
Attendance
AIMS	–5.300	2.378	–2.230	0.026	–9.962 to –0.639
Time	0.004	0.002	2.660	0.008	0.001 to 0.007
Constant	–8.529	29.057	–0.290	0.769	–65.480 to 48.422
sigma_u	20.985
sigma_e	10.856
rho	0.789
Variable	dy/dx	SE	z	p-value	95% CI
Marginal effects
Control sites	0.004	0.002	2.230	0.026	0.000 to 0.007
Intervention sites	0.007	0.003	2.600	0.009	0.002 to 0.012

Key performance indicators

Total time in the ED is widely reported as the main key performance indicator for ED waiting times. 89,90 During the EDARA study the target for this measure was that 95% of attendances should spend no more than four hours in the ED. We therefore decided that total time in the ED should be the main analysis and should focus on whether the 4-hour target was achieved, as there is evidence that EDs manage patients (and thus influence total ED time) in relation to this target.

Data for ED-related key performance indicators revealed a number of inconsistencies, particularly between English and Welsh data. The English data reported time to initial assessment, time to treatment and total time in the ED; the Welsh data reported time to first clinical contact, time to treatment and time to completion of treatment. Examining the data suggested that time to completion of treatment was the most consistently recorded variable in the Welsh data and appeared to be consistent with total time in the ED in the English data. Therefore, we used time to completion of treatment in the Welsh data alongside total time in the ED in the English data to estimate the effect of the intervention on the proportion achieving the 4-hour target for total time in the ED.

Attributable to alcohol intoxication

Table 41 shows the mean number of admissions with AAI per day (starting at midnight) during hours of AIMS activity. This is shown for intervention sites before and after AIMS implementation and for control sites before and after AIMS implementation at the paired intervention site. Numbers of admissions were low, with admission rates varying from less than one admission per week to an admission every 1 or 2 days. There was no obvious trend towards admissions increasing or decreasing across intervention or control sites.

Interrupted time series analysis

Tables 42 and 43 show the results of ITSA using intervention-site-only analysis (see Table 42) and paired analysis with pooled control sites (see Table 43). Bonferroni correction across seven ITSA tests resulted in a p-value of ≤ 0.0142 for statistical significance. We were unable to fit a time series model for the paired analysis with single control sites because the data were not balanced. Caution should be exercised in the interpretation of ITSA results; some of the models have poor model fit (F-statistic). Autocorrelation was explored and, in all cases excluding site C, no lags were significant; therefore, autocorrelation was not adjusted for.

The intervention-site-only analysis suggested a significant increase in admissions after intervention at site E. The effects across other sites were non-significant trends towards both increases and decreases in admissions. The paired analysis with pooled control sites showed a non-significant increase in admissions at site E and a decrease at site F that was not significant with Bonferroni correction. Again, the effects across other sites were variable and non-significant.

Linear regression analysis (estimated using ordinary least squares)

Tables 44–46 show the results of linear regression analysis using intervention-site-only analysis (see Table 44), paired analysis with single control sites (see Table 45) and paired analysis with pooled control sites (see Table 46). Bonferroni correction across sites and eight tests resulted in a p-value of ≤ 0.0125 for statistical significance.