Text message intervention to reduce frequency of binge drinking among disadvantaged men: the TRAM RCT

Key findings from the feasibility study

Focus groups investigated men’s attitudes about taking part in a text message intervention and their beliefs about cutting down. 71 They gave insight into the target group’s patterns of drinking, and their knowledge about alcohol-related harms and the benefits of reduced drinking. The common pattern of drinking was periods of abstinence interspersed with infrequent heavy-drinking days. Many men believed that their drinking behaviours, motives and desire to change were significantly different from when they were younger. They had adopted the role of the ‘mature drinker’, which came with social roles and responsibilities (employee, husband/partner, parent). Most men were aware of the harms associated with alcohol misuse, but had low perceived personal risk. They made it clear that in an intervention they would not want to be preached at or told what to do.

The feasibility study also found that the text message brief intervention could engage participants86 and had the potential to reduce binge drinking. 71 Participants enjoyed the interactive nature of the intervention and gave carefully considered personal responses to the questions asked in the text messages. They engaged with the cognitive antecedents to reducing drinking as they were discussed in the text messages. Although the intervention addressed intention to change, it did not provide sufficient assistance on methods to achieve behaviour change and maintain the new behaviour. The intervention would need to be modified to incorporate a psychological theory that addresses volition as well as motivation to change behaviour. In addition, a period of 1 month would not be sufficient to address all of the steps required to change behaviour, so it should be extended to 3 months.

Psychological theory underpinning the intervention

The behaviour change strategy was based on the Health Action Process Approach (HAPA). 87 The HAPA model suggests that behaviour change occurs through two distinct phases: a pre-intentional or motivational phase and a volitional phase. The volitional phase is made up of separate components: a planning and action phase, and a maintenance phase. HAPA was chosen as it addresses the intention–behaviour gap,88 identified as a weakness in some behavioural change theories. Although HAPA was used as the overarching structure around which the intervention was developed, it drew on other social cognition models, such as subjective norms from the theory of planned behaviour,89 self-monitoring from social cognitive theory90 and control theory. 91 It also used empirical data from the feasibility study. Finally, behaviour change techniques suitable for use in alcohol interventions were incorporated. 92

Alcohol brief interventions

The first trial of a brief intervention for heavy alcohol use was conducted in 1962. 93 Since then, over 50 trials have been conducted in primary care and their results have been synthesised in 26 systematic reviews. 94 The consistent finding is that the interventions were effective for hazardous and harmful drinking in middle-aged men. However, there is insufficient evidence of effectiveness in women and in older and younger drinkers. Evidence from other settings is less persuasive. For example, in patients from emergency departments, trauma care centres, hospitals and community pharmacies, the evidence is mixed. 95,96

Despite the evidence on effectiveness, the mechanism of action of brief interventions is not known,97–99 in part because alcohol brief interventions are very heterogeneous. The time for the delivery of the interventions varies between 5 and 60 minutes, with a median of 25 minutes. 5 The content of the interventions also varies substantially. 5,94 They can contain motivational interviewing, feedback and advice, self-monitoring of alcohol consumption, self-help manuals, counselling and cognitive–behavioural therapy. 5,94,100 It is unclear whether or not interventions based on the principles of motivational interviewing may be more effective than other approaches. 97,99 One review of reviews has suggested that effective interventions contain at least two of three elements: feedback on drinking, advice and goal-setting. 101 A more recent review found that promoting self-monitoring was the only technique that appeared effective. 92 Bien et al. ,100 in their very widely cited paper, summarised the common components of effective brief interventions with the acronym FRAMES: Feedback on current consumption; Responsibility of the individual for his drinking; Advice to change; Menu of change strategies; Empathy in the delivery; and Self-efficacy for action.

Narratives in interventions

A narrative was used to engage the participants and to illustrate the key steps in the behaviour change process. Braddock and Price Dillard102 describe a narrative as ‘a cohesive, causally linked sequence of events that takes place in a dynamic world subject to conflict, transformation, and resolution through non-habitual, purposeful actions performed by character’. Narrative is increasingly being used as a tool for behaviour change. 103 Instead of presenting facts and arguments for changing behaviour, a narrative intervention translates these into actions and experiences of characters within a chronological series of events. 104 Narratives have been used in a range of health behaviour change studies (e.g. risks associated with sunbeds, vaccination for human papillomavirus, smoking cessation and alcohol use104), although none delivered through text messaging has been reported. In the present study, the narrative follows a central character, Dave, as he gradually realises that he drinks too much and decides to reduce his consumption. It charts his efforts, successes and relapses, and concludes with Dave enjoying the financial and social benefits of moderated drinking.

There are several advantages of using narrative. Information presented in a narrative has a stronger effect on knowledge, attitudes and intentions than the same information in a non-narrative format. 105 Narrative is particularly useful for changing perceived social norms and behavioural intentions. 106 It can ‘strengthen existing prosocial beliefs and behaviours as well as counteract unhealthy ones’. 107 Resistance to behaviour change is overcome because individuals are engaged in the narrative. 103

To be effective for behaviour change, the narrative and the characters in it have to engage the reader, a process aided if the protagonists are culturally similar to the target audience. 108,109 The narrative also has to be plausible and internally consistent. 110 The depiction of a character who succeeds against the odds can boost motivation for personal goals. 111

Logic model

A logic model was used to aid the development of the intervention (Table 1). Logic models help to clarify how the intervention will lead to behaviour change and, in the longer term, to improved health and social well-being. 112 A review of a problem (behaviour to be changed and the target group) clarifies what is to be achieved. It can also identify challenges, such as resistance of disadvantaged groups to conventional health-promotion interventions. The intervention strategy, the intervention goals and the outputs specify how the aim will be achieved. In this study, the initial requirements for behaviour change were an increased awareness and personal relevance of the harms of alcohol and the benefits of moderated drinking. Altering risk perception and alcohol expectancies are prerequisites for increased motivation to change. Setting goals and making action plans would lead to reduced drinking, but only if self-efficacy for action was also increased. Furthermore, to prevent relapse, reduced drinking would have to be maintained. This could be achieved by increasing the salience of the benefits of reduced drinking. Thus, the short-term benefits of moderated drinking can be used to encourage longer-term reductions. This will lead to improved health and social well-being for the individual and a reduction in the costs of alcohol-related harms for society.

Stage 1: establishing the provisional structure

Building on the logic model, key elements of the intervention were reviewed to create a provisional structure. These were the behaviour change model (HAPA)87 and relevant behaviour change techniques. 92 This provided a framework to establish how the narrative would depict the process of behaviour change, who the characters in the narrative would be and how the intervention could be delivered within the time frame of 3 months.

Stage 2: drafting text messages

The complete narrative was rendered in a series of 112 text messages, each with one or more of the following purposes:

• delivering the narrative (to engage participants)

• increasing the salience of the harms of heavy drinking and the benefits of moderated drinking

• modelling steps in the behaviour change process

• giving information or facts (to augment the behaviour change strategy portrayed in the narrative)

• asking questions (to monitor, in real time, participants’ reactions to the components of the intervention)

• comments from other characters (anonymised quotations from the feasibility study participants to reinforce the part of the intervention being delivered)

• adding humour (to increase engagement).

The salience of harms was increased by asking participants if they or their friends had experienced harms. This avoided the possibly patronising approach of telling experienced drinkers about harms with which they were already familiar. The feasibility study showed that most men were well aware of these harms. 71 It also increased the personal relevance of the harms. A similar approach was used with benefits, in which participants were asked to identify how their lives might be improved if they drank less.

The major constraint in writing the text messages was the permitted length of a text message (160 characters). Thus, the storyline had to be fairly simple and straightforward enough to be delivered in a few words. Although smartphones can send messages that are more than 160 characters (i.e. combining two or more messages at once), there is a danger that the participants will be deterred by large blocks of text.

The text messages were constructed so that the main character, Dave, appeared to be a recipient of the intervention. Thus, he commented on the text messages, answered questions and modelled behaviours that were expected from the behaviour change strategy. To simplify the narrative, Dave was the only character who sent messages, although he discussed at length what was happening in the lives of the other characters. The narrative had to be sufficiently engaging so that participants could remember what was happening from day to day. Achieving this with one or two text messages per day was challenging. Messages containing narrative were identified either by Dave introducing himself or signing off at the end of the message. When possible, messages about changing behaviour and advice about reducing drinking were modelled by the characters. This avoided didactic delivery of the intervention, which preliminary work in the feasibility study had found to be unwelcome.

Stage 3: revision of the text messages

Stage 4: piloting and final revisions

The final piloting used 24 volunteers who were given background information on the study and were told about the fictional characters in the narrative. They were given all of the text messages on paper and asked to provide written comments, both on the overall approach and on individual text messages. The volunteers commonly engaged with the characters and the narrative as if they were real by responding to the text messages rather than commenting on their content. The volunteers’ comments gave reassurance that the texts were clear and readily understood. They also showed that the use of characters made the intervention appear more realistic and less daunting. The volunteers also found the overall approach supportive. Finally, they suggested how the narrative could be amended: for example, one of the minor characters, Stevie, had been left with an unresolved drink problem. The narrative was amended to show that Stevie successfully tackled his drinking and also found a girlfriend.

A second round of piloting involved volunteers who received the text messages on their mobile phones. This was primarily used to test the delivery system, but also helped determine whether or not the frequency and timing of the messages were acceptable.

Study group

Men aged 25–44 years from areas of high deprivation were recruited. Recruitment was conducted in four centres that cover major regions of Scotland: Tayside, Glasgow, Forth Valley and Fife. Level of deprivation was measured using the SIMD,72 which is similar to the English Index of Multiple Deprivation. Men were recruited from areas classified as being in the most disadvantaged quintile. Recruitment was conducted from March 2014 to December 2014.

Inclusion/exclusion criteria

Men were included in the study if they had ≥ 2 episodes of binge drinking (> 8 units of alcohol in a single session) in the preceding 28 days. Exclusion criteria were as follows: men who were currently attending care at an alcohol problem service and men who would not be contactable by mobile phone for any part of the intervention period.

Techniques to promote recruitment

Recruitment employed several evidence-based techniques. 40–43 It involved direct personal contact (face to face or by telephone), involved multiple attempts at contact and used an approach based on respectful treatment. 122,123 An opt-out strategy was used. A financial incentive was offered for participating in the study. High-street vouchers to the total of £50 were offered, although this was divided across the whole study: £10 at completion of the baseline questionnaire, £20 during the delivery of the text messages and a further £10 for completion of the questionnaires at 3 months and 12 months post intervention. The incentive was given for continued participation and was not linked to drinking behaviour.

To ensure good coverage of disadvantaged men, two recruitment strategies were employed, each to recruit half of the target of 798 men. One used primary care registers and the other used a community outreach method, time–space sampling (TSS).

Strategy 1: recruitment through general practice registers

Potential participants were identified from the practice lists of 20 general practices by staff from the Scottish Primary Care Research Network (SPCRN). These lists contain data on age, sex and postcode. Postcodes were used to derive the SIMD score. 124 Men who lived in the highest deprivation quintile were randomly selected by SPCRN staff to give a maximum of 200 potential participants from each practice list. General practitioners (GPs) screened the list and sent potential participants a letter inviting them to take part (see Appendix 1). The letter was personally addressed, mentioned the appropriate local university (Dundee, Stirling or Glasgow Caledonian) and stated that a financial incentive would be given. The accompanying participant information sheet (see Appendix 2) carried the university’s logo and stressed the confidentiality of the study. An opt-out strategy was used for recruitment. The name, address and telephone number of those who did not decline to take part were provided to the researchers by the SPCRN. The researchers contacted these individuals by telephone approximately 2 weeks after the GP letter was sent. Attempts at contact by telephone were made at different times of the day and on different days of the week.

Strategy 2: time–space sampling

Time–space sampling125 is a community outreach strategy that recruits participants from a number of venues and involves sampling at different times of the day and on different days of the week. The specific features of the strategy were based on findings from the feasibility study,71 augmented by fieldwork, to identify appropriate venues and suitable times for recruitment. A variety of venues were explored for their potential for recruitment, including town centres, workplaces, community groups, football grounds, charities that support long-term unemployed people, supermarkets, housing associations and main shopping streets in disadvantaged areas.

Fieldwork

Areas classified by the SIMD as being in the most deprived quintile were identified from a government website. 126 Maps of high-deprivation areas within towns were produced using an online mapping resource (Google Maps^TM, Google, Inc., Mountain View, CA, USA). The maps were printed with sufficient detail to show street names and reference points to provide a detailed guide for fieldwork. Sources of potential participants were public houses, job centres, community centres, pharmacies, sports facilities, bookmakers and supermarkets. These were identified by using Google Street View^TM (Google, Inc., Mountain View, CA, USA) and from local authority websites. Google Street View was also employed to identify routes of access and car parking.

The initial fieldwork established that few people were encountered in housing estates in areas of high deprivation. Instead, they tended to congregate in high streets in, or adjacent to, areas of high deprivation, where facilities such as public houses, bookmakers, convenience stores and supermarkets are common. The fieldwork showed that recruiting from the streets around these venues was more productive than recruiting inside the venues. It also established that distributing leaflets and using gatekeepers were largely unproductive.

Initial screening

A researcher approached men in the selected areas who appeared to be in the age range (25–44 years). Potential participants were asked about their age and their current drinking levels. The study was described to those who reported binge drinking (> 8 units of alcohol in a single session) at least twice in the previous 4 weeks. All participants were told that the study was about alcohol and health. They were given a participant information sheet (see Appendix 2) and a consent form (see Appendix 3) to read, and their mobile phone number was obtained. About 24 hours after the potential participants received the participant information sheet and consent form, the researcher telephoned them to discuss the study and ascertain their eligibility by administering the screening questionnaire (see Appendix 4).

Informed consent

Informed consent was obtained by text message. This method was successfully used in the feasibility study. 71 Individuals who verbally agreed to take part were sent a text message asking if they understood what was involved and if they were willing to take part. Consent was obtained when the participant positively responded to the text message. These messages were stored electronically. In addition, the research assistant completed the consent form while interviewing the participant and signed and dated the form, including the time at which the text message giving consent was received.

Randomisation

The randomisation was carried out using the secure remote web-based system provided by the Tayside Clinical Trials Unit. Randomisation was stratified by participating centre and the recruitment method and restricted using block sizes of randomly varying lengths. The allocation ratio was 1 : 1, intervention to control.

Allocation concealment

The researchers appointed to carry out the recruitment enrolled the participants. The researchers entered key data items (mobile phone number, study identification number and preferred first name) into the web-based randomisation system. This system automatically assigned men to one of the treatment arms and subsequently delivered the appropriate set of text messages. The researchers who conducted the baseline and follow-up interviews had no access to this system and were unaware to which treatment group the men had been assigned.

Training

The importance of staff training was recently emphasised in a survey of UK clinical trials units. 59 The research assistants who carried out the recruitment and baseline data collection received a formal training programme, comprising three 2-hour sessions of didactic lectures, tutorial sessions and role play. These sessions covered the background to the study and the details of the recruitment strategies and data collection techniques. The need for a sensitive approach to recruitment, based on ‘respectful treatment’,122,123 was described. Researchers were encouraged to value potential participants and to thank them for listening to the outline of the study. In addition to the formal training, two further sessions were held at which progress towards recruitment targets and experience with recruitment techniques were reviewed. Ongoing mentoring formed an important part of training, during which researchers’ recruitment experiences, successes and failures were discussed.

The training on data collection covered the purposes of all of the data items, but focused on the measurement of alcohol consumption. The diversity of alcoholic beverages was described, highlighting how bottled and canned drinks with a specific brand name could vary in volume and strength. The training enabled the researchers to explore this diversity. Role play gave practice in eliciting accurate details of specific drinks consumed, from which detailed drinking histories were prepared. At initial sessions, those playing the role of the drinker were forthcoming with the details of their drinking, but became progressively more reticent in subsequent sessions. This provided those playing the role of the researcher with experience of the careful probing needed to elicit full details of alcohol consumption. Finally, researchers practised calculating the frequency of binge drinking, heavy binge drinking and total alcohol consumption from detailed drinking histories. After each episode of training, supportive feedback was given as part of a group discussion.

Measuring binge drinking

In this study, binge drinking is defined as > 8 UK units of alcohol in a single session. This criterion is widely used in national surveys in the UK. 127 It corresponds to > 64 g of ethanol. The measure used in the USA is ≥ 5 drinks in a single session, which amounts to ≥ 70 g of ethanol. 128 Thus, the definitions are similar but not identical. The study recorded the number of binge-drinking episodes over the 28 days before the interview. Other approaches, such as recording the amount consumed on the heaviest drinking day in the past week, have been criticised. 127

This study also uses > 16 units of alcohol as the threshold for heavy binge drinking to identify those who are consuming very large amounts of alcohol in a single session. There is increasing concern about those who consume sufficient alcohol to be at risk of serious acute adverse effects (e.g. blackout or poisoning). 129 This measure was obtained by doubling the level for binge drinking. The same approach has been proposed for the USA, giving a threshold of ≥ 10 drinks. 129

Baseline data collection

Alcohol consumption was measured by the methods described above. To minimise research participation effects, which could influence the impact of the intervention, the number of data collected at baseline was kept to a minimum. Recent systematic reviews134,135 indicate that baseline questions can lead participants to re-evaluate drinking behaviour. Thus, questions on topics such as knowledge of the harms of alcohol, or intentions to reduce consumption, were not asked.

Individual-level sociodemographic status was assessed using marital status, employment status and educational attainment. The participants’ postcodes were used to derive the SIMD score. In addition, a single question from the Fast Alcohol Screening Test (FAST)136 was used to determine whether or not participants suffered episodes of memory loss following drinking sessions.

Data collection at the first follow-up

The first follow-up was carried out 3 months post intervention (see Appendix 6). Data on alcohol consumption were collected, using the method described above. Two secondary outcomes were measured: the proportion of men consuming > 8 units of alcohol, and the proportion consuming > 16 units of alcohol, on ≥ 3 occasions in the previous 28 days. No other data were collected at this follow-up to minimise question behaviour effects. 134,135 Participants were also asked for any change of address to ensure that their gift vouchers would be correctly delivered. The researchers collecting the data were blind to treatment allocation.

Data collection at final follow-up

The final follow-up was carried out 12 months post intervention (see Appendix 7). The primary outcome and three secondary outcomes were measured at this follow-up. The primary outcome was the proportion of men consuming > 8 units of alcohol on ≥ 3 occasions in the previous 28 days. Two secondary outcomes of alcohol consumption were also measured at final follow-up: the proportion of men with ≥ 3 occasions of heavy binge drinking (> 16 units of alcohol in a session) and the mean alcohol consumption over 28 days. The method of collecting the data on alcohol consumption was described above. Finally, the AUDIT137 was used to determine the frequency of hazardous and harmful drinking. Questions were also asked about attempts to reduce drinking and the success of these attempts. Questions on self-efficacy for refusing drinks were taken from a validated questionnaire. 138 Health status was measured by the EuroQol-5 Dimensions, five-level version (EQ-5D-5L),139 a validated quality-of-life questionnaire designed to be simple to administer. Contacts with police/criminal justice, plus accident and emergency and other health-care usage, were measured by the widely used short Service Use Questionnaire (S Parrot, Department of Health Sciences, University of York, 2012 personal communication). Well-being was measured using the four Office for National Statistics Personal Well-being questions. 140 Finally, questions were asked about the acceptability of the study methods and recall of the text messages.

The sequence of questions in the follow-up questionnaire was arranged to obtain data on the primary and secondary outcomes first, followed by perceived changes in drinking over 12 months and the data for the economic evaluation, and, finally, participants’ views on the study methods (see Appendix 7). This ensured that questions on alcohol consumption were not influenced by responses to questions on other topics. It also meant that the most important data would be obtained if the interview was terminated early.

Methods for descriptive statistics

Binary variables (including primary and secondary outcomes as well as baseline binary variables) were summarised as number of observations, number of missing values, and number and percentage overall and per treatment group. Continuous variables (total alcohol consumption at 12 months post intervention and total alcohol consumption at baseline) were summarised as number of observations, number of missing values, mean, standard deviation (SD), standard error of the mean, median, and range overall and per treatment group.

Analysis of primary outcome

Logistic regression141 was used to investigate the effect of the intervention on the primary outcome, that is, whether or not the participant had consumed > 8 units of alcohol on ≥ 3 occasions in the previous 28 days at 12 months post intervention (proportion drinking > 8 units of alcohol on ≥ 3 occasions at 12 months). Three models were fitted to the primary outcome:

1. the unadjusted model (only treatment group as a fixed factor in the model)

2. the model adjusted for one baseline drinking variable (whether or not the participant had consumed > 8 units of alcohol on ≥ 3 occasions in the 28 days before the beginning of the study)

3. a full model adjusted for baseline drinking as for model 2 and the baseline covariates of method of recruitment (general practice registers/TSS), recruitment centre, age group, living with a partner (yes/no), employed (yes/no), further education (yes/no), SIMD score (1–10) and question 2 from the FAST. 136

Model 3 is considered the primary analysis and models 1 and 2 are presented for information. The treatment effect of the intervention on the primary outcome was the difference in proportions, or odds ratios (ORs), between the two groups with the 95% confidence interval (CI). Unadjusted treatment effect estimates were compared with corresponding estimates adjusted for baseline and other model covariates.

Analysis of secondary outcomes

The binary secondary outcomes (> 8 units of alcohol on ≥ 3 occasions at 3 months post intervention, > 16 units of alcohol on ≥ 3 occasions at 3 months and > 16 units of alcohol on ≥ 3 occasions at 12 months, and an AUDIT score of > 7 at 12 months post intervention) were analysed using the same analysis plan as for the primary outcome based on the logistic regression model. The adjustment for baseline drinking used the alcohol measure that was the equivalent of the secondary outcome (e.g. for the proportion drinking > 16 units of alcohol on ≥ 3 occasions at 12 months, the adjustment used the proportion drinking > 16 units of alcohol on ≥ 3 occasions at baseline). However, as the AUDIT was not administered at baseline, the adjusted models for the proportion with an AUDIT score of > 7 at 12 months used whether or not the participant had consumed > 8 units of alcohol on ≥ 3 occasions in the 28 days before the beginning of the study (> 8 units of alcohol on ≥ 3 occasions at baseline).

For total alcohol consumption at 12 months post intervention, owing to the skewness of the data, a generalised linear model assuming a gamma distribution and log-link function141 was used in the analysis. Again, three models were fitted as described above to this secondary outcome: the unadjusted model (only treatment group as a fixed factor in the model), the model adjusted for baseline (total alcohol consumption at baseline) and the full model adjusted for baseline and the other covariates listed above for the analysis of the primary outcome. The treatment effect of the intervention on this outcome was measured as the mean difference in consumption between treatment groups with the 95% CI.

Sensitivity analysis for missing data

Multiple imputation methods were used to assess the sensitivity of primary outcome results to missing data. Generalised linear models were used for multiple imputations, assuming that data were missing at random. 142 Multiple imputation included the explanatory variables used in the fully adjusted model above plus the primary and secondary outcomes. All primary and secondary outcome variables at baseline and at the 3-month and 12-month follow-ups were used, as was additional information collected at the 12-month follow-up interviews. This included demographic data at the 12-month follow-up, several questions on participant experiences in the study, and items from the Service Use Questionnaire, the EQ-5D-5L and the Office for National Statistics Personal Well-being questionnaire.

Economic evaluation

The methods for the economic evaluation are presented in Chapter 9.

Patient and public involvement

The trial design was informed by findings from the feasibility study, particularly from the focus groups with disadvantaged men. In addition, the pilot trial with disadvantaged men explored their views of the study design and conduct to identify ways in which these could be improved. Two user group representatives were involved throughout the development and conduct of the full RCT. They attended project meetings and assisted in developing the recruitment methods, reviewing the use of incentives, commenting on the data collection questionnaires and assessing the overall acceptability of the study methods. Volunteers also reviewed the text message intervention and the narrative on which it was based.

Interpretation of baseline drinking

The pattern of drinking was one dominated by binge-drinking sessions interspersed with periods of sobriety. Overall, the mean consumption of alcohol was lower than in many other studies of alcohol brief interventions. In this study, the average consumption over 28 days was 134 units, corresponding to 268 g of alcohol per week. A systematic review5 found that in studies among men, the average consumption was much higher, at 377 g per week. However, in the present study, the frequency of binge drinking, 1.65 times per week, was much higher than in the systematic review5 (0.9 times per week). Our findings support previous studies that have shown that although socioeconomically disadvantaged groups may not drink more on average, they are more likely to binge drink. 15–17

A striking finding is that 93% of the alcohol that the participants drank was consumed in binge-drinking sessions. This could partly explain why disadvantaged men, who do not drink more heavily than more affluent men, experience much higher rates of alcohol-related harm. 14,147,148 It also emphasises the need for interventions directed at binge drinking.

Assessment of the recruitment process

The recruitment target for this study, 798 men,121 was exceeded in a social group considered to be hard to reach. 45–49 Several factors may have contributed to the success of recruitment. These include the use of evidence-based strategies to improve recruitment: financial incentives, direct contact (face to face or telephone) and, for those on general practice registers, an opt-out approach and multiple attempts at contact. 40–43,149 In addition, the study was designed to require little effort from the participants, and common barriers, such as the requirement to travel or to attend a clinic, were avoided.

The study also used a sensitive approach based on ‘respectful treatment’122,123 in which the needs and views of potential participants take priority. This was encouraged through formal training sessions and informal, but regular, mentoring of the researchers who recruited participants. The recruiters recognised that potential participants could have a distrust of the research or concerns about the effort involved in participation. 50–53 Thus, men who were willing to listen to a description of the study were viewed as helping us, and the men who agreed to participate have done us a great favour. This perspective may have aided recruitment and helped researchers deal more easily with the disappointment of refusal.

The opening remarks may also have been important for recruitment. For community outreach, some phrases would often initiate a conversation: ‘Excuse me, would you happen to be in the age range 25–44?’ or ‘Excuse me, could you help me out?’ Other phrases were found to prompt immediate refusal: ‘Have you got a minute?’ and ‘Can I have a moment of your time?’ When contacting men recruited from general practice registers, it was helpful to mention the letter that had been sent out by their GP. This established credibility and may have eased concern that the call might be from a sales person. Once a relationship was established, a fuller description of the study was given, resolving any doubts or concerns that the men may have had.

Previous studies have found that recruitment by community outreach is often less successful than that using other strategies. 50 A recent study150 reported that community outreach was ineffective in recruiting socially disadvantaged smokers. The question then is why the method succeeded in the present study. A possible reason is the method and manner of the approach. Preliminary fieldwork was used to establish promising venues. The approach involved direct personal contact with potential participants. Methods such as using gatekeepers to identify potential participants and distributing leaflets proved ineffective. Direct face-to-face contact has been found to be more successful149 because it enables potential participants to receive an explanation about what to expect and can clarify misapprehensions.

Comparisons of the groups identified by the two recruitment strategies

A key finding from this study was the difference between the men according to recruitment strategy. Those recruited by community outreach had a higher mean consumption and a higher frequency of binge drinking. The difference in alcohol consumption patterns between the two recruitment strategies can be partly explained by sociodemographic factors such as marital and employment status. However, within each level of the demographic factors, there are marked differences in alcohol consumption between the men recruited by TSS and those recruited from general practice registers. Those recruited by community outreach had consistently higher alcohol consumption. Furthermore, there is a multiplicative effect on consumption of community outreach with being single or being unemployed.

The explanation for the relationship between recruitment method, sociodemographic factors and alcohol consumption is complex. It would be expected that unemployed men would be more likely to be recruited by community outreach, as this was mainly carried out during working hours, when men in employment would be working. However, this explanation does not easily explain why fewer single men, and fewer heavy drinkers, were recruited from general practice registers. Possibly a form of self-selection bias was in operation. Those who drink heavily are less likely to participate in surveys. 151–153 Non-responders to surveys are also more likely to suffer alcohol-related morbidity and mortality. 49,154 This bias could operate when men on general practice registers are contacted by telephone. In contrast, the face-to-face contact of community outreach could provide a more sympathetic and supportive approach, which could also emphasise the non-demanding nature of the study. This may help overcome reluctance of heavy drinkers to participate. It could also reduce the barriers that are experienced by disadvantaged groups, such as distrust of research, concern about confidentiality, fear of authority and lack of benefit from participation. 50–54

In summary, the recruitment strategies were successful for recruiting disadvantaged men. Those recruited frequently engaged in binge drinking but had few moderate drinking sessions. Thus, most of the alcohol consumed was drunk in binge-drinking sessions, which may place those men at high risk of alcohol-related harm. The community outreach approach may be a better method of recruiting hard-to-reach groups with adverse social circumstances. The methods used in this study could be usefully employed in future studies wanting to test interventions to reduce inequalities in health by tackling adverse health behaviours in disadvantaged groups.

Fidelity of delivery of the text messages

Short Message Service messages can be tracked to determine whether or not they have been delivered to the recipient’s mobile phone. The computer system that sent the text messages recorded and stored information on delivery status. Delivery status was recorded as delivered (the phone had reception and was switched on) or undelivered (the phone was switched off or it had no signal for 24 hours). The programme could not record whether or not messages delivered to the phones were opened by the recipients. The proportion of text messages recorded as delivered to the participants’ mobile phone was monitored as a measure of fidelity of delivery of the intervention.

Responses to the text messages

The computer package that delivered the messages also electronically stored text message responses received from participants. Participants were told that responding to the questions was voluntary, but also that they could send a response at any time to any of the messages received. All of the responses received were anonymised and collated by the Health Informatics Centre at the University of Dundee.

Although both treatment groups received messages that prompted a response, the purpose of the questions posed differed between the groups. For the intervention group, questions were designed to reinforce key components of the intervention and to monitor engagement with the psychological constructs of the intervention in real time. Twenty-one of the 112 messages sent to the intervention group sought a response. The purpose of questions in the control arm was simply to maintain interest in the study. Thus, 12 multiple-choice questions were posed, one per week on health topics. These questions did not require the participants to reflect on their own health behaviour in any way or to give a personalised response.

Engagement with the study and the behaviour change intervention was measured in seven ways.

• Intervention and control groups

  1. counts of responses to the text messages

  2. counts of the use of key words that demonstrate satisfaction with the text messages

  3. counts of words that indicate reflection on drinking

• Intervention group only

  1. counts of responses to the multiple-choice questions

  2. the nature of responses to the multiple-choice questions

  3. counts of responses to the open-ended questions

  4. the nature of responses to the open-ended questions.

Characteristics of the participants who responded to the text messages

To investigate the factors that influenced engagement with the study, the demographic characteristics of the participants and their baseline drinking were compared with the frequency of responding to text message questions. Participants were divided into three groups (low, medium and high responders), according to the number of questions they answered. For the intervention arm, who were asked 21 questions, the groups were 0–5 questions answered (low responders), 6–14 questions answered (medium responders) and 15–21 questions answered (high responders). Control participants were asked 12 questions, so the groups were 0–4 questions answered (low responders), 5–8 questions answered (medium responders) and 9–12 questions answered (high responders).

Fidelity of delivery of the text messages

The intervention package comprised 112 SMS text messages. Thus, a total of 46,032 messages were sent to the 411 participants during the intervention period. The control group participants received 89 text messages each, amounting to 36,846 messages sent out. Of the total of 82,878 messages sent to all of the participants, 79,218 messages (95.6%) were delivered to the participants’ phones. Fidelity of delivery was similar in both treatment groups. Participants in the intervention group received 95.5% of messages sent, while those in the control group received 95.7%.

A few (4.4%) of the 82,878 messages were recorded as undelivered (the phone was switched off or it had no signal for 24 hours). Two hundred and seventy-six men (33%) failed to receive one or more messages. Of those who missed messages, the number of undelivered messages per participant in the intervention group ranged from 1 to 112, with a median of 6 messages missed. For the control group, the range was 1–74 (median 3.5).

Participants who missed more than six messages were more likely to be single and unemployed (Table 8). They were also heavier drinkers at baseline than those who received all of the messages, and were more likely to have been recruited by TSS.

Responses to the text messages

Frequency of responses received to the text messages

Participants in both groups responded enthusiastically to the text messages. Responses were received from 92% of participants in the intervention group (380 men) and from 94% of participants in the control group (388 men). The total number of responses received from participants in the two arms of the study are not comparable because of the differences in the nature of the text messages. The intervention group received many more text messages that asked questions, the majority of which required free-text responses. In contrast, almost all of the questions sent to the control group were multiple choice and required only a single-letter response.

Intervention group

A total of 7481 responses were received to the 112 messages from the men in the intervention group. The number of responses per participant ranged from 0 to 81 (mean 18.2, median 17). Although 35% of the participants (142 men) responded > 20 times, another 33% responded ≤ 10 times (Figure 3).

FIGURE 3.

Intervention group: frequency of responses to text messages.

Control group

The control group received fewer text messages than the intervention group, and also received fewer messages that prompted a response. Thus, 5605 responses were received to the 89 messages sent out to each man (Figure 4). Although the range of responses received was similar to the intervention group, 0–77, the mean and median number of responses received was lower (mean 13.5, median 10.5 per man). Only 19% responded > 20 times.

FIGURE 4.

Control group: frequency of responses to text messages.

Responses to individual text messages

Intervention group

The intervention group messages were divided into four types (Figure 5). The blue bars represent responses to text messages that presented the narrative; the black bars represent responses to facts, supporting information or humour; the dark green bars represent the responses to messages that sought a response from participants; and the light green bars represent responses to messages that asked a question, but also provided an answer, such that a response was not expected.

FIGURE 5.

Intervention group: frequency of responses by type of text message.

The mean number of responses per man to messages that asked a question was 10.7 (median 11). The number of replies to the 91 messages that did not seek a response was slightly less (mean of 7.5 messages per man, median 3), but each of these messages received some responses. Many of these were responses to events occurring within the narrative, such as offering support and sympathy to characters when they were struggling to reduce their alcohol consumption.

There was some attenuation in the number of men answering questions over the 12-week period. This could be partly due to the nature of the questions. For example, some questions asked the men about specific actions such as setting a goal to reduce drinking (message number 56) or making a plan (message number 64). Participants who had not considered doing this may not have responded because they felt that the questions were not relevant.

The final bar on the histogram represents the 184 responses that were not related to any particular message. These included queries about gift vouchers, details on change of addresses or telephone numbers and general comments about the study or how the participant was feeling.

Control group

Two types of messages were sent to the control group (Figure 6): facts and trivia on health topics (black bars) and multiple-choice questions (green bars). The control group participants also demonstrated high levels of engagement with the study. The frequency of responding to the questions was lower than in the intervention group (mean 6.7, median 7). For the 77 text messages that did not ask a question, the number of responses was also lower than in the intervention group (mean of 6.9 messages per man, median 2). Apart from answering the questions, men commented on the health topics or responded to the humorous messages, often coming up with jokes of their own. The multiple-choice questions, one per week, did not seek the personalised responses required by the intervention group. Nevertheless, the questions were sufficient to sustain the men’s interest over the 3-month period. As in the intervention arm, the first question received more responses than any other question (283 responses from the 414 men). There was less variation in the number of responses received to the control text messages. Some attenuation was observed over the period, particularly over the last 3 weeks of the intervention. The topics being discussed during these 3 weeks [e.g. foot health (206 responses), tiredness (189 responses) and hearing (175 responses)] may have had less appeal than topics covered earlier [e.g. physical activity (260 responses), sexual health (253 responses) or oral health (263 responses)].

FIGURE 6.

Control group: frequency of responses by type of text message.

The final bar on the histogram shows the 251 responses from the control participants that were not related to any particular message. In addition to the queries about gift vouchers and details of change of addresses and telephone numbers, many men came up with more trivia and jokes on the health topics.

Engagement with the text messages

To investigate further the level of engagement with the text messages, a count of key words was performed on the database of responses received from the participants. The first group of words addresses general engagement with the study (Table 9). Phrases such as ‘lol’ or ‘haha’ are consistent with enjoyment and pleasure, while ‘thanks’ and ‘no problem’ suggest that the participants appreciated the study. The number of responses were similar in both groups. The role of the character Dave differed depending on the arm of the study. In the intervention group text messages, Dave was portrayed as a peer, who demonstrated changing his behaviour through a narrative. In the control text messages, Dave was much less prominent and did not have a storyline to support his role. He simply passed on interesting or humorous information to the participants, and many replied to him.

The second set of key words relate to alcohol and behaviour change (Table 10). Whereas hundreds of the responses from the intervention group mentioned alcohol and drinking, few from the control group did so. This is because the control group received no messages about alcohol or about changing health behaviours. The number of references to different types of alcohol reflected the drinks most commonly consumed by the men (i.e. beer, vodka and lager).

Intervention group responses to text messages that sought a response

Text messages that sought a response were an integral part of the behaviour change intervention. These questions were designed to reinforce key components of the intervention by getting the men to reflect on their drinking and think carefully about the antecedents to changing their behaviour. Reviewing the content of these responses gave an assessment of engagement with the psychological constructs of the intervention and key steps in the behaviour change process in real time.

The questions were distributed over the first 10 weeks of the intervention period, with a maximum of four questions being asked during 1 week. Table 11 presents the questions asked of the participants, the psychological constructs being addressed and the number of responses received from the participants. Questions during the first few weeks were designed to address the motivational phase of the HAPA. Thus, early text messages asked the men to self-monitor, record and report how much they usually drank. Questions in this phase also encouraged men to think about the pros and cons of their current level of drinking and to consider the potential benefits of cutting down. Text messages in the volitional phase helped the men to set goals and make action plans by first demonstrating how to do it and then gently asking the men what their own goal and plans would be. Maintenance of a new behaviour, which presents many challenges, is a key component of the HAPA. The second half of the intervention period encouraged maintenance by asking questions about doing alternative activities and making plans on how to cope in situations in which drinking would be expected. Self-efficacy, which is essential throughout the behaviour change process, was addressed in both phases of the intervention. Study participants demonstrated engagement with the messages from the beginning of the intervention period, with the first question receiving the greatest number of responses.

Five of the questions were multiple choice and these attracted the most responses from the men (Table 12). The first question was designed to identify the type of drinkers in the study, and it was based on the Drinking Motives Questionnaire,155 which categorises reasons for drinking as social, coping or enhancement. Some men gave several reasons for drinking, but the majority of men who selected a single option (220 men out of the 327 who responded) reported that they drank ‘to have fun’.

Message 36, sent out at the end of the third week of the intervention, was used to evaluate whether the participants had reflected on their own drinking patterns in response to the intervention or had thought about cutting down. Forty-four per cent of the 260 men who responded said that they had thought about cutting down, while 35% of respondents said that they had not.

Two questions [at the end of week 5 (message 59) and at the beginning of week 10 (message 84)] assessed action self-efficacy and recovery/coping self-efficacy, respectively. More than half of the men who responded to the message on action self-efficacy (51%) expressed very high confidence levels, while 20% said that they lacked confidence. The number of men who responded to the question on recovery self-efficacy was smaller, but 56% of those who did respond had very high confidence levels.

Message 97 asked men to report on actual changes to their drinking during the intervention period. Of the 205 men who responded, 20% had set a goal to reduce drinking and 20% had made a plan of how to achieve their goal. Twenty per cent of the respondents revealed that they had never thought about changing their drinking, while 39% stated that they had at least thought about changing.

The remaining 16 messages that sought responses were open-ended questions that required the participants to reflect on the part of the behaviour change strategy being addressed (see Table 11). Many of the responses gave carefully considered answers and contained personal details. Very often these responses were longer than the 160 characters that are permitted in one text message. To illustrate the nature of engagement with components of the behaviour change process, verbatim responses to selected questions are presented.

Characteristics of the participants who responded to the text messages

The demographic characteristics of the high, medium and low responders were explored for the intervention group and control group separately, before the treatment groups were compared. Men in the intervention group who were high responders (compared with medium and low responders) were more likely to be in the older age group (35–44 years); be living with a partner; be employed; have attended further education; be living in the most disadvantaged areas; and have been recruited through general practice registers (Table 13). The characteristics of the high responders in the control group were very similar, although the differences between the high and low responders were less marked than in the intervention group (Table 14).

Responses requiring attention

In addition to responding to the content of the intervention, participants sent messages if they needed to get in touch with the research team. They had been asked to tell the study team if they had a change of address or telephone number, or if they wanted to withdraw from the study. Most importantly, they could report adverse events or any distress they were experiencing. Thus, anonymised responses from the participants were screened daily by the trial manager. A member of the research team responded to 80 text messages received from participants. The most frequent queries were about gift vouchers (27 messages). A few men, particularly at the beginning of the intervention period, had some misperceptions or misunderstandings about the nature of the intervention and the content of the text messages. Thus, 15 men were contacted to provide further explanation of what was involved and to reassure them. Nine men sent messages indicating that they no longer wanted to receive the messages. All of these men were called and, following a discussion with the researcher, two were withdrawn from the study. The other seven were happy to remain in the study. Messages from six men suggested that they were distressed in some way. Two men indicated that they wanted to die and two reported having other psychiatric problems. Some also reported drug-related problems. All of these men responded to, and appreciated, telephone calls from the researchers. They were given information on where to seek help and all of them remained in the study. A further six men, by answering questions posed in the intervention, reported drinking very large amounts of alcohol. These men were advised to contact their GP to seek help for their alcohol problems. Finally, 17 men were contacted for a variety of other reasons. For example, some asked if their friends or partners could take part, others reported a change of address, problems with their mobile phones or that they were going on holiday.

Loss to follow-up

The drinking patterns at baseline among those lost to follow-up was explored to identify whether or not bias could have been introduced. Table 17 shows that those lost to follow-up in the two groups had very similar drinking patterns at baseline and none of the differences approached statistical significance. Although there are small differences between the treatment groups, these sometimes favour one group and sometimes favour the other.

The men who were not followed up differed slightly in their drinking patterns from those who were followed up (Table 18). Those not followed up were more likely to have ≥ 3 binge, and heavy binge, drinking sessions. Their mean alcohol consumption was also higher. However, although the direction of effect is consistent, all of these differences were small and none was statistically significant.

The men who were not followed up differed markedly from those followed up in terms of their demographic characteristics (Table 19). Thus, a substantially larger proportion of those lost to follow-up were single, were unemployed or had lower educational attainment. They were also, on average, younger and were much more likely to have been recruited by TSS. All of these differences were statistically significant.

Descriptive analysis of the primary and secondary outcomes measured at 12 months post intervention

Model fitting and treatment effects

Summary of the treatment effects

The primary outcome showed a modest, statistically non-significant benefit of the intervention, which translates to a 5.7% reduction in the frequency of binge drinking. This is approximately half of the hypothesised value (11%) that was used as the basis for the sample size calculation. A prespecified analysis showed that the net reduction was larger in men recruited from general practice registers (8.6%) than in those recruited by TSS (2.1%). One secondary outcome shows a small benefit of the intervention but others suggest a small benefit for the control. None of these effects is statistically significant. The treatment effects are clustered around the value for no effect of the intervention and all have wide CIs.

Other measures of the impact of the intervention

Four questions from a drinking refusal self-efficacy questionnaire162 were used to assess the participants’ perceived ability to resist drinking. These revealed high or moderately high perceived self-efficacy, but with small, non-significant and consistent differences in favour of the intervention group (Table 28).

In response to a further question about whether or not they had tried to reduce their drinking, significantly more men in the intervention group said that they had tried to reduce their alcohol consumption (63.7% vs. 55.5%; p = 0.027). Table 29 shows that men who reported trying to drink less had reduced their alcohol consumption much more than men who said that they had not tried. The effects are similar in the two groups, although the fall in the frequency of binge drinking is greater in the intervention group and the fall in mean consumption is greater in the control group.

Participant recall of the content of the intervention (or control) text messages was assessed by a series of open questions. No prompting of specific issues was given and participants could give no answer, one answer or more than one answer. In response to a general question about what they remembered, substantially more men in the intervention group remembered the characters in the text messages (Table 30). The control group were more likely to give no answer, to remember the texts that asked questions or to say simply that they liked the messages.

Participants were also asked whether or not they found the comments from the characters helpful. Slightly more men in the intervention group (231, 67%) than in the control group (211, 59%) responded ‘yes’. Among those making a positive response, more men in the intervention group cited engagement with the narrative or increased awareness of alcohol consumption as their reason (Table 31). In contrast, those in the control group were more likely to say that they enjoyed the study or that it made them think in general.

About three-quarters of the participants said that they had benefited from participating in the study, with almost exactly the same number in the intervention (275, 79.3%) and control (273, 76.5%) groups. However, more men in the intervention group reported as benefits an increased awareness of drinking or of having reduced or stopped drinking (Table 32).

The acceptability of the study methods was assessed by four indirect questions that assessed the participants’ experiences of the study (Table 33). Almost all participants enjoyed taking part and would recommend the study to other people, and about four-fifths discussed the study with friends or family. Fewer participants, about half, showed the text messages to others, possibly because text messages are considered personal. The intervention and control groups gave similar levels of response.

Adverse events

Three serious adverse events (deaths) occurred during the study. Two of the deaths occurred while the participants were in the intervention phase of the study and one occurred after the first follow-up interview had been completed. No information on the circumstances of the deaths were received. However, there was no reason to suspect that participation in the study could have contributed to the participants’ deaths in any way. All serious adverse events were reported to the ethics committee.

Retention

The retention rate of 86% was high, particularly for a study of disadvantaged men. It is much higher than that seen in most previous brief interventions. The Cochrane review of brief interventions in primary care by Kaner et al. 5 found that the average retention rate was 73%. Furthermore, in the present study, retention was almost identical in the two treatment arms, suggesting that loss to follow-up bias would be low. In contrast, conventional alcohol brief intervention trials show, on average, a significantly higher loss to follow-up in the intervention group. 5 Possibly the use of a financial incentive, £10 each for the two follow-up interviews, aided retention.

Acceptability of study methods

The high acceptability of the study methods is consistent with an overt aim of the study design: to make participation as easy and enjoyable as possible. The use of financial incentives may also have contributed to the acceptability. It seems likely that the high retention rate is due to this level of acceptability. This finding has obvious implications for other studies targeting socioeconomically disadvantaged groups: use incentives and make participation as simple, enjoyable and worthwhile as possible.

Recall of the text messages

Similar numbers of men in the intervention and control groups recalled the content of the text messages. However, the details of what was recalled differed greatly between the treatment arms: the intervention group remembered specifics about the content and characters in the text messages, and being made to think about their drinking. In contrast, the control group’s recollections tended to be phrased in general terms, most often saying that they liked the messages. The important point about the differences in recollection is that they were sustained at 12 months after the messages stopped, indicating the strength of the impact of the text messages.

Impact of participating in the study

There was some evidence that the intervention group had thought more about their drinking and were more likely to have attempted to reduce their consumption. These reports did not translate into lower consumption than in the control group.

An unexpected finding was that many men in the control group reported that participating in the study made them think about their alcohol intake. Although more men in the intervention group gave this response, the reports from the control group need an explanation, particularly because alcohol was not mentioned in the control text messages. Possibly the detailed questions on alcohol, which were asked at the beginning of the interview, influenced replies to later questions. The men may have been attempting to please the researchers conducting a study on alcohol. This is the phenomenon of demand characteristics,163 which holds that participants in studies often try to please researchers. If this did occur in the control group, then an effect of similar size might be expected in the intervention.

In summary, the intervention had a modest statistically non-significant effect on the primary outcome and small inconsistent and non-significant effects on the secondary outcomes. There was considerable uncertainty in the estimates of the treatment effect. Interventions with disadvantaged individuals commonly have much smaller treatment effects than studies in the general population. Further research is needed to reduce the uncertainty around the treatment effect.

The sustained nature of the fall

Any explanation(s) for the fall would need to account for the sustained nature of the fall: the mean falls from baseline at the 3-month and the 12-month follow-ups were almost identical. Furthermore, at these two follow-ups the overall levels of consumption were similar. Three trials of alcohol brief interventions have included two follow-up assessments at times comparable with those in this study. Curry et al. 164 and Reiff-Hekking et al. 165 showed large sustained falls in the frequency of binge drinking and in the mean consumption in both intervention and control groups; Richmond et al. 166 did not report on binge drinking but showed large sustained falls in mean consumption.

Several possible explanations for the sustained fall in this study were considered. These include regression to the mean,167 social desirability bias,168 research participation effects,169 demand characteristics,163 contemporaneous change in alcohol consumption and self-selection bias.

Regression to the mean

Individuals who are selected for a study because they have a high value on a measure (such as alcohol consumption) will, on average, have a lower value when measured on a subsequent occasion. 170–173 This phenomenon, regression to the mean, will affect all trials that use a high value of a measure (e.g. alcohol consumption) as an entry criterion. An empirical demonstration of this in a cohort study showed that the size of the fall in AUDIT score increased progressively as the threshold for entry was increased. 174

The present study selected men who had a least two binge-drinking sessions in the previous 28 days, so some fall in alcohol consumption would be expected. Furthermore, it would result in a sustained fall. Thus, at least some of the observed fall in alcohol consumption in the intervention and control groups will be due to this phenomenon. The question is how much of the reduction is due to it, and whether or not an additional explanation needs to be sought for the full size of the fall.

We replicated the empirical study of McCambridge et al. ,174 who investigated the potential effects of regression to the mean by setting different entry criteria. The replication used data from the first follow-up and the final follow-up and included only those men who completed both follow-ups (n = 687). Three types of entry criteria were applied to the participants at the first follow-up: no criterion, a minimum of two binge-drinking sessions in the previous 28 days (criterion 1) and a minimum of three binge-drinking sessions in the previous 28 days (criterion 2). This was carried out separately for two measures of alcohol consumption: the percentage of men with ≥ 3 occasions of binge drinking (> 8 units of alcohol in a session) and mean consumption over 28 days. With no entry requirement, there was little change in alcohol consumption between the first and second follow-ups (Table 34). The application of criterion 1 resulted in a fall in both measures of alcohol consumption. Criterion 2 produced much larger falls in the alcohol consumption measures. However, even these larger falls were not as great as those seen in this study between baseline and final follow-up: the actuals falls were –33.6% (proportion of men with ≥ 3 occasions of binge drinking) and –50.5 units of alcohol. To put this into context, criterion 1 was the one used for selecting men for entry into this trial. To the extent that this analysis reflects what happened at recruitment, it suggests that regression to the mean can explain only a part of the observed fall between baseline and final follow-up. This is consistent with the empirical study of McCambridge et al. ,174 which showed that regression to the mean has only modest effects unless a very high bar to entry is imposed.

Social desirability bias

Social desirability bias is the extent to which individuals report their behaviour, for example alcohol consumption, in a way that they think is socially desirable. 168 Participants report what they think is appropriate, acceptable or desired by others. Substantial under-reporting of alcohol consumption in surveys is widely recognised. Much of this is likely to be due to social desirability bias, although part could be due to the incomplete recall of consumption. Self-reported alcohol consumption provides estimates which are only 40–60% of those from data on alcohol sales. Recent data from four English speaking countries show that the survey data account for only between 31% and 57.5% of the alcohol sold. 175 The figure for Scotland is 54%. 176 However, it is unclear how under-reporting would produce the large fall seen from baseline to follow-up. Social desirability bias, or incomplete recall, might be expected to operate to a similar extent at baseline and at the follow-up assessments, and thus might not produce the consistent large falls over time. A more nuanced argument would be that social desirability bias could increase because of participation in the research. This possibility is explored in the next section.

Research participation effects

Concerns about the effects of research participation on the behaviour of study subjects are longstanding. 169,177 In this trial, several study factors could influence participant behaviour: screening and recruitment,178,179 the process of gaining informed consent180 and the administration of the baseline questionnaire. 134,181 In particular, the detailed alcohol history that was taken at baseline interview in this study could have had such an effect. Moos182 has outlined a mechanism by which this could occur: assessment could raise awareness of risky drinking, leading to self-monitoring and to the recognition that consumption is higher than would be wanted. This could increase motivation to change and, thus, lead to reduced drinking. However, a recent overview of reviews134 and a commentary paper183 concluded that the effects of research participation are small and inconsistent, suggesting that they are unlikely to explain the size of falls in consumption seen in this study. Furthermore, Moos182 has suggested that an effect of research participation is likely to be short term and, if so, could not explain the large sustained fall in consumption observed at the 12-month follow-up.

Demand characteristics

Demand characteristics could be viewed as a type of research participation effect or a special type of social desirability bias. It holds that participants in research studies may act in ways that they think will please the researchers. 163 If participants guess the purpose of a study, in this case to reduce alcohol consumption, they may under-report their alcohol consumption to please the researcher. 184 Possibly the magnitude of under-reporting may depend on the amount of benefit participants believe they gained from the research. In this study, where the aim was to reduce alcohol consumption, participant satisfaction was high. This may have encouraged participants to under-report their consumption to a greater extent at follow-up than at baseline. Under this argument, it might expected that the intervention group, who received many text messages about reducing consumption, would have a stronger motivation to under-report their consumption. As the intervention and control groups reported similar large falls in alcohol consumption, demand characteristics may not account for the observed results. McCambridge et al. 184 have also pointed out the paucity of research on this topic outside laboratory settings. Although demand characteristics may have a role, their contribution at present is purely theoretical. In addition, the magnitude of the effect might be expected to attenuate over time, as occurs with most behaviour change interventions. 113 No attenuation of effect was seen between the 3-month and the 12-month follow-ups, suggesting that demand characteristics might not explain the observed falls in consumption.

Contemporaneous changes in alcohol consumption

The baseline data were collected from March to December 2014 and the follow-up interviews were conducted during the period August 2014–February 2016. If national alcohol consumption had fallen from 2014, then part of the fall observed in this study could have been due to contemporaneous change. In fact, alcohol consumption in Scotland (measured by alcohol sales data and self-reports in surveys) fell from 2008 to 2013, rose slightly in 2014176 and rose again in 2015 to the level seen in 2011. 185

Self-selection bias

A very large, high-quality trial (Project MATCH)186 of three treatments for alcoholics found that all three produced very similar outcomes: approximately 30% of participants were abstinent at the 1-year follow-up, and this was sustained at 3 years. A more recent reanalysis of the original data raised the possibility of a different explanation for the abstinence rates. 187 It showed that almost all of the treatment effect had occurred by the first week of the planned 12-week treatment programme. Furthermore, a group of participants who dropped out before receiving any treatment experienced two-thirds to three-quarters of the improvement seen in those who completed the treatment plan. A possible explanation for these findings is that those who ‘decide to enter treatment are likely to reduce their drinking’. 187 In other words, those who are highly motivated to reduce their drinking are more likely to participate in trials. Although Project MATCH was conducted with people with alcoholism, a similar phenomenon could occur among hazardous drinkers. This would explain the large falls in consumption frequently observed in control groups of alcohol brief intervention trials. 135,161 Self-selection would also explain the sustained fall in consumption. At present, selection bias remains a hypothesis that needs to be tested in further research.

In summary, only regression to the mean is capable of explaining a sustained fall in alcohol consumption, but most probably it can only explain part of the observed fall. At present, there is insufficient evidence to conclude whether social desirability, research participation effects or self-selection bias played a role.

Recent studies on alcohol brief interventions

Evidence from two systematic reviews5,6 shows that alcohol brief interventions are effective, although the effects are small. The reviews found that the proportion of men who engaged in binge drinking (heavy drinking) was reduced by 11%5 and 12%. 6 In contrast, this study found that the net reduction in binge drinking was 5.7%. It also found that the reduction in binge drinking in men recruited through general practice registers was higher (8.6%, 95% CI –18.7% to 1.5%) than in those recruited by TSS (2.1%, 95% CI –13.5% to 9.4%). Both of the reviews5,6 concluded that there was no evidence that brief interventions are effective with dependent drinkers.

Since the publication of these reviews, five large trials7–10,188 of alcohol brief interventions, which found that the intervention was ineffective or possibly harmful, have been published. Three of these trials formed the SIPS (Screening and Intervention Programme for Sensible Drinking) research programme in which the intervention was evaluated in primary care,7 accident and emergency departments8 and criminal justice. 9 All three studies found that the brief intervention was not effective, as the primary outcome was similar in the active treatment and control groups. A separate study10 in primary care found that, at follow-up, the intervention group had a statistically significantly higher alcohol consumption than the control group. Finally, a complex study188 that addressed four risk behaviours (smoking, alcohol use, exercise and healthy eating) found no significant effect on alcohol use. The last of these did report that the intervention increased intentions to change, the number of attempts at change and perceived success in achieving change, but these did not result in measured behaviour change.

Text message studies to reduce alcohol consumption

Several recent studies have explored the use of text messages to tackle hazardous or harmful drinking. Two studies, one with vocational school students81 and the other with mandated college students,189 used an uncontrolled before-and-after design, making it difficult to draw conclusions about effectiveness. Four studies78,190–192 on text messaging were conducted with college students and none showed a significant effect on alcohol consumption. One study190 stated that there was no significant effect of the intervention; another study,192 which intended only to assess feasibility and acceptability, also reported no significant effect on alcohol. The third study191 reported a significant effect in a subgroup, and, finally, a very small study78 reported a significant effect on a secondary outcome measure (readiness to change drinking). A further, large study193 of university students, which compared two smartphone apps with a control group, found that one of the apps significantly increased alcohol consumption.

Text message studies have also been conducted outside the educational environment. One study76 of emergency department attendees claimed that a text message intervention can lead to small reductions in the self-reported number of heavy drinking days and in the number of drinks consumed per drinking day. However, the authors reported that there was differential loss of heavy drinkers at follow-up (more heavy drinkers were lost in the intervention group). After adjustment for the losses by multiple imputation, the intervention effect became non-significant. Finally, a pilot study82 on dependent drinkers found a statistically non-significant difference in favour of the intervention. In summary, although the technology appears to work well, there is a lack of convincing evidence that text messages are effective in reducing alcohol consumption.

Potential limitations of the study

Randomised controlled trials can suffer from problems of design and conduct, which impair their ability to detect the effect of interventions. The following section places particular attention on the biases identified in the Cochrane risk-of-bias tool194 and those for which there is strong evidence of bias. 195 It also follows the recommendation of the Cochrane group and considers those design features of a trial that could have contributed to the inconclusive findings.

The nature of the target group

Behaviour change interventions are less effective, or ineffective, in disadvantaged/low-income groups, although they can still have modest statistically significant effects. 20–22 Evaluations of smoking cessation interventions suggest that the lower effectiveness is due not to lower initial uptake, but to lower sustained compliance with the intervention. 28,29 Fear of being judged and fear of failure have also been identified as barriers for disadvantaged groups. 32

Possibly the social and physical environment in which the men lived could have overwhelmed their intentions to drink more moderately. There is convincing evidence that area characteristics influence health outcomes. 203–205 A review of interventions to promote smoking cessation found that there were more barriers to change for disadvantaged groups, particularly pro-smoking norms, additional cues to smoking and increased stress. 23 Qualitative research suggests that disadvantaged individuals who live in poorly resourced and stressful environments are isolated from wider social norms and have limited opportunities for respite and recreation. 31

Living in a disadvantaged neighbourhood could also lead to heavier alcohol consumption. 206 Poverty and poor neighbourhoods can increase stress,207 which in turn increases the frequency of binge drinking. 208 Low-income groups encounter a higher frequency of daily hassles209 and alcohol outlets are more common in disadvantaged neighbourhoods. 210 Stress and alcohol cues (such as the ready availability of alcohol) could increase motivation to drink. 211 More research is needed into the impact of these barriers on intervention effectiveness, and on strategies to overcome their effects. 34

An interesting finding from a prespecified analysis was that the intervention had a much larger effect in men recruited from general practice registers than in those recruited through TSS. The latter group were more likely to be unemployed and single and to have lower educational attainment. They were also more likely to be heavier drinkers. Thus, the finding of lower effectiveness in the TSS group is consistent with the explanation that stress and the hassles of daily living reduced the effectiveness of the intervention. If true, it raises the question of whether it would be better to intervene on men recruited from general practice registers, because effectiveness is higher, or whether the TSS group, whose need may be greater, should receive higher priority. As all of the men in the study are at high risk of alcohol-related harm, this a challenging ethical issue.

Costs

Effectiveness

Two effectiveness outcomes were considered separately in the cost-effectiveness analyses in the short term: (1) frequency of binge drinking at the 12-month follow-up and (2) QALYs within the 12 months post intervention. The impact of the intervention on these outcomes and the uncertainty around these were estimated using the same methodology as described in Chapter 3, adjusting for participants’ baseline characteristics. QALYs were estimated using the EQ-5D-5L235 utility scores reported at 12 months post intervention and applied to the whole 12-month period post intervention. EQ-5D-5L data at baseline and during the intervention were not collected to minimise potential contamination through changed behaviour if too many questions on health were asked. 134 QALYs for participants who died before the 12-month post-intervention follow-up were assumed to be zero for the whole 12-month period post intervention. The impact of the intervention on QALYs was estimated using a generalised linear model framework with an identity link function and Gaussian distribution, controlling for the same baseline characteristics as the primary analysis. The distribution family of Gaussian was chosen to accommodate the negative EQ-5D-5L utility states, as these negative states are believed to be valid and no other standard distribution reflected the distribution. As changes in drinking frequencies are unlikely to fully translate into health changes in the first year after the intervention, it was expected that any change in QALYs due to the intervention was likely to be small at the 12-month post-intervention follow-up. The trial was not powered to detect small QALY differences and thus it was expected that there would be large uncertainties around the 12-month post-intervention follow-up QALY results.

At nationwide rollout, the effectiveness outcomes for the programme were assumed to be the same as the equivalent trial population for the TSS and general practice recruitment methods. For the combined recruitment strategy, we assume that the effectiveness outcome at nationwide rollout is the weighted average of the effectiveness of the two recruitment methods estimated from the trial.

Incremental cost-effectiveness ratios

The incremental cost-effectiveness ratios, defined as the incremental cost divided by the incremental effectiveness, were estimated to present the additional cost that is required to achieve one additional unit of effectiveness (cost per one less person with ≥ 3 occasions of binge drinking and cost per QALY). The cost-effectiveness of adopting TSS and general practice registers recruitment strategies were presented jointly and then individually as subgroup analysis had shown that the cost-effectiveness results of the combined strategies at nationwide rollout are essentially driven by the cost-effectiveness of the different recruitment strategies and their projected coverage.

Modelling long-term costs and outcomes post 12-month follow-up

The long-term impact of the intervention on costs and QALYs was estimated with economic modelling based on the alcohol consumption observed at the 12-month follow-up. We adapted the model structure of an existing alcohol policy appraisal model, the Sheffield Alcohol Policy Model,219–221 to estimate the impact of alcohol consumption on population health. The Sheffield model utilises potential impact fractions and alcohol-attributable fractions to estimate the harm resulting from both the mean (total weekly units of alcohol) and the peak (units of alcohol consumed on the heaviest drinking day in the previous 7 days) alcohol consumption of individuals. 219 A range of acute and chronic health conditions and social harms (crime and workplace harms) attributable to alcohol were considered in the Sheffield model, and impacts on costs were quantified in monetary terms and for health in QALYs.

Based on the Sheffield model, the impact of the intervention was modelled from the 12-month follow-up for another 29 years to allow the full impact of the consumption change on outcomes to be assessed. Health risks were estimated in age subgroups (defined as 25–34, 35–44, 45–54, 55–64 and ≥ 75 years) based on the mean and peak alcohol consumption. The perspective of the model is that of the government (costs to NHS and social services) and the drinkers (quality of life), counting the costs of morbidity, mortality, social harms and health-related quality of life. All costs and QALYs were discounted at a rate of 3.5% based on NICE convention222 and costs were presented in 2016 prices.

Sensitivity analysis

One-way sensitivity analyses were conducted for four alternative scenarios at nationwide rollout for the two effectiveness outcomes.

Cost per one fewer person with ≥ 3 occasions of binge drinking

For the cost per one fewer person with ≥ 3 occasions of binge drinking, the sensitivity analysis results were compared with the base-case results at nationwide rollout (Table 39). The uncertainties of these results for each alternative scenario were highlighted using acceptability curves as presented in Figure 8.

TABLE 39 - Sensitivity analysis: short-term government cost per one fewer person with ≥ 3 occasions of binge drinking

AS, alternative scenario.

The treatment effect of the intervention was estimated to be.0.022 (standard error 0.057) for TSS method and 0.096 (standard error 0.052) for the general practice registers method.

Scenarios	Recruitment method (£)
	Combined strategy	TSS	General practice registers
Base-case scenario at nationwide rollout	4576	41,196	3311
Sensitivity analysis
AS1: when the programme manager’s time overseeing the project rises 10-fold	5833	41,410	3326
AS2: when only a percentage of the projected population were reached	10% reached: 5856 90% reached: 5819	10% reached: 41,762 90% reached: 41,203	10% reached: 3351 90% reached: 3312
AS3: lost to follow-up included using multiple imputation	4148	39,323a	2998a
AS4: 10% of control group reduction in binge drinking was due to the control intervention itself (rather than what would have happened with no intervention)	3148	13,864	2363

FIGURE 8.

Acceptability curves for the primary outcome for combined strategy at nationwide rollout, base case and AS 1, 2, 3 and 4: short-term cost to government. AS, alternative scenario.

In alternative scenarios 1 (costs were varied by increasing the programme manager’s time overseeing the project) and 2 (reducing the projected population the programme could reach), there were minimal changes in the estimated incremental cost-effectiveness ratios. In alternative scenario 3 (effectiveness outcomes were varied by including the multiple imputation results for people who were in the trial during the intervention period but not available for the 12-month follow-up), the incremental cost-effectiveness ratios were estimated to be lower. For the combined strategy, confidence that the programme is cost-effective in the short-term was estimated to be approximately 58% for a government willingness-to-pay threshold of £5000.

In alternative scenario 4 [a control intervention effect (compared with a true ‘do-nothing’ approach) of 10% was assumed for the decrease in the primary outcome between baseline and 12 months post intervention], the cost-effectiveness ratios were estimated to be lower at £3148 for the combined recruitment strategy than for the base case. Confidence that the programme is cost-effective at a £5000 government willingness-to-pay threshold would be 70% for the combined recruitment strategy in this case.

Short-term cost per quality-adjusted life-year gained

For alternative scenarios 1–3, the point estimates for the QALY analysis suggest that the intervention was dominated (more costly and less effective). Thus, it is redundant to estimate the incremental cost-effectiveness ratios. Alternative scenario 4, with a possible control intervention effect, cannot be estimated, as EQ-5D-5L data were not collected at baseline.

The acceptability curves for the alternative scenarios on short-term cost per QALY gained are presented in Figure 9. As £20,000–30,000 per QALY gained is typically considered cost-effective under the NICE technology assessment framework,244 the willingness-to-pay thresholds of £20,000 and £30,000 were plotted.

FIGURE 9.

Acceptability curves for QALYs for AS 1, 2 and 3: short-term cost to government. AS, alternative scenario.

As shown in Figure 10, when considering only the QALY gains to 12 months post intervention we are only 15% confident that the intervention would be cost-effective at a government willingness-to-pay threshold of £30,000 per QALY.

FIGURE 10.

Estimated distribution of consumption at the 12-month follow-up for control and intervention groups (by combined recruitment strategy and by each recruitment method individually). Mean weekly consumption for (a) combined strategy; (c) general practice registers; (e) TSS; and peak weekly consumption for (b) combined strategy; (d) general practice registers; and (f) TSS.

Modelling the long-term cost-effectiveness using the Sheffield model

Based on the 12-month follow-up trial data, the projected distribution of alcohol consumption for participants in the control and the intervention groups are presented in Figure 10. Individuals are ranked from the lightest drinkers to the heaviest drinkers on each consumption measure. The predictions for participants recruited by TSS and general practice recruitment methods are also presented separately below. As shown in Figure 10, with combined recruitment for more moderate drinkers at the 12-month follow-up, it was estimated that the intervention group would drink approximately 2 units fewer each week and 1 unit less in their peak session than if they were in the control group. But at the heavier drinking end of the population at 12-month follow-up it was estimated that there would be very little impact of the intervention on peak drinking levels. The combined recruitment results are heavily driven by the large number of participants projected to be recruited via general practices and we can see the similar consumption patterns in the general practice subgroup in Figure 10. For the TSS subgroup, the intervention mostly appeared to have very little impact on mean weekly consumption (although in the upper tail of heavy drinkers, the intervention group were estimated to have higher mean weekly drinking levels than the control group) and the intervention was also estimated to have increased peak consumption especially for the heaviest drinkers at 12 month follow-up. Given that there seems to be little change in mean weekly consumption but higher peak consumption, it could be that men are having fewer drinking sessions (fewer binge sessions) but more intense drinking sessions when they do drink.

These predicted drinking levels at 12 months were then combined with the assumption that any differences observed between the intervention and control groups at the 12-month follow-up would linearly decline over the next 7 years, such that after 7 years the intervention group would again be drinking at the same levels as the control group. The modelling assumption that the impact of the brief intervention on both mean and peak alcohol consumption declines in a linear fashion over a period of 7 years is demonstrated in Figure 11. In particular for the intervention group, it is assumed that the mean weekly consumption across the whole population increases from 13 units per week at the 12-month follow-up to the control group level of 15 units per week after 7 more years. Similarly, peak consumption for the intervention group increases from an estimated average of 8.5 units of alcohol in their heaviest drinking session in the previous week at the 12-month follow-up to the control group level of an average of 9 units after 7 more years.

FIGURE 11.

Linear decline of the intervention effect on average projected (a) mean weekly consumption and (b) peak weekly consumption in the previous week.

Next, these estimated mean and peak drinking distributions for both the intervention and the control groups were used as inputs in the model to project the likely differences in alcohol-related harm over the next 29 years. The model results combined with the short-term follow-up results are summarised in Table 40, and the incidences for the specific health and social harms from the model are detailed in Appendix 8 (see Table 50). Overall, the intervention was estimated to be dominated when the short-term follow-up and the model results were combined. Owing to the small difference in both mean and peak consumption at final 12-month follow-up between the intervention and control groups, only a small cost difference and utility benefit was estimated from year 1 to year 30 and the utility benefit was smaller than the estimated utility loss within the initial 12-month follow-up period.

Table 41 shows the modelling results and long-term cost-effectiveness by the general practice and TSS recruitment strategies separately. For the general practice recruitment method alone, the projected benefits in QALYs gained within the modelled period were still outweighed by the loss observed within the 12-month follow-up. Because the intervention was estimated to increase mean weekly consumption in the TSS group for the heaviest drinkers and to have increased peak consumption, the model showed further substantial losses in QALYs over the year 1 to year 30 horizon. Thus, the estimated cost-effectiveness of the intervention for the TSS group becomes even less favourable once we consider the potential long-term impact of the intervention.