Project objective

This systematic review has the following objectives:

• To assess the effectiveness of digital interventions in reducing hazardous and harmful alcohol consumption and/or alcohol-related problems.
• To specify interventions according to their mode of delivery (e.g. functionality features) and assess their impact on outcomes
• To identify theories, models, theoretical constructs and behaviour change techniques (BCTs) that have been used in the development and/or evaluation of the intervention and assess their association with outcomes
• To compare reported costs and outcomes achieved by the different modalities of digital alcohol interventions
• ­To consider the acceptability of digitally delivered alcohol interventions to help reduce levels of excessive drinking or alcohol-related problems
• To identify factors that promote engagement with digital interventions in different groups of excessive drinkers who have experienced them

Changes to project objectives.

We originally submitted a protocol covering a combined review of both outcome and engagement data linked to digital alcohol intervention to the Cochrane Collaboration (Drugs and Alcohol group).  However, we received feedback that we should register only the outcome focused work as a Cochrane review. Consequently, we split the project into two linked systematic reviews:

(i) a Cochrane systematic review[1] incorporating a meta-analysis, to assess the effectiveness of digital interventions at reducing alcohol consumption;

(ii) a Prospero registered systematic review of the literature on acceptability and engagement factors impacting on the potential use digital alcohol interventions [2]. As this process delayed our work, a time only extension was agreed with the SPHR executive covering the engagement aspects of the review. For this second review, we were also advised by SPHR to apply a ‘lighter touch’ approach in our synthesis of the engagement data. Thus we have focused on a thematic synthesis of qualitative research work exploring ‘end user’ views about acceptability issues and barrier and facilitators to digital alcohol intervention use, and provided a summary table of related quantitative studies

Brief summary

Outcomes Review

Methods:

The following research questions were addressed: (i) Are digital interventions superior to no intervention (or minimal input) controls? (ii) Are digital interventions at least as effective as face-to-face brief alcohol interventions? (iii) What are the effective component behaviour change techniques of digital alcohol interventions and their mechanisms of action? (iv) What theories or models have been used in the development and/or evaluation of digital alcohol interventions?    

Types of studies: randomised controlled trials.

Types of participants: Community-dwelling participants, whose alcohol consumption had been screened and who were directed to digital intervention including web-based, mobile phone text messaging, smart phone apps, social networking, or ‘standalone’ computer-based technologies (including CD-ROMs). Recruitment was via a range of settings, including primary care (including emergency departments and community midwifery services), social care, educational settings, workplaces or the internet. No restriction was applied to where participants interacted with the intervention, since it could be delivered through mobile devices.  Studies were excluded if participants were (i) mainly alcohol dependent (or in recovery); not screened for hazardous or harmful levels of consumption (e.g. reported drinking at least once in the last month); or under obligation to complete the intervention (e.g. mandated college students); (ii) interventions were focused on primary prevention; or directed at someone other than the drinker (e.g. a significant other or someone serving alcohol); or delivered in a secondary or tertiary care setting.

Types of interventions: digital interventions were defined as being delivered primarily through a programmable computer or mobile device (laptop, phone, or tablet), and were responsive to user input in order to generate personalised content which aimed to change the participants’ alcohol-related behaviours.  The comparator condition was typically no intervention (screening only), usual care (in a health or social care setting), or a face to face brief intervention to reduce alcohol consumption or harm. 

Exclusion criteria: if digital technology (i) was merely used to convey a talk-based intervention (e.g. a conversation by mobile phone); (ii) did not provide personalised feedback or specific, tailored input (e.g. generic educational interventions); (iii) was used to screen participants but then intervention was delivered face to face; (iv) was compared to another digital intervention with no control arm.

Types of outcome measures: the primary outcome measure was alcohol consumption, converted into grams of alcohol per week, and reported at 1 month or more. Secondary outcomes were: other consumption measures (e.g. number of binges or heavy episodic drinking, frequency of drinking, number of participants exceeding limits as defined by study authors), indices of alcohol related harm or social problems to the drinkers or affected others, cost effectiveness, any reported adverse effects.

Search methods for identification of studies: 15 bibliographic databases and relevant websites (table 1) were searched by an experienced information specialist, using terms for hazardous or harmful alcohol consumption combined with terms for computer assisted therapy or digital interventions.  The search was not limited by publication status, language or date.  Reference lists of all included studies and relevant reviews were checked for eligible studies.

Study selection:  two researchers independently screened all titles and abstracts identified using Endnote, and then reviewed the full text of selected studies. Data extraction forms were developed and piloted in Excel for outcome extraction and BCT and theory coding, and used by two researchers independently to extract data from all included studies (8 researchers extracted outcomes data and 2 extracted theory and BCT data). Discrepancies were resolved by a third researcher throughout. 

Data extraction: outcome data was extracted as follows: details of the intervention (e.g. setting, duration, mode of delivery and costs), participants (size and characteristics of sample), trial design (to allow critical appraisal), and baseline and follow-up consumption data (all reported follow-up points). 

Studies were coded for BCTs as follows: intervention descriptions were read line-by-line, text that may indicate the presence of a BCT was highlighted and highlighted text was compared to the definition for the BCT given in the taxonomy[3]. A BCT was coded as included only when it was explicitly present. Studies were coded for use of theory using the Theory Coding Scheme[4]. Each of the 17 relevant items from the Theory Coding Scheme[4] were dummy coded as present (1) or absent (0). The reliability of both methods was developed and assessed in iterative rounds of coding.  Two coders independently coded a sample of five studies. Coding differences were resolved through discussion and the coding manual was reviewed and updated in the light of these discussions. If agreement could not be reached, the views of a behaviour change expert were sought. Inter-rater reliability was assessed with the PABAK statistic[5], which assess agreement on both the presence and absence of codes. The level of agreement for the BCT analysis was 0.96 and for the theory analysis 0.84, which reflects a substantial level of agreement.

Assessment of risk of bias in included studies: risk of bias was assessed for the trials independently by two researchers using the criteria recommended in the Cochrane Handbook[6] which are designed to assess risk of selection, performance, detection, attrition, and reporting biases. The Drummond checklist was used to assess the quality of economic studies.

Measures of treatment effect: the mean difference in alcohol consumption between intervention and control was examined.  Where outcomes had been assessed at more than one time, data for each time point was extracted.  Procedures in the Cochrane Handbook [7] were followed for dealing with unit of analysis issues and missing data.  The magnitude of heterogeneity was assessed using the I² statistic, and the statistical significance of the heterogeneity was assessed using P values derived from Chi² tests[8]. Heterogeneity was explored both narratively and using subgroup and sensitivity analyses. 

Data synthesis – effectiveness:  the mean difference in alcohol consumption was pooled using a random-effects model in a meta-analysis using RevMan that compared intervention and control arms. The weighted mean difference method was used to estimate pooled effect sizes and 95% confidence intervals (CIs).

Data synthesis: BCTs: the revised metareg command in Stata was used to conduct a series of random effects univariate meta-regression models to assess the associations between individual BCTs and effect size. The regression coefficients (B) represented an estimate of effect (mean difference in quantity of drinking, expressed in grams per week) per unit increase in the covariate (dummy-coded as 1 = used the BCT or 0 = did not use the BCT). Only BCTs uniquely present in experimental arms, i.e. not present in both experimental and control arms, were included in analysis. A negative coefficient for a BCT indicated that studies using that BCT produced a larger pooled effect than studies that did not. The approach used in a previous meta-regression study of the BCTs contained within physical activity and healthy eating interventions was adopted[9] and to be included in analysis, each BCT had to be used in at least four separate studies.  To assess the independent association after mutual adjustment, a multivariable meta-regression model was created including all behaviour change techniques that had a meaningful association with effect in the univariate models (i.e., β >23 [10]).  The associations in this model were regarded as providing the primary indication of association between BCTs and effect.  To assess the association between the total number of BCTs included in experimental arms and effect size a random effects univariate meta-regression model was created. Lastly, we assessed the overall fit of a model, in terms of adjusted R², containing only a theoretically derived cluster of self-regulatory BCTs. Control Theory congruent BCTs were grouped into four categories: Goals, Self-monitoring, Feedback and Action plans. Trials were dummy-coded as 1 = used BCTs from three or four of these groupings; or 0 = used BCTs from two or fewer of these groupings.

Data synthesis – theory: frequency counts and descriptive statistics were used to describe which theories were mentioned, and which items of theory use and categories of theory are used most frequently.  A series of random effects univariate meta-regression analyses were conducted to examine the association between intervention effectiveness and (i) the 17 individual items (required to be included by at least 10% of studies), (ii) the six areas of theory use, and (iii) total use of theory.  A multivariate meta-regression analysis was conducted, including all of the variables in the univariate models that had a meaningful association of B>.10[10], to investigate independent associations between use of theory covariates and intervention effectiveness.  Any predictor was dropped if there were issues of collinearity (if a correlation greater than 0.90 then only the item that had the largest effect in the univariate model will be included in the multivariate model).  The meta-regressions was conducted in STATA using the metareg command. In these analyses, the regression coefficient (B) is the estimated increase in the effect size per unit increase in the covariate(s)) and the p-value indicates whether the B weight is significantly different from zero. The adjusted R² value is reported to assess the proportion of between-study variance explained by each predictor.

Sensitivity analysis were conducted investigating the effect of omitting studies with a high risk of bias.

Results (including findings in relation to the objectives):

Characteristics of included trials: Fifty five RCTs (reported in 53 papers) were included in which a total of 33,899 participants were randomised (figure 1), including reasons for exclusion from meta-analysis).  Thirty studies considered teens, younger adults and/or students; the remaining studies were specific to adults. Thirty five studies were based on interventions delivered completely online, whilst in 19 studies the intervention was location specific. Follow-up times ranged from 1 to 24 months (median: 3 months).  Fifty four trials compared a digital intervention to a control arm, of which 40 reported appropriate information to be included in the primary meta-analysis.  Seven trials compared a digital intervention to a face to face intervention, of which five reported appropriate information to be included in the meta-analysis.  Six trials had a digital, face to face, and control arm.

Methodological quality of the trials: Many trials were judged to have a low risk of selection and performance bias because randomisation, allocation and administration of the intervention was automated. The main source of bias was due to attrition; follow-up was challenging because there was often little face to face contact between triallists and participants.  Trials were judged to be at high risk of attrition bias if loss to follow-up was >30% or if it was different between arms with no explanation to account for this; 19 (35%) trials were judged to be at high risk of attrition bias.

Effectiveness of digital interventions compared to control: the 40 trials in the primary meta-analysis (one of which contained two digital intervention arms that were analysed separately) included 19026 participants (9526 randomised to a digital intervention, 9500 randomised to a control condition).  The longest period of follow-up was from one month (n=8) to 12 months (n=7).  Overall, participants randomised to a digital intervention drank 23.6 (95% CI 16.0, 31.2) grams of alcohol per week (~ 3 standard drink units) less than controls (figure 2).  There was considerable heterogeneity in the estimate of the effect size between trials (I²=78%), although in only four trials was the quantity of drinking greater in the digital intervention arm than in the control arm.  The associated funnel plot indicates evidence of asymmetry (figure 3), suggesting under-reporting of results showing little or no evidence of an effect of digital intervention on alcohol consumption.  A smaller but still statistically significant reduction in consumption was demonstrated in a sensitivity analysis excluding 13 studies at high risk of bias.  The reduction appeared to be sustained across lengthening follow-up, although it was not statistically significant by 12 months (figure 4). Of the 40 trials in the main meta-analysis, 26 trials (one of which had two digital intervention arms) were solely of adolescents, young adults or college students.  There was a statistically significant, although smaller, reduction in consumption in this population of -14.0 grams per week (95% CI -19.9, -8.1); which was just under two UK standard drink units per week (figure 5).  Also, the findings for this analysis were less heterogeneous (I²=52%) than those from the main analysis.  Five trials provided information on alcohol consumption by gender.  There was no evidence from these trials that the difference in alcohol consumption between trial arms was modified by gender, but the available data were limited.

Effectiveness of digital interventions compared to face to face interventions: only five trials reported a direct comparison between a digital and face-to-face intervention.  There was no evidence from these trials of a difference in alcohol consumption between these arms.  However, the numbers of participants in this analysis were small (less than 200 in each arm).

Cost effectiveness: included were four economic evaluations which ran alongside clinical trials, one stand-alone modelling evaluation and two studies focused on measuring costs and benefits. This work was based in the US, UK or Netherlands, and assessed a variety of different types of interventions.  Four studies were rated as high quality, two as moderate quality, and one of weak quality; the main issues bringing down study ratings were insufficient economic analysis or intervention description.  Four out of five studies investigating cost effectiveness, reported the intervention was cost-effective.  There was no evidence to suggest that the length of the intervention or the specific type of digital intervention impacted on cost-effectiveness.  However, the sample size was too small to draw any definitive conclusions on the cost-effectiveness by type of digital interventions. 

Use of and associations with component BCTs:  The considerable heterogeneity described above (I²=78%) can potentially be explained by study-level covariates. Of the BCTs uniquely present in experimental arms, i.e. not present in both experimental and control arms, the five most frequently used were: ‘Feedback on behaviour’ (82.9%, n=34), ‘Social comparison’ (80.5%, n=33), ‘Information about social and environmental consequences’ (70.7%, n=29) ‘Feedback on outcomes of behaviour (65.9%, n=27) and ‘Social support (unspecified)’ (65.9%, n=27) (table 2). Of the ninety-three possible BCTs that could have been used, 15 were used in more than 20% of trials, 44 were used at least once and 49 were never used. The mean number of BCTs used in experimental arms was 9.1 (SD=5.3), the median was 9 and the range 1-21.  The BCTs of ‘Goal setting’ (B:-43.46, SE: 17.40, p=0.02, 95% CI: -78.66 to -8.27, R²_adj: 6.05%), ‘Problem solving’ (B: -47.67, SE: 15.00, p<0.01, 95% CI: -78.01 to -17.33, R²_adj: 24.01%), ‘Information about antecedents’ (B: -73.77, SE: 21.76, p= p<0.01, 95% CI: -117.79 to 29.75, R²_adj: 31.93%), ‘Behaviour substitution’ (B: -123.02, SE: 30.29, p<0.001, 95% CI: -184.28 to -61.76, R²_adj: 48.81%) and ‘Credible source’ (B:-39.91, SE: 16.50, p=0.02, 95% CI: -72.68 to -5.94, R²_adj: 14.49%) were significantly associated with reduced alcohol consumption in univariate models (table 3).  In a multivariate model that included BCTs with a B > 23 in the univariate model, the BCTs of ‘Behaviour substitution’ (B: -101.70, SE: 34.50, p< 0.01, 95% CI: -172.61 to -30.79) and ‘Credible source’ (B:-31.19, SE: 14.35, p=0.04, 95% CI: -60.70 to -1.69) were significantly associated with reduced alcohol consumption (table 4). The multivariate meta-regression model produced relatively good indices of fit and substantially reduced heterogeneity (I²: 66.72%, R²_adj: 59.19%, p<0.01) compared to the I² heterogeneity of 78.0% from the main outcomes.

Use of and associations with theory: the most frequently reported theory items were: a theory or model mentioned or cited (n= 20, 49%); theory or theoretical predictors used to select or develop intervention techniques (n=16, 39%); and targeted constructs were mentioned as predictors of behaviour (n=16, 39%) (table 5). Twenty-one interventions (51%) made no mention of theory at all, including any reference to the use of theory in either the design or evaluation of the intervention (table 5). Only two studies used theory to select recipients or tailor the intervention. No intervention used a theory or predictors to select recipients for the intervention, or to refine the theory, either by adding or removing theoretical constructs or by specifying that the interrelationships between theoretical constructs should be changed (table 5).  The most frequently mentioned theories or models were Motivational Interviewing (7/20), Transtheoretical model (6/20) and Social Norms Theory (6/20) (table 6).  Sixteen different theories or models were mentioned. The mean total use of theory score was 4.3 out of a possible 22 which indicates that studies are not using and/or reporting theory use in intervention development and evaluation (table 7). The considerable heterogeneity described above (I2=78%) can potentially be explained by study-level covariates. Univariate meta-regression analyses (table 8) were used to explore the relationship between reported theory use and intervention effectiveness. Items 4, 6, 9, 14b, 17a, 17b and category 6 were not included as the item was not present in more than 10% of the included trials. The results indicated that the Theory Coding Scheme items, categories of theory use and total use of theory score explained little of the heterogeneity. None of the Theory Coding Scheme items, categories of theory use or total use of theory score were significantly associated with intervention effectiveness (p>.076). A multivariate model was conducted entering the covariates (item 8 and item 13) that had a modest (albeit non-significant) association with effect size (B>23) in the univariate models (table 9). The results indicated no independent significant association between intervention effectiveness and Theory Coding Scheme items; ‘at least one of the intervention techniques are explicitly linked to at least one theory-relevant construct/predictor’ (item 8, B=27.97, p=.061) and ‘changes in measured theory-relevant constructs/predictor’ (item 13, B=-32.46, p=.061) (table 9).  Moreover, the multivariate model had little effect on the degree of heterogeneity identified in the primary analysis (I²=74%, adjusted R²=27%). Whilst there is no evidence that reporting of theory use is associated with the substantial heterogeneity between interventions, the current literature is insensitive to addressing the more general question of whether good quality use of theory in designing digital alcohol interventions may be associated with more effective interventions. Previous simulation studies have found that over 200 studies are required for 80% power[11]. Therefore we cannot draw a conclusion as to whether an association did not exist or whether there was insufficient power to detect one.

Conclusions:

The primary outcome meta-analysis suggests that digital interventions were superior to either no intervention or minimal input controls, and that they led to a reduction in alcohol consumption of approximately 3 (UK) standard drink units per week.  There was considerable heterogeneity in these findings, although this heterogeneity was lower amongst trials of adolescents, young adults or college students. There is no evidence from this work of a difference in effectiveness between digital and face to face interventions, although there was a paucity of evidence directly addressing this comparison.  Limited available economic evidence suggests that digital interventions are cost effective.  No evidence was found to suggest that this is dependent on the duration or type of intervention, but the evidence base may be too small to draw definitive conclusions. The BCTs of ‘Behaviour substitution’ and “Credible source’ are associated with the effectiveness of digital interventions to reduce alcohol consumption and warrant further investigation in an experimental context.  The small number of interventions available for analysis and the infrequent use of many BCTs (70 of the 93 were used less than four times), meant that the effects of most BCTs could not be evaluated. Other BCTs were used so frequently (‘Feedback on behaviour’ and ‘Social comparison’ were both present in more than 80% of trials) as to reduce the ability to determine their effectiveness.  Recognised issues with the incomplete reporting of intervention content[12] may have resulted in BCTs being incorrectly coded as present or absent. This may produce noise and undermine the power to test associations.  Simply recording a BCT as present or absent does not take into account the frequency, intensity or the quality in which it was delivered. Greater understanding the ‘dose’[13] of a BCT is needed in order to fully evaluate its effectiveness. The reporting of theory use in the development or evaluation of digital alcohol interventions is very limited and/or unclear. Over half of all interventions made no reference to any theories of behaviour and over a third used them to develop the intervention. The limited reporting of theory use is currently unable to explain the substantial heterogeneity in this literature. The current literature is insensitive to addressing the more general question of whether good quality use of theory in designing digital alcohol interventions may be associated with more effective interventions.

 Engagement review

Methods:

This engagement focused review considered for inclusion, studies which reported users’ experiences of digital interventions to reduce hazardous and harmful alcohol consumption, or assessed their views about acceptability, ease of use and/or adherence to the interventions.

Population of interest: people living in the community whose views on digital interventions for reducing hazardous or harmful alcohol consumption were reported.  Participant reported views concerned either an abstract idea of technological interventions to reduce alcohol consumption, or they had used and were expressing views on a particular intervention.  Studies were excluded if participants were mostly alcohol dependent, or if the digital intervention was targeted towards dependent drinkers.

Definition of digital interventions: Digital interventions were defined for the purposes of this review as being delivered primarily online or through a programmable computer or mobile device (e.g. laptop, phone, or tablet), responding to user input and generating personalised content which aimed to change the participants’ alcohol-related behaviours.   

Outcomes: any outcomes were accepted which described or could be a proxy for engagement with or adherence to the interventions, e.g. acceptability, satisfaction, user preferences, number of times  participants logged on or duration of use.

Study designs: we did not restrict the search by study design, but we separated qualitative from quantitative designs at the synthesis stage. 

Search methods for identification of studies: 15 bibliographic databases and relevant websites (table 1) were searched by an experienced information specialist, using terms for hazardous or harmful alcohol consumption combined with terms for computer assisted therapy or digital interventions.  The search was not limited by publication status, language or date.  Reference lists of all included studies and relevant reviews were checked for further eligible studies. The search and screening/assessment for this review and the previously reported effectiveness review took place together.

Study selection and data extraction:  two reviewers independently screened all titles and abstracts identified using Endnote, and then reviewed the full text of selected studies. Disagreements at these stages were resolved by a third reviewer (FB/EK).  A data extraction form was developed in Word for the qualitative studies.  One reviewer (GM) extracted relevant data (citation details, aim, location, study design, participant characteristics and sampling, theoretical approach, data collection, results, conclusions) for all studies, and another (FB) read each study thoroughly and added to the data extraction as appropriate, or noted points of disagreement for discussion. 

Quality assessment of studies:  Questions adapted from the Downe and Walsh[14] quality criteria for qualitative research were appended to the data extraction form.  One reviewer (FB) carried out the quality assessment for all studies. Quality assessments were later discussed with another reviewer (GM) and discrepancies were resolved by discussion between two reviewers (FB/GM). Studies were assigned High, Medium or Low reporting, reflecting on the quality of reporting as related to the detail on the methods employed, presentation and interpretation of the results (High=Very good reporting; Medium=Good reporting and Low=Limited reporting).

Data synthesis- qualitative studies: thematic synthesis was used to synthesise the results of qualitative studies.  One reviewer (GM) allocated initial codings, these were then discussed at length with a second reviewer (FB).  Direct quotes were labelled separately from study author description and interpretation.  In the first instance codes were generated inductively from the data; as a second step reviewers explored how themes fitted FRAMES, a model which is used to structure face to face brief alcohol interventions (Feedback, Responsibility given to participant for change, Advice, Menu of options, Empathy in style, Self efficacy promoted in participant to achieve change). 

Data synthesis – quantitative studies: studies were first tabulated according to study design and outcome. We then mapped them to the following categories: formative/feasibility work, surveys, and data allowing remote profiling and/or adherence of participants.

Results – qualitative synthesis:

Characteristics of included qualitative studies: We identified 14 qualitative studies that explored drinkers’ engagement with digitally delivered alcohol interventions for reducing hazardous or harmful alcohol consumption.  Of these studies, six[15-20] focused on students or younger people; two [21, 22] recruited patients in primary care practices; two studies[23, 24] targeted particular minority or ethnic groups (American Indian women and Latino people respectively); one study[25] explored the experiences of veterans from the armed forces; one study[26] recruited pregnant women from prenatal clinics; and one study[27] explored disadvantaged men’s experiences of a digital intervention delivered by mobile phone.  Six studies[19, 20, 23-26] took place in the USA, four[15, 18, 22, 27] in the UK, two[17, 28] in Australia, one[21] in New Zealand, and one[16] in Belgium.

Quality assessment: ten studies[15, 16, 18, 19, 21, 23-25, 27, 28] were considered to be core studies that provided rich data for the thematic analysis and four studies[17, 20, 22, 26] were considered “thin” studies which were eligible but provided much less data.  Most of the core studies were judged to be of high (n=6) or medium (n=3) quality overall, although none of them discussed researcher reflexivity and the potential impact of the researcher in the study.  Only one of the core studies[27] was considered to be of low quality due to very little detail in the reporting of the methods. 

Thematic synthesis of qualitative studies: We identified three interrelated factors that appeared to shape users’ engagement with digital alcohol interventions to reduce hazardous or harmful drinking. These were related to i) the inherent technical/functional capabilities and characteristics/attributes of the technology; ii) the specific components of the intervention as mapped onto the FRAMES ingredients for face to face interventions; iii) and the ‘fit’ between users’ perceptions/values/beliefs and the components of the intervention.  The following diagram illustrates these interrelated aspects:

Technical aspects

In terms of technology itself, there were a number of technical factors shaping users engagement with digital alcohol interventions such as the appearance of interface, navigation around intervention, ease of use, clear presentation of information; ability to leave and return to intervention at will and the option for self-reminders. Regarding some of the technical factors shaping users engagement with digital interventions participants commented:

 “Easy to follow. Didn’t take very long. Very easy to use” (Patient [P]016). (Goodyear-Smith 2013) A5

“I like colour… when it is very clinically written, I just sit back. But when it is colour and it gives a genuine feeling of welcome, then I wanted to have a look...” (Kay-Lambkin et al 2011) A1

“It was short, simple, to the point, which is always easiest with military people - very impatient, just get to the point.” (Veteran G) (Lapham et al 2012) A3

Intervention components

User engagement with digital interventions also depended on the specific functions or content of the intervention. Users responded well to components of the intervention that reflected the underlying philosophy and structure of face to face brief alcohol interventions as per the FRAMES approach. In particular, participants considered the following elements of digital interventions important:

•  Feedback: as well as weekly consumption information, participants noted the following types of feedback: alcohol calories, estimated money spent, peak BAC, hours of exercise. Some people thought information presented in terms of standard drinks or units was useful although others found it confusing.
• Normative feedback: this compared participants’ alcohol consumption to those of their peers; some interventions sought participants’ views of their peers’ drinking.
• Flagging personal Responsibility: interventions seen as helpful if they gave participants the opportunity for self-realisation and reflection and the motivation to change their behaviour, often raising awareness about health and other risks.
• Empathy: participants noted in several studies that they preferred non-judgemental approaches regarding advice or information about alcohol consumption or risks linked to drinking behaviour.

 Some illustrative comments were:

“I remember the blood alcohol content meter thing, you type in numbers and it tells you like what your blood alcohol content would be” (Nygaard and Paschall 2012)

“I was appalled at how much calories I had consumed... it was just like ‘Oh, geez.’” (Veteran G) (Lapham et al 2012)

“It was cool. I was like ‘Whoa, it shows, like, how much...food and stuff, like, it could be equalled out to’... I’d rather have a cheeseburger, obviously.” (Lapham et al 2012) C1

“You think to yourself ‘I don’t drink too much’ and ‘it causes no harm,’ but then it appears that it does...” (Fraeyman et al 2012) D1 D4

“it definitely makes me reflect. I think, “Well, hold on a second, you know, it would be better not to drink quite so much at those times.” (Veteran A) (Lapham et al 2012) D1

“...to me I think, when these websites say don’t do this, don’t do that, I don’t want to listen...whereas if it says if you are going to do it, do it responsibly... that is more interesting.” [Member of 16-25 year groups] (Kay-Lambkin et al 2011) F1

I found the results of the test interesting because I had a score of 7. A score of 8 would have put me in the second risk category. That made me think for a second: “maybe I should be aware...” Because I expected a lower score. (Fraeyman et al 2012)

“It’s good! It kind of scares you straight for a little bit. You don’t normally see this, you just think, Oh a couple of drinks, but you know, really, health-wise it’s a lot of calories. It hurts your health and your baby, it’s sad. But our generation or people that live around here, and who probably didn’t know they were pregnant, are out drinking.” (Gorman 2013)

"I liked this page, because I was like “Oh my god” (laughs). “Cause I just didn’t realize... putting it on the most that you drank in one night, I was like, “Wow, remind me not to do that very often.” I’ll think about that a lot more. (Lapham et al 2012)

User Perceptions/Values

Finally, users’ perceptions of the fit between intervention components with their own values and beliefs was a critical factor for the successful adoption of digital alcohol interventions. Study participants expressed various concerns including privacy and security issues related to the format and delivery mechanism of the intervention. In addition, trust seemed to be an important consideration, with participants explicitly noting in one study that they found the source of information credible, or in some cases stating a preference for information from organisations with a good reputation off-line. A number of concerns were also related to adherence/compliance with digital interventions. Some participants felt that it is easier to ignore or drop out from a digital intervention with no personal contact (i.e. face to face interaction makes treatment feel more real; digital intervention is more of a “one size fits all” approach). In addition, participants regularly expressed concerns regarding the relevance and cultural appropriateness values of the intervention. 

 “Could be honest about things you might not want to say to the doctor” (P087). (Goodyear-Smith 2013) I2

"I don’t like the idea of the answers being on record for anyone to see, e.g., insurance companies” (P078). (Goodyear-Smith 2013) I4

“...you could… send yourself an SMS text ... to remind yourself to eat between drinks, or alternate your drinks…” (Kay-Lambkin et al 2011) J2

Digital is “easier to ignore”, whereas face to face “make[s] treatment real” (Kay-Lambkin et al 2011) J1

“Oh, bull. I drink more drinks in a single day than 99 percent of male VA patients my age.” I don’t believe that for a second, okay?... (Lapham 2012) C4

This synthesis of qualitative studies suggested that many people react positively to a digital intervention and that it does appear to help people to reflect on their alcohol consumption.  Careful attention needs to be given to the visual design (for websites) and clarity (for websites and SMS) of the intervention in order to draw people in.  Particular consideration also needs to be given to normative feedback provided and to how advice is expressed, since many people in these studies did not self-identify as the target population (even where significant effort was made to personalise the information), and in some cases advice that was too directed or was perceived as judgemental put them off.  This may partly explain why even where people agreed that the intervention helped them reflect on their drinking, they didn’t always predict that it would change their behaviour.

Results – quantitative studies

Characteristics of included studies: we identified 57 studies containing quantitative data (table 2), of which six[16, 17, 21, 22, 27, 28] also contained qualitative data and were included above.  Six[29-34] were randomised controlled trials which provided data for the meta-analysis reported previously, but which also contained an embedded process evaluation which was used here.  Thirteen named interventions were the subject of more than one of the 57 studies.  Interventions were delivered via website (37 studies, 24 interventions, of which 4 were accessed in a specific location), stand-alone computer programme (13 studies, 10 interventions), mobile phone via SMS (3 studies/interventions, with additional MMS (n=1) or web (n=1) content), mobile phone app (1), CD-ROM (1), email (1), or IVR (interactive voice recognition, 1).  Participants were recruited from school, college or university (n=19); the community (e.g. newspaper or website adverts, online access from unspecified location, n=17); primary care (n=13); Emergency Departments (n=4); workplaces (n=3); and the marine corps (n=1).  The vast majority of studies reported Likert-style questionnaires with outcomes variously concerning (i) the look and feel of the intervention and ease of use; (ii) the quality or accuracy of the information provided; (iii) whether the intervention was useful, relevant or interesting; (iv) whether it had any impact in terms of increasing knowledge, or making the participants reflect on or change their behaviour (or seek support for behaviour change); (v) how participants felt about the privacy or security of using the intervention; (vi) whether participants would recommend the intervention;  (vii) remote profiling data such as number of times participants logged on or duration of use.

Table of quantitative studies:

A summary of the study designs and key areas of content reported in the quantitative descriptive work is found in Table 2 (see appendix). We plan to assess the results of these studies in future dissemination work and then link them, where possible, to the analytical themes from our qualitative review in the final cross-study synthesis. This final stage will be a comparative analysis of both syntheses to identify factors that promote acceptability of and engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations.

Conclusions

This is the first systematic review to identify factors that shape the acceptability to users of digital alcohol interventions and highlight features that promote or inhibit people’s use of them. Overall 14 qualitative studies explored a wide range of users’ views about digital interventions in some depth and 57 studies described some features of digital alcohol interventions that are positively valued or sometimes of concern to potential users. User engagement with digital alcohol interventions seems to depend on a complex interplay between: the inherent technological features of the platform delivering advice or information about alcohol; the specific content or components of the intervention programme (with positive views that reflect the principles and structure of face to face brief alcohol interventions); and users’ sense of ‘fit’ between their values and beliefs and those incorporated into digital programmes. Key issues included credibility (both of the information per se and also the source of the programme), the ability to be honest about drinking issues as well as assurance about data confidentiality and data security.  Just three qualitative studies specifically focused on people from lower socio-economic status background or minority ethnic groups. Although a number of studies flagged the potential of digital interventions to help widen access to information or advice about alcohol. Whilst owning digital devices (particularly newer smart phones) has a cost implication, rates of ownership are increasing across most population groups. Remotely delivered interventions could also help individuals who feel unable or unwilling to access services but the content needs to reflect a wide diversity of potential users with a range of abilities regarding digital and health literacy. Quantitative studies were mapped rather than synthesised for the purposes of this report (following earlier advice from SPHR); they mainly described web-based interventions or stand-alone computer programmes, and were mostly used in educational institutions, the general community or primary care. In summary, users and potential users of digital alcohol interventions seemed to be broadly positive about the potential helpfulness of this form of public health intervention, however, there were some important caveats regarding design, content and the need for reassurance about confidentiality and data security.

References

1. Kaner EFS, Beyer FR, Brown J, et al. Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. Cochrane Database of Systematic Reviews [Internet]. 2015 [accessed; (1). Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD011479/abstract.
2. Beyer FR, Kaner EFS, Hickman M, et al. Engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations: a systematic review: Prospero; 2015 [8th December 2015]. Prospero registration]. Available from: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015019790.
3. Michie S, Richardson M, Johnston M, et al. The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Annals of Behavioral Medicine 2013;46:81-95.
4. Michie S, Prestwich A. Are interventions theory-based? Development of a theory coding scheme. Health Psychol 2010;29:1-8.
5. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. Journal of Clinical Epidemiology 1993;46:423-9.
6. Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 510 (updated March 2011): The Cochrane Collaboration; 2011.
7. Deeks J, Higgins JPT, Altman DG. Chapter 9: Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 510 (updated March 2011): Cochrane Collaboration; 2011.
8. Deeks JJ, Altman DG, Bradburn MJ. Statistical Methods for Examining Heterogeneity and Combining Results from Several Studies in Meta-Analysis. Systematic Reviews in Health Care: BMJ Publishing Group; 2008. p. 285-312.
9. Michie S, Abraham C, Whittington C, et al. Effective techniques in healthy eating and physical activity interventions: A meta-regression. Health Psychol 2009;28:690-701.
10. Kaner EFS, Dickinson HO, Beyer FR, et al. Effectiveness of brief alcohol interventions in primary care populations. Cochrane Database of Systematic Reviews [Internet]. 2007 [accessed; (2). Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD004148.pub3/abstract.
11. Hempel S, Miles J, Booth M, et al. Risk of bias: a simulation study of power to detect study-level moderator effects in meta-analysis. Syst Rev 2013;2:1-10.
12. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008;27:379-87.
13. Voils C, King H, Maciejewski M, et al. Approaches for Informing Optimal Dose of Behavioral Interventions. Annals of Behavioral Medicine 2014;48:392-401.
14. Walsh D, Downe S. Appraising the quality of qualitative research. Midwifery 2006;22:108-19.
15. Chambers M, Connor S, McElhinney S. Substance use and young people: The potential of technology. Journal of Psychiatric and Mental Health Nursing 2005;12:179-86.
16. Fraeyman J, Van Royen P, Vriesacker B, et al. How is an Electronic Screening and Brief Intervention Tool on Alcohol Use Received in a Student Population? A Qualitative and Quantitative Evaluation. J Med Internet Res 2012;14:12.
17. Hallett J, Maycock B, Kypri K, et al. Development of a Web-based alcohol intervention for university students: processes and challenges. Drug Alcohol Rev 2009;28:31-9.
18. Moore SC, Crompton K, van Goozen S, et al. A feasibility study of short message service text messaging as a surveillance tool for alcohol consumption and vehicle for interventions in university students. BMC Public Health 2013;13:1011.
19. Nygaard P, Paschall MJ. Students' experiences with web-based alcohol prevention: A qualitative evaluation of AlcoholEdu. Journal of Drug Education 2012;42:137-58.
20. Reis J, Riley W, Lokman L, et al. Interactive multimedia preventive alcohol education: a technology application in higher education. Journal of Drug Education 2000;30:399-421.
21. Goodyear-Smith F, Warren J, Bojic M, et al. eCHAT for lifestyle and mental health screening in primary care. Annals of Family Medicine 2013;11:460-6.
22. Murray E, Linke S, Harwood E, et al. Widening access to treatment for alcohol misuse: Description and formative evaluation of an innovative web-based service in one Primary Care Trust. Alcohol and Alcoholism 2012;47:697-701.
23. Gorman JR, Clapp JD, Calac D, et al. Creating a culturally appropriate web-based behavioral intervention for American Indian/Alaska Native women in Southern California: the healthy women healthy native nation study. Am Indian Alsk Native Ment Health Res 2013;20:1-15.
24. Osilla KC, D'Amico EJ, Diaz-Fuentes CM, et al. Multicultural web-based motivational interviewing for clients with a first-time DUI offense. Cultur Divers Ethni Minor Psychol 2012;18:192-202.
25. Lapham GT, Hawkins EJ, Chavez LJ, et al. Feedback from recently returned veterans on an anonymous web-based brief alcohol intervention. Addict Sci Clin Pract 2012;7:17.
26. Witbrodt J, Kaskutas LA, Diehl S, et al. Using drink size to talk about drinking during pregnancy: Early Start Plus. Journal of Addictions Nursing 2007;18:199-206.
27. Irvine L, Falconer DW, Jones C, et al. Can Text Messages Reach the Parts Other Process Measures Cannot Reach: An Evaluation of a Behavior Change Intervention Delivered by Mobile Phone? PLoS One 2012;7:7.
28. Kay-Lambkin FJ, White A, Baker AL, et al. Assessment of function and clinical utility of alcohol and other drug web sites: an observational, qualitative study. BMC Public Health 2011;11:277.
29. Butler LH, Correia CJ. Brief alcohol intervention with college student drinkers: face-to-face versus computerized feedback. Psychology of Addictive Behaviors 2009;23:163-7.
30. Chiauzzi E, Green TC, Lord S, et al. My student body: a high-risk drinking prevention web site for college students. Journal of American College Health 2005;53:263-74.
31. Collins SE, Kirouac M, Lewis MA, et al. Randomized controlled trial of web-based decisional balance feedback and personalized normative feedback for college drinkers. J Stud Alcohol Drugs 2014;75:982-92.
32. Delrahim-Howlett K, Chambers CD, Clapp JD, et al. Web-based assessment and brief intervention for alcohol use in women of childbearing potential: a report of the primary findings. Alcohol Clin Exp Res 2011;35:1331-8.
33. Gajecki M, Berman AH, Sinadinovic K, et al. Mobile phone brief intervention applications for risky alcohol use among university students: a randomized controlled study. Addict Sci Clin Pract 2014;9:11.
34. Wagener TL, Leffingwell TR, Mignogna J, et al. Randomized trial comparing computer-delivered and face-to-face personalized feedback interventions for high-risk drinking among college students. Journal of Substance Abuse Treatment 2012;43:260-7.

Plain English Summary

Heavy drinking causes over 60 diseases in patients, as well as many accidents, injuries and early deaths each year. Brief advice or counselling, delivered by doctors or nurses, can help people cut down their drinking.  A summary of 22 studies compared brief advice or counselling with usual care, and found that it helped reduce drinking by 4-5 units a week.  This is around two pints of beer or half a bottle of wine each week. However, patients may be embarrassed by talking about alcohol with doctors or nurses. Many people use computers, smart phones or the internet to seek advice about health matters. We wanted to know if these gadgets can help people reduce their drinking. 

 

We carried out two systematic reviews.  The first looked at whether any of the gadgets are better at reducing harmful drinking than doing nothing, or than having a face to face conversation about drinking.  The second review summarised people’s view of these gadgets and what makes people more or less likely to use them in the first place. 

 

The studies we looked at for the first review included people in work places, colleges or health clinics and internet users. Everyone typed information about their drinking into a computer, telephone or internet site - which then gave half the people advice about how much they drank and the effect this has on health. This first group also got different suggestions about how to cut down on drinking. The other group did not get advice about their own drinking or suggestions about how to cut down. Sometimes they could read or print off a leaflet with general health information. Everyone was asked to return one to 12 months later to type in again how much they were drinking. Then the drinking levels in both groups were compared to each other.

 

We found 55 studies which compared the drinking of people getting advice about alcohol from computers, telephones or internet sites against those that did not. We also described the types of advice that people got, to see if any was better than another at helping to reduce drinking.

 

Most people did drink less if they received advice about alcohol from a computer, telephone or internet site compared to people who did not get this advice. The amount they cut down was about 1.5 pints of beer or a third of a bottle of wine each week. This difference was seen at 1, 6 and 12 months after the advice. There was not enough information to help us decide if advice was better from computers, telephones or the internet. We also do not know what particular bits of advice were the most important. However, advice from trusted places like doctors’ groups seemed helpful as well as suggestions about things to do instead of drinking.

 

For the second review we found fourteen studies which recorded people’s views about how well they could use and understand advice delivered by computers, mobile telephones or the internet. A further 57 studies asked people to score their thoughts on short rating scales (for example 1=didn’t like at all to 5=liked a lot).  We have focused here on the views reported by a wide range of people including patients in GP surgeries, students, pregnant women, disadvantaged men, American Indian women and Latino people.  People’s use of technology to access advice about alcohol depends on three linked features:

1. How easy the technology is to use and understand and how attractive it is. People liked uncluttered screens, coloured pictures and the ability to check in regularly or be sent reminders.
2. The type of advice being delivered. Many people found it useful (and sometimes surprising) to get feedback on how much they drank, and they liked a friendly rather than a ‘telling off’ approach.
3. People also preferred advice that fit with their day-to-day lives or views about the world. Some felt that the advice was meant for people unlike them and so it did not seem relevant.

Overall people seemed positive about advice delivered by computers or mobile telephone. Some preferred to talk to a doctor to help keep track of their drinking and some were nervous about entering personal information into computers. However, others some preferred the anonymity of using technology and felt it helped them to be more honest about their drinking.

Dissemination

Academic Dissemination

1. Kaner EF, Beyer FR, Brown J, Crane D, Garnett C, Hickman M, Muirhead C, Redmore J, Michie S, de Vocht F. Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations (Protocol). Cochrane Database of Systematic Reviews 2015, Issue 1.
2. Beyer FR, Maniatopoulos G, Hickman M, Michie S. Engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations: a systematic review. PROSPERO 2015:CRD42015019790 Available from http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015019790
3. Muirhead C, et al. (2015) Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. 11th UKSBM Annual Scientific Meeting: ‘Biology, Behaviour & Environment’.
4. Crane D, et al. (2015) Behaviour change techniques used in digital interventions to reduce excessive alcohol consumption. 11th UKSBM Annual Scientific Meeting: ‘Biology, Behaviour & Environment’.
5. Garnett C, et al. (2015) Theory content of digital interventions for reducing alcohol consumption: a systematic review. 11th UKSBM Annual Scientific Meeting: ‘Biology, Behaviour & Environment’.
6. Beyer FR, et al. (2015) Acceptability of and engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations: a systematic review. 11th UKSBM Annual Scientific Meeting: ‘Biology, Behaviour & Environment’.
7. Beyer FR, Maniatopoulos G, Hickman M, Michie S. Engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations: a systematic review. PROSPERO 2015:CRD42015019790 Available from http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015019790
8. Beyer FR, et al. (2015) Acceptability of and engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations: a systematic review. 11th UKSBM Annual Scientific Meeting: ‘Biology, Behaviour & Environment’, Newcastle, 8-9 December 2016.
9. Beyer FR, et al. (2016) Acceptability of and engagement with digital interventions for reducing hazardous or harmful alcohol consumption in community-dwelling populations: a systematic review. 2nd Behaviour Change Conference: ‘Digital Health and Wellbeing’, London, 24-25 February 2016.
10. Kaner E et al. (2016) New technologies for reducing excessive alcohol use. SPHR Annual Scientific Meeting: ‘Knowledge into practice’, Newcastle, 10^th March 2016.
11. Kaner E et al. (2016) Digital interventions for hazardous and harmful alcohol consumption: two linked systematic reviews (poster). SPHR Annual Scientific Meeting: ‘Knowledge into practice’, Newcastle, 10th March 2016.

We are planning to submit the fully completed outcome-focused review to the Cochrane Library in early 2016, thereafter a paper of these findings will be prepared for peer reviewed journal.

We submitted the review to the Cochrane Drugs and Alcohol Group (CDAG) in early 2016 and we have addressed comments from CDAG’s methodologist and assigned co-ordinating editor, the Cochrane Editorial Unit, and external peer reviewers.

We also plan to write two papers from the engagement focused review once it is completed.

We decided to focus on the qualitative literature for the engagement review and will submit to Journal of Medical Internet Research in July.

Four linked oral presentations relating to this programme of work were delivered at the 11th UK Society for Behavioural Medicine (SBM) Annual Scientific Meeting: ‘Biology, Behaviour & Environment’, Newcastle, 9th December 2015, at the 2nd Behaviour Change Conference: Digital Health and Wellbeing, London, 24th-25th February 2016, and the South West Public Health Scientific Conference, Bristol, 16th March 2016. Professor Kaner presented a summary of this work at the PHE Alcohol Project Team Engagement Event on the 1st December 2015, which was hosted by the London School of Hygiene and Tropical Medicine, and at the SPHR Annual Scientific Meeting at Newcastle on 10th March.

Non-academic conferences, workshops and seminars

A summary of review findings was presented to a meeting of policy makers from Public Health England, Department of Health and the Home Office on the 17th September 2015 at Wellington House. Professor Kaner has also discussed findings with the Behavioural Insights Team (Ross Broad, Simon Rudda) and the ‘What Works’ centre (David Halpern) which are linked to the Cabinet Office.

Public involvement

We have had a number of practitioner and policy-maker focused discussions relating to the two-linked systematic reviews and the outcome focused meta-analysis. We plan to set up a community panel to act as a sounding board to guide the interpretation of our findings overall but particularly the acceptability, engagement and adherence aspect of the systematic review.

Impact

Professor Kaner has prepared a confidential briefing relating to the two linked reviews for the scientific advisory panel of the four Chief Medical Officers’ review of current alcohol guidelines. This material is currently included as an appendix in the draft guidelines document which was launched in January 2016

This project was funded by the National Institute for Health Research School for Public Health Research (SPHR-SWP-ALC-WP1 (rev)) 

Department of Health Disclaimer

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR School for Public Health Research, NIHR, NHS or the Department of Health.