Notes
Article history
The research reported in this issue of the journal was funded by the PHR programme as project number 15/183/26. The contractual start date was in March 2018. The final report began editorial review in March 2020 and was accepted for publication in October 2020. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2021. This work was produced by Fulton et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
2021 Queen’s Printer and Controller of HMSO
Chapter 1 Introduction
Background
Although UK smoking rates have hit an all-time low, 6.1 million adults in the UK are current smokers. 1 The significant role of smoking as a cause of chronic illness and preventable death endures worldwide. 2 UK NHS Stop Smoking Services (SSS) offer free behavioural and pharmacological interventions proven to increase the likelihood of quitting compared with ‘going it alone’. 3 SSS are typically offered within general practice and pharmacy settings, although increasingly they are offered as part of a package of lifestyle services hosted in a variety of community settings (CSs), such as supermarkets, libraries and community centres. 4
The effectiveness of SSS is determined by smoking abstinence 4 weeks after the quit date set, confirmed by standard carbon monoxide (CO) testing. 5 Studies indicate that 4-week quit rates are a reliable predictor of long-term abstinence, with 6-month follow-up data revealing only a small increase in the accuracy of such predictions. 6
Service uptake by smokers is, however, in decline. 7 Potential explanations for this include reduced economic resources for promoting services, and that smokers now favour the use of electronic cigarettes (ECs) over behavioural support for cessation, although this remains unclear. 8 Nonetheless, a survey of stop smoking advisors found that the majority (95%) had encountered smokers who were also using ECs. 9 This suggests that smokers may recognise ECs as an adjunct rather than an alternative to behavioural support for smoking cessation, and the effectiveness of ECs as a stop-smoking aid is likely to be enhanced when coupled with SSS behavioural support. 10 The growing use of ECs need not impede SSS access if services promote a clear EC-friendly approach, with consistent training and policy. 11
Despite significant budget cuts, sustained investment in SSS is being called for by the All Party Parliamentary Group on Smoking and Health,12 to support the ambition of achieving a smoke-free population by 2030. 13 However, evidence suggests that SSS reach only 5–10% of current smokers. 14 Investment in promotion and interventions to increase uptake of SSS is warranted. The UK Government funded a series of mass media tobacco control campaigns between 1999 and 2010 to emphasise smoking harms and SSS support. These campaigns were suspended owing to significant costs associated with delivery in 2010, enabling analysis of their impact on SSS uptake. Although calls to stop-smoking telephone lines, smoke-free website visits and requests for cessation support packs decreased significantly, there was no effect on SSS attendance,15 suggesting that the campaigns were not effective in enhancing SSS uptake.
Digital interventions promoting SSS help-seeking by smokers offer a potentially cost-effective solution. Current data suggest that internet access and smartphone use are increasingly widespread, with an estimated 96% of 16- to 24-year-olds, 98% of 25- to 34-year-olds, 94% of 35- to 44-year-olds, 87% of 45- to 54-year-olds and 71% of 55- to 64-year-olds owning a smartphone. 16 However, questions remain regarding whether or not they increase health inequalities if users of digital technologies are principally from higher socioeconomic status (SES) groups. Interventions require careful testing to ensure that they do not promote a ‘digital divide’. This is pertinent given that smoking is most prevalent in more deprived communities. 17 However, a trial of ‘StopAdvisor’,18 a digital smoking cessation intervention, found that it was effective at achieving cessation for smokers from lower but not higher SES groups, suggesting that digital interventions can help to reduce rather than increase health inequalities.
The uptake of SSS from lower SES groups remains varied across geographical localities. 19 Several studies suggest that smokers from more deprived groups are less aware that SSS exist, or of the services that they offer. 20–24 Negative beliefs about SSS include uncertainties about whether or not they will help or if they will be stigmatising, guilt-inducing or judgemental. 20–23 Concerns from smokers about a lack of information and perceived availability of services have been highlighted, alongside work commitments and time constraints, which can impede attendance. 25 The authors conducted studies to explore these perceived barriers and facilitators in depth,26 and discovered similar findings, including beliefs that needing support to quit smoking is a sign of weakness27 and that smokers should not need help. Therefore, an understanding of the potential barriers to access, particularly in lower SES groups, is paramount, and should be addressed within the development and evaluation of interventions designed to increase SSS access.
These barriers to accessing SSS have not, to the authors’ knowledge, been addressed sufficiently within interventions or health promotion campaigns targeted at smokers. Research has shown that providing information about SSS may increase access; for example, when booklets explaining how and why services help smokers are given to those who have registered with SSS, this can increase attendance rates. 28 A more recent trial (Start2quit) provided letters to smokers with personalised risk information and the offer of SSS taster sessions, resulting in improved SSS uptake. 29
Given that public health budgets are under increasing pressure, and evidence suggests that smokers seeking help themselves are more successful at stopping than those referred by others,30 it will become increasingly important that SSS are promoted to smokers who otherwise may not have recognised the benefits of attending. This will also involve ensuring that there is informed decision-making and encouragement to attend, to reduce ‘did-not-attend’ (DNA) rates and the costs that these incur. Implementing inexpensive strategies to address these factors will maximise effectiveness in a resource-restricted public health context.
StopApp™ (Coventry University, Coventry, UK) is a brief, digital behaviour change intervention [web application (app)] developed with the aim of increasing booking and attendance at SSS, especially by those from more deprived groups. Based on evidence about the barriers to service access and designed using the Behaviour Change Wheel,31 it first aims to enhance motivation to attend SSS using a series of behaviour change techniques (BCTs)32 derived from a systematic assessment of the barriers and facilitators that smokers typically experience. 27 StopApp then supports instant appointment booking, providing choice regarding time and location via an application programming interface (API). This infrastructure is supported by pharmacy and general practice data management software (PharmOutcomes and Outcomes4Health, both owned by Pinnacle Healthcare Ltd, Teddington, UK) that is used extensively by UK pharmacists and general practices to manage SSS use data. For the purpose of this feasibility trial, we had access only to pharmacy-based SSS data software (PharmOutcomes) to test the trial methods. However, StopApp could be used to increase attendance at any SSS for which it is possible to embed an API link to the SSS data software system.
Although increasing numbers of digital interventions for smoking cessation support exist,33,34 to our knowledge, StopApp is the first digital intervention to target increased uptake of SSS. If effective, StopApp could provide an important contribution to maximise SSS uptake. Given the novel digital approach that StopApp represents, and the challenges of recruiting smokers into research studies,35 it is unclear from the current evidence base whether or not it is possible to recruit the required sample size for a full randomised controlled trial (RCT). In addition, given concerns around the potential for digital interventions to contribute to health inequalities, the extent to which a trial of StopApp would reach and engage smokers from a range of sociodemographic backgrounds, including those most at risk of poor health outcomes from smoking, needs to be established. Therefore, a feasibility RCT was deemed necessary in order to assess whether or not a future, definitive RCT is possible.
Aims and objectives
The aim was to establish the viability of a future, larger, multicentre RCT of the StopApp intervention. The specific objectives are outlined below.
Primary objective
The primary objective was to conduct a feasibility trial of StopApp to estimate recruitment and attrition rates of participants across three settings, namely general practitioner (GP) surgeries, CSs and online, at baseline, intervention access and 2 months’ follow-up.
Secondary objectives
The secondary objectives of the MyWay feasibility trial were to estimate:
-
the acceptability of randomisation and the StopApp intervention for participants
-
the acceptability of the outcome measures and measures required for cost-effectiveness analyses in a future trial
-
the key costs that would be incurred in delivering the intervention and usual care, including a comparison of DNA rates between each arm of the trial
-
the feasibility of accessing SSS and GP data (if recruited via a GP) on attendance, quit dates set and 4-week abstinence rates for trial participants
-
any differential recruitment and attrition rates across socioeconomic groups, age and sex
-
the rate of SSS booking and attendance in the intervention and control groups to estimate the event rate of the primary outcome measure for a future trial, and to support future trial sample size calculations.
Chapter 2 Methods
Design
This was a two-arm, 1 : 1 allocation, double-blinded, parallel-group, individual participant randomised feasibility RCT of StopApp (intervention) compared with usual promotion of and provision of contact details for SSS via an online leaflet (control; see Report Supplementary Material 1). The study included a nested qualitative process evaluation with participants and staff involved in participant recruitment across the three settings to assess the acceptability of the research processes, randomisation, measures and the intervention.
Study population for feasibility trial
Eligible participants were current smokers (of any tobacco amount and type) aged ≥ 16 years who were either (1) registered with one of six participating GP surgeries in Warwickshire, (2) accessing participating Warwickshire-based local community services (e.g. children’s centres, libraries, well-being hubs) or (3) viewing open-access advertising for the study on Warwickshire-based social media (SM) platforms [e.g. Twitter feed (Twitter, Inc., San Francisco, CA, USA; www.twitter.com) of local public health department; Facebook page (Facebook, Inc., Menlo Park, CA, USA; www.facebook.com) of ‘What’s on in Warwickshire’]. Participants were not excluded if they had previously attended SSS. Eligible participants had to be able to comprehend written English, have access to the internet and have a mobile phone to complete study measures, view intervention or control content, and be able to receive short message service (SMS) reminders. As SSS appointments were available only in Warwickshire, participants had to live or work in Warwickshire.
Study population for process interviews
Process interview participants comprised (1) smokers who had taken part in the main study who were willing to participate in an interview, and (2) professional staff from the pharmacies offering SSS appointments and two of the recruitment settings, including GPs and practice managers and staff from local community services.
Study protocol and intervention
The study protocol was approved by the NHS West Midlands – Edgbaston Research Ethics Committee (reference 18/WM/0170) and was later published. 36
Control
Participants received a link to a web page displaying Warwickshire’s standard stop smoking service provision information (‘Quit4Good’). The control website does not include behavioural science theory or BCTs to facilitate booking or attendance, nor does it include the facility to book an appointment online; rather, participants can telephone for an appointment.
Intervention
StopApp is a brief web app intervention designed to be used on a single occasion to address the psychological and practical barriers smokers may experience when considering help from SSS [Figure 1; see Report Supplementary Material 2]. It is intended to improve the reach and effectiveness of standard SSS promotion that local authorities (LAs) and SSS may use to recruit smokers. The process involved in the development of the StopApp, in terms of its theory and evidence base, and patient and public involvement (PPI) in content and functionality design, has been described elsewhere. 26,27,37 Figure 2 illustrates the participant flow and message tailoring content that StopApp recipients experienced. Initially, StopApp users are given persuasive messages applying BCTs tailored to past experiences of quit attempts and SSS use. They are given clear information about what to expect from the services and the benefits of SSS use; for example, the intervention provides key messages about the services and the multiple strategies for quitting that are available, the benefits of attending and the non-judgemental, friendly support provided. Some message content is derived from genuine feedback from SSS users. Subsequently, users are asked to select barriers to SSS use that they may be experiencing, and further persuasive content employing BCTs is tailored to personal perceptions. At all times, the opportunity to ‘book now’ is provided and anyone who does not yet feel ready but remains interested can request a reminder to come back to StopApp in the subsequent month. Navigation and user experience were designed using behavioural science insights to ensure that the intervention was intuitive, attractive and easy to use. To operationalise the ‘book now’ facility, StopApp was connected to the online outcomes reporting systems owned by Pinnacle Healthcare Ltd and used by SSS in pharmacies (PharmOutcomes) and GP surgeries (Outcomes4Health) in Warwickshire and other UK LAs. These secure systems support appointment booking and record SSS outcome data such as quit dates and CO-verified 4-week quit rates for service commissioners. StopApp was programmed to ‘talk to’ these systems and access live data about which services had appointment availability. StopApp users could then choose a location and could see appointment availability over the following 2 weeks. Participants selected an appointment time and received confirmation (date and location) via their mobile phone. In future, through collaboration with other providers of similar data management systems, StopApp could be adapted to link to a range of systems used by other types of service provider such as ‘QuitManager’ (Bionical Solutions Ltd, Willington, UK).
Recruitment settings
To assess the best method for recruitment in a potential future definitive RCT of StopApp, recruitment across three settings was trialled. The settings were recruiting (1) via participating GP practices at which smokers were registered, (2) via advertising and information provided in community services and settings, and (3) via SM channels.
The West Midlands Clinical Research Network (CRN) recruited a total of six GP surgeries and identified smokers on their patient lists. Where more than one smoker cohabited, the smoker whose first name was alphabetically first was selected for invitation to the study to avoid contamination across trial arms. Other members of the household who were smokers were sent control arm information about SSS at the end of the recruitment and follow-up period. Four surgeries sent study invites to smokers by text message containing a link to the study website. The two surgeries that were not enabled to send communications to patients by text message or e-mail sent invitations with study website details by post using ‘Docmail’ services (CFH Docmail Ltd, Radstock).
A wide variety of CSs were identified to promote the study. Initially, these included libraries, leisure centres, well-being hubs, children’s centres, the family information service and the registrar’s office, and, during recruitment, we also advertised via workplaces including the Warwickshire County Council’s intranet, newsagents and public houses. There was also a bus advert campaign, with adverts appearing at bus stops and inside buses in Warwickshire.
The marketing and communications department at Warwickshire County Council supported the SM campaign and promoted the study via a range of Warwickshire-based SM channels. They also set up paid targeted advertising via Google Ads (Google Inc., Mountain View, CA, USA) and Facebook Ads. The study team supported the online promotion via ‘retweets’ and posts of their own on relevant Facebook pages, with permission from administrators.
Recruitment and randomisation procedure
Smokers who received or viewed study invitations or promotional materials and were interested in participating in the research could access more information by clicking on or entering a link to the study web page online. Each recruitment setting had a different study URL in order to monitor where recruits were from. The study web page was delivered via secure, bespoke study management software (‘eNgage’) and this is where individuals could access the participant information sheet. Those who chose to participate were required to endorse mandatory consent statements before signing up (involving creating an account with their name and e-mail address) and were then directed by eNgage (St Louis, MO, USA) to complete the baseline measures set up in Qualtrics (Provo, UT, USA) survey software. On completion of the baseline survey, participants were routed back to eNgage, where the preprogrammed randomisation algorithm assigned participants to either the intervention arm or the control arm. Researchers and participants were blind to allocation. Randomisation used a 1 : 1 ratio, stratified by age, sex and level of deprivation, with pure minimisation to ensure balance. The randomisation algorithm was tested and approved by the Clinical Trials Unit at the University of Warwick. Our statistician regularly monitored allocation by condition using a dashboard in eNgage each week to ensure that a balance was achieved. Following randomisation, both groups received a near-identical message and web link. The web link took participants to either the control website or StopApp. At the same time, an e-mail containing the link, assigned to the participant’s unique identification number, was sent to participants to allow them to access the material at a later date. In relation to StopApp, participants could, therefore, go back and book at a later date. A reminder e-mail was sent to participants 2 weeks later. In the participant information sheet within eNgage, participants in both conditions were told that if they wished to book an appointment at the SSS they were free to do so, but, in taking part, were under no obligation to do so. An e-mail with a link to the control ‘Quit4Good’ leaflet was sent to anyone who contacted us after recruitment until the study ended.
Challenges with recruitment and deviation from planned protocol
Recruitment began on 7 January 2019 and, after an initial flood of participants in weeks 1 and 2, enrolment trailed off. We therefore sought approval for an amendment to the protocol from the National Institute for Health Research (NIHR) and NHS research ethics committees to provide an e-voucher payment of £5 per completed survey. This amendment to the protocol was included in the published study protocol. 36 Participants who had already participated were contacted and offered the e-voucher. At the same time as the e-voucher payment was introduced, our monitoring of participant access to the study website via linked analytics software demonstrated that there was a high ‘bounce’ rate (people clicking the link to the study and then immediately leaving the web page). Exploration of why that might be revealed that 86% of those accessing the study website were doing so on a mobile phone, and some information was difficult to read in this format. We therefore paused study recruitment for 1 week while the mobile optimisation features of eNgage were improved. Recruitment resumed and these amendments resulted in an improved recruitment rate.
Recruitment monitoring at the onset of the eighth week revealed a sudden influx of participants via the SM URL. Initial investigation of the data revealed that the majority of those recruited at this sudden point of influx were fraudulent and ineligible participants from other countries, most likely to be attempting to access the e-vouchers. Recruitment was paused again for a total of 4 weeks while further investigations took place and remedial action was taken. A clear set of indicators of ineligible participants were identified and then applied to the data in order to select and remove these cases. These indicators included checking the geographical location where each case had originated, the timestamp of the response, the e-mail address given and data provided in baseline questionnaires, to see if they were viable responses. It was generally very clear when ‘participants’ were completing several questionnaires in quick succession, as these responses were logged in batches with e-mail addresses that bore no resemblance to the name of the participant supplied in their eNgage account. These cases were removed and additional clarification was placed on the study website and the sign-up page explaining that only eligible participants who were smokers living and/or working in Warwickshire could be accepted into the study and given e-vouchers. We also included messages stating that evidence of eligibility may be necessary, to further deter ineligible participation. These pauses to recruitment resulted in the total period for baseline recruitment running until 31 May 2019, which included a total of 116 days on which recruitment via the study website was live.
Prior to the commencement of the feasibility trial, a data-sharing agreement (DSA) was drawn up between Coventry University and Warwickshire County Council, which commissions the SSS, in order to obtain access to the data on participants’ SSS use that are recorded by Pinnacle Healthcare Ltd. This was a requirement from Pinnacle Healthcare Ltd to ensure that permissions had been granted by Warwickshire County Council for it to release a report of the data it holds to Coventry University, to coincide with participant consent. We also sought written permission from the participating pharmacies at the point of inviting them to join the trial to ensure that they gave their approval for using SSS data. However, after the trial had commenced, concerns were raised about who owned the SSS data, and whether it was in fact the individual pharmacies that should give this more formal permission and, therefore, be required to sign a DSA to release their SSS data. As a result, it became necessary for the research team to action the creation of a series of DSAs between the council, Coventry University and each of the participating pharmacies.
Baseline measures
Baseline data included demographic information [including age, sex, profession, ethnicity and postcode – to identify Indices of Multiple Deprivation (IMD) score], current smoking status and tobacco products used (type and quantity), EC use, previous use of SSS, ease of internet access, and motivation to quit (measured using the one-item ‘Motivation to Stop Scale’38 and a single-item Likert scale39). Pregnancy status was collected to identify participants who would experience a separate NHS care pathway specifically for smoking in pregnancy. Level of social and economic deprivation was measured in several ways to identify the most suitable method for a full trial. This included using postcodes to calculate the standard IMD and quintile scores40 and a five-point version of the National Statistics Socio-economic Classification (NS-SEC)41 based on employment status. Health-related quality-of-life data were measured using the EuroQol-5 Dimensions, five-level version (EQ-5D-5L),42 and the ICECAP-A (ICEpop CAPability measure for Adults)43,44 instrument to inform the health economic analysis.
Measurement of engagement with StopApp
The eNgage system was programmed to provide individual-level data about StopApp use from Matomo (https://matomo.org; accessed March 2021) web analytics software. This allowed us to access individual participant-level information about whether or not StopApp was accessed, and how long participants spent on each page of the intervention, including whether or not they reached the booking confirmation page.
Follow-up procedure
Two months after each participant had been recruited to baseline and randomised, eNgage automatically sent them an e-mail with a request to complete an online follow-up questionnaire. Reminders to complete the questionnaire were sent again 2 weeks later. No further attempt to collect follow-up measures was made. Researchers checked that the automated procedures were functioning as planned, with no problems detected.
Outcome measures at follow-up
At the 2-month follow-up, participants were asked to provide self-reports of current smoking behaviour, any SSS appointment bookings made and how they were made (e.g. StopApp, booked with their own GP), and attendance at SSS in the last 2 months, quit dates set and if 4-week abstinence had been reached. Questions about resource use were also given to those who reported any service use, to verify costs to individuals and the public purse in conjunction with using StopApp compared with the control. This included costs and resource use associated with promoting the intervention and usual care (e.g SM marketing, bus advertising), delivering the intervention (e.g. web hosting, text messages) and delivering usual care (e.g. telephone calls to book appointments).
Data collection at participating Stop Smoking Services
The original plan for trial set-up had involved recruiting all GPs and pharmacies able to offer SSS appointments across Warwickshire to participate in offering appointments in StopApp during the trial; however, although GP surgeries were in the process of transferring to the Pinnacle Healthcare Ltd software when trial funding was awarded, they had not completed this process in time to support the trial. In addition, some pharmacies (n = 11) offering SSS appointments in Warwickshire declined to be involved in the trial and did not feature as venues in StopApp.
It had also been our original intention to collect consent to access the objective SSS data of trial participants at study sign-up in eNgage, and provide evidence of this along with a list of study participant names via secure means to Pinnacle Healthcare Ltd (the PharmOutcomes owners). Pinnacle Healthcare Ltd could then search for participants (who had given consent) in its secure database to provide evidence regarding any bookings, attendances, DNAs, quit dates set and 4-week quits achieved, by the end of the trial. Pinnacle Healthcare Ltd decided, however, that it wanted to re-collect consent at the point of service access. In order to facilitate this, all participating pharmacies (n = 28) adopted a procedure for checking the trial participation status of those attending appointments during data collection. Stop smoking advisors were requested to ask participants at the first appointment (1) to confirm that they were taking part in the MyWay study/trial, and, if they did so, (2) whether or not they consented to the research team accessing data on their SSS use, which would be stored anonymously, and (3) how they heard about the research study. To facilitate this process, Pinnacle Healthcare Ltd added these three questions as drop-down menu items in the ‘first appointment’ section of its PharmOutcomes software. This ensured that Pinnacle Healthcare Ltd, as the data controller, could create a report on SSS use by trial participants who consented to their data being shared at the end of the trial. The report identified appointments booked via StopApp, attendance at appointments, quit dates set and CO-tested 4-week quit rates. This report was sent to a secure nhs.net e-mail address within the research team, to match with the questionnaire data. After matching, all identifying information and the e-mail report were deleted. It should be noted that because there were alternative venues where smokers could seek a SSS appointment (i.e. their own GP practice or one of 11 pharmacies who did not agree to support the trial), objective data about SSS use from pharmacies were not considered complete. We had to also rely on self-report data to collect an assessment of numbers of bookings that occurred outside StopApp.
Process evaluation interviews
Participants were asked at the end of the baseline survey whether or not they would be willing to participate in an interview about their trial experience. Willing participants provided contact details. To ensure that a cross-section of views was captured, a sample of approximately 20 participants was targeted, with participants from both arms and different demographic groups represented. The intention was to oversample participants from more deprived backgrounds, as measured by their IMD scores and quintiles. This was to ensure that any factors potentially associated with deprivation (which may have influenced engagement with the intervention or the research process) were fully explored.
Staff from CSs, GP practices and pharmacies who took part were also invited to participate in telephone interviews about their trial involvement. Selected participants and staff were invited to participate in a telephone interview via a link that was sent by e-mail. The link took them back to eNgage, where they could view information about participating in the interviews and complete online consent statements. Participants who gave consent were then contacted by telephone to arrange a suitable time for the interview to take place. In some instances, this took place immediately. The interviews explored participants’ experiences of taking part in the trial, barriers to and facilitators of the process, suggestions for improvement, beliefs about the value of the research methods and the overall impact that it had on them. Interviews were conducted by KK and LS, were audio-recorded and transcribed verbatim, and qualitative data were subject to thematic analysis45 by EAF with input from KK and LS. A pragmatic, hybrid approach was used to conduct the analysis in that we were concerned with identifying positive and negative experiences with various predetermined processes within the study, but were open to identifying themes and ideas which summarised the data that had not been predetermined. All researchers involved were experienced in applying this thematic analysis approach to qualitative data. EAF, KK and LS analysed a 10% subset of the interview transcripts independently and met to consider their level of agreement and discrepancies. The transcripts were read in their entirety to gain familiarity with the data and then re-read to highlight examples of ‘meaning’, such as ideas, comments and descriptions, to form codes. Notes regarding these initial codes were made in the margin, for reflection and discussion with the other two researchers. Agreements about discrepancies were reached, and EAF applied agreed principles in all subsequent analyses. A complete list of codes, with example quotes from the text, from all the interviews by subgroup (e.g. all the codes for participants only) were reviewed and grouped together by commonality to create a list of overarching themes (based on the research questions) to describe the data. If themes could be grouped further to create higher-order themes, this was conducted. The final themes were reviewed a final time to ensure that codes were included in the most appropriate theme, and theme names were assigned.
Sample size
Based on RCT data investigating the recruitment of smokers to SSS via letters from their GPs,46 we would need to enrol 980 smokers to detect a 7% difference in attendance at SSS between control and intervention arms in a definitive trial. Based on this estimate, any trial would need to recruit 1.8 participants per day in total to achieve the required recruitment in 18 months. We planned to recruit for 3 months in a single LA provider of SSS, requiring one-sixth of the sample and providing a target of 162 participants in this time frame.
As a result of the challenges experienced during recruitment, outlined in Challenges with recruitment and deviation from planned protocol, a revised target of 120 participants was agreed with the NIHR Public Health Research (PHR) programme manager. This target adhered to recommendations from Teare et al. 47 about having a minimum of 60 participants per arm of a feasibility trial with a dichotomous primary outcome measure. 47 Given that this feasibility study was conducted in a single LA area and any future definitive RCT would be delivered as a multisite trial, a target of 120 participants still represents a conservative target for what may be possible in a national multicentre trial. With a sample size of 120 participants, the recruitment rate of smokers for a full RCT was calculated as estimable with a precision (95% confidence interval width) of ± 5%.
Data analysis
Data from Qualtrics were downloaded and analysed using the Statistical Product and Service Solutions (SPSS v25). The overall recruitment rate was calculated as the total number of participants recruited divided by the number of days recruitment was live, to provide a recruit per day figure for one LA. To enable comparison across recruitment settings, the same calculation was made for each of the different settings, and for GP surgeries the rate was also calculated as the proportion of those recruited from those identified as smokers and invited to participate, expressed as a percentage. Cost per baseline recruit was calculated by summing the total costs spent to promote and advertise in each setting divided by the number of recruits achieved there. Attrition rates across all three settings were calculated as the proportion of those lost to the 2-month follow-up, expressed as a percentage. Attrition at baseline (i.e. consented but no completion of measures) and at intervention access was also calculated in this way. Owing to low frequencies of data in some of the ethnicity categories, which would impair statistical analysis, the variable was reduced to four categories: ‘white British’, ‘white other’, ‘all mixed background’ and ‘African/Asian background’.
An analysis of variance (ANOVA) test was conducted for continuous data and the Kruskal–Wallis test was conducted for ordinal data, to assess any differential recruitment between the three settings. Statistical assessments were conducted at the 5% level of significance or 95% confidence intervals. We also asked participants in the process evaluation how they felt about the randomisation process and their experience of the study methods. To assess the acceptability of the StopApp intervention, we explored engagement using Matomo analytics and asked participants randomised to the StopApp arm for their views during the process evaluation interviews.
To assess the acceptability of measures, including outcome measures and measures needed to evaluate cost-effectiveness in a potential future definitive RCT, we considered the proportion of missing data across study measures. Missing value analysis identified data missing at > 5% on any single measure. Those interviewed were also asked about the acceptability of the questionnaires that they completed.
To assess the health equity of the study methods, we report on the spread of age, sex, ethnicity and SES represented in baseline recruits; those who completed follow-up and those who did not; and the differential attrition by sociodemographic characteristics of the sample.
To assess the feasibility of accessing SSS data on booking, attendance, quit dates set and 4-week quit rates, we report on the challenges experienced around data access. We also report rates of booking, attendance and quit rates from both the self-report data and the objective data, and have calculated the event rate of booking in the intervention arm in support of a future trial sample size calculation.
The key costs incurred in delivering the intervention and the usual-care data are examined in Chapter 5.
Data analytics
Data analytics software (Matomo) was used to measure the attrition rate at intervention access and the level of engagement with the intervention. This software enabled us to observe the number of StopApp pages viewed, where participants exited and the interactions they had with the intervention, including button clicks.
Patient and public involvement group
A PPI group of 11 smokers from a range of SES backgrounds was invited to support the delivery of the MyWay trial. A number from this group (n = 3) had previously been involved in the co-design and end-user testing of StopApp and were instrumental in contributing to several waves of improvements made to the content, functionality and user experience. We recruited new members to this group (n = 8) during preparation for our funding application, and we consulted and engaged with the wider group regularly on the design of this feasibility study. With the help of a NIHR Research Design Service PPI grant of £500, we held a workshop with and sent follow-up e-mail communications to the group to consult on study design. Payment via vouchers for all contributions was provided. In addition to several suggestions about locations and ways to recruit (e.g. through bus stop and public transport-based advertising), and recruitment messages, attendees also suggested that the follow-up measures should be collected at 2 months post baseline to encourage responding; any longer was deemed problematic, and would lead to a loss of interest and disengagement. The PPI group also read and approved the lay summary of the project for the funding application. A further three meetings were carried out following the commencement of the trial to (1) review the study materials and protocols to ensure that they were as inclusive and acceptable as possible, (2) discuss recruitment issues with suggestions for improvements and (3) review the acceptability of trial methods in the format of ‘think aloud’ sessions (members participated in the sessions as dummy participants). We provided training in research processes and met with members at regular intervals either face to face or via online methods, to consult and gain their input. Two representatives from the group also attended and contributed to the Study Steering Committee meetings. Detailed minutes and notes were taken by a co-facilitator throughout the PPI group sessions to record ideas and suggestions.
Chapter 3 Quantitative results
Recruitment took place between 7 January and 31 May 2019. Six GP practices with a combined patient list size of 52,608 identified 1602 smokers who met the eligibility criteria. A total of 1602 initial invitation texts or letters were sent, and 1597 reminders were sent 2 weeks later. Five participants were excluded before reminders were sent because they had stopped smoking, had contacted their GP to state they did not wish to take part, or had died (Figure 3).
A total of 78 community sites were recruited to promote the study to the Warwickshire population. Researchers sent posters and fliers out to sites by post, and attended major sites (e.g. children’s centres, libraries) to brief staff and provide additional support for recruitment (see Report Supplementary Material 3). Fliers for the study were distributed to newsagents that sold cigarettes, and to pubs and restaurants in Warwickshire. A bus and billboard campaign was also run in the Warwickshire area.
The online recruitment strategy comprised study invitation advertisements using Facebook Ads, Twitter Ads (Twitter, Inc., San Francisco, CA, USA), Google Ads and Gumtree (Gumtree.com, London, UK) for the duration of the recruitment period. The invitation was also posted on SM channels [including Twitter, Facebook and LinkedIn (LinkedIn Corporation, Sunnyvale, CA, USA) accounts] hosted by Coventry University, Warwickshire County Council and the local public health authority, and was added to a County Council online newsletter. These activities resulted in 721,602 SM ‘impressions’ and 3355 advert clicks.
Primary objective: recruitment and attrition rates across three settings (general practitioner, community setting, social media) at baseline, intervention access and follow-up
A total of 838 participants signed up on the study website, consented to take part in the study and completed the baseline measures. Of these, 715 (all accessing the study via SM) were excluded because they did not meet the eligibility criterion of living or working in Warwickshire (see Challenges with recruitment and deviation from planned protocol for details). A total of 123 eligible participants were, therefore, recruited over a period of 116 days and included in the baseline analyses (overall recruitment rate of 1.06 participants per day). Sixty-one participants were recruited via SM (0.53 per day), 36 from CSs (0.31 per day) and 26 from GP practices (0.22 per day). Using text messages and letters sent by post to recruit people from GP practices, we were able to identify that only 1.62% of those who received an invitation were recruited. All recruits were obtained via text message. Five participants were not computer randomised within the trial and were instead randomised manually by our statistician. Seven participants were not randomised. Interrogation of web analytics suggest that these seven participants completed the baseline measures but closed their browser before being assigned.
Of the 123 participants recruited, 116 were randomised (59 to the StopApp arm, and 57 to the control arm). A total of 60 participants completed the follow-up questionnaire (60/123; 48.8%) (n = 32, 54.2%, in the StopApp arm and n = 28, 49.1%, in the control arm) at 2 months post baseline (see Figure 3). This revealed an attrition rate of 51.2%. Loss to follow-up by site was as follows: SM (n = 24; 39.3%), CSs (n = 18; 50.0%) and GP (n = 21; 80.8%).
The demographic characteristics of participants are presented in Table 1. The distribution of participants by age, sex, ethnicity, SES and health-related quality of life between the two groups appeared balanced.
Characteristic | Total (N = 123a) | StopApp (N = 59) | Control (N = 57) |
---|---|---|---|
Age (years) | |||
Mean (SD) | 38.10 (12.6) | 38.27 (12.7) | 37.82 (12.4) |
Minimum, maximum | 16, 70 | 19, 70 | 16, 69 |
Sex (male), n (%) | 39 (31.7) | 19 (34.5) | 17 (30.4) |
Ethnicity, n (%) | |||
White British | 98 (79.7) | 43 (78.2) | 46 (82.1) |
White other | 6 (4.9) | 4 (7.3) | 2 (3.6) |
African/Asian | 6 (4.9) | 3 (5) | 3 (5.3) |
All mixed background | 10 (8.1) | 5 (8.4) | 5 (8.8) |
Have you done any paid work in the last 12 months? n (%) | |||
Yes | 103 (83.7) | 46 (83.6) | 45 (80.4) |
No | 20 (16.3) | 9 (16.4) | 11 (19.6) |
Current job description, n (%) | |||
Professional | 33 (26.8) | 9 (16.4) | 19 (33.9) |
Intermediate position | 23 (18.7) | 14 (25.5) | 6 (10.7) |
Routine and manual | 8 (6.5) | 4 (7.3) | 4 (7.1) |
Student | 0 | 0 | 0 |
Long-term employed/never worked | 20 (16.3) | 9 (16.4) | 11 (19.6) |
IMD score, n (%) | |||
< 8.49 (least deprived) | 16 (13.0) | 9 (16.4) | 7 (12.5) |
8.50–13.79 | 18 (14.6) | 7 (12.7) | 7 (12.5) |
13.80–21.35 | 36 (29.3) | 18 (32.7) | 17 (30.4) |
21.86–34.17 | 20 (16.3) | 8 (14.5) | 8 (14.3) |
> 34.18 (most deprived) | 18 (14.6) | 9 (16.4) | 8 (14.3) |
Recruitment source, n (%) | |||
GP practices | 26 (21.1) | 9 (16.4) | 13 (23.2) |
CSs | 36 (29.3) | 21 (39.6) | 20 (35.7) |
SM | 61 (49.6) | 23 (14.8) | 23 (41.1) |
How often do you usually smoke? n (%) | |||
Every day | 106 (86.2) | 45 (81.8) | 52 (92.9) |
Most days (3–5 days per week) | 8 (6.5) | 5 (9.1) | 1 (1.8) |
Fewer than most days (1–2 days per week) | 9 (7.3) | 5 (9.1) | 3 (5.4) |
How soon after waking do you smoke? n (%) | |||
Within 5 minutes | 29 (23.6) | 12 (21.8) | 15 (26.8) |
5–30 minutes | 55 (44.7) | 25 (45.5) | 26 (46.4) |
31–60 minutes | 22 (17.9) | 11 (20.0) | 7 (12.5) |
> 60 minutes | 17 (13.8) | 7 (12.7) | 8 (14.3) |
How likely is it that you will stop smoking in the next 6 months? n (%) | |||
Very unlikely | 13 (10.6) | 6 (10.9) | 6 (10.7) |
Unlikely | 16 (13.0) | 9 (16.4) | 6 (10.7) |
Maybe, maybe not | 48 (39.0) | 19 (34.5) | 25 (44.6) |
Likely | 31 (25.2) | 16 (29.1) | 13 (23.2) |
Very likely | 15 (12.2) | 5 (9.1) | 6 (10.7) |
Intentions to stop smoking, n (%) | |||
I don’t want to stop | 5 (4.1) | 2 (3.6) | 2 (3.6) |
I think I should stop smoking but I don’t know how | 18 (14.6) | 9 (16.4) | 9 (16.1) |
I want to stop smoking but haven’t thought about when | 15 (12.2) | 5 (9.1) | 8 (14.3) |
I really want to stop smoking but I don’t know when I will | 22 (17.9) | 10 (18.2) | 8 (14.3) |
I want to stop smoking and hope to soon | 31 (25.2) | 15 (27.3) | 15 (26.8) |
I really want to stop and intend to in the next 3 months | 11 (8.9) | 7 (12.7) | 4 (7.1) |
I really want to stop and intend to in the next month | 21 (17.1) | 7 (12.7) | 10 (17.9) |
Feeling settled and secure in all aspects of life, n (%) | |||
I feel settled and secure in all aspects of my life | 29 (23.6) | 11 (20.0) | 12 (21.4) |
I feel settled and secure in many aspects of my life | 50 (40.7) | 24 (43.6) | 24 (42.9) |
I feel settled and secure in a few aspects of my life | 32 (26.0) | 14 (25.5) | 16 (28.6) |
I feel unable to feel settled and secure in any aspects of my life | 12 (9.8) | 6 (10.9) | 4 (7.1) |
Having love and friendship, n (%) | |||
I can have a lot of love, friendship and support | 44 (35.8) | 21 (38.2) | 19 (33.9) |
I can have quite a lot of love, friendship and support | 46 (37.4) | 21 (38.2) | 20 (35.7) |
I can have a little love, friendship and support | 29 (23.6) | 12 (21.8) | 15 (26.8) |
I do not have any love, friendship and support | 3 (2.4) | 1 (1.8) | 2 (3.6) |
Being independent, n (%) | |||
I am able to be completely independent | 78 (63.4) | 38 (69.1) | 34 (60.7) |
I am able to be independent | 30 (24.4) | 12 (21.8) | 13 (23.2) |
I am able to be independent in many things | 8 (6.5) | 3 (5.5) | 5 (8.9) |
I am able to be independent in a few things | 4 (3.3) | 2 (3.6) | 2 (3.6) |
Having achievement and progress, n (%) | |||
I can have achievement and progress in all aspects of my life | 37 (30.1) | 17 (30.9) | 16 (28.6) |
I can have achievement and progress in many aspects of my life | 58 (47.2) | 24 (43.6) | 28 (50.0) |
I can have achievement and progress in a few aspects of my life | 24 (19.5) | 11 (20.0) | 11 (19.6) |
I cannot achieve in any aspects of my life | 4 (3.3) | 3 (5.5) | 1 (1.8) |
Having enjoyment and pleasure, n (%) | |||
I can have a lot of enjoyment and pleasure | 40 (32.5) | 18 (32.7) | 19 (33.9) |
I can have quite a lot of enjoyment and pleasure | 50 (40.7) | 23 (41.8) | 22 (39.3) |
I can have a little enjoyment and pleasure | 29 (23.6) | 12 (21.8) | 14 (25) |
I cannot have any enjoyment and pleasure | 3 (2.4) | 2 (3.6) | 1 (1.8) |
Data analytics findings
Matomo analytics data indicated that there were 60 unique views of the StopApp home page, which illustrated that one of the 59 participants in the intervention arm viewed the content more than once. Of these, nine proceeded to the next page(s) and 51 exited before interacting (e.g. clicking a button), indicating an attrition rate at intervention access of 85%. This meant that 15% of participants proceeded past the first page and were able to book an appointment; therefore, the majority of StopApp arm participants did not see information about SSS and how to book.
Secondary objectives
Acceptability of randomisation and the StopApp intervention for participants
Table 2 shows the demographic characteristics of participants by trial arm and compares baseline with follow-up. Similar rates of attrition were seen across trial arms, suggesting that being assigned to the control arm was as acceptable as being assigned to the StopApp intervention arm.
Characteristic | Baseline (N = 116)a | Follow-up (N = 56)a | ||
---|---|---|---|---|
StopApp (N = 59) | Control (N = 57) | StopApp (N = 31) | Control (N = 24) | |
Age (years) | ||||
Mean (SD) | 38.27 (12.7) | 37.82 (12.4) | 38.13 (12.53) | 35.29 (8.42) |
Minimum, maximum | 19, 70 | 16, 69 | 19, 70 | 20, 48 |
Sex (male), n (%) | 19 (34.5) | 17 (30.4) | 10 (32.3) | 6 (25.0) |
Ethnicity, n (%) | ||||
White British | 43 (78.2) | 46 (82.1) | 27 (87.1) | 22 (91.7) |
White other | 4 (7.3) | 2 (3.6) | 3 (9.7) | 0 (0) |
African/Asian | 3 (5.0) | 3 (5.3) | 0 (0) | 2 (8.3) |
All mixed background | 5 (8.4) | 5 (8.8) | 1 (3.2) | 0 (0) |
Have you done any paid work in the last 12 months? n (%) | ||||
Yes | 46 (83.6) | 45 (80.4) | 25 (80.6) | 21 (87.5) |
No | 9 (16.4) | 11 (19.6) | 6 (19.4) | 3 (12.5) |
Current job description, n (%) | ||||
Professional | 9 (16.4) | 19 (33.9) | 4 (12.9) | 7 (29.2) |
Intermediate position | 14 (25.5) | 6 (10.7) | 9 (29.0) | 3 (12.5) |
Routine and manual | 4 (7.3) | 4 (7.1) | 2 (6.4) | 3 (12.5) |
Student | 0 | 0 | 0 | 0 |
Long-term employed/never worked | 9 (16.4) | 11 (19.6) | 6 (19.4) | 3 (12.5) |
IMD score,b n (%) | ||||
< 8.49 (least deprived) | 9 (16.4) | 7 (12.5) | 3 (9.7) | 1 (4.1) |
8.50–13.79 | 7 (12.7) | 7 (12.5) | 1 (3.2) | 2 (8.2) |
13.80–21.35 | 18 (32.7) | 17 (30.4) | 3 (9.7) | 2 (8.2) |
21.86–34.17 | 8 (14.5) | 8 (14.3) | 0 | 1 (4.1) |
> 34.18 (most deprived) | 9 (16.4) | 8 (14.3) | 1 (3.2) | 1 (4.1) |
Acceptability of randomisation and the StopApp intervention was discussed during the process interviews. Participants understood the need for randomisation and thought it was acceptable. One participant expressed disappointment at being allocated to the control arm when this was disclosed during the process interviews (see Chapter 4, Acceptability of randomisation and self-report measures). StopApp was considered acceptable by those randomised to receive it, although some suggestions for improvements were also made [see Chapter 4, Acceptability of StopApp (for intervention arm participants only)].
Acceptability of primary and secondary outcome measures and measures required for cost-effectiveness and cost–utility analyses in a future trial
Missing data were interrogated to assess measure acceptability. Most questionnaire items had fully complete or near-complete data, with the latter defined as < 5% missing data from 123 questionnaire responses at baseline and 61 responses at follow-up. There was > 5% missing data for the following items: self-report data on whether or not the participant had booked an SSS appointment at follow-up (48.8% of participants did not respond); questions about job status: ‘Do you work as an employee or are you self-employed?’ (16.3% missing), ‘Do you supervise?’ (20.3% missing) and ‘Please read the following job descriptions, which best describes your role?’ (33.3% missing). Questions about whether or not participants were pregnant revealed missing data for 31.7% of female participants. Two items regarding smoking status, namely ‘Please tell us what you smoke’ and ‘How many cigarettes per day?’, each had 17.1% missing data.
There were also very few data entered in open-text boxes (e.g. 11.2% of participants listed that they smoked ‘other’ in a question about tobacco products, but no participants provided detail in the open-text box). Measure acceptability was also discussed during the process interviews. Participants in the trial who also took part in the process evaluation interview found the measures acceptable (see Chapter 4, Acceptability of randomisation and self-report measures).
Key costs that would be incurred in delivering the intervention and usual care including a comparison of did-not-attend rates between each arm of the trial
Key costs incurred are provided in Chapter 5. It was not possible to compare DNA rates between arms as a result of being able to confirm only two DNAs from the intervention arm in the objective SSS data.
Rate of Stop Smoking Services attendance in the intervention and control groups to estimate the event rate of the primary outcome measure for a future trial and support future trial sample size calculations
Self-report booking and attendance at Stop Smoking Services, quit dates set and 4-week quits
At follow-up, eight participants (6.9%) self-reported having made an NHS SSS appointment (Table 3). Six of these participants were randomised to the StopApp intervention arm and two to the control arm. Of these six participants, one had made an appointment in StopApp verified via the objective SSS data. The remaining participants made appointments outside StopApp. One participant made a face-to-face appointment (intervention arm), five made a telephone appointment (three from intervention arm and two from control arm) and two made a face-to-face and a telephone appointment (both intervention arm). Only three participants from the total study sample answered questions about how they booked these appointments. One reported to have telephoned (control arm) and two said that they had booked via StopApp (intervention arm, for which one was not recorded on the objective SSS data). Three participants said that they had set a quit date (one control participant and two intervention participants). One control participant said that they achieved a 4-week quit, one intervention participant said that they achieved a 4-week quit, and one intervention participant said that they did not achieve a 4-week quit.
Data source | StopApp, n (%) | Control, n (%) | ||
---|---|---|---|---|
Booked | DNA | Booked | DNA | |
Objective SSS data | 5 (8.5) | 2/5 (40) | 0a | 0a |
Self-report data | 5 (8.5)b | 1/5 (20)a | 2 (3.5) | 1/2 (50)a |
Total | 10 (16.9) | 3/10 (30) | 2 | 1/2 (50) |
Objective data about Stop Smoking Services booking, attendance, quit dates set and 4-week quits
The objective data accessed through the Pinnacle Healthcare Ltd system, PharmOutcomes, identified five MyWay trial participants who had booked an appointment in StopApp. Of these, two participants DNA and three attended their appointments. Of the three people who attended their appointments, only one is recorded as having responded ‘yes’ to the drop-down menu questions about participating in the MyWay trial and consenting to sharing their service use data with the research team. There was no indication in the objective SSS data that this participant set a quit date or achieved a 4-week quit.
Event rate calculation
Based on objective SSS data of five definitive bookings in StopApp (out of 59 participants randomised to this arm of the trial), the event rate is calculated to be 0.08 or 8% of those randomised. A standard margin of error calculation was employed [Z × √P(1 − P)/n], where Z is a constant 1.96, P is the booking rate (i.e. 5/59 = 0.08) and n is the potential sample size. Assuming a future trial recruitment size of 1000 and a future observed sample booking rate of 8%, we have calculated the margin of error to be 8% ± 1.68%, meaning that the true booking event rate at 95% confidence would lie between 6.32% and 9.68% of those randomised to StopApp. Applying the same calculation to the less conservative combination of objective and self-report data in the intervention arm provides a booking rate of 17%. Again, assuming a future trial sample size of 1000 and a future observed sample booking rate of 17%, the margin of error is ± 2.33%, suggesting that the true booking event rate at 95% confidence would lie between 14.7% and 19.3%.
In the control arm, the best indication we have is a booking rate of two out of 57 participants randomised to this arm. This provides a booking rate of 3.5%. Applying the margin of error calculation outlined above with an assumed sample size of 1000 and a future observed sample booking rate of 3.5%, the margin of error is ± 1.14%; suggesting that the true booking rate for the control arm would be between 2.36% and 4.64%.
Being conservative, if the difference was split between the lower of the two estimates of the event rate for the intervention arm (6.3% and 14.7%), which is 10.5%, and the higher estimate for the control arm event rate (4.6%) was deducted, this would result in a difference of 5.9% in the expected event rate of the control arm versus the intervention arm. A future definitive trial sample size calculation can be based on this percentage difference.
G*Power 3.1.9.448 was used to determine the sample size required to be able to detect a difference of 5.9% (where the event rates are 10.5% and 4.6%) for a dichotomous outcome. With the significance level held at 0.05 and power at a commonly accepted standard of 90%, we would need an overall sample size of 840 participants (420 in each arm) for a definitive trial. With the significance level held at 0.05, the power to detect a 5.9% difference would be increased to 95% with an overall sample size of 1038 participants (519 in each arm) for a definitive trial. Rounding the difference to be detected to 6% would push the required sample size down to 998 participants (499 in each arm).
Feasibility of accessing Stop Smoking Services data on attendance, quit dates set and 4-week quits for trial participants (including after the trial at the request of the National Institute for Health Research Public Health Research board)
Challenges with accessing objective service use data required for a full trial were identified in this feasibility study. One issue concerned the providers of SSS appointments within Warwickshire and the fact that not all of them were willing to support the trial. GPs were not involved in offering appointments at all, and some pharmacies also declined to be involved in providing appointments via StopApp. This should not have caused problems with accessing objective service data in isolation, but, in combination with the issues outlined below, trial participants booking appointments outside StopApp (at a non-participating venue) would only be reported using self-report.
A further issue relates to the fact that Pinnacle Healthcare Ltd, as the data controller for the SSS service user data, required consent from each participant for their service use data to be shared with the research team. Consent was required at the point of service access and at sign-up to the study online. To achieve this, Pinnacle Healthcare Ltd added three drop-down questions to its PharmOutcomes system in Warwickshire to establish whether or not service users attending during the trial period were trial participants and, if so, whether or not participants were willing to share their service use data with the research team. This created several issues. First, it transferred responsibility for obtaining consent to access important data to the participating pharmacists. Although the pharmacists were aware of these responsibilities, and had received training and support for setting up the trial, the research team cannot be certain that the pharmacists asked these questions, or asked them in a way that was understood. Some people were clearly study participants because they booked using StopApp and either answered ‘no’ to the question about being in the trial (in which case consent was not requested) or did not provide consent to their data being shared. We cannot be certain if they did not consent purely because of the way in which the questions were put to them or for another reason. This also highlighted that participants may not have understood what was meant by ‘participating in the MyWay trial’. A third issue is that, in the case of participants who booked but DNA an appointment, it was not possible to match the objective service data held in PharmOutcomes with self-report data because such participants DNA to provide consent.
Any differential recruitment and attrition rates across socioeconomic groups, ethnicity, age and sex
Baseline representativeness of the sample
The demographic characteristics of the sample were presented in Table 1. At baseline, females were over-represented in the sample (84/123, 68.3%). A large proportion of the sample were also white British (98/123, 79.7%). The most recent population census data show, however, that the population of Warwickshire is 93% white British, which suggests that we had over-represented black, Asian and minority ethnic groups in our sample. The mean age was 38.27 years [standard deviation (SD) 12.59 years] with a good representation across age groups (range 16–70 years). One participant reported being pregnant. Participant level of deprivation scores based on postcode (IMD quintiles) were evenly distributed between least and most deprived, with 38 (30.9%) participants in the two most deprived quintiles. In the NS-SEC item relating to job role and level of seniority/responsibility, the spread was relatively even, with 33 (26.8%) participants in a professional role, eight (6.5%) participants in a routine and manual role, and 29 (16.3%) participants who had never worked or were long-term unemployed. Most participants had undergone paid employment within the last 12 months (103; 83.7%) and most were employed (96; 78.0%) rather than self-employed (7; 5.7%). The data therefore demonstrated good reach across population groups and did not suggest that there were problems with health equity at study recruitment. Table 2 suggests, however, that there may have been a greater attrition rate in those from black, Asian and minority ethnic groups. Amendments to follow-up procedures for a full trial should aim to address this.
Analyses to assess any differential recruitment by the three settings
Demographic differences
Age, ethnicity and SES data by recruitment source are presented in Table 4. There were no significant differences in ethnicity [χ2 = 1.58, degrees of freedom (df) = 6; p = 0.950], measures of SES including NS-SEC (H = 0.335, df = 2; p = 0.846) and IMD quintiles (H = 1.566, df = 2; p = 0.457) between the three settings. However, there were significant differences in age between the three settings (F = 3.13, df = 2120; p = 0.047). The mean age was highest in the GP group (43.38 years) and lowest in the CSs group (37.16 years). However, the ANOVA test revealed only a very borderline significant difference in mean age between the three recruitment settings (F = 3.13, df = 2120; p = 0.047). These findings suggest that a slightly different age demographic was recruited dependent on recruitment setting. Further exploratory analyses were conducted to assess whether or not there were any further differences in relevant clinical outcomes by recruitment setting.
Characteristic | Recruitment source | ||
---|---|---|---|
GP practices | CSs | SM | |
Age (years) | |||
Mean (SD) | 43.4 (13.1) | 35.9 (10.6) | 37.2 (13) |
Minimum, maximum | 17, 67 | 16, 58 | 16, 70 |
Ethnicity (n) | |||
White British | 20 | 30 | 48 |
White other | 2 | 2 | 2 |
African/Asian | 1 | 1 | 4 |
All mixed background | 2 | 3 | 5 |
IMD score, n (%) | |||
< 8.49 (least deprived) | 2 (7.7) | 0 | 4 (6.6) |
8.50–13.79 | 1 (3.8) | 3 (8.3) | 4 (6.6) |
13.80–21.35 | 0 | 2 (5.6) | 6 (9.8) |
21.86–34.17 | 1 (3.8) | 2 (5.6) | 3 (4.9) |
> 34.18 (most deprived) | 4 (15.4) | 1 (2.8) | 2 (3.3) |
Clinical differences by recruitment setting
There were no significant differences in the mean health-related quality-of-life measure among the three recruitment settings (F = 1.29, df = 2115; p = 0.280). There were no significant differences in how soon after waking participants smoked, whether or not they had ever made an appointment at SSS, or made an appointment within the last 2 months. There were, however, significant differences in smoking frequency (H = 6.80, df = 2; p = 0.033) between the recruitment sites, with CS participants reporting the highest frequency, and GP the lowest (mean rank score 69.28 and 55.67, respectively). Intentions to stop smoking were also significantly different across sites (H = 6.60, df = 2; p = 0.037). GP participants had the greatest intentions to stop, and CS participants had the least (mean rank score 69.13 and 49.49, respectively). Perceptions about how likely it was that participants would stop smoking in the next 6 months were also significantly different across recruitment sites (H = 5.44, df = 2; p = 0.066). CS participants perceived that they were the least likely to stop, and GP recruits reported being the most likely to stop (mean rank score 50.89 and 67.96, respectively).
Differential attrition by socioeconomic status, ethnicity, age and sex
For sex or ethnicity, there was no significant association between participants who had completed follow-up data collection and those lost to follow-up. There were no significant differences in age (t = –0.203, df = 114; p = 0.839) between follow-up completers and non-completers. There were also no significant differences in any of the measures of SES (NS-SEC, IMD quintile and job description).
Participation in the MyWay process evaluation
There were significant differences in whether or not participants agreed to being contacted about the MyWay process evaluation among the three recruitment sites (H = 7.30, df = 2; p = 0.026). The mean rank scores indicated that CS participants were most likely to agree, and GP participants were least likely to agree.
Additional exploratory analyses
A range of exploratory analyses were carried out to identify any other differences or associations in the data. A significant association was found between internet access and ethnicity (χ2 = 21.56, df = 6; p = 0.001). Review of the frequencies suggests that participants of white background had greater access to the internet than those from mixed background, and African/Asian ethnicity. No other significant findings were found.
Harms
There were no known harms or unintended consequences to participants as a result of taking part in the MyWay trial.
Chapter 4 Process evaluation results
In addition to telephone interviews, there were several items completed by all participants on the self-report questionnaires that assessed items relevant to the process of participating. All participants described being happy with their level of access to the internet. Ninety-one (74%) participants had access on their mobile phone, 30 (24.4%) participants had access at home but not on their phone, and two (1.6%) participants had access at work but not at home or on their phone. Participants reported that they found out about the study in the following ways: online (51, 41.5%), by GP letter or text (28, 22.8%), via poster or leaflet (27, 22.0%), via word of mouth (13, 10.6%), from ‘the news’ (1, 0.8%) and via bus adverts (1, 0.8%). The majority of participants (91.3%) reported being happy for the researchers to access their data about SSS attendance and quit attempts. In the StopApp arm, 55 of 59 participants (93.2%) gave consent for this, compared with 50 out of 57 (87.7%) participants in the control arm. A total of 22 participants stated that they were happy to be contacted about the MyWay evaluation study process interviews.
Process interview results
Eleven process evaluation interviews were conducted, involving five trial participants and six members of staff involved in supporting the study.
Trial participant interviews
Twenty-two participants from the trial stated that they were willing to be contacted about taking part. All 22 participants were contacted, but only five (22.7%) agreed to take part; the remainder were unresponsive (lack of response to two e-mails, a telephone call and a voicemail) and were not pursued further. Two process study participants had been originally recruited via their GP, two via CSs and one via SM. Two participants had been assigned to the intervention arm and three participants had been assigned to the control arm. Participants reported that they struggled to recall what they had seen and the questions that they were asked because of the time delay since participation. Interviews lasted between 17 and 36 minutes and were all conducted over the telephone. Interviews were audio-recorded and transcribed verbatim. Analysis was conducted using the interview transcripts.
Participant interview results
Method for inviting to research
Participants described how they found out about the study. Responses included an e-mail (that was not provided by the research team) sent around the workplace from management, which the participants assumed was part of a ‘work health kick’ (804CS, lines 18–20). Text messaging was also mentioned (846GP, line 10) and was viewed as acceptable (846GP, line 32) and preferable to a letter:
When it’s a text you automatically look at it and read it and it’s saved on your phone so you can’t lose it. With a letter you just throw it in the bin.
846GP, lines 37–39
A SM participant was introduced to the study by friends (1003SM, line 7), and they later shared it (1003SM, lines 19–21). One participant reported a method the study team had not anticipated, in which their GP asked for the participant’s e-mail address during a consultation and sent something to them following this interaction (867GP, lines 2 and 3).
Reasons for taking part
Participants gave several reasons for choosing to take part. These reasons included the fact that the information was interesting (804CS, lines 26 and 27), they were happy to help and provide an opinion (867GP, lines 16 and 17) and they felt that the research might ‘help improve services’ (867GP, line 117):
You could gather more data of what things tend to be working for people, and what wasn’t.
867GP, lines 130–133
Participants noted how participation might aid smoking cessation attempts and be ‘worth a go’ (804CS, line 9; 743CS, line 6):
When I hear things about smoking it just puts me off and makes me think about quitting more.
846GP, lines 17 and 18
The provision of vouchers was also deemed a motivator for participation (846GP, line 14):
It pushes people who want to do it though, doesn’t it?
1003SM, line 284
Secondary reason was free money from Amazon [Amazon.com, Inc., Seattle, WA, USA].
743CS, line 8
Ease of participation, comprehension and time to complete
Ease of participation was mentioned, with some expressing that the process was not problematic (804CS, line 35):
I’d just say it was easy, easy to follow, straightforward.
846GP, lines 47–49
No part in it was difficult that I came across anyway.
1003SM, line 54
The log-in process within eNgage was described as ‘familiar’ and therefore something participants were accustomed to (804CS, line 55). Equally, one participant expressed that the process made it clear that they did not have to take part:
It was my choice to give my e-mail address. I don’t think in a way it forced anybody to do it.
867GP, lines 42–44
However, a lack of clarity about how the study would be conducted was conveyed (743CS, line 52), and concerns were expressed about the usability of the study research platform eNgage:
[The] website is not good so stopped using it, not user friendly and could not log on so gave up.
743CS, lines 87 and 88
Poor website access, poor communication, poor follow-up, incorrect platform.
743CS, lines 113–115
The online process was clunky, and it was difficult to participate in.
743CS, line 120
[Study hosted on] ineffective platform . . .
743CS, line 173
It leads to people disengaging.
743CS, line 190
Participants felt that the survey questions were relatively quick to complete:
. . . was just right – if it was too short you wouldn’t have taken the information in. If it had been long, you’d have probably given up before you deleted it.
804CS, lines 49–51
Literally takes you 5 minutes, not even that.
1003SM, lines 50, 51 and 67
Relatively quick and simple.
867GP, line 93
The number of questions was also deemed appropriate by participants:
There wasn’t too many questions, they were very easy to go through.
1003SM, lines 69–71
[It was] quite simple, quite understanding, and there was a lot of information on there about what it was on about and what it was for.
867GP, lines 29 and 30
Just the right amount.
867GP, line 71
Participants reported that they found the questions easy to comprehend (846GP, line 83) and were clear about what was being asked:
They were brilliant they were.
1003SM, line 105
Really explanatory, really simple, the questions were put in simple form. There was nothing too complicated about it.
867GP, lines 53 and 54
Participants all agreed that the level of personal detail collected was appropriate, relevant and not excessive:
I thought that it was easy and it was just enough to target what you were asking.
867GP, lines 75–77
Reminders and amount of contact from researchers
Reminders to complete the survey were seen as beneficial rather than obtrusive:
You get reminders which is good. Yeah I felt like since I did it there’s been a lot of contact after.
846GP, lines 47–49
They kept in touch, but you weren’t getting 100 e-mails a day.
846GP, line 65
You don’t feel like you’re being hounded but you don’t feel like you’re being ignored either.
846GP, lines 72 and 73
It was keeping me informed at times of what was happening, if I needed to do more, if I was to have telephone calls, so it kept me in the loop the whole time.
867GP, lines 180 and 181
There was an expectation about follow-up contact as participants all knew to expect further questionnaires, so this was deemed ‘fine’ (804CS, line 187). One participant felt it effectively nudged them to complete the survey again, which was good (1003SM, lines 221–223).
The role of participation on stopping smoking
A number of participants described how the process of taking part had spurred thoughts about stopping:
Made you think as you were completing it . . . I liked that.
804CS, lines 43–46
I think there should be more things out there like this that’ll encourage people to think about changing their life or giving up smoking . . . but yeah it was really good. Excellent [taking part in the study].
804CS, lines 273–281
It’s kind of made me urge to stop smoking a bit more. I booked as soon as I did them, its actually pushed me into going to the doctors. Taking part prompts you to stop – it sort of triggers me really to want to stop again.
846GP, line 210
It’s nice to have little reminders about smoking, it puts you off.
846GP, line 212
Its [the study] brought to light more in a way of how much I want to do it obviously for my children . . . I have cut down.
867GP, lines 195–198 and 200
One participant described how participation led to them booking a SSS GP appointment and nudged them closer to their goal with increased motivation:
It did prompt me to go to the SSS which actually, to be fair, has put it in my mind that even though I haven’t maintained that that I will be quitting smoking by the end of the year, because I don’t want to be a smoker . . . so it helped me in that way to resolve in my mind psychologically that when I hit that point of having I’m ready, that I will actually go and quit smoking.
743CS, lines 297–299
Acceptability of randomisation and self-report measures
When randomisation was explained to process evaluation participants, all participants reported an understanding of why randomisation was necessary and were comfortable with the process (e.g. 846GP, line 191), although one participant expressed disappointment to have not been assigned to the intervention arm (867GP, line 166). Receiving intervention content after the study ended for those in the control group was also deemed to be advisable (e.g. 846GP, line 195). The demographic questions were reasonable and expected for the study (804CS, lines 70 and 71; 743CS, line 196) and were seen to increase equality (867GP, lines 67 and 68). One participant suggested that data on sexual orientation and other protected characteristics (within the Equality Act49) to ensure ‘well-founded demographic data’ would have been beneficial additions (743CS, lines 196–207).
Smoking questions appeared to make sense and were relevant and thought-provoking (804CS, lines 75–77) (846GP, line 97):
It’s what the doctor would ask anyway and you need to give this information in order to stop.
1003SM, lines 109–112
Quality-of-life questions were also viewed in a particularly favourable light to demonstrate the impact of smoking and stopping on health in general:
I think they are just as relevant . . . makes you think about not just your smoking but your other health issues you have.
804CS, lines 84–86
Questions about SSS use were also considered to be acceptable (846GP, line 125):
I just thought it was brilliant that part.
1003SM, line 162
Participants themselves did not feel it was intrusive (804CS, line 114); however, they recognised that other people might find it to be (804CS, lines 116–118).
Acceptability of StopApp (for intervention arm participants only)
Participants randomised to the intervention arm were generally very positive about StopApp. Specifically, one participant liked the structure of the prompts to think about when to stop, how to stop and then receiving encouragement (804CS, lines 29 and 30). Participants appeared to like using StopApp (804CS, line 31) and how it provided a choice (804CS, line 126). They commented that it was useful to provide information about what services are out there:
. . . because they’re not advertised.
804CS, lines 124 and 125
Participants did not know that there were so many SSS out there:
I didn’t realise how many SSS there were around. I thought there was only one lot of SSS, but actually that surprised me. So that’s why, just think it encourages you to, you could try different ones as well. If you’re not happy with one, you can try a different one, so yeah.
804CS, lines 131–136
StopApp had prompted one participant to look at other websites and information about smoking:
It encouraged me to check out other sites and other research that was done.
804CS, line 170
Participants suggested that an improvement could be made by including more information on the SSS, as this might encourage people to go:
You have a bit of an idea yourself then before you get there.
804CS, lines 140–143
For instance, this information could comprise how long the session would last, what happens at the session and provision of information on vaping (804CS, line 153). More information was also needed up front about the choices available (804CS, lines 178–183), and the inclusion of more smoking cessation content and options for methods to try. This could include information about how to stop smoking and the different sources of help for smoking cessation (804CS, lines 58–60 and 62–64).
Another participant expressed a preference for an online intervention:
I don’t like talking to doctors about my problems, so I’d prefer to do it online because they gave you the answer back . . . with doctors I feel quite awkward talking to them . . . I felt a lot more comfortable doing it at home online.
1003SM, lines 99–102
Provision of incentives (vouchers)
It was argued that providing vouchers encourages people who ‘just need a nudge’ (804CS, line 258). However, concerns were also expressed, with one participant arguing that the data would be of a higher quality and have more integrity if no monetary incentive was provided (743CS, lines 379–390). Another participant argued that the vouchers should be offered after people complete the surveys:
Because I think a lot of people just do it for the vouchers and that’s not right.
804CS, lines 258–270
You shouldn’t always get paid to do stuff like that.
1003SM, lines 279 and 280
I’m not entirely sure whether you’re necessarily going to attract legitimate data.
743CS, line 375
They want to please the payment source.
743CS, line 377
Thoughts about why people may not want to take part in the study or complete follow-up measures
It was naturally recognised that ‘some [people] don’t want to be told about smoking’ (846GP, lines 228–230) and, therefore, were reluctant to participate:
People think of I’m not doing that, if I do, I might feel forced into quitting, not actually understand it’s for research.
867GP, lines 211–213
Additional thoughts about why people might choose not to take part in MyWay included time (804CS, line 250), privacy concerns (804CS, line 250) and capacity to use smart phones:
People are nervous about data.
743CS, lines 359–361
[If] not good at working phones . . . but it’s that straightforward you can’t really get it wrong.
846GP, line 224
One participant stated that there was a need to make it clearer who the follow-up e-mail was from so it was not accidentally deleted as spam (743CS, lines 288–291):
I think that realistically I had not made the cognitive connection to the interest and the desire to participate with any of the names or the titles included within the e-mail body or the e-mail address.
Another explained that their partner did not want to take part in the process interview because of anxiety; however, it was not clear whether or not the participant incorrectly thought that the telephone call was necessary to take part in the main study rather than just the process interviews:
[He’s] got very bad anxiety and he don’t like to talk on the phone, that’s the thing that put him off.
1003SM, lines 252 and 253
A further participant suggested that the study was not targeting the right age group and should instead focus more on younger smokers, whose habits are less entrenched (743CS, lines 24–28).
Other suggestions for additions and improvements to the study
Better advertising and marketing
More advertising (internet, press) was recommended (804CS, line 13), with a more strategic marketing plan necessary. Participants disclosed that ‘the leaflet was dreadful’ (743CS, line 34) and to instead ‘use a marketing model like AIDA [attention, interest, desire, action]’ (743CS, line 49). A need to create something to attract the younger demographic was also proposed. Participants felt that the current recruitment advertising was not memorable (743CS, lines 39–44), and needed to hold interest with marketing tools (743CS, lines 49–51) by offering something unusual or ‘interesting’ to generate a reaction’ (743CS, lines 56–58) and is ‘relevant to today’ (743CS, line 61). It was suggested that we needed to make it more fun and improve branding to ensure that the message ‘sticks in people’s minds’, use ‘psychological anchoring’ to get them ‘involved and invested’ (743CS, lines 19–325) and be memorable in terms of an experience rather than monetary reward (743CS, lines 337–339; 743CS, lines 19–325).
A suggestion was made to employ a marketer to design advertising that was non-public health focused (743CS, lines 72 and 73). Stickers on leaflets that emphasised free vouchers were appraised positively, as these highlighted the benefits of taking part (743CS, line 33).
Greater focus on social media recruitment
Participants suggested that recruitment methods should include SM channels such as Facebook and Instagram (846GP, line 21). Participants clarified that we should ‘use more social media that people actually go to’ (1003SM, lines 31 and 32) because the study did not have ‘a good social media presence’ (743CS, lines 11 and 12).
Alternative mode of study delivery
The need to include an app to go alongside participation was suggested (743CS, lines 53–55); however, it seemed that this participant thought that the trial’s objective was to get participants to stop smoking which was clearly not made clear enough to them. Concerns about poor communication and updates were also raised; for example, one participant said ‘it was a cool study and I thought it was a good idea . . . but the execution of communication was entirely wrong’ (743CS, lines 102–105), and another participant disclosing ‘the actual study but was like what’s going on, how are you studying me, you’re not interacting with me?’ (743CS, lines 109 and 110).
Additional questionnaires
One participant suggested that she would like to have received more surveys as it ‘helps you’ to think about stopping smoking (804CS, line 58). More regular updates were also requested as 2 months was deemed a long time between contacts from the research team (804CS, lines 212–214) and the provision of weekly updates and regular encouragement was suggested (804CS, line 217).
Follow-up timing
One participant felt that follow-up data collection should take place at a later date (e.g. 12 months post first survey) to ‘see if they’ve gone through with it [stopping smoking]’ (846GP, lines 215–217). Follow-up e-mails were described as being problematic as they ‘may look like spam when you get many’ (743CS, lines 284 and 285).
Consent
Participants also expressed the need to ensure that participants still gave consent for their data to be used at the point of attending SSS (743CS, lines 235–240).
General comments about stopping smoking
General thoughts about stopping smoking were also discussed in the interviews, with comments made about personal quit attempts, different methods tried, the psychological mindset needed to quit and reducing smoking behaviours to increase the likelihood of success at SSS (804CS, lines 199–209):
People will say they want to quit smoking whereas actually they probably don’t.
743CS, lines 353 and 354
Staff interview results
A total of 23 members of staff supporting recruitment to the study were invited to take part in the staff process interviews. This included professionals from a variety of CSs including public health services (PH), charities, GP practices and pharmacies. Six staff took part in the interviews. This included one GP (GPA), one GP administrator (GPB), one practice manager (GPC), one staff member from a charity providing PH and two pharmacy staff members (pharmacy32 and pharmacy14). The remaining 17 individuals did not respond to the contact or explained that they were unable to commit owing to time constraints. Interviews lasted between 5 and 14 minutes and were conducted by telephone.
Method by which staff found out about the study
Staff were introduced to the process evaluation study via a range of different means. GP-based staff were approached via the CRN contact (GPA, lines 2 and 3; GPC, lines 2 and 3) or through their managers or lead doctor (GPB, lines 2 and 3). The PH staff member received a flyer from their team manager (PH, line 2), whereas the two pharmacists were either contacted by the study team (pharmacy32, line 2) or through discussion with their manager (pharmacy14, lines 3–9). One pharmacy-based participant was a pharmacist with stop smoking advisor training (pharmacy32); the other was a pharmacy technician who had received the stop smoking advisor training.
Motivation to be involved
Staff stated that they were keen to be involved in research (GPA, lines 8–10) and were enjoying their involvement in the study recruitment process (pharmacy32, line 53):
They help people out in the long-run as well.
Pharmacy32, line 56
If not too onerous we just sign up to it.
GPC, lines 6 and 7
If nobody does it then you’re not going to get anywhere.
GPC, line 8
[The value is] we’re always learning about new research and things and what the outcomes going to be and it’s also interesting and it’s good to get involved.
GPC, lines 22 and 23
I think it’s quite interesting really and I think it can probably help you sort of like deliver a more focused service, I suppose, if you’re kind of seeing what people are leaning more towards.
Pharmacy14, lines 109–111
Positive experiences and thoughts about the study
Overall, staff reported that the study had minimal impact on their place of work and their time and their thoughts were largely positive:
It was fine, it was organised quite smoothly.
GPA, line 10
[that’s] the ones we like [minimal involvement]
GPC, line 10
For GP staff the study was largely managed by the CRN contact, whom they found very helpful and supportive:
Support fantastic [from CRN].
GPB, lines 21–24
[no involvement from research team but this was] absolutely fine.
GPB, lines 27 and 31
Well everything just worked, I mean like I was saying we only had one referral and they didn’t turn up.
Pharmacy32, lines 61 and 62
Pharmacy staff also described support, this time from the study team, which meant that the impact on them was minimal:
[Set-up support] was great.
Pharmacy14, line 75
They actually came to the pharmacy and they went through it, showed us everything that went on . . . they took a lot of time showing us exactly what to do [study team].
Pharmacy14, lines 78–83
I think its fine the way you conducted yourselves and the way the system was set up. There’s no problem at the pharmacy side, it’s all to do with the patients isn’t it.
Pharmacy32, lines 105 and 106
I’m quite happy with the way you ran it, and like I said it’s just a shame we didn’t have more patients. But that’s always going to be the case with stopping smoking.
Pharmacy32, lines 131 and 132
Pharmacy staff also described the ease of using the system (PharmOutcomes) (pharmacy14, line 113):
Not at all time consuming, seeing how the calendar works, fairly easy.
Pharmacy14, lines 69–72
The calendar system and intervention itself were also mentioned by pharmacy staff, including the provision of text reminders for appointments in the intervention arm, and the usefulness of booking appointments online for local SSS (GPC, lines 40 and 41):
I think some patients were quite grateful for it.
GPB, lines 39 and 40
One pharmacy staff member felt that participation in the study had an impact on the care she offered:
I [thought] . . . I’ve got to make sure I see this patient because he or she has been referred . . . I’ve got to make sure they have a good experience with us.
Pharmacy32, lines 114–116
Negative experiences of taking part
Negative experiences were described in terms of a lack of communication between the CRN and the study team, which resulted in sending out letters on a set day the contact reported being unaware of (GPA, lines 19–29); and a lack of funding available to send letters as expected. Pharmacy staff described feeling disappointed about the lack of bookings received (pharmacy14, lines 113–115):
It would have worked well if anything happened.
Pharmacy14, line 119
I would probably just be more than happy to carry on with this stage [awaiting referrals from StopApp].
Pharmacy14, line 150
The PH staff member felt that the study was not explained well by their manager, but did feel that the flyers that accompanied this had ‘a fair bit of information’ (PH, lines 5 and 6).
Feedback from the study team
One GP staff member was unhappy with the lack of contact and feedback from the study team (GPA, lines 39 and 40). Feedback about how the study had gone, how many patients were recruited and the demographic characteristics of who took part (GPA, lines 68–70) was judged to be missing:
Obviously you don’t want to be hounded or anything like that, but it was just silence, kind of forgotten about it really.
GPA, lines 45 and 46
We always like to know, especially from a mail-out point of view because you send these things out because the patients aren’t responding to you directly.
GPA, lines 49–51
It was suggested that updates were needed (GPA, line 52), as would be received for ‘commercial studies’, which could be delivered via a newsletter (GPA, line 53):
Did it work or did it not work or just different ways to maybe improve things.
GPA, lines 57 and 58
Suggestions for improvement
Staff suggested improvements that could be made to the study.
Feedback from the study team
Better feedback from researchers was requested (GPA, lines 89–92). This was echoed by the PH staff member, who felt that she was not as well informed as she hoped and did not have enough information to pass on; she suggested that a briefing would have been helpful (PH, lines 66–72). This had been offered to all CSs and, in fact, this organisation had withdrawn from the training after a training date was agreed.
Advertising and marketing
More advertising was proposed (GPA, line 95), including posters for the surgery (GPA, lines 97–105) and pharmacy (pharmacy14, lines 12–16), and more advertising in public areas ‘not just GP and children’s centres, etc.’ (pharmacy14, lines 28 and 29). Bus advertising was also mentioned by a pharmacy staff member who was not aware that the study had already utilised this method (pharmacy14, lines 33–35).
Training/face-to-face support for community settings
Training was suggested, however, concerns were raised that staff would forget the training as they do not use the system regularly enough (GPB, lines 59–62). Face-to-face support was also recommended:PH, lines 8 and 9
I think it possibly would have been good if someone did come in and deliver it maybe to staff . . . that might have helped just a little bit.
Vouchers
Vouchers were thought to have both positive and negative implications:PH, lines 74 and 75
. . . good thing but don’t want ‘to encourage people to do that just for the voucher’.
Intervention mode of delivery
The PH staff member queried whether the study could be more paper based (PH, line 100) or, alternatively, the provision of iPads (Apple Inc., Cupertino, CA, USA) ‘may have helped’ (PH, line 103), as potential participants might need computer support (PH, lines 107 and 108). She felt that PH staff could deliver such training (PH, line 114) but only with the necessary technology to present it (PH, lines 115 and 116).
Recruitment method
Both pharmacy staff members queried whether it is best to recruit smokers when these individuals enter the pharmacies directly (pharmacy32, lines 15 and 16). These comments emphasised the need for other ways in which people can access SSS as not everyone uses the internet (pharmacy14, lines 129, 130 and 143–145).
Adding technology to the pharmacy system
One staff member mentioned combining the existing PharmOutcomes calendar system that is linked to the intervention with a ‘PharmAlarm’, which indicates to pharmacists when appointments have been made/messages have been received, and is currently under development by Pinnacle Healthcare Ltd (pharmacy32, lines 32–38). This feature would be beneficial and would streamline how StopApp communicates with SSS advisors about appointments booked.
Thoughts about encouraging smokers to take part in the study
-
The PH staff member reported a reluctance in some smokers (PH, line 22) as they become defensive when discussing smoking (PH, line 25), making it difficult for staff to approach the issue (PH, line 26). She felt that ‘her’ smokers did not want to give up or were heavy smokers, which provided an additional challenge (PH, lines 58–63).
-
She felt that volunteers, in particular, found it uncomfortable (PH, lines 28–31). A lack of computer skills and age were also barriers (PH, lines 34–39), with participants primarily motivated by incentives rather than anything else (PH, line 45). She suggested the study was therefore more suited to a GP environment (PH, lines 50–56).
-
Pharmacy staff also reiterated the ‘difficulty of the patient group’:Pharmacy32, line 97
The onus is on the patient really.
Findings from the patient and public involvement group sessions during the study
Two meetings took place with the PPI group: one during data collection and one after data collection. The first meeting took place after recruitment had ended (June 2019) and was designed to review the user experience of participation. In order to do this, PPI members were asked to take part in the study as ‘dummy’ participants and to comment on the process throughout as part of a ‘think aloud’ exercise. There was also time to discuss some of the changes and issues within the study such as recruitment and the provision of vouchers. The second meeting took place at the end of the study (November 2019), and provided an opportunity to share with the group findings from the study, points of learning and future directions. The members had a final opportunity to comment on the research process and thoughts about a more targeted approach to SM in a full trial to include the use of external agencies offering services that enable targeted recruitment of smokers with the use of bespoke algorithms. A total of three PPI members attended both meetings and brief example notes were taken (see example PPI group meeting notes on the Journal Library website; URL: www.journalslibrary.nihr.ac.uk/programmes/phr/1518326/#/; accessed March 2021).
Chapter 5 Economic evaluation
Introduction
The aim of the economic component of the study was to assess the feasibility of methods to collect cost and outcome data to inform the parameters of a future definitive RCT designed to assess the cost-effectiveness of StopApp. As StopApp is a web-based behavioural change intervention to improve motivation and opportunities to access SSS, the additional resources required to deliver the app should be compared with usual care. This is important because any additional benefits associated with an intervention must be assessed in terms of any additional costs attributed. 50
The economic costs associated with smoking are considerable, and these costs affect the health-care sector, society and families. In England, the estimated costs to the NHS associated with smoking were £2.6B in 2015, with these costs including approximately 520,000 smoking-attributable hospital admissions among those aged ≥ 35 years. 51 In addition, there were broader costs for society estimated to be > £9B, associated with a reduction in years of disability-free life, increases in absences from work and early retirement, increased morbidity for those of working age and impacts on mortality. 52 Moreover, smoking contributes to health inequalities, as those from lower socioeconomic groups are more likely to take up smoking, and are less likely to quit successfully. 53
A detailed description of the intervention is reported in Chapter 2, Intervention. Briefly, the intervention involved a behaviour change intervention to improve motivation and opportunities to access SSS. StopApp had been developed prior to this study, with input from smokers and ex-smokers from a range of socioeconomic backgrounds. For the feasibility RCT, participants were randomised to receive either StopApp or standard promotion and referral to SSS. Those who were randomised to the intervention arm, and accessed StopApp, received tailored content to address negative perceptions and could instantly book an SSS appointment or receive a reminder to reaccess the intervention at a later date. The web app is linked to an online system that supports service providers to record and report SSS outcome data. Those randomised to the control arm received access to a web page with standard information and details about how to access SSS.
As fully online RCTs are relatively new, there is ongoing discussion about the best ways to recruit participants to such studies and the relative costs involved. 54,55 In addition to assessing the feasibility of methods to collect cost and outcome data to inform the design of a future RCT, this study component also aimed to inform decisions around recruitment methods by analysing the costs and outcomes associated with different recruitment approaches. Within the feasibility study, recruitment took place in three settings (GP practices, CSs and online). The economic component collected the costs associated with different recruitment settings to assess the resource use associated with recruitment via different methods to inform future discussions about the best methods to recruit to the definitive RCT.
Methods
The objective of the economic analysis was to assess methods for collecting the cost and outcome data for the StopApp arm and the control arm (usual care). We explored methods to measure costs from the perspective of the public sector (including the NHS, LAs and other public sector agencies) in line with the National Institute for Health and Care Excellence (NICE) recommendations for public health interventions. 56 The study also attempted to assess private costs incurred by participants such as productivity costs (e.g. time off work to attend the intervention sessions) and costs associated with travel and the purchase of stop smoking aids.
Recruitment costs
We captured costs associated with recruiting participants to take part in the study to inform the design of a definitive RCT. We assumed that these costs would not be required if the intervention was rolled out, as such costs would simply be absorbed within standard tobacco control activities conducted by LAs and health-care providers; thus, the analysis was conducted as a separate component. As described in Chapter 2, Recruitment settings and Recruitment and randomisation procedure, recruitment was conducted in three different settings using different methods. For recruitment costs incurred in GP settings, we gathered the costs associated with accessing patient details via GP lists and texting/writing to patients. Community-based recruitment involved staff recruiting participants in CSs such as libraries, children’s centres, well-being hubs and other locations, as well as using bus advertisements. We included costs associated with meeting staff at community locations and for the production of advertising materials. We also included the costs associated with producing and displaying bus advertisements. Data were collected on the costs associated with online recruitment via SM such as Facebook and Twitter, and online search engines.
Intervention costs
For both the intervention arm and the control arm, resource use data were collected prospectively by the trial research team. StopApp was developed in a previous study,27 and, hence, the costs of developing the app were not included in our analysis, as these are ‘sunk’ costs that would not need to be repeated if the intervention were to be rolled out. 57 The costs associated with the maintenance of StopApp and linkage to online systems to allow participants to locate and book SSS appointments were explored as part of the evaluation.
Ongoing costs associated with apps and online tools can be considered in terms of fixed and variable costs, with fixed costs being those that do not change according to the number of people who use them. 50 For interventions that involve apps or online elements, a high proportion of the total costs are often fixed; for example, costs could relate to system maintenance and data storage. These costs do not vary by the number of people who use them. This means that the cost of delivering the intervention (per participant) will be determined primarily by the number of participants assumed to be involved (the denominator). Thus, reporting costs using the number of participants in a pilot trial will not provide an accurate reflection of the true cost per participant if the intervention was rolled out to a wider group. 58 It is therefore recommended that the likely take-up rate is taken into account, to allow a more accurate estimate of the average costs per participant associated with delivering an app or internet-based intervention. 58 As a result, we adjusted the costs associated with the web app to give a more accurate estimate of these costs if the intervention was rolled out. Taking into account possible take-up rates, we allowed £2 per participant to cover the costs of the web-based tool, following methods adopted previously in similar studies. 59 In calculating this cost, we assumed that StopApp could be widely used across SSS, as it could be used by any service for which it is possible to embed an API link to the SSS data software system.
In addition, we included costs associated with the control arm. As is described in Chapter 2, Study protocol and intervention, participants randomised to the control arm were given access to a web page with standard information and details about how to access SSS. In line with the assumptions above, we assumed £1 per participant to cover the costs associated with maintaining and delivering the control information, given the scale at which this could be rolled out.
Health-care costs
Health-care resource use by participants was collected during the trial via an online questionnaire for participants in both trial arms. Data were collected on whether the patient had visited their GP/nurse at a GP surgery, or at a GP out-of-hours service, on accident and emergency (A&E) visits, and on the use of other services. Data were also collected on whether or not patients were prescribed any medications that were related to reducing smoking or trying to quit. Unit cost estimates were applied to resource use data to generate individual-level cost estimates. The sources of unit costs included routine and published literature. 60
Private costs
Time and costs spent travelling to and attending SSS lead to a cost incurred by participants and their families. This includes a cost linked to the time spent attending the appointment, any associated child-care costs, and potential time off work. In addition, there might be costs associated with purchasing over-the-counter medication and materials to support quit attempts, and other costs. Questions were included in the survey to measure these costs (see Chapter 2, Baseline measures).
Outcome data
Outcome data were collected for each trial arm. The main aim of this element was to test the feasibility and acceptability of collecting a range of outcome measures that could be used in a future definitive economic evaluation. This included data on health-related quality of life using the EQ-5D-5L42 at baseline and at 2 months, to allow a cost–utility analysis to be conducted in a future definitive trial, as recommended by NICE. 56 Utility values were calculated by mapping the five-level descriptive system data onto the three-level value set in line with NICE guidance. 61 As the trial was not powered to detect any changes in outcome measures, we calculated utility values for feasibility processes only. We also assessed the feasibility and acceptability of using the ICECAP-A instrument to measure general well-being at baseline and at 6 months. The ICECAP-A instrument has been developed as an outcome measure for economic evaluations in health and social care. 43,44 The ICECAP-A measures well-being in relation to capability, and allows for the measurement of outcomes that are broader than health,62 which is important in relation to smoking cessation, which has been demonstrated to have impacts beyond health. 63
Analysis
The analysis for the economic component was separated into two parts. The first part focused on identifying and assessing the incremental costs and benefits associated with different recruitment methods for the online pilot RCT. For this, we compared the costs attributed with various settings and methods of recruitment against the number of participants recruited. The second part focused on evaluating the feasibility of collecting cost and outcome data to inform methods for a future definitive economic evaluation. Issues such as trial processes for obtaining cost data and acceptability/completion of questions within the questionnaires were assessed. As this was a feasibility economic evaluation, the purpose was not to produce a definitive result with respect to the cost-effectiveness of the intervention, but to act as an exploratory analysis to refine the focus of methods for an economic evaluation alongside a future definitive trial.
Results
Key costs
Recruitment
To inform the design of a future definitive trial, information was gathered on the costs associated with various recruitment methods for the online pilot. We assumed that these costs would not be incurred if the intervention was rolled out, as such activities would be absorbed within usual tobacco control activities; hence, they are not included in the main analysis.
There were six participating GP practices that invited smokers to take part in the study. Recruitment by the GP practices included local co-ordination and set-up in each location as well as database searching and mail-outs/texts. These tasks were undertaken by GPs, practice managers and administrators (Table 5).
Activity | Time (hours) | Number of instances required | Total time (hours) | Unit cost (£) | Total cost (£) |
---|---|---|---|---|---|
Local co-ordination/set-up: GP involvement (per practice) | 0.5 | 6 | 3 | 156a | 468 |
Local co-ordination/set-up by practice manager (per practice) | 0.5 | 6 | 3 | 16.64b | 49.92 |
Preparation for database search and mail-out: level 1 simple search by practice manager (per practice) | 1 | 6 | 6 | 16.64b | 99.84 |
Simple database search: level 1 by practice manager (per practice) | 1 | 6 | 6 | 16.64b | 99.84 |
Checking of lists for exclusions by GP (per patient) | 0.003c | 1829 | 5.487 | 156a | 855.97 |
Sending of invitation letters by practice administrator (per patient) | 0.05d | 1678 | 83.9 | 9.63e | 807.96 |
Setting up for sending reminders by practice administrator (per patient) | 1 | 6 | 6 | 9.63e | 57.78 |
Sending of reminders by practice administrator (per practice) | 0.05d | 1597 | 79.85 | 9.63e | 768.96 |
Total cost | 3208.27 |
The costs incurred in CSs were primarily related to the printing and distribution of posters and leaflets. As shown in Table 6, a range of materials were created to publicise the study and were used in a range of locations.
Activity/items | Cost (£) | Number required | Total cost (£) |
---|---|---|---|
Strategy designa | 1250 | 0.5 | 625 |
Poster and leaflet designb | 280 | 2.5 | 700 |
Poster and leaflet designb | 140 | 2 | 280 |
Printing costs: A5 leaflets | 4000 | 230 | |
Printing costs: A4 posters | 250 | 54 | |
Printing costs: A3 posters | 50 | 31 | |
Printing costs: additional A5 leaflets | 4000 | 209 | |
Printing and displaying of adverts on inside of buses | 55.25 | 20 | 1105 |
Printing and displaying of adverts on bus streetliners | 150 | 10 | 1500 |
Printing and displaying of adverts on bus stops | 305 | 6 | 1830 |
Total cost | 6564 |
The final recruitment setting was via targeted adverts on SM and an online search engine and a variety of platforms were included (Table 7). The online recruitment strategy was developed by a specialist team and costs for this element were included. In addition, incentives were given to those who joined the trial but, as these were given across all trial arms, they are not included in the analysis.
Activity | Cost (£) | Number required | Total cost (£) |
---|---|---|---|
Strategy designa | 1250 | 0.5 | 625 |
SM and online poster/banner design | 280 | 2.5 | 700 |
Advertising via Google (per month) | 100 | 4 | 400 |
Advertising via Facebook (per week) | 83 | 9 | 747 |
Advertising via Twitter (per advert) | 60 | 6 | 360 |
2832 |
The online recruitment strategy was the most successful in terms of number of participants recruited. Costs per recruited participant appeared lower for this recruitment strategy (Table 8). The recruitment strategy in CSs had the highest level of cost per participant recruited, mainly due to higher costs associated with printing and designing posters and leaflets. However, it is important to note that this analysis does not give an indication of the level of engagement with the trial or interest in SSS; the only consideration is recruitment in terms of baseline measures being completed.
Recruitment strategy | Number of participants successfully recruited | Overall cost (£) | Cost per participant recruited (£) |
---|---|---|---|
GP practices | 26 | 3208.26 | 123.39 |
CSs | 36 | 6564.00 | 182.33 |
Online | 61 | 2832.00 | 46.43 |
Overall | 123 | 12,604.26 | 102.47 |
Intervention delivery and health-care resource use
Both the intervention and control arms involved an online intervention. As previously explained, we included a nominal cost per participant for receiving the StopApp intervention and control pathway, to reflect how both could be rolled out at scale to more participants than were included in the trial. We captured costs associated with attending SSS and using helplines (either by telephone or in person). A higher number of participants from the intervention arm reported that they had attended SSS and used the helpline (Table 9). In total, six participants from the intervention arm reported attending SSS (one booked via StopApp, one via face to face, three via telephone and one via both face to face and telephone). Data collection on the volume of attendances was less successful, with fewer than half of participants who attended giving details. In Table 9 we assumed that there had been one attendance per participant who indicated that they had attended.
Resource use | Cost item | Unit cost (£) | Control (n) | StopApp (n) | Total cost: control (£) | Total cost: StopApp (£) |
---|---|---|---|---|---|---|
Attendance at SSS | Per visit | 14a | 2 | 6 | 28 | 84 |
Calls to NHS national smokefree helpline | Per call | 6b | 5 | 30 | ||
Total cost | 28 | 114 |
As part of the study, data were collected from participants on health-care resource use associated with smoking-related conditions/illnesses and with stopping smoking. The main health-care resource use reported related to GP visits (Table 10).
Resource use | Control (n) | StopApp (n) | Total |
---|---|---|---|
GP consultation | 4 | 14 | 18 |
NHS outpatients | 6 | 6 | |
NHS walk-in centre | 4 | 4 | |
NHS 111 calls | 1 | 2 | 3 |
GP out of hours | 2 | 2 | |
Pharmacy | 3 | 3 | |
A&E | 4 | 4 | |
Total number of participants reporting use of health-care resources | 5 | 17 | 22 |
Total number of participants completing follow-up data | 25 | 32 | 57 |
Health-care resource use reported by participants and the associated costs are presented in Table 11. It is evident that the most frequently reported resource use related to primary care.
Resource use | Cost item | Unit cost (£) | Control (n) | StopApp (n) | Total cost: control (£) | Total cost: StopApp (£) |
---|---|---|---|---|---|---|
GP consultation (face to face) | Per visit | 39.23a | 4 | 23 | 156.92 | 902.29 |
GP telephone consultation (nurse led) | Per call | 6b | 3 | 9 | 18 | 54 |
NHS outpatient appointment | Per visit | 208c | 12 | 2496 | ||
NHS walk-in centre | Per visit | 39.23a | 2 | 78.46 | ||
NHS 111 calls | Per call | 6b | 1 | 6 | ||
GP out of hours | Per visit | 43.15 | 1 | 43.15 | ||
Pharmacy | Per visit | 14d | 2 | 28 | ||
A&E | Per visit | 93e | 12 | 1116 | ||
Total cost | 174.92 | 4723.90 |
The study included information about prescription costs associated with reducing cigarette consumption or quitting. Only a small number of participants reported that they had been given prescriptions for relevant medication, with only one participant from the control arm recording that they had received a prescription (Table 12). The data on the volume of the treatment prescribed were not comprehensively completed by participants, and thus the data below are based on standard doses only.
Resource use | Cost item | Unit cost (£) | Control (n) | StopApp (n) | Total cost: control (£) | Total cost: StopApp (£) |
---|---|---|---|---|---|---|
Nicotine replacement guma | Per item | 8.26 | 2 | 0 | 16.52 | |
Nicotine replacement lozengea | Per item | 7.40 | 3 | 0 | 22.20 | |
Nicotine replacement inhalera | Per item | 4.87 | 1 | 3 | 4.87 | 14.61 |
Nicotine replacement spraya | Per item | 13.03 | 1 | 0 | 13.03 | |
Nicotine replacement patchesa | Per item | 9.12 | 2 | 0 | 18.24 | |
Total cost | 4.87 | 84.60 | ||||
Total number of participants accessing prescriptions | 1 | 6 | ||||
Total cost per participant accessing prescriptions | 4.87 | 14.10 |
Costs borne by participants and families
We attempted to capture the costs incurred by those attending SSS, in terms of travel costs and time away from work and other responsibilities. As previously stated, only eight participants reported that they had attended SSS; hence the data gathered on costs associated with SSS were very limited. None of the participants recorded that they had needed to take time off work to attend their appointment. Two participants reported that they required assistance from a family member (who was not working) to attend their appointment. As very few participants had attended SSS, few data were captured on travel costs. Those participants who did attend SSS reported that they had walked or travelled by car, with no parking charges incurred.
We also attempted to capture costs that participants experienced that were related to smoking or trying to reduce cigarette consumption/quitting. A wide range of participants reported that they had purchased aids to help them to reduce or stop smoking (Table 13). However, data on the actual costs incurred were less comprehensive. The costs shown below are based on published costs. Those in the intervention arm reported a higher number of purchases relating to aids to help reduce or stop smoking.
Resource use | Cost item | Unit cost (£) | Control (n) | StopApp (n) | Total cost: control (£) | Total cost: StopApp (£) |
---|---|---|---|---|---|---|
Nicotine replacement gum | Per item | 12 | 2 | 24 | 0 | |
Nicotine replacement lozenge | Per item | 10.33 | 1 | 3 | 10.33 | 30.99 |
Nicotine replacement inhaler | Per item | 20 | 2 | 0 | 40 | |
Nicotine replacement spray | Per item | 15 | 2 | 0 | 30 | |
Nicotine replacement patches | Per item | 9 | 1 | 2 | 9 | 18 |
Acupuncture | Per treatment | 50 | 1 | 3 | 50 | 150 |
Hypnotherapy | Per treatment | 60 | 1 | 0 | 60 | |
ECs | Per item | 20 | 9 | 10 | 180 | 200 |
Apps | Per item | 1 | 2 | 0 | 0 | |
Books | Per item | 10 | 5 | 0 | 50 | |
Total cost | 273.33 | 578.99 | ||||
Total number of participants who purchased aids | 12 | 18 | ||||
Total cost per participant purchasing aids | 22.78 | 32.17 |
Summary of costs
Table 14 demonstrates the key data that were collected during the trial. The summary is illustrative only, owing to issues with collecting certain data on costs. However, the data collected demonstrate the importance of a comprehensive approach to cost collection, particularly in relation to the need to collect costs incurred by participants. As is shown in the table, participants reported a range of costs themselves, particularly in relation to the purchase of stop smoking aids. It is worth noting that, although the short-term costs are important to understand, in the longer term these costs could be heavily outweighed by savings associated with stopping smoking.
Type of resource use | Total cost: control (£) | Total cost: StopApp (£) |
---|---|---|
Intervention costs | 57 | 118 |
SSS attendance and helpline use | 28 | 114 |
Health-care service resource use | 174.92 | 4723.90 |
Prescription costs | 4.87 | 84.60 |
Private costs | 273.33 | 587.99 |
Total costs | 538.12 | 5628.49 |
Total cost per participant attending SSS | 269.06 | 938.08 |
Outcome data collection
As part of the feasibility trial, a range of outcome measures were successfully collected. This included data on health-related quality of life via the EQ-5D-5L instrument, and on general well-being using the ICECAP-A instrument. The EQ-5D-5L instrument was administered to inform the economic analysis, as it is a preference-based measure and can be used to calculate quality-adjusted life-years. Table 15 presents the results, demonstrating that the completion rates were similar for the other outcome measures. The completion rates for the ICECAP-A instrument were also similar to other outcome measures. The trial was not powered to detect any changes in outcome measures over time or differences between trial arms; rather, the aim was to assess the feasibility of collecting the outcome measures. It is worth noting that, in the short term, stopping smoking may have a negative impact on perceptions of well-being due to the side effects associated with withdrawal.
Outcome measure | Control, n (%)a | Control, mean (SD) | StopApp, n (%)a | StopApp, mean (SD) |
---|---|---|---|---|
EQ-5D-5Lb baseline | 57 (100) | 0.760 (0.292) | 59 (100) | 0.740 (0.277) |
EQ-5D-5L follow-up | 25 (43.86) | 0.779 (0.243) | 32 (54.24) | 0.687 (0.310) |
ICECAP-Ac baseline | 57 (100) | 0.769 (0.215) | 59 (100) | 0.780 (0.195) |
ICECAP-A follow-up | 25 (43.86) | 0.794 (0.181) | 32 (54.24) | 0.729 (0.175) |
Health economic discussion
The economic component of the study aimed to design processes and capture data on the costs and outcomes associated with the intervention and comparator programmes, to inform the methods for a future economic evaluation alongside a definitive trial. On the whole, processes were successfully put in place to measure the costs associated with delivering the intervention and the control. A range of outcome data were successfully collected, including the use of the ICECAP-A instrument. The cost data collected were comprehensive and, in particular, the private costs borne by the participants were well captured. The findings of this feasibility trial suggest that, in a definitive trial, it would be important to capture a broad range of costs as participants reported that they had incurred a range of private costs that should be considered for any future trial.
However, some challenges were also encountered. In particular, difficulties were encountered in terms of capturing data on health-care resource use (including information on prescriptions) for the control arm. As part of the study, we aimed to collect data on the private costs borne by participants and their families that were related to smoking or trying to quit. Again, data were less complete for those in the control arm. We recommend that, for the future trial, attention is paid to increasing completion rates for the control arm in relation to resource use and private costs. In addition, for a definitive trial, it would be more efficient to collect data on attendance at SSS, missed SSS appointments, and so on, using data directly obtained from SSS. Similarly, primary care resource use could be obtained through accessing primary care patient data (via Outcomes4Health, which is also owned by the data controller Pinnacle Healthcare Ltd). We would therefore recommend that appropriate DSAs are put in place as part of the trial development to facilitate this.
The outcome data were collected successfully. For a full trial, further outcome data might be collected directly from SSS to increase understanding of the potential impacts of the intervention. Collecting a range of outcome data would permit different types of economic evaluation to be conducted in a future definitive trial. Firstly, a cost-effectiveness analysis could be conducted, using the primary clinical outcomes included in the trial, including cost per SSS attendance and cost per quit attempt. In addition, collection of EQ-5D data would allow a cost–utility analysis to be undertaken as recommended by NICE. Finally, collection of data on more general well-being (via ICECAP-A) would enable a broader analysis to be undertaken, which would focus on outcomes beyond health.
The feasibility trial has provided valuable information about the costs and benefits associated with different methods of recruitment. It would seem that online methods are cheaper, based on the cost per recruit. However, there would need to be further consideration of the quality of engagement with the trial and the impacts on follow-up.
In a definitive trial the costs associated with attempting to reduce cigarette consumption and stop smoking would need to be offset against the longer-term savings associated with reductions in smoking-related morbidity and mortality. As is the case with many public health preventative programmes, longer-term costs and outcomes would need to be considered. 65 Hence, it will be necessary to use a decision-analytic model to extrapolate costs and benefits beyond the follow-up period. There is a range of evidence that could guide the development of the model, including the cohort model used to inform NICE guidance on interventions for smoking cessation66 and other more recent work in this area,67,68 and adaptation of an existing model will be explored. The development of a model will require data from meta-analyses on long-term abstinence, relapse and lifetime abstinence69 and utility estimates for smokers and smoking-related diseases. 70–72 The aim of the decision-analytic model will be to provide information on the potential longer-term costs/savings and health benefits associated with the StopApp intervention.
Chapter 6 General discussion
Recruitment, attrition and equity of methods
The primary objective of this study was to estimate recruitment and attrition rates of participants for a future definitive RCT of StopApp across three settings (GP surgeries, CSs and online) at baseline, intervention access, and 2 months’ follow-up. Our data show that it was possible to recruit 123 smokers over 116 days in a single LA area through an overall recruitment rate of 1.06 smokers per day. Over 12 months it should therefore be possible to recruit 387 smokers within a single LA SSS provider, and the number of LAs could be increased to achieve a required full-trial target. For example, recruitment via four LAs with a similar population size might be expected to achieve baseline recruits in excess of 1500 participants. We also found that withdrawal of consent prior to randomisation did not occur. All participants who signed up to the study online and completed consent procedures also completed baseline measures in full.
Recruitment rates varied across the three recruitment settings. SM recruitment outperformed the other two settings, with 61 eligible recruits, compared with 36 recruits in CSs and 26 recruits in GP practices. There was, however, a significant problem with fraudulent sign-ups from people via SM who were not eligible to take part as a result of not living or working in Warwickshire. Changes to the protocol to prevent fraudulent activity are needed for any future definitive trial, and are discussed in Key costs that would be incurred in delivering the intervention and usual care, including a comparison of did-not-attend rates between each arm of the trial.
Social media recruitment was also the most cost-efficient method of recruitment at £50.20 (including e-voucher payment) per recruit, compared with £184.42 per recruit in CSs and £123.39 per recruit in GP practices. SM also achieved the lowest rates of attrition at the 2-month follow-up, with 61% of participants completing follow-up measures compared with only 50% of participants in CSs and 27% of GP recruits. These rates of attrition are relatively high. Researchers73 who recruited smokers to a ‘taster session’ of smoking cessation support via a tailored letter from their GP achieved a 77% follow-up rate. Other studies, however, have tended to use more intensive methods to pursue follow-up data, including phoning participants and pursuing only the primary outcome measures for those most resistant to completing follow-up measures. 46,73 We only sent two auto-generated e-mails, 2 weeks apart, with a request to complete measures online via a web link and the offer of a £5 e-voucher for doing so, and achieved follow-up with 61% of participants in the SM setting. In a future definitive trial, we could aim to increase follow-up to a target of 80% of participants by employing more intensive follow-up procedures as other studies have done.
Even with the very unintrusive follow-up methods employed in the current study, the numbers achieved suggest that a full-scale trial would be feasible if using SM as the main driver of recruitment. If using SM alone, in six LA areas over 18 months with a similar population size to Warwickshire, it may be possible to achieve as many as 1830 baseline recruits and 1116 participants’ self-report data at follow-up (based on 61% follow-up achieved in the present study), or more with more intensive follow-up methods. The same calculation, applied to CS data alone, suggests a baseline of 1080 recruits and 540 participants at follow-up. The most problematic recruitment setting was the GP surgery setting. Follow-up rates were very poor and, based on the data obtained in this feasibility study, to recruit 1000 smokers at baseline we would need to invite smokers from 228 GP surgeries. Gilbert et al. 46 had to expand the number of GP settings that they recruited from during their RCT of an invite letter to a taster SSS session, but they ultimately achieved more than 4000 baseline recruits through 99 GP surgeries.
It seems likely that the relative success of the SM recruitment method is due, in part, to the fact that we ran the entire trial online, and, to our knowledge, we were one of the first to attempt this for a smoking cessation-related trial. Those who came across study information online and signed up were therefore likely to be smokers who were familiar with and comfortable in online spaces. Participants recruited via SM and the CSs were also more likely to have an interest in stopping smoking before deciding to take part. This will potentially have affected participants in both the intervention group and the control group and reduced the difference in SSS booking between groups. Engaging in the study will have been more effortful for those who found out about it in CSs (e.g. saw a poster or bus campaign message). We also know that no-one who received a paper-based letter from their GP signed up to the study, and a preference for texts was discussed in the process interviews. All sign-ups through GP surgeries came from those who were sent text messages containing a link to the study website. Because StopApp is a digital intervention designed to reach smokers and motivate those interested in quitting to use the convenient SSS booking system it provides, it is not surprising that those who could achieve a rapid and seamless engagement process by simply clicking on a link signed up in greater numbers. Furthermore, it seems likely that the differential attrition rate across settings was, to some extent, due to the different motivational drivers that may affect smokers in each setting. Smokers who signed up to the study following a message from their GP may have been more reluctant to participate in the 2-month follow-up if they had not been prompted to book and attend a SSS appointment than those who had the greater perceived anonymity of participation via other routes. No outcomes from the study were shared with GPs, but GPs will remain aware of the smoking status of their patients in the future; therefore, social desirability effects may have played a role in the low follow-up from smokers via GP recruitment.
Given the novelty of our intervention and online trial methods, it is difficult to make direct comparison with recruitment to other smoking-related trials. Recruitment via GP settings does, however, provide some notable points for comparison. First, based on the number of adult smokers in England, which currently stands at nearly 15%,1 and the fact that there was a combined list size of 52,608 in the six GP surgeries from which we recruited, we should have expected to invite close to 7890 smokers to the study. Instead, only 1602 smokers were identified on participating GP lists by the CRN that supported the study and sent either a letter or a text message invitation. We therefore invited an average of 267 smokers per participating surgery, compared with an average of 1075 smokers per surgery in previous research. 73 It is not clear why so few smokers were identified. Of those ineligible, 76 people were ineligible because their first name was not listed first alphabetically out of the known smokers in the household (as per the criterion applied by the CRN). It is possible that the data that these surgeries held on the smoking status of patients were simply inaccurate and left many smokers unidentified in the searches of patient lists. Second, of the 1602 smokers who were invited to participate, only 1.62% responded to that invitation and completed baseline measures. Other studies have achieved recruitment at closer to 11%,74 although recruitment has been as low as 0.27% in other smoking studies. 75 It is possible that the very brief information about the study that was included in the text messages was not sufficient to entice individuals to find out more. Participants in the Gilbert et al. 46 study received a much more detailed, personalised invitation that contained health risk information tailored to their own current health status, age, sex and smoking habits. Furthermore, participants in another trial76 were invited to attend SSS in correspondence sent by their GP that sought to update their medical records on smoking, which may have encouraged and normalised participation, leading to a higher recruitment rate. It is possible, therefore, that recruitment via GP practices in a future definitive RCT may be improved by a combination of better identification of smokers on GP patient lists and changes to the brief text message invitation.
Differences in some participant demographic characteristics were noted between the recruitment settings, with slightly older participants consenting via GP recruitment and younger participants consenting from CSs. CS participants also reported smoking the most cigarettes, with GP recruits smoking the fewest. GP participants also reported the greatest intentions and likelihood of stopping smoking, with CS participants reporting the least. The fact that the samples recruited from each setting vary slightly is not surprising, but needs to be considered in any decision about the recruitment methods in a future definitive RCT.
Attrition at the point of intervention access was measured using analytics data from Matomo, which indicated that StopApp received 60 home page views from the 59 participants randomised to receive this content, indicating that one participant viewed it more than once. Of the 59 participants, we can see that nine participants (15%) proceeded beyond the home page, interacting further with the site. We also know that five participants (56% of those who engaged with it) booked a SSS appointment using StopApp. Of the 60 home page views, a total of 51 exits occurred. It remains unclear whether the 51 exits from the home page occurred without any reading of the content or scrolling, and whether or not the exits were intentional. During the PPI group think-aloud session, it became apparent that one member of the group had been randomised to the intervention arm, but had been unprepared for being presented with the home page for StopApp and clicked the back button on the assumption that they had missed some introductory text regarding what it was. This inadvertently took them out of eNgage, and therefore the study, and they were unable to return to StopApp; therefore, it is possible that some of the exits may have been unintentional and we may have lost some people who would have otherwise booked an appointment at SSS via StopApp. Consequently, as we cannot be clear whether some attrition at intervention access was accidental or intentional, our event rate for booking at SSS may be under-reported. A future definitive trial may benefit from the provision of information at the end of the baseline measures to brief participants about what they are about to view, including clear messages to avoid clicking the back button, if the facility to do this cannot be removed. The research team has also received intensive training in Matomo analytics since the end of the trial so that we can make better use of the sophisticated participant tracking options available in the software for both trial arms in a planned future definitive trial.
We observed recruitment and attrition rates across socioeconomic groups, ethnicity, age and sex. The only issue identified was a small imbalance in sex compared with the general population profile, with a greater number of females recruited. However, this is not an unusual finding as it has been experienced in other smoking trials (e.g. 70% females77 and 59.4% females78). One way to address this may be to use a targeted approach in SM recruitment to monitor imbalances and respond accordingly. This might include the possibility of utilising algorithmic approaches within SM advertising that enable reach to particular groups, at particular times and in particular contexts; for example, this may include recruiting from particular workplace settings where there are known to be higher numbers of male smokers.
The demographic and smoking status profiles of those who completed follow-up and those who did not were broadly similar, suggesting that the study did not incur an attrition bias. There was, however, some indication from reviewing demographic data in Table 2 that those from black, Asian and mixed ethnic groups were lost to follow-up in greater numbers than those from white ethnic groups. Given the need for more intensive follow-up methods in general, this could be addressed by pursuing these follow-up data, in particular from minority groups, to avoid a lack of representative follow-up data in a future trial. In addition, some exploratory data analysis identified that a lack of internet access was associated with being from a mixed, black or Asian ethnic background. Recent government data1 suggest that overall internet use by ethnicity is actually lowest for white people (90.5% have access) and Asian ethnic groups (92.4% have access), compared with 92.8% of black people, and 95.1% of ‘other including mixed’ ethnicity groups. Interestingly, the data showed that internet access was lowest across ethnic groups and regions (i.e. among Asian people and people living in the West Midlands), compared with other UK regions, where this study was conducted. This suggests that our sample was not necessarily representative of internet use among ethnic minorities across the general UK population; therefore, the likelihood is that a definitive RCT of the StopApp intervention would reach a sufficiently broad sample of the population.
Acceptability of randomisation methods
Our data suggest that the randomisation methods used were feasible and acceptable to participants, as disclosed during the process evaluation interviews. We did identify an issue with some participants missing randomisation; however, this was likely to have been because participants closed the browser too early, not realising that there was something to see after the measures. For the full trial, careful guidance would be provided to participants and embedded monitoring and alerts would be used to ensure that randomisation was occurring for all participants, as this cannot adequately be managed by the research team or a clinical trials unit manually without time lapse and error.
Acceptability of the trial measures
The measures used were largely completed as expected, although some items contained consistent missing data or participants provided contradictory answers, suggesting that people were not willing to complete these measures or were finding them confusing or boring. Further work with our PPI group is required to revise these particular measures before including them in a full trial. Feedback on the measures was principally positive from the process interview findings, with agreement that item measures were not too onerous and were easy to comprehend and that there was a clear rationale for their inclusion.
Key costs that would be incurred in delivering the intervention and usual care, including a comparison of did-not-attend rates between each arm of the trial
Key costs incurred in delivering the intervention and usual care have been set out and discussed in Chapter 5. It is clear that the ongoing costs of delivering the intervention rather than usual care are negligible for commissioners and, therefore, the main costs to be considered are the resource use costs of those in the intervention arm versus those in the control arm. The provision of e-vouchers may have contributed to the high rate of participants who DNA their appointments. In a full definitive trial, the provision of e-vouchers needs some consideration to avoid attracting fraudulent behaviour. There is an indication in the data that resource use was higher among intervention arm participants, but there were also issues with the completeness of data from control arm participants on these measures that would need to be addressed for a full trial, in conjunction with our PPI group. Data on DNA rates were not sufficiently reliable nor available in the local Warwickshire LA for a comparison of associated costs to be carried out (see Feasibility of accessing Stop Smoking Services data). However, in supporting the feasibility trial, Pinnacle Healthcare Ltd did add a field to indicate whether an individual attended the appointment or not (i.e. DNA) to their routine process (as a result of supporting the MyWay trial), which we believe is a successful tangible impact on SSS attendance reporting. However, this approach did demand additional effort from staff completing the fields and puts the onus on these staff members to accurately report attendance.
Feasibility of accessing Stop Smoking Services data
This feasibility trial demonstrated that data on participant SSS use and smoking outcomes could be accessed for a future definitive trial; however, some amendments to the study set-up procedures and protocol are needed to ensure that there is accurate and complete data collection. For the data to be released to the research team, Pinnacle Healthcare Ltd required additional, direct consent from participants, collected by the pharmacist at the first appointment, rather than online in eNgage. We found that some participants who had signed up for the trial stated that they were not part of the trial when prompted by pharmacists and, therefore, were not invited to consent and, as a result, their data could not be accessed. Consequently, it will be necessary to consider more carefully how consent for access to SSS data is collected, when and by whom, to ensure the collection of accurate, objective smoking outcomes. We recognised that some participants may choose to book an appointment at SSS within a GP practice or a non-participating pharmacy (which could include participants in the StopApp arm who chose to book an appointment via alternative means to the web app, despite being prompted to do so via StopApp content). In the trial design that we were ultimately forced to adopt for this feasibility study, we were unable to collect data about these services and could only rely on self-reports of appointment bookings, which may be subject to bias. This could be addressed in the planning of a future definitive RCT of StopApp by working with Pinnacle Healthcare Ltd and the tobacco control commissioners in participating LAs or public health departments to set up a clear data-sharing protocol that meets the research needs. This could be organised with all relevant parties from the outset and alongside preparation of an application for trial funding.
Rate of Stop Smoking Services booking in the intervention and control groups to estimate the event rate of the primary outcome measure for a future trial and to support future trial sample size calculations
The booking event rate calculations based on the available objective and self-report data from participants have suggested that a future definitive RCT should be powered to detect an approximate 6% difference between the control and intervention arms. This is a conservative estimate of the likely difference between trial arms, since the true difference in booking rate between trial arms could be nearly three times higher. This more conservative estimate is in line with other smoking trials targeting attendance at SSS. 46,76 A sample size calculation based on this difference for a dichotomous outcome such as booking a SSS appointment with the significance level held at 0.05 and power at 0.95 suggests that we need an overall sample size of 1036 participants or 518 participants in each arm of a definitive trial to detect a difference of 6%. This sample size has been identified as being possible to achieve if using a well-targeted SM campaign with or without other recruitment settings also included.
Limitations, proposed improvements for a full trial and potential of StopApp
Based on our findings, a full trial would benefit from some amendments to the feasibility trial protocol. First, a more proactive approach to reducing attrition and improving follow-up data collection is warranted; for example, in many smoking trials follow-up data are collected via telephone calls with participants,46,73,79 whereas in this trial two e-mails were sent. This would also help to ensure representation of all demographic groups in follow-up data. Second, we demonstrated that use of SM was the most effective and cost-effective recruitment method. However, the approach to recruitment was not sufficiently targeted towards smokers and could benefit from employing specialist digital marketing agencies to recruit on our behalf to design bespoke algorithms to strategically target specific groups online for trial recruitment. Third, as discussed in Data collection at participating Stop Smoking Services, we experienced difficulties in collecting information from SSS, including issues with pharmacists obtaining consent. Prior to commencing a future definitive trial, there is a need to work closely with the data controller, Pinnacle Healthcare Ltd, LAs that commission SSS and are interested in participating in a trial, and the SSS providers using the software within pharmacies and GP surgeries to ensure that we can collect this information without attrition. For example, consent might be collected by the research team via text messaging at the point of SSS booking, or evidence of baseline consent could be provided to the data controller as proof for data release. Last, because of issues with the study management software (i.e. eNgage not being fully optimised for mobiles initially, and problems identified with hitting the back button in StopApp after randomisation), we would need to fully test digital trial procedures in each site of a multisite full RCT ahead of starting recruitment.
If a proposed full trial establishes that StopApp is effective and cost-effective in increasing attendance at SSS, over and above usual SSS promotion activity, then it also meets the APEASE criteria (affordability, practicality, effectiveness/cost, acceptability, side effects/safety and equity),80 in that it can be instantly scaled up nationally owing to the widespread existing use of PharmOutcomes systems by SSS. It could also be incorporated into the GP-based version (Outcomes4Health), which is also owned by Pinnacle Healthcare Ltd, which uses an identical platform. StopApp would incur minimal additional costs to local commissioners, who will require only an updated map of services for their own locality; therefore, it offers a highly affordable addition to existing SSS provision, targeting smokers who might otherwise have avoided services as a result of mistaken beliefs about what they offer, their approach and how to access them. A future trial will compare cost savings associated with StopApp, including reduced administrative burden in arranging appointments and the possibility of fewer wasted sessions due to non-attendance or attendance by those who were referred by health professionals and are, therefore, less motivated to stop smoking. The seventh World Health Organization report on the global tobacco epidemic81 states that its principal focus is the need for greater comprehensive cessation support worldwide. One-third of the global population (2.4 billion) in 23 countries now have access to SSS, two billion more people than in 2007. Given the clear recommendations to offer continued cessation support, reaching the harder-to-reach entrenched smokers from more deprived backgrounds becomes ever more imperative.
Conclusions
This feasibility RCT has demonstrated that it is possible to recruit and retain sufficient smokers to assess the effectiveness and cost-effectiveness of StopApp at increasing booking for and attendance at SSS compared with usual SSS promotion. The trial methods were identified as acceptable and equitable. A number of challenges relating to the feasibility of a definitive RCT have, however, been identified. It is arguable that these could be addressed in the process of the planning of a future RCT, before a funding application is submitted. The acquisition of service use data will demand careful planning with all relevant data controllers and processors in order to ensure that there is complete objective data on all bookings, attendances/DNAs, quit dates set and 4-week quits achieved among participants. We foresee this resolution involving detailed discussions with all relevant parties (including Pinnacle Healthcare Ltd) and a funding application for the full RCT being submitted only when everyone is in agreement with the appropriate data access. A clear SM recruitment strategy is also required to successfully target smokers and reduce the likelihood of fraudulent activity, and a NIHR-preferred provider has been identified for targeted SM marketing to address this need. In addition, in line with methods in other smoking trials, greater efforts to pursue follow-up data from participants would be applied to minimise missing data.
Research recommendations
The following specific research recommendations based on the experience of delivering this study should be considered and appropriately incorporated in the planning of a future definitive RCT of StopApp, and may also be of relevance in the planning of similar digital or smoking-related feasibility and full RCTs:
-
Agree data-sharing protocols with the data controller and all data processors, which, in the case of the current trial, includes Pinnacle Healthcare Ltd, LA commissioners and LA service providers. Where LAs wish to be involved as a study site, they could use commissioning contract arrangements to require data sharing for research purposes.
-
Booking and attendance remain the primary outcome measure but a broader measurement approach to secondary outcome measures, including recording any positive changes in smoking behaviour, may encourage those who have not successfully booked an appointment and/or quit smoking to respond to follow-up measures about their smoking behaviour and outcomes.
-
Obtain better access to objective service use data to reduce the reliance on self-report data on resource use by participants for health economic analyses.
-
Apply more intensive follow-up procedures (e.g. telephone calls and requests to access the primary outcome items) to reduce attrition and to ensure that demographic representation is achieved at follow-up compared with baseline.
-
Work with the PPI group ahead of trial data collection commencement to refine and streamline all digital trial processes and the experience for participants – address any identified glitches or poor experiences. In particular, check that the study management software being used is functioning seamlessly and monitor this regularly.
-
Make sure follow-up communications with participants are very clear about who they are from to avoid being viewed as ‘spam’ and consider reducing an initial follow-up from 2 months to avoid participants ‘forgetting’ about involvement.
-
Improve the quality of study and digital marketing by using a professional digital marketing company to support targeted marketing to potential recruits via SM.
-
Engage a stakeholder and public involvement co-ordinator to support not just the PPI group activities but communicate regularly with all relevant stakeholders about progress, delivery and outcomes of the research.
Acknowledgements
The feasibility trial was funded by the NIHR PHR programme (NIHR portfolio number 38004). The authors wish to thank NIHR, all the participants who took part in the study, all the staff who supported the research and, in particular, the members of the PPI group for their valuable contributions.
The authors would also like to thank Dr Maxine Whelan for her contributions in addressing reviewers’ comments following peer review.
Ethics approval
Ethics approval for the study was granted by the West Midlands – Edgbaston Research Ethics Committee (reference 18/WM/1070).
Contributions of authors
Emily A Fulton (https://orcid.org/0000-0002-5979-3207) (Assistant Professor Health Psychology) conceived the idea for StopApp and led the stage 1 application for funding to NIHR, co-led the research project and co-wrote this monograph.
Katie Newby (https://orcid.org/0000-0002-9348-0116) (Associate Professor Health Psychology) supported the set-up of the trial data management system and took responsibility for the information governance of the study.
Kayleigh Kwah (https://orcid.org/0000-0003-2307-1285) and Lauren Schumacher (https://orcid.org/0000-0003-3430-1816) (Research Assistants Psychology) conducted the day-to-day management of the study, liaised with all study sites and ran the process interviews.
Kajal Gokal (https://orcid.org/0000-0002-2020-1876) (Research Fellow Health Psychology) managed the ethics application and liaison with the CRN and GP practices.
Louise J Jackson (https://orcid.org/0000-0001-8492-0020) (Lecturer Health Economics) conducted the health economic assessment.
Felix Naughton (https://orcid.org/0000-0001-9790-2796) (Senior Lecturer Health Psychology) and Tim Coleman (https://orcid.org/0000-0002-7303-4805) (Professor Primary Care) advised on the trial design and supported study decision-making throughout.
Alun Owen (https://orcid.org/0000-0002-4358-7118) (Associate Professor; Statistician) advised on all statistical analysis and reporting of statistics.
Katherine E Brown (https://orcid.org/0000-0003-2472-5754) (Professor Health Psychology) led the stage 2 application for funding, co-led the research project and co-wrote this monograph.
All authors contributed to the drafting of this monograph and read and approved the final version.
Publications
Fulton EA, Brown KE, Kwah KL, Wild S. StopApp: using the behaviour change wheel to develop an app to increase uptake and attendance at NHS Stop Smoking Services. Healthcare 2016;4:31.
Fulton EA, Kwah KL, Wild S, Brown KE. Lost in translation: transforming behaviour change techniques into engaging digital content and design for the StopApp. Healthcare 2018;6:75.
Fulton E, Newby K, Gokal K, Kwah K, Schumacher L, Jackson LJ, et al. Tailored digital behaviour change intervention with e-referral system to increase attendance at NHS stop smoking services (the MyWay project): study protocol for a randomised controlled feasibility trial. BMJ Open 2019;9:e028721.
Kwah K, Fulton E, Brown K. Accessing National Health Service Stop Smoking Services in the UK: a COM-B analysis of barriers and facilitators perceived by smokers, ex-smokers and stop smoking advisors. Public Health 2019;171:123–30.
Data-sharing statement
All data requests should be submitted to the corresponding author for consideration. Access to available anonymised data may be granted following review.
Patient data
This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health and Social Care.
References
- Office for National Statistics . Adult Smoking Habits in the UK: 2018 2019. www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/healthandlifeexpectancies/bulletins/adultsmokinghabitsingreatbritain/2018 (accessed 30 October 2019).
- World Health Organization (WHO) . WHO Global Report on Trends in Tobacco Smoking 2000–2025. Second Edition 2020. www.who.int/tobacco/publications/surveillance/trends-tobacco-smoking-second-edition/en/ (accessed March 2021).
- West R. Stop Smoking Services: Increased Chances of Quitting. National Centre for Smoking Cessation and Training Briefing: 8. London: National Centre for Smoking Cessation and Training; 2012.
- NHS . Smokefree 2020. www.nhs.uk/smokefree/help-and-advice/local-support-services-helplines (accessed 5 January 2020).
- Department of Health and Social Care . Local Stop Smoking Services: Service Delivery and Monitoring Guidance 2011 12 2011. www.ncsct.co.uk/usr/pub/delivery-and-monitoring-guidance-2011-and-2012.pdf (accessed 30 October 2019).
- Shahab L. Why Use CO-verified 4-week Quit Rates as the Primary Measure of Stop Smoking Service Success? NHS Centre for Smoking Cessation and Training Briefings 14. National Centre for Smoking Cessation and Training. London: National Centre for Smoking Cessation and Training; 2014.
- Kmietowicz Z. Action is needed to boost uptake of stop smoking services, say campaigners. BMJ 2015;351. https://doi.org/10.1136/bmj.h4578.
- McNeill A, Brose LS, Calder R, Bauld L, Robson D. Vaping in England, an Evidence Update, February 2019. A Report Commissioned by Public Health England 2019. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/821179/Vaping_in_England_an_evidence_update_February_2019.pdf (accessed 30 October 2019).
- Beard E, Brose LS, Brown J, West R, McEwen A. How are the English Stop Smoking Services responding to growth in use of electronic cigarettes?. Patient Educ Couns 2014;94:276-81. https://doi.org/10.1016/j.pec.2013.10.022.
- NHS Digital . Statistics on NHS Stop Smoking Services 2017. https://digital.nhs.uk/data-and-information/publications/statistical/statistics-on-nhs-stop-smoking-services-in-england (accessed 30 October).
- Farrimond H, Abraham C. Developing E-cigarette friendly smoking cessation services in England: staff perspectives. Harm Reduct J 2018;15. https://doi.org/10.1186/s12954-018-0244-8.
- All Party Parliamentary Group on Smoking and Health . Delivering the Vision of a ‘Smokefree Generation’. The All Party Parliamentary Group on Smoking and Health Response to ‘Prevention Is Better Than Cure’ 2019. https://ash.org.uk/wp-content/uploads/2019/03/2019-APPG-report.pdf (accessed 30 October 2019).
- Department of Health and Social Care . Smoke-Free Generation: Tobacco Control Plan for England 2019. www.gov.uk/government/publications/towards-a-smoke-free-generation-tobacco-control-plan-for-england (accessed 30 October 2019).
- Dobbie F, Hiscock R, Leonardi-Bee J, Murray S, Shahab L, Aveyard P, et al. Evaluating Long-term Outcomes of NHS Stop Smoking Services (ELONS): a prospective cohort study. Health Technol Assess 2015;19. https://doi.org/10.3310/hta19950.
- Langley T, Szatkowski L, Lewis S, McNeill A, Gilmore AB, Salway R, et al. The freeze on mass media campaigns in England: a natural experiment of the impact of tobacco control campaigns on quitting behaviour. Addiction 2014;109:995-1002. https://doi.org/10.1111/add.12448.
- Statista . Smartphone Usage in the United Kingdom (UK) 2012–2019, by Age 2002. www.statista.com/statistics/300402/smartphone-usage-in-the-uk-by-age/ (accessed 30 October 2019).
- Office for National Statistics . Smoking Inequalities in England, 2016 2016. www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/drugusealcoholandsmoking/adhocs/008181smokinginequalitiesinengland2016 (accessed 30 October 2019).
- Brown J, Michie S, Geraghty AW, Yardley L, Gardner B, Shahab L, et al. Internet-based intervention for smoking cessation (StopAdvisor) in people with low and high socioeconomic status: a randomised controlled trial. Lancet Respir Med 2014;2:997-1006. https://doi.org/10.1016/S2213-2600(14)70195-X.
- West R, May S, West M, Croghan E, McEwen A. Performance of English stop smoking services in first 10 years: analysis of service monitoring data. BMJ 2013;347. https://doi.org/10.1136/bmj.f4921.
- Roddy E, Antoniak M, Britton J, Molyneux A, Lewis S. Barriers and motivators to gaining access to smoking cessation services amongst deprived smokers – a qualitative study. BMC Health Serv Res 2006;6. https://doi.org/10.1186/1472-6963-6-147.
- Benson FE, Stronks K, Willemsen MC, Bogaerts NM, Nierkens V. Wanting to attend isn’t just wanting to quit: why some disadvantaged smokers regularly attend smoking cessation behavioural therapy while others do not: a qualitative study. BMC Public Health 2014;14. https://doi.org/10.1186/1471-2458-14-695.
- Copeland AL, Businelle MS, Stewart DW, Patterson SM, Rash CJ, Carney CE. Identifying barriers to entering smoking cessation treatment among socioeconomically disadvantaged smokers. J Smok Cessat 2010;5:164-71. https://doi.org/10.1375/jsc.5.2.164.
- Ussher M, Etter JF, West R. Perceived barriers to and benefits of attending a stop smoking course during pregnancy. Patient Educ Couns 2006;61:467-72. https://doi.org/10.1016/j.pec.2005.06.021.
- Vogt F, Hall S, Marteau TM. Examining why smokers do not want behavioral support with stopping smoking. Patient Educ Couns 2010;79:160-6. https://doi.org/10.1016/j.pec.2009.10.007.
- Kale D, Gilbert H, Sutton S. An exploration of the barriers to attendance at the English Stop Smoking Services. Addict Behav Rep 2019;9:005-5. https://doi.org/10.1016/j.abrep.2018.10.005.
- Kwah KL, Fulton EA, Brown KE. Accessing National Health Service Stop Smoking Services in the UK: a COM-B analysis of barriers and facilitators perceived by smokers, ex-smokers and stop smoking advisors. Public Health 2019;171:123-30. https://doi.org/10.1016/j.puhe.2019.03.012.
- Fulton EA, Brown KE, Kwah KL, Wild S. StopApp: using the behaviour change wheel to develop an app to increase uptake and attendance at NHS Stop Smoking Services. Healthcare 2016;4. https://doi.org/10.3390/healthcare4020031.
- Matcham F, McNally L, Vogt F. A pilot randomized controlled trial to increase smoking cessation by maintaining National Health Service Stop Smoking Service attendance. Br J Health Psychol 2014;19:795-809. https://doi.org/10.1111/bjhp.12078.
- Gilbert H, Sutton S, Morris R, Parrot S, Galton S, Nazareth I. Evaluating the effectiveness of using personal tailored risk information and taster sessions to increase the uptake of smoking cessation services: study protocol for a randomised controlled trial. Trials 2012;13. https://doi.org/10.1186/1745-6215-13-195.
- Borland R, Partos TR, Yong HH, Cummings KM, Hyland A. How much unsuccessful quitting activity is going on among adult smokers? Data from the International Tobacco Control Four Country cohort survey. Addiction 2012;107:673-82. https://doi.org/10.1111/j.1360-0443.2011.03685.x.
- Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011;6. https://doi.org/10.1186/1748-5908-6-42.
- Michie S, Hyder N, Walia A, West R. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav 2011;36:315-19. https://doi.org/10.1016/j.addbeh.2010.11.016.
- Das S, Tonelli M, Ziedonis D. Update on smoking cessation: e-cigarettes, emerging tobacco products trends, and new technology-based interventions. Curr Psychiatry Rep 2016;18. https://doi.org/10.1007/s11920-016-0681-6.
- Chevalking SKL, Ben Allouch S, Brusse-Keizer M, Postel MG, Pieterse ME. Identification of users for a smoking cessation mobile app: quantitative study. J Med Internet Res 2018;20. https://doi.org/10.2196/jmir.7606.
- Herbec A, Brown J, Shahab L, West R, Raupach T. Pragmatic randomised trial of a smartphone app (NRT2Quit) to improve effectiveness of nicotine replacement therapy in a quit attempt by improving medication adherence: results of a prematurely terminated study. Trials 2019;20. https://doi.org/10.1186/s13063-019-3645-4.
- Fulton E, Newby K, Gokal K, Kwah K, Schumacher L, Jackson LJ, et al. Tailored digital behaviour change intervention with e-referral system to increase attendance at NHS stop smoking services (the MyWay project): study protocol for a randomised controlled feasibility trial. BMJ Open 2019;9. https://doi.org/10.1136/bmjopen-2018-028721.
- Fulton EA, Kwah KL, Wild S, Brown KE. Lost in translation: transforming behaviour change techniques into engaging digital content and design for the StopApp. Healthcare 2018;6. https://doi.org/10.3390/healthcare6030075.
- Kotz D, Brown J, West R. Predictive validity of the Motivation To Stop Scale (MTSS): a single-item measure of motivation to stop smoking. Drug Alcohol Depend 2013;128:15-9. https://doi.org/10.1016/j.drugalcdep.2012.07.012.
- Hummel K, Candel MJ, Nagelhout GE, Brown J, van den Putte B, Kotz D, et al. Construct and predictive validity of three measures of intention to quit smoking: findings from the International Tobacco Control (ITC) Netherlands Survey. Nicotine Tob Res 2018;20:1101-8. https://doi.org/10.1093/ntr/ntx092.
- Ministry of Housing . Communities and Local Government. The English Indices of Deprivation 2019. Research Report 2019. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/833947/IoD2019_Research_Report.pdf (accessed 30 October 2019).
- Office for National Statistics . The National Statistics Socio-Economic Classification (NS-SEC) 2019. www.ons.gov.uk/methodology/classificationsandstandards/otherclassifications/thenationalstatisticssocioeconomicclassificationnssecrebasedonsoc2010 (accessed 30 October 2019).
- Devlin N, Shah K, Feng Y, Mulhern B, van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. Health Econ 2018;27:7-22. https://doi.org/10.1002/hec.3564.
- Al-Janabi H, Flynn TN, Coast J. Development of a self-report measure of capability wellbeing for adults: the ICECAP-A. Qual Life Res 2012;21:167-76. https://doi.org/10.1007/s11136-011-9927-2.
- Al-Janabi H, Peters TJ, Brazier J, Bryan S, Flynn TN, Clemens S, et al. An investigation of the construct validity of the ICECAP-A capability measure. Qual Life Res 2013;22:1831-40. https://doi.org/10.1007/s11136-012-0293-5.
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77-101. https://doi.org/10.1191/1478088706qp063oa.
- Gilbert H, Sutton S, Morris R, Petersen I, Wu Q, Parrott S, et al. Start2quit: a randomised clinical controlled trial to evaluate the effectiveness and cost-effectiveness of using personal tailored risk information and taster sessions to increase the uptake of the NHS Stop Smoking Services. Health Technol Assess 2017;21. https://doi.org/10.3310/hta21030.
- Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials 2014;15. https://doi.org/10.1186/1745-6215-15-264.
- Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods 2009;41:1149-60. https://doi.org/10.3758/BRM.41.4.1149.
- Great Britain . Equality Act 2010 2010.
- Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Oxford: Oxford University Press; 2015.
- Public Health England . Cost of Smoking to the NHS in England: 2015 2017. www.gov.uk/government/publications/cost-of-smoking-to-the-nhs-in-england-2015/cost-of-smoking-to-the-nhs-in-england-2015 (accessed 30 October 2019).
- Department of Health and Social Care . Towards a Smokefree Generation: A Tobacco Control Plan for England 2017. www.ncsct.co.uk/usr/pub/Towards%20a%20Smoke%20free%20Generation%20-%20A%20Tobacco%20Control%20Plan%20for%20England%202017-2022.pdf (accessed 30 October 2019).
- Hiscock R, Bauld L, Amos A, Fidler JA, Munafò M. Socioeconomic status and smoking: a review. Ann NY Acad Sci 2012;1248:107-23. https://doi.org/10.1111/j.1749-6632.2011.06202.x.
- Akers L, Gordon JS. Using Facebook for large-scale online randomized clinical trial recruitment: effective advertising strategies. J Med Internet Res 2018;20. https://doi.org/10.2196/jmir.9372.
- Whitaker C, Stevelink S, Fear N. The use of Facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res 2017;19. https://doi.org/10.2196/jmir.7071.
- National Institute for Health and Care Excellence . Methods for the Development of NICE Public Health Guidance (Third Edition) 2012. www.nice.org.uk/process/pmg4/resources/methods-for-the-development-of-nice-public-health-guidance-third-edition-pdf-2007967445701 (accessed 30 October 2019).
- Tate DF, Finkelstein EA, Khavjou O, Gustafson A. Cost effectiveness of internet interventions: review and recommendations. Ann Behav Med 2009;38:40-5. https://doi.org/10.1007/s12160-009-9131-6.
- Graham AL, Chang Y, Fang Y, Cobb NK, Tinkelman DS, Niaura RS, et al. Cost-effectiveness of internet and telephone treatment for smoking cessation: an economic evaluation of The iQUITT Study. Tob Control 2013;22. https://doi.org/10.1136/tobaccocontrol-2012-050465.
- Estcourt C, Sutcliffe L, Mercer CH, Copas A, Saunders J, Roberts TE, et al. The Ballseye programme: a mixed-methods programme of research in traditional sexual health and alternative community settings to improve the sexual health of men in the UK. Programme Grants Appl Res 2016;4. https://doi.org/10.3310/pgfar04200.
- Curtis L. Unit Costs of Health and Social Care 2018. Canterbury: PSSRU; 2019.
- National Institute for Health and Care and Excellence (NICE) . Position Statement on Use of the EQ-5D-5L Value Set for England n.d. www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/technology-appraisal-guidance/eq-5d-5l (accessed 27 January 2020).
- Lorgelly PK. Choice of outcome measure in an economic evaluation: a potential role for the capability approach. PharmacoEconomics 2015;33:849-55. https://doi.org/10.1007/s40273-015-0275-x.
- Piper ME, Kenford S, Fiore MC, Baker TB. Smoking cessation and quality of life: changes in life satisfaction over 3 years following a quit attempt. Ann Behav Med 2012;43:262-70. https://doi.org/10.1007/s12160-011-9329-2.
- Joint Formulary Committee . British National Formulary n.d. www.medicinescomplete.com (accessed 30 October 2019).
- Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health 2013;13. https://doi.org/10.1186/1471-2458-13-1001.
- Flack S, Taylor M, Trueman P. Cost-effectiveness of Interventions for Smoking Cessation. York: York Health Economics Consortium; 2007.
- Coleman T. A dynamic, modifiable model for estimating cost-effectiveness of smoking cessation interventions in pregnancy: application to an RCT of self-help delivered by text message. Addiction 2019;114:353-65. https://doi.org/10.1111/add.14476.
- Leaviss J, Sullivan W, Ren S, Everson-Hock E, Stevenson M, Stevens JW, et al. What is the clinical effectiveness and cost-effectiveness of cytisine compared with varenicline for smoking cessation? A systematic review and economic evaluation. Health Technol Assess 2014;18. https://doi.org/10.3310/hta18330.
- Etter JF, Stapleton JA. Nicotine replacement therapy for long-term smoking cessation: a meta-analysis. Tob Control 2006;15:280-5. https://doi.org/10.1136/tc.2005.015487.
- Ara R, Brazier JE. Populating an economic model with health state utility values: moving toward better practice. Value Health 2010;13:509-18. https://doi.org/10.1111/j.1524-4733.2010.00700.x.
- Tengs TO, Lin TH. A meta-analysis of quality-of-life estimates for stroke. PharmacoEconomics 2003;21:191-200. https://doi.org/10.2165/00019053-200321030-00004.
- Trippoli S, Vaiani M, Lucioni C, Messori A. Quality of life and utility in patients with non-small cell lung cancer. Quality-of-life Study Group of the Master 2 Project in Pharmacoeconomics. PharmacoEconomics 2001;19:855-63. https://doi.org/10.2165/00019053-200119080-00007.
- Naughton F, Jamison J, Boase S, Sloan M, Gilbert H, Prevost AT, et al. Randomized controlled trial to assess the short-term effectiveness of tailored web- and text-based facilitation of smoking cessation in primary care (iQuit in practice). Addiction 2014;109:1184-93. https://doi.org/10.1111/add.12556.
- Gilbert HM, Sutton SR, Leurent B, Alexis-Garsee C, Morris RW, Nazareth I. Characteristics of a population-wide sample of smokers recruited proactively for the ESCAPE trial. Public Health 2012;126:308-16. https://doi.org/10.1016/j.puhe.2011.11.010.
- Buller DB, Meenan R, Severson H, Halperin A, Edwards E, Magnusson B. Comparison of 4 recruiting strategies in a smoking cessation trial. Am J Health Behav 2012;36:577-88. https://doi.org/10.5993/AJHB.36.5.1.
- Murray RL, Coleman T, Antoniak M, Stocks J, Fergus A, Britton J, et al. The effect of proactively identifying smokers and offering smoking cessation support in primary care populations: a cluster-randomized trial. Addiction 2008;103:998-1006. https://doi.org/10.1111/j.1360-0443.2008.02206.x.
- Gram IT, Larbi D, Wangberg SC. Comparing the efficacy of an identical, tailored smoking cessation intervention delivered by mobile text messaging versus email: randomized controlled trial. JMIR Mhealth Uhealth 2019;7. https://doi.org/10.2196/12137.
- Iacoviello BM, Steinerman JR, Klein DB, Silver TL, Berger AG, Luo SX, et al. Clickotine, a personalized smartphone app for smoking cessation: initial evaluation. JMIR Mhealth Uhealth 2017;5. https://doi.org/10.2196/mhealth.7226.
- Free C, Knight R, Robertson S, Whittaker R, Edwards P, Zhou W, et al. Smoking cessation support delivered via mobile phone text messaging (txt2stop): a single-blind, randomised trial. Lancet 2011;378:49-55. https://doi.org/10.1016/S0140-6736(11)60701-0.
- Michie S, Atkins L, West R. The Behaviour Change Wheel: A Guide To Designing Interventions. Sutton: Silverback Publishing; 2014.
- World Health Organization (WHO) . WHO Report on the Global Tobacco Epidemic – Offer Help to Quit Tobacco Use 2019. https://apps.who.int/iris/bitstream/handle/10665/326043/9789241516204-eng.pdf?ua=1%26ua=1 (accessed 30 October 2019).
List of abbreviations
- A&E
- accident and emergency
- ANOVA
- analysis of variance
- API
- application programming interface
- BCT
- behaviour change technique
- CO
- carbon monoxide
- CRN
- Clinical Research Network
- CS
- community setting
- df
- degrees of freedom
- DNA
- did not attend
- DSA
- data-sharing agreement
- EC
- electronic cigarette
- EQ-5D-5L
- EuroQol-5 Dimensions, five-level version
- GP
- general practitioner
- ICECAP-A
- ICEpop CAPability measure for Adults
- IMD
- Indices of Multiple Deprivation
- LA
- local authority
- NICE
- National Institute for Health and Care Excellence
- NIHR
- National Institute for Health Research
- NS-SEC
- National Statistics Socio-economic Classification
- PH
- public health services
- PHR
- Public Health Research
- PPI
- patient and public involvement
- RCT
- randomised controlled trial
- SES
- socioeconomic status
- SM
- social media
- SSS
- Stop Smoking Services
Notes
Supplementary material can be found on the NIHR Journals Library report page (https://doi.org/10.3310/phr09050).
Supplementary material has been provided by the authors to support the report and any files provided at submission will have been seen by peer reviewers, but not extensively reviewed. Any supplementary material provided at a later stage in the process may not have been peer reviewed.