The decision-making process in recommending electronic communication aids for children and young people who are non-speaking: the I-ASC mixed-methods study

Primary data collection activities

Primary data collection activities were focus groups, individual semistructured interviews and surveys. All data collection was UK-wide.

Data management

Data were managed in accordance with the General Data Protection Regulation and Manchester Metropolitan University’s data protection policy.

Coding scheme design

Two methods of data coding were adopted to support the analysis of focus group and interview data: thematic analysis52 and framework approach. 53 The former supported the inductive development of a network from open coding, and the latter enabled the deductive and inductive development of themes. With both approaches, analysis followed the stages of data interpretation, that is familiarisation with the data through to mapping and interpretation.

Intercoder reliability testing

Intercoder reliability testing was set up for all qualitative activity. This included lead researchers for the relevant WPs reading the transcripts to gain a sense of the data. Two researchers independently re-read and assigned initial codes to meaningful segments of the data, which was followed by discussion and some preliminary consensus on coding and then core research group members sharing and discussing coding. The key researchers led an iterative process of code refinement to develop the thematic network or map to the existing framework. Finally, the network and frameworks were illustrated using quotations from the data and presented to the wider research group for sense checking, credibility and transferability. Two researchers external to the core research group (critical friend group or additional co-investigator staff involved in the public involvement WP) provided independent coding reliability reviews to reduce the impact of researcher bias.

Quantitative data analysis

Two stated preference surveys investigated the decision-making of AAC practitioners. The first method, termed best–worst scaling (BWS) case 1, allowed the relative importance of factors in decision-making to be assessed. It quantified what AAC professionals regarded as the most important factors related to both children and their AAC systems. The second method, a discrete choice experiment (DCE), built on the findings from the BWS. Professionals completed this survey by making choices related to which of three hypothetical AAC systems they would choose for a stated hypothetical child.

Analysis was grounded in random utility theory and for BWS included estimates of the β parameters obtained from random parameters logit models. Analysis in the DCE included a one-sample Kolmogorov–Smirnov test.

Generic point

Guidance sought from the NHS NW Research Design Service on the extent of double screening required indicated (Sarah A Rhodes, NIHR Research Design Service – North West, and University of Manchester, 8 October 2019, personal communication) that 100% double reviewing was not necessary as the reviews did not seek evidence of the effectiveness of interventions. The reviewers for systematic reviews 1 and 3 were very experienced and had several published systematic reviews, including Cochrane reviews. One of the reviewers for systematic review 2 had no previous experience of systematic reviewing; accordingly, it was decided that, for systematic review 2, 100% of the records would be double reviewed.

Inclusion criteria

Screening process

The search process yielded 53,158 records, which were imported into EndNote™ [Clarivate Analytics (formerly Thomson Reuters), Philadelphia, PA, USA] for screening. One researcher conducted the title and abstract review and full-text review. An independent researcher completed inter-rater reliability checks. This researcher independently applied the inclusion and exclusion criteria to a sample of search records. The sample combined half of the included papers with a sample of the excluded papers as a reliability measure and the agreement rate was 100%.

Quality appraisal

The inclusion/exclusion criteria were used to screen all returned records, resulting in 112 studies being included in the full-text review. Following the full-text review, 14 papers were identified for quality appraisal (Figure 3). Two quality appraisal tools appropriate to the study designs were used: the Critical Appraisal Skills Programme (CASP) Qualitative Checklist58 and the Evidence-Based Management Survey Checklist. 59 The 14 papers were appraised using relevant quality indicators and weight of evidence and relevance to the research question. 60 Two researchers independently completed this process before comparing the results. Eleven studies were agreed to meet the quality threshold. Three studies were excluded as they did not meet the agreed quality threshold: one from the USA,61 one from Ireland62 and one from Canada. 1

FIGURE 3.

Systematic review 1 PRISMA flow chart of language and communication characteristics of children.

Data extraction

Two researchers extracted relevant data from the studies. Information on study characteristics was extracted.

Language understanding

Two studies16,64 reported on language understanding. In one study,16 68 children demonstrated their spoken sentence understanding. The results indicated that, although children followed a typical pattern of development, spoken sentence understanding was more typical of a child younger than their (chronological) age. In the second study,64 of seven young men who had been using AAC for ≥ 15 years, all participants scored below average (i.e. younger than their age) on language understanding. Participants did not have identified learning difficulties, so the results suggested that factors affecting language development included language learning opportunities and the influence of communicating with graphic symbols. Currently, we know too little; further research is needed to identify how AAC systems and language learning opportunities best support children to achieve their potential.

Expressive use of symbols

Three studies68,70,71 considered how participants generated graphic symbol output. There was variation across participants, with many using limited vocabularies successfully to communicate a range of messages and language structures. In one study, 72 adolescents used an average of two symbols per message to tell stories. It was clear that they used extra skills to communicate with fewer words, for example using strategies such as ‘it sounds like’. Some participants used different word orders from the spoken language in their environment. These word-order variations were used to make the intended message clearer. Word-order patterns did not appear to be related to age or motor impairment severity but did correspond to the child’s language-understanding abilities. In summary, there was wide variation in the language abilities of and graphic symbols used by children using aided AAC, and some children had developed highly creative skills to overcome the restriction of small expressive vocabularies. The limited data suggest that research is needed to understand how children develop these strategies and how to support the effective use of these.

Narrative skills

Two papers from one research project looked at the storytelling abilities of children who use AAC,66,67 which varied widely across participants. One study66 found that some children could give many details in their story, while others were unable to do this. All participants had some difficulty with independently telling stories and using different story elements. The results indicated that maintaining the topic of the story was an area of strength. Children overly relied on one-word messages and because of the limitations of their communication aids they often depended on their communication partner and allowed the partner to take control of the story. Specifically, children did not use many action words (e.g. ‘run’ or ‘jump’) or pronouns (e.g. ‘he’ and ‘she’) in their stories. Nor did they use joining words (e.g. ‘and’ or ‘because’). As a result, it was harder to follow their storylines. These findings suggested that children may need more opportunities to tell stories and may need AAC vocabularies that support storytelling.

Grammatical morphemes

Grammatical morphemes are small words or word parts of a language that can be used to change meaning. For example, adding the past-tense ending ‘–ed’ turns ‘I look’ into ‘I looked’ to show that the action happened in the past. Some studies looked at how children who use AAC understand and use grammatical morphemes. 30,65,69 The results suggest that children who use AAC find it harder to understand and use grammatical morphemes than children who are speaking. Children may leave out grammatical morphemes because they have limited opportunities to use them or because communicating with symbols may make it harder to learn grammatical morphemes. These results indicate that children may need more opportunities to learn grammar morphemes even if they are not expected to use them in everyday communication (e.g. drill and practice activities). It can be proposed that more thought needs to be given to AAC system design to support children learning to use grammatical morphemes.

Social competence

Using a test of social competence, one study63 looked at the ability of four young adults to adjust their communication abilities based on different social situations. Participants had most success with communicating messages that could be fulfilled with one word (e.g. ‘yes’) and with providing information (e.g. ‘That’s my coat’). Participants had difficulty expressing messages more formally (e.g. ‘Could I look at that?’) and expressing feelings (e.g. ‘I love it!’). The participants’ performance scores did not reflect their age, the number of symbols available on their AAC system or their language understanding. A further result was that participants who interacted with more people on a weekly basis achieved higher scores on the social competence test. This suggests that having more communication opportunities is important in supporting ongoing communication skill development.

Search procedure

A review protocol was drawn up using the PRISMA-P51 template and a search strategy was developed based on the research question. The search strategy is detailed in Box 2. The review commenced in March 2016.

Searches were carried out on the EBSCOhost, EMBASE, ProQuest, Scopus, Web of Science, the Cochrane Library and Augmentative and Alternative Communication journal electronic databases. When possible, searches were refined by excluding categories that could not be related to AAC (e.g. animal studies).

Inclusion and exclusion criteria

Studies were included if their focus was the implementation of a graphic symbol AAC system. Papers were excluded if the focus was entirely on literacy. Any primary research study of any design conducted from 1970 to 2018 was included. Searches were conducted in English, but records returned in any language were considered for the review. Unfortunately, a translation of one paper from Korean could not be sourced within the available timeframe.

Screening process

The citations were downloaded to a local database and managed using the JabRef software tool (www.jabref.org). Owing to the number of papers, the initial review process was carried out in two stages. An initial title and abstract review stage excluded articles not related to AAC. Two researchers independently carried out a title and abstract review of the remaining literature to screen for relevance to the research question. Any paper marked by either researcher as meeting the inclusion criteria was retained for full-paper review. Both researchers reviewed the full text of the remaining papers independently. Papers included by both researchers were included in the review. Papers included by only one researcher were discussed until consensus was achieved, with a third researcher available if needed. Figure 4 details the review screening process.

FIGURE 4.

Systematic review 2 PRISMA flow chart of language and communication characteristics of communication aids. This figure has been reproduced from Judge et al. 72 with permission from Taylor & Francis Group (https://www.tandfonline.com/). The figure includes minor additions and formatting changes to the original figure.

This figure has been reproduced from Judge et al. 72 with permission from Taylor & Francis Group (https://www.tandfonline.com/). The figure includes minor additions and formatting changes to the original figure.

Quality appraisal

The Crowe Critical Appraisal Tool73 was used as the basis for quality appraisal as it supports the inclusion of a variety of study designs. It was chosen also because it allowed the outcome of the appraisal to be used as a criterion for acceptability. A score of < 40% on the tool was agreed to indicate that the paper was rated as weak. Quality appraisal was carried out by the two researchers independently. Papers rated as weak by both researchers were excluded from the review. 74–80

Data extraction

Two researchers worked jointly to extract relevant data from the studies. The following study characteristics were extracted: study design, participant sample size and characteristics, existing graphic symbol system(s) used by participants, language or communication attribute studied, intervention, measures and results.

Themes

Thematic analysis of the included papers resulted in three main themes of vocabulary organisation and design, symbol system and encoding, and vocabulary selection.

Vocabulary organisation and design

Three papers reported data from studies related to vocabulary organisation and design. These studies involved participants trialling communication aids with different combinations of static and dynamic organisational schemas. The primary aim of the first study81 was to investigate the nomothetic approach; however, the study had the secondary aim of examining the effect of display levels and vocabulary concreteness on the use of a communication aid. The study compared organisation schemas described as single level or dual level with a small number of symbols in a task where the participant was asked to match a symbol to a word spoken to them. The single-level display produced fewer errors, and concrete items were found to be easier to recall by participants than abstract ones. The second study82 was of similar design. In this study the static organisation promoted higher rates of vocabulary recognition during initial learning with the dynamic organisation scheme achieving higher rates after training (the seventh and eighth trials in the study). One study83 alternated organisational schemas between schemas they termed fixed, dynamic passive and dynamic active. The study involved a symbol-to-photograph matching task using a 30-symbol set with a single participant described as having ‘severe mental retardation’. For this participant there was no significant difference between dynamic active and fixed organisations tested in terms of speed or accuracy of symbol selection. The quality appraisal process identified two potential challenges to the validity of this result when considering it in the context of communicative use. First, it is not clear that the symbol-to-photograph matching task would transfer to unprompted use in communicative environments. Second, the method does not adequately explain the results for the ‘dynamic active’ condition. The method states that page changing occurred every time a symbol was pressed, with each screen displaying only half of the available symbols; this would suggest that for a randomly presented photograph the matched symbol would not be present on the communication screen for around half of all responses. The reported accuracy results are all > 60% (rising to > 90%) and so it appears that the experimenter chose the photograph to correspond to the current screen or that the method or condition was not fully described.

Symbol system and encoding

Three papers reported data from studies related to either the symbol system or the encoding methods used in symbol communication aids. One study84 had the aim of establishing which of two symbol systems was more easily acquired and maintained when an individual is trained in their use. Blissymbolics, a predominantly ideographic symbol system,93 was compared with line-drawn iconic pictures illustrated with the intention of showing a high degree of similarity to the object they represent. Twenty of each of the symbols were placed on a single-page communication board and provided to three males with cerebral palsy as part of a within-subjects study. Stimulus generalisation was evident in both symbol systems, but higher scores were reported with iconic pictures. Although students made spontaneous responses using both symbol types during daily activities, iconic pictures were used more frequently. A number of factors were identified in quality appraisal as limiting the interpretation of these results. Participants were all described as having ‘severe retardation’; however, the inclusion/exclusion criteria were not listed and it was reported that teachers felt that participants’ receptive language was above that reported in the test results. The test used to assess receptive and expressive language is not validated for this level of physical disability and it is not made clear if the assessment was carried out by the researchers or taken from records. The choice of items for the intervention was based on items that were readily visible in the environment, which limited the symbol vocabulary to nouns. Furthermore, in the spontaneous use task, both types of symbol were included on the communication board, which is unlikely to be representative of use in a naturalistic communication task. No description of the analysis or statistical methods is provided.

Two studies85,86 investigated the process of using short codes to create longer messages, termed ‘message encoding’. Both studies compared letter codes based on the first letters of salient words in the message (e.g. CE would expand to ‘can I have something to eat’), letter category codes based on the first letters of a category plus a specifier (e.g. RE would mean a ‘Requests to do with Eating’) and iconic codes derived from the icons and semantic associations proposed by Baker,94 that is, Minspeak^® (e.g. the icon of an apple followed by a question mark would stand for food and requests). The first study found that the salient letter technique was associated with higher recall than the letter code or iconic technique. In both studies, concrete messages were also found to have significantly higher recall than abstract ones. There was no interaction effect between these two factors. The accuracy of code recall increased for all learning and testing sessions in both studies. The quality appraisal identified that the participant cohort in the first study was biased towards functionally literate individuals and those with cerebral palsy. The authors noted this and attempted to address it in the second study; however, in this study, all of the participants included were above the age of 6–7 years in reading ability and all but one had cerebral palsy.

Vocabulary selection

Three papers investigated the process of selecting the symbol vocabulary to use on a communication aid. One study87 aimed to investigate the social validity of a specific vocabulary set by determining the importance of identified vocabulary items to 12 adults who used AAC. The results suggested that participants concurred with most (80%) of the vocabulary selected by a variety of knowledgeable informants. The authors identified that the study had a low response rate, and participants were recruited as a purposive sample, which may have provided skewed data. There were also no test–retest or internal consistency reliability measures of the data collection tool. Two studies looked at vocabulary selection. The first study89 involved nine participants who contributed their vocabulary lists, which were compared with each other and then against standard vocabulary lists. The second study89 presented a case description of the process of vocabulary selection and a comparison of the selected vocabulary against standard vocabulary lists. Inspection of participants’ vocabulary lists highlighted that these were small vocabularies compared with estimates of common English words or standard vocabulary lists. Comparing against the standard lists showed that the larger vocabulary lists contained a greater proportion of users’ vocabularies, but no standard vocabulary list contained all words included in even relatively small user vocabularies.

Search procedure

Owing to the dispersed nature of AAC research,50 a multifaceted search strategy was developed to navigate the literature. Five electronic databases were selected, namely EMBASE, ProQuest, EBSCOhost, Scopus and Web of Science. A hand-search of the Augmentative and Alternative Communication journal was completed and reference lists from a range of sources were also examined. Box 3 summarises the search strategy.

Inclusion criteria

Screening process

The search process yielded 29,591 records, which were imported into EndNote for screening. The lead researcher conducted the title and abstract review and full-text review. An independent researcher completed inter-rater reliability checks. This researcher independently applied the inclusion and exclusion criteria to a sample of search records. The independent researcher reviewed a sample of 403 records of included and excluded papers using agreed evaluation criteria. The agreement rate was 100%.

Quality appraisal

The inclusion/exclusion criteria were used to screen all returned records, resulting in 56 studies being included in the full-text review. Following the full-text review, six papers were identified for quality appraisal (Figure 5). Quality indicators were derived from two quality appraisal tools appropriate to study designs: the Critical Appraisal Skills Programme (CASP) Qualitative Checklist58 and the Evidence-Based Management Survey Checklist. 59 The six papers were appraised using relevant quality indicators and weight of evidence and relevance to the research question. 60 Two researchers independently completed this process before comparing the results. Five studies were agreed to meet quality thresholds. One study was deemed acceptable following consensus discussion.

FIGURE 5.

Systematic review 3 PRISMA flow chart of clinical decision-making by professionals.

Data extraction

Two researchers worked jointly to extract relevant data from the studies. The following study characteristics were extracted: sample size and characteristics, data collection and analysis, team composition and service delivery model, experience level of team members, decision-making processes, child and parent involvement in decision-making, language representation and organisation considerations.

Terminology

Across the included studies, different terminology was used to describe the professionals involved. For ease of reading, the term speech and language therapist (SLT) has been applied to both speech and language therapists and speech–language pathologists (SLPs). The term specialist speech and language therapist describes a SLT with reported expertise in communication aid recommendation. The term generalist SLT describes a SLT who is involved in aid recommendation but is reported to have broad experience across clinical areas. The term professional describes any person involved in the aid recommendation processes in a paid capacity, including health and education professionals.

Professionals

The managers of specialised communication aid services from across the UK were contacted in writing and invited to participate. Purposive sampling ensured that the specialised services from which the managers were recruited had different funding structures (e.g. government-funded public services, charitable organisations) and were from across the UK. The inclusion criteria were being a professional team member (e.g. SLT, occupational therapist) involved in making a communication aid recommendation for a child aged 0–18 years referred to a specialised service. All professional team members involved in the clinical decision-making for a particular child were invited to participate, so this included the local service staff who completed the specialist referral.

Children and families

Each participating team of professionals identified families who had been referred to the specialised service for an assessment. These families were contacted and provided with study information. They were asked to give written consent to their consultation being discussed, so, although not participants in the traditional sense, they were facilitators of the process. This was to ensure that the research agenda had no impact on their assessment. The researchers provided the specialised teams with a list of broad demographic child characteristics to encourage diversity among those invited to take part.

Competing considerations

This global theme summarised the perspectives of feature-matching that related to the child, to the communication aid and to effective access.

Cultural and contextual influencers

This global theme summarised the context within which decisions are enacted. This includes people, places, learning resources and situations.

Clinical implications

Taking cultural and contextual influencers collectively, the process of making a communication aid recommendation is shaped by many factors. The specific cultural and contextual influencers varied across children and services, suggesting that each decision involved balancing a range of influencers.

The perceived complexity of communication aid decision-making resulted in the development of an I-ASC explanatory model of AAC decision-making. Figure 6 provides a schematic representation of the process. ‘Competing considerations’ was identified as the first global theme as it denotes the interaction between child characteristics, access features and communication aid attributes. The ‘competing’ label describes the process of choosing the best fit based on a series of trade-offs rather than necessarily arriving at the perfect option. In the explanatory model, these competing considerations are illustrated by the cog wheels. The interaction of the elements of the child, access and communication aid is shown by the potential of the cog wheels to rotate in either direction as the team debates and considers issues within the decision and, ultimately, moves towards consensus. The cog–wheel interplay indicates an intensity in the trade-off between one decision made related to a characteristic, feature or attribute and how that instantly has an impact on the interlinked cogs that depict the remaining two organising themes. Once a decision is made about these factors, it can be influenced by issues external to the child or the aid. In the second global theme, ‘contextual and cultural influencers’, we found that environmental factors related to both the child’s culture and context and the professional’s work setting influenced decision-making. These cultural and contextual influencers, as illustrated by the funnel and its contents, denote factors that further influence clinical decision-making outside the feature matching process. The model denotes the two global themes and the arrows are indicative of the fluid and iterative process of decision-making.

FIGURE 6.

The I-ASC explanatory model of AAC decision-making.

Augmentative and alternative communication users

Augmentative and alternative communication users were recruited to discuss their experiences of communication aid assessments. Using purposive sampling, 15 AAC users were recruited. This process aimed to identify people with a range of profiles (Table 10).

Parents

Parents were invited to participate, and 16 agreed (Table 11). The participants comprised 14 mothers and two fathers.

Professional and support teams

Team members involved in any aspect of a communication aid assessment process were invited to participate. Forty-four participants from a range of backgrounds contributed (see Table 9).

Parents and professional support teams

Interviews were carried out in clusters, with the AAC user, parents and team members all interviewed on the same day (or on consecutive days for the AAC user). Interviews were conducted in a setting convenient to the participants, which included home, school and clinic.

Interviews were conducted in a semistructured way with a topic guide used to support data collection (for the interview topic guides, see the project web page: www.journalslibrary.nihr.ac.uk/programmes/hsdr/1470153/#/; accessed 20 April 2020). The topic guides were informed by the systematic literature review and the findings from WP 2 focus groups. Interviews lasted from 20 to 60 minutes and were audio-recorded and transcribed verbatim.

Augmentative and alternative communication users

The interview transcriptions of those AAC users who completed interviews in a traditional format (i.e. gave spoken responses) were imported into NVivo. For AAC users who completed interviews using supported formats, the Talking Mats photographs, researcher interviewer field notes and video transcription notes were imported into NVivo. These data were held in a separate NVivo record from the parent and team member data for analysis purposes; however, the same framework analysis approach was used. 113 Using this approach, two researchers read and reread the interview records and viewed the images to obtain an overall sense of the data and to develop their initial impressions. An initial coding process identified how meaningful data segments related to framework codes. The two researchers discussed initial coding, recognising commonalities and discrepancies, and arrived at consensus that informed coding development. Other research team members undertook a data review process. The purpose of this review was to support robustness of the qualitative process. These researchers reviewed portions of coding against the operational coding definitions and asked questions about the overall procedure, meanings and interpretations. Coding consensus was agreed. The NVivo software provided an audit trail and supported credibility.

Parent and professional teams

Interview transcripts were imported into NVivo for data management purposes. Following the inductive development of a thematic network in WP 2, a framework analysis approach was used in this WP to support the coherent development of themes. 113 Two researchers who were specialist AAC SLTs read and reread the transcripts to obtain an overall sense of the data and to develop their initial impressions. An initial coding process explored how meaningful data segments related to framework codes. The two researchers discussed initial coding, recognising commonalities and discrepancies that informed the coding development. Further independent coding reviews were conducted by two external researchers to ensure credibility and transferability. 52 In addition, a peer review process was undertaken by other research team members, as described in the preceding section, Augmentative and alternative communication users.

Overview of augmentative and alternative communication users’ perspectives

I love my communication aid, because I can pretty much say anything.

The majority of AAC users were able to express opinions about their communication aids and their communication aid assessments. Some could provide views on specific aspects of the communication aid (e.g. the colour, battery life reliability or aid size) (Figures 11 and 12).

FIGURE 11.

A visual record of a young person’s view on their communication aid. This young person indicated that they liked the size, colour, voice and look of their communication aid. They also like the (speaking) volume. They did not like the battery life, how difficult it was to program and the time it took them to communicate. They did not have an opinion on what the communication aid was like for playing games.

FIGURE 12.

A visual record of a child’s view of their communication aid. In this example from a very young child, they demonstrate that they can express that they liked the size, colour and voice of the communication aid. They liked that it worked quickly and was easy to use. They did not like how to put in new messages and that it was hard to take around places.

They could also express their preferences about how they liked to use their communication aids [e.g. talking to grandparents on Skype™ (Microsoft Corporation, Redmond, WA, USA), talking to friends, talking about places they like to use their communication aids, being able to go to the shops independently]. Young children (as young as 4 years) and those with multiple challenges were able to express their views with support (see Figures 11 and 12).

Augmentative and alternative communication users have the ability to take an active role in decision-making related to communication aid assessment and recommendation. Building opportunities for AAC users to express their views during assessment processes may support the selection of communication aids that are more closely aligned with their views and preferences.

Communication aid attributes: hardware aesthetics

Hardware aesthetics were expressed as preferences for smaller and lighter communication aids: ‘As small, I can use it on my knees’. For one participant, size was the most important decision-making factor:

My previous communication aid was really heavy and difficult to carry around and I was just about to go to secondary school so we were looking for a communication device that was lighter, smaller and easier to carry around.

Another participant indicated they would prefer a larger screen than they currently had, highlighting the individual nature of each decision-making process and the need to seek individual views.

Some participants highlighted challenges of not having access to their communication aid during the whole day, for example if they were not able to use it in a swimming pool or when travelling in a car, limiting their communication options (Box 4).

In addition to weight and size, some participants expressed preferences for more modern and mainstream-looking devices:

My parents have suggested going back to a typewriter because people are able to see what you are typing on both sides of the device but I prefer a tablet-styled device because it is modern and blends better with other devices.

This supports views expressed in other aspects of the study that devices having a mainstream look was important, particularly for adolescents. One participant highlighted the importance of a good voice that can be understood as a priority communication aid attribute: ‘I think good voice and you understand it’.

Communication aid attributes: hardware reliability

A key consideration was the operational reliability of the aid and how long the battery lasted. Both were priorities for individuals who had experienced insufficiently reliable systems: ‘It is bad because every week they break’.

Given the impact that these attributes have on the AAC users in this study and the relatively low importance ratings these received in the BWS (see Chapter 7), these findings reinforce the notion that obtaining users’ views and preferences and incorporating them into decision-making needs to be considered in assessment processes.

Ways of working: inheriting decisions

A factor some AAC users raised was the impact of decisions made in previous communication aid recommendations and the implications that such decisions had for the future. Graphic symbol sets and systems are used in AAC to represent language. Learning to use these symbols to communicate takes considerable time, learning effort and support, similar to learning how to read. The graphic symbol sets used are proprietary and, as a result, some graphic symbols and language organisation packages are available only on specific communication aids. As one participant noted, once the time and learning has been invested in one language representation system, there are significant opportunity costs in switching to another form of language representation:

I wanted same language because if I move away from my language system, I guess I will be not happy because I don’t know nothing.

This limitation of being able to use a language representation system on only one retailer’s communication aids essentially means that individuals are precluded from choosing another type of aid. This restriction implies that, when a communication aid needs to be replaced, individuals do not receive a full assessment but move to the newest aid that can run a similar language representation system. New developments in other aids that may enhance communication are not easily available to them. This highlights the long-term ramifications of the choices about communication aids made when a child is very young. It is important to recognise that these early decisions may be taken forward into multiple future communication aid options. The clinical implication of this finding suggests a need for extra care to be taken over early communication aid recommendations with a mind to the long-term impact of the choice made; alternatively, communication aid manufacturers need to recognise how to support all graphic symbol communication platforms.

Ways of working: service delivery model

Data on service delivery models mirror findings from other aspects of the I-ASC study. AAC users reported varied levels of support and service delivery models, which had an impact on their experience of communication aid assessment and the subsequent intervention. Some participants were able to access good services in a timely way – ‘They [the AAC service] were fantastic, very quick’ – whereas others waited long periods for services or reported the need to fight for the services they needed: ‘I had a very long fight with people about paying [for the communication aid]’. Service availability appeared to be affected by geographical location (across the UK) and by the age of person. Young adults experienced being discharged from services and struggling to access support to enable them to continue to communicate independently. Further research and service appraisal could determine equity of access to services, especially at the time when young people who use AAC transition into adulthood.

Transitions: technology change

In contrast to inheriting decisions that may impede choice, technology change was identified as an important and positive factor for some participants; technology change opens up new ways of communicating, interacting with other technology such as computers, and enhances access speed and efficiency: ‘I started using my eye equipment last August which is fantastic and easier and quicker’. Some of the young adults reflected on the changes in technology during their lifetimes and welcomed what systems can do now: ‘When I was at school, no equipment like this was around but then they started coming’.

Available resources: intervention and team knowledge and skill

The AAC users valued having access to the intervention support and resources needed to fulfil communication aid recommendations. For example, one participant described being fortunate to have access to these resources in a timely manner: ‘I was very lucky, because 2 years I had speech therapy every day for an hour or two’. This reinforces the belief that detailing appropriate support for an aided communication intervention at the time of recommendation may enable appropriate provision.

Communication opportunities and progress

Positive aspirations for the young person were a main factor when considering how to build communication opportunities:

He is going to go on and have a career so we are thinking long term. He is going to have to build friendships, relationships, so we need something that’s going to help him to achieve his full potential really, making sure that communication isn’t the barrier to any of these things happening.

SLT

When appropriately recommended and implemented, communication aids were recognised as supporting AAC users to communicate independently and reach their potential:

The high-tech device really has opened up her world to being able to communicate what she wants to say independently.

SLT

Participants described how the use of communication aids had the power to shift perceptions of what was possible. These ‘little eureka moments’ showcased the young person’s potential and raised expectations about their future:

She told her she had a headache and she wanted a tablet, she had a pain which was a very big first for mum. She had never told her mum that she had a pain in her head before. So there were those little eureka moments that we talk about that were really lovely.

Assistive technology professional

The young person communicating something they had never expressed before or independently communicating for the first time were important watersheds in people’s perception of their young person’s communication development:

In a café, he asked for a hot chocolate and a cookie, which was pretty mind-blowing really for him. The bit that got [child name] mother what she was so excited about was, it was that spur of the moment. They just went off and did it. It was a case of, we are going to the pub as a family, and what do you fancy for dinner? There’s the menu, and they read him out the things and then he found it on the [device name] and ordered what he wanted, and it was lovely, it was really lovely.

Teaching assistant

By demonstrating what they could do with their communication aids, AAC users changed others’ perceptions of them. This highlights the need for having the right communication supports in place so that children and young people can show their potential:

She threw a complete 2-year-old tantrum and was taken out into the kitchen. She told her dad she didn’t want to watch that on the telly through the [device name]. But she still continued having this big, big strop. So, we gave her 20 minutes to calm down and then we said, ‘Hello [child name] how do you feel now?’ ‘I feel sorry.’ Up until then, we didn’t know that she had any empathy, that she felt those feelings. Nobody knew whether she was just playing up because she was playing up or whether it was a disability-linked issue.

Parent

The communication opportunities experienced by AAC users, coupled with their ability to demonstrate communication skills, influenced the perceptions and expectations of those around them. Such abilities also affected their access to services. For example, in some areas, being assessed for a specialised communication aid was predicated on being able to demonstrate existing abilities with AAC that met the referral criteria. One SLT observed that being able to demonstrate these abilities often required particular communication opportunities:

It is often about what low-tech AAC supports they’ve had in place and how the children are [used] to communicating in any way but particularly [by] combining symbols (to suggest linguistic competence).

SLT

This particular commentary highlights that children with limited or no access to appropriate AAC support at a local level may not be able to demonstrate the level of ability required to access specialised services and therefore may miss support at both levels. Research needs to address how children access appropriate support at a local level to ensure that they experience the best possible communication opportunities to develop skills that may later enable them to meet specialised referral criteria. For example, young children may need highly motivating and lower-effort opportunities to build their experience of being successful and active communicators:

We just gave him access to communication from the start. The very first thing we did is we made a human jack-in-the-box, we put his teaching assistant in the box and every time [child name] hit a switch for a different message, he actually jumped out of the box.

Teaching assistant

There was a recognition that these opportunities did not just happen but required skilled individuals in the child’s environment to structure opportunities to afford the child control and the experience of active communication; in typically ‘speaking’ children, this is taken for granted:

Everybody’s communication opportunities have to grow. What concerns me the most is, yes, we have given him this aid, but for it to grow with him, that’s reliant on us understanding how it grows. We have been fortunate here because we’ve got a specialist teaching assistant, who’s had lots of training and is very motivated and enthusiastic, so he can oversee those elements.

Teaching assistant

This example highlights how critical the right support is for an AAC user to benefit from the communication aid recommended. Experienced and knowledgeable staff members are needed to help children learn the power of communication through technology.

It was notable in the I-ASC data that those who progressed quickly and demonstrated greater proficiency were offered more communication opportunities than those who made slower progress. Where rapid progress was observed, there was a greater focus on making a communication aid available for more of the day, for example by providing appropriate mounting systems:

Mounting it to her wheelchair would probably be our long-term plan, but at the moment we just want her to be a bit more proficient and use it in the classroom.

Assistive technology professional

Some AAC users had their opportunities with their communication aids limited for different reasons. For example, sometimes communication aids could be used only in school or for certain activities:

I’m not sure if it always goes out onto the playground with him but I’ve had conversations with his teaching assistant before that it really should go out with him because it’s one of his social times of the day.

Teaching staff member

The demands of keeping up with vocabulary needs and customisation was another reason why opportunities to use a communication aid could be limited:

He brings it home on weekends because the school keep it in the week to add things on it. So he brings it home at weekends but he hasn’t been using it until lately at home.

Parent

Software graphic symbol representation and organisation

Graphic symbol systems are used to represent language. There are many types of graphic symbol that can be chosen for children at different stages of language skill. Different symbol sets are available on different communication aids and symbols can be organised in many different ways. Many participants indicated that, despite being core to how children learn how to communicate through aided means, consideration of the type of graphic representation or how it was organised did not feature strongly in the decision-making debate:

It is not particularly the software because we customise most of it anyway. It’s really about the people around the child that are the key. To be able to programme it yourself, to be able to use it, to encourage [child name] by modelling, I think all of those skills are actually more important than the software itself.

Teacher

Data suggest that the choice of graphic representation may be influenced by the acceptability of different systems to those in the environment, rather than by the child’s learning needs and learning style preferences:

Don’t get me wrong, there is a place for [named software 1]. But we have quite a few new members of staff and I think sometimes they got a bit flustered by it. Whereas with [named software 2], the response we got from the staff and their ability to support him was quite different. It wasn’t as daunting for them as well and the same with his parents.

SLT

Participants reported that the type of graphic representation used by children often changed. Changes occurred as children changed communication aids or contexts, for example when they started school. Participants reported that they often aimed to retain previous learning in terms of vocabulary layout, but it was not seen as important to retain the same graphic representation system, particularly with children who had a good understanding of their current symbol set:

He had gone down the route of [symbol set 1]. I don’t have too much of a problem with changing to something like [symbol set 2] because for some children that isn’t really an issue but his mum was very, very keen that he stick with the [symbol set 1].

SLT

It is unclear from this finding if decisions related to the choice of graphic representation and organisation are driven by the preferences of those supporting the child or by the resources in the child’s environment, rather than by the system preferences of the child.

The need for customised child-specific vocabulary layouts was considered for individuals starting out with AAC to build early motivation to communicate:

We fostered the choice-making, but did it in a way that he would find fun and interesting. And those boards just grew arms and legs. There was more and more and more of them. So she put all that into a book but it got so big with so much vocabulary that it became unmanageable and it became far too big.

Teaching staff member

Choosing how the vocabulary was organised was made in one of two ways. The first way was the use of ‘try it and see’. Participants reported giving a child a few different layouts to try and they then gauged the child’s response to these layouts, choosing the one that they felt worked best. The second way was by selecting the vocabulary layout that was already being used in the child’s school or other setting. This lends further support to the notion that contextual factors often have a greater prominence in decision-making than child characteristics do. This implies a need for more structured assessment of children’s symbolic understanding and potential for communicating with graphic symbols, so that this information can inform the choice of graphic symbols and vocabulary organisation.

Roles and responsibilities

There was considerable comment about who took responsibility for different aspects of the decision-making and implementing processes. Across the interview clusters, there was variation in team size, team perspectives and the way decisions were made. There were examples of good practice and outcomes. Good practice appeared to be related to either having a comprehensive team with the required expertise involved in the appraisal or having a few people assuming multiple roles to ensure that all aspects of the process were completed:

Because we had that team approach, I think we did cover everything. It would have been different if we hadn’t had the expertise of everybody involved. I think having that team approach really worked. If we had taken any particular member of the team away, it would have fallen down then. I am hopeful that he feels the same way but I think we covered it quite comprehensively.

SLT

There were less positive examples of information gaps or an absence of collaborative working. For example, one professional discussed how an aid had been provided to a child and their family without any instructions or training:

We feel responsible for the communication aid, but because it is another organisation’s kit. You know, you presume. You can presume that it has come with a set of instructions. Actually, they have just provided it [the communication aid].

SLT

Although there were several examples of different service structures and processes, there was a sense that, as long as roles were clearly defined and mutually understood, team responsibilities could work well for the AAC user. One example of good practice was having someone in the role of champion or driver of the process:

The class teacher was highly involved and was always really supportive, and there was a very supportive learning support assistant as well. They always attended all the assessments and when he moved class, both teachers attended one of the assessments so the school were very supportive and I personally think that was part of the success.

SLT

Sometimes a professional took on this role but often a parent fulfilled this role and created the drive for success:

His mum’s involvement and her being so proactive in taking the lead in putting all that stuff in there, for me was the crux of it being so successful and in such a short period of time. It could be quite laboured and it could take a very long time if you didn’t get the support of parents.

Teaching staff member

This view was echoed by a parent who had taken on this role for her child:

It’s all about advocating and you’ve got to play a significant role in pushing things forward, and if you’re not able to do that then you’re stuffed basically because the system cannot cope with a parent who isn’t heavily involved in all the processes.

Parent

Another suggestion of good practice was having a case holder for each child who uses a communication aid. A case holder could be described as an individual who takes responsibility, follows up, checks in and makes sure that the aid is still meeting the child’s needs and that any roadblocks are addressed:

I didn’t get to go in myself to do the pick-up and the feedback to classroom staff. But then I probably wouldn’t have felt so anxious about that if there’d been a case-holding SLT involved.

SLT

Participant recruitment

The target population was any UK-based professional involved in AAC decision-making and who worked, either wholly or partly, with children. To make statistical modelling as robust as possible, the aim was to obtain as large a sample as feasible and to reach a geographically widespread UK audience. Recruitment e-mails were sent to a mailing list developed by the research team of attendees at previous project events. The e-mails were also sent to the mailing list of Communication Matters, a UK-wide AAC charity (www.communicationmatters.org), and to the administrators of various service providers with a request to forward it to their members. In addition, personalised invitations were sent to the research team’s professional contacts with a request to circulate these. Responses were collected between 24 March 2017 and 15 May 2017.

Procedures

Analysis

Analysis was grounded in random utility theory. 121 This means that individuals are assumed to choose whichever option gives the greatest utility. The utility of an option is modelled as having a deterministic component, determined by the option’s attributes, and a random component, which represents aspects of the decision-making environment not observable by the researcher. All analysis was carried out using R version 3.3.1 (The R Foundation for Statistical Computing, Vienna, Austria).

The utility individual i receives from choosing option k ∈ {1,2, . . ., 6} in choice situation j is modelled as:

where x_jkl is a dummy variable indicating whether or not option k includes attribute/characteristic l, β_il is a parameter representing an individual i’s preference for attribute/characteristic l, ε_ijk is an independently and identically distributed extreme value error term, and N_A is the total number of attributes/characteristics.

Estimates of the β parameters were obtained from random parameters logit (also commonly known as mixed logit) models. For a given attribute, l, individuals’ parameters were assumed to be normally distributed with mean β¯il and variance σl2. The distribution mean was allowed to depend on respondent characteristics according to:

where β_l0 is a constant, the Z_im are M variables representing characteristics of individual i, the δ_lm are parameters giving the dependence of preference on characteristics, and η_il is a normally distributed error term.

Results are presented using relative importance scores (RISs), which give the importance of characteristics/attributes on a ratio scale. Thus, a characteristic/attribute with a RIS of 10 is twice as important as one with a RIS of 5, and a characteristic/attribute with a RIS of 2 is only half as important as one with a RIS of 4. The RIS of all characteristics/attributes is transformed to sum to 100, implying that a characteristic/attribute with a RIS of 100/19 ≈ 5.26 for children and 100/18 ≈ 5.55 for AAC devices was of average importance. The RIS of attribute/characteristic l for individual i was calculated using:

where β^il is the estimated individual-level coefficient on attribute/characteristic l for respondent i. 122 The mean RIS for each attribute was then calculated across participants.

We performed t-tests of the null hypotheses that each attribute/characteristic was of average relative importance, and also for each pair of attributes/characteristics of the null that they were of equal relative importance. Whether or not the RIS differed according to each of the demographic variables in Appendix 3 was examined using t-tests. Based on test results, characteristics and attributes were divided into three groups: (1) those with a RIS significantly higher than average, (2) those with a RIS not significantly different from average, and (3) those with a RIS significantly lower than average. We tested whether or not all characteristics and attributes differed in importance, and each RIS was tested to determine if there were differences according to respondent demographics (Table 12).

Response quality was assessed as follows. Kolmogorov–Smirnov tests were used to check whether or not individuals were biased towards selecting an attribute/characteristic in a given position (e.g. at the top of the list). We calculated the proportion of times participants made either contradictory choices (e.g. stating that attribute A is more important than attribute B in one question, and then stating the opposite in another question) or choices that violated transitivity (the principle that if attribute A is more important than attribute B, and if attribute B is more important than attribute C, then attribute A is more important than attribute C). As individuals who make logical choices should be consistent and transitive, and should make an even spread of choices in each position, poor performance may indicate a lack of understanding or inattentiveness.

No responses were excluded from the main analysis based on these measures. Respondents were split according to whether their choices displayed above or below the median proportion of consistency and transitivity and we examined whether or not the relative importance score differed significantly between the two groups. Models were re-estimated including only participants making consistent and transitive choices at least 80% of the time (this removed participants with the greatest number of inconsistent and intransitive choices while retaining sufficient data to estimate models).

Statistical significance was judged at the 5% level, with adjustment for multiple testing using Holm’s sequential Bonferroni correction. 123

Survey development

Attributes for the DCE were selected from the attributes used in study 1 (BWS) during consensus discussions between the research team with lived AAC experience and expertise in AAC service delivery and research and health economics. The criteria for selecting the attribute sets were that they should (1) form coherent and realistic descriptions of children and systems, (2) address the research aims of the wider research project, (3) include mainly attributes with high relative importance scores in the BWS study and (4) contain a small enough number of attributes that the choice tasks would not overburden respondents. This resulted in four child attributes and five system attributes.

A set of levels was generated for each attribute during further consensus discussions, with the criteria that levels should (1) reflect the characteristics of children and systems usually encountered by AAC professionals, (2) cover a wide range of characteristics of a given attribute and (3) be few in number to aid model estimation. The language used for attributes was refined from that used in the BWS to reflect a different type of choice task.

The final list of four child attributes with associated levels and descriptions is given in Table 14 and the list of AAC system attributes with associated levels and descriptions is given in Table 15.

A decision situation in this DCE had two components: a child vignette and a set of systems to allocate to that child. For the first component, there were 54 possible vignettes. Research colleagues with expertise in AAC and speech and language therapy identified and removed 18 vignettes that represented unrealistic combinations, leaving a total of 36. Each participant was asked to answer questions about three vignettes randomly selected from these 36.

Experience with the BWS study led to the belief that it would be difficult to recruit a large sample to this study. To maximise the number of data, we chose a relatively heavy response burden of 12 choices among three systems. From the 432 possible AAC systems, researchers removed 158 unrealistic combinations. From the remaining 274 combinations, a D-efficient survey design was generated using NGene (© ChoiceMetrics; www.choice-metrics.com) with five blocks, meaning that there were 60 choice tasks in total. Participants were randomly allocated to a block independently of the child vignettes.

At the start of the survey, participants were asked whether or not they contributed to decision-making about AAC systems. Those who answered ‘no’ were not shown the DCE tasks and answered only demographic questions. For the first four DCE questions, participants were asked to imagine choosing a system for the first randomly selected child vignette, for the second four questions they chose for the second child vignette, and for the final four questions they chose for the third vignette. The children in the first, second and third vignettes were referred to as child A, child B and child C, respectively. The order of system attributes was randomised between participants but consistent within the choices seen by a given participant. Finally, participants answered some questions about themselves and their experiences with AAC (Table 16).

The DCE was administered online for ease of recruitment. It was tested by five AAC professionals, after which the wording of some attributes and levels was altered.

Recruitment was completed using e-mail lists gathered at previous project events, as well as the mailing list of Communication Matters and authors’ professional contacts. Owing to the small pool of potential respondents, we had to allow those who may have completed the BWS survey to complete the DCE. Responses were collected between 20 October 2017 and 4 March 2018.

Analysis

Responses were again analysed using a random utility framework. The utility to participant ε of allocating AAC system s to child c is modelled as:

where α_s is an alternative specific constant, x_s is a vector of dummy variables indicating the level of each attribute for system s, β_ic is a vector of coefficients describing i’s assessment of the appropriateness of each level of each system attribute for child c and ε_i is an independently and identically distributed extreme value error term.

The coefficient on level l of system attribute a, β_ialc, depends on the characteristics of the child according to:

where γ_ial0 is a constant, z_c is a vector of dummy variables indicating the level of each attribute for child c, and γ_ial is a vector of coefficients indicating how i’s valuation of level l of system attribute α depends on child attributes.

For each child and system attribute, one level was selected as a baseline (indicated by asterisks in Tables 14 and 15).

A full model with all interaction terms and two alternative specific constants implies 98 parameters, too many to reliably estimate given the number of data collected. A strategy was required to allow the identification of a suitable model with fewer parameters. The first stage was a series of stepwise multinomial logit regressions, beginning with a model with all 98 parameters. The parameter with the lowest p-value, excluding the γ₀ constant terms, was eliminated, and a model with 97 parameters was estimated. Then the parameter with the lowest p-value was excluded and a new model was run. This continued an iterative process until only the 12 γ₀ constant terms remained (one for each non-baseline system level).

The Bayesian information criterion was used to select the preferred multinomial logit model. This model was then re-estimated as a mixed logit model to accommodate heterogeneity of participants’ preferences. The β-coefficients on system attribute levels were assumed to be drawn from a normal distribution with both mean and variance depending on child attributes. If p is the number of parameters of the preferred multinomial logit model, then models with between p – 2 and p + 2 parameters were re-estimated as mixed logit models. The mixed logit model minimising the Bayesian information criterion was chosen as the final preferred model.

Multinomial logit models were estimated using maximum likelihood and mixed logit models were estimated using simulated maximum likelihood, both implemented using the Choice Modelling Centre’s code for R version 1.1 (University of Leeds, Leeds, UK). Analysis was carried out using R version 3.3.1.

The statistical significance of parameters was assessed at the 5% level after adjusting for multiple testing using Holm’s sequential Bonferroni correction. 123

It is possible to gain an insight into the size of impact child attributes have on AAC professionals’ decision-making. A measure is introduced termed relative interaction attribute importance, similar to relative attribute importance, which is often used to present DCE results. Relative interaction attribute importance measures the amount that preferences for attributes of choice objects are impacted by a given interaction attribute associated with a choice situation relative to other interaction attributes. In this study it measures how much preferences for AAC systems are influenced by child characteristics.

Relative interaction attribute importance is calculated with respect to a single choice object attribute by taking the difference between the greatest increase an interaction attribute causes to a choice object attribute’s part worth utility and the greatest decrease, expressed as a percentage of the differences for all interaction attributes. Formally, the relative interaction attribute importance for interaction attribute i with respect to choice attribute c is:

where γicmax and γicmin are, respectively, the maximum and minimum coefficients for interaction attribute i with respect to choice attribute c, and N_I is the number of interaction attributes. The overall relative interaction attribute importance for i is similarly calculated as:

where now γimax and γimin are, respectively, the maximum and the minimum coefficients for interaction attribute i across all choice attributes.

The research context

The I-ASC research focused on the interplay between the symbol communication aid, the young person benefiting from it, and the contexts of use and interaction.

In the documentary space available, key points are developed in this chapter, with cross-references given to our online resource. The online resource provides a comprehensive synthesis of findings that have been translated into guidance to inform research and clinical decision-making (https://iasc.mmu.ac.uk/).

Child and young person

Consideration of the existing evidence in terms of child or young person characteristics reveals a confusing literature that appears focused on the experiences the child or young person needs to access support for language development, rather than describing the inherent characteristics and abilities that might inform decision-making (e.g. current language profile, sensory impairments).

In a small number of studies, child characteristics, such as their medical diagnosis, their ability to learn more abstract symbols and their receptive language abilities, were reported as factors in the decision-making process. 96 An ability to demonstrate skills was also reported as a factor, resonating with other aspects of I-ASC research (e.g. referral criteria to specialised services requiring the demonstration of AAC-related communication skill). Existing research suggests that, if such skill was evident, a more abstract representation system would be chosen, even if this was not the easiest system for the child to use. 14 This suggests that SLTs focus on maximising linguistic and communicative potential and consider developmental trajectories in their decision-making. It also suggests evidence of a gap in provision of support for young people who do not yet meet specialised referral criteria.

Children and young people having access to high-quality communication opportunities with more conversational partners was viewed as likely to promote their language development, but only if they were afforded access to sufficient vocabulary on their communication aid. Although not explicitly stated, the characteristics of resilience and motivation were implied in the recognition that the child/young person persevered in trying to convey a message with often limited vocabulary options.

Communication aid

The literature review demonstrated that little research exists to help us understand what evidence professionals use to appraise the attributes of specific symbol communication systems. This was reinforced by the considerable inconsistencies in the terminology used across the research cited; where attributes have been studied, they have been inconsistently defined.

Research objectives

• To qualify and quantify processes that support public involvement across all aspects of co-production in the research process.

• To quantify the protocols that facilitate marginalised and vulnerable public involvement groups to make meaningful contributions to the research process.

• To appraise the costs and benefits of extensive public involvement in research.

• To develop guidance and practical tools to facilitate the co-production of research with public involvement co-researchers from diverse, hard-to-reach cohorts.

• To disseminate this guidance and the results of the project to improve the quantity and quality of public involvement meaningfully in the co-production of research.

Research questions

1. How and what can we learn from an evaluation of public involvement in a nationally funded project focusing on vulnerable and hard-to-reach people?

2. How can public involvement research, implementing current guidance with vulnerable and hard-to-reach groups, be structured to avoid pitfalls and improve impact?

Participant recruitment

Participants in all groups had been identified already as they had all been involved previously in the I-ASC project, as researchers/co-researchers/support staff/Advisory Group and critical friend. The chief investigator provided the contact details of these individuals, with their consent, to the WP lead. An electronic participant information sheet and the topic guide were sent to all potential participants. Those who wanted to take part were invited to attend an interview and provided written informed consent on a consent form at the beginning of the interview.

Data collection

Across the participant groups, three possible data collection activities were completed: (1) individual interviews, (2) one focus group and (3) a resource utilisation questionnaire. Individual semistructured interviews were conducted face to face in accessible, confidential rooms on university premises or over Skype. Interviews lasted between 25 and 100 minutes depending on each participant’s responses. The focus group for group 1 took place on university premises and lasted approximately 90 minutes. For group 1, individual interviews took place before the focus group to ensure that group thinking did not influence individual viewpoints. Two participants who used AAC were invited to prepare their responses in advance and pre-programme their AAC devices if they wanted to do this. One participant prepared written responses to questions and e-mailed these to the WP lead before the interview. This participant’s responses were read aloud to them during the interview and they were asked if they wanted to revise these or add any information. This participant received communication support during their individual interview from their support assistant, who was also a participant. The second participant who used AAC and their support assistant, also a participant, requested to be interviewed together, so that personal and communication support could be provided where needed. This AAC user did not pre-programme their responses using their AAC device. All interviews were recorded digitally and field notes were completed. After they had completed their interviews, participants in groups 1, 2 and 3 were asked to complete and return the resource utilisation questionnaire.

Analysis

A mixed-methods approach was adopted. Interview and focus group data were analysed thematically using the framework analysis approach. 53 This approach was selected because it enables organising themes and basic themes to be generated both deductively from the research objectives and the findings of the literature review (see above) and inductively from open data coding. The digital recordings from the interviews and focus group were transcribed verbatim into a Microsoft Word (Microsoft Corporation, Redmond, WA, USA) file. Each Microsoft Word file was imported into NVivo to facilitate data analysis.

Analysis followed the five-stage framework analysis process53 of familiarisation with the data through to mapping and interpretation of the data. This enabled the overall thematic framework to be modified (see Appendix 7).

The resource utilisation questionnaires were analysed thematically using the framework analysis approach. 53 The questionnaire responses were collected, scanned and stored electronically in accordance with agreed project protocols for data management. As above, each Microsoft Word file was imported into NVivo to facilitate data analysis, and a similar five-stage framework analysis approach was completed. A financial analysis of proposed and actual budgetary spend was completed so that quantitative data relating to resource use could be considered.

Rigour

Different approaches were used to ensure methodological rigour. 152 An experienced qualitative researcher reviewed the initial thematic framework and was able to suggest slight changes to it. Another researcher used the thematic framework to code a sample of four transcriptions (one per participant group). Coding consensus was achieved through discussion and slight amendments were made to the thematic framework in response to differences in coding identified.

Findings from individual and focus group interviews

Five themes were generated from the deductive and inductive coding of individual interview and focus group data: (1) the nature of public involvement in the I-ASC project; (2) the resources used to enable public involvement; (3) the benefits provided by public involvement; (4) the challenges associated with public involvement; and (5) the facilitators of successful public involvement. Organising themes and basic themes were also identified in the data; these are discussed below.

The nature of public involvement in the I-ASC project

This theme was generated from data relating to how public involvement was planned and implemented in the I-ASC project (Table 20).

Participants’ responses indicated that purposeful public involvement was integral to and achievable during all stages of the I-ASC project:

[The chief investigator] had a very clear view as to what she wanted and that she wanted PI [public involvement] involvement right from the start to the finish of the project.

Co-researcher, group 1

So I’ve been working on the I-ASC project as a co-researcher and I’ve been working on interviewing people, doing presentations regarding the project, and I have done some work on the website and social media.

Co-researcher, group 1

Having them there, very evident and very involved, not just people who we were reporting to. I think that worked well . . . It made us reflect on what . . . we were saying and what we were planning all the way through the project . . .

Researcher, group 1

Participants described how the roles of the co-researchers evolved from early conceptualisations in response to the needs of the project with the skills and experience that the co-researchers were able to provide. Notably, one co-researcher brought many professional skills that were utilised in the project and contributed more time to the project than had been initially anticipated. Researchers in the team emphasised that this set of circumstances should be neither perceived as commonplace in public involvement research nor expected in future projects:

[The co-researcher] has been involved over and above what might be well what we previously expected or what might be reasonable for any other projects . . .

Researcher, group 1

Participants perceived different aspects of the co-researcher role: (1) to provide the perspective of a representative of a particular population (in this case, a user of AAC or personal assistant/family member); (2) to challenge established and accepted norms in the research process (e.g. the way in which research outcomes were described and reported); and (3) to contribute actively to research activities (e.g. data collection, analysis and dissemination). Researchers valued all contributions and were able to provide examples of the benefits associated with each aspect to the conduct and outcomes of the project.

This theme provided information about how researchers and co-researchers worked in partnership to deliver the project. Participants perceived the co-researchers to be well integrated in the research team and actively involved in the project; they rejected the notion that public involvement in the project could be viewed as tokenistic:

. . . you know the last thing this research has been . . . is tokenistic. It has taken full account of the views of [the co-researchers] in making certain that people who use AAC and families are accounted for.

Co-researcher, group 1

I think the project has gained from its genuine, rather than tokenistic inclusion . . .

Researcher, group 2

. . . I do believe that they have been actively involved in it, and not just token box-ticking, which is what I frequently see when you include any stakeholder . . .

Critical friend, group 4

Participants suggested that there were differences in individual levels of integration and involvement between the co-researchers and proposed a number of potential reasons for this:

I think I would say that [one co-researcher] was a very integrated member of the research team. [They are] familiar with being involved on research projects and [are] an assertive communicator in a very positive way . . . I think for [a different co-researcher], the experience of being involved in a research project was newer and more challenging . . . And there are sort of practical logistical reasons which probably made some of that challenging.

Researcher, group 2

Participants’ responses enable us to identify potential barriers to co-researchers’ integration within research teams and to their involvement in research activity: (1) lack of experience of public involvement (on the part of researchers and co-researchers), (2) perceived power imbalances resulting from different professional status and experience, (3) structural barriers that exclude people with disabilities from inhabiting traditional research roles and (4) differences in researchers’ and co-researchers’ communicative abilities, skills and methods.

Resources used to enable public involvement during the I-ASC project

This theme was generated from data relating to the resources used to plan and implement public involvement in the I-ASC project (Table 21).

Participants indicated that resources were utilised to ensure that meaningful public involvement was integrated into the project. They suggested that research projects that involve public involvement take more time and cost more money than traditional research projects:

I’d be really keen to be involved again in this kind of project but recognise that it needs a lot of time and money to support that to happen properly.

Researcher, group 1

Various members of employed staff inside and outside the core research team needed to devote time to facilitate the involvement of the co-researchers. Initially, the chief investigator and the co-researchers used unfunded resource in terms of time to incorporate public involvement within the original project funding application. Participants reported that an important amount of researcher time was required to overcome structural barriers to the co-researchers gaining access to their role on the project. This included time spent ensuring that the building was physically accessible to a co-researcher who uses a wheelchair (in addition to the purchase of specialist equipment) and time spent attempting to obtain employment contracts and research passports for both co-researchers.

Considerable resources were used to provide training and support for researchers and co-researchers to work together effectively. This included supporting co-researchers to be able to understand and participate in the research environment (e.g. research methods training); supporting researchers to engage in teamwork successfully with co-researchers; and specific training for personal assistants and researchers to deliver effective, personalised care and communication support for one of the co-researchers. This last type of support can be resource intensive, resulting in significant cost.

Participants identified several ‘hidden costs’ of public involvement that were not anticipated when the original project funding application was submitted. At the project planning stage, public involvement contributions were unfunded, apart from out-of-pocket expenses. Additionally, the project funding projections required to support the delivery of the public involvement elements of the project were underestimated; subsequently, the chief investigator needed to use unfunded time to manage some aspects of the project and secure additional non-NIHR funding to support enhanced co-researcher involvement. Participants emphasised that the co-researchers also used unpaid time to work on the project during the project conceptualisation and funding application stages. At times, the financial reward received by co-researchers may not have been commensurate with the level of professional skill and experience informing their contributions to the project.

Benefits provided by public involvement during the I-ASC project

This theme was generated from data relating to the ways in which participants perceived the I-ASC project to benefit from public involvement (Table 22).

Some participants reported that they found it difficult to identify or quantify concrete benefits that they could attribute to the public involvement contribution to the project; there was also a perception that benefits may become more apparent in the long term. However, the majority of participants were able to identify a range of important benefits to the project that they associated with its public involvement elements.

Researchers, particularly those in clinical service delivery, suggested that their experience of working with the co-researchers had made them more aware of the lived experience of people who use AAC and their families; one participant stated that their involvement with the co-researchers had inspired them to make changes to the way in which clinical service delivery was organised locally to ensure that service users’ voices were represented:

. . . it’s probably inspired me to do more of that within our service outside of research, which we are doing. So we got a member of staff that’s now an honorary member of staff. He’s someone that uses AAC.

Researcher, group 1

Participants indicated that the public contributions made by the co-researchers during the data collection and dissemination phases have challenged public perceptions about the ability of people with disabilities to be involved in research. Furthermore, the co-researchers have proved empowering role models to other AAC users, their families and the people who work with them:

Having co-researchers involved in the interviews, again, really had an impact on participants . . . So I think the message that sends to families of young AAC users is very powerful and very important.

Researcher, group 1

. . . people listening to [the co-researchers] have really been fascinated, interested in their role and viewed them as very competent people who are doing the presentation . . .

Researcher, group 1

The data suggest that the opportunities provided by being involved in the project empowered the co-researchers; their involvement enabled them to learn new skills, engage in meaningful and financially rewarding work activity, gain confidence and feel valued for their contributions:

Being on this project as a co-researcher has been a huge learning curve for me in plenty of aspects . . .

Co-researcher, group 1

. . . it’s been good to get the opportunity to work within the field again.

Co-researcher, group 1

. . . it has helped me to feel valued for skills that I haven’t used for a good number of years.

Co-researcher, group 1

The data suggest that the I-ASC project research process benefited in different ways from public involvement. Researchers were able to identify methodological insights into data collection and analysis that originated from the co-researchers’ unique perspectives. Actively involving co-researchers in data collection was associated with improvements in the quantity and quality of data sourced:

I think engaging the co-researchers in actual data collection, as I said, allowed data to be accessed.

NIHR Advisory Board member, group 4

Researchers commented that as a result of the direct contributions by the co-researchers they had learned new approaches to making research dissemination more accessible and engaging to the public:

. . . [the co-researcher] might say, ‘I don’t understand that,’ . . . that would make us go away and think, ‘Well . . . yeah, actually it doesn’t make sense logically what we’ve just said’ or ‘it’s just not clear enough’ . . . So it’s definitely helped frame how we’ve reported the results, which is really important.

Researcher, group 3

Researchers reported benefiting from witnessing the positive value of public involvement and from new learning about how to involve the public in research (e.g. in terms of how to recruit and support co-researchers with specific needs):

. . . it has made me think twice about how I go about recruiting PPI, PI involvement [patient and public involvement/public involvement] . . . paying much more heed to what I suspect are the skill and experience sets required for a particular activity.

Researcher, group 1

Participants associated public involvement during the project with superior research outputs. They reported that they believed that clinical resources originating in the project would be more useful and useable and that any publications would be stronger. Participants described the project outputs as having increased face validity and thereby greater credibility with research consumers as a result of the co-researchers’ involvement. This was associated with the potential for enhanced research impact:

And its impact will be greater because it was a co-produced project.

Researcher, group 2

Researchers reported that working alongside co-researchers was rewarding. They associated public involvement with improvements in team communication processes and identified ways in which the co-researchers’ contributions helped the team to feel grounded. One participant suggested that comments from co-researchers during discussions about research dissemination had helped the team to feel more ambitious about the way they conceptualised and communicated their research findings:

. . . it gave the project team a bit more braveness in being able to do it I think.

Researcher, group 1

Challenges associated with public involvement during the I-ASC project

This theme was generated from data relating to the challenges that participants associated with incorporating public involvement into the I-ASC project (Table 23).

Different types of challenge were identified in the data that could have an impact on researchers and co-researchers involved in other projects. Participants suggested that identifying individuals who want to be co-researchers and are representative of a specific clinical population could be difficult. Researchers expressed concerns that the co-researchers working on the I-ASC project might not be representative of the wider population of people who use AAC and their families:

. . . I think that was a challenge in that [the co-researchers] might say things or interpret bits of data or interpret things that had happened in one way, whereas I think we might know from our practice in going to see the very wide variety of people in lots of different socioeconomic circumstances, that there is a wider reflection, there is a wider spread of opinions, perspectives, realities in terms of how people live than they had.

Researcher, group 1

Participants identified that the co-researchers were not representative of younger people or children who use AAC, the project’s targeted clinical population. Researchers suggested that the research outputs might have been strengthened by including a wider group of co-researchers. Although the critical friend group was intended to go some way to addressing this concern, it was clear that the group demographic did not deliver a child-oriented perspective.

As reported above, the team needed to overcome structural barriers to enable co-researchers to take part in research activities. It was not possible to obtain an NHS research passport for one co-researcher because of their employment status; they could not be employed by the university as this would have jeopardised their entitlement to state disability benefits, and this excluded them from certain data collection activities. Researchers indicated that involving people with communication and physical disabilities in research is inherently challenging and takes more time than research that does not involve these groups. For example, practical adjustments needed to be made to the research environment to make it more accessible to co-researchers who use wheelchairs. The project team experienced considerable difficulty in recruiting consistent support from the personal assistants of one of the co-researchers to enable this individual to participate fully during the early stages of the project. Participants also acknowledged challenges in making research activities accessible to people with different types of educational background and life experience.

Participants emphasised that the team encountered challenges when attempting to pay the co-researchers equitably for their time. Structural barriers related to employment contractual arrangements and the welfare benefits system prevented the co-researchers from being paid on an equal basis to the researchers. Researchers associated this with a lack of parity within the team:

So that actually made the relationship different. Even though we wanted you [the co-researchers] as equal partners, actually, we weren’t able to have you as equal partners in terms of reimbursement at that point.

Researcher, group 1

Perceived power relationships between researchers and co-researchers also presented a source of challenge to communication within the project team. Researchers suggested that they felt challenged by some of the co-researchers’ views or responses during team discussions but did not always feel empowered to challenge those views themselves:

I think there was sense that everyone was equal but then I think some of the researchers felt maybe we couldn’t always challenge [the co-researchers] . . . so then it’s not fully equal, in a way.

Researcher, group 1

Individuals may not have sufficient time or resources to be involved in research activities, or they may not be able to secure sufficient personalised support to be involved. Individuals may not feel empowered to challenge researchers’ views or to ask for support to understand research concepts. This is especially important for people who use AAC, whose communication in group discussions is likely to need to be supported by the whole group:

. . . I think if [the co-researcher] wasn’t understanding something, I don’t necessarily think [they] challenged it back said, ‘I don’t understand that’.

Researcher, group 1

Individuals who use AAC may find other people’s reactions to their involvement challenging and potentially disempowering. For example, one research participant appeared surprised that a person who used AAC was actively involved in the project:

. . . there was one person we were interviewing, he was then surprised that [the co-researcher] was asking questions. And I was like, I couldn’t, I cannot believe this has happened to [the co-researcher].

Researcher, group 1

Facilitators of successful public involvement on the I-ASC project

This theme relates to potential methods to facilitate public involvement in research and was constructed from participants’ responses relating to their experience of and reflections about the I-ASC project (Table 24).

Participants indicated that they felt it was important that the co-researcher role was clearly defined and firmly established within the research infrastructure to facilitate public involvement in future studies. Their responses suggest that research teams need to have a clear vision of what the co-researcher role involves while acknowledging that this will depend on the aims and objectives of individual projects. For example, it would be beneficial for researchers to understand and acknowledge that co-researchers bring unique perspectives to each project and cannot represent the views of an entire clinical population. Members of the public need to be motivated to engage in public involvement roles and supported to participate in ways that correspond to their abilities and preferences.

Participants indicated that public involvement could be facilitated if members of the public were able to access employed research roles on an equitable basis:

. . . I think one of the things that would have improved that was if there had been a budget to actually reimburse [the co-researchers] at that preparatory stage, because [they] were completely volunteering [their] input there.

Researcher, group 1

To achieve this, various mechanisms for enabling public involvement at local and national levels were proposed: funders need to explicitly encourage full costing for public involvement roles in their award application processes; support needs to include recruitment to public involvement roles, in terms of how to cost salaries, define the role (e.g. proposing a template job description) and identify potential candidates; policy could be enhanced and practical support and research training made available to enable people to access co-researcher roles; research teams should publish real-world exemplars of public involvement research to share learning and good practice with others.

Participants identified several ways of improving the planning of individual projects to facilitate public involvement. They emphasised the importance of understanding local processes (e.g. human resources) for identifying and recruiting co-researchers:

. . . know your university processes, has it been done before? . . . and actually try and pre-empt those challenges and barriers and start those initial conversations earlier . . .

University manager, group 3

Participants recognised a need for researchers to be realistic about the time, funding and other resources (e.g. accessible environments) required to facilitate co-researchers’ participation. In addition, participants highlighted the importance of enabling clear and open reciprocal communication between researchers and members of the public about each group’s expectations of the co-researcher role, available participation options and individual preferences. The co-researchers suggested that it was important to plan for co-researchers to be involved throughout the lifetime of a project, rather than at discrete stages, to help them to fully understand and contribute to the research activity:

It’s having that broad base of knowledge from the start of the project right through has actually helped to add the value, hopefully, rather than being a bit of a butterfly and coming in on a particular element.

Co-researcher, group 1

Participants emphasised the value of securing consistent, personalised, flexible and high-quality support for co-researchers with disabilities during research projects. It is also important that other members of the research team feel supported to work confidently with co-researchers with disabilities:

. . . if they’ve got cognitive impairment or mobility issues or communication, dealing with those kinds of issues, I guess it would be easy for most clinicians if they’re familiar with this. I suppose I’m in a different [position] because I’m a methodologist . . . so getting help on that front would be important.

Researcher, group 2

Additionally, co-researchers should be offered training in research methodology and flexible working options (e.g. the opportunity to work from home) to support their participation.

Another theme identified in the data related to the need for leaders to create a team ethos that facilitates public involvement. Participants’ responses indicate that an asset-based, inclusive approach to project management can facilitate participation by members of the public:

It’s about the value, the recognition as equals. The fact we’ve looked at [the co-researchers’] individual needs and built it in, not made it a problem. You know, it’s that attitude of . . . it’s asset-based rather than a sort of negative-based approach.

University manager, group 3

Co-researchers need to feel confident in expressing their needs and strengths when making decisions about how they might contribute to projects. All members of the research team need to take time to get to know one another so that they can understand each other’s roles, backgrounds and expectations. Participants suggested that spending some social time together, especially in the early stages of a project, could facilitate such team-building. This may support team members to appreciate their colleagues’ viewpoints and better value their contributions to group discussions:

Try to get to know your colleagues and value and understand what they’re bringing to the project.

Co-researcher, group 1

Participants highlighted the importance of open, inclusive and respectful communication within the team; they identified that some team members may need support to manage challenging conversations between researchers and members of the public, which might arise when individual perspectives appear to conflict. Importantly, they were able to identify practical approaches to making communication during research activities more accessible to people with communication disability who use AAC.

Public involvement (work package 8) research team

Liz Moulam, Professor Sue Powell, Professor Francis Fatoye and Dr Mark Jayes. Francis Fatoye led the development and delivery of the cost–consequence analysis component of the WP. Liz Moulam was the lead delivery researcher for this component. Sue Powell offered project management support to Liz Moulam and Mark Jayes.

Support team

Kelly Lavender-Smith, Zoe Lingard, Zoe Johnston, Christopher Wills, Helen Bell, Gareth Preston, Dr Sue Caton, Dr Christina Sotiropoulou-Drosopoulou, Dr Sam Wright, Radek Loucka, Monwara Begum, Marie Basting, Charlotte Saunders, Chloe Levett-Smith, Sarah Brown, Wanausha Khafaf, Kerry Wilkins and Natasha Robinson.

Support team

Andrea Lee, Gemma Sleman and Beccy Masters.