Responsiveness of primary care services: development of a patient-report measure

Inequalities in patient experience in primary care

One consistent finding across primary care patient experience surveys is that certain patient groups tend to report less positive experiences. 1 Patients from black and minority ethnic (BME) backgrounds, patients with poor health status, poor mental health or disabilities, less affluent patients, and younger patients tend to report poorer experiences of accessing and using primary care. 2–5 The reasons for this variation in experiences across different patient groups are not well understood. 1,5 In the case of patients from BME groups, some of this variation may be explained by the fact that these patients tend to be clustered in generally poor-performing practices, although many BME patients report less positive experiences than other patients within the same practice. 5 The variation in patients’ experiences of care between practices and across different patient groups has been described as unacceptable. 6 These inequalities are a cause for concern, as poorer experience of primary care is likely to impact on and perpetuate inequalities in health across diverse patient groups.

Responsiveness as means of addressing inequalities: an emerging concept

These concerns led to a focus in English policy from around 2008 onwards, under the Labour government, on efforts to improve access and responsiveness in primary care. Primary care providers were encouraged to work to be more ‘responsive’ – to move from a ‘one size fits all’ approach to delivering their services, towards a more flexible and responsive approach that takes into account the specific needs of their local populations. Guidance from the Department of Health (DH) underlined this aspiration, stating that, ‘there is no “one size fits all” model, it is for practices to personalise their services to meet their patients’ preferences’7 (see Chapter 3, Inequalities and the needs of diverse groups).

The concept of responsiveness was seen as closely linked with access, with the DH explicitly conceptualising practice responsiveness as a component of patient experience of access in primary care. 5 Definitions of the concept incorporate communicating and engaging with patients and carers, making efforts to identify and meet their non-clinical needs (i.e. their needs in terms of accessing and using services, rather than health needs or needs relating to the consultation or the clinician–patient relationship) and engaging in efforts to support patients from specific groups who may be vulnerable or experience particular difficulties in using primary care.

Practice responsiveness is the way in which a practice communicates and engages with its patients and their carers and responds to their non-clinical needs and preferences, reflecting the different ways in which they might prefer to access the service and an appropriate clinician, book, or indeed cancel an appointment. It includes the practice’s attitude to customer service and friendliness of staff, the environment in which patients wait to be seen and the way in which they interact and support patients from particular groups, such as those with hearing or sight loss or people from a black or minority ethnic background. 7

p. 15

The importance of supporting patients from diverse groups to access and use primary care services was underlined by a subsequent report on improving the experience of BME patients. 8 The report identified multiple reasons for the poorer access and experience of primary care of patients from BME groups; these included language and culture barriers, poorer health status, variable quality of GP services, and different expectations of BME patients. The report suggested that practices needed to work in partnership with BME patients, plan for their needs and ensure that the services were personalised to meet identified needs, and concluded that, ‘more responsive and personalised care will mean benefit for all members of the community – black and white – through the lessons learnt in service development’ (p. 3). 8

This focus on identifying and responding to the non-clinical needs of patients from diverse groups represented a shift in thinking, in that much of the work on inequalities and patient experience in primary care has tended to be concerned with clinical or health needs, and features of the consultation. This has included research into communication skills and barriers, patient-centredness and cultural competence, and issues such as patient–physician gender and race concordance. 9–11

Improving responsiveness

A review of key English policy and discussion documents highlights the key domains of service which have been identified in policy and guidance as those that GP practices should attend to in improving their responsiveness. These include engagement and communication with patients; practice opening hours; ease of contacting the practice; the ability of patients to easily make appointments that meet their needs for urgency, timing, and choice of practitioner; equity of access; alternative modes of consultation; the use of information technology (IT) to facilitate information and access; availability of premises and appropriateness of the physical environment, including accessibility; customer care; and adjustments for specific groups, including minority ethnic groups, disabled people, young people and carers. 7,12–15

Policy directives on responsiveness from 2008 onwards were accompanied by efforts to support and incentivise primary care providers to become more responsive. In 2008–9, a Primary Care Service Framework for accessible and responsive primary care13 was developed, setting out standards for care provision to improve responsiveness, and providing guidance for primary care trusts (PCTs) as to how practices might demonstrate their responsiveness. Under this framework, GP practices could claim financial reward for demonstrating their uptake of elements of responsiveness set out in the framework. Alongside this, the Practice Management Network produced extensive guidance on how to improve access and responsiveness in the form of a ‘how to’ manual,14 and a DVD to train reception staff on improving the patient experience. 15 These guides remain available, and give examples of good practice, and advice for practices in improving customer care, improving access for groups who are disadvantaged, and effectively engaging with patients. At the time that these resources were launched, they were accompanied by a training programme organised by the Royal College of General Practitioners (RCGP).

Responsiveness in the current context of primary care

Responsiveness was a highly prominent theme in English primary care policy under the Labour government (predominantly between 2003 and 2010). Despite significant political changes and NHS reorganisation, responsiveness remains a core aspiration for primary care. The DH White Paper Equity and Excellence, produced under the coalition government, calls for a ‘genuinely patient-centred approach in which services are designed around individual needs, lifestyles and aspirations’. 16 The new Clinical Commissioning Groups (CCGs) have a constitutional duty to reduce inequalities between patients in relation to their ability to access primary care services, and the outcomes achieved,17 and as such, issues of accessibility and responsiveness of local primary care providers are likely to be of concern to them. There have been some slight shifts in focus. Under the coalition government, responsiveness has become associated with the personalisation and choice agenda:16

In future, patients and carers will have far more clout and choice in the system; and as a result, the NHS will become more responsive to their needs and wishes.

p. 16

In recent years there has also been a growing emphasis on the role of patient engagement and involvement in promoting responsiveness. Interest groups such as the National Association for Patient Participation (NAPP) have taken a keen interest in responsiveness, emphasising the importance of ‘effective patient participation [in] ensuring the quality and responsiveness of services and health outcomes for patients and the wider community continuously improve’. 18

Responsiveness has also been highlighted as a key element of ‘the right culture’ post Francis, with a focus on responding to patients’ needs and preferences, encouraging a patient-centred philosophy, responding to complaints, and listening to patients. 19

A range of initiatives and programmes continue to be made available to support efforts to improve responsiveness in primary care. These include the ongoing RCGP Quality Practice Award programme; its ‘patient centred care module’20 sets standards relevant to responsiveness, and indicates evidence that practices can provide to demonstrate their performance in these areas. Initiatives such as Productive General Practice (developed by the now defunct NHS Institute for Innovation and Improvement, and still in use),21 and The King’s Fund’s experience-based codesign toolkit,22 provide guidance and support for primary care providers in involving patients in the task of identifying unmet needs within local populations, and aligning the design and delivery of services with the needs of the patients. There remains a need for research to inform approaches to addressing inequalities on health care in cost-effective ways; research into access, quality, cost and effectiveness of services for disadvantaged groups has been identified as important. 23

Measures of responsiveness

Responsiveness can best be understood as a feature of the PCO: the extent to which the organisation (1) has processes and systems in place to help them to understand the non-clinical needs of patients and how best to address them; (2) designs their services in ways that are directed at meeting these identified needs; and (3) takes a flexible and personalised approach to delivering services and helping patients gain access to the services they need.

It is evident that responsiveness of a PCO can be assessed through relatively ‘objective’ measures of these efforts, for example the extent to which the practice has clearly-defined strategies to identify the needs of their patient population, or the extent to which the practice provides different routes for patients to book appointments that are appropriate to the needs of the local population.

Alongside this, it is also important that the views of patients are sought. Responsiveness is ultimately reflected in patients’ perceptions of whether or not the service provided actually meets their non-clinical needs. Users of a service will have a range of needs and preferences, and the patterns of these will vary across services and localities; changes in service provision targeted at improving responsiveness may meet the needs of some groups of patients, but fail to address the needs of others, meaning that groups who are disadvantaged may be no better served.

While there are a number of validated and widely used patient experience questionnaires for primary care, including the annual national GP patient survey,24 there are currently no patient-report questionnaires that are designed specifically as measures of patient experience of responsiveness in primary care. Existing questionnaires include some non-clinical aspects of care (primarily issues relating to access and the behaviour of reception staff), but also include more extensive questions on the clinical aspects of care (quality and experience of the consultation).

There is extensive evidence on what matters to patients (across clinical and non-clinical domains of primary care25,26). A useful distinction has been made between relational aspects of care (e.g. dignity, empathy), and ‘functional’ aspects (e.g. access, waiting):1 non-clinical aspects of care may fall within both of these dimensions. The recent NHS Patient Experience Framework brings together evidence on patient experience and sets out the key elements of patient experience across the NHS. 27 The framework includes some ‘non-clinical’ elements, particularly access, privacy, and transition and continuity.

The framework is designed to be used flexibly, and it is acknowledged that, ‘different areas of the framework will be more significant in particular settings for different groups of patients and therefore demographics, equality, and environment will need to be considered when applying the framework’. 28 It is not clear, however, which non-clinical features of primary care are most salient for patients from different groups in judging the responsiveness of primary care services to their needs.

It is important to recognise that the boundary between clinical and non-clinical needs may be blurred, but the focus on responsiveness brings attention to the specific issues of whether or not patients find services easy to access and use, and whether or not they feel that services are aligned with their personal, cultural or lifestyle-related needs. If a questionnaire is to act as a valid measure, it is essential that it cover these issues.

A further consideration is that existing patient experience measures tend to achieve a low response rate from patients from traditionally disadvantaged groups. The term ‘seldom heard’ has been used to describe these groups, defined as those who do not have an effective voice in relation to public and voluntary service providers, and who are the most vulnerable to ill health because of social and economic disadvantage. This includes minority ethnic groups, people with disabilities or mental health problems, refugees, travellers, homeless people, ex-offenders, migrants who do not speak English and others who are socially excluded. 29,30 Accessing the views of these ‘seldom-heard’ groups is critical in assessing whether or not practices are responding fairly to the needs of all. There has been some recognition of the need to find ways of gathering more targeted and detailed information about experiences across patients from different groups. 1

In this study, we aim to develop a measure of patient experience of responsiveness that includes the non-clinical needs identified as most important for patients across diverse groups; is worded to tap into responsiveness in terms of ease of getting needs met (rather than simply measuring experience); and is designed to be appropriate for use with patients who are traditionally seen as ‘hard to reach’ and hence ‘seldom heard’.

Diversity of primary care providers

We aim to develop approaches to measuring responsiveness across a range of primary care providers. While the GP practices remain core providers of primary care, there is a diverse set of organisations that fall under the umbrella of primary care, and these organisations differ in the types and nature of services they deliver. Walk-in centres, minor injury units and urgent care centres all provide primary care services which are accessible without appointments and are open to patients outside traditional working hours. Traditional general practices have diversified, with the emergence of specialist practices providing services to vulnerable groups such as asylum seekers and homeless patients; large health and social centres enable patients to access a wide range of services under a single roof. In addition, patients consult pharmacists for advice, and pharmacies commonly offer extended health services such as health checks. There may be specific challenges to responsiveness across these different types of organisations, and patients are likely to have different expectations of the types of services they can access and the way in which these will be organised and delivered across the different providers. These differences have implications for approaches to measuring responsiveness in primary care.

Summary of methods

Development and testing of the questionnaire had three main stages. The overall study design is described below and in Figures 1–4. Further details of the methods are provided for each part of the study in the corresponding chapter of this report.

Stage 1: literature review and interviews with patients and staff

This stage commenced with a literature review on the meaning and measurement of responsiveness (Figure 1). Different fields in which responsiveness has emerged as a core concept were scoped, and from this literature we identified key items and followed narrative threads.

FIGURE 1.

Stage 1: literature review and qualitative interviews. a, In 12 PCOs both staff and patient interviews were conducted, while in one PCO only staff interviews were conducted.

We then conducted qualitative work with patients and professionals in order to understand how patients experienced responsive primary care, the ways in which practices had attempted to deliver responsive service, and the barriers and facilitators to this. We also explored the important features of a questionnaire on responsiveness for patients and for PCOs. We conducted semistructured interviews with staff (n = 27), patient representatives (n = 6) and patients (n = 58) from 13 PCOs in the East Midlands. Recruitment was supported by the Primary Care Research Network (PCRN) for the East Midlands and South Yorkshire; practices in the South Yorkshire region were not included. Thirteen PCOs were identified through the PCRN and were selected to include mainstream GP practices of varying size, population demographics and location, as well as other types of PCO, including a walk-in centre, pharmacies and a specialist GP practice for homeless patients (Table 1).

TABLE 1 - Characteristics of participating PCOs

WIC, walk-in centre.

Size key for GP practices: small is < 5000 patients, medium is between 5001 and 10,000 patients, large is > 10,000 patients; for pharmacies, subjective view from the pharmacists based on number of prescriptions; for WICs, subjective view based on footfall.

From Practice Profiles. 31 Pharmacy and walk-in centre deciles are derived from the postcode of the nearest GP practice: 1 is the most deprived, 10 the least deprived.

PCO01 withdrew after the staff interviews and so did not contribute to patient interviews.

GP practices that took part in the mapping population study.

GP practices that took part in the substudy of seldom-heard groups.

PCO	Type	Location	Sizea	Region	Deprivation decileb	Stage 1	Stage 2 pilot 1	Stage 2 pilot 2	Stage 3
01	GP surgery	Urban	Large	East Midlands	5	✓c	–	–	–
02	Pharmacy	Inner-city	Large	East Midlands	4	✓	–	✓	✓
03	Pharmacy	Rural	Large	East Midlands	10	✓	–	✓	✓
04	GP surgeryd	Town centre	Large	East Midlands	5	✓	✓	–	–
05	GP surgery	Rural	Medium	East Midlands	10	✓	–	–	–
06	GP surgery	Inner-city	Small	East Midlands	4	✓	–	✓	–
07	Homeless GP surgery	Inner-city	Small	East Midlands	2	✓	–	–	✓
08	Health and social care centred	Inner-city	Large	East Midlands	1	✓	–	–	✓
09	WIC	Urban	Large	East Midlands	4	✓	–	–	✓
10	GP surgery	Urban	Large	East Midlands	4	✓	–	✓	–
11	GP surgery	Urban	Medium	East Midlands	6	✓	✓	–	–
12	GP surgery	Rural	Small	East Midlands	8	✓	✓	–	–
13	GP surgeryd	Inner-city	Small	East Midlands	3	✓	–	–	✓
14	Pharmacy	Inner-city	Large	Greater London	1	–	–	–	✓
15	Pharmacy	Inner-city	Large	Greater London	2	–	–	–	✓
16	GP surgerye	Inner-city	Small	Greater London	1	–	–	–	✓
17	WIC	Inner-city	Large	Greater London	1	–	–	–	✓
18	GP surgery	Urban	Medium	Greater London	1	–	–	–	✓
19	GP surgery	Recruited for stage 3 but withdrew prior to data collection
20	GP surgery	Rural	Small	Northern & Yorkshire	10	–	–	–	✓
21	GP surgery	Rural	Medium	Northern & Yorkshire	8	–	–	–	✓
22	GP surgerye	Town centre	Small	Northern & Yorkshire	2	–	–	–	✓
23	WIC	Town centre	Large	Northern & Yorkshire	2	–	–	–	✓
24	Pharmacy	Town centre	Large	Northern & Yorkshire	3	–	–	–	✓

Based on the analysis of these interviews, we generated descriptions of the key components of responsive primary care as experienced by patients, and used these to develop a set of potential questions for the questionnaire.

Stage 2: developing the questionnaire

We conducted three patient focus groups specifically targeted at patients from ‘seldom-heard’ groups, to explore and validate understanding and acceptability of the questions. As a result of the focus groups, the questions were refined. Although we had intended to develop a short, generic questionnaire that would be applicable to all primary care providers, including GP practices, pharmacies and walk-in centres, the findings from the interviews and focus groups suggested this would not be feasible or acceptable to patients. We elected to lead with the development of a questionnaire for GP practices, which would then be adapted and modified to make it more appropriate to other specific primary care settings.

A draft questionnaire was developed including the revised questions, open questions to allow comments, and sociodemographic questions. A version was produced in paper and another online. The draft questionnaire was subject to two rounds of piloting, involving administration of the questionnaire to patients in primary care, semistructured interviews with responders to assess face validity and acceptability to patients, and cognitive interviews to assess comprehensibility. Piloting informed the refinement of questions, and provided insight into the feasibility of use of the questionnaire across different service providers (Figure 2).

FIGURE 2.

Stage 2: piloting of the questionnaires.

Version 1 of the questionnaire related to GP surgeries only, with the plan to adapt for other primary care providers. The first pilot was conducted in three GP practices selected from the 12 PCOs remaining after stage 1 (one practice withdrew during stage 1). A patient survey was handed out to 150 patients in each practice and a purposive sample of responders who expressed an interest were invited to take part in semistructured interviews to assess face validity and acceptability. The questions were modified and a pharmacy version was developed. The second pilot was conducted in a different set of two GP practices and in two pharmacies. In this second pilot, 450 patients received a questionnaire and cognitive interviews were conducted with a sample of responders to identify problems with question wording or questionnaire design.

We also conducted a consultation with relevant groups and individuals, using contacts through the advisory group (e.g. learning disability organisations, community organisations representing patients from ethnic minority groups, people with visual impairment) to identify preferences for a range of formats, and approaches to administering the questionnaire; members of the project team also contributed to this.

The questionnaire and the approaches to administering the questionnaire were revised iteratively in the light of the findings from the pilots. Version 1 of the GP surgery questionnaire was tested in pilot 1, and then amended to version 2 in pilot 2. Version 1 of the pharmacy questionnaire was developed for pilot 2. In stage 3 (see Figure 3), version 3 of the GP surgery questionnaire and version 2 of the pharmacy questionnaire were tested, and version 1 of the walk-in centre questionnaire was developed and tested.

Stage 3: testing the questionnaire

To assess the reliability, validity and acceptability of the questionnaire, a large-scale pilot was undertaken in 16 PCOs across three regions in England: six of the PCOs recruited for stage 1 of the study, five new PCOs from the Northern and Yorkshire/North West region, and five new PCOs from Greater London (Figure 3). PCOs were selected to ensure a diverse sample, and included eight GP surgeries, five pharmacies, three walk-in centres and one Health and Social Care centre (Table 1). The questionnaire was handed to a sample of up to 200 users attending most PCOs, except the two pharmacies and a small specialist homeless practice recruited from stage 1 that handed out 100 each. As one practice withdrew before it had handed out any patient packs, these 200 were split between four other practices; hence four GP practices were asked to hand out 250. Two practices did not manage to hand out their full quota during the time scale of the study. Responders who expressed an interest were mailed a second copy of the questionnaire, to enable test–retest reliability to be assessed. Feedback was gained from staff in participating PCOs on their experiences of administering the questionnaire, and the value of the data to them in informing changes to practice.

FIGURE 3.

Stage 3: large-scale testing of the questionnaires.

As part of stage 3, we worked with a sample of PCOs to explore ways of including patients from ‘seldom-heard’ groups in the survey process. We conducted informal interviews with staff co-ordinating the survey to identify the ‘seldom-heard’ groups within their patient population and to discuss ways of supporting these patients to become involved. The study team provided support and resources to practices to enable them to undertake additional work to include ‘seldom-heard’ groups in the survey, in ways that were tailored to the needs of specific patient groups. We also engaged a specialist in Easy Read materials to produce an Easy Read version of the questionnaire. This was subject to small-scale testing with people with learning disabilities.

The questionnaire developed as a result of this process is available in three versions: a GP surgery version, a pharmacy version and a walk-in centre version. The majority of questions are common across the three versions, but each contained some questions specific to the particular type of PCO. The questionnaire was designed as a self-completion paper questionnaire in standard and Easy Read formats, but can also be interviewer-administered (including via an interpreter) or completed online.

Mapping population diversity substudy

We also undertook a substantial substudy to explore approaches to and attitudes towards mapping diversity in primary care. This involved a review of available methods for population mapping and the challenges to this, and interviews with staff in four participating GP surgeries about their experiences of and attitudes to exploring the characteristics of their patient populations (Figure 4).

FIGURE 4.

Population mapping.

Data analysis

For the initial staff and patient interviews, and for narrative interviews in pilot 1, data were analysed using a combination of framework analysis and the constant comparative approach, which involved both deductive and inductive elements. An initial coding frame was generated from the research questions, which acted to guide, but not constrain, the analysis; themes and subthemes were added and iteratively revised as additional interviews were coded. The cognitive interviews were coded and charted with a focus on comprehension and response selection. NVivo 8 (QSR International, Warrington, UK) was used to organise the qualitative data.cr.

For pilots 1 and 2, we used SPSS (IBM Corporation, Armonk, NY, USA) to generate descriptive statistics for each question. In pilot 3, we carried out an exploratory factor analysis to investigate the structure of the GP questionnaire, and to identify key questions to form the basis of scores for the questionnaire which could be used as a measure of patient experience of responsiveness. We conducted analysis to provide initial evidence about reliability and validity. 32,33 We used Cronbach’s alpha coefficients to assess internal consistency for scores from identified subfactors. Pearson correlations were calculated between the scale scores from our questionnaire, and three indicators of construct validity. Kappa and intraclass correlation coefficients (ICCs) were calculated for the test–retest questionnaires to assess test–retest reliability.

Assessment of the method

Methodological challenges

The greatest challenge was to provide the time and material resources to support the PCOs in their participation. As with most primary care research, there was a continuum of commitment and engagement from the participating PCOs. Some were extremely keen and contributed quickly and efficiently, with little input from the research team. At the other extreme were PCOs that needed regular reminders and prompting to ensure their continued participation. Some required material input, such as having ready-printed letters provided for use rather than printing the letters themselves on their headed paper. Unfortunately, one GP practice withdrew due to internal staffing changes during stage 1 (after the staff interviews, prior to the patient interviews). Another practice that agreed to participate in stage 3 eventually withdrew after many failed attempts on behalf of the research team to get data collection started.

Ensuring that the ‘seldom heard’ were included in stage 3 pilot was difficult. The two practices that engaged with the substudies to reach out to ‘seldom-heard’ groups were keen to participate and found the experience extremely rewarding. However, other PCOs were less interested in committing the time and resources. Administration of the substudy required additional administrative time and constant management.

Some of our methods were iterative and reactive to previous experience. During the 3-year project, we submitted seven substantial amendments to the research ethics committee and associated RM&G offices. These were time-consuming in terms of both preparation and approval processes. Although all amendments were ultimately successful, the project was held up on several occasions while we waited for approvals.

Two further methodological challenges arose. The first related to our aim of developing a measure of patient experience of responsiveness. Our initial intention was to design this to include a small number of generic, high-level and relatively abstract questions which would produce a single score. However, testing this approach with focus groups demonstrated that this was not acceptable to patients, and was unlikely to be valid. Patients wanted concrete questions that they could relate to their experiences. As a result, the questionnaire we developed was more specific and detailed than planned. However, our work in pilot 3 demonstrates that, despite this, it can be used to produce responsiveness ‘scores’.

The second methodological challenge relates to our aim of producing a measure that would be generic. For the reasons described above, this was not seen as acceptable to patients, so we instead developed a core questionnaire for GP practices and adapted this for pharmacies and walk-in centres. This means that the additional versions were not subject to the same level of development and testing as the GP questionnaire and are likely to require more work before they are ready for use.

Service quality

Responsiveness has a long history of being considered as a core element of service quality. 34 Service quality has been defined as the ability of the organisation to meet or exceed customer expectations. 35 Parasuraman and colleagues’ widely accepted model of service quality includes five key dimensions: tangibles (features of the service environment); reliability; responsiveness; assurance (the extent to which the organisation and its employees are perceived as credible and trustworthy); and empathy. Within this context, responsiveness has been defined as an organisation’s employees’ ‘willingness to help customers and provide prompt service’ (p. 23). 35

Service quality is seen as important for customer-serving organisations in marketing,36 achieving success and profitability through helping to attract and retain customers,37 and promoting customer satisfaction. Responsiveness is recognised as particularly important to the public image of customer-serving organisations; in the UK, the Institute of Customer Service38 makes a high-profile annual People’s Choice award for ‘Most Responsive Organisation’.

The notion of responsiveness as an element of service quality also features in the health literature. Several papers speak of responsiveness as part of a high-quality service, and as a feature of customer service or interactions between staff and patients. For instance, in a qualitative study of patients’ expectations and experiences of public and private providers, patients were found to expect more responsiveness and better quality of care and to be willing to pay for it. 39 A study of predictors of hospital patient satisfaction ratings in the USA found that nursing staff communication with patients had an important bearing on perceived responsiveness. 40

The field of work on service quality has generated a number of measures; the most widely used of these is the SERVQUAL scale. 35,41 This incorporates the five dimensions of service quality described above. The SERVQUAL scale includes a subscale of four questions to measure responsiveness:

• P10: employees of XYZ tell you exactly when services will be performed

• P11: employees of XYZ give you prompt service

• P12: employees of XYZ are always willing to help you

• P11: employees of XYZ are never too busy to respond to your requests.

SERVQUAL has been used extensively across a wide range of settings to measure service quality, either in its original form or adapted to the specific setting. 42,43

The SERVQUAL dimensions have been found to be applicable and stable in the measurement of service quality in health care,44–46 and specifically in primary care,47 with responsiveness remaining a core element of service quality in this context. SERVQUAL has been adapted and used in studies of service quality in a range of health-care settings48–51 but has not found wide application in the context of primary care. A version of SERVQUAL designed to assess the quality of hospital services has been developed,52 and SERVQUAL has been adapted for use in a primary care clinic in the USA. 53 The revised scale was assessed for factor structure, reliability and validity; responsiveness remained a core element of the scale. Four questions on responsiveness were included, relating to prompt service; employee willingness to help; staff never too busy to respond to requests; and convenient opening hours.

The conceptualisation of responsiveness as an element of service quality has several key implications. First, responsiveness is tied to customer service – the quality and promptness of interactions between employees and customers (or between staff and patients) – rather than other features of the organisation. Second, the notion of service quality focuses on improving customer experience, exceeding expectations, increasing satisfaction, and even ‘delighting’54 the customer. It is seen as a route to attracting and retaining customers, and increasing market share. Within this literature, responsiveness is not seen as a ‘duty’ or essential feature of an organisation, but as a ‘value adding’ feature of service. Third, the focus is not on diverse or disadvantaged groups, but on improving the experience of all customers/patients.

Inequalities and the needs of diverse groups

The concept of responsiveness is also prominent in the field of work relating to health inequalities. Seminal work undertaken by the World Health Organization (WHO) in the late 1990s and early 2000s identified responsiveness as one of the three intrinsic goals of health systems (along with good health and fair financing). 55 Responsiveness involves health systems meeting the needs of the patients they serve, and is based on the idea that there are fundamental needs, or basic human rights, that health systems should meet for all patients. These needs are seen as relating to the non-clinical domains of health service provision. Responsiveness is defined as ‘how well the health system meets the legitimate expectations of the population for the non-health enhancing aspects of the health system. It includes seven elements: dignity, confidentiality, autonomy, prompt attention, social support, basic amenities, and choice of provider’ (p. 1). 56 It is suggested that if a health system is responsive, the interactions that people have within the health system may improve their well-being, irrespective of improvements to their health. 57

The WHO argues that the measurement of responsiveness is essential to assessing the performance of health systems, and importantly, involves measuring ‘both the overall level of achievement (average over the whole population) as well as the distribution (equitable spread of this achievement to all segments of the population)’ (p. 1). 56 It is clear that this conceptualisation of responsiveness involves fairness and avoiding inequalities across the patient population.

By their definition, the WHO sought to measure responsiveness objectively by looking at how patients perceive what happens in their experience of using health care. 56 They developed a responsiveness instrument58 and tested it across 35 countries. Factor analysis revealed the seven elements used in their definition, above. 59 Levels of responsiveness were measured using a patient-report questionnaire, split into two sections: patients were asked, first, to rate their experience with the system, and second, to rate how important each element was to them. Each element comprised 3–7 questions. 56 Questions were formatted to assess the extent to which patients felt their needs for each dimension were met, for example:

• In the last 12 months, how often did the office staff, such as receptionists or clerks there, treat you with respect?

  • always

  • usually

  • sometimes

  • never.

Data on inequalities in distribution of responsiveness were generated through surveys completed by key informants in each country; informants were asked about whether or not they felt that particular patient groups were discriminated against with regard to responsiveness in their country. 60

The WHO responsiveness measure has been widely used in studies to assess health system responsiveness internationally. 61,62 This has included work to identify predictors of health system responsiveness; evidence suggests that health-care expenditures per capita, and educational development, are positively associated with responsiveness, while public sector spending is negatively associated with responsiveness. 63 The WHO definition has also been used as the basis of work to identify unmet needs in disadvantaged groups, including a literature review to identify the extent to which people who access mental health services were not having their needs met. 64

The WHO work focuses on responsiveness of health systems, rather than individual providers such as GP practices. The WHO definition was adapted for primary care by Canadian researchers to produce a definition of responsiveness as the ‘ability of the primary care unit to provide care that meets the non-health expectations of users in terms of dignity, privacy, promptness, and quality of basic amenities’ (p. 341). 65 They argued that the operationalisation of responsiveness was problematic due to lack of distinctness from other concepts such as whole-person care. Their earlier work to develop national indicators suggested two core questions to ask about responsiveness: ‘Are patients satisfied that the Primary Health Care organization and providers respect their right to privacy, confidentiality and dignity? Are patients confident that PHC organizations and providers are responsive to their culture and language needs?’66

The focus in health on responsiveness as involving providing services equitably and meeting the needs of all, across diverse groups, is echoed in work on responsiveness in the field of education. Responsiveness to learners’ needs has been considered in terms of meeting individual student needs, information, support, respect and opportunities to air views; these dimensions have been included in the National Learner Satisfaction Survey. 67

As discussed earlier, arguments for the need to promote responsiveness in primary care tend to draw heavily on the inequalities agenda with a recognition that responding to specific groups’ needs might help to reduce inequalities. 68 However, needs are framed in a slightly different way from the notion of universal legitimate expectations that is central to the WHO work. Instead, the focus is on the diverse needs of different patient groups, and ensuring that individual patient needs are met through individualised and proactive care. 69,70 Alignment with the needs of different patient groups is seen to be important, and there has been particular focus on the need for cultural alignment with minority groups. Research has shown how cultural self-reflection and self-awareness on the part of staff can be helpful, and how developing a reciprocal understanding of needs can lead to a flexible responsive service. 71 However, the evaluation of a human immunodeficiency virus mental health service that actively valued cultural responsiveness (acknowledging clients’ cultural identities, taking their beliefs, norms and values into account in the interventions) struggled to separate out whether cultural responsiveness or integrated care affected observed findings independently or in combination. 72

With responsiveness defined as an approach to reducing inequalities, the onus is on providers to ensure that they understand their practice populations, and can segment them based on need. 13,73 There is also an emphasis on finding proactive ways to reach out to marginalised groups who may find it difficult to access health care. 74,75

One challenge raised within this field relates to the nature of needs, expectations and demands. A distinction has been made between needs and demands in the context of health needs assessment, and this distinction may be useful in conceptualising needs in relation to non-clinical aspects of care. Under this framework, needs have been described as areas in which there is capacity to benefit, and as ‘normative’, i.e. they should be met. 76 Demands are what patients ask for (implicit in this is the notion that it may or may not be appropriate for providers to respond to demands). One study, exploring the views of staff about needs and demands in relation to public services, concluded that responsiveness related to identifying unmet needs (with a focus on cultural needs), and finding the right balance of managing needs and demands. 77 The authors define needs as rational demands (consistent and evidence-based) as opposed to demands as ‘desires’. They argue that public services should try to meet users’ demands, but that other forms of demand management may be required to realign these demands with users’ needs. They suggest that practical demand management in needs-based public service requires knowledge of users’ demand for services; content analyses of users’ demands to identify any misinformed demands; conversion of any misinformed demands into evidence-based specifications of needs; and formulating coherent evidence-based demands on behalf of users who cannot do so themselves. They acknowledge the tension between needs assessment being professionally controlled rather than responsive to users. 77

The key implications of considering responsiveness as relating to inequalities include, first, that responsiveness relates to a broad set of non-clinical features of service organisation and delivery. Second, responsiveness is seen as a core duty of an organisation – providing a level of service that meets patients’ needs fairly across all patient groups (although there are some differences in emphasis in relation to whether this refers to basic universal needs or specific needs of different patient groups or individual patients, and a recognition that responsiveness may involve managing demands). Third, the focus is on diverse groups and ensuring that no patient groups are disadvantaged in their experiences of receiving services. Under this definition, achieving responsiveness requires more than just good customer service; it requires an understanding of population characteristics, and proactive planning to meet needs and avoid disadvantage. Finally, measures of responsiveness need to pick up on differentials between different groups, and identify whether or not certain groups are disadvantaged (e.g. BME groups, deprived groups, males, females).

Consumerism and patient involvement

The focus on responsiveness has been linked with the shift towards consumerism and consumer demand for services that are tailored to individual needs. Responsiveness is seen as a way of bringing the NHS in line with other services such as retail and banking, which are geared towards being adaptable to individual needs, and offering more convenience, choice and flexibility. Fundamental to this shift is the idea that services should be geared to the interests of users rather than the convenience of producers. 78 Increasing user participation is described as core to this; an advisory document reporting on group discussions and a citizens’ forum found that:

Overall, people think that a responsive public service is one that: provides easy and appropriate access to services; encourages the individual to use and shape services in ways that suit them; actively seeks to learn from public involvement and develop services accordingly. 70

p. 5

People’s ideas on how public services could be made more responsive included making communication simple and obvious, keeping people informed throughout, involving people as early as possible, and shared responsibilities and shared outcomes. 70 User ‘choice’ and ‘voice’ are also seen critical: exit and voice (communicating user demands) have been described as two ‘recuperation’ mechanisms for making organisations responsive. 79

In the context of public service and administration literature, responsiveness has been contrasted with collaboration. 80 The authors described responsiveness as responding to requests for action or information. They viewed responsiveness as a mostly passive, unidirectional reaction to people’s needs and demands, whereas collaboration was a more active, bidirectional act of participation, involvement and unification of forces between two or more parties. Some definitions of responsiveness in primary care and patient involvement literature are more in line with this notion of collaboration.

Responsiveness in primary care has been defined as synonymous with patient participation, engagement and involvement. 81 NAPP suggest that being responsive requires practices to engage with patients; that patient experience is a key part of a responsive practice; and that improved communication and responsiveness are needed for a successful practice and patient participation group (PPG). 81 A discussion piece summarising the meaning of responsiveness included involving patients with service planning. 74 The authors found two ways that this was enacted. First, some seek to involve patients in the planning of care. Second, others make attempts to reach out to groups who find it difficult to access health care. A Scottish study of Local Health Care Co-operatives highlighted the need to engage patients and local communities, and defined responsiveness in terms of proactively engaging patients in planning services. 82

The close link between responsiveness and patient engagement is highlighted by the inclusion in a directed enhanced service (DES) framework on responsiveness of an eligibility criteria relating to engagement with patients,13 as the report states:

Improving access and responsiveness needs to be strongly founded on engagement with patients and should be a dynamic process. Providers should be required to demonstrate active engagement with people and local communities in developing services . . . Providers should demonstrate how they respond to patient feedback and this is to be used to shape and improve services . . . Local Involvement Networks (LINks), the voluntary sector and patient advocacy organisations are all further mechanisms to seek active involvement in service planning, delivery and monitoring.

p. 6

There are limitations to the conceptualisation of responsiveness as dependent on choice, voice and patient involvement. Changing providers is not always cost-neutral for users, and exercising voice adds practical burdens with little reward, hence users who exercise voice may be few, self-selected and apparently ‘unrepresentative’. 79 Many people with common health conditions, such as mental health problems, are reluctant or unable to engage in the ‘user movement’, hence undermining the effectiveness of patient involvement. 69

The key implications of definitions of responsiveness within the consumerism/patient engagement literature are, first, responsiveness framed in this way is about patients as consumers taking responsibility for defining78 and asserting their needs. The implication of this is that less responsibility is placed upon providers to proactively plan for and support disadvantaged groups. This is in tension to some extent with the notion of responsiveness as a duty of providers and as a way of reducing inequalities (as discussed above). Reliance on patient choice and voice may result in responsiveness to those who are most eloquent and demanding, at the expense of the vulnerable and needy. There has been much focus on the need for groups who are disadvantaged to have a voice, and on providers working to involve, and hear the voices of, ‘seldom-heard’ groups. Second, a distinction can be made within this literature between the notion that responsiveness can be defined as the extent to which providers engage with patients and enable choice and voice, and the view that patient involvement and engagement is a key means of achieving or improving responsiveness (by helping providers better understand patient characteristics and needs, particularly those of groups who are disadvantaged). Third, this suggests that measures of responsiveness should assess aspects of patient choice and engagement, and the extent and quality of dialogue between providers and patients.

Recruitment

Semistructured interviews were conducted with members of staff recruited from 13 PCOs in the East Midlands of England (see Table 1). Purposive sampling ensured that a range of PCO staff (management, clinical and reception) were involved. Staff were approached directly by a study researcher, with permission from a senior member of staff, or were approached by our main contact at the PCO. Two members of staff were invited to be interviewed at each PCO. One PCO was a practice managed by a large national company; here, we invited a member of staff at the head office, in addition to the practice staff.

Interviews

Most of the interviews were conducted by EA or JW on PCO premises, and one was conducted over the telephone. A topic guide was developed and used flexibly to guide the conduct of the interviews. Participants were asked to give their views on what responsiveness meant to them and their PCO, how responsiveness was enacted in their PCO, and successes and difficulties they had faced in their attempts to be responsive to patients’ needs and preferences. The topic guide for the GP surgery and walk-in centre staff is included in Appendix 3; this was adapted slightly for pharmacy staff. Interviews were digitally recorded and transcribed verbatim.

Analysis

Analysis progressed alongside the interviews using NVivo to organise the data. Data were analysed using a combination of framework analysis83 and the constant comparative approach, which involved both deductive and inductive elements. An initial coding frame was generated from the research questions and focused on the meaning of responsiveness, examples of implementation, meeting the needs of different groups, and barriers and facilitators to providing a responsive service. We used this coding frame to map staff descriptions of the strategies available to them to develop responsive primary care services. The coding frame acted to guide and bound the analysis of staff interviews to ensure a focus specifically on ‘responsiveness’, not just broad experiences of primary care (e.g. ensuring that the focus remained on ‘non-clinical’ aspects of care). However, we were careful to ensure that the coding frame did not unduly constrain the analysis; themes and subthemes were added and iteratively revised as additional interviews were coded. As such, themes were tested and developed inductively to produce a description of the strategies PCOs use to be responsive, grounded in staff’s views and experiences. In interpreting the data, we drew on concepts around market orientation and customer orientation developed in the marketing literature. 84,85

In the initial part of the findings, we focus on data from the GP practices; we reflect on similarities and differences in the accounts of walk-in centre and pharmacy staff in a separate section.

General practitioner practices

What approaches do primary care organisations use to make their services more ‘responsive’ to the needs of patients?

Staff described various activities undertaken at their PCO to improve the extent to which they could meet the needs of their patients. From these descriptions we developed a typology of approaches to providing a responsive primary care service. We found that approaches fell into three distinct categories that varied in the extent to which they were proactive or reactive, and the extent to which they were focused at the level of the practice population or the individual patient. These categories were (1) proactive population orientated approaches (approaches that involved PCOs planning ahead to design systems and services around the anticipated needs of the different groups of patients that used their services); (2) reactive population orientated (attempts to elicit and respond to patients’ expressed needs, feedback and complaints); and (3) individual patient orientated (providing flexibility and support at the level of the individual patient), which is also primarily reactive. These themes are summarised in Figure 5.

FIGURE 5.

Strategies for responsiveness: themes.

Proactive population-orientated strategies

One prominent theme in staff accounts of strategies for responsiveness reflected the attempts of practices to develop an awareness of the diverse patient groups they served and their specific needs, and to proactively design and organise their services so that they could respond to the needs of specific patient groups.

It’s looking at different groups of people and trying to work out what they want and need, and looking at ways to provide that, . . . it’s trying to find ways to firstly kind of identify the problems and then work out ways to change things.

S15, GP

A minority of practices had developed an understanding of their patient populations through formal mapping, but the majority described their understanding as based on informal staff knowledge of the local patient population. (The issue of practices’ awareness of the characteristics of their patient population is explored in more detail in Chapter 8. )

[Site 1] has a lot of Muslim patients and South Asian patients, [site 2] has some as well but less . . . a lot of Polish patients as well. I don’t know about our Afro-Caribbean population, no idea. There definitely are some but I have no idea what proportion we would have.

S02, GP

The steps that practices took to proactively plan for and support the needs of diverse groups included adapting the physical environment and technology, providing interpreters and other communication support, staff training on diversity, considering the design of appointment and booking systems, and using tailored approaches to contacting specific groups of patients and delivering services to meet their needs.

The physical environment was talked about commonly in the context of how improvements to the building had been made in ways that met the needs of diverse groups. This was particularly in relation to patients with disabilities, although staff also talked about how consideration had been given to the way the environment could be set up to better meet the needs of other groups, including teenagers and families with young children.

One of the things when we had the building refurbished was improving disabled access, because we used to have a horrible desk where . . . the receptionist [was] standing behind a chest-high hatch with a grill, whereas we have got that low desk now.

S12, practice manager

The teenage population . . . for them, . . . as you come just inside the surgery, there is a huge notice board there and we put things there, because we feel that the teenagers coming in might not want to come fully into the surgery, they can just come into that first bit the foyer, see what they need to see and if anything there, does interest them, then they can come in further.

S14, receptionist

Staff also talked about provision for communication support for specific patient groups, including non-English speakers and patients with a sensory impairment:

We have got quite a lot of patients who don’t speak English as their first language. So things like the screen that patients use to book in when they arrive at the surgery: . . . if they speak a different language then that is all written in the language that they speak.

S15, GP

Staff training in diversity issues, and in supporting the specific needs of specific patient groups, was also seen as critical to providing a responsive service.

I think probably [we need] more training and awareness for staff . . . It’s maybe that education about the range of issues that may be faced from mental disabilities, visual impairment, hearing impairment, physical disabilities . . . to make sure that we try and consider things from the perspectives of every one of our patients.

S27, head office

For some groups of patients, the key issue was seen to be planning to enable diverse patient groups to access the services, particularly groups of patients who would be disadvantaged by standard opening times and appointment systems. Staff described ways that their practice had adapted the systems for appointments around the needs of these patient groups.

We have obviously got some surgeries that support care homes, you know, you have got people that have mobility issues, so we tend to do regular GP visits, . . . we will say the GP is going to be here on a Monday afternoon and a Thursday afternoon. [. . .] So we do try to accommodate the needs that we have.

S27, head office

Many staff recognised that tailored approaches could be required to help patients from disadvantaged groups to access services and have their needs met. Being responsive often involved practices ‘doing things differently’ in the way that they communicated and engaged with patients.

We have got a few [patients] in [location] that couldn’t read or write. So we were sending out absolutely loads of letters, for all chronic diseases, for immunisations, follow-ups for smears. Our postage was extortionate, but I think we were having no results [. . .]. So then we started to actually telephone, to actually ring them up . . . and that was quite successful.

S03, practice manager

We do the learning disability health checks here . . . and they gave us literature for people with learning disabilities and told us how to set things out, for somebody with a learning disability . . . Just sending them a standard letter, they are going to think ‘oh no’, but this had pictures and things on it so that was very useful.

S25, practice manager

In some practices, initiatives that had been developed to help improve the services or support provided to specific groups within the practice population. Setting up registers of patients with specific needs could help practices to identify patients who might benefit from additional support and to direct them to appropriate services.

Initially I set up the carers register in [the practice computer system], well there was one there [already], but updated it, [I] . . . now we have a specific carer support worker, so any GP can refer to [her]. She comes in every Wednesday and she takes it from there. [She] contacts the carers, and basically they have an assessment, she lets them know what the benefits are due to them, [and] where they can go for support.

S20, practice manager

There were examples of practices going ‘one step further’ in their efforts to meet the needs of their patients: actively working with local community to engage ‘seldom-heard’ patients in using services.

We have worked very closely with the PCT and the local community to manage the Romany population. [. . .] What we agreed was that every Wednesday morning we would run a clinic where they could just walk in and they would be seen, and we would have two translators there, and then the community leaders spread that round the community so that everybody knows. So that worked so well.

S27, head office

Overall, being responsive through proactive planning was not a discrete task but a process, involving the ongoing monitoring by practice staff of the changing needs of their practice population, and the willingness to proactively seek solutions to these problems.

Recently I have noticed about the Slovakian people . . . there is language assistance problem, so [we are working towards getting] the language assistance.

S26, receptionist

Reactive population-orientated strategies

The second theme that emerged related to attempts to listen to, involve and engage with patients, and to react to their expressed priorities and needs. In contrast to the first theme, this approach to responsiveness was one which was reactive – being led by the expressed needs and priorities of patients.

Staff felt that listening to and involving patients was important in ensuring that they were aware of the needs of their patient population, and how best to address them. Many staff felt that responsiveness involved taking patient feedback seriously, and being willing to make changes, and try new things, to address the issues that were important to patients.

We are very open here, I mean we are as a practice very much for change, we don’t just say ‘well we will do it this way and that’s it’, we look at ways. And if something doesn’t work, or if we get a complaint, or we get a suggestion we do look at it and see if we can act upon it, and we discuss it. So we are very open to change and we are changing all the time really to try and improve services.

S03, practice manager

Practices used a range of processes to listen to patients. These included collecting and acting on patient survey data, suggestions and complaints, and involving patients through a PPG or engagement with the local community.

We have worked hard with the patient participation group in the practice, we run every 2 months, and they have been great in two ways – one is telling us how they have observed our practice and what other people have been saying so they are giving us feedback. [. . .] The other thing, the patient participation group just gives us ideas about how we can keep improving.

S09, GP

In a special case of a GP service for homeless patients, a GP cited how important it was to get feedback, not only from patients themselves but also from partner organisations that also worked with the same patient group.

We have learnt a lot over the last 10 years about being responsive to the needs of our patient group. So we try to listen very hard, to what their needs are. We have a multidisciplinary meeting every week with our partner agencies, with the hostel staff and the mental health team and the drug workers, so information gets shared and fed back at that point.

S10, GP

Acting on patient survey data or patient feedback was seen to be a key element of responsiveness, but not always sufficient in itself. Another feature of providing a responsive service was described in terms of ‘closing the loop’ – finding ways of communicating to patients about how changes would (or would not) be made in response to their comments or complaints. This was seen as important not only in ensuring that patients felt that the practice was listening to them, but also so that patients could get to know about changes that had been made to better meet their needs.

We have introduced [appointment] the system, it’s currently working, we need to advertise it a bit more, we need to get the patient participation groups more involved so they are advocating it. [. . .] We have got leaflets that we can hand out to [patients]. Whenever I get a complaint that says ‘can’t get through’, I immediately use [leaflets] to show them this new service.

S01, practice manager

This element of responsiveness, therefore, involved engaging in dialogue with patients about their needs and the practice’s response to these needs.

Patient-orientated strategies

While the first two themes include approaches that are oriented towards the patient population as a whole, or specific groups of patients, staff also commonly talked about their efforts to be responsive in terms of meeting the needs of individual patients at the point of delivery. For many staff, responsiveness was most evident at the point at which individual patients accessed the service.

You respond individually to that person that is in front of you . . . depending on their needs.

S21, pharmacist

Responsive service delivery was seen to be dependent on attitudes and behaviour of staff (particularly reception staff) towards patients; willingness to operate systems flexibly around the needs of patients; and the approach of the practice towards helping and supporting patients who had higher levels of need or faced particular challenges in accessing services.

Delivering a responsive service was seen to depend heavily on the extent to which a practice had a patient-oriented philosophy. This was most important in relation to the way that reception staff treated patients. Interviewees commonly emphasised the need for reception staff to be friendly, approachable and helpful towards patients, to treat patients as individuals, and to be sensitive to their needs and circumstances. They felt staff needed a willingness to help and to support patients who faced difficulties in using the service.

I think just overall it’s just a positive approach to patients . . . what we have got to remember, when these patients are ringing up, no matter how cross they are, or frustrated or confused, or even happy, we have got to remember we are here to help and we have to do everything that we can, within our capabilities, to help them.

S04, practice manager

We have a large elderly population . . . and you know we are just very aware and more helpful to the elderly.

S14, receptionist

Interviewees emphasised the role of reception staff as gatekeepers, in terms of managing demand on the service, but felt that reception staff needed to be willing to work with patients to help find the most appropriate solution for them within the constraints of the system.

We don’t ever say ‘we can’t do it’, see you have got to try and find a solution for the patient’s needs at that time. Because you know, that is what you are here for, to provide a service.

S19, receptionist

Responsiveness was reflected in a willingness to operate systems flexibly around the needs of patients, particularly patients from groups who are disadvantaged who may struggle to access the services they needed.

We don’t take prescription requests over the telephone, [but] if a patient is real sick, like say for cancer [patients] or elderly can’t get out, we do take them over the telephone, so that is quite responsive.

S06, practice manager

Staff highlighted the value of systems that alerted them to individual patients who had particular needs, so that these needs could be accounted for at the point at which the patient accessed the service.

You can put alerts on the patient home screen and you can put it in order of severity . . . On the front screen, you can have an alert. For example if someone you know is extremely unwell with mental health issues, or they are Down’s or anything like that, an alert can pop up on the screen to say ‘please accommodate where possible’.

S20, practice manager

Walk-in centres

Staff at the walk-in centre shared many of the views of staff at the GP surgeries. Responsiveness, as interpreted by walk-in centre staff, is responding to what patients want.

. . . basically providing the service that the patients want and need.

S18, nurse

It means answering or dealing with their problem as quickly as you can, and as appropriately as you can.

S19, receptionist

Because of the way walk-in centres are set up, staff felt that they contributed to the ability of primary care services across a locality to respond to patient needs, by offering patients the option of quick and easy access to an appointment. An additional benefit was seen to be that medical care could be provided to patient groups who might otherwise struggle to access mainstream general practice.

We are reacting to people who say ‘we want evening appointments.’ [. . .] If somebody rings up after 5 tonight then you know we have probably got two appointments on each doctor that we could book into. [. . .] so I think that is responding to a need of the patients . . . because people don’t work normal hours anymore.

S19, receptionist

Because of the nature of the walk-in anybody can walk in from anywhere . . . anyone can walk in. There have been a few travellers that just use the walk-in centre, because they will get treated and they will get seen.

S20, practice manager

Strategies for responsiveness described by walk-in centre staff tended to be reactive. The walk-in centre did not have policies for dealing with specific patient groups; staff felt that all patients were treated the same. They did, however, recognise the need to plan and to be prepared for patients who may have difficulties using the service, such as patients who do not speak English.

I think everybody is generally treated the same, I mean, people with disabilities have an equal access to anybody else. It’s harder sometimes, especially with the Polish population, because a lot of them really struggle with their English. To me they are making a really good effort, I take my hat off to them because I can’t speak another language, but I have an app on my iPhone that translates and I have used it a few times now and it’s cracking.

S18, nurse

There were particular challenges to providing a responsive service at the walk-in centre. Patient perceptions and expectations were seen as the main barriers (not understanding what nurse practitioners can offer, not wanting to use the touch-screen appointment system, or expecting to be able to see a particular doctor any time).

They want to see the doctor of their choice, at the time of their choice. Patients are still very uneducated about nurse practitioners, they still see a doctor [as] better than a nurse.

S20, practice manager

There were also practical considerations, such as maintaining the safety of women in refuges.

Women who live in refuge centres, that is difficult because they don’t have a postcode, it’s like a PO Box number and you have to go through a few avenues to get to them and only certain members of staff do because of the confidentiality side of things.

S20, practice manager

One member of staff spoke about the difficulties that arose at the GP surgery adjoined to the walk-in centre.

I mean ‘yes, in an ideal world you would be able to be seen now’, but where I work is not a walk-in centre. But they expect, I think, because there is a walk-in centre [next door], that everyone will perhaps work the same.

S18, nurse

Pharmacies

The pharmacy staff’s definition of responsiveness was more literal (akin to a dictionary definition) than that of staff at GP surgeries and walk-in centres. They felt that it was about the way in which and how quickly they responded to patients’ requests.

. . . a lot of it is responding immediately to phone calls or to patients wanting something or asking us something.

S21, pharmacist

Pharmacy staff enacted responsiveness in similar ways to GP surgeries. Their patient-orientated strategies included responding to the needs of customers on an individual basis: providing a speedy and efficient service at the customers’ convenience, such as delivering goods and services to homes; being open 365 days a year; stocking the products and brands that people wanted; or recognising that first-time mothers need reassurance or that visually-impaired customers may need extra help and time.

. . . we liked to think that we go that extra bit for the patients in the traditional [way], I mean we deliver if they want delivering and we do everything we can to make the patient happy.

S21, pharmacist

Well, as soon as you see them or hear [a blind person] coming through the door you speak to them across everybody else, so they know that they have been noticed and somebody will be with them, and then you help them to the counter. And then you just go through everything with whatever [medication] they want, you spend time showing them stuff and explaining how tops open and things.

S22, pharmacy staff

The business model appeared to affect staff’s views of responsiveness. While GP surgeries are also small businesses, their primary source of income is the NHS. Surgeries try to keep patient lists to a manageable size, and work towards meeting the needs of patients as they present at the surgery. Pharmacies, on the other hand, are in direct competition with other pharmacies: the key to attracting ‘customers’ (as opposed to patients) is the quality of customer service.

. . . the only way we can differentiate ourselves in this market is by customer service so we try and make sure that they always get a smile when they come in and just make them feel as if we genuinely care. It’s not just a case of, you know, bring them in and let’s show them the door as quickly as possible.

S23, pharmacy manager

Many members of GP surgery staff spoke about ‘bending over backwards’ for patients, but pharmacy staff took this even further.

So the opening hours are 9 a.m. ‘til 8.30 p.m. [. . .] but the key, the determining factor is 7 days a week, 365 days a year, so Christmas Day, Boxing Day, New Year’s Day . . . Yeah, bank holidays, never close, it never closes.

S23, pharmacy manager

Pharmacies’ reactive population-orientated strategies involved staff using customer surveys, approaching people who were obviously smokers to see if they wanted to attend the smoking cessation clinic, and looking out for patients who might need a medicines-use review. Staff were also aware of the characteristics of their potential customers, for example providing an emergency hormonal contraception service in an inner-city area with a large student population. Pharmacies wanted to know about the characteristics of their population and to gain feedback from customers, mainly to understand how to improve their customer service and to attract and retain customers.

. . . we obviously have our own annual customer survey that we do and generally, we’re always quite impressed with the results that we see so, it’s always good, I mean, you always get a few, couple of negative comments which you’ve got to take constructively and try and work on and try and improve upon the but general feedback is always well received.

S23, pharmacy manager

There were no obvious references in the pharmacy staff interviews to demonstrate proactive population strategies, such as surveying their populations to identify and quantify the various patient groups. However, there were a multiple examples where the pharmacies recognised that different groups had different needs and had proactively designed and organised their services to meet these needs, by providing cheap baby products, smoking cessation clinics, additional training to provide flu vaccines, health promotion initiatives for people with hypertension or diabetes, different processes for people with drug addiction, and additional staff.

. . . it’s all about prevention rather than cure, so we are trying to [focus] on health promotion activities and trying to get the public and the patients that we look after to a better place with any little bits of lifestyle advice that we can give, especially to the hypertensive or the diabetics. The smoke cessation clinics, that’s really done well out of this branch.

S23, pharmacy manager

Although the priority of pharmacies appeared to be speed and quality of service and providing for the customers’ wants and needs, there were limits. Staff recognised that people expected a quick service and to be able to buy whatever they want, but that there were limits to the speed at which a prescription could be filled and rules about what could be sold.

It’s not like a doctors’ system or the dentist system where it’s just inherent that there is an appointment. That’s often kind of the drawback in pharmacy, people feel that access should be immediate, they feel they shouldn’t have to wait, and sometimes they walk in with a big prescription, they want it yesterday.

S23, pharmacy manager

The challenges to pharmacies are quite different from those experienced by GP surgeries, mostly due to their business model. Smaller pharmacies have to compete with larger pharmacies, chains and supermarkets that have superior buying power and can offer cheaper products. PCT-led services can be at the mercy of budget cuts and are restricted to certain patients. Private health checks can be more flexible and responsive, but the customer has to pay.

. . . the supermarkets, the Boots, the likes of them, are able to offer things, they sometimes retail at prices that we’re buying at. It’s sheer muscle power that they have when they’re doing their procurement which, when we have a customer perhaps thinking ‘well I can get this cheaper in Asda’, it’s a frustration . . .

S23, pharmacy manager

Practical constraints for pharmacies included opening times that are not convenient for working people, lack of toilet facilities and wheelchair access, having time to do the extra things, and training required for certain jobs. Staffing could also be an issue because of the long opening hours.

Yes, sometimes you think, you know, oh perhaps there was a little bit more you could have done, but time is restricting you. You are just so busy.

S22, pharmacy staff

One of the difficulties of having a pharmacy that’s open all the hours is that you don’t always have the same staff working there. So one of the main difficulties we’ve found is . . . it always has been a challenge to try and make sure that the customer would get the same level of service whether they come on a Monday afternoon or a Sunday evening. That’s always been very difficult, you know, just trying to make sure that the staffing is right, not just in numbers but in terms of calibre.

S23, pharmacy manager

Pharmacy staff felt that they wanted to accommodate both needs and expectations as part of delivering a service that would attract and retain customers.

I suppose their day-to-day requirements, or if they have got an illness then they need something to sort that out, but the wants and needs are very, they are very close. And most of the time, would be the same, sometimes won’t.

S21, pharmacist

Attitudes and expectations

Being responsive requires tolerance and flexibility, and a willingness to help to support patients who were struggling to get their needs met. Notions of responsiveness became challenging when patients were perceived to be unduly demanding or aggressive.

I think they are quite demanding, certainly in the last few months the abuse that has taken place from patients to receptionists . . . is outrageous and I do think if you went to McDonalds, and you kicked off you would be asked to leave. [. . .] You get like a select few that are really quite hard to deal with.

S20, practice manager

While recognising the value of flexibility, and of staff going ‘beyond the call of duty’ to help vulnerable patients, staff were very aware of the potential negative consequences of this flexibility.

We ask people to allow 48 hours [for prescriptions] but there is always somebody who, with the best will in the world, realises that they have made a mistake and they have woken up and found they have got no tablets left, and we will do that on the day. But we actually found that that was happening so often, that we have had to clamp down . . ., try and educate people and make a note of the ones who keep on doing it, and just try and encourage them to organise their lives a bit better.

S12, practice manager

Staff were concerned that efforts to respond to patient needs could mean that excessive demands were put on staff. In one example, a member of staff felt that the PCO’s explicitly flexible attitude could have negative repercussions for her team, and needed to be brought into check.

As far as me and the little nursing team are concerned, I think we are generally [responsive] [ . . . but] you feel so put upon sometimes, you think ‘right, now I am putting my foot down’ and you just say ‘no, no, it can’t be done that way because we are not doing that, why should we miss our lunch break because of that?’

S18, nurse

Resources

Lack of funding, staffing and time due to internal and external pressures were commonly cited as barriers to a PCO’s ability to be more responsive to patients.

We will undertake looking after patients with substance misuse, we do people with alcohol, . . . not for the money but again we do it simply because it’s good patient care. [. . .] Really, sometimes it’s an awful lot of work and it can put pressure on top of you. . . . Unfortunately, the resources don’t always meet the need.

S03, practice manager

Practical constraints

The physical environment caused problems for some PCOs. Some staff felt that the building in which the PCO was hosted was unsuitable, and made it difficult for them to provide the accessibility and comfort that patients expected. Staff expressed frustration when they were unable to make improvements, such as providing extra services or additional opening hours, owing to practical constraints.

It’s an old building. So it is sometimes challenging to get around the building because we have got lots of floors and lots of sets of stairs, but there is a lift, there are little stair lifts up even just two or three stairs, and ramps and things as well. [. . .] It’s not an ideal building for disabled people but I think we hopefully try and get round that the best we can.

S15, GP

From a practical point of view . . . our waiting area is shared with another service; when they close, we can’t, at the moment, open [. . .]. That poses us with a problem.

S10, GP

Internal and external context

The ability of PCOs to be responsive to their population needs was seen to be influenced by features of the organisation and the population they served. In particular, staff felt that, in larger practices, it could be more difficult to respond to the needs of individual patients at the point of service, and to respond to patient preferences for continuity of care.

[Our patients] would probably say we are growing too much, too fast, they are not seen as patients any more, they are seen as numbers, and we are not offering the amount of routine appointments with their own GPs, this is the biggest thing – it’s the patient’s GP that they want to see.

S20, practice manager

Several barriers external to the PCO were identified by staff. Curiously, only one staff member referred to the NHS restructuring that was taking place during 2011 in the wake of the change of government. More commonly mentioned was the plethora of bureaucratic rules by which PCOs are expected to abide. These could take the form of targets or actions dictated by the (former) PCTs or the wider government through locally or centrally enforced policy objectives, some of which were felt to be inappropriate for local populations.

Time is the only [thing], I would say time and pressure of work. Deadlines . . . working to, jumping through hoops sometimes for the PCT. [. . .] When you have got all of those other pressures and things that you have to do, you can’t always do what you want to do?

S03, practice manager

In some cases, a lack of support from organisations higher up the chain of command was felt to be obstructive.

The health authority [are] a very difficult bunch to deal with. [They] are asking me to [do something that is] just highly impossible. So from that I don’t feel like I have got much support.

S20, practice manager

Use of data from existing patient experience surveys

Staff at the GP practices used various sources of patient feedback to learn about issues that patients considered important. The most common source was the GP patient survey. Some staff found these data useful for identifying problems common to most patients, such as not being able to get through by telephone or not being able to make an appointment with a preferred GP.

I think it’s important that we get the patient’s point of view, and what they want from the surgery, so that we can work towards doing what they want to, if it’s feasibly possible. Yes and [the survey feedback] shows us areas where we need to work on, again one thing that always comes up is the access to the phones, and we are looking into getting extra lines put in, and having a direct service so you get directed to different places for different issues or queries that you have got.

S04, practice manager

Questionnaires about specific elements of the service were also used. Staff found these useful, especially if the results showed that patients’ reactions were positive.

We did have, for the extended hours there was a questionnaire asking, you know, ‘would you use this service again?’ ‘Did you get an appointment [and] in what time scale?’ . . . ‘Were you happy with what happened, by coming to a different surgery?’ There were questionnaires about that, which seem to be quite positive, you know.

S19, receptionist

Although the national GP patient survey was used in many of the practices, most of the staff interviewees identified problems in using the data. The response rate was consistently low, and only certain patient groups returned their survey, leading to ‘biased’ results. In addition, some staff were concerned that this lack of representativeness may lead to the ‘seldom-heard’ being excluded, such as people with low literacy, non-English speakers, people with dyslexia, homeless people and older people. In some cases, these were the people from whom staff wished to hear. On the other hand, some staff were concerned that the surveys were also sent to people from whom they did not wish to hear, such as patients who have not used the services recently.

We don’t see much meaning in them because our response rate from our practice is about 15/20% and we know that patients who are more likely to complain are more likely to fill them in. So I am not sure whether that’s the representative sample that they are capturing. [. . .] I really don’t know how one could capture a representative sample because the response rates are so low.

S09, GP

I think they are given out to patients who may not have accessed the practice anyway. We know for a fact [that someone known to us] got one and he hasn’t been near a GP in several years. How is he supposed to know whether he feel the opening hours are appropriate [. . .]. Their answers will be: ‘it’s a shame they don’t offer evening appointments’ yes, we do. It will be: ‘a shame they close on Thursdays’, which we don’t . . .

S02, GP

In addition to low response rates and lack of representativeness, the questions that were asked in the national survey caused concern among staff. Some felt that the questions were difficult to understand and that the survey was too long. Others were aware that sending questionnaires by post was not necessarily the most effective way of reaching some patients.

We have got the national survey but I think that’s rubbish. [. . .] Because as I say I think the questions are hard to understand.

S02, GP

The survey, the mechanism [by] which [it] is delivered is not helpful for this population, it’s by post. I don’t know what percentage but I am guessing 50% of our patients use this building as their c/o address. So we simply receive the questionnaire, which sits in our reception waiting for us to see the patient and hand it out, so a significant reduction is made in the number of people who even receive their questionnaire. And then it’s a postal measure, and I know that our numbers for returns are very low. The quality of the information received or the positive feedback that we get seems to be very good but I know that the numbers must be a handful really.

S10, GP

Some members of staff felt that existing questionnaires were worded in a leading way and that questionnaires should be solution-based rather than problem-based.

I think specific [questions] can sometimes lead to, I guess, coercion sometimes, can’t it. Yes I would like a very open ‘how do you feel?’ Very, it’s very qualitative, there is no exact measure. [. . .] I would like to see some encouraging solutions, based on responses, as opposed to ‘this is a problem’. I guess if you get presented with a number of problems it’s actually very difficult to accommodate that, whereas if you say ‘actually that sounds like a very good solution to a problem’.

S17, GP

The delay in receiving the results was also considered a problem by staff. Staff received not the raw data but, rather, aggregated feedback, often with targets to reach in response to the issues raised. While this may be useful for incentive-based systems such as Quality and Outcomes Framework (QOF), the delayed nature of the feedback prevented staff from making changes to practice in real time.

For us the national patient survey has got a real time lag on it obviously, because it takes a long time to collate all the information, so it’s always kind of 6 months out of date, and it’s rolling averages, and it can be a bit difficult to extract very quick changes.

S27, head office

So the annual patient survey . . . you get points you know they give you a percentage. . . . I think there is a target, you know, what’s acceptable, sort of 90% of patients and you know, how you measure up. I mean I think we usually came either round about just over, or with the odd thing just slightly under, so we were obviously sort of more or less on target for that. But there hasn’t been, there is nothing that I am aware of recently to give us any sense of how we are doing, other than feedback from patients, which by its nature tends to be negative.

S12, practice manager

Other sources of feedback about patients’ experiences and views

The most common sources of patient feedback that practices reported using were informal and in real time. This included ad hoc feedback from individual patients (n = 1), through comments forms and boxes (n = 5), monitoring complaints received (n = 17) or not received (n = 2), exit questionnaires and postcards (n = 4), websites (including NHS Choices, n = 4) and the PPG (n = 17).

Patient participation groups were a popular source of feedback. Staff tended to use feedback from PPGs in a similar way to survey results, identifying issues upon which staff could act. However, staff expressed concern that membership of such groups was likely to be ‘biased’ and may not include patients who are hard to reach.

. . . having spoken to the PPG they would like to see continuity from the partners and are happy to see any doctor for the urgent stuff so we are hoping to try and move towards a model like that.

S02, GP

It’s mostly elderly that come [to the PPG meetings] I am trying to get somebody that is young with a baby but that hasn’t happened as yet.

S03, practice manager

We tend not to get a decent representation of the practice. Often these sorts of things, there tends to be a little bit of bias, I find, from the type of patient who will come to that type of group. And so in reality that tends to be unproductive.

S17, GP

Complaints were also a common source of feedback, through either not receiving them (no action required by the practice) or receiving them (action may be required by the practice or the issue may be resolved on the spot). Complaints were submitted in writing, verbally or via the NHS Choices websites.

We are not getting any complaints anymore, so [laughs] that’s good.

S25, practice manager

There is a complaint form but that obviously goes to the managers. So if a patient does want to complain we always tell them to either write, we always get them to write in, erm and it is dealt from there. [. . .] if it’s something significant, and that needs to be improved then yes it does, be informed in the [team] meetings.

S16, receptionist

Comments forms and suggestion boxes were also used by staff, as trends could be identified and acted upon.

I think the comments box in reception can sometimes not be where they would like to leave their comments. It’s not about a blame culture at all, it’s about getting some honest feedback, and you know you can see when trends start developing, that you can work with those staff to manage their performance and to give them the training that they need, but that means patients can be very honest about their feedback which is really valuable for us.

S27, head office

Ad hoc feedback from individual patients could also be a useful source of evidence that there may be an issue that needs addressing, bearing in mind the bias that this may introduce.

. . . the fairly informal feedback that you get from patients as they come into the room tends to be fairly actually quite useful, as in: ‘I have struggled to get an appointment this week’ and then if you hear that on a number of occasions, that tends to, I guess, bear some sort of interest in your mind that actually there is perhaps, perhaps more of a problem here than one person’s particular opinion. So I guess you have got, it is important to listen to individual opinions . . . but again sometimes individual opinions do have a little bit of bias and they do have different agenda.

S17, GP

Question topics

Fifteen members of staff gave advice about the topics we should ask patients about. Five members of staff wanted to know how patients felt about the service they provided, whether or not they were happy, specifics about what the PCO was doing right or wrong, and whether or not patients felt they were receiving a quality and timely service. Two members of staff suggested that it would be most useful if patients could explain why they felt that the service was poor, rather than ticking yes/no boxes. Four members of staff would be interested to hear how patients might suggest how to improve the system and what changes they would make to achieve this improvement.

I would ask them what they thought of the service, how we could improve the service, why, have they got examples of what we have not done right, because you need to get their perception of things. So I would delve into more deeply I suppose, why you don’t think we achieve that, you know.

S03, practice manager

Some members of staff cited particular areas of interest that they would like to see on the questionnaire, such as the telephone system (n = 4), the appointment system (for instance how soon and far ahead you can book) (n = 6), opening times (n = 2), receptionists (n = 2), information provision (such as how easy it is to get information and advice, whether or not staff get back to patients if they do not know the answer) (n = 2), and accessibility of the building for people with physical disabilities (n = 1).

To see are they are happy, I suppose, with the practice, really the main thing is as long as they are happy we are happy as such. But, if they can get the appointments and things like that, that’s really all it is.

S05, receptionist

Three members of staff suggested that the questionnaire should include questions about the clinical consultation, perhaps due to a misunderstanding of responsiveness. Another two interviewees were keen that the clinical aspects should be separated out.

Some staff felt that it was important that the questionnaire included ‘something different’, beyond appointments and telephone systems. Suggestions included what patients thought about backroom staff and the referrals process, how well complaints are handled, how services had been improved in recent times, and how service reconfigurations such as practice mergers had affected patients.

But [I’d] like to come up with a question that is a bit more, that we can get explanation from, not ‘I can’t see my doctor in 3 days, I was on the phone for 20 minutes’. [. . .] I would like to see how they, how they felt about backroom staff, if they have ever had any dealings with, I don’t know, their referrals, how did that go, you know ‘have you ever complained?’ . . . It would be nice to ask if they feel something has improved, if they feel the service has improved . . .

S20, practice manager

One member of staff also felt that it was important to include customer-focused questions – asking the ‘awkward’ questions, the answers to which might prove challenging for the PCO to respond to.

Two interviewees, both practice managers with a background in business, felt that it was more important to measure responsiveness objectively through systems and procedures as well as seeking the patients’ views.

Question wording

Twelve members of staff provided advice about how the questions might be worded. Seven were keen to stress the importance of open-ended questions, to allow patients to express their views in their own words to create a rich source of data to inform service improvement. Four wanted the questions to be solution-focused – how can the service be changed to make it better – rather than simply asking patients what they like and don’t like.

. . . how would you change it, how would you change the appointment system, how would you change you know the phone queuing system. Yes, do that. Definitely do that, I would like to see what they say.

S20, practice manager

The ability to measure change over time was important to three members of staff.

something that can measure change, that, you know, if services are changing you could see a change, it was responsive to change, in terms of you can measure it and the measure would be valid, that there has been an improvement.

S09, GP

Answer formats

As well as the wording of the questions, 11 members of staff referred to the format of the answers that patients could give. Eight members of staff spoke about tick boxes and six about yes/no questions. Five members of staff felt that closed questions were of little value and encouraged ‘satisficing’. In contrast, four members of staff preferred closed questions as they were simple for patients to complete and less work for staff to interpret. Two members of staff had a mixed response to closed questions – they were easy to analyse and prevented difficult issues arising, such as suggestions that would not be implementable, but were limited in usefulness.

Tick boxes and yes/nos are useful, but actually there is going to need to be some space within it, for people to flesh out what their experience has been and/or what their issue has been.

S10, GP

Data analysis

Two members of staff talked about the importance of data analysis: being able to feed back to specific members of their team and being able to provide feedback to patients quickly. Staff also felt that comparative national data would be valuable.

I think the time frame of getting the responses needs to be turned around very quickly, that it’s not 3 months later you are addressing a problem, you are actually addressing it there and then, and then the patients feel like ’I have made a comment and someone has listened to me’.

S27, head office

I think some information about national averages should be provided.

S07, practice manager

Participants and format

Nine members of staff provided advice on participants and format. Seven interviewees were keen that the responsiveness questionnaire be accessible to particular patient groups, including those with dyslexia (n = 1), those with low levels of literacy (n = 2), non-English speakers (n = 4) and older people (n = 1). Four members of staff talked more generally about ensuring that a PCO targets the appropriate patients.

I think trying to get the populations involved, that you want to hear from. So trying to engage with people with learning disabilities and their carers, and people who don’t speak the language . . . trying to get those people who maybe wouldn’t put themselves forward or, maybe, you know, would have difficulties in communicating with you trying to get their views.

S15, GP

Six members of staff were concerned about the format of the questionnaire. They wanted it to be accessible (n = 3), anonymous (n = 1), short (n = 3) and tailored for different patient groups (n = 1). Two interviewees hoped that a simple format would help with analysis.

It couldn’t just be in the written form, because it would have to either, well there would have to be different options, I don’t think one size would fit all.

S15, GP

Mode of delivery

The mode of delivery was important to 18 interviewees. The modes of delivery they discussed included using technology, such as the booking-in touch-screens or hand-held tablets (n = 10), paper-based by post or in the PCO building (most of these acknowledged the difficulties with this method) (n = 6); online (n = 5); face to face (n = 5); providing help with completion (n = 4); using lay interviewers (n = 2); offering incentives (n = 2); and using patient groups to help deliver it in the PCO (n = 1).

There is no reason why you can’t have machines in the reception area where the patients can fill it in. Or a tablet you give out, they fill it in, give it to the next patient.

S09, GP

Recruitment

A diverse group of patients and patient representatives were recruited from 12 PCOs in the East Midlands of England, comprising eight GP practices with varying characteristics, a primary care walk-in centre, two pharmacies and a specialist GP practice for homeless patients (see Table 1). Patients were recruited from only 12 of the 13 practices from which we recruited staff, as one of the practices (PCO1) withdrew owing to staffing changes and time pressure.

In mainstream GP practices, random samples of adult patients stratified by age were drawn from practice lists and contacted by post and patients who expressed an interest were purposively sampled for interview. In other PCOs, the receptionist or counter assistant introduced the study to consecutively attending patients and a study researcher approached those who expressed an interest in person. To ensure the inclusion of patients from ‘seldom-heard’ groups and ‘groups who are disadvantaged’, targeted mailings to carers and people with sensory disabilities were conducted in three PCOs; we also recruited via a BME group charity, a learning disabilities specialist and an organisation for travelling families.

Interviews

Semistructured interviews were conducted by EA or JW, in the participant’s own home (34 people) or on PCO (n = 18) or local organisation (n = 6) premises. All bar one patient contacted by letter chose to be interviewed at home and all patients recruited in the homeless GP practice, walk-in centre and pharmacies chose to be interviewed on the PCO premises. The travelling women and the BME patients requiring an interpreter chose to be interviewed in the local support organisation premises. Some interviews included more than one participant (at the participants’ request), for example a husband and wife, a mother and daughter, and a group of travelling women. All were counted as individual participants in reporting the total number of participants. Six stakeholders were recruited to represent the views of particular groups of patients, based on their knowledge, experiences and priorities of these patient groups, representing carers (n = 2), people with learning difficulties (n = 2), BME patients (n = 1) and travellers (n = 1).

A topic guide was developed and used flexibly to guide the conduct of the interviews. The topic guide included open-ended questions about participants’ recent experiences of non-clinical aspects of care from PCOs, and about ways in which services did or did not respond well to their needs. We included prompts derived from the literature and policy documents on responsiveness in primary care. The topic guide was adapted slightly for different patient groups (e.g. homeless patients) and for different PCOs. The core topic guide is included in Appendix 4. Interviews were digitally recorded and transcribed verbatim.

Analysis

Data were analysed using a modified version of framework analysis, which involved both deductive and inductive elements. 83 Themes identified from a review of English policy documents acted as an initial organising framework for the analysis. Our initial analysis focused on identifying core themes relating to responsiveness of GP practices. We used the thematic framework to code patients’ accounts relating to GP practices using NVivo. The framework acted to guide and bound the analysis of patient interviews to ensure a focus specifically on ‘responsiveness’, not just broad experiences of primary care (e.g. ensuring that the focus remained on ‘non-clinical’ aspects of care). However, we were careful to ensure that the initial framework did not unduly constrain the analysis, in line with the recognition that the elements of patients’ experience of responsive service might not mirror the elements required to deliver responsive service. Themes and subthemes were added and iteratively revised during coding. Themes were tested and developed inductively to produce a description of responsiveness of GP practices grounded in patients’ experiences. Following the initial analysis, we used the themes generated from the analysis of data relating to GP practices to code data relating to pharmacies and walk-in centres, while ensuring that we were sensitive to identifying divergent issues.

Attributes of responsiveness for general practitioner practices

We identified five key attributes of the organisation and delivery of primary care that patients directly experienced, and that impacted on their perceptions of how well the GP practice responded to their needs. These were ease of access to services; the alignment of the physical environment with patient needs; diversity awareness and cultural alignment; staff attitudes and behaviours; and co-ordination and support for their ongoing care. Although issues relating to keeping patients informed, and patient involvement and engagement, are prominent in recommendations for the delivery of responsive primary care, these elements featured less prominently in patients’ accounts of their experiences of responsive primary care.

These five key attributes were common across all patient groups, and across different primary care providers, but certain groups were more likely to have negative experiences of certain dimensions of care.

Attributes of responsiveness for other primary care organisations (walk-in centres and pharmacies)

The key attributes of responsiveness identified from the analysis of data relating to GP practices mapped well onto the findings on patients’ experiences of responsiveness from the walk-in centre and from pharmacies. In Tables 5 and 6, we summarise similarities and differences across the PCO types organised by theme, and provide illustrative quotes.

Methods

We conducted three focus groups with patients from diverse groups to test the initial set of questions. We wanted to access the views of patients from diverse groups to ensure that the questions were relevant to their specific needs and experiences, and to enable us to test the questions with patients who are traditionally ‘seldom heard’,87 to ensure that our questionnaire included issues that were important for these patient groups. We chose to hold one group with patients who were members of a PPG at a large inner-city practice in a deprived locality (seven participants), one with individuals from a BME background with few or no English-speaking skills (Gujarati-speakers) (10 participants) and one with people with physical disabilities (eight participants). PPG members were recruited by a postal invitation issued by the GP practice, accompanied by a visit by the researcher to a PPG meeting to introduce the study and answer questions. Patients who did not speak English, or had limited English, were recruited through information leaflets (translated into Gujarati) distributed to learners at a community IT education project. The focus group involving patients with physical disabilities was held at a social group set up by a local charity.

Focus groups were facilitated by JW, with EA in attendance to take notes and act as scribe, using a flipchart to display the main points. The focus group with Gujarati speakers was jointly facilitated by JW and an interpreter from the community IT project. The topic guide is given in Appendix 5. Groups lasted between 1 hour 10 minutes and 2 hours 8 minutes, and were held at locations convenient for the participants. With participants’ permission, the discussions by each group were digitally recorded and transcribed verbatim. A framework approach83 was used for the analysis of the data. A coding framework was developed in NVivo by EA, based on notes taken during the focus groups. The framework described each question in terms of issues that participants talked about when answering the questions, and misunderstandings and suggestions for improvement to wording. Verbatim transcripts were imported and coded to the framework. The coded data were summarised into a chart.

Results

Revisions to the questions

Key decisions made at this stage included (1) the decision to include a larger number of more detailed questions in the questionnaire – a common theme across focus groups was that patients felt that some of the high-level questions were too abstract, and it was clear from inspection of the topics discussed during the focus group that the question wording for some questions was not cueing patients effectively to consider the issues that we expected; and (2) the decision to frame questions by asking patients about the ‘last time’ they visited the GP, to overcome the problem of patients struggling to answer a general question relating to making an appointment (they felt that their experiences depended on what they wanted on a particular occasion, such as to see their usual GP or to be seen quickly).

We consulted widely used and validated primary care patient experience/satisfaction questionnaires to identify relevant questions and question wording to inform development of questions for the first draft of the responsiveness questionnaire. 88–91 During the process of drafting the initial version of the questionnaire, the advisory group were consulted for feedback on the wording and ease of use, and to gain insight into the appropriateness of the pilot version for different patient groups. Feedback was collated and version 1 of the questionnaire was developed for testing in the first pilot.

Questions on demographics were developed, drawing on other validated measures where possible, including the national GP patient survey (questions on sex, age, occupation, parent/carer status, sign language, sexuality), the General Practice Assessment Questionnaire (question on ethnicity), and the Scottish Patient Experience Survey (question on disability).

Version 1 related to GP surgeries only, consistent with our approach, which was to develop a GP version of the questionnaire which could then be adapted for other primary care providers. Version 1 of the questionnaire is included in Appendix 7. Version 1 was produced in paper and online (using SurveyMonkey: www.surveymonkey.com) formats.

Methods

The first pilot took place over 3 months (June–August 2012). Three mainstream GP practices were chosen for the first pilot (one in a central area of a large town, another in a deprived suburban area of a town, and the third in a small village in rural surroundings). Practices were asked to hand out 150 patient packs to consecutively attending patients coming into the surgeries during a 2-week period. Patients could return their questionnaire to a box on the reception desk, or directly to the research team using a freepost envelope. The questionnaire included a website address for patients to complete the survey online, if they preferred.

Patients were asked to return a reply slip with the questionnaire if they were willing to be interviewed. On receipt of the reply slip (or e-mail for the online version), participants were contacted by telephone by the interviewer and a date was arranged with patients who were agreeable: patients were offered the choice to be interviewed either in their own home or at their GP premises. A topic guide was used in interviews (see Appendix 8) to elicit patients’ accounts of their recent encounters with their GP practice. It included direct questions about the content of the questionnaire: whether or not patients felt that the questions allowed them to articulate their views; whether any questions were missing or redundant; and how easy it was to complete. Interviews were recorded on a digital recorder and transcribed verbatim.

Interview transcripts were uploaded into NVivo software and coded according to a predefined coding framework which included the broad themes of the questionnaire. Additional codes were generated for issues that did not fit this framework, and for issues relating to problems and suggestions for improvement. A validation exercise was conducted in which the coded text from the nine randomly selected interviews was compared against the list of questions in the questionnaire by EA and CT. An approach based on content analysis was used to identify whether or not each theme was discussed by each individual patient, and to identify any additional themes. We collated examples of the issues people discussed within these themes, and the terms that were commonly used to describe these issues, to inform refinements to the wording of the questionnaire. The additional transcripts were read to ensure that no further issues arose. Across the whole sample, we also explored participants’ expressed views on the wording, content and format of the questionnaire.

This analysis from the interviews was used in conjunction with analysis of descriptive statistics from the pilot survey to develop the next iteration of the questionnaire. The draft was circulated to members of the team for feedback, and the content and format of version 2 was agreed.

Results

Revisions to questions: changes from the first to the second pilot

The changes suggested by patients were considered along with the distribution of responses from the survey, tables of correlations between questions, and the extent to which questions discriminated between practices. The following changes were made.

Developing version 1 of the pharmacy questionnaire

A first version of a parallel questionnaire for use by pharmacies was developed for testing in pilot 2. This mirrored the GP questionnaire, but was adapted to make it appropriate for use in pharmacies, informed by the findings of the interviews in stage 1. The key differences between the pharmacy version 1 and the parallel version 2 GP questionnaire are summarised in Table 14.

Version 1 of the pharmacy questionnaire is also shown in Appendix 10.

Administering the questionnaire: feedback from practices and consultation about alternative formats

Summary of pilot 1

Pilot 1 allowed initial testing of the GP version of the questionnaire. From inspection of responses to the survey, and checking content validity based on patient interviews, we identified changes required to improve the questionnaire. Based on feedback from the team and members of the advisory group about formats to improve accessibility of the questionnaire, we chose to focus on lay and interpreter administration, and developing an Easy Read version. We also developed a pharmacy version. We went on to test these further in pilot 2.

Methods

The second pilot took place over 3 months (October–December 2012). For the second pilot, two GP practices were chosen (one small practice in a non-English speaking inner-city area, and one very large practice in a deprived town). In addition, two pharmacies (one inner-city and one rural) were invited to pilot the pharmacy version.

While the first pilot exclusively employed face-to-face methods to distribute the patient packs, the second pilot included postal methods in one GP practice. Based on evidence on improving response rates,93,94 the process entailed:

1. Sending patients a prenotification letter informing them that they would soon receive a questionnaire through the post.

2. Sending patients the patient pack (containing an invitation letter, information sheet, questionnaire and reply-paid envelope) 1–2 weeks later.

3. Sending patients a follow-up letter 1–2 weeks later. This letter thanked patients who had returned the questionnaire, and reminded those who had not to complete and return the questionnaire. It included some frequently answered questions about concerns that patients may have about completing questionnaires.

The questionnaire was posted out to 150 patients in one practice, selected randomly from the patient list for patients who had visited the surgery in the previous 6 months. An online version was made available.

The other GP practice (PCO10) asked members of their PPG to distribute the patient packs in the waiting rooms; this enabled us to assess the feasibility and effectiveness of involving lay people in administering the questionnaire, which was identified in the consultation in pilot 1 as a potentially important way of improving accessibility and uptake. PPG members handed waiting patients a patient pack, briefly explained the study, and offered to help patients to complete the questionnaire if they wished. It was hoped that this offer of help might increase the response rate from those who may have literacy or sight problems, and those for whom English is not their first language.

In the pharmacies, 75 packs were handed out to adult patients attending for medical advice or prescriptions. Customers were excluded if they were simply using the pharmacy as a shop, for example to buy toiletries, or if staff felt that it would be inappropriate to include them, for example due to a recent bereavement.

Patients completing the questionnaire were invited to return their contact details to express an interest in being interviewed. The aim was to interview up to 20 patients. Cognitive interview techniques,95 drawing on the ‘verbal probing’ method,96 were used to identify problems with questions and misinterpretation: specifically, how participants understood each question on the questionnaire; how they answered it; and how the question might be improved. The topic guide for the interviews is shown in Appendix 11. Interviews were conducted by JW, mostly in participants’ homes (two chose to be interviewed on university premises). We aimed to involve non-English speakers in the cognitive interview process; we recruited Polish speakers in the waiting room of one of the participating GP practices (PCO06), and Gujarati speakers from the community organisation involved in recruitment to the focus groups. An interpreter assisted patients in completing the questionnaire and participating in interviews. The interpreters were interviewed by JW about the experience and difficulties of translating the questionnaire for responders.

Transcripts were uploaded into NVivo for analysis, and coded and charted to identify key issues relating to comprehension, response options and interviewer difficulties97,98 (for the interpreted questionnaires). We identified suggestions for improvements.

The findings from the cognitive interviews were used in conjunction with data from the survey to refine the questionnaire to develop a near-final version. Version 3 of the GP questionnaire and version 2 of the pharmacy questionnaire were then tested for reliability and validity in pilot 3, along with version 1 of the walk-in centre questionnaire.

Results

Revisions to questions: changes from the second to third pilot

Based on inspection of the pilot data, and informed by the findings of the cognitive interviews, we made the following changes to the GP and pharmacy questionnaires:

Developing version 1 of the walk-in centre questionnaire

A first version of a parallel questionnaire for use by walk-in centres was developed for testing in pilot 3. This mirrored the GP questionnaire, but was adapted to make it appropriate for use in walk-in centres, informed by the findings of interviews in stage 1. The key differences between the walk-in centre version 1 and the parallel version 3 GP questionnaire are summarised in Table 20.

Version 1 of the walk-in centre questionnaire is also shown in Appendix 13.

Summary of pilot 2

Pilot 2 allowed additional testing and refinement of the GP version of the questionnaire based on inspection of responses to the survey and findings from cognitive interviews. The pharmacy questionnaire was also subject to testing, and a parallel walk-in centre version was developed for testing in phase 3. The involvement of PPG members in handing out the questionnaire and helping patients complete it was welcomed by PPG members and patients, although it did require commitment of time from PPG members and did not generate a high response rate. Postal administration was used in only one practice, but they found it burdensome and were concerned about a low response rate from ‘seldom-heard’ groups, particularly non-English speakers.

Data entry and cleaning

Data were entered onto SPSS v20. Descriptive summaries (frequency tables and charts) were used to check for any incorrect data (see Appendix 14). If a responder had ticked more than one option, then the higher (more negative) number was taken. For the SERVQUAL scale, where a respondent had circled two adjacent numbers, the lower (more negative) number was taken. Where a respondent had circled two numbers within two points of each other, the middle value was taken (e.g. if 3 and 5 circled, 4 was entered). Where a respondent circled two non-adjacent numbers more than two points apart, this was counted as missing data. We calculated scores for the two dimensions of the SERVQUAL scale: responsiveness (four questions: A–D) and empathy (five questions: E–I), by calculating the mean score on each dimension for each respondent. SERVQUAL scores were calculated for individuals with up to one missing data point.

Questionnaire structure: factor analysis

We carried out an exploratory factor analysis to investigate the structure of the GP questionnaire, and to identify key questions to form the basis of scores for the questionnaire which could be used as measures of elements of responsiveness. Questions were excluded from the factor analysis on the basis of initial results, and based on consideration of the content of questions in the questionnaire. We aimed to include generic questions (relating to universal needs and relevant to the majority of patients, regardless of which group they belonged to), and remove questions specific to certain groups (where there were a high proportion of ‘don’t know’ answers, indicating that these questions were only relevant to certain patients, e.g. the questions about getting a home visit and getting to see a doctor or nurse who spoke the patient’s language).

The final round of factor analysis included 15 generic questions. The analytic approach involved varimax rotation, with factors generated based on the criteria of MINEIGEN = 1. Missing data were handled using ‘pairwise deletion’.

Factor analysis generated a three-factor solution. Together, the three factors explained 57.7% of the variance. The factors do not correspond exactly with the sections of the questionnaire, but can be described as:

• factor 1 (five questions): ease of access to a suitable appointment

• factor 2 (seven questions): welcome, help, and support for ongoing care

• factor 3 (four questions): ease of using the building and facilities, and information.

In Table 24, coefficients below 0.3 are suppressed.

Internal consistency of scales: Cronbach’s alpha

Cronbach’s alpha coefficients were calculated to assess internal consistency of three subscales identified through factor analysis. High alpha scores suggest that scales are measuring the same underlying construct (Table 25). Missing data were deleted list-wise.

Factors 1 and 2 showed ‘good to excellent’ internal consistency (0.7–0.9), while factor 3 was ‘acceptable to good’ (0.6–0.7). The lower scores for factor 3 may partly reflect the smaller number of items in factor 3, but there would be value in testing the measure again with another data set to confirm the factor structure. This suggests that it would be appropriate to compute scores for each of the three factors.

Calculating scores for the questionnaire

We calculated scores from the questionnaire: we calculated an average score for each of the three factors for each patient, by averaging the scores across the questions included in the factor. For each factor, we calculated scores for individuals who had answered all questions or omitted no more than one question. Table 26 shows mean scores and standard deviations for each of these scores. Lower scores indicate more positive experiences.

External validity: correlations between the responsiveness scores and SERVQUAL dimensions

Pearson correlations were calculated between the scale scores from our questionnaire, and scores on the ‘responsiveness’ and ‘empathy’ subscales of SERVQUAL. We expected a modest negative correlation, as the content of the SERVQUAL scales differs somewhat from the content of the responsiveness questionnaire, and SERVQUAL is scored in the reverse direction to the questions on our questionnaire, i.e. higher scores are more positive. Correlations are shown in Table 27.

The correlations between the factors and the SERQUAL scores were all in the expected direction. All correlations were significant at p < 0.001. The correlations between factor 3 and the SERVQUAL scales were relatively low. This is to be expected, as the issues to which this factor relates (buildings and facilities, and supporting communication) are not covered directly in the SERVQUAL scales.

Test–retest reliability

The consistency of individual responses between the original and retest questionnaire were assessed using Kappa for individual questions and ICCs for scores.

Data display: data from the questionnaire for individual practices

Practices may wish to look at the frequency of positive responses for each individual question in comparison with other practices, in order to identify areas in which their patients report less positive experiences. An example for practice 16 factor 1 is given below (Figure 6). This reports the percentage selecting ‘very easy’ on each question within the practice, compared with the percentage selecting this option in the overall sample across all practices. The graph suggests that this practice performs better than average on making it easy for patient to get an urgent appointment on the same day, to see a doctor or nurse of their own sex, and to get a home visit if they need one, but worse than average on seeing their preferred doctor, making it easy for patients to see someone who speaks their language, and getting a telephone consultation.

FIGURE 6.

Percentage responding ‘very easy’ to the questions ‘How easy is it for you to . . .’: overall sample vs. practice 16.

It is important to note, however, that responsiveness relates to the extent to which different patient groups are supported in accessing and using services. In order to assess responsiveness in a valid way, the data produced must be based on a sample that is representative of the patient population; achieving this may require a range of strategies, as discussed later in this chapter.

Data display: comparative data across practices

The questionnaire could be potentially be used to compare practices to provide feedback on how their level of responsiveness differs from other similar practices. This may be useful at a local level, for example to help identify where quality improvement efforts or where additional funding may be needed, to help practices improve their responsiveness.

The scores for each practice are shown in Figures 7–9. A higher score indicates a less positive overall experience.

FIGURE 7.

Means and confidence intervals (CIs) for factor 1 scores (error bars 95% CI).

FIGURE 8.

Means and confidence intervals (CIs) for factor 2 scores (error bars 95% CI).

FIGURE 9.

Means and confidence intervals (CIs) for factor 3 scores (error bars 95% CI).

Caution should be exercised in drawing conclusions from these comparative data, as scores are likely to be highly influenced by local contextual factors (such as practice 22 sharing its premises with a walk-in centre), and characteristics of the local population (such as practice 8 serving a population with a high level of deprivation; this population might be expected to find it more difficult to use services).

However, this type of comparative data is likely to be of value in identifying areas where efforts are required to improve the alignment of services with patient need. This potential would be strengthened by further research to test the discriminant validity of the measure.

Revisions of the questionnaires to produce final versions

Decisions about which questions we would remove were based on inspection of the descriptive data, and checks against the findings from the set of patient interviews conducted in stage 1. We were also informed by the exploratory factor analysis. We aimed to reduce the number of questions to make the questionnaire more manageable. In summary, revisions included:

1. Removing some redundant or poorly performing questions to reduce the length of the questionnaire. These included questions on finding out about the PCO’s opening hours and services; whether or not the reception staff are polite; how easy it is to let staff know that you have arrived for your appointment; and how good the PCO is at acting on patients’ suggestions and complaints.

2. Removing the question on travel to the surgery: this had been the source of some dispute as it is an issue important to patients. It was decided to remove this as it is outside the control of PCOs. The question on home delivery of prescriptions was removed as this could be assessed objectively.

3. Minor rewording to improve clarity.

Final versions of the questionnaire are included in Appendix 15.

Summary of analysis

Testing suggests that the questions are reliable and valid, and can be used to produce scale scores. We suggest that the results would be easier to interpret if scoring were reversed, i.e. so that high scores indicate better performance.

More research is required before these are put into use to develop guidance for sampling and for administering the questionnaire, and for interpreting scores. There needs to be consideration of how this measure can be integrated into a more holistic assessment of responsiveness, which may include ‘objective’ measures of practices’ responsive activities, and systematic sampling and surveying of diverse patient groups.

Practices may also like to explore the data from different patient groups to identify whether or not certain groups have less positive experiences, and why. This would provide a starting point for identifying how to address these problems. The questionnaire could be used to compare the experiences of patients within certain groups, for example patients with disabilities, within and between practices. We have provided some examples in this section of how data from the questionnaire could be displayed for use by providers. Sophisticated approaches to making data accessible, and displaying them in ways that are intuitive and easy to interpret, have been developed for existing patient-experience and other quality measures,31,102,103 and there would be value in learning from these approaches to optimise the display of data on responsiveness.

Overview of evolution of questionnaires

Tables 29–31 provide a summary of the evolution of questions in the GP, pharmacy and walk-in centre versions of the questionnaire, from pilot 1 through to the final version. Comments on the reasons for changes are also included.

A general practitioner practice with a large non-English-speaking Turkish population

The area in which PCO18 is based has a low literacy rate. The practice manager identified three ‘seldom-heard’ groups: Somali women; males, in particular Caribbean males; and Turkish patients. Turkish patients make up 15% of the practice list. Those who can speak some English still find it difficult to communicate with non-Turkish staff members. The practice manager noted that Turkish patients highly frequent the practice: ‘40 times a year as opposed to the 3–4 times a year on average from other patients. Each time they demand the Turkish doctor and refuse to see another’.

At the time of the interview, the practice employed a Turkish doctor and two part-time Turkish receptionists but there were days when no Turkish speakers were present at the surgery. When the Turkish doctor is away on leave, the vast majority of the Turkish patient population cancel any pre-booked appointments. The interim data analysis indicated a lack of questionnaires returned by patients self-ascribing as Turkish or Kurdish.

For previous patient surveys, staff sat in the waiting room with as many Turkish patients as possible, going through each question and noting their answers. They had not had the resources or the time to do this with the stage 3 responsiveness questionnaires. This approach also has disadvantages in that patients may not feel able to be totally honest about their experiences when a staff member is acting as a translator.

A general practitioner practice providing enhanced services

PCO22 is a GP surgery providing enhanced services (substance misuse, homeless, sexual health and mental health). Enhanced service workers proactively search for ‘seldom-heard’ groups and groups who are disadvantaged rather than relying on them attending the practice. This includes visiting their homes, shelters, bed and breakfast hotels, a local community group (which is very well attended), and a centre for substance misuse (also very well attended).

The practice had previously attempted to engage with these groups by organising a PPG but there was always a low turnout at scheduled meetings. The few who attended regularly complained about their treatment and the speed of the services delivered to them as individuals. None commented on practice processes. The practice never receives suggestions for improvement or feedback from patients.

Discussion

We worked with practices to find methods that were practical for PCOs during the research project but these would be equally feasible for use as part of routine surveys in order to include ‘seldom-heard’ groups. PCOs need to understand their practice populations and which groups will be ‘seldom heard’ using traditional approaches, and to invest time and resources in accessing these ‘seldom-heard’ groups. We found very good acceptance of questionnaire by ‘seldom-heard’ groups when they were offered the questionnaire in a tailored way.

Methods

Systematic and non-systematic literature searches were conducted of published materials, ‘grey’ literature and websites, to identify data sets containing information about practice populations, and publicly available information about practice patient profiles. Initial searches yielded results from local PCT websites, the DH and other informational NHS-related pages. A search was conducted of the databases MEDLINE, Scopus, The King’s Fund Library, Web of Science and Science Direct from January 2001 to March 2011. Search terms included terms such as ‘PATIENT DATA’, ‘DEMOGRAPHICS’ and ‘PROFILING’ combined with terms such as ‘PRIMARY CARE’ and ‘GENERAL PRACTICE’. Articles from the year 2000 onward were included in the primary search. When search combinations produced results of 2500 or more, the searches were refined using subject headings. Articles were then read to identify instances where practice population data from a particular general practices (or a group of known general practices) had been stated, included, referenced or displayed. Over one-third of the search results were duplicated throughout all the databases. Reference snowballing was also conducted on articles, and papers of relevance were added. Secondary searches of the literature were conducted using the titles of the local primary care toolkits and the names of general practice resources found within the initial ‘grey’ searches, coupled with primary care-related terms.

Results

Discussion

Electronic patient records have the potential to be used for mapping population characteristics, but this is dependent on the quality and completeness of the data recorded, and it may be complicated and time-consuming for individual practices to extract and display these data. Tools and approaches for practices or CCGs to map the features of individual practices’ patient population, based on electronic records and other information, are described in Part 2: existing population mapping techniques, below.

A number of large searchable databases of primary care patient data exist, but these are not primarily designed to provide detailed, practice-level information about the characteristics of the patient population. The national databases (DIN, GPRD and THIN) are primarily designed to be used for research purposes based on anonymised individual patient data. It would not be possible to identify and single out the data for one GP practice.

There are a number of sources of practice profiles that are easy for practices to access but the information they contain is often incomplete. The PHO NGPPs are easily accessible and provide very basic categories of practice data. It is possible for practices to see an overview of the practice in clinical and non-clinical terms, but the profiles do not provide defined patient population characteristics. In addition, the depth of information displayed is not adequate for viewing detailed information on a particular patient population: the QOF scores display the performance of the practice under the various indicator categories, but further details about the practice population characteristics are not available. The NHS iView displays a greater number of non-clinical categories that could potentially be used to improve practice responsiveness, but the total number remains low. The tool is focused on displaying figures on clinical information. QOF data are also of limited value in assessing practice patient characteristics.

Methods

Informal literature searches were performed to draw out existing mapping techniques in use or proposed to generate information about population characteristics at the level of individual practices. A literature search was conducted with the terms ‘MAPPING’, ‘PROFILING’, ‘NON-CLINICAL DATA’, ‘PRIMARY CARE’, ‘GENERAL PRACTICE’ and their variants, from 2000 onwards. These terms were used to search online databases including Science Direct, MEDLINE and Web of Science, and searches refined using subject headings. Identified articles were screened for relevance by reviewing titles. A ‘grey’ literature search was conducted using similar terms, and key websites/reports identified in the formal literature were retrieved.

Results

Discussion

There are a number of tools and methodologies for mapping and displaying data on population characteristics in primary care, but their usefulness depends heavily on the quality and completeness of existing diversity data.

Concerns and challenges: ethnicity data

Ethnicity data in primary care are not collected systematically or recorded to a high standard. 162 Missing ethnicity data within routine data sets in primary care have hindered comprehensive assessments of health inequalities experienced by ethnic minority groups. Any drive to reduce inequalities for these groups has tended to lack focus and measurable elements for evaluating successful initiatives. 163 Without a comprehensive data set, the NHS and other public services cannot say whether they have or have not delivered race equality outcomes. 164 There has been a growing interest in filling in the gaps in ethnicity data to enable effective equality monitoring. 165

The DH recommends the collection of ethnicity data ‘as indicated by the national minimum ethnicity dataset’ (p. 7), covering both ethnicity and first language. 166 There has been a drive to improve race equality in relation to mental health patients in both primary and secondary care, prompting the launch of the Mental Health Minimum Data Set (2003–8) to include new collections of data (including ethnicity) and the National Mental Health and Ethnicity Census in 2005. 167

The NHS Practice Management Network suggests that general practices begin to understand their communities. It forms the first step of the five-step plan to ‘improve access for patients from black and minority ethnic groups’ (p. 95). They encourage the practice managers to put measures in place to monitor ‘ethnicity and preferred language plus all other diversity pointers’, seeing this as vital to providing a responsive service to patients. 14

Studies have explored the practicalities surrounding ethnicity data collection within general practices. 165,168,169

One of the main barriers to systematic collection of ethnicity data is the lack of understanding within primary care teams (and the NHS as a whole)170 as to the potential added benefits to service delivery from collecting and analysing these data. It is thought that the unsuccessful attempts to relay the benefits around the mass collection of ethnicity data, coupled with the lack of incentives for ethnicity data collection at PCO level, has led to low recording rates overall. 171,172 The collection of ethnic monitoring data is incentivised in QOF, attracting one QOF point, providing some financial incentive for practices to engage in this data collection. Systematically collecting ethnicity data within PCOs requires significant time and resources. 81,173 The financial value of the QOF incentive is fairly minimal in terms of the effort and resources necessary.

Staff resistance to asking patients for ethnicity and other personal data, and patient resistance to providing such data, are key challenges. Staff resistance emerged an a significant barrier to the collection of ethnicity data in a Liverpool initiative;174 staff could not envisage the added benefits in the collection of ethnicity data, and felt that the additional value of the collection was less than the additional workload demand it would cause. A study of primary care patients’ perspectives on the collection of ethnicity data found that patients had reservations about data collection and the possibility of abuse of information they deemed sensitive. 170

Another key challenge to collecting and recording ethnicity data relates to the level of detail to be recorded. Collecting and storing data under multiple self-ascribed classifications would enable a detailed overview of the practice population. However, general practices that have attempted to record ethnicity data about their patients have generally done so in order to monitor and target treatment for chronic diseases more prevalent in certain ethnic groups. 175 For this purpose, using less descriptive categories is often preferred;176 this makes for easier data analysis when studying disease prevalence and management at a practice level. The problem with this approach is that, when considering tailoring service delivery to practice populations, the broader categories may systematically suppress the subgroups that are potentially more useful. Broader categories are less useful to tailoring to non-clinical aspects of care that are concerned with improving patient access to and their experience of care. If practices already record less ‘fine-grained’ ethnicity data, they may be unwilling to collect more detailed data concurrently.

A recent review by the Race Equality Foundation162 summarised the challenges in routinely collecting ethnicity data:

• difficulties in obtaining an accurate classification

• reluctance to request data for fear of giving offence

• no meaningful incentive to collect or provide data

• reluctance on the part of service users to supply data

• inability to supply data owing to language barriers

• lack of understanding as to how data can or will be used.

The report argued that ‘successful ethnic monitoring requires a strong regulatory framework complemented by proactive, committed leadership’ (p. 1), and emphasised the need to increase the extent and quality of ethnic data collection and to ensure that data are used to inform and improve practice.

Examples of projects to improve the collecting and recording of diversity data at practice level

Methods

Semistructured interviews were conducted with practice managers and GPs at four practices with diverse demographics (multicultural city centre; high-deprivation city fringe; suburban, largely elderly population and less deprived; and inner-city, high minority ethnic population). Three practices were chosen from those already involved in the main study. One GP worked for the head office of a large private company (of which PCO13 is a part) and spoke about the local GP practice where he also worked. Inclusion criteria were largely pragmatic and included practices with diverse patient populations and that were willing to be involved. The topic guide included questions about types of data collected by the practices and about staff views on mapping practice population characteristics and on using these data (see Appendix 17). Data were analysed using a combination of framework analysis and the constant comparative approach. 83 A stage of familiarisation involved reading the transcripts and making notes. An initial coding frame was generated based on themes of interest, relating to what data were collected within the practices; the extent to which practices used the data, and the barriers to collecting and using diversity data. Data were entered into the coding frame using NVivo, and care was taken to add new codes for data that did not fit the existing coding frame. Data summaries were produced for each theme.

Results

Seven interviews were conducted, with four GPs and three practice managers. Quotations are labelled GP for general practitioners and PM for practice managers.

We describe variation between practices in the extent of non-clinical data they recorded for each patient, and how they went about capturing, recording and using these data. We also explore staff views on the value of such data, and of the barriers and challenges to both collection and use of non-clinical diversity data.

Discussion

All four practices collected some non-clinical data, usually when a patient first registered with the practice. Subsequent changes to patient information were less likely to be recorded.

Some staff understood that non-clinical data could improve clinical outcomes for patients and could be used for tailoring services, for example recruiting multilingual staff or restructuring the appointment system. Financial incentives and government policies provided motivation to collect some types of data, such as ethnicity, but not other types of non-clinical data. Practices did little with the data they collected.

Hearing the seldom heard

Responsiveness is about making sure that primary care services can meet the needs of all patients, even the ‘hard to reach’ or ‘seldom heard’. 29,30 This has implications both for efforts to improve responsiveness and for the use of the questionnaire. We reflect on how successful we were in involving ‘seldom-heard’ groups in the research process, and identify lessons for future research.

Efforts to improve responsiveness

This study found that PCO staff do not necessarily think about responsiveness in the complex way outlined in this report; PCOs tend to think of responsiveness in terms of being reactive: responding to individual patients as they access the service, and responding to patient complaints or suggestions, or patient survey data. There was less evidence of PCOs thinking about responsiveness proactively. Few had attempted to systematically map their patient population or plan actively for their patients’ needs. This restricted approach has significant limitations in that PCOs might respond only to those who make themselves heard and who take an active role. The patient interviews indicated that most patients were not interested in or able to engage or be involved and that this is this is particularly likely to be the case for patients from disadvantaged and vulnerable groups.

To be responsive to all, PCOs need to invest in finding out more about the disadvantaged groups in their patient population, particularly those who may not access services because they face significant barriers in doing so. This may mean PCOs (or CCGs) having to give extra support or make extra effort for some patient groups, and there are implications of this in terms of time and resources.

It is important to recognise that, while the questionnaire assesses how easy it is for patients to get their needs met (such as their need to easily access an appropriate appointment), what this means to an individual might vary tremendously. For example, ‘a convenient time’ will vary, with daytime appointments perhaps preferred by the elderly and early or late appointments by those at work. Being responsive involves identifying gaps in meeting needs, and understanding how best to meet those needs for underserved patients.

This reinforces the need for each practice to identify their particular ‘seldom-heard’ (and underserved) patients. Practice populations differ considerably, and patients’ needs and wants will vary accordingly. For example, a practice with high numbers of retired British-born nationals may need to focus on physical aspects of the practice to maximise accessibility, whereas an inner-city multiethnic practice will need to give special attention to responding to multilanguage challenges. There is a potentially large number of different and overlapping groups. The complexity of differing patient populations and their needs and wants means that there is not a ‘one-size-fits-all’ solution to increasing responsiveness. The challenge of being inclusive and responsive is significant.

Interestingly, although patient involvement, choice and voice are seen in policy as key ways of ensuring responsiveness, we found that notions of involvement, choice and voice did not feature in patients’ accounts of responsiveness. Instead, we found that, while many patients want to be involved in decisions on an individual level186 – in terms of their individual treatment and care – on the whole this desire for involvement did not seem to extend to collective decisions at the organisational level. Most patients were not interested in being or did not feel able to be engaged or involved in shaping the design or delivery of services. For most patients, particularly those from vulnerable groups, responsiveness was reflected in the extent to which practices ‘took the strain’, i.e. did the work of identifying needs, planning service delivery and providing support to patients to use services and to help organise their care. Patients from disadvantaged groups may struggle to act as active and involved partners in the design and delivery of health care. They need primary care providers to take responsibility for enabling them to overcome barriers to using services.

This study suggests that pluralisation of primary care providers can have a positive impact on responsiveness, but that this is unlikely to be due the effects of choice and competition driving up responsiveness. Rather, this is due to the emergence of specialist providers that are better able to accommodate to and meet diverse needs. Examples include the introduction of walk-in centres that enable patients to access services without the need to wait or make appointments, and the emergence of specialist providers such as specialist practices for homeless patients and asylum seekers.

Models of health-care provision that focus on empowering patients to drive improvement through choice and involvement may, in fact, disadvantage the most vulnerable. Instead, there is a need to ensure that primary care providers are better placed to support the least empowered.

Use of the questionnaire

Reaching diverse groups with the questionnaire was a challenge for this study, and will remain a challenge for those who want to use the questionnaire in practice. The questionnaire has been designed to maximise its acceptability to a range of diverse patient groups: patients with disabilities, homeless patients, travellers and non-English speakers were closely involved in its design, and it was piloted with different ‘seldom-heard’ groups. This included conducting cognitive interviews with non-English speakers to check for problems in translation if the questionnaire is used with an interpreter. It also included targeting homeless patients, vulnerable patients and non-English speakers in pilot 3. We worked with a specialist organisation to develop an Easy Read version.

It is critical that future users of the questionnaire recognise the need to use appropriate strategies to access the views of diverse groups. Simply handing out questionnaires to a consecutively attending sample of patients or mailing to a random patient sample will not ensure that the views of diverse patient groups are heard. The validity of the measure is dependent on steps being taken to include the ‘seldom-heard’. There are multiple barriers to be overcome, ranging from a lack of willingness to engage with surveys, to language barriers, to barriers to completing questionnaires caused by limitations arising from physical, sensory or cognitive impairment. Gaining a genuine picture of responsiveness to the needs of diverse groups requires (i) information on the characteristics of the population to be surveyed; and (ii) the time, resources and motivation to invest in efforts to access the views of these groups.

There is guidance available on accessing the views of hard-to-reach groups. 187 This is not as challenging as it may be assumed to be. In pilot 3, we conducted a substudy specifically aimed at accessing hard-to-reach groups in two PCOs. Neither PCO had reliable data on the characteristics of their patient population, but both were able to identify vulnerable patient groups whom they served, who were unlikely to be reached through traditional survey methods. The first practice identified that they had a significant number of patients who spoke only Turkish or Kurdish. We arranged for an interpreter to be available in the practice for three mornings (12 hours), and were able to get 20 completed questionnaires. This involved a small cost, but generated data from a hard-to-reach group. The second practice provided a substantial enhanced service to patients with complex needs (substance misuse, homeless, sexual health and mental health). They had struggled to hear from and involve this patient group, but were able to gain 20 completed questionnaires and huge insight into the experiences of these patients by assigning two staff members (not core practice staff) to spend a small amount of time over a 4-week period on their routine visits to a breakfast club for vulnerable people, encouraging and assisting patients to complete the questionnaire.

Our experience suggests that with a small commitment of resources, and careful consideration of appropriate strategies, the views of ‘seldom-heard’ people can be accessed. Such efforts are also likely to be valuable in opening up lines of communication with hard-to-reach groups.

It is a key consideration as to how much effort practices will be willing to put in to get the participation of people who find it particularly difficult to participate. It is unknown what practices need to see, in terms of benefits for them, to engage in such a labour intensive process which requires commitment and investment of resources.

Involving the ‘seldom heard’ in research

We were very keen to involve patients from hard-to-reach, or ‘seldom-heard’, groups in our study, as we felt that this would be critical in designing a measure that was accessible and appropriate for all. We had considerable success in this, using the following approaches:

• making links with experts, specialist practices, and community organisations

  • having a study advisory group which included representatives of diverse and hard-to-reach patient groups; advisory group members helped with snowball sampling, and reviewing the measure

  • working with local community groups and voluntary organisations to recruit patients and get feedback on the questionnaire

  • working with specialist providers (e.g. a GP practice serving homeless patients

  • attending local events (e.g. a ‘Big Health Day’ for people with learning disabilities)

  • working with a specialist organisation to develop Easy Read version.

• seeking to involve diverse groups actively in the study

  • providing interpreters and involving carers/advocates in focus groups and interviews

  • flexibility in research methods (e.g. face-to-face recruitment with homeless patients).

Using these approaches, we were able to access the views of diverse patient groups, including travellers, non-English speakers and patients with learning disabilities.

We found that ‘seldom-heard’ groups were not so hard to involve in the research process. Our success was dependent on the advice we received from our advisory group; efforts to build relationships with key individuals, local community groups and voluntary organisations; and having a flexible and adaptive approach to conducting the research, such as going out from the PCOs and into the community setting for the purposes of administering the questionnaire, and working with interpreters and carers. We found that working with diverse patient groups was extremely valuable for questionnaire design, helping to improve the validity and acceptability of the final product, and understanding how to access the views of diverse groups. The support we received from the specialist organisation in developing and testing the Easy Read version was particularly important.

Use of the measure for service planning

Primary care in England is facing several related challenges. 188 The demand for care is increasing as the population ages and the numbers of people with several comorbidities rise. The supply of GPs is insufficient to meet the demand,189,190 funding is constrained,191 action to reduce the burden on hospital services is needed,192 and at the same time general practice has taken on lead responsibility for commissioning many services, a role that requires the acquisition of new attributes, including a community perspective. 193 Responses to these challenges have included proposals for the creation of federations of practices that are able to share policies and resources to improve efficiency and effectiveness,6 greater integration of care with general practice playing a key role,194 whole-person care,195 and the greater adoption of a public health perspective in primary care. 195 Although a detailed and national plan for the reform of general practice and primary care to meet the current challenges has not yet been set out or implemented, all of the pressures and suggested responses indicate that primary care will need to improve the planning of services (both those delivered in primary care and those commissioned from other health and social care sectors), a process that will rest on an improved understanding of the needs of populations and communities, and better monitoring of the extent to which those people with potential to benefit from health care do have access to the appropriate care. Responsiveness to the needs of populations is, therefore, fundamental to the delivery of primary care in the coming years.

The instrument we have developed has several potential roles in assisting primary care meet the new need for responsiveness. Providers (practices, CCGs, primary care federations) will be able to use the measure to monitor whether or not care is responsive, i.e. whether or not care is reaching those sections of the population in need to care, and therefore reducing the need for the secondary care of complications or late-stage disease or reducing the need for social care. A key influence in acceptance of the questionnaire will be the local PPG. The questionnaire could even be ‘owned’ by the PPG and form a focal part of the PPG–practice relationship (or, more widely at a local level, the public–CCG relationship). PPG guidance would be beneficial in leading efforts to reach the ‘seldom heard’. Perhaps the ideal approach at practice level may be a ‘ground-up’ push led by the local PPG, with a top-down push and support from the CCG.

Population mapping

Mapping the population to understand the characteristics of the populations is important for several reasons: first, to gain greater insight to the characteristics/demographics of the practice population; second, to help identify specific groups who may be ‘seldom heard’ via traditional survey method; third, to interpret the results in an informed way; and, finally, to identify areas for improvement.

Population mapping aids the federation, CCG or practice to plan their population (reactive and proactive)-orientated strategies by informing the practice of the characteristics of their population. Once ‘seldom-heard’ groups are identified, then a plan can be developed to ensure these groups are heard when the questionnaire is used. Learning gained from this study can illuminate approaches to successfully reaching the ‘seldom heard’ and can be utilised by PCOs.

Improving responsiveness

The definition of responsiveness developed in this study has several implications for efforts to measure and improve responsiveness. The first is that PCOs may wish to target the questionnaire at specific patient groups as part of a process of improving the experiences of these groups. The second relates to the potential for the measure to be used more widely across a CCG, a region, or nationally. This could involve, for example, using the measure comparatively to identify specific GP practices where patients report lower levels of responsiveness. In doing this, it would be important to recognise that the measure would not, in itself, be an objective measure of ‘quality’ of the provider. Instead, the measure would be better viewed as a tool to be used in the early stages of service delivery planning, as a means of identifying problems and gaps, and for the purposes of issue stimulation.

The scores can be seen as reflecting the level of alignment between the way the PCO organises and delivers its services, and the needs of the local population. Clearly, responsiveness not only depends on the way a PCO delivers its services, but is also highly dependent on the level and type of need of local patients, and the challenges they may face in accessing and using services. Addressing this lack of alignment may require the PCO to (1) improve the way in which they organise and deliver services by attending to the three key components of responsiveness; and (2) work with local patients to manage needs and expectations (e.g. in our study one practice worked with local patients who were recent immigrants, and who had little understanding of how English primary care worked, to educate them about how to use services and hence bring their expectations in line with what the PCO could provide). However, it may require attention to the range and nature of services available locally in order to identify if there are gaps in meeting patients’ needs that would be better met by specialist primary care services or by working closely with other services (such as social care or charities). This level of consideration implies a need for strategic involvement and planning, possibly at a CCG level.

This study raises an important question about the most appropriate ways of improving responsiveness across different care providers. Our findings are most pertinent for GP practices. Identifying the features of the business model, or model of operation, that underpins the work of particular organisations, and considering the implications of this for what responsiveness means in terms of patient benefit, and how best to encourage and support it, is likely to be important.

Lessons learned

We reflect on decisions made during the questionnaire development process, and highlight the implications of these decisions in terms of the content and format of the questionnaire, the need for further development and testing, and for the potential future use of the measure.

Conclusions

Responsiveness is a complex concept, involving alignment of service delivery and the needs of diverse patient groups. Achieving responsive primary care requires PCOs to use reactive and proactive strategies at individual and population level. In practice, responsiveness tends to be understood and enacted in a limited way; PCOs focus on reacting to patients’ requests and being accommodating when patients access the service, and/or on reacting to patient feedback and complaints. There is less evidence of proactive planning for diverse patient needs across primary care, or of understanding how to best manage and meet the needs of diverse patient groups.

We found that responsiveness is important to patients, and patients from vulnerable groups often experience primary care as failing to be responsive to their needs. This reinforces the emphasis in policy on improving responsiveness, and suggests that there is still much work to be done in this area. Being responsive means providing good care equally to all, and some groups may require extra support. What this extra support is will differ reflecting the different patient populations, and so knowledge of the practice population is essential to be responsive. However, practices do not systematically collect or use diversity data, and lack of understanding of the value of such activities, as well as lack of resources, is a significant barrier.

We developed parallel versions of a patient-report measure of responsiveness for GP practices, walk-in centres and pharmacies. We provided preliminary evidence of reliability and validity for the GP measure, but additional testing is required. Our questionnaire has the potential to be used as a measure within PCOs, between PCOs, and across different patient groups. It will have most value when used as a part of a process of mapping local populations, surveying patients across diverse groups to identify where there is lack of alignment between needs and service provision, and developing efforts to address this gap. The validity of the measure depends on PCOs’ efforts to involve hard-to-reach groups; a challenge exists in that PCOs that are less responsive are less likely to invest in the time and effort required to hear from these groups. An understanding of how to incentivise and support PCOs to engage in this is required.

Future research

The following research requirements emerge from this study.

MEDLINE strategy (primary care), revised

Web of Science: strategy 1 (primary care)

1. #24 #21 AND #20 Refined by: Publication Years=(2006 OR 2010 OR 2008 OR 2007 OR 2009 OR 2001 OR 2003 OR 2002 OR 2004 OR 2005 OR 2011) AND General Categories=(SCIENCE & TECHNOLOGY OR SOCIAL SCIENCES)

2. #23 #21 AND #20 Refined by: Publication Years=(2006 OR 2010 OR 2008 OR 2007 OR 2009 OR 2001 OR 2003 OR 2002 OR 2004 OR 2005 OR 2011)

3. #22 #21 AND #20

4. #21 #19 OR #18 OR #17 OR #16 OR #15 OR #14 OR #13 OR #12 OR #11 OR #10 OR #9 OR #8 OR #7 OR #6 OR #5 OR #4 OR #3 OR #2 OR #1

5. #20 Title=(primary same care or "general practi*" or "family practice" or "family doctor*" or "family physician*" or "primary care same nurs*" or "health visitor*")

6. #19 Title=(consumer* same preference*)

7. #18 Title=(consumer* same experience*)

8. #17 Title=(consumer* same perspective*)

9. #16 Title=("non-health benefit")

10. #15 Title=("patient expectation*")

11. #14 Title=(responsive*)

12. #13 Title=(respond*)

13. #12 Topic=("inverse care")

14. #11 Title=("service delivery")

15. #10 Title=("service quality")

16. #9 Title=("patient participation")

17. #8 Title=("patient preference*")

18. #7 Title=("patient need*")

19. #6 Title=("patient consultation*")

20. #5 Title=("patient experience")

21. #4 Title=("patient report*")

22. #3 Topic=("non-health expectation*")

23. #2 Topic=(divers* same workforce)

24. #1 Topic=(segmentation same population)

Web of Science: strategy 2 (public services), revised

#30 #27 AND #20

Refined by: Publication Years=(2009 OR 2001 OR 2010 OR 2005 OR 2000 OR 2008 OR 2007 OR 2004 OR 2002 OR 2006 OR 2003 OR 2011) AND General Categories=(SOCIAL SCIENCES OR SCIENCE & TECHNOLOGY)

#29 #27 AND #20

Refined by: Publication Years=(2009 OR 2001 OR 2010 OR 2005 OR 2000 OR 2008 OR 2007 OR 2004 OR 2002 OR 2006 OR 2003 OR 2011)

1. #28 #27 AND #20

2. #27 #26 OR #25 OR #24 OR #19

3. #26 Title=(user* same experience*)

4. #25 Title=((user* same preference*))

5. #24 Title=((user* same need*))

6. #23 #20 AND #19

7. Refined by: General Categories=(SOCIAL SCIENCES OR SCIENCE & TECHNOLOGY) AND Publication Years=(2009 OR 2005 OR 2010 OR 2006 OR 2008 OR 2007 OR 2004 OR 2002 OR 2003 OR 2001)

8. #22 #20 AND #19

9. Refined by: General Categories=(SOCIAL SCIENCES OR SCIENCE & TECHNOLOGY)

10. #21 #20 AND #19

11. #20 #10 OR #9 OR #8 OR #7 OR #6 OR #5 OR #4 OR #3 OR #2 OR #1

12. #19 #18 OR #17 OR #16 OR #15 OR #14 OR #13 OR #12 OR #11

13. #18 Title=(consumer* same service*)

14. #17 Title=(consumer* same need*)

15. #16 Title=(consumer* same preference*)

16. #15 Title=(consumer* same experience*)

17. #14 Title=(customer* same experience*)

18. #13 Title=(customer* same preference*)

19. #12 Title=(customer* same need*)

20. #11 Title=(customer* same service*)

21. #10 Title=(performance same index)

22. #9 Title=(cross-cultural same approach)

23. #8 Title=(market segmentation)

24. #7 Title=(measuring same quality)

25. #6 Title=(measuring same service)

26. #5 Title=(measure same quality)

27. #4 Title=(measure same service)

28. #3 Title=(service same quality)

29. #2 Title=(respond)

30. #1 Title=(responsive*)

Web of Science: strategy 3 (public services: SERVQUAL)

#1 Title=(SERVQUAL) Refined by: Publication Years=(2009 OR 2004 OR 2005 OR 2008 OR 2006 OR 2010 OR 2007 OR 2001 OR 2003 OR 2002)