A pragmatic randomised controlled trial of the effectiveness and cost-effectiveness of 'PhysioDirect' telephone assessment and advice services for physiotherapy

Setting up the service

In order to establish a PhysioDirect service, each participating site needed to train sufficient physiotherapists to provide telephone assessments using the assessment software and to provide a suitable office. Apart from the training and set-up costs, no extra physiotherapist resources were used within the study – the time of the existing staff was reallocated within existing resources.

Training the physiotherapists

All physiotherapists operating the PhysioDirect telephone services undertook a structured training programme led by the physiotherapy service in Huntingdon which involved attending a 2-day course of teaching, demonstrations and observation of live calls. Each PCT trained about eight physiotherapists in order to provide sufficient capacity across the week (with some contingency in case of absence) and without any one physiotherapist having to spend more than half of their time working on the PhysioDirect service.

The PhysioDirect software (see below) was installed at each of the PCTs in the PhysioDirect trial prior to the trial physiotherapists attending the Huntingdon training so that following the training session the physiotherapists could practise and hone their skills. The Huntingdon PhysioDirect trainers were available by telephone throughout this period to provide advice if problems were encountered.

A visit to each participating PCT was undertaken by a PhysioDirect trainer approximately 2 weeks after completion of the training programme. The trainer observed calls and facilitated a problem-solving session. The trainer then listened to individual calls and, using a structured format, assessed each physiotherapist's competency to utilise the system safely and effectively. If physiotherapists did not reach the required competency at the first site visit a further visit to recheck competency was carried out approximately 6 weeks after the initial training. The competency check consisted of the trainer assessing 53 aspects of the telephone assessment process and completing a checklist indicating whether or not each aspect was performed to a satisfactory level. Each section was evaluated on a yes/no basis, with overall comments about performance, issues to be addressed and an agreed action plan if required. All physiotherapists had to be certified as competent to undertake PhysioDirect before they assessed patients in the trial.

Office facilities

Each site required a quiet office with at least two computers with installed software, and a telephone system with headsets for the physiotherapists, a fax machine and an answering machine. Ideally (but not necessarily), the receptionist would work close by in an adjacent office.

The PhysioDirect software

Shortly after its inception, the Huntingdon PhysioDirect system was translated into a computer-based system to assist the safe, efficient and effective delivery of the telephone assessment and advice service. The challenge during the development of the computerised system was to maintain sufficient structure to guide an effective and efficient assessment by prompting physiotherapists to cover all key aspects of the patient assessment, while being flexible enough to be responsive to presentations of individual patients and taking account of the lack of visual clues provided by a patient during a face-to-face consultation. The system had to be as simple as possible as physiotherapists undertaking telephone assessment are required to perform a number of skills simultaneously, including asking questions, visualising the patient and their presentation, analysing the patient's responses, formulating the next question and typing responses into the system. The solution was to include mandatory fields with dropdown menus and tick boxes for key aspects of the assessment, with text boxes provided to allow the physiotherapists to record responses to supplementary questions eliciting further information or clarification from the patient.

The software is a based on a program developed using Microsoft Access 2003 (Microsoft Corporation, Redmond, WA, USA), which first allows the patient's demographic details to be checked, then brief details regarding the patient's occupation, hobbies/activities and main problem for which they have been referred. When using the software the physiotherapist selects an appropriate assessment framework, depending on the area of the body affected. This ensures any relevant ‘special questions’ are asked that relate to that area. For example, if the patient is complaining of low back pain, questions regarding symptoms suggestive of cauda equina compression are included by the computer software in the assessment algorithm. This assessment, prompted by the software, includes the investigation and recording of the patient's presenting, medical and drug history, details of aggravating, easing and diurnal patterns and assists the physiotherapist in reaching a clinical diagnosis. An example screen from the system is shown overleaf.

Further information about the PhysioDirect software used in this trial can be obtained from Mrs Jill Gamlin, Physiotherapy Department, Hinchingbrooke Hospital, Hinchingbrooke Park, Huntingdon PE29 6NT, UK (e-mail: jill.gamlin@nhs.net).

Standardisation of PhysioDirect across the sites

All four physiotherapy services participating in this trial set up a PhysioDirect service following the same model of organisation and using the same assessment software as developed in Huntingdonshire (Figure 1).

FIGURE 1.

Example screen shot from PhysioDirect assessment software. Reproduced with permission from Cambridgeshire Community Services NHS Trust.

As far as possible, the PhysioDirect service was implemented in the same way in each of the four sites. The physiotherapists received the same training and were accredited in the same way, they used the same PhysioDirect software, and they used the same suite of advice and exercise leaflets. Furthermore, all recruitment and other research procedures were standardised in all sites. In this way, a high degree of consistency in how the intervention was delivered in each local service was ensured.

After the physiotherapists had been trained and assessed as competent to assess and advise patients via PhysioDirect, each physiotherapy site carried out a run-in period of at least 1 month to ensure smooth running of the service before patients to be included in the main trial were recruited. During this run-in period, all trial procedures of recruitment and randomisation were followed, making it possible to fully pilot the research procedures in parallel with the physiotherapy services gaining experience of the new way of working. Follow-up data collection for most outcome measures was also undertaken, although with less intensive use of reminders than in the subsequent main trial because of resource limitations.

Staffing the service

Based on advice from Huntingdon, it was decided that only experienced physiotherapists would be involved in providing the PhysioDirect telephone advice. Each service therefore trained several of their more senior staff who were on Agenda for Change Band 6 or above to conduct the PhysioDirect sessions. Details of the training and experience of these physiotherapists were collected at the training sessions. These are described in Table 1.

Because (in the context of a trial) only a minority of patients were randomised to PhysioDirect, the number of patients did not justify having a physiotherapist available to answer the telephone at all times in the week. Each site was asked to provide telephone sessions spread across a range of days of the week and at different times of day, to allow patients some choice about when to telephone. These opening times were included in the letter inviting patients to telephone the service. If patients telephoned at other times, an answerphone message asked them to telephone again when the service was open.

Process of the encounter

As soon as possible after consent to participate in the trial was received, patients were sent a letter inviting them to telephone an experienced physiotherapist for initial assessment and advice and explaining the times during which the PhysioDirect service was available each week. When a patient called the service, if a physiotherapist was not engaged in a call, they would be the first person the patient spoke to and the assessment would start immediately. If the staff were all engaged with other calls then the first person they would speak to would be a receptionist. The receptionist would take the patient's details and place them on a ‘call-back’ list along with information regarding when would be most convenient for this to happen. The physiotherapists would either field any calls arriving if they were free, or consult the call-back list for available patients.

The physiotherapist responding to the telephone call followed the computer-assisted assessment system to assess the patient and record the findings.

Process of care following the encounter

There were several possible outcomes following the initial telephone call:

• In some cases, at the end of the call the physiotherapist posted a relevant advice leaflet about self-management and exercises to the patient, inviting them to telephone back to report progress after about 2–4 weeks, if appropriate. If the patient telephoned back they could be given further advice or be booked for a face-to-face consultation if necessary.

• In some cases, the initial telephone call established that face-to-face assessment and/or care was needed, in which case this was arranged either by putting the patient on the PhysioDirect waiting list for face-to-face care or by arranging an urgent appointment if necessary.

• In other cases, the initial assessment established that physiotherapy was unlikely to be effective and the patient was given appropriate advice or referred to another service, and discharged from physiotherapy.

• In a few cases the initial assessment revealed that urgent medical intervention was required and these patients were sent to accident and emergency (A&E) with an accompanying fax detailing the need for attendance. If the medical intervention was not considered ‘an emergency’ then patients were asked to attend their GP, for example to obtain a radiograph.

The assessment process allocates patients to one of five categories. These categories and the subsequent management pathways are summarised in Table 2.

Providing PhysioDirect in the context of a randomised controlled trial

Separate waiting lists were held for patients randomised to PhysioDirect and to usual care, so that any impact of the PhysioDirect pathway on how long people waited for a face-to-face appointment could be detected. For the same reason, we sought to allocate physiotherapist time in proportion to the number of people who were randomised to the PhysioDirect arm of the trial. Based on experience in the pilot study it was possible to estimate the proportion of patients referred for physiotherapy who would be eligible for the trial and the proportion of those who would consent to take part. Using these data, and also taking into account that one in three of the eligible, consenting patients would be randomised to usual care, it was possible to estimate that approximately 20% of all patients would be allocated to PhysioDirect rather than usual care. Each site was therefore asked to allocate 20% of their total physiotherapist staff resources to patients allocated to the PhysioDirect arm of the trial, and within this proportion to allocate about half the time to telephone sessions and the remainder to face-to-face care for people in the PhysioDirect arm.

Patients randomised to the PhysioDirect arm could state at any time that they did not want to discuss their problem on the telephone but instead wanted to wait for a face-to-face appointment. This was allowed within the PhysioDirect pathway and did not mean that the patient needed to withdraw from the trial. It is important to note that the PhysioDirect ‘intervention’ was based on offering patients an initial telephone assessment with a physiotherapist, with face-to-face care available if necessary. The ‘intervention’ is not the telephone call, and a face-to-face consultation should not be equated with ‘usual care’. The PhysioDirect intervention is the care pathway, which can encompass both telephone advice and face-to-face care.

Sensitivity analysis

• Repeating the primary analysis adjusting also for any variables exhibiting marked imbalance at baseline to check that this did not influence the findings.

• Investigating the effect of missing primary outcome data using multiple imputation methods.

• Investigating clustering of outcomes by (a) practice and (b) physiotherapy service.

Secondary outcomes

• Assessing equivalence in clinical outcome using the MYMOP score at 6 months.

• Examining clinical outcome at 6 weeks using the SF-36 PCS.

• Comparison of the proportion of patients who ‘respond to treatment’ in each arm, in line with the OMERACT-OARSI recommendations,72 using the SF-36 physical function and bodily pain scales and the main problem score from the MYMOP.

• Analyses of the other secondary outcomes using similar approaches as described for the primary outcome.

• Investigation of process measures such as physiotherapy consultation rates, waiting times, physiotherapy DNA rates and consultation rates with other health-care services in the NHS and private sectors.

• Repeated measures analysis using an interaction term for time to investigate any divergent/convergent pattern in the SF-36 PCS over 6 weeks' and 6 months' follow-up.

Subgroup analyses

Appropriate interaction terms were entered into the primary regression analysis for SF-36 in order to conduct pre-specified subgroup analyses according to site of presenting MSK problem, patient age group, socioeconomic status and PCT physiotherapy service. As the trial was powered to detect overall equivalence between the groups rather than interactions of this kind, these analyses were essentially exploratory and would need to be interpreted with due caution.

Adjustment for baseline imbalances

Some imbalance at baseline was observed for the employment status variable, but including it as a covariate in the ANCOVA did not change the conclusion of the original primary analysis, namely no evidence of any difference between PhysioDirect and usual care on SF-36v2 PCS at 6 months (Table 16).

Effect of missing primary outcome data

Although only about 15% of the primary outcome data were missing in each trial arm, multiple imputations were conducted to determine what effect those data might have had on the primary analysis. The conclusion that there was no evidence of difference between the trial arms was the same with or without multiple imputations of missing data (Table 17).

Clustering effects

The possibility of clustering effects due to patients belonging to the same PCT or GP practice was explored through multilevel models with random effects attributed to those clusters. Each one of the models was compared with the original one (fixed effects) for its goodness of fit relative to the data by means of a log–likelihood ratio test or through its Akaike information criterion (AIC, lower value corresponding to a better fit), which accounts both for likelihood and parsimony of the model.

The only model for which there was some evidence of a better fit to the data than the one adopted for the primary analysis was a model accounting for a PCT clustering effect, but it maintained the conclusions of the original model (i.e. equivalence between PhysioDirect and usual care) and the clustering effect appeared to be very modest as indicated by an intracluster correlation (ICC) of 0.005 (Table 18).

PhysioDirect consultations

Patient-level data were collected for each appointment with PhysioDirect. These included the length of the telephone call and the grade of the physiotherapist making that call. Data about the length of the telephone call were obtained by download from the PhysioDirect assessment software, which recorded which physiotherapist conducted each telephone call, and the duration of each call, based on when the physiotherapist logged in to and out of the patient's electronic record.

A small time and motion study comprising observation of sessions at each site was conducted to determine how physiotherapists occupied themselves when assigned to PhysioDirect but not on the telephone to patients. Observation of this non-contact time was required to estimate the opportunity cost of that time. Time was categorised as consultation time, administrative time related to PhysioDirect appointments, administrative time related to face-to-face appointments, general administrative activity (e.g. dealing with e-mails) and work breaks. The aim was to undertake observational time and motion studies at points in the study when sites were expected to be fully operational, across a mix of day, time of day and location. Observations were undertaken over a 4-month period (September to December 2009) of four PhysioDirect sessions in Bristol, three in Somerset and two in each of Stoke and Cheshire, with physiotherapists ranging from Agenda for Change Bands 6 to 8a (representing the full range of grades of staff providing PhysioDirect sessions).

During the time and motion study it was noted that physiotherapists had to continue some administrative activities following each telephone call, such as collating information to send to the patient by post. The time spent on these activities was not captured within the times logged in to the PhysioDirect software. However, in Bristol (but not in the other three physiotherapy sites), physiotherapists recorded details of telephone calls in the same routine records as they recorded face-to-face consultations. Comparison of matched data about individual calls demonstrated that the mean duration of calls recorded in the routine record was 5 minutes longer than that recorded in the PhysioDirect software. It was therefore assumed that this represented the extra administrative time spent after each call, and furthermore that this could be applied to calls in the other three sites. Consequently, 5 minutes was added to the duration of call time for each call. In a small number of instances we noted that physiotherapists had logged in to patients records for a very short time, sometimes several times in quick succession on the same day. We assumed this was due to problems telephoning a patient, or physiotherapists accessing the records for administrative purposes. These short ‘calls’ were included in the costing, but 5 minutes was only added once for any given patient on any one day, even if more than one call from the patient was recorded on the same day.

Information about the capital requirements for PhysioDirect was obtained through dialogue with the study sites and research teams and relevant costs were obtained through these routes. The capital requirements for each site varied but in general the teams needed far less space than for treating patients face to face. Based on the data collected, capital requirements were allocated at 50% of the capital requirements for face-to-face physiotherapy sessions utilised by Curtis. 87

Face-to-face physiotherapy consultations

Data about face-to-face consultations were obtained from routine records maintained by physiotherapists at each site. These included the length of appointment, the grade of the physiotherapist seen and information about missed appointments. For most sites data were collected through the routine community databases. For Bristol and Somerset the ICS database was used; for Stoke the relevant database was Lorenzo. For Cheshire, where some sites did not use computerised systems, data were extracted from the patients' written notes.

Primary care consultations

Information was obtained from general practice records by researchers using a standard pro forma. Relevant consultations were identified as follows:

• If the MSK condition for which the patient was referred was mentioned, the consultation was recorded, even if this was not the primary reason for the consultation.

• If a patient had a MSK condition, but had a consultation about other problems and the MSK condition was not mentioned, these consultations were not recorded.

• If a repeat prescription was issued on the same date as a consultation (i.e. was probably issued at the consultation) but there was no mention of the MSK condition at the consultation then the consultation was not recorded (but the prescription was recorded along with other repeat prescriptions for MSK problems).

Primary care consultations were recorded by type (e.g. face to face, telephone, out of hours, home) and by type of professional seen (e.g. GP, nurse).

Prescribed medication

Information was obtained from practice records by researchers using a standard pro forma. Relevant medication was defined as that prescribed for MSK conditions. It was not feasible to distinguish between medication prescribed for the condition for which the patient was referred for physiotherapy and any other MSK problem and so all MSK medication prescribed was included.

Any prescribed item that appeared in the chapters of the British National Formulary (BNF) detailed below was assumed to be a relevant prescription and details were recorded:

• 4.7.1–4.7.2 Analgesics

• 10.1.1 Non-steroidal anti-inflammatory drugs

• 10.1.2.2 Local corticosteroid injections

• 10.3 Drugs for the relief of soft-tissue inflammation.

Medication was recorded by name, route of administration, quantity and strength.

NHS Direct/walk-in centre consultations

Information about visits to NHS Direct or to walk-in centres related to the condition for which the patient was referred for physiotherapy was collected through the 6-week and 6-month questionnaires.

Visits to accident and emergency

Information about visits to A&E services was obtained from the questionnaires administered at 6 weeks and 6 months.

Secondary care outpatient consultations

Information about secondary care outpatient consultations related to the condition for which the patient was referred to the physiotherapy service was obtained from the questionnaires administered at 6 weeks and 6 months. This included information about the clinic name or department visited and the number of appointments. Where questionnaire data were ambiguous, for example with regard to the specialty, information about these consultations was obtained from GP records.

Secondary care inpatient stays

Information about secondary care inpatient stays related to the condition for which the patient was referred to the physiotherapy service was obtained from the questionnaires administered at 6 weeks and 6 months. Information obtained directly from this questionnaire was just whether or not the patient had received an inpatient stay. Any patient who answered ‘yes’ to this question was then telephoned to obtain further details about the location and length of stay.

Travel costs

Information about travel costs associated with health-care consultations for the condition being treated by physiotherapy was obtained from the patient questionnaires. Costs associated with travel to the GP and to the physiotherapy services were collected separately and then applied to all appropriate consultations.

Over-the-counter medication

Information about over-the-counter medication associated with the condition being treated by physiotherapy was obtained from the patient questionnaires. Individuals were asked to specify whether or not they had purchased over-the-counter medication and the total costs associated with any such purchases. They were not asked for details about the type of medication taken or the dosage.

Expenditure on prescriptions

Information about whether or not individuals paid for prescriptions was collected through the questionnaire. Individuals were asked to specify whether or not they paid for prescriptions, and if so, whether they paid on a per-item basis, or by purchasing a 3- or 12-month prepayment certificate.

Out-of-pocket expenditure

Information about out-of-pocket expenditure associated with the condition being treated by physiotherapy was obtained from the patient questionnaires. Individuals were asked to specify whether or not they had purchased any equipment or devices for the condition, or had had to pay for any extra help in the home (e.g. cleaning, gardening, ironing) or had paid for any form of private care (e.g. private physiotherapy, chiropractor, osteopath, other complementary and alternative therapies).

Loss of earnings

Information about whether or not the individual had experienced any loss of earnings associated with their condition was obtained directly from the individual through the patient questionnaires.

Disability payments

Information about disability payments received as a result of the condition for which the physiotherapy was provided was obtained from the questionnaires administered at 6 weeks and 6 months. This included information about the type(s) of benefit(s) received and the total amount paid over the previous 6 weeks (for the 6-week questionnaire) and since the previous questionnaire (for the 6-month questionnaire).

PhysioDirect consultations

Methods used by Curtis87 in her estimation of a face-to-face physiotherapy appointment for a Band 5 member of staff were used in developing detailed costs for PhysioDirect appointments for each band of staff and in each location, based on median pay rates for each band from 1 April 2009 using the Agenda for Change. 88 Data on national insurance payments, overheads and capital overheads were taken from Curtis. 87 Capital overheads for telephone consultations were adjusted to 50% of the capital overheads for a face-to-face appointment to reflect the smaller amount of space required. Assumptions about the length of the working week and the number of weeks worked per year were also taken from Curtis. 87 More detailed information about salary and the associated superannuation costs was then utilised to obtain a cost for each band of staff from 3 to 8a.

In order to obtain a cost per hour of telephone contact, we combined the Curtis87 estimates of cost per hour with information from the computerised records of the PhysioDirect service once it had reached a steady state of operation. These records identified the proportion of time spent by physiotherapists actually dealing with PhysioDirect patients. To this was added data from the time and motion study, which identified activities undertaken during non-contact time, for example administration for face-to-face patients (rather than PhysioDirect patients) or general administration. All estimates were carried out on a site-specific basis.

Face-to-face physiotherapy consultations

Methods used by Curtis87 in her estimation of a face-to-face physiotherapy appointment for a Band 5 member of staff were used to develop more detailed costs for physiotherapy appointments for each band of staff and in each location, based on median pay rates for each band from 1 April 2009 using the Agenda for Change. 88 Data on national insurance payments, overheads and capital overheads were taken from Curtis. Assumptions about the length of the working week and the number of weeks worked per year were also taken from Curtis. 87 More detailed information about salary and the associated superannuation costs was then utilised to obtain a cost for each band of staff from 3 to 8a. Finally, information obtained from physiotherapy service managers about the level of non-contact time in each study site for each band of staff was used to obtain a site-specific cost for each band of staff.

Primary care consultations

Information about the costs associated with primary care consultations was primarily obtained from Unit Costs of Health and Social Care (Curtis87) on a per-surgery consultation or home visit basis as appropriate. Costs used are detailed in Table 29. Costs excluding qualification are used for all visits, as this service is not likely to have implications for the numbers or training of primary care staff. Costs for out-of-hours appointments with a GP were obtained using recent publications. 89

Prescribed medication

The value of prescriptions issued in general practice (both when prescribed during a consultation or as a repeat prescription) was based on an estimate of the full cost to the NHS and was made up of four elements:

• The basic price of the drug – from BNF90 – to cost the prescribed medication.

• An average deduction for discount. This depends on the total value of prescriptions dispensed from the pharmacy in a month and varies from a 5.63% deduction to a 11.5% deduction (May 2010). A mid-point deduction of 8.56% was applied to all basic drug prices.

• The professional fee: 90p per prescription (obtained from the Drug Tariff England and Wales: www.nhsbsa.nhs.uk/924.aspx).

• The container allowance: 3.24p per prescription (obtained from the Drug Tariff England and Wales).

NHS walk-in-centre consultations

The national evaluation was used to value a consultation at a walk-in centre,91 with values adjusted to 2009 values using the pay and prices index. 87 The consultation value used was £31.99.

Visits to accident and emergency

Information about the costs of visits to A&E services was obtained from the National Schedule of Reference Costs for 2009–2010, using the value for A&E attendances that do not lead to subsequent admission. 92 The value used for an A&E consultation was £103.

Secondary care outpatient consultations and procedures

Information about the costs of visits to outpatient consultations in secondary care was largely obtained from the National Schedule of Reference Costs for 2009–2010,92 supplemented by direct information from the Bristol Homeopathic Hospital. Values used are shown in Table 30 for those specialties that were utilised by patients included in the trial.

Information about the costs of procedures conducted in outpatient departments in secondary care was primarily obtained from the same source. 92 Values used are shown in Table 31 for those procedures that were utilised by patients included in the trial.

Secondary care inpatient stays

Information about the costs of inpatient hospital stays was based upon the healthcare resource group (HRG) for the particular admission. HRGs were coded using an earlier version of HRGs than the current version, because of the fuller information available, and so cost data were used from 2007 to 2008 and inflated to 2009 prices. Costs were obtained from the National Schedule of Reference Costs 2007–2008 (www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_098945). Values used are shown in Table 32.

Cost of telephone calls to the physiotherapy service

The cost of telephone calls to the physiotherapy service was borne by patients. These were costed at the standard British Telecom rate prevailing at the time, i.e. 5.25p per minute plus 9p connection charge. Patients using the face-to-face service are required to call to make an appointment; in these cases a nominal charge relating to a 3-minute call was included.

It is recognised that patients may have used mobile phones on a ‘pay as you go’ basis, making calls more expensive, or may have had contract mobile phones or belonged to landline telephone contracts with included ‘free’ minutes. However, no information was available about these factors and so the standard British Telecom call rate was used.

Travel costs

Information about travel costs was obtained directly from the patient questionnaires except for the case of car mileage. Information about mileage costs was obtained from the Automobile Association schedule and a value of 46p per mile was used.

Over-the-counter medication

The total cost associated with over-the-counter medication was recorded by the individual in the questionnaire.

Expenditure on prescriptions

Information about the cost of prescription charges was obtained from the Department of Health.

Out-of-pocket expenditure

Information about the costs of out-of-pocket expenditure was obtained directly from the patient through either the initial questionnaire or the supplemental telephone interview.

Loss of earnings

Information about direct loss of earnings experienced by patients because of their condition was obtained through the questionnaire.

Disability payments

Costs associated with disability payments were obtained directly from the patient questionnaire.

Prescribed medication

Information about the amount of prescribed medication in primary care, and the cost of that medication, is shown in Table 38. Medication provision through primary care was extremely similar across the two groups with mean medication costs for those allocated to PhysioDirect being very slightly lower than amongst those allocated to usual care.

Impact of mimicking an efficient service

If the PhysioDirect services had run during the trial in the more efficient way as seen in Bristol following the trial, the amount of non-contact physiotherapist time would have been reduced in all four centres. During the trial, the physiotherapists spent about 35% of their time during PhysioDirect sessions on the telephone or dealing with directly related administration. In Bristol, after the trial, this was increased to 57%. The cost implications of this are shown in Table 53.

Under this scenario, cost per patient in the PhysioDirect group is £14.53 less than under trial conditions and £2.11 less per patient in the usual-care group. The ICER is therefore lower, at £1045, and the NMB correspondingly higher at £127 (λ = £20,000). The probability of PhysioDirect being cost-effective at λ = £20,000 is 0.89; at £30,000 it is 0.90. This is illustrated in Figures 9 and 10.

FIGURE 9.

Cost-effectiveness plane for PhysioDirect (sensitivity analysis on cost of PhysioDirect mimicking an efficient service): complete case data, adjusted QALYs; showing 5000 bootstrapped replicates of the ICER.

FIGURE 10.

Cost-effectiveness acceptability curve for PhysioDirect (sensitivity analysis on cost of PhysioDirect mimicking an efficient service), showing the probability that the intervention is cost-effective at different levels of willingness to pay; complete case data, adjusted QALYS.

Impact of excluding hospital costs

Hospital costs accounted for 75% of all NHS costs yet only 19% participants reported using any secondary care. The main element of hospital costs was inpatient admissions but only 15 patients were admitted. The effect of removing hospital costs from the analysis is shown in Table 54 and Figures 11 and 12.

FIGURE 11.

Cost-effectiveness plane for PhysioDirect (sensitivity analysis excluding hospital costs), showing 5000 bootstrapped replicates of the ICER.

FIGURE 12.

Cost-effectiveness acceptability curve for PhysioDirect (sensitivity analysis excluding hospital costs) showing the probability that the intervention is cost-effective at different levels of willingness to pay.

This analysis is based on all patients for whom we had primary care and medication costs and QALY data: n = 1317 (59%). There is almost no difference in NHS costs between the two groups; the cost difference is due entirely to the difference in the cost of physiotherapy. The ICER is therefore reduced (£1084) and the NMB higher at £142 (λ = £20,000) and £217 (λ = £30,000). The probability of PhysioDirect being cost-effective at λ = £20,000 is 0.93; at £30,000 it is 0.94.

Imputation of missing data

The results of imputing missing NHS cost and QALY data are shown in Table 55 and Figures 13–16. The mean cost of physiotherapy was lower for all patients after imputation than for those for whom we had complete data (overall mean £72.60 vs £84.04). This was true of patients in both groups, meaning that our base-case estimate of the cost of physiotherapy is conservative. Conversely, the imputed NHS cost data suggest that the complete case analysis may underestimate the true cost: the estimated cost in all patients after imputation was £131.48 compared with the base case of £108.38. The effect of imputing NHS costs is more marked in the usual-care group, which reduces the difference between the two groups. The overall effect is a reduction of the incremental cost of PhysioDirect compared with usual care from £19.30 (complete cases) to £4.46 (imputed cost data).

FIGURE 13.

Cost-effectiveness plane for PhysioDirect (sensitivity analysis after imputing missing data, adjusted QALYs), showing 5000 bootstrapped replicates of the ICER.

FIGURE 14.

Cost-effectiveness acceptability curve for PhysioDirect (sensitivity analysis after imputing missing data, adjusted QALYs), showing the probability that the intervention is cost-effective at different levels of willingness to pay.

FIGURE 15.

Cost-effectiveness plane for PhysioDirect (sensitivity analysis after imputing missing data and with efficient service scenario, adjusted QALYs), showing 5000 bootstrapped replicates of the ICER.

FIGURE 16.

Cost-effectiveness acceptability curve for PhysioDirect (sensitivity analysis after imputing missing data and with efficient service scenario, adjusted QALYs), showing the probability that the intervention is cost-effective at different levels of willingness to pay.

Sensitivity analysis using imputed cost data in the ‘more efficient service’ scenario suggests a possible cost saving of £6.02 per patient.

The effect of imputing missing EQ-5D data is to reduce QALYs gained from 0.3299 (complete cases) to 0.3216 (imputed data). The difference is greatest in the PhysioDirect group, giving a reduced incremental QALY gain of 0.0020 as against 0.0067.

Using imputed cost and QALY data the probability that the PhysioDirect service is cost-effective at £20,000 per QALY is 0.66 and at £30,000 it is 0.67. The probability under the ‘efficient service’ scenario of the sensitivity analysis is 0.72 at both £20,000 and £30,000 per QALY.

Use of Short Form questionnaire-6 Dimensions rather than European Quality of Life-5 Dimensions

It is possible to use the data from the SF-36 questionnaire to estimate QALYs using the SF-6D measure. 96 In a further sensitivity analysis (not prespecified a priori but in response to the request of a peer reviewer) we compared the findings using the SF-6D rather than the EQ-5D to generate QALYs. The results are shown in Tables 56 and 57 and Figure 17.

FIGURE 17.

Cost-effectiveness acceptability curves: sensitivity analyses comparing use of EQ-5D or SF-6D to generate QALYs, with or without imputation of missing data, showing the probability that the intervention is cost-effective at different levels of willingness to pay. (a) QALYs from EQ-5D, n = 1124. (b) QALYs from SF-6D, n = 1124. (c) QALYs from EQ-5D, imputed data. (d) QALYs from SF-6D, imputed data.

In order to compare the findings using the SF-6D or the EQ-5D it is necessary to include only patients who have data on both outcomes, as well as complete cost data. This reduced the denominator from 1272 patients to 1124 patients. In this group of patients, the cost-effectiveness of PhysioDirect assessed using the SF-6D was slightly lower than when assessed using the EQ-5D (see Table 56). The probabilities that PhysioDirect was cost-effective at the £20,000 and £30,000 willingness-to-pay thresholds were 76% and 80%, respectively.

After imputing missing data, PhysioDirect again appeared to be slightly less cost-effective when QALYs were assessed using SF-6D rather than EQ-5D. The probabilities that PhysioDirect was cost-effective at the £20,000 and £30,000 willingness-to-pay thresholds were 55% and 57%, respectively (see Table 57).

Summary of sensitivity analyses

Table 58 summarises the findings from the various sensitivity analyses that were conducted to assess the robustness of the conclusions under a range of different scenarios and assumptions.

Several trends are apparent from the above sensitivity analyses. The incremental cost per QALY (ICER) for the intervention is low. However, because the benefits in terms of QALYs and the costs are both very small, the NMB is also small (and the confidence estimates overlap zero). Under the most optimistic scenario (based on the more efficient service as observed outside the trial and after imputing missing data), the intervention is both more effective and less expensive. Under the least optimistic scenario (using the SF-6D to generate QALYs and imputing missing data), the NMB from PhysioDirect is negligible. It is notable that in all scenarios the probability that PhysioDirect is cost-effective is > 50%.

Patient interviews

Interviews were carried out after the patient's discharge from their physiotherapy episode of care in the trial. The aims were to explore the accessibility and acceptability of the physiotherapy service, the influence of PhysioDirect and usual physiotherapy care assessment and treatment on patients' perceptions of their care and its impact on their MSK problem, and to explore whether and how patients perceived that services could be improved.

Purposive sampling based on age, sex, presenting clinical problem and participating PCT was used to generate a diverse range of patient perceptions and experiences. In total, 388 patients were invited to take part in the qualitative interview study between August 2009 and April 2010, with 82 agreeing to be interviewed and 57 interviews being conducted. The reasons why the number agreeing to be interviewed and the final number of interviews were different were due to the difficulties in arranging convenient times for interviews and saturation of the data, i.e. no new themes emerged after the first 57 interviews.

Table 59 provides a summary of the characteristics of patients who took part in interviews, showing their key characteristics according to each of the sampling criteria. Slightly more women were interviewed than men, and although the average age was 58 years, there was a large range of ages, with the youngest person interviewed being 19 years and the oldest being 87 years. More patients were interviewed in the group that was randomised to the new PhysioDirect service than in the usual-care group, including those who received some or all of the components of the new service, as well as those who were randomised to the new service but who never telephoned or contacted the service. This was deliberate in order to fully explore patients' perceptions of the new service. To facilitate comparisons between the new PhysioDirect service and the usual physiotherapy service, nine interviews were conducted with patients randomised to usual care. Interviews were conducted with patients from each of the four participating PCTs, and patients were sampled to ensure a breadth of MSK problems affecting different bodily regions.

Interviews with physiotherapists and their managers

The aim of the interviews with physiotherapists and their managers was to identify the salient issues involved with providing and managing the new PhysioDirect service and to explore its acceptability and implementability. A longitudinal design was used with the participating physiotherapists so that each was interviewed twice: once before treating patients in the PhysioDirect trial, and again when the trial had finished. The aim of the longitudinal design was to obtain insight into each physiotherapist's views, hopes and concerns prior to using the new service with patients and then in the second interviews to explore their experiences of using PhysioDirect with patients, following up on the key issues arising from the first interviews, while also exploring if there were any changes over time. The physiotherapy service managers were interviewed once after the trial had finished and these interviews explored the perceived impact of the PhysioDirect service, how it had been set up in each PCT, what it was like to manage the new service and other issues of importance to service managers which were perceived to facilitate or hinder its implementation or use.

From a total sample of 32 physiotherapists trained in the delivery of the PhysioDirect service, 16 physiotherapists were identified to be interviewed – four from each PCT. In each of the four PCTs, the key physiotherapy service manager overseeing the operational issues of the physiotherapy service was also invited for interview. The purposive sampling of physiotherapists was based on sex, clinical experience, whether they worked exclusively in the NHS or also in private practice, their experience of using telephone assessment systems previously and the circumstances under which they had became involved in the trial. All physiotherapists and managers who were invited for interview agreed and were interviewed.

Table 60 provides a summary of the physiotherapists and their managers who took part in the interviews, showing their key characteristics according to sampling criteria. Of the 16 physiotherapists interviewed across the four PCTs, 75% were female. They had a range of MSK outpatients experience ranging from 1 to 30 years, with an average MSK experience of 12 years. The majority (81%) worked exclusively in the NHS and 62.5% of the physiotherapists had previous experience of using telephone assessment within physiotherapy. Half of the physiotherapists reported that they had been invited to participate in the PhysioDirect trial by their managers, while the other half reported that they had volunteered to deliver the new PhysioDirect service in the trial. Of the four managers interviewed, 75% were female and had a range of experience of managing MSK services from 1 to 8 years, with an average of 4 years.

Interviews with GPs and commissioners

The aim of the GP and commissioner interviews was to explore some of the key organisational and contextual issues that might influence the acceptability and implementation of the new PhysioDirect service. In each PCT, GPs and commissioners of MSK services were interviewed after the trial had finished but before the results of the trial were known. The sampling criterion for the commissioners was based upon whether they had a clinical or non-clinical background. A total of eight commissioners, two from each PCT, were identified and invited to take part in the interview study, of whom four (one from each PCT) responded and agreed to participate. GPs based in practices that had participated in the trial were invited to take part in the trial, based upon two key criteria: the PCT and their referral patterns to physiotherapy (high vs low referrers to physiotherapy). Eighty GPs were sent postal invites of whom 26 replied, 15 agreed to be interviewed and eight GPs in total were eventually interviewed (two from each PCT area). The GPs and commissioners were interviewed at their place of work.

Table 61 provides a summary of the GPs and commissioners who took part in the interviews, showing their key characteristics according to the sampling criteria. Of the eight GPs interviewed across the four PCTs, five were male. They had a range of general practice experience ranging from 10 to 30 years with an average experience of 20 years. Of the four commissioners interviewed, one was female, three had a non-clinical background and overall they had a range of experience of managing MSK services from 1 to 5 years with an average of 2 years.

Recording and management of data

All of the interviews were digitally recorded. The audio recordings were stored on the researchers' password-protected drive and the audio files on the Dictaphone were then erased. The audio files were transcribed in full and saved in Microsoft Word (Microsoft Corporation, Redmond WA, USA) format and then anonymised. The transcripts were saved to both a password protected computer and the Framework software programme. Each case was given an identification number.

Cost of the telephone call

The cost of the telephone call, or its perceived cost, was clearly an issue that some patients considered in deciding whether or not to use the PhysioDirect service and whether or not to telephone back after the initial telephone assessment and advice. The interviews revealed that some patients telephoned PhysioDirect from their mobile telephones. Other patients, when prompted on how they thought the service should be improved for the future, suggested that the telephone call should be free for patients. Patients had been told in the trial information leaflet that it was likely to take around 20 minutes to be assessed by the physiotherapists over the telephone. One clear recommendation for the future is that patient information should make it clearer that the physiotherapist can call the patient back at a convenient time, therefore transferring much of the cost of the telephone call to the PhysioDirect service.

I found it perfectly clear and they said this could take up to 20 minutes or a bit longer or whatever and it came down to funds again for phoning.

Patient 20,645 (randomised and did not ring)

Accessed private physiotherapy instead

A number of patients interviewed had access to private health care and some used their private health-care cover to provide physiotherapy instead of contacting the PhysioDirect service.

I was on holiday for three weeks and then I came back and got in contact with the firm and they said they'd pay and that's when I thought I don't need to bother with the national health if they're going to pay for it.

Patient 10,556 (randomised and did not ring)

Perceived lack of effectiveness of physiotherapy

Some patients' reasons for not contacting the PhysioDirect service were centred on the perceived lack of efficacy of physiotherapy for their MSK problem; therefore, they did not feel the need to contact the PhysioDirect service, as they did not believe that physiotherapy would aid their recovery. This is likely, however, to be a similar reason for non-attendance within usual physiotherapy care.

I ended up with the telephone thing, which, well, I would probably eventually have got around to ringing up and saying I didn't think physiotherapy was what I needed but I didn't get around to doing that.

Patient 20,469 (randomised and did not ring)

Lower perceived value of the PhysioDirect telephone call in comparison with face-to-face care

Some patients explained that although they did not contact the PhysioDirect service they would have made the effort to attend a face-to-face appointment if they had been randomised to receive usual care in the trial. This is important, as it appears there is a difference in how patients view physiotherapy face-to-face contact compared with telephone contact. Some patients felt that it was worth making an effort to obtain face-to-face physiotherapy but this was less true for a telephone contact. This seemed to suggest they placed lower value on the PhysioDirect telephone call as a method of accessing physiotherapy care.

So obviously, I can do things like that because they fitted in. Why did that one fit in? I was able to fit in it but I can't make the phone calls. Every time I phone them, sorry, they're not in.

Patient 40,770 (randomised and did not ring)

If you had a face-to-face appointment?

Interviewer

I would make the effort to do it.

Patient 40,770 (randomised and did not ring)

And:

You know that there's somebody sitting there waiting for you turn up and you don't or you're cancelling your appointment that somebody's gone to the trouble to make for you whereas a phone call's just a phone call and it can be any time and anywhere.

Patient 20,469 (randomised and did not ring)

Patients with other pressing priorities

Further reasons to explain why some patients did not ring the new PhysioDirect service were their other health problems or ‘busy and complicated’ lives. In these cases, patients did not prioritise contacting PhysioDirect to have their MSK problem assessed. Other more pressing issues in their lives, for example caring for disabled relatives or other health complaints, were of greater priority than the telephone call to the PhysioDirect service.

I got fed up and I just had so much to do and it was just another phone call, another, you know what I mean?

Patient 20,318 (randomised and did not ring)

I did fill them in and send them back and then my anxiety started to take over again really bad so, my, sort of, pain problems and stuff weren't so important.

Patient 11,465 (randomised and did not ring)

A quick efficient service

Patients' narratives were full of descriptions of how the PhysioDirect telecare service was experienced as quick, efficient, convenient and preferable to the long waits for physiotherapy face-to-face care. In a previous study, Hills and Kitchen103 showed that waiting times are important with respect to patients' satisfaction with treatment. In the interview data, it was clear that the acceptable features of PhysioDirect centred on the service being perceived as quick, efficient and convenient.

Well, the thing I liked about it really, it didn't take long for them to get in touch with me.

Patient 10,552 (telecare only)

And:

It's a very good service, very quick, very prompt, very thorough.

Patient 30,196 (telecare only)

A good first step

Patients also described the PhysioDirect service as a good ‘first step’ to access physiotherapy services. The interviews provided an insight that PhysioDirect was perceived by patients to be the first stage in a physiotherapy ‘staged-care approach’, where levels of contact by the physiotherapist would increase depending on the complexity of the problem. Patients understood that there was another step or ‘stage’ if they needed it and saw PhysioDirect as a service within a larger, existing system.

It's a good first, yeah, I think it's a good starting point.

Patient 10,104 (telecare only)

I was fine because as I say, it's like anything. You've got to try something to see if you can resolve the problem and it's easier to resolve it in the simplest ways rather than go into the extreme ways, because maybe you don't need to go to the extreme, you can do the first stage first and that maybe resolves it. Or maybe you might have to go to the second stage and that resolves it.

Patient 20,353 (telecare only)

Helpful physiotherapist

From the patients' perspective, the attributes of the physiotherapist providing the PhysioDirect service were very important. Patients interviewed were very complimentary about the physiotherapists in the PhysioDirect service, describing them as thorough in their assessment, polite and helpful.

I found her very clear, thorough and very pleasant. She was very pleasant. She really was good.

Patient 10,290 (telecare only)

Other patients saw the physiotherapist as the ‘knowledge giver’, able to advise, provide information on the MSK condition and give time frames for the patient to phone back if things were not improving.

I did find the physiotherapist themselves very good and better able to explain to me what I needed to do and what they thought was causing the problems.

Patient 31,575 (telecare + face to face)

Communication difficulty

Some patients found it difficult to describe their symptoms over the telephone, especially without the apparent benefit of physically pointing to and demonstrating their problems to the physiotherapist in person. It is acknowledged that describing pain is often challenging for patients. 107 However, the specific difficulty patients experienced appeared to relate to pinpointing and describing the exact anatomical areas of their MSK pain. Most patients also appeared to find it challenging to explain the movements they found painful or difficult to achieve, in order to respond to the physiotherapist's questioning during the telephone assessment of their MSK problem. Reliance upon their own descriptions and difficulties in their interpretation of what the physiotherapist meant in the assessment of their problem created areas of doubt for patients.

It's a bit difficult to physically explain what you're doing down the phone.

Patient 20,353 (telecare only)

And:

Yeah, I found it a bit, quite difficult, because it's hard to explain isn't it, even, not just on the phone but to anybody. I mean, the pain I was in was really, really bad, so, I would have preferred to have saw somebody, you know, because when you try and explain the areas or, you know, where the pain was, which it goes all the way down, down to there, it's a bit hard to describe on the phone, so, that's when I would have liked to have seen somebody.

Patient 10,253 (telecare only)

PhysioDirect as an impersonal service

PhysioDirect telephone care was seen by some patients who experienced it as impersonal. The words ‘not personal’ were used by patients in their narrative when describing the features they disliked about PhysioDirect. In comparison, none of the usual care patients described the service they received as impersonal. There was evidence that some patients felt they were unable to create a therapeutic relationship with the physiotherapist in PhysioDirect and felt that it inhibited any continuity of that relationship. Some also found that the telephone call was easily forgotten.

I just mean somebody who, you know, I just feel that this PhysioDirect, you are just a number on a piece of paper, but, like I say, if you rang me back in a month's time and actually had a conversation with me, I would feel that that was more personal than me having to ring and getting somebody completely different every time. That's all.

Patient 30,272 (telecare only)

And:

It didn't feel very real, a phone call is an odd, very easily dismissed from my day-to-day life, I've done the phone call and goodbye whereas if you have an ongoing relationship, an ongoing face to face, it becomes more part of your daily life.

Patient 20,081 (telecare only)

The role of the referring general practitioner

Looking to the future, patients expressed mixed views about the role of the GP in referral to physiotherapy. Some patients suggested that direct access to a physiotherapist without the referral from a GP would be more suitable for them. There was evidence in the patients' accounts that some would prefer not to involve the GP in the management of their MSK condition but this mostly centred on face-to-face care rather than the telephone-based care, and this was mostly from patients who had previous experience of physiotherapy.

In an ideal world, how would, how would you like to see a physio service?

Interviewer

Walk-in.

Patient 41,454 (usual care)

Other patients thought that the role of the GP was pivotal in the decision to have physiotherapy treatment, although this might be explained by patients' current experience where, in most services, patients must be referred to physiotherapy by their GP.

It's difficult because you can't, to access physio over the phone, you don't, you have to go through your GP first anyway, so, you need to have spoken to your GP first. Well, that's how I think you should do it.

Patient 20,053 (telecare only)

Future technologies

The uses of the internet and video or camera technologies were also mentioned in the patients' accounts when asked about services in future, with some patients clearly looking at health information online before and after the use of PhysioDirect.

I've printed off things, I've had information sent me, I've watched videos of chiropractors talking about whiplash.

Patient 10,253 (telecare only)

Patients also described how other technologies could work in future PhysioDirect services, creating innovative ways of amalgamating the process of providing treatment advice, along with the use of telemedicine and the internet. They could envisage integrated PhysioDirect services directing patients to approved websites where they might find helpful and evidence-based information about their condition and available treatments.

I suppose you could always have a website which contains the booklet information which people can download.

Patient 10,104 (telecare only)

And:

I suppose your service could be done on a one-to-one on broadband but that's only if everybody had the webcam, you know. The way things are going, I mean, these things come out with webcams on them.

Patient 20,353 (telecare only)

The use of technology and health highlighted by Wyatt and Sullivan108 suggests that the role of e-health for individual patients offers opportunities for prevention, choice, home-based care and chronic disease management, widening the access to health care for most patients. This view of an increased shift towards information for patients is also advocated by the Department of Health. 16,109

Patient visualization

The second round of interviews provided some insight as to how the physiotherapists adapted their way of working in order to deliver the PhysioDirect service. The majority of physiotherapists interviewed used techniques of visualisation in assessing patients in order to overcome the lack of visual feedback in the assessment process:

I think you've just – you've just got to totally switch off from everything else that's going on and put yourself in the – in the shoes of the patient, you've got to kind of imagine that they're at the end of the phone, they're, you know, sat on the bottom step or, you know, what's – what's going on and – and you're just trying to totally focus on what that patient is, you know, describing to you. And imagine almost as if you are in a cubicle with that patient. And chatting to them so that, you know, you're not doing anything different to, you know, you're still using the same skills that you would use if you were seeing somebody face to face. That's quite, quite difficult.

Physiotherapist 13, time point 2

As part of the PhysioDirect training package, physiotherapists received a published article,110 which described how NHS Direct nurses visualise their patients while triaging them over the telephone. It is clear from the data that the physiotherapists used similar techniques while assessing their patients over the telephone. However, it is less clear whether or not they would have visualised their patients in their assessments if they had not been trained to do so. On the one hand, this evidence may suggest that the training for the PhysioDirect system worked well, but, on the other hand, it could highlight that physiotherapists intrinsically also use these techniques. Nevertheless, there is evidence to suggest that some physiotherapists visualise patients while assessing them over the telephone.

Joining in the assessment

Physiotherapists described how they joined in with the assessment with the patient, moving their own arms and legs around to test if their explanations, questions and descriptions were accurate and comprehensible.

But you know, you've just got to try and switch off from, you know, if – if I'm moving my arm up and I know that a physio is laughing at me as I'm, you know, you've just got to shut everything else and just, you know, sort of stare into space and – and get on with it. And I think what you would do is try and sort of move your arm or move your leg because you're trying to think well how can I explain this to the patient so that they understand what I'm asking them to do? So it's almost like you've got to join in as well so that you know your instructions are clear.

Physiotherapist 13, time point 2

Enhanced communication skills

The physiotherapists described the need to enhance their communication skills in order to deliver the PhysioDirect service. This included the need to facilitate patients with their questioning in sufficiently lay language in order to ascertain the appropriate information from the patient:

I think it normally takes me quite a long time, because obviously that's really important for exactly you know, where you're talking about and obviously you know, different people have different perceptions of where their shoulder is, you know, is it up by the neck or is it kind of, down by the arm or you know, your exact and you know, you do need to know quite specific areas when you're kind of, talking about different pathological structures, so I think that is quite difficult over the phone.

Physiotherapist 10, time point 2

Other physiotherapists described focusing and listening attentively to the patient which they identified as a key component in the assessment in order to achieve a correct diagnosis:

From the patient and I think from that information, provided you're listening carefully and not just assuming answers, provided you do listen really carefully and that, you, you can I think our diagnoses were fairly good.

Physiotherapist 06, time point 2

Thus, physiotherapists' telephone assessments of patients combined a mixture of enhanced questioning, listening and visualisation techniques, as well as simultaneous participation in movements along with the patient.

Generalised rather than individualised treatment

The physiotherapists in their initial interviews expressed concerns that the PhysioDirect service might limit them to providing only rather generalised rather than individualised treatment to patients with MSK problems, given their inability to observe patient posture and specific movements.

The other ones that are involved on the phones, from the peer discussions that we've had, they seem to be feeling similar things to me, from what I can gather. One of them said to me that it's more of a generalised, analysis at the end because you can't look at which specific movements that are stiff or week, you're making a more general feeling that yes, this is probably a stiff knee or a weak knee and therefore, you're more likely to be giving them general exercises.

Physiotherapist 09, time point 1

And:

Yes, and I don't like the idea of being standardised practitioners because I don't think we are and I don't think we should be. I think we should have the scope to treat people differently, according to their individual, not just their, yes, their clinical needs but also their mental needs and the whole attitude of the patients, they'll all be slightly different. I just feel that we're, kind of, being squeezed into boxes and you're got to fit into the box where you're not going to get anywhere.

Physiotherapist 21, time point 1

There was some evidence in the second round of interviews that this concern remained and that the PhysioDirect service appeared to restrict their practice to the provision of generalised advice. Physiotherapists as a profession are autonomous practitioners, who have the knowledge to make clinical decisions and deliver the most appropriate intervention to individual patients. 111,112 Therapists felt that PhysioDirect standardised and protocolised care might limit their professional autonomy. This was, on the whole, in reference to the information leaflets provided within the PhysioDirect service in the trial, with some physiotherapists feeling restricted in what treatment they could advise.

It would have been better if I could, give, if there was flexibility for me to just give the exercises I wanted to do, so I'd find that, I get a leaflet, there is a couple of things I'd like to give to the patient, one was in one leaflet, one was in the other, which was obviously difficult, so there's things that I probably thought clinically, I want them to do this, but the leaflet that was more relevant to them did not contain that.

Physiotherapist 22, time point 2

PhysioDirect effective at providing self-management advice

Many of the physiotherapists felt that PhysioDirect was an effective medium through which to provide patients with advice about self-management, for example patients with osteoarthritis (OA).

I think the OA knees like those sort of situations. The osteoarthritic patients generally who need more range of movement and strengthening exercises just generally those patients and I think really what I said before, those patients who need more self management; those people that maybe are on the mend but just need that extra push to get better. Those sort of patients are ideal for, are ideal for PhysioDirect.

Physiotherapist 1, time point 1

The second round of interviews highlighted that physiotherapists felt that as patients were ‘only’ receiving PhysioDirect telecare, they felt obliged to spend longer on self-management advice during telephone calls than they would normally do in usual care:

The telephone assessment almost feels like it's more of, makes a bigger proportion of your treatment because you can't do any manual treatment perhaps. You're giving them advice and exercises and that's the package and so I think maybe I, I put a bit more emphasis on those sort of self treatment approaches then perhaps I would have done face to face. It feels that way. I'm not sure if that's accurate but I feel I gave good general self management advice.

Physiotherapist 29, time point 2

Impact on the physiotherapist and patient relationship

In the initial interviews, physiotherapists expressed concerns regarding how they were going to communicate with the patients over the telephone, and they perceived that some aspects of physiotherapy treatment would not be easily communicated over the telephone. They felt that physiotherapy ‘is more than just words’, rather that the relationship with the patient is integral to effective physiotherapy.

I think it's more the rapport that you have to build up with a patient, to get them to comply with what's going on. You need to get some trust there and if you keep on talking to different people, it's hard to build up any trust.

Physiotherapist 22, time point 1

And:

I don't know whether it's just because of the ability to communicate with them, that I'm not sure they would understand just with the spoken word, whether they needed me to see or touch or do because that's what I find with physio, it's not so much what you do, it's the way in which you do it and you can't see it over the telephone.

Physiotherapist 11, time point 1

These concerns were realised. In the second round of interviews, physiotherapists felt that the PhysioDirect service tended to inhibit the normal therapeutic relationship that develops between the physiotherapist and patient. They perceived that there was an inability for patients to fully trust them and thus disclose information over the telephone. The lack of good rapport between the two parties may mean the physiotherapist might be less likely to identify what might be specifically causing the patient's problem.

But when you see a patient face to face, you build up a bit more of a rapport, you get to know them a little bit better, you know how to approach it better, and you can't do that on the phone, so part of your experience, your, what you're getting paid more to do, is lost, so.

Physiotherapist 22, time point 2

PhysioDirect impairs continuity of care

In PhysioDirect, patients often spoke to a different physiotherapist than the one that had initially assessed them over the telephone, when they called back to discuss progress several days or weeks later. The physiotherapists felt that this would negatively impact on their therapeutic relationship with the patient, and reduce the continuity of care with patients.

It's just nice, you can build up a really nice rapport with patients and I like that, whereas, you wouldn't necessarily get that over the phone because perhaps it would be more of a one-off, or, you wouldn't necessarily be the person taking the call off the same patient, if they phoned back. So, in terms of, maybe, kind of, consistency, I enjoy being there physically for my patients and being there in person.

Physiotherapist 07, time point 1

In the second round of interviews, there was some evidence that this reduction in continuity of care in the PhysioDirect service made developing a good rapport with patients more challenging:

They're loads and loads of things around this issue that might be influencing their physical problem. So that only comes out sometimes at first contact to a certain degree. If you then get that patient back in, they'll often tell you a little bit more and you can, you can get a better insight as to exactly what is going on and if you are treating that patient face to face, I think that is, is easier. On the telephone, you're a stranger; you're just a voice and they don't maybe want to give you things in as much detail. So you might be trying to delve but they're putting the anchors on and saying this is all I want to tell you at the moment.

Physiotherapist 04, time point 2

PhysioDirect disengages patients

Evidence from the first round of interviews with physiotherapists highlighted their concern that the PhysioDirect service might disengage patients. It appeared that some physiotherapists had concerns about patients not contacting the PhysioDirect service and not calling back if their MSK problem did not improve:

My concern is that sometimes patients, you speak to the general public about seeing the doctor and they say I spoke to the doctor and he said this, this and this but it hasn't really helped and I didn't go back. And you say why didn't you go back. Oh well, what he gave me for it, it didn't help. And, I think one of my concerns would be it empowers the patient, it puts the ball in the patient's court, but actually, if they think I tried those exercises, I had this assessment on the phone and we took half an hour, but it didn't really help, they might write off physio, they might say it didn't help and I'm not going down that route thank you, I want to go down a different route and we'll never know.

Physiotherapist 21, time point 1

This was also a concern in the second round of interviews and although many physiotherapists were surprised at how many patients they could successfully manage over the telephone, they were also surprised at how few patients seemed to recontact the service. The over-riding concern appeared to be that PhysioDirect might therefore disengage patients from physiotherapy through disappointment with the new service:

I've had two in a year. So no. They're not my own. But whether my patients call back on a different day and speak to someone else, there's no way of knowing that. So … but on the days that I work I've only had two people … we were only saying this the other day because I can't ever remember having to get into the right bit to write the comments when patients call back.

Physiotherapist 32, time point 2

And:

So I've been staggered so I'm taking from that that, that what we are giving them is actually proving to be quite helpful or they've been so disappointed in the advice given that they've decided not to bother coming back. I don't know. That's what's going to be really interesting.

And how would you feel if that was the case?

Interviewer

That they were so disappointed that they didn't? Very disappointed. Very disappointed.

Physiotherapist 29, time point 2

Evolving professional identity

Initially, one of the main concerns for the physiotherapists was that, if they were to spend the majority of their clinical time on the telephone assessing patients, they might become deskilled with respect to their physical and manual assessment and treatment techniques. They clearly saw these skills as key to their own professional identity.

I'd hate to be a physiotherapist who only talked to people on the phone, that would be terrible, you know, you'd really deskill and obviously, that wouldn't be good, but, I think it would be, if it is helpful, that's going to be good.

Physiotherapist 10, Time point 1

In the second interviews, they still expressed the desire not to spend the majority of their working week delivering the PhysioDirect service:

I can see the benefit of a mixture but for me, I wouldn't want to do more than say two sessions a week on the phone. I'd prefer my face-to-face contacts.

Physiotherapist 07, time point 2

And:

I wouldn't want to do any more than that, and I couldn't do more than half a day in any one stint, you couldn't do a day of that, it would be very difficult, I wouldn't do that as a job, I wouldn't be happy with that. If that was the way physio went, I'd go and get another job.

Physiotherapist 22, time point 2

Another reason as to why the physiotherapists did not want to spend the majority of their time on the telephone was that the PhysioDirect assessments were perceived to become rather monotonous and they felt they were repetitive if there were many patients to assess during the opening times of the PhysioDirect service:

By the end of the day, I'm sick of saying the same questions, you know, because you're kind of repeating the whole, this format of questions if you're kind of, you know, just repeating those continuously.

Physiotherapist 10, time point 2

Another more subtle issue related to the acceptability of PhysioDirect from the initial interviews was that some physiotherapists did not perceive that physiotherapy could be delivered over the telephone.

PhysioDirect is a different system I think you're doing it automatically, as a physio. You're making note of things automatically and it might be, you know, it might come out later on, but, you can't put that in a subjective assessment on a telephone consultation.

Physiotherapist 04, time point 1

Do you think there's any way you could get around that, other than bringing them in?

Interviewer

No. It's not, that's the problem, you can't compare it with like value to a face-to-face consultation.

Physiotherapist 04, time point 1

It appears that the professional identity of physiotherapists is not only influenced by the type of physiotherapy they deliver, but also there is evidence from the second round of interviews that it is linked to continuity of care. The physiotherapists reported that feedback from patients (whether their symptoms had improved or worsened) enabled the physiotherapist to reflect on their decisions and further develop themselves as a practitioner. PhysioDirect was viewed as limiting this feedback, as few patients seemed to call back to discuss how their problem was progressing.

You don't get the closure and the feedback loop and so it's difficult, more difficult to learn as a clinician using PhysioDirect about how your patients are doing 'cos you don't get the feedback loop whereas when we see them face to face and we say I want to see you again in a month to see how you're getting on, we get that feedback of how they're progressing and that helps us learn and develop.

Physiotherapist 21, time point 2

And:

I think the big concern is if all you did was assess on the phone, if that was what the whole profession did in the NHS, you know I think yes there'd be a real issue there with professional clinical development so it is very, potentially a very useful and effective service for managing resources and cost effectiveness and may have acceptable outcomes. If it was all you did, I think there would be real big shortfalls. Yes, long term it would be very damaging probably to yes, to the skill level within the profession.

Physiotherapist 21, time point 2

PhysioDirect physiotherapist skill set

In the first round of interviews, some physiotherapists recognised that there were certain additional skills they needed to assess patients well over the telephone. The additional development of these skills was suggested as a reason why some physiotherapists took part in the PhysioDirect trial (i.e. they hoped that their subjective assessment skills and communication skills would improve):

There's skill, obviously, in trying to get the information out of someone who struggles to relate and that's obviously part of the physio's ability to take that information. So, you know, for the patient, the experience is going to depend a lot on the person they're speaking to and how effective they are at getting that information and understanding what the patient's trying to tell them. So, there's a two-way thing, if the physio misses the point completely what the patient's trying to say, they're going to get the wrong diagnosis and the patient's going to feel that they've not really listened or understood.

Physiotherapist 23, time point 1

In the second round of interviews, they had clearer ideas as to what skills PhysioDirect physiotherapists should have. There was consensus from both the managers and the physiotherapists that the PhysioDirect service should be delivered by an experienced physiotherapist with at least Band 6 MSK experience (note: the Agenda for Change pay band is the tariff that NHS staff are allocated to on the basis of their knowledge, responsibility, skills and effort needed for the job). The skills that the PhysioDirect physiotherapist needed were perceived to centre on the ability to listen, visualise and make quick decisions.

A kind of an upgrade Band 6. So … but, but then saying that, I mean PERSON_H, who is in the study, he, he did very well and he's probably mid, mid kind of grade. I mean somebody with a lot of musculoskeletal experience. I mean we certainly wouldn't be looking at a Band 5 junior. They need to have had a good three, three years minimum.

Manager 3

PhysioDirect training

The physiotherapists involved in the trial undertook a 2-day training package, which was held in Huntingdon, Cambridgeshire, before the start of the trial. The experience of the physiotherapists using the PhysioDirect software consists of two subthemes: training at Huntingdon and familiarisation with the PhysioDirect system. The physiotherapists interviewed were positive about their training at Huntingdon and thought that the training experience was beneficial, as it helped them to conceptualise how it might be operationalised within their own departments.

That was really good, really well delivered and really good to see the software in action and have a go. Yes, really, really helpful. I think without that, we'd be struggling a lot more to get our head around the software and stuff. It was essential, I would say.

Physiotherapist 21, time point 2

Familiarisation

This subtheme also explores how the physiotherapists became familiar with the computer software program and explores the difficulties that some physiotherapists had of becoming more confident to assess a patient over the telephone without the visual component of an traditional face-to-face assessment. Familiarisation with the computer software and being able to clinically reason over the telephone were other themes that emerged from the physiotherapists' views and experiences of the PhysioDirect training.

I think there's two sides to the practice; one is getting your head around operating the software whilst talking, which some people call multi-tasking, some people in this Trust suggest there are gender differences around that but, so one half of the challenge has been getting used to taking down the assessment in electronic form whilst doing your subjective examination and the second half has been actually the clinical challenge of working just on subjective information and not being able to do any clinical testing, not being able to see the patient, observe anything, and so trying to clinically reason and be, have a good level of certainty in your impression at the end, so, it's basically clinical reasoning in a different way, with less information.

Physiotherapist 21, time point 1

The second set of interviews with the physiotherapists suggested that they did get used to this way of assessing patients and reported that their confidence eventually improved with practice using the PhysioDirect service with patients:

So obviously that was challenging for a while, but we'd soon get into a routine and remember to ask it and, why you're asking it, to do it.

Physiotherapist 22, time point 2

And:

So … but yeah, you just get used to when to ask, when it best fits into the assessment. But that just comes with doing it. The more you do it, the more slick you become.

And when did you start to feel slick?

Interviewer

After about 6–8 weeks, probably.

Physiotherapist 32, time point 2

Length of telephone call times

A key physiotherapist concern was whether or not they would be able to complete a patient clinical assessment over the telephone within 20 minutes. This recommendation of a 20-minute telephone assessment time from the training team at Huntingdon caused some anxiety and unease amongst physiotherapists, as they felt it was too short to be able to glean all of the information from the patient required to make an accurate diagnosis. Some suggested that although this was the ‘target time’ for a telephone assessment, they would not let their clinical judgement be affected by a time frame:

If I can get all the clinical information out that I need to make an appropriate decision then yes. I haven't got to that stage yet, but, I may do, I don't know. I'm quite open-minded about it really. I'm not going to compromise my clinical judgement because of the time but I'm prepared to be not exactly proved wrong.

Physiotherapist 09, time point 1

Data from the PhysioDirect IT system and the qualitative interviews suggests that some physiotherapists achieved this guide of 20 minutes per telephone call during the trial while others did not. The average call time was 24 minutes overall. The second interviews also suggested that as physiotherapists became familiar with the process of assessing patients over the telephone, their call times decreased. Reasons as to why some did not were associated with not feeling sufficiently familiar with the PhysioDirect system owing to the relatively few patients telephoning the service while they were on duty and the perceived inadequacy of their own computer skills.

Computer literacy

Some of the older physiotherapists involved had reservations about their competence to use the IT systems within PhysioDirect and it was clear that they felt they were not sufficiently computer literate to assess the patient simultaneously with inputting assessment findings and information into the PhysioDirect software. The physiotherapists who had these concerns tended to be those who were older:

I didn't realise, when I initially signed up for it or said that I would be interested in doing it that it was going to be so computer based, because I'm an old lady and I tend to be handwriting based as opposed to screen and at the moment, I'm still finding, I mean, I've got used to it now but I'm finding it very hard to work on the screen instead of handwriting the notes, partly because I've used the same sort of assessment forms for donkey's years and I always ask my questions in a very set order so I know that I haven't missed anything out and it doesn't come up in that order on the screen and I find it hard to dodge backwards and forwards, because I'm just not competent with it yet, but, I'm sure it will come with practice.

Physiotherapist 2, time point 1

Concerns regarding how to navigate IT systems did not seem to affect the younger physiotherapists, as they were quite happy with the software and the computer.

Yeah, well, I've grown up with computers and I can touch type, so, typing and listening at the same time doesn't worry me.

Physiotherapist 16, time point 1

And:

Being quite young, I can quite easily work my way through a computer package, so, that's not a problem.

Physiotherapist 22, time point 1

Again, in the second round of interviews the physiotherapists who struggled with the IT aspects said that their typing skills did improve; however, there were others who did not have this experience:

Well, typing, I knew, would be a problem for me. I am quite slow and I tend to, I do quite like to have things written neatly rather than not having capitals or full stops, so, I do have to go over it a little bit, but, it's one thing I have done a lot more at home is do a lot more typing on the computer, so, I am a bit faster now.

Physiotherapist 06, time point 2

And:

And I knew I'd identified that you know, before I went into the study. As something that I, yes that, yes that I was concerned about because I don't sit and type and I haven't got the background of sitting and typing. I haven't done so I though right I'll probably improve as I go along and I did and my times came down so I was quite happy but again it was like lack of experience, lack of practice time so that was a bit of disappointment to me but I know that if, once you become more familiar with what's coming up on screen, then that would come.

Physiotherapist 04, time point 2

This was a problem for some physiotherapists as they generally worked only one or two PhysioDirect clinics per week (each of 3–4 hours) in the trial and felt that the PhysioDirect service did not have sufficient patients coming through in order for them to practice and feel sufficiently confident with the computer system.

Environment to provide PhysioDirect practice

The physiotherapists were concerned about the physical environment in which they delivered the PhysioDirect service, with the most common concern being the background noise in the offices in which they were working, as it made it difficult for the physiotherapists to assess patients with the PhysioDirect system. All used headsets as part of the trial, rather than relying on the handset of the telephone, as they needed their hands free to use the computer system in parallel with talking with the patient on the telephone:

I did dislike it when you were trying to assess and it was noisy, really annoying. ‘Cos you just, yes it's really annoying and people don't mean to be noisy but if you can't hear anything and then you can hear yourself talking, you can't concentrate, it's really off putting. You're kind of sitting there, finger in your ear, really trying to listen to the patient and there's a big conversation going on beside you.

Physiotherapist 25, time point 2

PhysioDirect as an inefficient use of PhysioDirect time

An initial concern for the physiotherapists prior to their use of the PhysioDirect service in the trial was that they might not be kept sufficiently busy during PhysioDirect clinics:

I don't mind that, I hope the phone goes just to keep busy.

Physiotherapist 22, time point 1

And:

That's not such an issue because although they're sitting on the telephone, they're still doing physio, they're still assessing a patient and giving them advice for a problem. It's not like they're sitting doing nothing, that'd be worse.

Physiotherapist 23, time point 2

This concern was realised as physiotherapists reported that the PhysioDirect service was not ‘busy enough’. This resulted in some therapists holding the opinion that the PhysioDirect service was an inefficient use of their time:

So that was a hard part of it actually because we were sitting feeling like we were wasting important clinical time so from that perspective, I have to just you know, mention that [laughing]. That was quite a large part of it. And that, I think influenced how we felt about things in general. So I think if we had, if I had been in a busier session I might feel a little bit more positive about it.

Physiotherapist 4, time point 2

Managing a PhysioDirect service

The interviews with physiotherapy managers highlighted that the unpredictable number of calls in each PhysioDirect clinic session made it difficult to manage the service. However, the physiotherapy managers did report, at the end of the trial, that they were overseeing this aspect of the service better:

Or when they got there the phone was absolutely mad. It seemed to be all or nothing. So I mean we had staff going there, I mean going there, sitting there, nothing to do, bored. A lot of them got a lot of CPD [continuing professional development] and discharges done, but other times they were going and then they were feeling under a lot of pressure because the phone was ringing.

Manager 3

And:

I think we'd have to – we would have to look it quite differently as to how we rolled it out because if from what we're scoping, if patients aren't going to use it we can't afford to have a physio in every hospital, sitting waiting for phone calls. Because that would be half my workforce, you know, it would – it just wouldn't be feasible. Not with 31,000 referrals.

Manager 2

The physical geography of the participating PCT and the population each served were perceived as important factors in managing the PhysioDirect service. It was felt unfeasible for physiotherapists to have to travel long distances to the PhysioDirect office to provide the PhysioDirect service. This was particularly an issue for the physiotherapy managers in charge of services serving largely rural areas:

There were quite a few logistical things which you wouldn't … if you were running it yourself out with the trial, you wouldn't have some element of that. So if you were working in an acute trust it would have been much more straightforward, because everybody would have been in one building, your call centre would have been in the building and it, it wouldn't have been … but we had to move staff, as I say, from one part of the patch to another part of the patch and be mindful of what was happening at their original base.

Manager 3

Implementation of PhysioDirect after the trial

Two of the physiotherapy services (out of the four that participated in the trial) carried on using the PhysioDirect service after the trial ended, but importantly both changed the way in which the service was offered and operationalised. The main difference was in how they managed patient referrals and calls into the service. In the amended services, GP referrals were screened by a physiotherapist and then, depending on the MSK problem and length of the local waiting list for physiotherapy, the patient was sent a letter inviting them to call the PhysioDirect service or they were put on a waiting list for face-to-face care. For example, patients with shoulder pain problems were viewed by physiotherapists as needing to be seen in traditional face-to-face consultations and so these patients were not given the choice of PhysioDirect but listed for face-to-face care, whereas those with back or lower limb problems were viewed as suitable for PhysioDirect and were automatically routed to physiotherapy via that pathway:

And we've obviously now in a way cherry picked what patients we think do best, so now when the patient referrals come into the department if they're shoulders we just straightaway put them on face-to-face waiting list and if they're kind of lower limb or backs that we think would do well then we put them to the – to the PhysioDirect method of working.

Manager 2

By essentially changing PhysioDirect to a call-back service only, the two sites that carried on using PhysioDirect did so in a way that enabled them to better predict the number of people ringing into the service. They did this through taking calls from patients and arranging convenient call-back times for the physiotherapist to contact the patient later, essentially using PhysioDirect only as a call-back service. Despite these changes, they still faced some challenges in predicting how many patients they could assess in 1 day.

And the other challenge was when we totally overloaded the system and had about 20 calls in one day and only one physio. Well, it wasn't that bad but it really overloaded the system and so, managing the right capacity and demand was quite difficult.

Physiotherapist 06, time point 2

The managers perceived PhysioDirect as a way to manage the long waiting lists in their services.

We used PhysioDirect very much to siphon off some of the bulge in referrals, so that some of the more urgent stuff could go through there, and that, you know, it was used as a sort of management tool really in that respect; worked quite well.

Manager 1

The reasons why the two other PCTs did not continue to provide the PhysioDirect service centred upon wishing to wait until the results of the randomised trial were known, as well as lack of support from service commissioners given current financial constraints.

The official reason they used was that; if you do a drug trial okay, for a drug you don't know its effectiveness, you wouldn't continue giving it to patients until you know, if it's really effective or not. So basically they [physiotherapy service commissioners] asked me to stop providing the service. If it's proven to be effective, to start again, but until then, they wouldn't pay for such a service.

Manager 4

Future of PhysioDirect services

Most of the physiotherapists could see PhysioDirect working within their PCT in the future and were not adverse to the idea of offering a future PhysioDirect-type service. They saw it as a tool to reduce waiting times or to help with the management of waiting lists. The majority of the physiotherapists mentioned the future of PhysioDirect working well within a service that offered self-referral to physiotherapy (direct access without the need for a referral from a GP) for patients, in order to promote patient choice in how to access the physiotherapy care they needed:

They could ring so I think on its own, it would be OK but it would be better in conjunction with other direct access systems because then you, it gives the patient more choice of how to have physio and also it wouldn't inundate the PhysioDirect system because they've got other options if they don't like the phone option.

Physiotherapist 23, time point 2

This was supported by physiotherapist managers, who suggested that PhysioDirect could be used to offer patient choice in how they access and receive physiotherapy. They envisaged that the choice would be given to the patient at the time of ringing the physiotherapy department of either a face-to-face appointment or a telephone assessment and advice session. This was thought to appropriately complement a self-referral system already in place within one of the participating PCTs.

Yeah, I don't know, that's my question and I wonder whether we could – I suppose our gold standard where we're always heading is if, if direct access is sustainable can we run the two side by side? So when a patient rings up for their same day, next day appointment we can ask them would you like this over the telephone or would you like it as a face to face?

Manager 2

Okay, so giving them a choice?

Interviewer

It is all about patients' choice isn't it, about giving them the options to access physiotherapy in whatever mode they want to.

Manager 2

And:

I believe it has a place in physio, okay, however, I don't think this will be the only way for patients to access a service, okay; there have been issues in their own economy that patients were not given a choice really so if it looked like a choice service.

Manager 4

It was evident that PhysioDirect was not seen as the only way to deliver physiotherapy services to patients but the managers clearly saw it as one option.

But I think, I think deep down I knew, ‘Well, it's not going to solve everything but it's definitely kind of got a, a kind of a role’.

Manager 3

Evaluation of new services

There is always a tension involved in evaluating newly established services. On one hand, it is important to conduct evaluation fairly early in the development of a new initiative. This makes it possible to identify and disseminate benefits from a new way of working, but also means that if there are disbenefits, wasted resources or safety concerns then these are identified quickly. On the other hand, services evolve and run more smoothly as they overcome initial teething problems, staff become more familiar with new procedures, and services are fine tuned in the light of experience. For these reasons, all of the new services in this trial had a run-in period of at least a month before they recruited patients to the main trial. During this period they practised providing PhysioDirect sessions and also piloted the research procedures.

However, several aspects of the service continued to evolve during the trial which will have had an impact on the cost-effectiveness of the service. For example, in some sites a high proportion of patients assessed by PhysioDirect were initially invited for face-to-face appointments. Through discussion with sites and by feeding back comparative performance data, this proportion reduced over time and became more similar between sites. It also became clear that the length of telephone assessments became shorter over several months, as the physiotherapists became more confident and more familiar with the software.

Conducting a randomised controlled trial in the context of a routine service

Conducting a RCT in the context of a routinely provided service inevitably has an effect on the way in which care is provided, which can have important implications for the results. The constraints which are inherent in a RCT, such as the need to standardise the intervention over a period of time (when in reality services constantly change in response to external events), to isolate and minimise contextual factors in multicentre studies (when in reality the service is appropriately adapted to the context) and to run different systems of care in parallel, tends to lead to compromises either in the service or in elements of the research design. For these reasons, some commentators have questioned the role of RCTs in the evaluation of new services,115 although others have argued that more pragmatic trials of this type are needed. 116

These considerations are very relevant to this study. First, only a small proportion of all patients referred for physiotherapy were offered PhysioDirect in this trial. Usual care continued to be provided for patients referred by hospital specialists, those referred from primary care or self-referred who were ineligible, those who declined to participate and those randomised to receive usual care. The overall effect of these factors meant that about 80% of all patients referred were still offered usual care, even although two out of three patients in the trial were allocated to PhysioDirect. Attempting to provide a PhysioDirect service on a small scale meant that it was not possible to provide a physiotherapist to answer telephone calls at all times in the week. Instead, sessions were conducted across a range of times on different days of the week, but for a substantial proportion of each week the service was not available. Outside the constraints of a trial, a local service is likely to make PhysioDirect the normal first point of contact for all patients referred from primary care or self-referred. A much higher call volume would mean that the service could be open at a wider range of times, making it more convenient for patients, and also more efficient for reasons described below.

The trial design also made it necessary to run separate waiting lists for face-to-face appointments for patients in the usual care and PhysioDirect arms. This reduced the flexibility for patients to obtain appointments at convenient times.

Allocation of physiotherapy resources in the context of the trial

From the outset of the study, we sought to ensure that the total physiotherapy resources in terms of the number of physiotherapist hours were distributed between the two arms of the trial in proportion to the number of patients we anticipated would be allocated to PhysioDirect or usual care. These estimates were based on routine data and the experience gained in a pilot study, and meant that 80% of resources were devoted to usual care. If resources had been allocated disproportionately, this would have been likely to have an impact on waiting times for treatment in the arm which was under-resourced, which may in turn have affected patient satisfaction and possibly clinical outcomes. Although one aim of PhysioDirect was to reduce waiting times, we wanted to ensure that any such effect was due to the effect of the intervention rather than an artefact created by uneven distribution of resources. However, because patients in the PhysioDirect arm of the main trial had fewer appointments in total, this meant that the physiotherapists' time allocated to PhysioDirect was not fully utilised. In the main analysis, and following conventional principles for economic analysis, this unproductive time was re-allocated to the PhysioDirect arm. The consequence of the decision to allocate physiotherapist time proportionately between usual care and PhysioDirect patients meant that the costs of physiotherapy for patients in the two arms were inevitably similar, using this approach to analysis.

In real life, outside the context of a trial, a physiotherapy service would reduce the numbers of physiotherapist hours provided for telephone sessions if the physiotherapists' time was not being used productively or would increase the number of patients having access to the service. Over time the managers of the service would ensure that the number of staff on the telephone matched demand for the service. They would also modify the operation of the service in the light of experience to ensure efficiency, for example by introducing a call-back system, as occurred in the two services that continued the PhysioDirect service after the trial. For this study it was important to conduct sensitivity analysis to model the costs of providing PhysioDirect under a realistic scenario using data about productivity from one of these continuing PhysioDirect services.

It is clear from this sensitivity analysis that PhysioDirect could be provided less expensively and be more cost-effective than it was in this trial, if the physiotherapists' time was used more productively when they are manning the telephone sessions. The proportion of productive time is never likely to approach 100%, even if the physiotherapists make outgoing telephone calls to patients rather than waiting for incoming calls, as this incurs wasted time phoning patients who are not available (as well as the additional costs of call charges). Another way to improve the efficiency of PhysioDirect would be to provide it on a larger scale. It would be feasible for one call centre to provide PhysioDirect for several areas, and providing the service on a larger scale would help to even out peaks and troughs in demand, as well as achieving other economies of scale (e.g. in terms of maximising the investment in telephone equipment). However, this change to a call-centre model may have other implications, for example for staff recruitment and turnover, as our qualitative research suggests that some physiotherapists would not want to spend most of their working time in this kind of environment.

Absence of a no-treatment arm

One limitation of pragmatic trials of this type, which compare a new treatment with usual care, is the absence of a ‘no-treatment’ or placebo/attention control group. The CONSORT guidance on reporting non-inferiority and equivalence trials emphasises the need to establish the efficacy of the reference ‘usual care’ treatment. 82 One limitation of this trial is that the inclusion of a broad range of patients makes it difficult to establish the efficacy of ‘usual care’, as most research about physiotherapy is based on patients with specific conditions. This trial replicates the way in which PhysioDirect services are implemented in real life, in that it is used to assess patients with a wide range of MSK problems. Current PhysioDirect services are not restricted to patients with, for example, knee pain. This study therefore maximises generalisability but at the expense of a good estimate of efficacy in the usual-care arm. In the most extreme sceptical position, one could argue that both treatment arms are equivalent because neither is effective. However, as a counter to this argument, there are a number of studies of the efficacy of physiotherapy for many of the conditions suffered by patients treated in this trial, as described in the introduction to this report. Furthermore, it would not be ethically justifiable or feasible to recruit patients to a study in which participants were offered the possibility of being randomised to a no-treatment arm, when physiotherapy is routinely available on the NHS. Finally, as it is very likely that physiotherapy is going to continue to be available on the NHS for the foreseeable future, a less costly service can be more cost-effective whether it is equally effective or equally ineffective.

Improvement in symptoms over time

It is notable that patients in both treatment arms improved similarly and considerably over the 6-month follow-up period (based on an improvement in SF-36v2 PCS scores of about 7 points) and that most of this improvement occurred by the time of the first follow-up questionnaire that was sent after 6 weeks. This finding may be interpreted as indicating that both types of treatment pathway were equally effective, or it could suggest that a large amount of natural resolution in symptoms occurs in the first few weeks after referral, or it could reflect a placebo or attention response. If, as seems likely, at least some of this improvement is due to natural resolution, the scale of this change makes it more difficult to detect small differences in the effectiveness of different treatment pathways.

Blinding

As is usual in pragmatic trials, it was not possible to blind participants to whether they were allocated to the PhysioDirect or usual-care arms of the trial. This may have led to bias if participants believed that one or other arm was more effective or preferable. Such bias may have been reduced if we had included an objective measure of clinical function rather than relying on self-reported measures. However, we did use a range of well-validated outcome measures, and patient-reported outcomes are more appropriate measures of the benefits of these different approaches to service delivery than objective measures of function. It is unlikely that patient attitudes to services would affect their responses to the SF-36v2 PCS, although they may have affected their responses on the patient satisfaction questionnaire.

As previously described, as far as possible all data were collected and entered blind to treatment allocation. The unequal allocation ratio made it impossible to maintain blinding during analysis.

Patient satisfaction

There is an important limitation to the analysis of patient satisfaction. Six of the nine questions in this questionnaire related to consultations with the physiotherapist. Of the remaining three questions, which formed the access scale, one referred to the hours the service was open, one to the convenience of the service and one to how long the patient had to wait to get advice from a physiotherapist. All but the last of these questions would not have been relevant to patients who had not received any physiotherapy. The questionnaire therefore instructed respondents to skip the questionnaire if they had not had any contact with a physiotherapist (including telephone advice or a face-to-face consultation).

It is likely that those people who were still waiting for physiotherapy by the time they were sent their first follow-up questionnaire after 6 weeks would have been most likely to be dissatisfied with access to the service but they did not have the opportunity to respond to the questionnaire. As people in the usual-care arm were much less likely to have had a consultation by 6 weeks, comparison between the PhysioDirect and usual-care arms in relation to the access questions at 6 weeks is not a randomised comparison and may be misleading.

For this reason, we explored in more depth the free-text comments in relation to access to care and waiting times for treatment that many patients made at the end of the questionnaire. These provided useful insights into the factors that determine patient satisfaction with the different services and ways in which they can be improved.

Limitations of routine data

Some of the data used to collect resource use for the economic evaluation came from routine records and may have been of limited reliability. Information about use of secondary care resources was based on patient questionnaires and we did not attempt to validate this against hospital records. Data about the duration of telephone calls in PhysioDirect came from electronic call records stored within the software, and these times may be unreliable if, for example, a physiotherapist did not log off the system when they finished a call. It became evident during time and motion studies that further time was incurred after many telephone calls, for example by collating leaflets to send to patients. We estimated the length of this ‘administrative time’ from the mean difference between the call times recorded by the PhysioDirect software and that recorded manually by physiotherapists in Bristol in routine records, but the latter may itself not have been a reliable gold standard, as the data exhibited clear digit preference for 5-minute intervals.

One advantage of using routine data is that it is almost complete, whereas there is considerable missing data in the variables obtained from patient questionnaires. This particularly affects the EQ-5D data and information about secondary costs, both of which are important in estimating cost-effectiveness. It is debatable whether or not the primary analysis should be based on complete case data, which omits a large number of cases and therefore has limited power due to the small sample size, or the imputed data, which has a larger sample but introduces a range of assumptions about the extent to which data are missing at random, and also the validity of the imputation model.