Abstract
Introduction
A prior study examining perceptions of Allied Health Professions (AHP) telehealth services at a metropolitan hospital highlighted multiple issues impacting service uptake, operationalisation, and delivery. Concept mapping methodology was utilised to address these issues and prioritise actionable telehealth service improvements.
Methods
Representatives (n = 22) from seven AHP departments and consumers generated statements addressing the question: ‘What do we need to do to enhance and sustain telehealth services?’ Statements were synthesised and then clinicians and managers sorted them into similar groups and assigned each statement a ranking of perceived (a) importance and (b) changeability. Multivariate and multidimensional scaling was undertaken to develop a final prioritised set of goals for change.
Results
Ninety-six unique statements were generated as actionable goals for change. Statements were grouped into 13 clusters relating to improvements in staff support, infrastructure, consumer support and organisational processes. All clusters were rated >50% for importance (range 3.3–2.4 out of 4) and changeability (range 2.6–2.1 out of 4). Twenty-six statements were ranked highest for importance and changeability. Key prioritised areas were staff training, consumer advocacy and engagement, telehealth operations and workflow.
Conclusion
Concept mapping was an effective process for generating a prioritised list of actions to enhance AHP telehealth services.
Introduction
During COVID-19, health services rapidly implemented telehealth models to provide care during social distancing restrictions. While adapting to these new models was challenging,1–5 it demonstrated the benefits of telehealth for consumers and health services on a scale. With the return to usual care, organisations need to capitalise on the expertise gained from implementing these models and embed telehealth into mainstream clinical practice.1,2,6
This research team recently explored consumer, clinician, and administration staff perspectives of an allied health professions (AHPs) telehealth service (specifically videoconferencing) during COVID-19. 1 Results confirmed telehealth was viewed positively, however, improvements were needed for long-term sustainability. While organisational support was confirmed, it was acknowledged that re-designing telehealth services across a diverse range of disciplines would require multiple stakeholder consultations, and commitment from healthcare staff to advocate for, and adopt service change.
Hence the aim of this study was to engage key stakeholders to identify and prioritise actionable goals that would enhance and sustain AHP telehealth services.
Methods and results
Setting and participants
This study was conducted within the AHPs service line of a quaternary facility in Brisbane, Australia. Seven AHP departments participated: audiology, nutrition & dietetics, occupational therapy, physiotherapy, psychology, social work, and speech pathology. Participants included managers (n = 9), clinicians (n = 7), administration officers (n = 4) and consumer representatives (n = 4). All participants provided informed consent. A waiver from ethical review was provided by the Institutional Human Research Ethics Committee (LNR/2020/QRBW/64414).
Concept mapping process
Concept mapping is a mixed methods research approach that can be used to guide initial change management processes. It has been used successfully to integrate clinician and consumer preferences for health care7–9 and inform service improvement.10–12 Guided by Foley et al., 10 the first five stages of the concept mapping process (Figure 1) were undertaken for this study: (1) preparation, (2) statement generation, (3) structuring, (4) representation, and (5) interpretation. As each stage influences the next, the methods and results pertaining to each phase are presented together below. The final stage, Stage 6 – utilisation, which involves planning the application of the results, will be undertaken in a future study.

Stages 1 to 5 of the concept mapping process.
Stage 1 – preparation
Firstly, all participants attended a group meeting, where the results of a recent study exploring consumer, clinician, and administration staff feedback on the AHP telehealth service were provided. 1 Concept mapping methodology was also explained.
Stage 2 – statement generation
Then, small focus groups were conducted with each department and the consumer group, facilitated by a member of the research team. Participants considered telehealth service feedback (Stage 1) and brainstormed solutions to the question: ‘What do we need to do to enhance and sustain telehealth services?’ Discrete statements were generated for each idea/solution. Following each focus group, the statements were sent to participants for member checking and to add additional statements as necessary. The research team then compiled all statements and removed duplicates. On completion of Stage 2, a list of 96 unique statements was generated as potential areas for telehealth service enhancement.
Stage 3 – structuring
Structuring involved two steps undertaken individually by each participant. Firstly, participants sorted statements (provided on individual cards) into groups based on similarity. They could create as many groups as needed but could only put each statement into one grouping, which they then named by theme. The cards were shuffled between participants to avoid a priming effect. Next, participants were provided with the list of statements in an excel spreadsheet and asked to rank each statement according to (1) how important it was (0 = not important to 4 = extremely important) and (2) how easy it was to change (0 = not at all changeable to 4 = extremely changeable).

Dendrogram illustrating 13 cluster themes.
Stage 4 – representation and Stage 5 – interpretation
Stage 3 data was analysed using free open-source software R-CMap. 13 Firstly, multidimensional scaling was used to generate a stress value, which relates to a ‘goodness of fit’ between the raw and processed data to examine the statistical strength of the results. The stress value (0.327) was within the accepted range for concept mapping,14,15 indicating full analysis could proceed.
Hierarchical cluster analysis examined how the general themes underlying the statements were grouped by participants, depicted on a dendrogram. A dendrogram using a 13-cluster arrangement (Figure 2) was generated and the clusters were named by four members of the research team after reviewing the statements within each cluster. These covered a wide range of themes including governance, strategic planning and evaluation, infrastructure and interoperability, workflow processes, consumer awareness and staff training. The cluster names, theme descriptions, example statements, and mean importance/changeability ratings are presented in Table 1.
Characteristics of cluster themes.
AO: administration officer; HP: health practitioner.
A cluster rating map examined the relationship between the statements within each cluster (Figure 3). This displays: (1) how closely related statements are to each other, as depicted by their proximity on the map (i.e. closer the proximity = closer the relationship); (2) level of participant agreement between statement themes within each cluster, illustrated by cluster size (i.e. high agreement = smaller cluster size); and (3) the uniqueness of each cluster theme illustrated by the distance between each cluster (i.e. greater distance between clusters = clusters are more unique). All clusters were comparable in their uniqueness of theme except for clusters 3 (infrastructure and interoperability) and 8 (resources tailed to consumers’ needs) which overlapped for Statement 32 ‘Utilise a generic phone support service that provides support for patients to set up telehealth’.

Cluster rating maps with numbered statements listed within each cluster.
The pattern matching graph (Figure 4) displays the mean rating of importance and changeability for all statements within each cluster. Cluster 10 ‘Workflow processes’ was rated highest for importance whilst Clusters 4 ‘Consumer feedback’ and 13 ‘Governance’ had the equal highest rating for changeability. For 12 of the clusters, the gradient between importance and changeability was negative, indicating that whilst participants viewed the themes as being important, they also felt that enacting change might be difficult. One cluster ‘Strategic planning and evaluation’ had a neutral gradient with a mean score of 2.4 for both importance and changeability. Although clusters were ranked as having higher importance than changeability, overall, all the clusters were rated >50% for both importance (range 3.3–2.4 out of 4) and changeability (range 2.6–2.1 out of 4).

Pattern matching comparing the importance and changeability of 13 clusters.
Finally, the Go Zone (Figure 5) is a scatterplot where each statement is placed into one of four quadrants according to its overall importance and changeability. The ‘green zone’ contains statements ranked highest in mean importance and changeability and should therefore be prioritised in goal setting to achieve both meaningful and efficient change. There were 26 (27%) statements in the green zone (Table 2) representing 11 of the 13 clusters. Cluster 6 ‘Staff training’ had the most statements falling within the green zone. In the opposite quadrant, the ‘red zone’ shows statements with the least importance and lowest changeability. There were 30 statements (31%) in the red zone predominantly coming from Cluster 1 ‘Dedicated telehealth staff’ and Cluster 3 ‘Infrastructure and interoperability’. As this study focused on identifying actionable areas for improvement, the red zone results are not discussed in detail.

Go zone graph illustrating the importance and changeability rating for each statement.
Individual statements are identified within the ‘Go Zone’.
AO: administration officer; AHP: Allied Health Profession; CALD: culturally and linguistically diverse; MDT: multidisciplinary team.
Discussion
This study utilised a concept mapping approach to identify and prioritise telehealth service improvements across an AHP service line. Consultation with managers, clinicians, administration officers and consumers highlighted enhancements across several key areas. Statements relating to staff training, consumer engagement, and telehealth operations/workflow rated highest in the prioritised list of actionable goals. These findings have provided the initial direction for service improvement to establish a sustainable telehealth model that delivers effective and efficient care while meeting end-users needs. 16
A well-trained and confident workforce is essential for effective service delivery.17–19 The group acknowledged this with ‘Staff Training’ contributing the highest number of statements in the ‘green zone’. Improved training across technical and clinical/operational aspects of service delivery was suggested including staff having dedicated time to learn (1) evidence-based telehealth models, (2) how to adapt in-person skills to telehealth delivery and (3) how to address safety issues and/or risks during remote consultations. These recommendations are consistent with published telehealth guidelines17,19,20 as is maintaining staff skills through task re-orientation when changing workloads and having access to quick reference guides and technical support. Improving staff knowledge of funding arrangements for telehealth appointments compared to other models (e.g. telephone) was also suggested as a service sustainability strategy.
The rapid adoption of telehealth during COVID-19 challenged consumer readiness and impacted engagement.1,2,19,21 The group identified this as an area for enhancement by prioritising >30% of statements in the Go Zone on this topic. Acknowledging that the offer of telehealth appointments is commonly clinician directed, 1 the group suggested strategies to empower consumers to become more actively involved in choosing how they access their healthcare appointments. This included education on (1) the benefits/limitations of telehealth, (2) which AHP services could be accessed via telehealth and (3) providing avenues for feedback on telehealth sessions. Actively discussing telehealth models during clinic appointments and/or providing written information (e.g. brochures) were considered appropriate forms of consumer education. Ensuring equity of access for clients was also highlighted through the recommendation of designing relevant and culturally appropriate resources, in a range of formats (e.g. verbal and written/online), to increase engagement for all consumer groups (e.g. First Nations peoples).
Telehealth operations/workflow was the third prioritised area of service enhancement, evidenced by ‘Workflow processes’ receiving the highest rating for importance. A critical element of effective and efficient telehealth services is the integration of appropriate infrastructure and streamlined work processes.16,19 Prioritised actions for change focused on improving operational, clinical and consumer interactions. This included the establishment of operational procedures across the service line along with department-specific guidelines addressing clinical care and safety issues. Access to familiar and accessible technology for staff and consumers was also prioritised, as well as procedures to facilitate flexible appointments (e.g. change in-person to telehealth) and efficient access to support services (e.g. interpreter services). To streamline operations across departments and optimise engagement with consumer groups, dedicated support roles, such as the nomination of staff champions were recommended.
All clusters except for ‘Strategic planning and evaluation’ were rated higher in importance than changeability. Given the fiscal climate post-COVID-19, it is understandable that goals requiring additional equipment or staff may be more challenging to achieve. However other clusters requiring limited additional resources such as the development of clinical models and guidelines were also rated at a similar level of changeability. While not explicitly explored, lack of knowledge and/or experience in telehealth is a known barrier to service engagement and enhancement.22,23 Stage 6 of the concept mapping process, ‘Utilisation’ involves undertaking the change management process. It is acknowledged that execution of the prioritised goals into clinical practice will require the adoption of a planned implementation strategy. Frameworks that consider not only the adopters (e.g. staff and consumers), but also organisational (e.g. leadership and readiness for change) and wider system (e.g. regulatory) issues 24 such as the Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) Framework 25 and Consolidated Framework for Implementation Research 26 have successfully guided multi-site/service AHP telehealth models.2,27 Given this telehealth service functions within a large public health facility that supports a range of professions and clinical services, a multi-level implementation framework, such as these, will be required. In addition, the combined stakeholder support for service enhancements will also facilitate the use of known telehealth strategies aligned with these priorities such as nomination of staff champions,28,29 standardisation of operational guidelines, and the development of staff training and consumer engagement procedures. 29 Ensuring a coordinated approach that incorporates stakeholder consultation across implementation, evaluation, and monitoring phases of the change management process will assist in the adoption and sustainability of these telehealth services enhancements. 29
It is acknowledged that this study was conducted in an AHP service line within a single public health facility with existing telehealth infrastructure. Different health services, funding structures and technology access/supports may reveal alternative priorities and results.
A dedicated effort is required to integrate and sustain telehealth services into mainstream clinical practice. Concept mapping was an effective approach to bring together key stakeholders and facilitate the identification and prioritisation of improvements required for AHP telehealth services. Results demonstrated the need to focus on improving staff support, consumer advocacy and engagement, and operational workflow processes to optimise service sustainability.
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethics approval
Approval to conduct this study was granted by the Royal Brisbane & Women's Hospital (LNR/2020/QRBW/64414).
Funding
The authors received no financial support for the research, authorship, and/or publication of this article
