Tele-Assessments in Rural and Remote Schools – Perspectives of Support Teachers

Abstract

Tele-assessments may enable specialist evaluation of students in schools and their progress following intervention. The aim of this project was to evaluate the feasibility of using videoconferencing technology to assess students with reading difficulties in the school setting and obtain the perspectives of teachers supporting the students during the assessment. Teachers (n = 57) of 71 primary school age students participated in the study. Teachers provided feedback on the tele-assessment by completing questionnaires about students’ behaviours and the quality of the technology and assessment process. Randomly selected teachers (n = 24) were invited to participate in a semi-structured interview to provide qualitative feedback. Tele-assessments were completed in 93.4% of the sample. Support teachers reported satisfaction with the audio and visual quality as well as the assessment process. The majority of students completed the tele-assessment with good compliance, engagement and attention. Feedback from support teachers also reflected the strengths of tele-assessments, such as better access to service, cost saving and convenience. Barriers to tele-assessments included technical difficulties, assessment limitations and equipment issues at schools. Tele-assessment is a feasible method of objectively measuring outcomes of students following an intervention. Tele-assessment at school is largely accepted by teachers who supported students in this study. Feedback obtained from this research may be used to offer guidance on undertaking tele-assessments with students in the school setting.

Keywords

evaluation school student teacher experience tele-assessment

Reading difficulties in school age children can have lifelong impacts on physical and mental health, employment and health literacy (Law et al., 2009). In the long term, such difficulties are associated with fewer years of education, lower rates of employment and higher rates of mental health problems (Smart et al., 2017). Children in rural and remote areas show greater levels of educational disadvantage and gaps in skills that widen over time, compared to their city-based counterparts (Goss & Sonnemann, 2016). It is important to assess reading skills in order to identify the need for intervention and areas to target when planning reading programs. Such assessments are also helpful to monitor response to reading intervention. Standardised assessments of reading such as Wechsler Individual Achievement Test – Fourth Edition (Wechsler, 2020), York Assessment of Reading for Comprehension (Snowling et al., 2009), Test of Word Reading Efficiency – Second Edition (Torgesen et al., 2012) and Woodcock Reading Mastery Test – Third Edition (Woodcock, 2011) have been designed to be delivered face-to-face by speech pathologists and psychologists. However, there is limited access in rural and remote areas to such specialist clinicians who can assess and support difficulties associated with reading (Australian Institute of Health and Welfare, 2019).

Over the last three decades, videoconferencing technology has helped address barriers to accessing services for those living in rural and remote communities (Hirko et al., 2020; Marcin et al., 2016). In addition to improved access, telehealth has been helpful in reducing travel time and costs associated with traditional face-to-face appointments (Burns et al., 2017). Up until the COVID-19 pandemic, uptake of this method of service delivery had remained modest, with barriers including costs, technological concerns and capability, and cultural considerations preventing clinicians and clients alike from using these models (Jang-Jaccard et al., 2014; Nepal et al., 2012). These barriers apply particularly to the use of videoconferencing technology to administer standardised assessments.

Most of the research to date in tele-assessment has focused on adult populations (Carotenuto et al., 2018; Galusha- Glasscock et al., 2016; Settle et al., 2015; Stead & Vinson, 2019), and has considered the feasibility and reliability of undertaking cognitive and language assessments. A growing number of studies have explored the application of tele-assessments in evaluating cognitive, language and literacy skills in clinic settings. Few studies have examined the feasibility of tele-assessments in children. Wright (2020) assessed 256 school-aged children (aged 6–16) by using the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) either via videoconferencing or face-to-face. Results indicated there was no significant difference in the scores between the two delivery modes. Earlier, Waite and colleagues (2010) demonstrated the feasibility of conducting literacy assessments such as the Queensland University Inventory of Literacy, South Australian Spelling Test and Neale Analysis of Reading Ability via videoconferencing in 20 children. However, technical issues (echo, audio latency and break-up) were noted. Previous studies also demonstrated the feasibility and reliability of tele-language, intelligence and literacy assessments at baseline prior to commencement of intervention, albeit in modest sample sizes (Hodge et al., 2019a, 2019b; Sutherland et al., 2017, 2019). Hodge and colleagues (2019b) measured reliability based on the comparisons of the scores of the same individual assessed in face-to-face and telehealth conditions (Hodge et al., 2019a) while measures of the feasibility of conducting tele-assessment was operationalised as the ‘number of sessions completed and local staff ratings of audio and video quality’ (Hodge et al., 2019b, p. 281).

To our knowledge, there has been limited research into the perspectives of participants involved in tele-assessments. Stead and Vinson (2019) evaluated the acceptability of conducting cognitive assessments by videoconferencing in healthy adults and results demonstrated that 81% of participants preferred face-to-face assessment. Feedback from those involved in tele-assessment commonly identified issues with internet connectivity, technology and sensory strain that could affect their performance as well as fatigue and frustration during assessment (Carotenuto et al., 2018; Galusha-Glasscock et al., 2016; Settle et al., 2015). There is growing literature investigating consumer feedback about language and psychological interventions for children delivered by videoconferencing technology (Bradford et al., 2018; Fairweather et al., 2016, 2017; Hepburn et al., 2016). To our knowledge, no published research to date has evaluated the user experiences of tele-assessment in children.

As Farmer and colleagues (2021) indicated, much of tele-assessment work has been conducted in controlled clinical settings. We are not aware of any studies that have examined the feasibility of tele-assessments with children in real-world settings such as schools. School settings are generally better equipped than family homes with the appropriate technology and internet access for such assessments as these factors vary in homes, which may impact access to such services (Farmer et al., 2021; Peterson et al., 2020). Schools are often better able to provide an appropriate environment, such as access to quiet, distraction-free spaces. Within this setting, children are primed for learning and therefore more likely to engage with assessments evaluating their academic progress. Assessments in school also offer the opportunity for more objective and standardised administration, removing the need for parents to provide support for their child (Farmer et al., 2021). This is enabled through the support of teachers who already possess skills for supporting children’s behaviour and participation and who generally understand the role of standardised assessments.

The current study, therefore, aimed to explore the feasibility of conducting tele-assessments in schools, and to obtain the perspectives of teachers supporting students during these assessments at their local schools. The results will enable the refinement of the tele-assessment process and improve understanding of factors influencing uptake of such assessments, as well as the barriers and enablers of tele-assessment.

Methods

Ethics Approval

This study was approved by The Sydney Children’s Hospital Network Human Research Ethics Committee (HREC Reference: 2020/ETH00150) and the New South Wales (NSW) Department of Education (SERAP: 2019362). Oral and written informed consent were obtained from students, parents/carers of students and support teachers in this study.

Procedure

The NSW Centre for Effective Reading (the Centre) receives referrals for primary school students with reading difficulty from rural and remote parts of NSW, Australia in order to obtain individualised assessment and intervention. Students receive one-on-one reading intervention for a maximum 12 months and, as part of the program, the students’ local school teachers administer a post-intervention, curriculum-based assessment to review their progress.

In this study, the locally administered post-intervention assessments were augmented with standardised assessments of reading delivered via tele-assessment. The school staff and parents/carers of students who completed the reading intervention between June 2020 and December 2020 were contacted. Two standardised reading assessments, the Test of Word Reading Efficiency – Second Edition (TOWRE 2; Torgesen et al., 2012) and Word Attack, Word Identification and Passage Comprehension subtests of the Woodcock Reading Mastery Tests – Third Edition (WRMT-III; Woodcock, 2011), were completed via tele-assessment. Students were asked to read words and non-words presented on the screen. Short passages (some with illustrations) were also presented on the screen and students were asked to provide a missing word.

The digitised version of the TOWRE 2 was developed by Pro-Ed, Inc. Permission from Pearson Australia Group Pty Ltd. was granted to use scanned test materials in the tele-assessments. These digitised tests were shared with the students on a computer screen, via the videoconferencing platform, ‘Coviu’, which was developed by the CSIRO in Australia. Coviu ensures security and privacy through the use of full encryption of all video, audio and data that is delivered end-to-end. Assessments were delivered using standard desk top computers with external webcams, utilising the available hospital broadband internet access. Information about the computer equipment used in schools (laptop/desktop, type of internet) was not collected.

All tele-assessments took place at the local school during school time, and were facilitated by support teachers who were nominated by the school principal. These teachers were responsible for setting up the equipment (a computer with webcam and speaker) in a room with minimal distraction and stable internet connection, supporting the students during the tele-assessment, and troubleshooting any technical issues. Prior to the assessment, a phone call was made to each support teacher to explain the project as well as their expected roles in the tele-assessment before consent. Questions from teachers were answered during the phone calls or in email correspondences prior to the tele-assessment. An instruction manual was also sent to the support teachers which provided step-by-step guidelines of how to set up the computer and video conference platform, organise the room and prepare the student for the tele-assessment. Two questionnaires, a sticker chart and a participation certificate for students were also provided.

The tele-assessments were approximately 30 minutes each and were administered by the principal investigators AH and RS based at The Children’s Hospital at Westmead, Sydney, Australia. RS is a qualified speech pathologist with undergraduate and postgraduate training. AH is a qualified neuropsychologist with undergraduate and postgraduate training. RS and AH had 4 years of experience with tele-assessments.

Immediately following the tele-assessment, the support teachers completed the questionnaires to provide feedback on the assessment. A subset of teachers was randomly selected to participate in a semi-structured interview. These teachers were interviewed by author EC and their responses were typed as verbatim notes during the interview with the consent of the participants. All personal information was deleted in transcription.

Participants

All support teachers had prior experience with tele-assessments. The schools were located between 100 km to 798 km from the assessors who were situated in a metropolitan city. Age of students ranged from 8 to 12 years. Students were enrolled in Year 3 to Year 6 at time of assessment with 56.9% of the students enrolled in government schools. Students’ intellectual abilities were measured by the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V; Wechsler, 2014), and showed verbal comprehension standard scores ranging from 59 to 121 (M = 85.69, n = 65) and visual spatial standard scores from 53 to 122 (M = 87.45, n = 64). Students’ language ability (Core Language) ranged from standard scores of 57–110 (M = 83.01, n = 70), as measured by the Clinical Evaluation of Language Fundamentals–Australian and New Zealand-Fifth Edition (CELF-5 ^A&NZ; Wiig et al., 2017). The WISC-V and CELF-5 ^A&NZ were administered either in person or via telehealth.

Tele-assessments were conducted across 55 primary schools. School staff and parents/carers of 104 students were contacted and provided with information about the study. Seventy-one students completed the assessments. Thirty-three students did not participate in the tele-assessment due to student refusal, school unavailability, and other personal, parent/carers and school factors (see Figures 1 and 2). Fifty-seven support teachers were recruited to the current study, with some supporting more than one child. Sixty-eight of the 71 sets of questionnaires were completed (96%) and 24 support teachers completed the semi-structured interview.

Figure 1.

Recruitment flowchart.

Figure 2.

Factors contributing to the feasibility of tele-assessment.

Measures

Support teachers completed two questionnaires: (a) the remote assessment project feedback form (see Table 1) and (b) the behavioural observation checklist, which were specifically developed for this study (see Table 2). Completion time was less than 5 minutes.

Table 1.

Support teacher remote assessment project feedback form.

Domain	Question	Response
Students’ behaviour and performance	1. Describe the child’s attention and behaviour during the assessment (please tick all that apply)	a. Significantly reduced attention b. Reduced attention c. Appropriate attention d. Noncompliant e. Reduced compliance f. Compliance g. Calm and settled h. Anxious i. Other
	2. Did the child require constant redirection to the tasks on the computer during the assessment?	Yes/No question
	3. Do you feel that the website or remote assessment technology affected the child’s performance (i.e. as compared to face-to-face assessment)?	Yes/No question
	4. Did the child or parent/carer comment on the remote assessment technology or website? If yes, what did the child/parent/carer say?	Yes/No question and free text response
	5. Do you have any other comments about the child’s behaviour and performance?	Free text response
Technology and software	6. Do you feel that the child enjoyed using the remote assessment technology during the assessment?	Yes/No question
	7. Did you enjoy the remote assessment technology during this assessment?	Yes/No question
	8. What did you think of the audio and visual quality?	a. Audio quality was poor b. Audio quality was good c. Visual quality was poor d. Visual quality was good
	9. Did you encounter any technical issues during the assessment session?	Free text response
Other	10. Do you have any other comments regarding this assessment session?	Free text response

Table 2.

Support teacher behavioural observation checklist.

Domain	Question	Response
Level of interaction	1. The child was	a. Compliant b. Compliant under duress c. Noncompliant
Physical activity level	2. The child’s activity level was	a. Appropriate b. Too active c. Not active enough
	3. The child was fidgety/restless	a. Never b. Some of the time c. Most of the time
	4. The child required breaks	a. Never b. At appropriate times c. Too frequently (interfered with testing)
Attention to task	5. The child maintained attention	a. The whole time b. Most of the time c. Some of the time d. Never
	6. The child was distracted by test-related factors or equipment (e.g. stimulus book, recording of scores)	a. Never b. Occasionally c. Often
	7. The child was distracted by non-test-related factors (e.g. environmental sounds, window, pictures on walls, etc.)	a. Never b. Occasionally c. Often
	8. The child engaged in off-task behaviours	a. Never b. Occasionally c. Often
Response latency	9. Responses were generally given in	a. 0 to 15 seconds b. 16 to 30 seconds c. More than 30 seconds
Response latency	10. Response rate was generally	a. Appropriate b. Too fast c. Too slow
Fatigue/Boredom/Frustration	11. The child demonstrated fatigue, boredom and/or frustration during the test	a. Never b. Rarely c. Sometimes d. Often
Fatigue/Boredom/Frustration	12. Fatigue, boredom and/or frustration became evident	a. Never b. After 15 minutes c. After 10 minutes d. After 5 minutes
Anxiety	13. The child demonstrated anxiety/nervousness during the test	a. The whole time b. Most of the time c. Some of the time d. Never
Request help, support or clarification	14. The child requested help, support or clarification	a. Never b. Occasionally c. Often

The semi-structured interview explored the teacher’s experiences of, and opinions about the assessment using a set of open-ended questions. The interview took approximately 10–15 minutes.

The TOWRE 2 is a standardised reading assessment to measure students’ ability to read printed words and decode non-words within 45 seconds. The TOWRE 2 has demonstrated good reliability ranging from .83 to .94 (Torgesen et al., 2012). It has very large correlations with criterion tests of reading such as Woodcock Reading Mastery Test - Revised (Woodcock, 1998) and has good construct validity as a measure of word level reading skills (Torgesen et al., 2012).

The WRMT-III assesses students’ readiness, comprehension and basic skills in reading. The WRMT-III has shown good internal consistency with reliability estimates for reading test scores ranging from .83 to .95 (Woodcock, 2011). Test retest reliability coefficients for the WRMT-III ranging from .85 to .94 (Brunsman, 2014). The WRMT-III is highly correlated with other tests of reading such as Wechsler Individual Achievement Test – Third Edition (Wechsler, 2016) with correlation coefficients ranging from .89 to .93, indicating good construct validity (Woodcock, 2011).

Data Analysis

As per Hodge and colleagues (2019b), feasibility was defined as the completion of sessions with adequate video and audio quality.

The teacher questionnaire data was analysed descriptively. The open-ended questions from the questionnaires were analysed qualitatively by data familiarisation, coding and development of themes.

The Framework Approach (Gale et al., 2013) was adopted to analyse the interview data, as this method has been commonly used in applied qualitative studies and requires little training. This is an inductive approach where there is no hypothesis to begin with but, rather, themes are derived from the data. The raw data from transcripts is analysed using the framework methodology looking for codes, categories, subthemes and themes (Gale et al., 2013). Often the case numbers are dictated by saturation of themes. This ensures that no new data is gained from the subsequent interviews and ensures that a range of views have been captured. After transcription and familiarisation of the interview data, author EC coded the transcript by paraphrasing and labelling the underpinning interpretations of each line. Author NO reviewed the codes independently. Authors EC and NO then compared the codes of the transcript and any differences were resolved by consensus. Author EC then grouped the codes into categories and formed a working analytical framework until no further codes emerged. The codes within these categories were then synthesised into subthemes and themes.

Results

Perspectives of support teachers

Questionnaires

Students’ behaviour and performance

Teachers’ ratings indicated that 97% of students were compliant and showed appropriate activity level. Eighty-seven per cent (87%) of students maintained appropriate attention for the duration of the tele-assessment with less than 15% of students distracted by task factors and less than 20% of students distracted by non-task related factors. Approximately 6% of students occasionally engaged in off-task behaviours (see Table 3).

Table 3.

Students’ behaviours during tele-assessment.

Variables	n (%)
Level of interaction
Compliant	66 (97)
Compliant under duress	1 (1)
Noncompliant	1 (1)
Physical activity level
Appropriate	66 (97)
Too active	1 (1)
Not active enough	1 (1)
Fidgety/restless
Never	35 (51)
Some of the time	24 (35)
Most of the time	9 (13)
Student required breaks
Never	63 (93)
At appropriate times	5 (7)
Too frequently	0 (0)
Maintained attention
The whole time	59 (87)
Most of the time	8 (12)
Some of the time	1 (1)
Response latency
0–15 seconds	42 (62)
16–30 seconds	21 (31)
More than 30 seconds	5 (7)
Response rate
Appropriate	56 (82)
Too fast	1 (1)
Too slow	11 (16)
Fatigue/Boredom/Frustration
Never	47 (69)
Rarely	10 (15)
Sometimes	10 (15)
Often	1 (1)
Anxiety
The whole time	3 (4)
Most of the time	6 (9)
Some of the time	26 (38)
Never	33 (49)
Student requested help, support or clarification
Never	47 (69)
Occasionally	20 (29)
Often	1 (1)
Student required redirection to tasks
Yes	4 (6)
No	64 (94)

Within the 30-minute tele-assessment, close to 70% of students never demonstrated signs of fatigue, boredom or frustration and only 4% and 7% of students showed signs of fatigue after 5 and 10 minutes, respectively. Approximately half of the students demonstrated nervousness during the assessment, however, no sessions were discontinued due to anxiety or distress.

In 95% of sessions, support teachers felt that the student enjoyed using the technology during the assessment and 97% of the teachers themselves enjoyed the tele-assessment. Additionally, 83% of support teachers believed that the tele-assessment did not affect the students’ performance.

Technology and software

Teachers’ ratings indicated that 93% of support teachers reported good audio quality during the assessment, and 90% reported good visual quality. No technical difficulties were experienced in 44 of the 68 (65%) tele-assessments. Five tele-assessments experienced internet connectivity issues, 7 had small font sizes and 9 experienced poor audio quality such as echoing and delay. Some support teachers reported having issues accessing the tele-assessment platform (n = 2), issues with screen-sharing (n = 2), lagging (n = 1) and mismatch of cursor location on the assessor’s and students’ screen (n = 1). No session was discontinued due to sound or video issues.

Interview

Overall impression of tele-assessment

All support teachers reported an overall positive experience of the tele-assessment. Some support teachers (n = 2) indicated that they did not find any difference between tele-assessment and face-to-face assessment: ‘It works much the same as compared to face-to-face…I think using videoconferencing does not affect his performance’ (T7).

Conducting tele-assessment at schools

The interview responses were further grouped into different themes as presented below (see Table 4).

Table 4.

Themes and subthemes of the factors contributed to the feasibility of tele-assessment in students at local schools.

Theme	Sub-theme	Description	Quote
Set up	Easy set up (n = 8)	The setup of the tele-assessment was easy, straight-forward and intuitive	‘The setup of the tele-assessment was super easy. I just clicked on the link and followed the prompts. It was quite intuitive’ (T10)
Set up	Provision of instruction (n = 1)	There was clear instruction for setting up the tele-assessment	‘The instructions for setting up the tele-assessment were also clear and concise’ (T4)
Assessment	Assessment delivery (n = 13)	The tele-assessment was easy to follow (n = 2)	‘I think in general the assessment was easy to follow’ (T15)
		There was clear instruction (n = 3)	‘I think the assessor was clear with instructions’ (T14)
		The length of assessment was appropriate (n = 8)	‘It was quick and did not take up much time. I think long assessment time might lose the student and might have to break it up into two parts’ (T6), ‘the timing was precise’ (T11)
	Assessment materials (n = 7)	The assessment materials were clear (n = 3)	‘The images used in the tele-assessment were clear. We used a laptop today and the images were ok on a laptop screen’ (T7)
		The sticker chart was useful (n = 4)	‘The sticker chart was good too. Student could see how quickly he went and where he was heading to’ (T17)
		The font size of a particular assessment task was small (n = 6)	‘The word list was also too long and font was small’ (T23)
Technical	Visual and audio quality (n = 9)	Good visual and audio quality (n = 6)	‘The video and sound were good. It went well and was smooth’ (T13)
	Visual and audio quality (n = 9)	Some audio issues (n = 3)	‘Audio is quite scratchy today and there are some glitches. These could have an impact on the student as there was delay in response and student had to repeat some words’ (T14)
	Connectivity (n = 5)	Good internet connection (n = 2)	‘The internet worked well today’ (T8)
	Connectivity (n = 5)	Some connection issues (n = 3)	‘We had connection issue in between the two assessments. We closed the videoconferencing meeting, got back on and no problem afterwards’ (T15)
	Features on the videoconferencing platform (n = 2)	The videoconferencing platform has the function to adjust the font size of the assessment materials (n = 1)	‘The videoconferencing platform that we used today was able to accommodate according to the students’ needs in which the test materials could be zoomed in or out to adjust the size’ (T5)
	Features on the videoconferencing platform (n = 2)	The cursor on the videoconferencing platform was not precise (n = 1)	‘It was not always clear to what the assessor was pointing at’ (T10)
Student	No difference between tele-assessment and face-to-face assessment (n = 8)	Anxiety level (n = 4)	‘I think it was daunting for students to sit in an assessment. The assessor is a stranger to the student and it is an unknown assessment. However, I think students will react the same in face-to-face assessment as compared to the tele-assessment’ (T15)
		Concentration level (n = 1)	‘The student has attention issue and he took medication before the assessment. I think he was ok with engagement in the tele-assessment which would be the same as in a face-to-face assessment’ (T19)
		Familiarity with technology (n = 3)	‘Students are so used to technology and zoom meetings now and kids are so adaptive. They have no problem using videoconferencing as long as the technology works. Teachers are also used to technology. Therefore I don’t see any differences between videoconferencing and face-to-face assessment’ (T17)
	Videoconferencing increases students’ engagement (n = 5)	Use of computer screen (n = 2)	‘I think the computer screen in the tele-assessment has a game-like feeling for students, which students may have a better attention to details than face-to-face assessment’ (T19)
		Novelty (n = 2)	‘The tele-assessment is new to student and is different to what he used to do in the reading intervention. This kept him engaged as he did not know what to expect’ (T13)
		Better sense of control (n = 1)	‘I think using videoconferencing for assessment has a calming effect on students with autism, ODD and ADHD as they want to be in control. Students feel that they can do what they want when they use computer’ (T7)
	Tele-assessment might impact on students positively or negatively (n = 2)	Concentration level	‘The level of engagement that the students have in a tele-assessment depends on the students. Some students may be distracted but some might have good attention due to the visuals presented using videoconferencing’ (T18). ‘The process of assessment also depends on the child. Student’s concentration may affect the assessment and I am not sure how inattentive students will perform in tele-assessment. However, students with poor concentration will encounter difficulties in face-to-face assessment regardless’ (T8)
	Tele-assessment increases students’ anxiety (n = 1)	Limited videoconferencing experience	‘The challenge for students when doing a tele-assessment is that they have little exposure of using videoconferencing and completing assessment. It might be beneficial for students to have a practice run…so that…they feel less anxious’ (T4)
Support teacher	Level of experience (n = 3)	Teacher’s previous videoconferencing experience was an important factor in supporting student in tele-assessment	‘When doing the tele-assessment, inexperienced teachers might not be as competent in supporting students. For example, they might not have the knowledge or experience of how to motivate students in assessments’ (T1). ‘I felt comfortable when I used videoconferencing as the school has been using videoconferencing to deliver speech intervention and occupation therapy for student’ (T5)
	Role of support teacher (n = 5)	The conflicting role of the support teacher in the tele-assessment	‘I was not sure about my role in the assessment as a support person. I was uncertain about whether I should have prompted the student or reminded the student about the skills that have been taught… I was also not sure about how much reassurance I should have given’ (T3). ‘It was hard not to step in to assist the student from a teacher’s perspective’ (T21)
	Preparation (n = 1)	The support teacher had a preparatory role in the tele-assessment	‘Teachers and parents discussed with the student about the assessment so that the student was fully aware of the assessment. This reduced the anxiety of the student during the tele-assessment’ (T17)
	Comfort to student (n = 1)	The support teacher was a comforting factor for students	‘Having someone that the student is familiar with as the support person would be a comforting factor for the student’ (T22)
Team	Provision of materials (n = 4)	The provision of materials in preparation for the tele-assessment and how the materials were implemented during the assessment	‘I received all the assessment materials in advance and I could prepare them before the assessment… the sticker chart that was used in the tele-assessment also helped the student to keep on track’ (T3)
Team	Communication (n = 3)	The communication between the research team and the school staff	‘I am happy with the way the videoconferencing review assessment went. The communication with the team has been great. I received good and appropriate preparation for the assessment’ (T1)

Setup of the tele-assessment

Support teachers commented that the setup of the tele-assessment was easy, straight-forward and intuitive and one noted that the instruction was clear.

Assessment factors

Some support teachers (n = 13) were positive about the delivery of tele-assessment in terms of the process and length. They also found the assessment materials were clear (n = 3) and useful (n = 4). However, some support teachers (n = 6) commented that the font size of a particular assessment was small.

Technical factors

Some support teachers (n = 5) reported technical issues during the assessment. Three main subthemes were derived from the technical factors: visual and audio quality, internet connectivity and tele-assessment platform.

Support teachers identified factors that contributed to the quality of internet connectivity such as weather (‘Sometimes when the weather was not good, for example when it was windy, the internet connectivity would be affected’ (T8)) and geographical location (‘We learn to live with poor connectivity and some satellite schools can’t get access to services due to poor internet’ (T21)).

Even though 65% of the tele-assessments had no technical issues, the quality of the technology was perceived by some support teachers (n = 3) as a key factor that contributed to the quality of the tele-assessment: ‘As long as the technology works, it will work quite well. Otherwise, the technical issues can affect assessment results’ (T4). In contrast, in assessments that experienced technical issues, the support teachers did not think the assessment results were affected: ‘The words were a bit blurry and not sharp enough. The sound was echoing but the technical issues were not a problem’ (T12).

Student factors

A number of support teachers (n = 8) reported no difference between tele-assessment and face-to-face assessment in terms of students’ anxiety and concentration level. They indicated tele-assessment would be similar to face-to-face assessment due to student familiarity with technology, whereas some support teachers (n = 5) said the use of technology increased students’ engagement in assessment due to the use of a computer screen, the novelty and having a better sense of control. Only one teacher reported that tele-assessment increased the students’ anxiety due to limited experience with technology.

Support teacher factors

Teachers’ previous videoconferencing experience was reported as an important factor in supporting students in tele-assessment. Support teachers indicated that they had a preparatory and comforting role in the tele-assessment (n = 2). However, one support teacher mentioned reported being unsure about their role in the tele-assessment.

Team factors

Some support teachers (n = 3) mentioned their positive experiences working with the research team. They appreciated receiving the assessment materials in advance (n = 4). They also reflected that the communication with the research team in preparation for the tele-assessment was smooth.

Perceived strengths of conducting tele-assessment at schools

The perceived strengths of tele-assessments included: enhancing the accessibility of professional services to students in rural areas; and reducing costs in terms of time, money and travel. Further advantages included enhancing inclusiveness of people living in rural areas, convenience and capacity building of local school teachers (see Table 5).

Table 5.

Advantages of conducting tele-assessment in students at local schools.

Theme	Description	Quote
Ease of access to services (n = 11)	Enhance the accessibility of professional services to students in rural areas	‘Videoconferencing provides fantastic access to services and enables schools to provide more support to students. As a remote school, we can access support via videoconferencing which we are not able to access on a regular basis’ (T5)
Cost saving (n = 11)	Reduce the cost of time, money and travelling	‘The technology enables access to services without families travelling to other cities, driving, booking accommodation or students missing schools’ (T2)
Reduce students’ anxiety (n = 3)	Conduct tele-assessment at the local schools potentially reduces students’ anxiety	‘Videoconferencing requires less travelling, which travelling adds anxiety to children and families’ (T13)
Feeling of inclusiveness (n = 3)	Tele-assessment provides a sense of inclusiveness to people living in the rural areas	‘Using technology for assessment is great for us due to where we are in the country. Technology makes us feel connected and less isolated’ (T17)
Convenience (n = 2)	Conduct assessment at the local schools is convenient to students and families	‘It also makes life easier when students can receive tele-assessment at the local school because parents often have different commitments. It is also easier for students to have tele-assessment at their schools.’ (T15)
Upskilling of local school teacher (n = 1)	Having tele-assessment at schools provides training opportunities to local school teachers	‘Videoconferencing also offers an opportunity to upskill teachers. It will be great if tele-assessment can be routinely available to remote schools’ (T5)

Perceived limitations of conducting tele-assessment at schools

The perceived limitations of tele-assessments were categorised into three main themes in relation to the assessor, technical difficulties and finding an appropriate environment and equipment for tele-assessments at the local school (see Tables 6 and 7).

Table 6.

Disadvantages of conducting tele-assessment in students at local schools.

Theme	Sub-theme	Quote
Assessor (n = 9)	Assessor might not be able to pick up the students’ subtle behaviours via tele-assessment (n = 6)	‘I was not sure if the assessor was able to pick up the visual cue and body language the student displayed by looking through the computer screen’ (T3)
Assessor (n = 9)	Assessor’s limited observations on students’ subtle behaviours might have an impact on students’ engagement (n = 4)	‘I think face to face assessment might be more engaging as you can look at the kid’s body language’ (T14)
Technical difficulties (n = 7)	Poor audio quality (n = 6)	‘Face to face assessment can better pick up on those soft sounds as compared to the tele-assessment’ (T1)
	Unstable internet connection (n = 1)	‘From my experience of using videoconferencing, sometimes the internet does not work and it could be very frustrating’ (T2)
	Student could not physically touch the assessment materials in tele-assessment (n = 1)	“She could not hold it physically and point to the words she is reading. I think that would impact on the student’s reading ability” (T23)
Environment and equipment (n = 5)	The need to find a suitable assessment location and additional support person on the students’ side	“It is necessary to find a quiet place at school with less distraction” (T5) ‘I think students need another person to support them and check on how they are going when students undertake tele-assessment’ (T4)

Table 7.

Support teacher interview.

Question
1. Could you tell me your experience of using this technology for the evaluation/assessment process upon the completion of the reading program?
2. How did you find the process of setting up? (Explore practical, technological, personal aspects of using the technology)
3. Can you tell me more about the telehealth assessment process itself?
4. In what way do you think telehealth assessment will affect student’s engagement as compared to face-to-face assessment?
5. How can we improve this experience for you further?
6. Is there anything else you would like to talk about?

Teacher’s suggestions for improving tele-assessments

Administration

Support teachers (n = 4) suggested that providing students with exposure to tele-assessment prior to the actual online assessment would enhance their experience: ‘I think the assessor can do a practice run with the student before the assessment to go through the process of the assessment. It could show the student how to share the screen and introduce the assessor to the student’ (T23). Larger font size of the assessment materials (n = 2) and the use of noise-cancelling headphones (n = 2) were suggested.

Support teacher

Some support teachers (n = 2) requested more information about the tele-assessment: ‘I think the tele-assessment can be improved by informing the support person about how long the whole assessment and each task will take. The timing of the assessment can help the support person to determine whether it is time for the student to have a break when the student shows signs of tiredness’ (T2). There was also one support teacher who suggested more time in preparation for the tele-assessment: ‘If the rural schools are given plenty of notice, then I believe schools can arrange the tele-assessment and are supportive of providing more services to students’ (T6).

Assessor

Some support teachers (n = 3) provided suggestions to the assessor, which included providing more breaks in between assessment tasks (‘I suggest providing longer breaks during the assessment…Having some chit chat in between tasks allows students to focus better and know that they are moving to the next task’ (T6)) and checking if the students understand the instructions (‘Some improvements could be done including…asking the students to repeat the instructions’ (T6)).

Discussion

The current study is one of the first to demonstrate the feasibility of tele-assessment with students at their schools, as defined by the number of tele-assessments completed with adequate video and audio quality. Information collated from teachers supporting students during tele-assessment reflected positive feedback about the process, technology and level of participation of students. Furthermore, the majority of teachers felt that students enjoyed the experience.

In terms of feasibility, a high percentage of tele-assessments were able to be completed with primary school-aged students with learning difficulties in rural and remote locations of NSW with IQ skills ranging from below average to above average. This adds to the much needed and growing literature in assessing students in real-world settings (Hodge et al., 2019a; Waite et al., 2010; Wright, 2020).

Although there was a high number of students who completed the tele-assessment, it is important to identify the barriers that may prevent this occurring. In our study, students who declined to participate were reported to have pre-existing psychological difficulties such as anxiety and poor engagement at school. Previous studies have shown that anxiety can impact on test completion (McDonald, 2001). In future assessments, prior knowledge of such personal factors should enable assessors to make appropriate adjustments in order to ensure that all students are able to participate in such assessments. For example, if it is known beforehand that a student will be anxious about the assessment, strategies may be employed such as meeting with the student prior to the assessment to prepare them for what will be involved and provide necessary reassurance.

Whilst schools are often ideal settings for undertaking tele-assessment, it is important to be mindful of the staffing, time and resources required for this to occur. Close collaboration and negotiation with the school principals and teaching staff were essential for enabling the assessments to be completed. Allowing sufficient time for the support teacher to prepare in advance for the assessment was deemed necessary by many. Enhanced collaborations and communication were also essential to better support the teachers during the videoconferencing session (Fairweather et al., 2017).

Regarding the perspectives of support teachers, many expressed high levels of satisfaction with the tele-assessments. The support teachers highly valued the convenience and quality of technology and considered tele-assessment to be comparable to traditional face-to-face assessment. They anticipated the benefits of such a method of delivery for improving access to assessments and reducing cost. Conducting tele-assessments at schools could also overcome the barrier of limited access to technology faced by families of low socio-economic status (Farmer et al., 2021) as well as address the paucity of psychology/school counselling services faced by rural and remote schools.

The findings of this study have the potential to inform and refine tele-assessment service models and guidelines currently being developed (Farmer et al., 2020, 2021; Pritchard et al., 2020). One of the key factors that was emphasised in the current study was the technical quality in relation to internet connectivity, audio and visual quality. As Farmer and colleagues (2021) suggested, consideration needs to be given to technology such as hardware and reliability of internet connection and type of platform to be used (Australian Psychological Society, 2020). Increasing bandwidth may be one way to address the issue of connectivity in the future and support stable access to videoconferencing. Publishers of standardised tests should also consider the font size of the assessment materials and how they are displayed on videoconferencing platforms to improve user experience.

The design of this study necessitated the involvement of a trained teacher in supporting the student and to provide insights about the process. It is important to note, however, that the completion of the assessments in this tele-assessment model is not contingent on the presence of a teacher. Rather, students can be well supported by other individuals, including teacher assistants, who have been trained in supporting students in tele-assessment. Such training includes information about standardised testing, troubleshooting technology issues and alerting the tele-assessment examiner to student fatigue or restlessness. As suggested by Farmer and colleagues (2020), a trained support person is needed to facilitate tele-assessments to provide technical and emotional support to children. The support person also facilitates the tele-assessment by offering informative data to the assessor such as observations on the students’ behaviours. Assessors should provide sufficient information about the tele-assessment and specific guidelines on the expected role of the support person.

Furthermore, this study highlighted the importance of having an experienced assessor administer and facilitate the tele-assessment. The role of the assessor is of paramount importance for tele-assessment in terms of providing effective communication with the local schools and ensuring collection of valid data whilst troubleshooting technical difficulties during the assessment process. With the increasing demand for tele-assessment, relevant training in tele-assessment becomes essential for clinicians who engage in this type of service delivery (Farmer et al., 2021), including how to work with support persons. Ongoing supervision and professional development in tele-assessment is necessary (Pritchard et al., 2020).

A limitation of the study may have been selection bias in that only those who agreed to participate were included in the study. This may mean that only those schools and support teachers who were already favourably inclined towards tele-assessments volunteered to participate. In addition, the current study did not examine the students’ perceptions of tele-assessment as well as students’ behaviours during face-to-face assessments. This study was limited to examining the perceptions of teachers of tele-assessment only, without their perceptions of face-to-face assessment. There was a limited test battery and narrow age range included in the study. The small font size of the stimuli in the TOWRE 2 assessment was also a limitation in this study. Furthermore, the study design was limited to survey data and qualitative research. Lastly, no operational definitions were given in this study for terminologies such as fatigue, boredom and anxiety.

Future research may evaluate the feasibility of tele-assessment for a wider range of tests or longer test battery across a variety of disciplines; as well as in students younger than 8 years with a variety of needs. Future research should examine the demographic information of teachers including their level of experience with tele-assessment and years of experience with supporting students, as well as teacher perceptions of face-to-face assessment. Future studies also need to include operational definitions of the behaviours of interest and explore the psychometric data of the survey used in the study. Furthermore, as Hewitt and colleagues (2020) suggested, it will be important to identify the individuals for whom tele-assessment is not appropriate.

This research was one of the first to demonstrate the feasibility of using videoconferencing technology to administer reading assessments in the real-world context of schools. While tele-assessment is feasible, it is essential to ensure that technology is available and working, to prepare the child and support person for the assessment, and consider whether or not this is the most appropriate method of assessment delivery given what is known about the child. Our findings have the potential to inform the development of clinical guidelines about how to conduct tele-assessment to students at schools.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by NSW Department of Education and Training.

ORCID iDs

Antoinette Hodge

Esther Chan

References

Australian Institute of Health and Welfare . (2019). Rural & remote health. https://www.aihw.gov.au/reports/rural-remote-australians/rural-remote-health.

Australian Psychological Society . (2020). Principles for choosing videoconferencing technology. https://www.psychology.org.au/for-the-public/Medicare-rebates-psychological-services/Medicare-FAQs-for-the-public/Telehealth-services/Principles-choosing-videoconferencing-technology.

Bradford

N. K.

Caffery

L. J.

Taylor

Meiklejohn

Smith

A. C.

Langbecker

(2018). Speech-language pathology services delivered by telehealth in a rural educational setting: The school’s perspective. Journal of the International Society for Telemedicine and eHealth, 6, e201–e208. https://doi.org/10.29086/JISfTeH.6.e20.

Brunsman

(2014). Woodcock reading mastery tests - third edition. In The nineteenth mental measurements yearbook. The Buros Center for Testing.

Burns

C. L.

Kularatna

Ward

E. C.

Hill

A. J.

Byrnes

Kenny

L. M.

(2017). Cost analysis of a speech pathology synchronous telepractice service for patients with head and neck cancer. Head & Neck, 39(12), 2470–2480. https://doi.org/10.1002/hed.24916.

Carotenuto

Rea

Traini

Ricci

Fasanaro

A. M.

Amenta

(2018). Cognitive assessment of patients with Alzheimer's disease by telemedicine: Pilot study. JMIR Mental Health, 5(2), e31. https://doi.org/10.2196/mental.8097.

Fairweather

G. C.

Lincoln

M. A.

Ramsden

(2016). Speech-language pathology teletherapy in rural and remote educational settings: Decreasing service inequities. International Journal of Speech-Language Pathology, 18(6), 592–602. https://doi.org/10.3109/17549507.2016.1143973.

Fairweather

G. C.

Lincoln

M. A.

Ramsden

(2017). Speech-language pathology telehealth in rural and remote schools: The experience of school executive and therapy assistants. Rural and Remote Health, 17(3), 1–13. https://doi.org/10.22605/rrh4225.

Farmer

R. L.

McGill

R. J.

Dombrowski

S. C.

Benson

N. F.

Smith-Kellen

Lockwood

A. B.

Powell

Pynn

Stinnett

T. A.

(2021). Conducting psychoeducational assessments during the COVID-19 crisis: The danger of good intentions. Contemporary School Psychology, 25(1), 27–32. https://doi.org/10.1007/s40688-020-00293-x.

10.

Farmer

R. L.

McGill

R. J.

Dombrowski

S. C.

McClain

M. B.

Harris

Lockwood

A. B.

Powell

S. L.

Pynn

Smith-Kellen

Loethen

Benson

N. F.

Stinnett

T. A.

(2020). Teleassessment with children and adolescents during the coronavirus (COVID-19) pandemic and beyond: Practice and policy implications. Professional Psychology: Research and Practice, 51(5), 477–487. https://doi.org/10.1037/pro0000349.

11.

Gale

N. K.

Heath

Cameron

Rashid

Redwood

(2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 1–8. https://doi.org/10.1186/1471-2288-13-117.

12.

Galusha-Glasscock

J. M.

Horton

D. K.

Weiner

M. F.

Cullum

C. M.

(2016). Video teleconference administration of the repeatable battery for the assessment of neuropsychological status. Archives of Clinical Neuropsychology, 31(1), 8–11. https://doi.org/10.1093/arclin/acv058.

13.

Goss

Sonnemann

(2016, March) Widening gaps: What NAPLAN tells us about student progress . Grattan Institute. https://grattan.edu.au/wp-content/uploads/2016/03/937-Widening-gaps.pdf

14.

Hepburn

S. L.

Blakeley-Smith

Wolff

Reaven

J. A.

(2016). Telehealth delivery of cognitive-behavioral intervention to youth with autism spectrum disorder and anxiety: A pilot study. Autism, 20(2), 207–218. https://doi.org/10.1177%2F1362361315575164.

15.

Hewitt

K. C.

Rodgin

Loring

D. W.

Pritchard

A. E.

Jacobson

L. A.

(2020). Transitioning to telehealth neuropsychology service: Considerations across adult and pediatric care settings. The Clinical Neuropsychologist, 34(7-8), 1335–1351. https://doi.org/10.1080/13854046.2020.1811891.

16.

Hirko

K. A.

Kerver

J. M.

Ford

Szafranski

Beckett

Kitchen

Wendling

A. L.

(2020). Telehealth in response to the COVID-19 pandemic: Implications for rural health disparities. Journal of the American Medical Informatics Association, 27(11), 1816–1818. https://doi.org/10.1093/jamia/ocaa156.

17.

Hodge

M. A.

Sutherland

Jeng

Bale

Batta

Cambridge

Detheridge

Drevensek

Edwards

Everett

Ganesalingam

Geier

Kass

Mathieson

McCabe

Micallef

Molomby

Ong

Pfeiffer

Silove

(2019a). Agreement between telehealth and face-to-face assessment of intellectual ability in children with specific learning disorder. Journal of Telemedicine and Telecare, 25(7), 431–437. https://doi.org/10.1177%2F1357633X18776095.

18.

Hodge

M. A.

Sutherland

Jeng

Bale

Batta

Cambridge

Detheridge

Drevensek

Edwards

Everett

Ganesalingam

Geier

Kass

Mathieson

McCabe

Micallef

Molomby

Pfeiffer

Pope

Silove

(2019b). Literacy assessment via telepractice is comparable to face-to-face assessment in children with reading difficulties living in rural Australia. Telemedicine and E-Health, 25(4), 279–287. https://doi.org/10.1089/tmj.2018.0049.

19.

Jang-Jaccard

Nepal

Alem

(2014). Barriers for delivering telehealth in rural Australia: A review based on Australian trials and studies. Telemedicine and E-Health, 20(5), 496–504. https://doi.org/10.1089/tmj.2013.0189.

20.

Law

Rush

Schoon

Parsons

(2009). Modeling developmental language difficulties from school entry into adulthood: Literacy, mental health, and employment outcomes. Journal of Speech, Language, and Hearing Research, 52(6), 1401–1416. https://doi.org/10.1044/1092-4388(2009/08-0142.

21.

Marcin

J. P.

Shaikh

Steinhorn

R. H.

(2016). Addressing health disparities in rural communities using telehealth. Pediatric Research, 79(1), 169–176. https://doi.org/10.1038/pr.2015.192.

22.

McDonald

A. S.

(2001). The prevalence and effects of test anxiety in school children. Educational Psychology, 21(1), 89–101. https://doi.org/10.1080/01443410020019867.

23.

Nepal

Jang-Jaccard

Alem

Dods

(2012, November 7). Telehealth services for remote Australia . CSIRO. https://publications.csiro.au/rpr/download?pid=csiro:EP129006&dsid=DS1

24.

Peterson

R. K.

Ludwig

N. N.

Jashar

D. T.

(2020). A case series illustrating the implementation of a novel tele-neuropsychology service model during COVID-19 for children with complex medical and neurodevelopmental conditions: A companion to Pritchard et al. The Clinical Neuropsychologist, 35(1), 99–114. https://doi.org/10.1080/13854046.2020.1799075.

25.

Pritchard

A. E.

Sweeney

Salorio

C. F.

Jacobson

L. A.

(2020). Pediatric neuropsychological evaluation via telehealth: Novel models of care. The Clinical Neuropsychologist, 34(7-8), 1367–1379. https://doi.org/10.1080/13854046.2020.1806359.

26.

Settle

J. R.

Robinson

S. A.

Kane

Maloni

H. W.

Wallin

M. T.

(2015). Remote cognitive assessments for patients with multiple sclerosis: A feasibility study. Multiple Sclerosis Journal, 21(8), 1072–1079. https://doi.org/10.1177%2F1352458514559296.

27.

Smart

Youssef

G. J.

Sanson

Prior

Toumbourou

J. W.

Olsson

C. A.

(2017). Consequences of childhood reading difficulties and behaviour problems for educational achievement and employment in early adulthood. British Journal of Educational Psychology, 87(2), 288–308. https://doi.org/10.1111/bjep.12150.

28.

Snowling

M. J.

Stothard

S. E.

Clarke

Bowyer-Crane

Harrington

Truelove

Nation

Hulme

(2009). YARC York assessment of reading for comprehension passage reading. GL Assessment.

29.

Stead

Vinson

(2019). Cognitive assessment using face-to-face and videoconferencing methods. Nursing Older People, 31(5), 34–39. https://doi.org/10.7748/nop.2019.e1160.

30.

Sutherland

Trembath

Hodge

Drevensek

Lee

Silove

Roberts

(2017). Telehealth language assessments using consumer grade equipment in rural and urban settings: Feasible, reliable and well tolerated. Journal of Telemedicine and Telecare, 23(1), 106–115. https://doi.org/10.1177%2F1357633X15623921.

31.

Sutherland

Trembath

Hodge

M. A.

Rose

Roberts

(2019). Telehealth and autism: Are telehealth language assessments reliable and feasible for children with autism? International Journal of Language & Communication Disorders, 54(2), 281–291. https://doi.org/10.1111/1460-6984.12440.

32.

Torgesen

J. K.

Wagner

R. K.

Rashotte

C. A.

(2012). Towre 2: Test of word reading efficiency Second Edition. Pro-Ed.

33.

Waite

M. C.

Theodoros

D. G.

Russell

T. G.

Cahill

L. M.

(2010). Assessment of children's literacy via an internet-based telehealth system. Telemedicine and E-Health, 16(5), 564–575. https://doi.org/10.1089/tmj.2009.0161.

34.

Wechsler

(2014). Wechsler intelligence scale for children - fifth edition. Pearson.

35.

Wechsler

(2016). WIAT-3: Wechsler individual achievement test - third edition. Psychological Corporation.

36.

Wechsler

(2020). WIAT-4: Wechsler individual achievement test - fourth edition. Psychological Corporation.

37.

Wiig

E. H.

Secord

W. A.

Semel

(2017). Clinical evaluation of Language Fundamentals: Fifth edition, Australian and New Zealand standardised edition (CELF-5 ^A&NZ). Pearson Clinical Australia.

38.

Woodcock

R. W.

(1998). Woodcock reading mastery tests - revised. Pearson.

39.

Woodcock

R. W.

(2011). Woodcock reading mastery tests - revised. Pearson.

40.

Wright

A. J.

(2020). Equivalence of remote, digital administration and traditional, in-person administration of the wechsler intelligence scale for children, (WISC-V). Psychological Assessment, 32(9), 809–817. https://psycnet.apa.org/doi/10.1037/pas0000939.