Use of the Weekly Calendar Planning Activity to Assess Executive Function in Adolescents With Autism Spectrum Disorder

Abstract

Importance: Youths with autism spectrum disorder (ASD) have challenges with executive function (EF). However, there are few measures to objectively assess EF, such as time management and prioritization. The Weekly Calendar Planning Activity (WCPA) is a promising performance-based EF measure that has rarely been used with youths with ASD.

Objective: To characterize the performance of youths with ASD on the WCPA and the association of WCPA performance with caregiver-rated EF.

Design: Observational and summary data obtained during a baseline evaluation for a research study.

Setting: Middle school.

Participants: Thirty-nine adolescents with ASD and without an intellectual disability.

Method: Youth were administered the WCPA; caregivers completed EF ratings. Descriptive statistics characterized WCPA performance. Correlations between WCPA scores and age, IQ, and EF ratings were computed.

Results: The majority of youth performed poorly on the WCPA and lacked insight into their poor performance. Many struggled to enter appointments. Appointment accuracy was <30%. The most common errors were location errors. Appointment accuracy and planning time were moderately associated with IQ but not with caregiver-rated EF or age.

Conclusions and Relevance: Overall, the WCPA was cognitively demanding for youths with ASD without an intellectual disability. These results have implications for using evidence-based strategies to teach adolescents with ASD how to manage their time and plan and prioritize homework assignments, upcoming tests or quizzes, and extracurricular activities.

Plain-Language Summary: Youths with autism spectrum disorder have significant problems with executive function, particularly organization, prioritization, time management, and planning. There are few objective, performance-based measures of executive function, especially those that can be used with autistic youths. To our knowledge, this study is the first to use the Weekly Calendar Planning Activity (WCPA) with autistic adolescents who do not have an intellectual disability. The WCPA captured their deficits related to executive function and could be a useful tool for assessment and treatment planning.

This is the first study to use the Weekly Calendar Planning Activity with autistic adolescents who do not have an intellectual disability as a performance-based measure of executive function.

Executive function (EF) is broadly defined as the overarching regulation of goal-directed, future-oriented, higher order cognitive processes (Lezak et al., 2004). Although there is general agreement on the broad concept of EF, the theoretical models and underlying processes (i.e., cognitive, clinical, behavioral, and neurobiological frameworks) vary considerably (Lezak et al., 2004). In one model, EF has been conceptualized as having at least three subcomponents, each linked to specific frontal systems in the brain:

▪ goal setting, which includes initiating, planning, problem solving, and strategic behavior;

▪ cognitive flexibility, which includes working memory, attentional shift, and conceptual transfer; and

▪ attentional control, which includes selective attention, sustained attention, and response inhibition (Anderson, 2002).

These EF subcomponents can be conceptualized as an integrated functional system, with goal setting and cognitive flexibility undergoing significant maturation from middle childhood to adolescence (Anderson et al., 2008). Dysfunction within this EF system may be characterized by difficulties generating and implementing strategies for problem solving, poor planning and organization, perseveration, inability to correct errors or use feedback, and mental inflexibility (Anderson, 2002; Anderson et al., 2008). Qualitative features of impairments may include disorganization, reduced self-control, impulsivity, perseveration, erratic or careless response behaviors, and deficient high-level communication skills (Anderson et al., 2008). This EF model is clearly relevant for people who have been diagnosed with autism spectrum disorder (ASD) and who exhibit many of these characteristics (American Psychological Association, 2013).

In fact, approximately 35% to 70% of youths with ASD present with EF deficits, including challenges with planning, concept formation, mental flexibility and shifting, fluency, response inhibition, and working memory (Demetriou et al., 2018; Friedman & Sterling, 2019). Although poorer EF is more typically observed in youths with ASD with an intellectual disability (i.e., with an IQ < 70), compared with EF in those without ASD, EF deficits are observed regardless of cognitive functioning (Benallie et al., 2021; Demetriou et al., 2018). EF challenges are so ubiquitous in ASD that EF has been proposed as an endophenotype (Demetriou et al., 2019). Notably, youths with ASD without an intellectual disability demonstrate particular challenges with flexibility, generativity, and working memory when equating comparison groups on cognitive functioning (Lai et al., 2017). Specific challenges in planning, response inhibition (i.e., inhibit prepotent responses), and strategy formation have also been reported for youths with ASD without intellectual disability, compared with typical controls (Robinson et al., 2009). Persistent EF deficits not only are predictors of poor outcomes in ASD but also contribute to poorer academic performance (Demetriou et al., 2019; Tamm et al., 2020). Thus, understanding the EF profiles of people with ASD is critical to inform treatment planning and intervention efforts.

The assessment of EF typically involves the use of neuropsychological assessments. However, traditional neuropsychological EF tasks are not predictive of real-world complex task performance that requires interplay among numerous aspects of EF (Chan et al., 2008). Although real-world behavioral ratings of EF address this concern in part, ratings are not necessarily objective and do not measure the same EF constructs that neuropsychological tasks do (Braconnier & Siper, 2021; Toplak et al., 2013). Tasks that allow for the direct observation of performance may be useful for capturing difficulties when performing higher level, cognitively challenging activities (Toglia, 2015). The Weekly Calendar Planning Activity (WCPA) is a complex, cognitively demanding, performance-based measure that involves entering 18 appointments into a weekly schedule while adhering to rules and avoiding conflicts (Toglia, 2015). Successful completion requires coordinated EF, including planning, flexibility, inhibition, and prospective memory (Toglia, 2015). Thus, the WCPA holds promise as an objective measure of these EF deficits in youths with ASD; however, we are unaware of any studies that examined the performance of adolescents with ASD on the WCPA.

Studies using the WCPA in examining other disorders with similar EF deficits suggest that the measure is useful in capturing functional cognition challenges. For example, college students with and without attention deficit hyperactivity disorder (ADHD) differed in WCPA performance, duration of performance, and in the number of strategies used, with students with ADHD performing less well (Lahav et al., 2018). Similarly, adolescents with ADHD performed more poorly than non-ADHD controls on the WCPA (e.g., fewer appointments entered, lower accuracy, fewer rules followed, and lower efficiency; Fisher, Berger, Grossman, Tal-Saban, et al., 2022). Adolescents with acquired brain injury performed more poorly on the WCPA than typically developing controls (e.g., lower accuracy, more errors; Doherty et al., 2022). Adolescents with genetic generalized epilepsies demonstrated significantly lower accuracy, lower efficiency, and fewer strategies used on the WCPA, compared with their typical peers (Zlotnik et al., 2020). Additionally, adolescents with ADHD tend to overestimate their performance on the WCPA, compared with typically developing controls (Fisher, Berger, Grossman, & Maeir, 2022). It is important to note that performance on the WCPA has been associated with caregiver-rated EF (Doherty et al., 2022; Zlotnik et al., 2020) and performance on other neuropsychological assessments (Doherty et al., 2022). The WCPA has also been used to document EF deficits and as an outcome measure for a time management intervention for adults with neurodevelopmental disorders, including ASD and ADHD (Wingren et al., 2022).

In the present study, we aimed to characterize the performance of adolescents with ASD without intellectual disability on the WCPA and to explore whether WCPA performance is associated with the caregiver report of EF. On the basis of the literature reviewed previously with other clinical groups, we hypothesized that youths with ASD would perform poorly on the WCPA (i.e., enter fewer appointments with less accuracy, use fewer strategies because of less cognitive flexibility) and that poorer WCPA performance would be associated with caregiver-reported EF deficits. We also expected participants to follow the WCPA rules, because people with ASD are reported to be more “rule abiding” than those without ASD (Bolling et al., 2011; Huang et al., 2012), likely because of the inflexible adherence to routines and rituals that is characteristic of ASD (American Psychological Association, 2013).

Method

Participants

Participants included 39 youths with ASD without an intellectual disability who were engaged in an ongoing research study. Participants ranged in age from 11 to 14 yr (M = 12, SD = 1.0) and were predominantly male (87.2%). In terms of race and ethnicity, on the basis of the caregiver report, 74.4% were White, 15.4% were Black, 2.6% were Asian, 2.6 were American Indian/Alaska Native, 10.3% identified as more than one race, and 2.6% were Hispanic/Latinx. All participants had an IQ of 80 or higher (M = 102.6, SD = 15.8). On the basis of the Autism Diagnostic Observation Schedule classification system (2nd ed.; ADOS–2; Lord et al., 2012), 35 participants were classified as having autism and 1 participant was classified as being on the autism spectrum. (Note that we were unable to complete the ADOS–2 for 3 adolescents; thus, caregivers completed the Social Communication Questionnaire, with a score higher than 15 [i.e., likely to be on the autism spectrum] considered eligible.) In addition to meeting the diagnostic criteria for ASD and having an IQ of 80 or higher, the inclusion criteria for the ongoing trial included that the adolescent be enrolled in school in the general education setting for the majority of the school day and have Behavior Rating Inventory of Executive Function (2nd ed.; BRIEF–2; Gioia et al., 2015) scores higher than 60 on the Cognitive Regulation Index or one of its component subscales (see the Measures section). Caregivers reported on their child’s diagnoses in addition to ASD, which included ADHD (n = 21; 53.8%), anxiety disorder (n = 15; 38.5%), and mood disorder (n = 4; 10.3%). Twenty-four (61.5%) participants were taking psychotropic medications.

Procedures

The study was approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board. Participants in the ongoing study were recruited by letters sent to families by school personnel. Interested caregivers underwent an initial phone screen (e.g., to confirm an ASD diagnosis and screen out those with a known intellectual disability) and provided informed consent electronically. Caregivers completed the BRIEF–2 (Gioia et al., 2015) electronically. Youths who met initial eligibility criteria provided written assent and participated in an eligibility evaluation, including the ADOS–2 (Lord et al., 2012), which was administered by a research reliable assessor, and the Kaufman Brief Intelligence Test (2nd ed.; KBIT–2; Kaufman & Kaufman, 2004), which was administered by trained research coordinators or graduate-level psychology students. Eligible participants completed the WCPA administered by trained research assistants or graduate-level psychology students. On average, the WCPA was administered within a week of the caregiver’s completion of the BRIEF–2. Assessments, which lasted up to 2 hr, including the ADOS–2, KBIT–2, and WCPA, were conducted in the participant’s school in a private classroom or conference room for one or two testing sessions. Data for the present study were obtained from the eligibility and baseline evaluations for the ongoing trial in which they were participating.

Measures

Kaufman Brief Intelligence Test, Second Edition

The KBIT–2 (Kaufman & Kaufman, 2004) is a culturally sensitive standardized assessment that estimates verbal and nonverbal intelligence. This test generates an overall composite score and has good reliability and validity (Kaufman & Kaufman, 2004).

Weekly Calendar Planning Activity

The WCPA (Toglia, 2015) is an objective measure of metacognition (i.e., planning, inhibition, problem solving, keeping track of rules, and monitoring passage of time) while performing a multiple-step activity (i.e., scheduling multiple appointments while adhering to various rules). The WCPA involves scheduling a list of 18 appointments into a weekly calendar while following rules, avoiding conflicts, monitoring time, and inhibiting distractions. The WCPA is scored for accuracy (number of accurate appointments scheduled out of 18), number of appointments entered, planning time (time from the beginning of the assessment until the first appointment is scheduled), total time, error types, number of rules followed, and number of strategies used. Additionally, the difference between actual appointments entered accurately and what the adolescent thought that they entered accurately derived during the after-task interview (i.e., accurate appointments minus estimated accurate appointments) is used to capture awareness of performance (overestimate, negative values, or underestimate, positive values). An efficiency score comparing the raw score to a predetermined weighted score can also be computed (i.e., total time/weighted score). However, in this sample, we were not able to compute the efficiency score for most participants, given the low level of accuracy; thus, we used accuracy to index performance. A high level of interrater reliability (.99) for total accuracy score has been established in at-risk adolescents (Weiner et al., 2012), as well as moderate to high (.60–.85) test–retest reliabilities for performance measures in college students (Lahav et al., 2018). Discriminant validity between typical adolescents and at-risk youths (Toglia & Berg, 2013) and between adolescents with epilepsy and controls has been reported (Zlotnik et al., 2020). Finally, discriminant validity between college students with and without ADHD on various performance measures was found (Lahav et al., 2018). Normative data for adolescents ages 12 to 18 yr are available (Zlotnik & Toglia, 2018).

Behavior Rating Inventory of Executive Function, Second Edition

The BRIEF–2 (Gioia et al., 2015), which has strong psychometric properties, assesses real-world EF behaviors in the home and school environments. It is important to note that the BRIEF–2 has been validated in youths with ASD without intellectual disability (Gioia et al., 2015; Granader et al., 2015; Panerai et al., 2014). The BRIEF–2 has nine subscales that load onto three index scores; specifically, the Behavioral Regulation Index (Inhibit, Self-Monitor), Cognitive Regulation Index (Initiate, Working Memory, Plan/Organize, Task-Monitor, Organization of Materials), and Emotional Regulation Index (Shift, Emotional Control). The nine subscales also load onto an overall General Executive Composite score. Higher T scores reflect greater EF deficits, with T scores from 65 to 69 considered potentially clinically elevated and T scores of 70 or higher considered clinically elevated.

Data Analysis

Descriptive statistics were used to characterize WCPA performance (i.e., number of appointments entered, number of accurate appointments, number of rules followed, number of strategies used, difference in accuracy estimation, planning time on task in minutes, and total time on task in minutes). Frequency counts were computed for each of the WCPA error types (missing, incomplete, repetition, location, time, and self-recognition) on a scale measuring 0, 1, 2, or ≥3 error types as well as descriptive statistics. Frequency counts were computed for each of the WCPA strategies. Spearman’s ρ correlations (to address any outliers) between WCPA (i.e., number of appointments entered, number of accurate appointments, number of rules followed, number of strategies used, difference in accuracy estimation, planning time on task in minutes, and total time on task in minutes) and BRIEF–2 summary scores (i.e., Behavioral Regulation Index, Cognitive Regulation Index, Emotional Regulation Index, and General Executive Composite) were also computed. We also explored the association between WCPA scores (i.e., number of appointments entered, number of accurate appointments, number of rules followed, number of strategies used, difference in accuracy estimation, planning time on task in minutes, and total time on task in minutes) and both IQ and age, using Spearman’s ρ correlations. Finally, we explored the association of the difference in accuracy estimation with WCPA scores (i.e., number of appointments entered, number of accurate appointments, number of rules followed, number of strategies used, planning time on task in minutes, and total time on task in minutes), using Spearman’s ρ correlations.

Results

Although significant heterogeneity is indicated by large standard deviations and range of scores (Table 1), the median scores suggest generally low WCPA performance for most adolescents with ASD. On average, less than 50% of the 18 appointments were entered into the calendar, and of these entered appointments, most were wrong (<5 of 18). In fact, 4 (10.3%) adolescents did not enter any appointments at all, and 13 (33.3%) got 0 accurate appointments. The majority of the sample followed three or more of the five rules (n = 32). On average, fewer than 3 of the 12 possible strategies were used. Notably, adolescents lacked insight into their poor performance; that is, 29 (74.4%) overestimated and 4 (10.3%) underestimated the number of appointments accurately entered. (Note that 1 participant indicated that they had entered 26 appointments, and recall that only 18 appointments are made on the WCPA; this adolescent also entered their own personal weekly appointments into the calendar.)

Table 1.

Descriptive Data for the Weekly Calendar Planning Activity and Caregiver-Rated Executive Functioning

Variable	M (SD)	Range	Median	Skewness	Kurtosis
WCPA (n = 39)
No. of appointments entered	8.62 (6.04)	0 to 18	7.00	0.15	−1.44
Boys (n = 34)	8.24 (6.01)	0 to 18	6.00	0.25	−1.39
Girls (n = 5)	11.20 (6.30)	2 to 18	13.00	−0.73	−0.37
No. of accurate appointments	4.28 (5.14)	0 to 18	3.00	1.28	0.60
Boys (n = 34)	3.94 (4.74)	0 to 16	3.00	1.33	0.70
Girls (n = 5)	6.60 (7.60)	0 to 18	5.00	0.89	−0.21
No. of rules followed	3.51 (1.14)	0 to 5	4.00	−0.70	0.78
Boys (n = 34)	3.56 (1.16)	0 to 5	4.00	−0.90	1.16
Girls (n = 5)	3.20 (6.30)	2 to 5	3.00	1.29	2.92
No. of strategies used	2.38 (2.20)	0 to 8	2.00	1.18	0.67
Boys (n = 34)	2.29 (2.25)	0 to 8	2.00	1.31	0.99
Girls (n = 5)	3.00 (1.87)	1 to 5	2.00	0.38	−2.90
Difference in accuracy estimation a	−4.89 (6.19)	−26 to 5	−3.00	−1.28	2.36
Boys (n = 33)	−5.30 (6.24)	−26 to 3	−3.00	−1.35	2.26
Girls (n = 5)	−4.00 (7.18)	−14 to 5	−4.00	−0.26	−0.04
Planning time on task, min	1.27 (0.85)	0 to 3.33	1.03	0.40	−0.67
Boys (n = 34)	1.25 (0.89)	0 to 3.33	1.02	0.44	−0.61
Girls (n = 5)	1.43 (0.80)	0.65 to 2.55	1.57	0.39	−1.03
Total time on task, min	12.18 (7.81)	0 to 32	10.00	1.04	0.36
Boys (n = 34)	11.68 (7.33)	0 to 32	10.00	1.16	1.09
Girls (n = 5)	15.60 (10.88)	4 to 29	12.00	0.39	−2.49
BRIEF–2 (n = 37)
Behavioral Regulation Index	68.62 (10.32)	41 to 87	70.00	−0.39	0.01
Cognitive Regulation Index	69.78 (7.23)	50 to 84	71.00	−0.49	0.36
Emotional Regulation Index	72.27 (9.35)	45 to 89	73.00	−0.58	1.03
General Executive Composite	72.95 (7.86)	51 to 87	74.00	−0.31	0.51

Note. BRIEF–2 T scores from 65 to 69 are considered potentially clinically elevated, and T scores ≥70 are considered clinically elevated. BRIEF–2 = Behavior Rating Inventory of Executive Functioning (2^nd Edition); WCPA = Weekly Calendar Planning Activity.

The WCPA Difference in Accuracy Estimation could not be computed for 1 participant.

In terms of errors, the majority (n = 33) had three or more missing appointment errors (Table 2). Many adolescents (n = 20) made two or more location errors (i.e., entering an appointment for the wrong day or for the right day but the wrong time slot). Few made repetition, incomplete, or time (i.e., length of time scheduled for the appointment) errors. Additionally, only 6 (16.7%) adolescents recognized or attempted to address errors (Table 2).

Table 2.

Weekly Calendar Planning Activity Error and Self-Awareness Analysis

Type of Error	Description	Frequency of Error Types				M (SD)
Type of Error	Description	0	1	2	≥3	M (SD)
Missing	Omitting appointment	3	2	1	33	2.60 (0.9)
Location	Difficulties placing appointment	14	5	9	11	1.42 (1.3)
Incomplete	Not entering appointment completely	23	10	0	6	0.69 (1.0)
Time	Missing day or time	23	7	3	6	0.75 (1.1)
Repetition	Entering same appointment twice	36	3	0	0	0.08 (0.3)
Self-recognition	Identify and attempt to correct errors	33	4	2	0	0.19 (0.5)

With regard to specific WCPA strategies, half of the adolescents paused and reread the instructions and appointments list, whereas about a third used their finger and/or crossed off or checked off appointments as they were entered (Table 3).

Table 3.

Weekly Calendar Planning Activity Strategy Use

Strategy	Frequency (%)
Pauses and rereads	19 (50.0)
Uses finger	14 (36.8)
Crosses/checks off or highlights appointments entered	12 (31.6)
Talks out loud about strategy, method, or plan	9 (23.7)
Self-checks	7 (18.4)
Rearrangement of materials	6 (15.8)
Repeats key words or instructions out loud	4 (10.5)
Underlines, circles, or highlights key words/features	3 (7.9)
Enters fixed appointments first	3 (7.9)
Categorizes or organizes appointments first	1 (2.6)
Makes a written rough draft or plan first	0 (0)
Crosses off specified free day	0 (0)

Note. N = 38.

Caregiver-reported BRIEF–2 scores were in the potentially clinically elevated range (Behavioral Regulation Index and Cognitive Regulation Index) and the clinically elevated range (Emotional Regulation Index and General Executive Composite; Table 1). WCPA scores (i.e., number of appointments entered, number of accurate appointments, number of rules followed, number of strategies used, difference in accuracy estimation, planning time on task in minutes, and total time on task in minutes) were not significantly associated with caregiver-reported BRIEF–2 summary scores (Table 4).

Table 4.

Correlations (Spearman’s ρ) between WCPA Performance and Caregiver-Rated Executive Functioning

Variable	Spearman’s ρ
Variable	Behavioral Regulation Index	Emotional Regulation Index	Cognitive Regulation Index	General Executive Composite
Number of appointments entered	−.005	.135	.024	.036
Number of accurate appointments	.145	.149	.144	.175
Number of rules followed	−.229	−.149	−.196	−.251
Number of strategies used	.289	.180	−.010	.108
Difference in accuracy estimation	−.032	−.009	−.001	−.029
Planning time on task, in minutes	.121	.180	−.027	.059
Total time on task, in minutes	−.105	.245	−.069	−.033

We explored the association of IQ with number of appointments entered, number of accurate appointments, number of rules followed, number of strategies used, difference in accuracy estimation, planning time on task in minutes, and total time on task in minutes. Number of accurate appointments was positively associated with IQ (rs = .46, p = .003), and planning time was negatively associated with IQ (rs = −.36, p = .026); all other WCPA scores were not significantly associated with IQ (ps ≥ .186). We also examined the association of age and WCPA performance, given that our study included 15 (38.5%) adolescents, who were age 11 yr, which is younger than has been reported in the literature. Notably, no significant associations with age were observed (ps ≥ .336). Finally, to explore the association between performance and self-estimation, we examined the association of the difference in accuracy estimation variable with the other WCPA performance variables for the 38 participants for whom accuracy estimation could be calculated. All of the correlations were nonsignificant (ps ≥ .147) with the exception of number of appointments entered; the overestimation of performance was associated with entering more appointments (rs = .33, p = .047) but not with accuracy (rs = .07, p = .689).

Discussion

This study is the first, to our knowledge, to report on the performance of adolescents with ASD on the WCPA. Overall, the WCPA was quite cognitively demanding for youths who have ASD with average cognitive abilities, with many failing to successfully schedule any appointments and demonstrating an average accuracy lower than 25%. The most common error, other than failing to enter all appointments, was entering an appointment at the incorrect day or time into the calendar. As expected, adolescents were able to follow the five WCPA rules most of the time. However, youths with ASD only used a few strategies, including pause and reread, using fingers, and crossing or checking off entries, or highlighting appointments entered. Caregiver-rated EF was not significantly associated with WCPA performance. Exploratory correlations showed that lower IQ was associated with lower accuracy and less planning time and that an adolescent’s tendency to overestimate their performance was associated with entering more appointments.

Although the present study did not include any comparison group, published normative data (Zlotnik & Toglia, 2018) with typical adolescents (ages 12–18 yr) show that they have higher accuracy scores (>13), enter more appointments (18), and use more strategies (>4) than observed for youths with ASD in this study. In fact, as can be seen in Table 5, which provides a brief review of the literature using the WCPA in various clinical groups, adolescents with ASD in the present study performed more poorly (with fewer appointments entered and lower accuracy) on the WCPA than all other clinical groups, including adolescents (Fisher, Berger, Grossman, Tal-Saban, et al., 2022) and college students (Lahav et al., 2018) with ADHD; at-risk adolescents (Toglia & Berg, 2013); adults with neurodevelopmental disabilities and/or mental disorders, including ASD and ADHD (Wingren et al., 2022); and adolescents with acquired brain disorder (Doherty et al., 2022) and epilepsy (Zlotnik et al., 2020). It is quite telling that typical adolescents entered all 18 appointments on average (Zlotnik & Toglia, 2018), whereas youths with ASD entered only 8.7 on average. It is interesting that youths with ASD also took much less time than all other clinical groups and typical adolescents to complete the WCPA. Taken together, these findings (fewer appointments entered and shorter completion time) may indicate that the task was too difficult to execute and that students then gave up to avoid feeling stressed or overwhelmed. Another possible explanation for this finding is that students may have rushed through the task and, as a result, made errors caused by inattention (e.g., not checking that all appointments were entered). Both of the aforementioned explanations are in line with the subjective observations made by the WCPA examiners. The relatively high number of adolescents who entered incomplete appointments (n = 16) is also consistent with this hypothesis, as incomplete entries reflect a tendency to attend to only part of the appointment and miss key information or details (Toglia, 2015).

Table 5.

Abbreviated Summary of Relevant WCPA Literature

Variable	This Study	Fisher et al. (2022)	Lahav et al. (2018)	Toglia & Berg (2013)	Wingren et al. (2022)	Doherty et al. (2022)	Zlotnik et al. (2020)	Zlotnik & Toglia (2018)
Population studied	ASD	ADHD	ADHD	At risk	Various disorders, including ASD and ADHD	Acquired brain injuries	Epileptic	Typical
Sample size, n	39	52	61	104	38	14	20	109
Age, M (SD)	12.9 (0.96)	14.4 (1.6)	26.1 (2.6)	18.4 (1.3)	35.9 (10.7)	15.3 (2.1)	15.8 (1.5)	15.2 (1.9)
Age range	11–14	12–17	20–34	16–21	20–62	12–18	NR	12–18
M (SD) for
No. of appointments entered	8.6 (6.0)	16.8 (2.3)	NR	15.7 (2.8)	NR	16.2 (3.5)	15.9 (4.2)	18.0 (0)
No. of accurate appointments	4.3 (5.1)	12.9 (3.4)	12.8 (3.5)	7.8 (3.9)	10.3 (10.0)	13.0 (3.5)	10.7 (4.8)	13.9 (2.4)
No. of rules followed	3.5 (1.1)	3.8 (0.91)	NR	4.0 (1.0)	4.3 (4.0)	4.6 (0.64)	NR	NR
No. of strategies used	2.4 (2.2)	5.0 (2.7)	5.9 (2.1)	3.1 (1.9)	3.3 (3)	5.0 (2.5)	2.7 (1.5)	4.9 (2.7)
Planning time, in minutes	1.3 (0.9)	NR	NR	0.86 (1.2)	NR	3.6 (7.0)	NR	NR
Total time, in minutes	12.2 (7.8)	33.6 (14.1)	20.4 (9.9)	15.9 (5.4)	17.6 (17.5)	18.4 (6.7)	15.8 (5.1)	28.9 (14.6)

Note. Total time and planning time were converted to minutes. ADHD = attention deficit hyperactivity disorder; ASD = autism spectrum disorder; NR = not reported; WCPA = Weekly Calendar Planning Activity.

Although many of the adolescents with ASD in this study used strategies such as using their finger and crossing off or checking off entries, their use of these strategies was much lower than reported in typical adolescents who used these two strategies, as well as other strategies (e.g.,, entering fixed appointments first, self-checks) much more frequently (>37%; Toglia & Berg, 2013). In contrast, adolescents with ASD used the pause and reread strategy much more often than typical adolescents, who were observed using this strategy less than 25% of the time (Toglia & Berg, 2013; Zlotnik et al., 2020). It is possible that the use of the pause and reread strategy may reflect the adolescent being overwhelmed, inattention to directions, and/or lack of understanding of the task so that adolescents needed to implement this strategy rather than more active strategies (e.g., entering all fixed appointments). It has been noted that participants who take more time, use more strategies, and follow more rules have more accurate task performance (Weiner et al., 2012). Successful WCPA completion requires coordinated EF, including planning, flexibility, inhibition, and prospective memory (Toglia, 2015), and it would appear that the adolescents in this study were unable to overcome significant EF challenges to accurately enter appointments in this real-world proxy for EF. Cognitive strategies may help people process information efficiently and cope and manage challenges successfully (Toglia, 2015).

The fact that adolescents with ASD used less than three strategies on average is consistent with studies showing information-processing deficits in ASD (Belmonte et al., 2004), and with one other study that included participants with neurodevelopmental disorders, including ASD, which reported that the mean number of strategies used was 3.3 (Wingren et al., 2022). Note that adolescents with ADHD were reported to use approximately five strategies on average and took much longer (i.e., M = 33.6 min) to complete the WCPA than controls (Fisher, Berger, Grossman, Tal-Saban, et al., 2022). It has been suggested that deficits in cognitive flexibility and planning may be specific to ASD when comparing ASD, ASD plus ADHD, ADHD, and typically developing people (Craig et al., 2016), and the present findings are consistent with that suggestion. The lack of strategy implementation places greater demand on mental tracking, thereby increasing task difficulty and making accurate completion of the task more challenging (Toglia & Berg, 2013). In fact, several adolescents did not enter any appointments, were observed to be overwhelmed, and/or appeared confused about how to get started on the task. Furthermore, WCPA accuracy was moderately and significantly correlated with IQ, suggesting that cognitive abilities contribute meaningfully to performance, even in a sample with average IQ.

Although the task was markedly difficult for the adolescents in the present study, their scores were higher than in the Low Performance category (i.e., number of appointments entered, <4; accuracy of appointments entered, <4; rules followed, <0; and strategy use, <0) described in the WCPA manual (Toglia, 2015). This suggests that the 18-appointment version of the WCPA could still be used for youths with ASD. However, given that there are no norms for 11-yr-olds and that we observed possible floor effects (i.e., >10% of the sample did not enter any appointments at all and >30% did not enter any appointments accurately), we suggest that the short version (only 10 appointments) and the simplified format (Level 1, which includes copying a list of fixed and flexible appointments into a calendar) of the WCPA may be alternative options to consider for use with younger adolescents with ASD. (It is our understanding that there is no Level 1 in the adolescent version of the WCPA, which the task developers may want to consider developing.)

Perhaps not unexpectedly, given their ASD diagnosis and related metacognitive deficits (Carpenter & Williams, 2023), adolescents with ASD lacked insight into their poor performance. They frequently overestimated the number of appointments that they correctly entered into the calendar. This tendency toward overestimation has also been observed in adolescents with ADHD (Fisher, Berger, Grossman, & Maeir, 2022) and at-risk adolescents (Weiner et al., 2012). A significant association between performance and self-estimation has been reported with poorer performance associated with greater overestimation (Zlotnik & Toglia, 2018), and we observed that overestimation in youths with ASD was associated with more, but not more accurate, appointments entered. This phenomenon is known as the Dunning–Kruger effect, so that people lacking skills in various domains reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it (Fisher, Berger, Grossman, & Maeir, 2022; Kruger & Dunning, 1999; Zlotnik & Toglia, 2018). Metacognitive interventions have the potential to reduce this discrepancy (Goodman & Corkum, 2017). However, Fisher et al. speculated that poor awareness of task performance could contribute to poor treatment engagement and uptake of skills, so that not being able to detect poor performance on daily tasks may detract from youths’ motivation to learn and apply effective coping strategies (Fisher, Berger, Grossman, & Maeir, 2022). In fact, youths with ASD do seem to struggle with self-monitoring and the ability to apply feedback in the context of neuropsychological task performance (Robinson et al., 2009). In the present study, the majority of participants (n = 33) appeared to lack insight into their performance, with 29 of them overestimating the number of accurate appointments entered. More work is needed to explore whether overestimation mediates treatment response.

Surprisingly, WCPA performance was not significantly associated with any of the EF domains captured by the BRIEF–2. This is inconsistent with the literature that has reported significant correlations between higher caregiver-reported BRIEF General Executive Composite scores and lower WCPA accuracy in adolescents with acquired brain injuries (r = −.34; Doherty et al., 2022) and epilepsy (r = −.39; Zlotnik et al., 2020), as well as in typical adolescents divided into high and low WCPA accuracy subgroups (Zlotnik & Toglia, 2018). Thus, when used with autistic adolescents with ASD, the WCPA may measure aspects of EF that are not captured by the BRIEF–2 parent report, which suggests that it may be a complementary, real-world assessment of planning, working memory, and prioritizing. Alternatively, the mechanisms contributing to poor WCPA performance in adolescents with ASD may not be caregiver-rated EF deficits. It should be noted that the range of scores observed for the BRIEF–2 are within the range reported for people who have ASD without intellectual disability on the BRIEF–2 (Gentil-Gutiérrez et al., 2022; Gioia et al., 2015; Granader et al., 2015; Hutchison et al., 2020). Note that the average BRIEF scores in the Doherty et al. (2022) and Zlotnik and Toglia (2018) articles were in the normal range (<60; Zlotnik et al. [2020] did not report the BRIEF values) and that WCPA accuracy was much higher (>10 in the two studies); it is possible that the association between EF and WCPA performance is more evident when WCPA accuracy is less extreme.

Study limitations include the relatively small sample size, particularly given the heterogeneity of the sample, and that there was no comparison group without ASD. Also, because so few appointments were entered on average, there was less opportunity to investigate the pattern of errors (other than missing appointments) made by adolescents with ASD. The majority of participants in the sample were White males, potentially limiting generalizability and precluding our ability to explore possible gender differences. Additionally, we did not have other neuropsychological tasks to assess the individual EF constructs involved in completing the WCPA (inhibition, working memory, self-monitoring, etc.), so we could not explore their association with WCPA performance. Similarly, we do not include information on functional abilities, social skills, and school performance such as grades. Also, we only examined caregiver report on the BRIEF–2; there are often significant informant differences between caregivers and teachers (Pino Muñoz & Arán Filippetti, 2021), which may be partially due to observation setting (Tschida & Yerys, 2022). Although we did observe significant correlations between IQ and WCPA accuracy and planning time, the strength of these correlations were weak to moderate. Finally, the age range of participants included adolescents ages 11 yr, which is younger than in previously published work with the WCPA; although age did not correlate significantly with WCPA performance, the WCPA does not have norms for this age group, and the 18-item version of the WCPA may not be appropriate for use with adolescents younger than age 12 yr. Future work with larger samples, using alternate versions of the WCPA and including additional neuropsychological assessments, information on functional abilities, and EF informants (e.g., teacher) may help further explicate the performance and utility of the WCPA in adolescents with ASD.

Implications for Occupational Therapy Practice

Autistic youths have significant problems with EF, particularly organization, prioritization, time management, and planning, that affect them at home (e.g., getting started on homework assignments, organizing materials in their backpack) and school (e.g., writing down and prioritizing homework assignments in a planner). However, there are few objective, performance-based EF measures, especially those that can be used for youths with autism, that provide clinicians with an understanding of the profile of EF strengths and challenges. The WCPA assesses EF (i.e., planning, time management, prioritizing) by having youths schedule multiple appointments in a planner while following rules, avoiding conflicts, monitoring time, and being aware of distractions. The WCPA provides scores including accuracy, frequency of appointments entered, total planning time, and error types.

▪ Administering and interpreting the WCPA will allow clinicians to understand the adolescent’s ability to accurately record appointments (e.g., correct day and time, essential details) and use strategies to effectively organize appointments (e.g., how to enter fixed appointments vs. variable appointments, what specific strategies may increase accuracy and efficiency).

▪ Information gleaned from the WCPA could be used to develop a treatment plan to address EF difficulties by learning, practicing, and mastering strategies that an adolescent is not consistently using (e.g., entering fixed appointments first, entering all academic appointments before extracurricular appointments, double-checking the accuracy of appointment entered) to more effectively use a planner to manage and prioritize academics (e.g., recording daily homework assignments accurately, breaking down the steps of a project and writing down the steps on specific days, indicating when and how to study for a test) and other activities (e.g., doctor appointments, days off of school, extracurricular activities).

▪ Treatment could focus on teaching evidence-based organizational strategies using the adolescent’s own planner, which has more real-world applicability and relevance for middle school and beyond (e.g., high school, college, workplace).

▪ Understanding an adolescent’s challenges on the WCPA also has broad implications across a range of other tasks at home (e.g., following morning and nighttime routines, doing laundry from start to finish, getting chores done on the weekend) and school (e.g., writing down assignments accurately in a planner, bringing the necessary materials home from school each day in a backpack). For example, daily living skills such as getting ready to go to school in the morning typically have more than 18 steps (e.g., wake up, get dressed, brush teeth, wash face, clean glasses, take medicine, make breakfast, pack lunch, put laptop in backpack) and, as such, may be difficult for adolescents to complete independently and successfully without compensatory strategies (e.g., checklist of steps, phone alarm or reminder).

▪ Ideally, the WCPA should be used with youths ages 12 yr and older. Regardless of age, less complex or shorter WCPA versions may be most appropriate for adolescents with ASD (e.g., the short version with 10 appointments, or Level 1).

Conclusion

Overall, results show that the WCPA is a potentially useful tool for assessing EF deficits in adolescents with ASD. Practically, the results of this study suggest that adolescents with ASD may struggle to use a planner at school to write down their assignments and include the relevant, accurate information (e.g., due date, materials needed). Higher level tasks, such as breaking down the steps of a long-term assignment or the steps of studying for a test, may be particularly difficult to record in a planner. These results have implications for the use of evidence-based strategies to teach adolescents with ASD how to manage their own time and to plan and prioritize homework assignments, tests and long-term projects, extracurricular activities, and so forth. Such interventions are currently in development for youths with ASD without intellectual disability, with preliminary work showing improvement in these domains (Tamm et al., 2021, 2022). Adolescents with ASD may also overestimate their performance on tasks that require planning, which suggests that metacognitive interventions (Goodman & Corkum, 2017) or the use of specific strategies, such as double-checking things for accuracy or asking for clarification, may need to be specifically taught and practiced.

References

American Psychological Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596

Anderson

(2002). Assessment and development of executive function (EF) during childhood. Child Neuropsychology, 8, 71–82. https://doi.org/10.1076/chin.8.2.71.8724

Anderson

P. J.

Jacobs

, & Spencer-Smith

(2008). Development and assessment of executive function: From preschool to adolescence. In Anderson

Jacobs

, & Anderson

P. J.

(Eds.), Executive functions and the frontal lobes: A lifespan perspective (pp. 123–154). Taylor & Francis.

Belmonte

M. K.

Cook

E. H.

Jr ., Anderson

G. M.

Rubenstein

J. L.

Greenough

W. T.

Beckel-Mitchener

et al. Tierney, E. (2004). Autism as a disorder of neural information processing: Directions for research and targets for therapy. Molecular Psychiatry, 9, 646–663. https://doi.org/10.1038/sj.mp.4001499

Benallie

K. J.

McClain

M. B.

Bakner

K. E.

Roanhorse

, & Ha

(2021). Executive functioning in children with ASD+ADHD and ASD+ID: A systematic review. Research in Autism Spectrum Disorders, 86, 101807. https://doi.org/10.1016/j.rasd.2021.101807

Bolling

D. Z.

Pitskel

N. B.

Deen

Crowley

M. J.

McPartland

J. C.

Kaiser

M. D.

et al. Pelphrey, K. A. (2011). Enhanced neural responses to rule violation in children with autism: A comparison to social exclusion. Developmental Cognitive Neuroscience, 1, 280–294. https://doi.org/10.1016/j.dcn.2011.02.002

Braconnier

M. L.

, & Siper

P. M.

(2021). Neuropsychological assessment in autism spectrum disorder. Current Psychiatry Reports, 23, 63. https://doi.org/10.1007/s11920-021-01277-1

Carpenter

K. L.

, & Williams

D. M.

(2023). A meta-analysis and critical review of metacognitive accuracy in autism. Autism, 27, 512–525. https://doi.org/10.1177/13623613221106004

Chan

R. C.

Shum

Toulopoulou

, & Chen

E. Y.

(2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23, 201–216. https://doi.org/10.1016/j.acn.2007.08.010

10.

Craig

Margari

Legrottaglie

A. R.

Palumbi

de Giambattista

, & Margari

(2016). A review of executive function deficits in autism spectrum disorder and attention-deficit/hyperactivity disorder. Neuropsychiatric Disease and Treatment, 12, 1191–1202. https://doi.org/10.2147/NDT.S104620

11.

Demetriou

E. A.

DeMayo

M. M.

, & Guastella

A. J.

(2019). Executive function in autism spectrum disorder: History, theoretical models, empirical findings, and potential as an endophenotype. Frontiers in Psychiatry, 10, 753. https://doi.org/10.3389/fpsyt.2019.00753

12.

Demetriou

E. A.

Lampit

Quintana

D. S.

Naismith

S. L.

Song

Y. J. C.

Pye

J. E.

et al. Guastella, A. J. (2018). Autism spectrum disorders: A meta-analysis of executive function. Molecular Psychiatry, 23, 1198–1204. https://doi.org/10.1038/mp.2017.75

13.

Doherty

Dickerson

Kadela

Noetzel

Wolf

T. J.

Berg

, & Dodd

(2022). Initial validation of the weekly calendar planning activity with adolescents with ABI. OTJR: Occupation, Participation and Health, 42, 305–314. https://doi.org/10.1177/15394492221087191

14.

Fisher

Berger

Grossman

E. S.

, & Maeir

(2022). Online and intellectual awareness of executive functioning in daily life among adolescents with and without ADHD. Journal of Attention Disorders, 26, 870–880. https://doi.org/10.1177/10870547211031982

15.

Fisher

Berger

Grossman

E. S.

Tal-Saban

, & Maeir

(2022). Weekly Calendar Planning Activity (WCPA): Validating a measure of functional cognition for adolescents with attention deficit hyperactivity disorder. American Journal of Occupational Therapy, 76, 7606205010. https://doi.org/10.5014/ajot.2022.049028

16.

Friedman

, & Sterling

(2019). A review of language, executive function, and intervention in autism spectrum disorder. Seminars in Speech and Language, 40, 291–304. https://doi.org/10.1055/s-0039-1692964

17.

Gentil-Gutiérrez

Santamaría-Peláez

Mínguez-Mínguez

L. A.

González-Santos

Fernández-Solana

, & González-Bernal

J. J.

(2022). Executive functions in children and adolescents with autism spectrum disorder, Grade 1 and 2, vs. neurotypical development: A school view. International Journal of Environmental Research and Public Health, 19, 7987. https://doi.org/10.3390/ijerph19137987

18.

Gioia

G. A.

Isquith

P. K.

Guy

S. C.

, & Kenworthy

(2015). Behavior Rating Inventory of Executive Function, Second Edition, Professional Manual . Psychological Assessment Resources.

19.

Goodman

L. R.

, & Corkum

(2017). A metacognitive training pilot study for adolescents with autism spectrum disorder: Lessons learned from the preliminary stages of intervention development. Journal of Intellectual and Developmental Disability, 42, 204–210. https://doi.org/10.3109/13668250.2016.1210104

20.

Granader

Isquith

P. K.

Dudley

, & Kenworthy

(2015, February). Updated BRIEF profiles in children with autism spectrum disorders [Abstract]. Presented at the 43rd Annual International Neuropsychological Society Meeting, Denver, CO. https://www.the- ins.org/files/meetings/den2015/files/INS-Denver-2015_Preliminary_ Abstracts.pdf

21.

Huang

Kao

Curry

A. E.

, & Durbin

D. R.

(2012). Factors associated with driving in teens with autism spectrum disorders. Journal of Developmental and Behavioral Pediatrics, 33, 70–74. https://doi.org/10.1097/DBP.0b013e31823a43b7

22.

Hutchison

S. M.

Müller

, & Iarocci

(2020). Parent reports of executive function associated with functional communication and conversational skills among school age children with and without autism spectrum disorder. Journal of Autism and Developmental Disorders, 50, 2019–2029. https://doi.org/10.1007/s10803-019-03958-6

23.

Kaufman

A. S.

, & Kaufman

N. L.

(2004). Kaufman Brief Intelligence Test (2nd ed.). AGS.

24.

Kenworthy

Yerys

B. E.

Anthony

L. G.

, & Wallace

G. L.

(2008). Understanding executive control in autism spectrum disorders in the lab and in the real world. Neuropsychology Review, 18, 320–338. https://doi.org/10.1007/s11065-008-9077-7

25.

Kruger

, & Dunning

(1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134. https://doi.org/10.1037/0022-3514.77.6.1121

26.

Lahav

Ben-Simon

Inbar-Weiss

, & Katz

(2018). Weekly Calendar Planning Activity for university students: Comparison of individuals with and without ADHD by gender. Journal of Attention Disorders, 22, 368–378. https://doi.org/10.1177/1087054714564621

27.

Lai

C. L. E.

Lau

Lui

S. S. Y.

Lok

Tam

Chan

et al. Cheung, E. F. C. (2017). Meta-analysis of neuropsychological measures of executive functioning in children and adolescents with high-functioning autism spectrum disorder. Autism Research, 10, 911–939. https://doi.org/10.1002/aur.1723

28.

Lezak

M. D.

Howieson

D. B.

, & Loring

D. W.

(2004). Neuropsychological assessment (4th ed.). Oxford University Press.

29.

Lord

Rutter

DiLavore

P. C.

Risi

Gotham

, & Bishop

S. L.

(2012). Autism Diagnostic Observation Schedule , Second Edition (ADOS-2), Modules 1–4. Western Psychological Services.

30.

Panerai

Tasca

Ferri

Genitori D’Arrigo

, & Elia

(2014). Executive functions and adaptive behaviour in autism spectrum disorders with and without intellectual disability. Psychiatry Journal, 2014, 941809. https://doi.org/10.1155/2014/941809

31.

Pino Muñoz

, & Arán Filippetti

(2021). Confirmatory factor analysis of the BRIEF-2 Parent and Teacher Form: Relationship to performance-based measures of executive functions and academic achievement. Applied Neuropsychology: Child, 10, 219–233. https://doi.org/10.1080/21622965.2019.1660984

32.

Robinson

Goddard

Dritschel

Wisley

, & Howlin

(2009). Executive functions in children with autism spectrum disorders. Brain and Cognition, 71, 362–368. https://doi.org/10.1016/j.bandc.2009.06.007

33.

Tamm

Duncan

Vaughn

McDade

Estell

Birnschein

, & Crosby

(2020). Academic needs in middle school: Perspectives of parents and youth with autism. Journal of Autism and Developmental Disorders, 50, 3126–3139. https://doi.org/10.1007/s10803-019-03995-1

34.

Tamm

Risley

S. M.

Hamik

Combs

Jones

L. B.

Patronick

et al. Duncan, A. (2022). Improving academic performance through a school-based intervention targeting academic executive functions— A pilot study. International Journal of Developmental Disabilities. Advance online publication. https://doi.org/10.1080/20473869. 2022.2095690

35.

Tamm

Zoromski

A. K.

Kneeskern

E. E.

Patel

Lacey

H. M.

Vaughn

A. J.

et al. Duncan, A. W. (2021). Achieving independence and mastery in school: An open trial in the outpatient setting. Journal of Autism and Developmental Disorders, 51, 1705–1718. https://doi.org/10.1007/s10803-020-04652-8

36.

Toglia

(2015). Weekly Calendar Planning Activity: A Performance Test of Executive Function. AOTA Press.

37.

Toglia

, & Berg

(2013). Performance-based measure of executive function: comparison of community and at-risk youth. American Journal of Occupational Therapy, 67, 515–523. https://doi.org/10.5014/ajot.2013.008482

38.

Toplak

M. E.

West

R. F.

, & Stanovich

K. E.

(2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, and Allied Disciplines, 54, 131–143. https://doi.org/10.1111/jcpp. 12001

39.

Tschida

J. E.

, & Yerys

B. E.

(2022). Real-world executive functioning for autistic children in school and home settings. Autism, 26, 1095–1107. https://doi.org/10.1177/13623613211041189

40.

Weiner

N. W.

Toglia

, & Berg

(2012). Weekly Calendar Planning Activity (WCPA): A performance-based assessment of executive function piloted with at-risk adolescents. American Journal of Occupational Therapy, 66, 699–708. https://doi.org/10.5014/ajot.2012.004754

41.

Wingren

Lidström-Holmqvist

Roshanai

A. H.

Arvidsson

Janeslätt

White

, & Holmefur

(2022). One-year follow-up after the time management group intervention Let’s Get Organized. Scandinavian Journal of Occupational Therapy, 29, 305–314. https://doi.org/10.1080/11038128.2021.1954687

42.

Zlotnik

Schiff

Ravid

Shahar

, & Toglia

(2020). A new approach for assessing executive functions in everyday life, among adolescents with genetic generalised epilepsies. Neuropsychological Rehabilitation, 30, 333–345. https://doi.org/10.1080/09602011.2018. 1468272

43.

Zlotnik

, & Toglia

(2018). Measuring adolescent self-awareness and accuracy using a performance-based assessment and parental report. Frontiers in Public Health, 6, 15. https://doi.org/10.3389/fpubh.2018.00015