Validation of the Pictorial Infant Communication Scale for preschool-aged children with autism spectrum disorder

Joint attention and ASD

Although various definitions for the term “joint attention” can been found in the literature, in this article, it is used to refer to an individual’s capacity to coordinate or establish a shared focus of visual attention with a social partner in relation to some third object or event (Bakeman and Adamson, 1984; Bruner and Sherwood, 1983). ¹ Early joint attention skills allow the child to initiate or respond to interactions with social partners, and take various behavioral forms, including alternating eye contact and gestures such as reaching, giving, pointing, and showing. These nonverbal social communication behaviors can be further distinguished on the basis of function, or the purpose of the communicative message (Bates, 1979; Bruner, 1981; Wetherby and Prizant, 2002). Some behaviors are proto-declarative in nature and focus on coordinating attention primarily for the purpose of sharing interest or experiences (e.g. commenting). Others, however, serve an instrumental-imperative function and are used to elicit assistance in obtaining or manipulating objects (e.g. requesting).

In typical development, different behavioral manifestations of coordinated attention begin to emerge between 6 and 18 months of age, and continue to evolve during early childhood (Carpenter et al., 1998; Morales et al., 1998). However, joint attention remains an important component of social communication and cognition throughout childhood and well into adulthood (e.g. Adamson et al., 2013; Dunham and Moore, 1995; Mundy and Newell, 2007). Thus, while the majority of research examining joint attention development has historically focused on the infancy period, there has been a recent shift toward examining these skills in the preschool years, and beyond (Gillespie-Lynch et al., 2012; Gulsrud et al., 2014; Sullivan et al., 2015). It has been suggested that individual differences in these behaviors provide important information about the development of mental processes in infancy that are crucial to aspects of later social and cognitive development (Mundy, 2016). Supported by a pivotal skill theory, a pivotal skill (such as joint attention) is a skill that, when honed, produces enhancement of functioning in other areas (Hurwitz and Watson, 2015; Koegel et al., 1999). Many studies demonstrate that the development of joint attention skills is critically related to subsequent developmental outcomes, such as language, social competence, and overall cognitive development (e.g. Beuker et al., 2013; Carpenter et al., 1998; Freeman et al., 2015; Hurwitz and Watson, 2015; Sheinkopf et al., 2004; Tomasello and Todd, 1983; Vaughan Van Hecke et al., 2007). Therefore, the developmental connection between early joint attention and both early and later childhood development is well established in the literature.

Numerous labels have been used to refer to the various distinct, yet related, dimensions of coordinated attention. Consistent with the terminology originally proposed by Seibert et al. (1982) and sustained by Mundy and Newell (2007) and Mundy et al. (1988), responding to Joint Attention (RJA) refers to the ability to follow the gaze, head turn, and/or pointing gesture of another person (Scaife and Bruner, 1975). This social behavior allows the child to begin to map important input from a social partner to a specific referent, which is crucial to later social communication and language development (Mundy and Gomes, 1998; Tomasello and Farrar, 1986). Initiating Joint Attention (IJA) involves the use of eye contact and/or conventional gestures (e.g. pointing or showing) to spontaneously initiate coordinated attention with a social partner for the purpose of sharing affective experiences (Kasari et al., 1990). This component is related to the child’s ability to develop proper social communication and expressive behaviors. Initiating Behavior Requests (IBR) involves the use of eye contact and/or gestures (e.g. pointing or giving) to initiate attention coordination with another person for the purpose of eliciting aid in obtaining an object or event (Bates, 1976), and is an important foundation for functional communication. Together, these components allow the child to better cultivate his or her interaction skills, and are among the most widely studied in the development of early nonverbal social communication.

Triadic attention deficits are a persistent behavioral feature of ASD (Mundy et al., 1986). While research has long suggested that children with ASD do not exhibit an absolute deficit in coordinated joint attention skills per se, it does suggest that these children instead display a unique profile of abilities in this domain across time (Paparella et al., 2011). Specifically, while not all shared attention skills develop in children with ASD, research has found that those skills that do emerge in children with ASD may be sequentially or qualitatively different than typically developing children with similar language levels (Gulsrud et al., 2014; Paparella et al., 2011). Furthermore, relative to typically developing children and children with developmental delays, children with ASD also demonstrate a significant delay in the rate of their skill acquisition over time (e.g. Hurwitz and Watson, 2015). For example, Mundy and colleagues demonstrated that children with ASD may point or give to request as often as typically developing children (Mundy, 1995; Sigman and Ruskin, 1999), although it appears they use less coordinated eye contact during these IBR bids than mentally aged matched children with intellectual disabilities. Similarly, other studies report that the emergence of nonverbal requesting gestures (i.e. reaching, giving, and pointing) is largely similar among children with ASD and typically developing children (Charman et al., 1997; Paparella et al., 2011). The most seriously impaired nonverbal social communication behaviors in children with ASD are those related to IJA. Across a wide range of developmental levels, children with ASD rarely use and coordinate eye contact and gestures (i.e. pointing, showing) for the social purpose of sharing their experience of a toy or event (Gulsrud et al., 2014; Mundy, 1995; Mundy et al., 1994). RJA is also impaired in ASD, although children may show improvement with cognitive development (Mundy et al., 1994).

The delayed acquisition of joint attention abilities in children with ASD seems to have a detrimental impact on the child’s subsequent social development. Specifically, a failure to develop or refine basic communicative behaviors may serve to isolate the child with ASD from a typical pattern of social exchange (Mundy and Crowson, 1997). As a result, the child may struggle to establish shared states of attention during which social learning opportunities can occur, further negatively impacting social development. This suggests that impairments in joint attention both precede, and contribute to, the social behavioral impairments observed in children with ASD as they develop. This, combined with the demonstrated importance of joint attention skills to the development of language, has established joint attention as a primary target in early intervention programs (e.g. Kasari et al., 2008, 2012). Recent research demonstrated that children with ASD who used joint attention had significantly better language than children with ASD who did not, even after controlling for chronological and mental age (Hurwitz and Watson, 2015). Furthermore, the targeted treatment of joint attention skills in early childhood appears to influence growth in specific skills (coordinated eye contact, showing, pointing—i.e. IJA) over time (Gulsrud et al., 2014).

Precise measures of joint attention and related behaviors may therefore be particularly useful in research and clinical practice for numerous reasons. When examining specific aspects of joint attention, the literature over time consistently suggests that various types of joint attention skills differentially relate to developmental outcomes. For example, studies have shown that RJA is predictive of language skills (e.g. Morales et al., 2000) and pro-social behaviors (e.g. helpfulness) in early childhood (Sheinkopf et al., 2004). Additionally, IBR is a determinant for later “instrumental” attention-directing behaviors such as requesting assistance on a task. Furthermore, IJA aids in the development of the tendency to initiate social interactions and perspective-taking abilities; both of which are negatively affected in children with ASD (Mundy et al., 1987; Sigman and Ruskin, 1999). Additional lines of research indicate that IJA and IBR are also differentially related to measures of frontal brain activity (Henderson et al., 2002; Mundy et al., 2000). Given that these distinct, yet related, measures of coordinated attention display different patterns of growth and correlates with language development (Mundy et al., 2007), as well as different patterns of neural network correlates (Mundy, 2016; Mundy and Newell, 2007), quantitative measurement of these behaviors may be particularly helpful in identifying deficits and growth in this highly pivotal skill.

Measurement of joint attention skills

Measurement of nonverbal social communication skills poses a challenge for researchers and clinicians because it is influenced by many variables, including the social partner, the interactive context, the source of information, and the psychometric features of the measurement scale itself (Wetherby, 2006). Furthermore, joint attention can vary from person to person in both frequency and quality, often subtly, thus having a set guideline for measurement is very difficult. Nevertheless, given its importance, precise and quantitative measurement of joint attention is both valuable and essential. Two main approaches have traditionally been employed in assessing joint attention skills in young children with typical and atypical development. These include interactive, semi-structured assessments administered in a clinical setting, and parent- or teacher-report measures.

Methods

Participants

A total of 205 children were initially recruited; however, 7 children were excluded from data analysis because they did not meet study diagnostic criteria outlined below. A total of 198 children were enrolled in the study; however, one parent did not complete the PICS. Therefore, complete data were available for 197 children with ASD. Children were between the ages of 3 and 5 years (mean (M) = 47.60 months, standard deviation (SD) = 7.49 months) at enrollment. Autism Diagnostic Observation Schedule, Generic (ADOS-G; Lord et al., 1999) severity scores (M = 7.24, SD = 1.65) were used to obtain estimates of current ASD symptoms and to verify diagnosis. The Mullen Scales of Early Learning (MSEL; Mullen, 1995) Standard scores were used as an estimate of the child’s developmental level and scores in this sample ranged from 49 to 136 (M = 64.10, SD = 19.21). Expressive language skills were measured by the Preschool Language Scale, Fourth Edition (PLS-4; Zimmerman et al., 2002). Standard scores ranged from 50 to 129 (M = 69.73, SD = 23.02). The PLS-4 Auditory Comprehension (AC) standard scores ranged from 50 to 135 (M = 68.83, SD = 19.96). Relevant demographic data for the current sample are presented in Table 1, and a more comprehensive breakdown of participant demographics can be found in Boyd et al.’s (2014) article detailing the results of the parent project.

OPEN IN VIEWER

Table 1.

Child demographics.

	n	%
Child race
White	154	78.17
Black	26	13.20
Asian	10	5.08
Multi-racial	7	3.55
Child ethnicity
Non-Hispanic	129	65.48
Hispanic	68	34.52
Child gender
Male	162	82.23
Female	34	17.26
Missing	1	0.51
Household income
<US$20,000	25	12.69
US$20,000–39,999	35	17.77
US$40,000–59,999	28	14.21
US$60,000–79,999	25	12.69
US$80,000–99,999	20	10.15
>US$100,000	52	26.40
Missing	12	6.09
Caregiver education
Partial high school	3	1.52
High-school graduate/GED	40	20.30
AA	51	25.89
BS/BA	58	29.44
MS/MA/doctorate	41	20.81
Missing	4	2.03

GED: General Educational Development.

Caregiver education data are presented for the primary caregiver.

Inclusion/exclusion criteria

Students were enrolled in the larger study if they were between the ages of 3 and 5 years, had a prior diagnosis of ASD or developmental delay, and met diagnostic criteria on the ADOS-G or Social Communication Questionnaire (SCQ; Rutter et al., 2003). Overall, families were required to be proficient enough in English to complete parent rating scales, and children with any significant visual or hearing impairment, uncontrolled seizure disorder, or traumatic brain injury were excluded from the study (see Boyd et al.’s (2014) publication for complete characterization of the sample).

Measures

PICS

Developed by Delgado et al. (2001), the PICS is a brief, 16-item parent-report questionnaire in which parents are asked to rate how frequently their child has displayed joint attention behaviors during the previous 2-week period using a 4-point Likert scale (i.e. “not sure,” “never,” “sometimes,” and “frequently”) (see Table 2 for sample items). Appropriately named the “PICS,” the instrument employs photographs to aid parents’ understanding of the behaviors of interest. The PICS yields subscale scores for IJA, IBR, and RJA, as well as a Total Score. Subscale and Total PICS scores may range from 0 to 2; high scores indicating more optimal communication.

OPEN IN VIEWER

Table 2.

Items 1–16 of the PICS.

How often does your child show objects to you without giving them to you?

If you point to something behind your child that is interesting to see, how often does your child turn his or her head and look behind?

How often does your child let you know that he or she wants an object by looking at you and reaching for the object at the same time?

How often does your child look at you when he or she sees an interesting object?

How often does your child give an object to you to get help operating or opening it?

How often does your child point to an object to enlist your aid in obtaining the object?

When you point and look at something how often does your child look at the same object or event?

How often does your child point to indicate his or her interest in an object or event?

How often does your child show you an object but not let you take the object from him or her?

How often does your child use reaching as a sign to you to help him/her get an object?

When you look at and point to a toy how often does your child turn and look at the same toy?

How often does your child point to draw your attention to something?

How often does your child hand (or push) an object to you in order to give it to you?

When you point and look at something how often does your child look at the same object, even if that object is behind him or her?

When your child sees something interesting how often does he or she look at you like he or she is trying to share the event with you?

ESCS

As discussed earlier, the ESCS was initially created to capture nonverbal communication skills in typically developing toddlers (Morales et al., 2000; Mundy and Gomes, 1998), and was later utilized with children with ASD and other developmentally delayed populations (Mundy et al., 1994, 1995). This semi-structured assessment evaluates a child’s tendency to initiate and respond to coordinated joint attention and requesting behaviors with a trained clinician. Data from the ESCS measures of IJA, RJA, and IBR were examined in this study (see the “Coding” section for further definitions). Measurement of joint attention behaviors using the ESCS has been shown to successfully differentiate between typically developing children, children with ASD, and children with intellectual disabilities (e.g. Mundy et al., 1987). Trained examiners administered the ESCS in accordance with procedures outlined in the abridged ESCS manual (Mundy et al., 2003). Video-recorded administrations were coded for fidelity of procedures on both general administration guidelines (e.g. does the examiner only present one item/task at a time?) and task-specific administrations (e.g. was each mechanical toy and hand-operated toy presented three times?). Fidelity with the administration manual was rated at 80% or higher.

MSEL

The MSEL (Mullen, 1995) is a standardized, clinician-administered assessment that measures cognitive, language, and motor ability from birth to 68 months on five scales: Gross Motor, Visual Reception, Fine Motor, Expressive Language, and Receptive Language. The MSEL’s psychometric properties consist of test–retest reliability median values ranging from 0.76 to 0.84, inter-rater reliability ranging from 0.91 to 0.99, and median values of internal consistency ranging from 0.75 to 0.83 (Mullen, 1995). Furthermore, the validity of the measure for children with ASD has been established (Akshoomoff, 2006). Each subscale raw score yields a T-score, percentile rank, and age equivalent. In addition, an Early Learning Composite (ELC) standard score is calculated that entails the cognitive scales (Visual Reception, Fine Motor, Expressive Language, and Receptive Language) and serves as a general measurement of development and intelligence.

PLS-4

The PLS-4 is a clinician-administered assessment of language. The PLS-4 was designed for children from birth to 6 years 11 months, and it functions to assess language comprehension and expression of vocabulary, concepts, sentence structures, and grammatical markers. The PLS-4 has two subscales: Auditory Comprehension (AC) and Expressive Communication (EC). The AC subscale has 62 items that assess receptive language behaviors by testing the ability to comprehend and follow oral commands. The EC subscale has 68 items which assess the ability to communicate either through gestures and signs or spoken language. The subscale scores’ psychometric properties consist of test–retest stability coefficients ranging from 0.90 to 0.97, internal consistency reliability coefficients ranging from 0.66 to 0.95, and an inter-rater reliability coefficient of 0.99 (Zimmerman et al., 2002).

Autism diagnostic observational schedule, generic (ADOS-G)

The ADOS-G is a semi-structured, clinician-administered assessment that evaluates three core components of ASD, namely, communication and language, reciprocal social interaction, and restricted repetitive behaviors or interests (Lord et al., 1999). This play-based assessment is used to elicit desired behaviors or responses from a child, from which an autism severity score is derived (Gotham et al., 2006). The ADOS is one of the gold-standard autism assessments currently in use and has sound reliability and validity (Lord et al., 2000). In this study, administration was completed by research reliable assessors, along with the SCQ, to confirm ASD diagnosis, as part of the initial inclusion criteria. In addition, the IJA (i.e. spontaneous IJA) and RJA items on the ADOS Modules 1 and 2 were used as an additional index of concurrent validity. ADOS items are scored on a 3-point scale from 0 (no evidence of abnormality related to autism) to 2 (definite evidence). Some items include a code of 3 to indicate abnormalities so severe as to interfere with the observation; however, scores of 3 were converted to a 2 for analyses.

Social Communication Questionnaire (SCQ)

The SCQ is a 40-item parent-report questionnaire used to screen for symptoms associated with autism (Rutter et al., 2003). It includes questions about behaviors characteristic of autism between the age of 4 and 5 years, and currently. Total raw scores can range from 0 to 39, and the clinical cut-off score is 15. It was originally developed as a companion measure for the Autism Diagnostic Interview-Revised (ADI-R; Lord et al., 1994) and includes items similar to those on the ADI-R that were found to have discriminative diagnostic validity. The SCQ has established validity with the ADI-R and clinical diagnoses of autism, as well as adequate sensitivity (0.85) and specificity (0.75; Berument et al., 1999; Rutter et al., 2003).

Results

Prior to conducting targeted statistical analyses, potential effects of child and parent demographic characteristics on PICS scores were examined using Statistical Package for the Social Sciences (SPSS) version 21. Spearman’s rho correlations between PICS subscales (i.e. IJA, RJA, and IBR) and demographic variables (i.e. child ethnicity, child race, child gender, total household income, and caregiver education) yielded no significant associations. Next, reliability analyses were conducted using Statistical Analysis Software (SAS) 9.2 and revealed that the Total Score and subscales of the PICS were found to have a high degree of internal consistency (Cronbach’s alpha coefficients ranging from 0.72 to 0.89; Table 3). Results from the CFA were conducted using Mplus Software Version 6.12, and provided support for the established three-factor model, broadly representing IJA, IBR, and RJA (Table 4). Model fit was as follows: χ²(101) = 241.666, p < 0.0001, Comparative Fit Index (CFI) = 0.957, Root Mean Square Error of Approximation (RMSEA) = 0.084 (90% confidence interval (CI): (0.071, 0.098)), Weighted Root Mean Square Residual (WRMR) = 1.016. This model was judged to also have good fit to the data. ³ Construct validity was supported by intercorrelations among the factors. Specifically, the IBR and IJA scales were correlated at r = 0.93, p < 0.001, while RJA and IJA were correlated at r = 0.78, p < 0.001, and RJA and IBR were correlated at r = 0.61, p < 0.001. The large correlation between IBR and IJA suggested that they might not be distinguishable as separate factors, so a two-factor solution combining the items from the IBR and IJA scales onto a single latent variable was fit to the data. Fit for this two-factor model was as follows: χ²(103) = 268.659, p < 0.0001, CFI = 0.949, RMSEA = 0.090 (90% CI: 0.077, 0.104), WRMR = 1.091. A χ² difference test was conducted using the Mplus “difftest” utility and had the following results: Δχ²(2) = 22.769, p < 0.0001, indicating that further constraining the model to a two-factor solution significantly worsened model fit. Therefore, it was concluded that the three-factor solution should be preferred over the two-factor solution, although the difference in fit according to the relative fit indices was minimal.

OPEN IN VIEWER

Table 3.

Internal consistency reliability for subscales.

Scale	n	Alpha
Total	188	0.89
IJA	188	0.78
IBR	189	0.72
RJA	189	0.88

IJA: Initiating Joint Attention; IBR: Initiating Behavior Requests; RJA: Responding to Joint Attention.

OPEN IN VIEWER

Table 4.

Confirmatory factor analysis results.

Factor	Item	Loading	SE
IJA	1	0.501	0.060
	4	0.695	0.039
	8	0.849	0.025
	9	0.459	0.063
	12	0.835	0.025
	16	0.633	0.041
IBR	3	0.549	0.055
	5	0.522	0.060
	6	0.762	0.034
	13	0.605	0.053
	15	0.910	0.021
	10	0.338	0.068
RJA	2	0.816	0.036
	7	0.896	0.023
	11	0.902	0.021
	14	0.924	0.022

SE: standard error; IJA: Initiating Joint Attention; IBR: Initiating Behavior Requests; RJA: Responding to Joint Attention.

Factor variances fixed to one and all loadings free (n = 197).

Descriptive data for the PICS subscales scores and ESCS frequencies are presented in Table 5. To establish concurrent validity, Spearman’s rho correlations between the PICS and direct clinical measures of child joint attention and language skills were conducted (Table 6). As expected, significant and positive correlations were detected between the PICS IJA and ESCS IJA subscales (p < 0.05). Likewise, the PICS RJA subscale was positively correlated with the ESCS RJA subscale (p < 0.01) and negatively correlated with the RJA item on the ADOS (p < 0.05). In addition, PICS IJA ratings were significantly associated with ESCS IBR and RJA scores (ps < 0.05). Significant correlations were found between the PICS Total Score and both the ESCS IBR and RJA subscales (ps < 0.05). Additional analyses indicated that the IJA subscale on the PICS was significantly correlated with receptive language as measured by the PLS-4 (AC) and expressive language as measured by the MSEL (ps < 0.05). The PICS RJA subscale was also associated with the AC subscale of the PLS-4 (p < 0.05). The PICS IBR subscale was not significantly or marginally associated with either the ESCS, ADOS, or language scores.

OPEN IN VIEWER

Table 5.

Descriptive data for PICS ratings and ESCS frequencies (and percentages).

Measure	n	M	SD	Minimum	Maximum
PICS
IJA	25	1.19	0.46	0.33	1.83
IBR	25	1.47	0.39	0.67	2.00
RJA	25	1.22	0.52	0.00	2.00
Total	25	1.30	0.38	0.44	1.88
ESCS
IJA	26	10.73	10.28	0.00	38.00
IBR	26	18.04	7.46	2.00	37.00
RJA %	26	69.27	35.01	0.00	100.00

M: mean; SD: standard deviation; PICS: Pictorial Infant Communication Scale; ESCS: Early Social Communication Scales; IJA: Initiating Joint Attention; IBR: Initiating Behavior Requests; RJA: Responding to Joint Attention.

OPEN IN VIEWER

Table 6.

Associations between the PICS ratings, ESCS scores, ADOS scores, and language standard scores.

Scale	n	PICS: IJA	PICS: IBR	PICS: RJA	PICS: Total Score
Joint attention
ESCS: IJA	25	0.41*	0.04	0.13	0.27
ESCS: IBR	25	0.44*	0.24	0.30	0.41*
ESCS: RJA	25	0.60**	0.10	0.55**	0.51**
ADOS: IJA	23	−0.33	−0.03	−0.26	−0.26
ADOS: RJA	13	−0.52	−0.55	−0.64*	−0.63*
Language
PLS-4: AC	25	0.48*	−0.06	0.44	0.37
PLS-4: EC	25	0.34	−0.08	0.33	0.25
MSEL: RL	25	0.26	−0.20	0.26	0.13
MSEL: EL	25	0.40*	−0.06	0.28	0.27
ASD symptomatology
ADOS Severity Index	186	−0.27**	−0.05	−0.23*	−0.20*
SCQ Total Raw Score	180	−0.49**	−0.36**	−0.46**	−0.51**

PICS: Pictorial Infant Communication Scales; ESCS: Early Social Communication Scales; PLS-4: Preschool Language Scale, Fourth Edition; MSEL: Mullen Scales of Early Learning; IJA: Initiating Joint Attention; IBR: Initiating Behavioral Requests; RJA: Responding to Joint Attention; AC: Auditory Comprehension; EC: Expressive Communication; RL: Receptive Language; EL: Expressive Language; ADOS: Autism Diagnostic Observation Schedule; SCQ: Social Communication Questionnaire.

*

p < 0.05; **p < 0.01.

In addition, to explore how PICS scores may vary by autism symptom severity, the relationship between the PICS subscales and Total Scores and the SCQ Total Scores (n = 180) and ADOS-G-calibrated severity scores (n = 186) was examined using Spearman’s rho correlations. The ADOS-G-calibrated severity score was significantly and negatively correlated with PICS IJA (p < 0.001) and RJA (p < 0.01) subscales and PICS Total Score (p < 0.01; Table 6). Similarly, associations were detected between the SCQ Total Score and PICS IJA, RJA, and Total Score. Thus, parent ratings of better PICS IJA and RJA skills were associated with lower ASD symptom scores as observed by an examiner on the ADOS, and as rated by a parent on the SCQ. PICS IBR was not significantly related to ASD symptomatology on the ADOS (r = −0.05, ns), but it was significantly and negatively related to the SCQ Total Score (p < 0.001; Table 6).

Due to the lack of associations between the PICS IBR subscale and direct measures of nonverbal social communication, a within-subjects comparison of subscale scores was performed using a series of paired samples t-tests. Results revealed that the PICS IBR subscale score was significantly higher than scores on both the PICS IJA subscale (t(24) = 3.19, p < 0.01) and the RJA subscale (t(24) = 2.67, p < 0.05; Table 6). There was no significant difference detected between the PICS IJR and RJA subscales (t (24) = −.45, ns). Likewise, children had significantly higher scores on the ESCS IBR subscale than the ESCS IJA subscale (t (25) = −3.48, p < 0.01). This suggests that children’s performances and perceived competence varied across the different subscales, with IBR seemingly better developed than IJA and RJA.

Discussion

This study aimed to evaluate the internal reliability, construct validity, and criterion validity of the PICS in preschool children with ASD. Results from a large sample of preschoolers with ASD (n = 197) suggest that the PICS may be a reliable parent-report measure of joint attention. Specifically, our data suggest that there is high internal consistency within the subscales of this measure, as well as with the Total Score. Our results also suggest strong construct validity of the PICS in measuring nonverbal social communication skills among children with ASD, which was assessed by analyzing the intercorrelations between the subscales. Additionally, the CFA supports the appropriateness of a three-factor model (i.e. RJA, IJA, and IBR) over a two-factor model, where IBR and IJA are collapsed. This reinforces the multidimensional nature of joint attention (Mundy, 2016; Mundy et al., 2007; Mundy and Newell, 2007) and substantiates the need to distinguish function (requesting versus commenting) when evaluating social communication skills.

Finally, strong correlations between the PICS and direct clinical assessments of joint attention and language skills suggest that the PICS may be a promising tool for measuring joint attention in preschool-aged children with ASD. Specifically, our results demonstrated that the PICS IJA is positively correlated with the ESCS IJA. Likewise, the PICS RJA was positively correlated with the ESCS RJA and negatively correlated with the ADOS RJA. This suggests that in this sample, the PICS is a comparable measure of IJA and RJA to a clinician-administered measure such as the ESCS and ADOS. In addition, significant correlations were found between the PICS Total Score and both the ESCS IBR and RJA subscales. Additional analyses indicated that the IJA subscale on the PICS was significantly correlated with receptive language, as measured by the PLS-4, and expressive language, as measured by the MSEL. The PICS RJA subscale was also associated with the receptive language subscale of the PLS-4, again suggesting criterion validity across measures. The consistency with which PICS IJA was related to both receptive and expressive language suggests the PICS’ potential utility in measuring communication skills more broadly.

In addition to within-domain correlations (e.g. PICS IJA with ESCS IJA), associations across domains of nonverbal social communication were also detected. For example, the PICS IJA subscale was also highly correlated with both the ESCS RJA and ESCS IBR subscales. This is a somewhat surprising finding given that prior literature highlights the unique functions, different patterns of growth, and distinct correlates with language development between IJA, IBR, and RJA (e.g. Mundy, 2016; Mundy et al., 2007). Nevertheless, these measures also share a common mental process that involves triadic attention deployment (self, other, object, event), which is a necessary step to establish a common point of reference with others. Furthermore, the commonality among measures is especially high for IBR and IJA since their measurement involves some of the same behavioral forms (e.g. pointing), albeit utilized for different purposes (i.e. sharing interest versus requesting; P. Mundy, 27 October 2015, personal communication). Therefore, it is understandable that these measures were correlated in our sample.

Interestingly, the PICS IBR subscale was not significantly or marginally associated with either the ESCS or language scores. Given this pattern of findings, it appears that the PICS may be a more sensitive measure of IJA and RJA skills than IBR skills. It is possible that the lack of significant associations for the IBR subscale on the PICS may be due to the fact that IBR skills are fairly well developed in this sample, thus narrowing the range of data and limiting our ability to detect individual differences. Furthermore, results of the within-subjects comparison of subscale scores revealed that the PICS IBR subscale score was significantly higher than scores on both the PICS IJA and the RJA subscales, and that there was no significant difference detected between the PICS IJA and RJA subscales. Likewise, children had significantly higher scores on the ESCS IBR subscale than the ESCS IJA subscale, suggesting that children’s performances and perceived competence varied across the different subscales, with IBR seemingly better developed than IJA and RJA. This would be consistent with prior literature demonstrating that the development of IBR skills often occurs at an earlier age than those children included in our sample and is often among the first joint attention skills acquired in children with ASD (Gulsrud et al., 2014; Sigman and Ruskin, 1999; Yirmiya et al., 2006). An alternative explanation for these results may be that parents had some difficulty distinguishing between the communicative function of joint attention behaviors, even with detailed descriptions and pictures of such behaviors. As mentioned above, IJA and IBR share many of the same behaviors such as pointing; thus, it may be difficult for parents to completely differentiate them. If this is true then this may indicate a limit in the capacity of parents to make fine-grained distinctions about developments in different types of early social communication skills.

Yet, another possible implication of this pattern of results is that parent report on the PICS and direct observations on the ESCS provides sufficiently non-overlapping sources of information to suggest that they may be combined to provide a more complete picture of the development of young children with ASD. Semi-structured, observational measures of joint attention and related behaviors, such as the ESCS, offer a comprehensive assessment of multiple dimensions of nonverbal social communication, capturing the form, function, frequency, and quality of such behaviors. However, parent report is an essential component in evaluating joint attention and related behaviors as they occur in more natural environments. Ideally, in clinical and research settings alike, the combination of observational and parent-report measures from multiple sources (e.g. parent, teacher/caregiver, therapist) should be utilized to offer the most comprehensive picture of a child’s communicative repertoire (Tager-Flusberg et al., 2009). Thus, the PICS aims to complement the structured assessment process and provide a more rapid and cost-effective method of gaining parent-reported behavioral data. Information gathered with the PICS, when used in combination with the ESCS, ADI-R, or SCQ, may help decrease errors of commission. If a child scored low in IJA on the ESCS but was rated high in IJA on the PICS, one might question the possibility of an inaccurate or invalid ESCS assessment. Similarly, if an ADI-R or SCQ were to result in clinical level scores, but parent report on the PICS suggested typical social communication development, this may raise the question of inconsistent parent reporting and trigger the need for more precise symptom verification (e.g. ADOS).

This measure may also afford an index of risk of ASD based on parent report in young children if utilized as a screening instrument. The PICS is brief, and requires no training or a priori assessment to determine whether the report is completely knowledgeable about the behaviors being tested. Therefore, these data could conceivably be collected on more children, more quickly, and in more settings, than is possible with examiner-administered observational assessments like the ADOS. For example, the PICS could be used as an initial screener to provide an accurate index of risk of ASD based on parents’ report in young children. If a child were to display IJA delays alone, or in combination with IBR and RJA delays, this may indicate the need for further assessment of joint attention skills with more in-depth assessments (e.g. ESCS, ADOS). Data from the analyses investigating the relationship between PICS subdomains and ASD symptomatology provide support for the PICS utility in detecting risk of ASD. In particular, higher parent ratings of all PICS subdomains and Total Score were associated with lower total raw scores on the SCQ. The associations between PICS IJA, RJA, and Total Score and ASD severity on the ADOS were somewhat attenuated, although results were still statistically significant. Given the common method variance, however, it is not surprising that more significant associations were observed among parent-report measures of the PICS and SCQ than were observed between the PICS and ADOS. Of course, further investigation of specific patterns of scores, ranges, and/or cut-offs (e.g. 1.33 vs 1.75) is necessary to determine the full clinical impact of social communication delays as measured by the PICS. Nevertheless, the PICS could be used to supplement existing diagnostic measures to provide a more comprehensive profile of a child’s nonverbal communication skills.

The PICS may also be useful in assessing joint attention and related abilities for the purpose of predicting and/or monitoring response to treatment. As increasing numbers of interventions are focused on teaching or enhancing joint attention skills (e.g. Kasari et al., 2006, 2008), and therefore a quick and efficient measurement of these behaviors becomes increasingly more important. Measures of treatment response may provide information about which aspects of behavior are most (and least) likely to change in response to intervention (Rogers, 2001) and may also allow researchers to identify characteristics of children who will benefit most from specific kinds of intervention approaches (Mundy and Crowson, 1997). However, it will remain challenging to engage in research on the important dimensions of nonverbal social communication and joint attention development in ASD without more efficient, cost-effective measures, like the PICS.

Our findings, taken together with previous reports of high test–retest reliability and criterion validity in typically developing infants (Delgado et al., 2004), provide initial evidence of validation of the PICS as a promising parent-report measure of joint attention skills. As discussed earlier, weaknesses in joint attention are core features of ASD (Mundy, 1995). However, as supported by pivotal skill theory, joint attention skills that are increasingly developed produce enhancement of functioning in other areas (Hurwitz and Watson, 2015; Koegel et al., 1999). Therefore, as the prevalence of ASD is increasing (1 in 68 children; Centers for Disease Control and Prevention (CDC), 2014) and deficits in joint attention are solidly implicated in future language and cognitive development and increasingly targeted in early intervention studies, the PICS may be a useful tool to include in the comprehensive measurement of joint attention behaviors.

Limitations and future directions

There are, however, several procedural and sample-based limitations in the present study. First, the lack of the inclusion of a control group (neither typically developing nor developmentally delayed) is a limitation of this study. While this study provides evidence of adequate internal reliability, construct validity, and criterion validity, replication of the criterion validity analyses with a larger sample is a critical next step. Because only one site in the multisite study had research-trained ESCS administrators, only a small subset of the children who participated in the larger study were administered the ESCS (n = 25). A notable limitation of this study, authors also suggest that a lack of trained research staff is likely the case in many clinical and research settings, and it therefore strengthens the argument for the utility of parent-report measures. While our results are encouraging, replication with larger samples of children would certainly strengthen the proposed incremental value of the PICS.

Another noted limitation is the age range of children in the current sample. The photographs on the PICS version used in this particular study portray infants, whereas the participants in this study were of preschool age. The development of joint attention skills begin to emerge in the first year of life, but this skill continues to develop through the preschool years, particularly in children with ASD (Gillespie-Lynch et al., 2012; Gulsrud et al., 2014; Sullivan et al., 2015). While further validation of the PICS as a reliable tool for the measurement of joint attention in the earliest of years is a critical next step, results from this preschool population of children with ASD are encouraging. Additionally, longitudinal studies using the PICS in a preschool-aged sample to assess test–retest reliability would further determine whether the PICS is a sensitive measure for detecting behavioral change over time. This also has implications for the application of the PICS as a measure of development and treatment response. Taken a step further, validation of the PICS with very young (infant) populations will assist in determining the ability of this screening tool to identify young children with disabilities, particularly language delays and ASD.

While there is some evidence that PICS scores can successfully differentiate children with ASD from children with developmental delay (Thorp, 2004), any conclusions that may be drawn about the ability of parents to provide an accurate report of social communication behaviors are limited, as parents in this study were aware of their child’s diagnosis. It is possible that parents of children with ASD may have an altered expectation of their child’s social and communicative abilities, possibly reducing the PICS predictive value. Therefore, future research is needed to investigate the psychometric properties of the PICS with parents who have not yet received a clinical diagnosis. Additional studies comparing a mixed sample of at risk and typically developing children would be helpful in evaluating the usefulness of the PICS as a complementary measure that supports formal diagnostic tools. Further investigation of specific patterns of scores and/or cut-offs would also be useful in evaluating the clinical significance and potential index of risk for those children scoring. Finally, regarding participant demographics, the majority of our sample was Caucasian, which limits the generalizability of our results. Thus, although results are encouraging, the PICS requires further investigation in order to assure its validity as a primary measure of joint attention behaviors.

Conclusion

In conclusion, although direct observational measures of joint attention (e.g. ESCS, CSBS) exist, they are often laborious to administer in clinical and research contexts and do not often provide parent-reported descriptions of their child’s abilities at home. Alternative measures that are easier, cost-effective, and more parent-friendly are needed to supplement the assessment of these critical, pivotal skills. We believe that with the further psychometric validation suggested above, and with the promising psychometric properties already demonstrated in this study, the PICS may be one possible step in that direction. We are hopeful that this unique parent-report measure will prove to be a useful and impactful tool for clinicians (e.g. pediatricians, educators) and researchers alike in measuring joint attention skills in young children.