Prosodic cues in second-language speech processing: A visual world eye-tracking study

Abstract

Listeners interpret cues in speech processing immediately rather than waiting until the end of a sentence. In particular, prosodic cues in auditory speech processing can aid listeners in building information structure and contrast sets. Native speakers even use this information in combination with syntactic and semantic information to build mental representations predictively. Research on second-language (L2) learners suggests that learners have difficulty integrating linguistic information across various domains, likely subject to L2 proficiency levels. The current study investigated eye-movement behavior of native speakers of English and Chinese learners of English in their use of contrastive intonational cues to restrict the set of upcoming referents in a visual world paradigm. Both native speakers and learners used contrastive pitch accent to restrict the set of referents. Whereas native speakers anticipated the upcoming set of referents, this was less clear in the L2 learners. This suggests that learners are able to integrate information across multiple domains to build information structure in the L2 but may not do so predictively. Prosodic processing was not affected by proficiency or working memory in the L2 speakers.

Keywords

anticipation contrastive pitch accent eye tracking interface prosody second language processing

I Introduction

Speakers use prosody to convey particular meanings by manipulating the duration, loudness, pauses, or stress over particular words or sentences. For example, in We ate Angela’s cake but saved BENjamin’s . . . , the contrastive pitch accent placed on Benjamin’s indicates to the listener that two of the same objects will likely be contrasted (Angela’s cake and Benjamin’s cake). If the word cake receives a contrastive pitch accent instead, as in We ate Angela’s CAKE . . . , now the listener expects that Angela’s cake will be contrasted with another object Angela possesses (as in We ate Angela’s CAKE, but not her ice cream). Note that the initial clauses in both examples contain the same words and structure yet differ in the placement of the contrastive pitch accent. Processing prosody thus is a critical aspect of speech comprehension that informs the listener as to what information is given, new, or focused in a dialogue (Halliday, 1967), facilitating communication between interlocutors.

Despite the importance of prosody for language understanding, it is still unclear how second language (L2) listeners use prosodic cues. Evidence from prior studies suggests that native speakers are able to integrate information from different representational domains, such as prosody and syntax, while reading and listening (Ito and Speer, 2008; Patterson et al., 2017; Sedivy et al., 1995). On the other hand, theories of L2 processing predict difficulties for learners either in the integration of information across various representational domains (Interface Hypothesis; Sorace, 2011) or in complex mapping of information across domains (Patterson et al., 2017). Producing and recognizing L2 prosodic cues is difficult for L2 speakers, especially if prosody differs in form and function between the languages being compared (Mennen and de Leeuw, 2014). In the current study, we used a visual world eye-tracking paradigm (Tanenhaus et al., 1995) to investigate whether L2 learners are able to use contrastive pitch accent to build information structure and, if so, to what extent L2 learners use this prosodic cue predictively, and to what extent this depends on working memory and proficiency. The following sections provide an overview of prior studies investigating anticipatory processing of L1 (first language) and L2 prosody and the potential sources of differences between the two language groups.

1 Anticipatory processing of prosody in native speakers

Intonation can be used as an indication of information structure status, namely whether an element of a sentence has been introduced earlier, is new or is focused (Halliday, 1967). Informational focus is acoustically marked with a wider pitch range, an increased intensity, and increased duration compared to other speech elements that are not in focus (Wagner and Watson, 2010). This makes I gave the APPLE to John a felicitous answer to the question What did you give to John?, in which the direct object is probed; but not to a question in which the apple is given and the indirect object is probed, such as Who did you give the apple to?

Another type of focus is contrastive focus. In English, contrastive focus is marked by a low-falling, then high-rising F₀ contour, or L+H* pitch accent (Pierrehumbert and Hirschberg, 1990). Empirical data substantiates the claim that this type of pitch accent is used to denote a new referent, while a deaccented word in a set of referents indicates that the referent has already been mentioned (Terken and Nooteboom, 1987).

The literature suggests that native speakers are able to identify and integrate prosodic information to build information structure in real time (Dimitrova, 2012; Heim and Alter, 2006; Wang et al., 2011). ERP experiments in English (Johnson et al., 2003) and German (Hruska and Alter, 2004) show an increased N400 response at words that should have a focus pitch accent but do not, suggesting that prosody and expectations about focus status may immediately influence processing. Many studies have also looked at focus processing during reading. In these studies, focused target words typically elicit an increased late positive amplitude (e.g. Bornkessel et al., 2003; Cowles et al., 2007), which has been interpreted as reflecting integrative processing. Taken together, ERP evidence from both written and spoken language suggests that focus status influences processes related to both lexico-semantic interpretation and information integration.

Prior work on anticipatory processing of acoustic cues has investigated the role of contrastive pitch accent using a focused referent which is understood with respect to a set of alternatives. Evidence from such research suggests native listeners can use prosodic cues to anticipate upcoming information (Dahan et al., 2002; Ito and Speer, 2008; Nakamura et al., 2012; Sedivy et al., 1995). As an example, using a visual world paradigm, Ito and Speer (2008) had native English speakers decorate a Christmas tree according to spoken instructions. Objects were grouped in the same cell by type, and each object within a cell was a different color. When the L+H* pitch accent was placed on the second adjective, participants made anticipatory looks to the target object. When hearing an infelicitous L+H* contrastive pitch accent on the adjective (e.g. Hang the green ball. Next, hang the BLUE (L+H*) angel), participants produced garden-path fixations, that is, they looked initially to the cell with the same objects as the one previously mentioned. No such effect was found for the more neutral H* pitch accent. Native English listeners were able to use the prosodic cues to make predictions about the upcoming referent and were garden-pathed when the target noun was not the noun they had anticipated.

2 L2 processing of prosody

The literature on L2 learners’ use of L2 prosody shows mixed results. Some studies suggest that learners have problems producing and perceiving prosodic cues, especially if these differ from those prosodic cues in their L1 (Mennen and de Leeuw, 2014). For instance, Akker and Cutler (2003) compared phoneme detection in speakers of Dutch and English, languages that use prosodic cues similarly to inform information structure. Native speakers of both languages were quick to identify target phonemes when the target word was both in semantic focus and contained a predictive prosodic pitch in their respective L1s. However, Dutch L2 learners of English were not able to map pitch accent to semantic information as effectively as native speakers of English. Additionally, ERP studies using written question-answer pairs (Reichle, 2010; Reichle and Birdsong, 2014) reported that low-proficiency English learners of French did not show ERP indices associated with words in contrastive versus informational focus (clefts). High-proficiency learners, on the other hand, did show native-like negativity for contrastive focus, further suggesting that the online perception of information structure is affected by proficiency. Takahashi and colleagues (2018) found in a behavioral task that reaction times for Chinese learners of English were shorter for sentences with felicitous contrastive pitch accent on an adjective as compared to infelicitous sentences (e.g. Click on the scarlet necklace. Now click on the SCARLET (L+H*) mittens). The authors concluded that Chinese speakers were able to effectively use English contrastive prosodic cues likely due to the similarity in pitch contour shape for prosodic focus in English and Mandarin Chinese. The evidence thus far therefore suggests that L2 speakers are sensitive to modulations of L2 prosody but may have difficulty combining this information with other sources, especially when the L2 makes distinctions not present in the L1 and/or when the learner has not reached high proficiency.

3 Predictive processing in L2

As discussed above, native speakers appear to use linguistic cues, including prosody, rapidly to anticipate upcoming information. Research on prediction in L2 learners has yielded mixed results. On the one hand, a number of studies on L2 learners suggests that non-native speakers do not anticipate to the same extent as native speakers, even though they know the specific rules and words used when probed offline (Dussias et al., 2013; Grüter et al., 2012; Grüter and Rohde, 2013; Hopp, 2013; Lew-Williams and Fernald, 2010; Martin et al., 2013). Grüter et al. (2012) found a reduced ability for L2 learners to predict, substantiating this finding by proposing that the association between nouns and gender markers are weaker for L2 learners than for native speakers, affecting their ability to effectively predict in the L2. Namjoshi and Tremblay (2014) investigated English learners of French in an eye-tracking experiment and found that the L2 learners did not use contrastive pitch accent to anticipate upcoming referents using a visual world paradigm. However, neither did native speakers of French in this case.

Alternatively, other studies report that L2 learners can use linguistic information predictively, especially those with greater L2 proficiency (Hopp, 2013). Dussias et al. (2013) also found that only highly proficient English learners of Spanish used determiner information predictively, but not to the same extent as native speakers. Most recently, in Curcic et al. (2019), participants were taught a subset of Fijian nouns, verbs, adjectives, and determiners. The researchers interpreted the eye-tracking data as evidence that overall, learners were able to predict upcoming nouns based on determiner information. Yet, once the learner group was divided into those who were aware of the gender agreement between determiner and noun and those who were not, they found that only learners who claimed they were aware revealed significant anticipatory processing

Clearly prediction is not all or nothing, but affected by various factors (Kaan, 2014). Variation in the ability to predict has not only been attested among L2 speakers (Dussias et al., 2013; Hopp, 2013), but also among native speakers (Borovsky et al., 2012; Federmeier et al., 2002; Hopp, 2013; Huettig and Janse, 2012; Huettig et al., 2011). Previous studies report a relation between the ability to predict, vocabulary knowledge (Borovsky et al., 2012; Federmeier et al., 2002), consistency and speed of lexical access (Hopp, 2013), and working memory (Huettig and Janse, 2012; but see Otten and Van Berkum, 2009). In the current study, we test whether the ability to use prosodic cues predictively is modulated by working memory span or proficiency. If the mechanisms underlying predictive processing are the same in native speakers and L2 learners, one expects that factors that affect predictive processing in L2 speakers will also affect predictive processing in native speakers.

4 Accounting for differences between L1 and L2

Various accounts have been proposed to explain why L2 learners differ from native speakers in some aspects of sentence processing. We will discuss these approaches below and derive predictions as to L2’s ability to integrate information from disparate sources (in this study, syntactic, lexical, and prosodic) to build information structure and do so predictively.

First, according to the Shallow Structure Hypothesis (Clahsen and Felser, 2006), L2 learners do not build complex syntactic structures, but instead rely on contextual and lexical information and processing heuristics to understand sentences. Under this account, it is predicted that L2 learners will not build detailed information structure either and will not combine prosodic cues with other information to inform the information structure.

According to processing-based approaches, L2 learners can fully attain the knowledge of an L2; differences between L2 and native sentence processing can be attributed to a lack of resources in L2 speakers. L2 learners have fewer automatic processing routines and need to suppress their native language, which takes up processing resources. L2 speakers therefore differ from native speakers particularly in more complex processing situations, e.g. when different kinds of information need to be combined (Hopp, 2010; Sorace, 2011), or, in the case of predictive processing, when information needs to be accessed and combined rapidly to form predictions (Grüter and Rohde, 2013).

The Interface Hypothesis specifically posits that L2 learners specifically have difficulty in integrating information across domains when processing sentences (Sorace, 2011). These domains may be lexical, semantic, syntactic, visual, contextual, and most relevant to this article, prosodic in nature. Native speakers are experts at incorporating information from various domains to make meaning, but L2 learners tend to have more difficulties. Sorace and Serratrice (2009) found that for English-Italian bilingual children acquiring language, integrating contextual discourse-pragmatic information with syntax using overt and null subject pronouns resulted in greater difficulty processing sentences than when only coordinating syntactic and semantic information. Furthermore, this ability to integrate discourse-pragmatic information was not affected by structural overlap or lack thereof between the speakers’ two languages, suggesting learners are not able to reach native-like levels of processing when tasked with integrating information across domains even if they can identify the linguistic elements involved.

A slightly different take has been proposed by Patterson et al. (2017). Patterson et al. found that neither the Interface Hypothesis (Sorace, 2011), which predicts difficulty in integration of information across multiple representational domains, nor the Shallow Structure Hypothesis (SSH; Clahsen and Felser, 2006), which posits that learners rely primarily on semantic information over syntactic or prosodic information, could fully account for L2 performance in linguistic processing. Instead, results from their focus study on L2 pronoun resolution suggested that while L2 learners were likely able to combine information across domains when the mapping between them was one-to-one, complex mapping affected the extent to which they were able to successfully do so. In this article, we refer to Patterson et al.’s (2017) discussion of their findings as the complex mapping account. In sum, if one step of the process is broken in L2 processing (recognizing a prosodic cue, knowing its purpose, applying that knowledge to the overall syntactic structure and processing in terms of the affected lexical item), processing will not be native-like.

II The current study

This study focuses on L2 learner use of prosodic and lexico-semantic information (contrastive pitch accent and lexical information) to build information structure and restrict upcoming referents during listening. The questions investigated were as follow:

Do L2 speakers use contrastive pitch accent to build information structure while listening?

Do native speakers and L2 learners use this prosodic information predictively to restrict the set of upcoming referents?

Does each language group use contrastive pitch accent as a function of proficiency and working memory?

We used a paradigm inspired by Ito and Speer (2008) to investigate the use of contrastive pitch accent in native and L2 listeners. A visual world paradigm was used as an indirect measure of speech processing. Two avatars and two objects (two pictures of each object) were displayed on a computer screen while participants listened to short passages. Two context sentences preceded the critical sentence in each passage introducing the objects on the screen as belonging to one of the two avatars. Example critical sentences for each of the four conditions are provided in Table 1.

Table 1.

Example of the four experimental conditions.

	Contrastive pitch accent	Types of first noun	Example
			Context: Angela and Benjamin came to our potluck. They brought dessert.
A	–Accent	different	We ate Angela’s ice cream but saved Benjamin’s cake in the fridge.
B	+Accent	different	We ate Angela’s ice cream but saved BENjamin’s cake in the fridge.
C	–Accent	same	We ate Angela’s cake but saved Benjamin’s cake in the fridge.
D	+Accent	same	We ate Angela’s cake but saved BENjamin’s cake in the fridge.

The target proper name (Benjamin) either carried the contrastive pitch accent (+Accent conditions B, D) or carried no contrastive pitch accent (–Accent conditions A, C). Because the contrastive pitch accent signals to a listener that an upcoming referent is similar to one mentioned earlier, condition D (+Accent Same Object) is felicitous while condition B (+Accent Different Object) is not. On the other hand, no contrastive pitch accent is expected when the two mentioned objects are different. Therefore, condition A (–Accent Different Object) is felicitous while condition C (–Accent Same Object) is not as a contrastive pitch accent would be expected on the second proper noun when the two mentioned objects are the same.

If both native speakers and learners are able to integrate prosodic information in speech processing to aid with construal of information structure and lexico-semantic representations, we expect that participants will look towards the “same” object (i.e. Benjamin’s cake in C, D, and to Benjamin’s ice cream in A, B) relatively more or longer in the +Accent (B, D) than –Accent conditions (A, C). If this is the case, then findings from this study will challenge theories such as the Interface Hypothesis (Sorace, 2011), which would predict learners would not be able to integrate lexico-semantic and prosodic information, or the Shallow Structure Hypothesis (Clahsen and Felser, 2006), which would posit that learners apply less detailed processing of sentences compared to native speakers.

As for predictive ability, the evidence to date lends support to the notion that native speakers will be able to use the contrastive pitch accent to make anticipatory eye-movements toward the same-object referent (more anticipatory looks to Benjamin’s cake in D, and to Benjamin’s ice cream in B versus the –Accent conditions (A and C)). If L2 learners do not use contrastive pitch accent to predict the upcoming referent, they will not show a difference in looks to same-object referent in the +Accent (B, D) versus –Accent condition (A, C) before the target word is presented. The ability to use prosodic information predictively may be related to working memory capacity or English proficiency levels. We expect that those L2 speakers with higher working memory capacity and higher proficiency scores will look more often to the same object in the +Accent conditions and to the different object in the –Accent conditions (Hopp, 2009; Reichle and Birdsong, 2014). If this is borne out, this would contribute evidence that L2 speakers may not be lacking in L2 linguistic knowledge, and that differences in performance between L2 and native speakers are subject to processing resources or other factors (Kaan, 2014), including but not limited to task effects, motivation, or cognitive control.

Another consideration is that differences in predictive processing between native speakers and learners may be partly attributed to the disparity in the way languages encode and use certain information (Dussias et al., 2013; Foucart et al., 2014); we aimed to reduce effects of cross-linguistic differences by using languages wherein contrastive focus is encoded relatively similarly. In Mandarin, as in English, contrastive focus is prosodically cued. This differs from, for example, Spanish, in which focus is typically expressed through word order. It is important to note, however, differences in contrastive focus marked by prosody between English and Mandarin Chinese. While English contrastive focus is marked by an L+H* pitch accent (Pierrehumbert and Hirschberg, 1990; Watson et al., 2008), namely the shape of the fundamental frequency (F₀), in Chinese, information structure is denoted by the duration and range of the F₀ contour (Chen and Braun, 2006; Jin, 1996). As a tonal language that uses F₀ contour to convey semantic information, the shape of a lexical tone must be preserved for word recognition. English and Mandarin Chinese thus do have some inherently different factors that contribute to prosodically marked information structure.

III Method

1 Participants

We recruited 57 participants in the study. Data from five native English speakers and one Chinese learner of English were not recorded successfully due to technical issues in Data Viewer. Two more Chinese participants were omitted from analysis as their familiarity with the nouns used in the study was far lower than in other learners (83% and 73% compared to the 98% average). The remaining participant sample consisted of 27 native speakers¹ of English (4 males; Mage = 20.56, range 18 to 24) and 22 Chinese learners of English (6 males; Mage = 25.09, range 20 to 31) recruited from the University of Florida. On average, the L2 learners began acquiring English acquisition at the age of 8.75 years, with a range of ages 3 to 15 years. Participants completed the MELICET (MELICET; University of Michigan, 2001) and LexTALE English proficiency task (Lemhöfer and Broersma, 2012). The MELICET includes English grammar, cloze, vocabulary, and reading questions. The LexTALE provides a visual lexical decision task intended to measure vocabulary knowledge in advanced learners of English. In this study, we evaluate the composite z-score of results from both the MELICET and the LexTALE as an indicator of proficiency. Finally, participants completed an auditory digit span task in English (forward and backward). The sum total from both the forward and backward digit span tasks is presented below. Table 2 provides an overview of the proficiency and memory data for the two groups.

Table 2.

Mean values (and standard deviations) for behavioral tasks by language group.

	Age (years)	Age of acquisition (years)	MELICET	LexTALE	Digit span
Native speakers	20.6	n/a	45.9 (2.7)	92.4 (6.5)	15.8 (4.2)
Second language (L2) learners	25.1	9.0 (3.4)	33.8 (6.3)	67.2 (9.3)	14.0 (4.0)

Native speakers were significantly younger than the L2 learners (t(25) = −6.14, p < .001) and scored significantly higher on the MELICET (t(27) = 8.31, p < .001) and LexTALE (t(36) = 10.75, p < .001) than L2 learners. However, the two language groups did not differ in terms of their working memory scores based on the digit span task (t(46) = 1.58, p = 0.12).

Participants either received course credit or monetary compensation ($7.50/hour) for participating. All participants had normal hearing as tested on-site and normal or corrected-to-normal vision. The protocol was approved by the University of Florida Institutional Review Board, and all participants gave written informed consent.

2 Materials

a Experimental passages

We created 32 short passages that served as critical trials and 32 passages that served as filler items. The critical items consisted of three sentences as illustrated in Table 1. The first sentence introduced a male and a female subject using their proper names. Next, situational context was introduced, followed by the target sentence. The target sentence included the two actors and two objects (e.g. We ate Angela’s ice cream but saved Benjamin’s cake in the fridge). The second proper name was always three syllables long with contrastive pitch accent on the first syllable to allow time for effects of contrast to arise before listeners heard the second object.

Four types of fillers were created (eight of each type): passages in which the third sentence used a different connective between clauses than the target trials (e.g. and), passages that used different prosodic cues (e.g. Larry LOVES to paddle down the river, but Maggie thinks this is not very exciting), passages that mention of only one subject in the third sentence (e.g. Margaret found a nice shell and later saw a huge starfish), and passages in which the third sentence begins with a what-clause (e.g. What Lexi really hates are cigarettes, but she does not hate pipes as much). To check that participants were paying attention, comprehension questions followed 16 trials (25%), either critical or filler. Questions targeted the content of the passage and participants gave a ‘yes’ or ‘no’ response using a game controller.

Four lists were constructed. Every participant saw eight sentences from each of the four critical conditions, and no item was presented in more than one version on a list. Information from COCA (The Corpus of Contemporary American English; Davies, 2008) was used to ensure that for each of the four conditions, items were balanced in terms of frequency so that no one condition contained more frequent words than the others in a list. Each list contained all 32 fillers. Critical trials and fillers were pseudorandomized such that two critical trials never appeared twice in a row.

b Auditory stimuli

All audio materials were recorded by a female native speaker of English in a sound-attenuated booth. In order to elicit the auditory materials, the speaker read felicitous sentences out loud in blocks of 20. In the first half of each block, the speaker read only sentences in the –Accent Different object condition (condition A) for 20 trials. After this, the speaker read only the critical sentence in the +Accent Same object condition (condition D). The amplitudes of all the audio recordings were normalized using Praat (version 6.0.36: Boersma and Weenick, 1992–2017). For the –Accent conditions, the pitch contour was realized as L* for the second proper name (e.g. Benjamin) across all trials (see Figure 1), while the pitch contour was L+H* for the +Accent conditions (Figure 2).

Figure 1.

Example ToBI (tone and break indices) transcriptions of target proper name and noun for the –Accent conditions.

Figure 2.

Example ToBI (tone and break indices) transcriptions of target proper name and noun for the +Accent conditions.

Tables 3 and 4 provide the duration, minimum, maximum, and mean F₀ for the proper names and nouns in both the –Accent and +Accent conditions for the first and second clause, respectively. No significant differences were found between the first proper names and first nouns for any factors.

Table 3.

Mean duration and F₀ information (with standard deviation in parenthesis) for the first proper name (Angela’s) and first noun (ice cream or cake).

	Proper name		Noun
	–/+Accent different	+/–Accent same	–/+Accent different	+/–Accent same
Duration (ms)	441.98 (90.0)	476.41 (74.3)	487.22 (102.7)	475.67 (97.4)
Minimum F₀ (Hz)	203.47 (33.2)	208.76 (36.4)	169.32 (40.5)	162.98 (54.3)
Maximum F₀ (Hz)	161.58 (31.3)	165.24 (30.0)	131.11 (36.5)	115.18 (35.7)
Mean F₀ (Hz)	253.90 (94.3)	268.40 (114.7)	228.10 (118.2)	212.92 (99.7)⁺

Notes. ⁺p < .10 as per pairwise t-tests between +Accent and –Accent.

Table 4.

Mean duration and F₀ information (with standard deviation in parenthesis) for the second proper name (Benjamin’s) and second noun (cake).

	Proper name		Noun
	–Accent	+Accent	–Accent	+Accent
Duration (ms)	484.39 (53.6)	576.85 (52.7)***	367.32 (88.3)	444.46 (123.2)**
Minimum F₀ (Hz)	147.28 (31.0)	147.04 (21.2)	128.08 (32.5)	127.53 (33.8)
Maximum F₀ (Hz)	256.30 (129.1)	332.11 (61.7)**	215.03 (110.1)	235.4 (133.1)
Mean F₀ (Hz)	185.54 (34.1)	239.59 (17.3)***	164.73 (35.9)	164.6 (38.2)

Notes. ** p < .01 as per pairwise t-tests between +Accent and –Accent. *** p < .001.

For the second proper noun, items carrying a contrastive pitch accent had significantly longer duration (t(62) = −6.85, p < .001) and higher maximum F₀ values (t(62) = −2.95, p < .01) as well as higher mean F₀ values (t(62) = −7.87, p < .001) than non-accented items, as depicted in Table 4. The second nouns only differed in duration, wherein those in the +Accent conditions were longer than those in the –Accent conditions (t(62) = −2.88, p < 0.01).

Next, all recordings were spliced at the silence immediately following the final consonant in the conjunction but in the target sentence. Following this, the audio recordings were cross-spliced to create the infelicitous conditions (conditions B and C). Time stamps were set for the onset of the second proper name and the second noun in the target sentence.

c Visual display

Two avatars were presented in the order they were introduced from left to right. Avatars were depictions of a male and female, and both of their names were printed (in size 20 Arial font) underneath the respective picture to help the listeners link the names to the avatars. For critical items, 64 unique avatars were created (32 male and 32 female). The avatars were presented on the top third of the screen with a height of 190 pixels. Four objects then appeared underneath the avatars 5 seconds before onset of the second proper name. The objects corresponded to the two possessed items mentioned in the target sentence as indicated by a vertical line down the middle of the display. As such, underneath each avatar the same two objects were presented. In the case of the example in Table 1, a picture of ice cream and cake appeared under each avatar. The position of the target item (middle or bottom of the screen with width of 180 pixels) was counterbalanced across lists. Together, 128 unique pictures were used (64 critical trials and 64 fillers trials), and two of each object was used in each trial, as in Figure 3, for a total of 256 color pictures from an open source collection of clipart. AOI (areas of interest) were defined for the target object, the competitor, and the two other items (those images underneath the first proper name presented in the critical sentence).

Figure 3.

Example of a visual display in one trial of the eye-tracking experiment.

3 Procedure

Participants were seated in front of a computer screen and fitted with an SR EyeLink II head mounted eye-tracker. Next, the location of gaze fixations was calibrated. A set of instructions appeared on the screen, and participants advanced through the trials using a controller. Participants listened to the passages through headphones. A central fixation point appeared on the screen for 500 ms at the start of each trial. Participants saw five practice trials at the beginning of the experiment, two of which were followed by questions. Participants responded to comprehension questions by pressing one of two shoulder buttons on the controller. After all questions were answered, participants began the experiment. Upon completion, participants were administered a digit forward and digit backward task in English, followed by the LexTALE test (Lemhöfer and Broersma, 2012), the Michigan English Language Institute College English Test (MELICET; University of Michigan, 2001), the LEAP-Questionnaire (Marian et al., 2007), and a noun familiarity test. In the latter task, participants were presented the names of the avatars and nouns that were used in the eye-tracking portion and were prompted to rate their familiarity with each name and object on a Likert scale from 1 (very unfamiliar) to 7 (very familiar). The experiment took about 1 hour per participant for native speakers of English and 1.5 hours for Chinese learners of English.

4 Analysis

Eye fixation data were analysed using linear mixed-effects models with the lmer function (lme4 version 1.1.10; Bates et al., 2015) and lmerTEST package (version 2.0-33, Kuznetsova et al., 2016) in R version 3.4.2 (R Core Team, 2017). Mixed-effects models allow for the evaluation of empirical data for groups of different sample sizes, and trial and subject as random effects (Baayen, 2008). The dependent variable for the eye-tracking data was the natural log gaze probability ratio of looks to the same object versus looks to the different object beneath the avatar representing the second proper noun (e.g. Benjamin) ( $\log \frac{p r o p S A M E}{p r o p D I F F E R E N T})$ (see Barr, 2008; Jaeger, 2008; Ito et al., 2012). The proportion of looks to the same object was chosen as the dependent variable to directly assess the effects of the contrastive pitch accent on listeners. If contrastive pitch accent is used predictively to restrict the set of referents, listeners will look more at the same object as previously mentioned in the +Accent conditions (e.g. will look more at Benjamin’s cake than Benjamin’s ice cream after hearing . . . cake, but not BENjamin’s . . ., and will look more at Benjamin’s ice cream than Benjamin’s cake after hearing . . . ice cream, but not BENjamin’s . . . compared to the –Accent conditions). Fixation data were coded as either 1 or 0 depending on whether gazes were inside or outside of each area of interest (AOI), respectively. Mean log gaze probability ratios were calculated for five time windows of 200 ms each between –400 ms and 600 ms from the second-noun onset (e.g. cake). The analyses were time-locked to the second noun to investigate whether participants revealed anticipatory looks to the same object in +Accent conditions and to mitigate minor differences in the lengths of the proper name that precedes the second noun (see Table 2).

We analysed data separately for the two language groups. The fixed effects in the models constructed for analysis of each language group were contrastive pitch accent (negative for –Accent, positive for +Accent), target object type (negative for different and positive for the same target object from the first object mentioned), working memory (based on the sum total of the forward and backward digit span task scores), and proficiency (calculated as the composite z-score of each participant’s LexTALE and MELICET score after the scores were found to correlate). Values for each factor were centered around the mean. The mixed-effects models were run for the five time bins. The final model structure was:

log gaze ~ Accent + Target Object + WM + Proficiency + Accent:Target + Accent:WM + Target:WM + Accent:Proficiency + Target:Proficiency + Accent:Target:WM + Accent:Target:Proficiency + (1 | Subject) + (1 | Trial ID).

Using the natural log gaze probability of looks to the same object versus different object as the dependent variable, we aimed to directly assess the effect of contrastive pitch accent for native speakers and learners. We included working memory and proficiency to investigate whether these factors played a role in processing of contrastive pitch accent. Models with a more complex random effect structure failed to converge. We did not include interactions between working memory and proficiency as we made no predictions with regard to those effects.

IV Results

In an effort to reduce the likelihood that participants were using segmental information from the target noun and because one of the aims was to determine whether prosodic information was used predictively, we limited the length of the time bins analysed to 200 ms, as it takes 200 ms to 300 ms for listeners to plan and launch saccades to a target (Hallett, 1986; Viviani, 1990). Figures 4 and 5 show the log gaze probability ratio of looks to the same object by condition for English and Chinese speakers from 400 ms before to 600 ms after the second noun onset (e.g. cake) divided into five time bins of 200 ms each. The onset of the second noun (e.g. cake) occurs at 0 ms. The filled shapes represent the –Accent conditions and the open shapes represent the +Accent conditions. The Different conditions are marked by triangles and the Same conditions by circles. The felicitous conditions are the filled triangles (–Accent Different) and the open circles (+Accent Same).

Figure 4.

Log gaze probability ratio of looks to same vs. different objects (see main text) for the time range starting –400 ms before the second noun (cake) onset to 600 ms afterwards: Native speakers.

Figure 5.

Log gaze probability ratio of looks to same vs. different objects (see main text) for the time range starting –400 ms before the second noun (cake) onset to 600 ms afterwards: L2 learners.

As evidenced from Figures 4 and 5, listeners in both language groups preferred looking at objects that have not been mentioned: logit values tend to be lower than 0 preceding the onset of the second noun in all conditions, regardless of the presence or absence of the contrastive pitch accent over the second proper name. That is, participants were more likely to fixate on the cake when they heard ice cream in the first clause of the critical sentence and more likely to fixate on ice cream when they heard cake in the first clause. Eventually, these looks were modulated by contrastive pitch accent in the +Accent conditions. Below we break down results by language group.² Additionally, we assessed performance on the comprehension questions. An analysis revealed that L1 speakers (90.24% accuracy) performed better than L2 learners (80.99% accuracy) in responding to these questions (t = −3.2016, df = 40.085, p = 0.002).³

1 Native speakers

Results for native speakers are presented in Table 5. In Time Bins 1 through 4 (–400 ms to 400 ms after noun onset), the intercept was significantly smaller than 0, revealing that participants preferred to look at the different object versus the same object before and after the onset of the critical noun, which is also reflected in the top panel of Figure 4. In Time Bins 3 (0 ms to 200 ms) and 4 (200 ms to 400 ms), the effect of Accent was significant (Time Bin 3: β = −0.47, SE = 0.19, t = 2.51; Time Bin 4: β = −0.65, SE = 0.19, t = 3.38). That is, speakers looked more to the same object in +Accent conditions from 0 ms to 400 ms: in the corresponding time bins, the logit proportions of looks to the same object in the +Accent conditions (unfilled shapes, conditions B and D in Figure 4) are larger than those in the –Accent conditions. Also, Time Bin 3 reveals that Proficiency was a main effect (β = −0.23, SE = 0.10, t = −2.34), indicating that native speakers with higher proficiency scores looked more to the different object than the same object.

Table 5.

Results from mixed-effects models for native speakers of English for five separate time bins.

	Estimate	SE	T
Bin 1 (–400 ms to –200 ms):
(Intercept)	−0.37	0.09	−4.32***
Accent	−0.12	0.17	−0.74
Target object	0.24	0.17	1.40
WM	−0.02	0.03	−0.90
Proficiency	−0.07	0.09	−0.74
Accent:Target	−0.02	0.34	0.07
Accent:WM	−0.02	0.05	0.41
Target:WM	−0.01	0.05	−0.26
Accent:Proficiency	−0.22	0.18	1.21
Target:Proficiency	−0.11	0.18	−0.64
Accent:Target:WM	−0.18	0.10	1.75
Accent:Target:Prof	0.22	0.35	−0.63
Bin 2 (–200 ms to –400 ms):
(Intercept)	−0.55	0.11	−5.16***
Accent	−0.18	0.18	1.03
Target object	0.15	0.18	0.83
WM	0.01	0.003	0.19
Proficiency	−0.14	0.11	−1.20
Accent:Target	0	0.35	0
Accent:WM	−0.05	0.05	0.93
Target:WM	−0.02	0.05	−0.30
Accent:Proficiency	−0.17	0.18	0.93
Target:Proficiency	0.21	0.18	1.12
Accent:Target:WM	−0.17	0.12	1.60
Accent:Target:Prof	0.15	0.37	−0.41
Bin 3 (0 ms to 200 ms):
(Intercept)	−0.53	0.09	−5.73***
Accent	−0.47	0.19	2.51*
Target object	0.26	0.19	1.42
WM	0.03	0.03	0.90
Proficiency	−0.23	0.10	−2.34*
Accent:Target	−0.02	0.37	0.05
Accent:WM	−0.09	0.06	1.63
Target:WM	0	0.06	0.03
Accent:Proficiency	−0.15	0.19	0.77
Target:Proficiency	0.25	0.19	1.27
Accent:Target:WM	−0.21	0.11	1.84
Accent:Target:Prof	0.43	0.39	−1.10
Bin 4 (200 ms to 400 ms):
(Intercept)	−0.42	0.10	−4.43***
Accent	−0.65	0.19	3.38***
Target object	0.87	0.19	4.53***
WM	0.01	0.03	0.21
Proficiency	−0.16	0.10	−1.55
Accent:Target	0.25	0.38	−0.64
Accent:WM	−0.05	0.06	0.82
Target:WM	0.05	0.06	0.78
Accent:Proficiency	−0.25	0.20	1.23
Target:Proficiency	0.12	0.20	0.60
Accent:Target:WM	−0.01	0.12	0.05
Accent:Target:Prof	1.13	0.40	−2.82**
Bin 5 (400 ms to 600 ms):
(Intercept)	−0.10	0.09	−1.03
Accent	−0.19	0.19	1.00
Target object	2.27	0.19	11.96***
WM	0	0.03	−0.15
Proficiency	0.02	0.10	0.23
Accent:Target	0.48	0.38	−1.27
Accent:WM	0.01	0.06	−0.25
Target:WM	0.03	0.06	0.51
Accent:Proficiency	−0.25	0.20	1.25
Target:Proficiency	0.07	0.20	0.37
Accent:Target:WM	0.05	0.12	−0.43
Accent:Target:Prof	0.61	0.40	−1.54

Notes. * p < 0.05. ** p < 0.01. *** p < 0.001. Prof = proficiency composite score. WM = working memory score.

As expected, by Time Bins 4 (200 ms to 400 ms) and 5 (400 ms to 600 ms), native speakers looked more to the same object when it was the target object, and to the different object when this was the target (Effect of Target Object type). Finally, a three-way interaction of Accent, Target Object type and Proficiency was found in Time Bin 4 (β = 1.13, SE = 0.40, t = −2.82). This significant interaction revealed that looks to the same object decreased as proficiency increased when the target was a different object in –Accent conditions. Those with lower proficiency scores looked to the same and different object roughly equally for the –Accent conditions. There was no effect of Working Memory in any time bin.

As reflected by the significant effect of Accent in Time Bins 3 and 4, and as demonstrated by the upper panel in Figure 4, native speakers looked more often to the same object in both of the +Accent conditions compared to the –Accent conditions. This effect started prior to 200 ms, which is too soon to be driven by the target noun, and hence is indicative of predictive use of the prosodic cues.

2 L2 learners

The same model as that used for native English speakers was constructed for Chinese learners of English. The results are presented in Table 6. Similar to native speakers, Time Bins 1 (–400 ms to –200 ms) to 4 (200 ms to 400 ms) revealed that Chinese learners also preferred to look at the different object as compared to the same object (intercept significantly smaller than 0). We found a significant interaction of Accent, Target Object and Proficiency in Time Bin 1 (β = −1.03, SE = 0.47, t = 2.19). However, since the target object has not been presented at this point, we interpret this effect as spurious and will not further discuss this. In Time Bin 3 (0 ms to 200 ms), the effect of Accent was marginally significant (β = −0.35, SE = 0.19, t = 1.80). In Time Bin 4, Accent is a main effect (β = −0.43, SE = 0.21, t = 2.03), revealing that Chinese learners looked more often to the same object versus the different object in the +Accent conditions. In Time Bin 5, learners looked more to the same object when it was the target object, as expected (β = 1.33, SE = 0.21, t = 6.31). Furthermore, the interaction between Target Object and Proficiency suggests that learners with higher proficiency had a larger Target Object effect, with more gazes to the same object when the target object was the same as the one mentioned previously, and fewer when the target object was different (β = 0.77, SE = 0.31, t = 2.49). Working memory did not have any effect.

Table 6.

Results from mixed-effects models for Chinese learners of English for five separate time bins.

	Estimate	SE	T
Bin 1 (–400 ms to –200 ms):
(Intercept)	−0.27	0.08	−3.43***
Accent	−0.84	0.16	0.54
Target object	−0.04	0.16	−0.27
WM	−0.02	0.03	−0.64
Proficiency	0.21	0.12	1.75
Accent:Target	0.29	0.31	−0.94
Accent:WM	0.06	0.06	−1.02
Target:WM	0	0.06	0.06
Accent:Proficiency	0.19	0.24	−0.82
Target:Proficiency	0.09	0.24	0.39
Accent:Target:WM	−0.04	0.12	0.31
Accent:Target:Prof	−1.03	0.47	2.19*
Bin 2 (–200 ms to 0 ms):
(Intercept)	−0.26	0.10	−2.66*
Accent	−0.07	0.17	0.42
Target object	0.14	0.17	0.82
WM	0	0.04	−0.12
Proficiency	0.07	0.15	0.49
Accent:Target	0.29	0.35	−0.83
Accent:WM	−0.04	0.07	0.64
Target:WM	0.02	0.07	0.32
Accent:Proficiency	0.03	0.26	−0.13
Target:Proficiency	−0.08	0.26	−0.33
Accent:Target:WM	0.18	0.14	−1.34
Accent:Target:Prof	−0.03	0.52	0.06
Bin 3 (0 ms to 200 ms):
(Intercept)	−0.33	0.10	−3.36***
Accent	−0.35	0.19	1.80
Target object	0.27	0.19	1.40
WM	−0.03	0.04	−0.69
Proficiency	0	0.15	−0.02
Accent:Target	0.40	0.39	−1.03
Accent:WM	−0.09	0.08	1.16
Target:WM	0.02	0.08	0.25
Accent:Proficiency	−0.31	0.29	1.08
Target:Proficiency	0.07	0.29	0.24
Accent:Target:WM	0.18	0.15	−1.20
Accent:Target:Prof	−0.50	0.58	0.85
Bin 4 (200 ms to 400 ms):
(Intercept)	−0.45	0.12	−3.88**
Accent	−0.43	0.21	2.03*
Target object	0.34	0.21	1.62
WM	0	0.04	−0.07
Proficiency	0.13	0.17	0.77
Accent:Target	0.12	0.42	−0.28
Accent:WM	0.02	0.08	−0.24
Target:WM	−0.02	0.08	−0.31
Accent:Proficiency	−0.13	0.31	0.42
Target:Proficiency	0.60	0.31	1.91
Accent:Target:WM	0.20	0.16	−1.22
Accent:Target:Prof	−0.26	0.63	0.41
Bin 5 (400 ms to 600 ms):
(Intercept)	−0.21	0.11	−1.81
Accent	−0.35	0.21	1.67
Target object	1.33	0.21	6.31***
WM	0.06	0.04	1.25
Proficiency	0.16	0.15	1.02
Accent:Target	−0.27	0.42	0.63
Accent:WM	0.06	0.08	−0.77
Target:WM	0.03	0.08	0.41
Accent:Proficiency	−0.04	0.31	0.14
Target:Proficiency	0.77	0.31	2.49*
Accent:Target:WM	0.09	0.16	−0.54
Accent:Target:Prof	−0.11	0.62	0.19

Notes. * p < 0.05. ** p < 0.01. *** p < 0.001. Prof = proficiency composite score. WM = working memory score.

In sum, like the native speakers, learners showed more looks to the same object in the +Accent conditions than in the –Accent conditions. However, this effect become significant somewhat later than in the native English group.

3 Summary of the results

Both native speakers and L2 learners revealed a different-object bias as indicated by the significant negative intercepts in bins 1 through 4. However, contrastive pitch accent modulated this preference: when the proper name was accented, native speakers started to look more at objects that were the same as the object mentioned before (i.e. ice cream in condition B, cake in condition D). This effect of Accent was significant in the predictive time window, before 200 ms. L2 learners showed a similar effect but somewhat later than the native speakers. Furthermore, native speakers had increased looks to the different object immediately following the onset of the noun in the –Accent condition as proficiency increased. On the other hand, more proficient L2 speakers looked more to the target object in the final time bin.

V Discussion

In this study, we investigated whether L2 learners can use contrastive prosodic cues to build information structure and do so predictively. Furthermore, we investigated to what extent this effect was modulated by working memory capacity or proficiency. Results suggest that L2 speakers can use prosodic information to build information structure during listening, but we do not have strong evidence that they can predictively restrict the set of upcoming referents on the basis of prosody or that proficiency or working memory scores affected the use of the contrastive pitch accent to predict or process the subsequent noun phrase. In addition, we found a large bias for Different objects in both L2 and native speakers. These effects are discussed in detail below.

1 Different object bias

Both speaker groups investigated revealed a strong bias towards new objects. That is, speakers tended to look at the new object (as opposed to the already mentioned object) after having heard the first possessed noun. We postulate that the nature of the materials constructed resulted in this type of behavior. All critical sentences used the coordinator but (e.g. We ate Angela’s cake but saved Benjamin’s cake in the fridge). Recall that in building the materials, the recordings of the felicitous conditions (A and D) were cross-spliced after the silence in the conjunction but. The presence of this coordinator may have indicated to the listener that a contrast was upcoming, resulting in more looks to the new object prior to the listener hearing the second proper name. However, regardless of the different-object bias, in the +Accent conditions, native speakers started to look more to the same object after hearing the second proper name produced with an L+H* pitch accent but before they could have heard the target referent, indicating that they adjusted their predictions (cf. Ito and Speer, 2008).

2 The use of prosody by L2 speakers

The first aim of the study was to explore whether the contrastive pitch accent L+H* facilitates looks to particular referents for L2 speakers. Starting at 200 ms to 300 ms after the onset of the critical noun, Chinese learners of English looked more to the same object in +Accent than in the –Accent conditions. These findings reveal that learners were able to incorporate prosodic information to interpret linguistic information in this online study. Our results are in contrast with Namjoshi and Tremblay (2014)’s observations that English L2 learners of French were not able to integrate contrastive pitch accent and syntactic information. Our study differs in the timing of avatar presentation and layout of the visual display. In Namjoshi and Tremblay (2014), each image of the noun used was accompanied with an image of the possessor (an avatar). In our design, we presented the avatars and their names at the beginning of each experimental trial. A line down the middle of the display separated the two avatars and their possessions, making it easier for participants to determine which avatar had what. Participants in our study could therefore focus more on information in the speech stream, allowing them to map prosodic information to lexico-semantic information and information structure with greater ease.

We find that L2 learners are indeed able to integrate prosodic and lexico-semantic information quite similarly to native speakers, albeit with difference in timing (see next section). This challenges the Interface Hypothesis (Sorace, 2011), which predicts difficulty in L2 processing when integrating linguistic cues across different domains. The Shallow Structure Hypothesis (Clahsen and Felser, 2006) cannot explain these results in full since this approach posits learners predominantly rely on semantic information, suggesting they are less likely to interpret and incorporate prosodic information in speech processing. Instead, the complex mapping account (Patterson et al., 2017) and more general processing accounts (Hopp, 2013; Kaan, 2014) allow for the effective mapping of semantic and prosodic information to elicit sentence meaning, although this mapping requires more time and perhaps more effort on the part of the learners as compared to native speakers. Additional evidence in favor of the processing and complex mappings account is provided by an ERP study conducted by Lee et al. (2019). Lee et al. used participants drawn from the same L2 population and used the same materials as in the current study but found qualitative differences in brain responses between native and L2 speakers. Specifically, native speakers showed an increased N400 effect at the noun following the proper noun in the +Accent Different versus +Accent Same conditions, whereas the L2 speakers showed an N400 effect only in the –Accent Different conditions. According to Lee et al. (2019), these results suggests that L2 speakers used the prosodic cues differently from native speakers. In particular, the pitch accent may have facilitated lexical access in the L2 speakers, so the lack of accent and use of a non-repeated noun in the –Accent Different conditions may have elicited an increased negativity. Note that in the ERP study, no visual display was used. This could have made it harder to access the lexical items and restrict the set of referents, making the task more effortful and mapping of the prosody and lexico-semantic information more complex than in the present study. This supports the view that L2 processing can be similar to native speakers’ when the task is simple, as in the current study, but can be qualitatively different when the task is more complex.

3 Anticipatory processing of L2 prosody

Regarding anticipatory effects of the contrastive pitch accent, our results replicated those of previous studies on native speakers (e.g. Ito and Speer, 2008) supporting the view that native speakers make predictions based on cues provided by contrastive pitch accents. On the other hand, our learner data do not provide strong enough evidence that L2 learners used the contrastive prosodic cue predictively, or at least reliably so.

The delayed effect of Accent in the L2 speakers can be attributed to several factors, which are not mutually exclusive. First, the reduced ability to predictively use prosody to build information structure for learners, or to do so consistently, may also be attributed to the slight differences between marking of contrastive focus in English and Mandarin. As discussed in the introduction, empirical data suggest that in Mandarin, information structure is conveyed by the range of F₀ contours rather than the shape of the pitch contours (Chen and Braun, 2006; Jin, 1996). As a tonal language that uses F₀ contour to convey semantic information, the shape of a lexical tone must be preserved for word recognition. It is thus possible that even though learners are able to integrate the information sources across domains, they may not use all potential sources of information to anticipate referents because of the complexity involved and may rely on lexical and semantic information primarily and later integrate prosodic information (Clahsen and Felser, 2006; Kaan, 2014; Dussias et al., 2013; Hopp, 2013; Martin et al., 2013).

Alternatively, learners may be able to use prosodic information predictively if given enough time as processing is generally slower for learners compared to native speakers, especially if it requires the integration of information from various domains, and thus, the experimental paradigm used in this study may have not captured this phenomenon. Supporting evidence for our L2 speakers being overall slower to process the sentences is that the Target Object effect (which is dependent on the actual target word) is significant only 400 ms after the onset of the target word, whereas native speakers show this effect already 200 ms to 300 ms after onset of the word. A follow-up eye-tracking study including a set of materials that separates the proper noun from the possessed noun using an adjective (e.g. but saved Benjamin’s tasty cake . . .) would grant a clearer understanding of whether learners use contrastive pitch accent effectively to predict an upcoming referent.

Finally, we acknowledge that the apparent qualitative similarity in eye movements between the native and L2 groups does not mean that native and L2 speakers interpreted the contrastive pitch accent in the same way, or dealt with the infelicitous use of prosody in the same way. Different interpretational processes may also give rise to timing differences. Follow-up studies which do not include infelicitous sentences and which more explicitly probe the meaning of the sentence are needed to investigate this issue.

4 Effects of working memory and proficiency

Finally, we had posited that L2 proficiency and working memory span would affect learners’ ability to use prosodic information predictively during sentence comprehension. We did not find effects of working memory in either native speakers or learners. Using a forward and backward digit span task may not have been appropriate for our purposes, or may not have given us a wide enough range of scores. Instead, a Listening Span Task (Daneman and Carpenter, 1980) may have been more appropriate for testing working memory in relation to auditory sentence processing.

Proficiency did have an effect in some time windows, but these effects are hard to interpret. L2 speakers who were more proficient showed more looks to the target in bin 5. This suggest that these speakers were faster in recognizing the target word. We did not find any effects of proficiency on the L2 use of prosody, however. Unexpectedly, we did find that native speakers with higher English proficiency scores looked more at different objects from 0 ms to 200 ms after target noun onset and looked more at target objects in the –Accent Different condition between 200 ms to 400 ms after target noun onset. We speculatively interpret this as more proficient native speakers using the lack of accent more effectively as a cue signaling a different object.

VI Conclusions

Our findings show that L2 speakers are able to integrate prosodic and lexico-semantic information to build information structure. We, however, do not have strong evidence that learners do so predictively, or that this is affected by proficiency or working memory capacity. Our findings stand in contrast to other studies conducted to date that report that L2 learners have problems using prosodic information. Our results therefore challenge the Interface Hypothesis (Sorace, 2011). They provide support for approaches according to which L2 speakers can map information across representational domains, at least when the mapping is simple (Patterson et al., 2017) and not many resources are required (Hopp, 2013). Although further research is needed, our observations suggest that L2 learners can learn to successfully use L2 prosody if the task is relatively easy and involves a restricted number of referents. As prosody is not a topic commonly taught in the second-language classroom and is notoriously difficult for L2 speakers to acquire (Mennen and de Leeuw, 2014), these insights can help L2 curricula by designing simple scenarios in which prosody can be incorporated and by introducing more complex mappings and more complex situations step-by-step.

Footnotes

Acknowledgements

The authors thank Naomi Braun, Ethan Kutlu, and Julia Barrow for their assistance in constructing materials, and to Lisha Yang, Aleuna Lee, and Souad Kheder for running participants.

Authors’ Note

Michelle Perdomo is now affiliated with Vanderbilt University, TN, USA.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Michelle Perdomo

Edith Kaan

Notes

References

Akker

Cutler

(2003) Prosodic cues to semantic structure in native and nonnative listening. Bilingualism: Language and Cognition 6: 81–96.

Baayen

(2008) Analyzing linguistic data: A practical introduction to statistics using R. Cambridge: Cambridge University Press.

Barr

(2008) Analyzing ‘visual world’ eyetracking data using multilevel logistic regression. Journal of Memory and Language 59: 457–474.

Bates

Maechler

Bolker

Walker

(2015) Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67: 1–48.

Boersma

Weenick

(1992–2017) Praat: Doing phonetics by computer: Version 6.0.36 [program]. Available at http://www.praat.org (accessed September 2019).

Bornkessel

Schlesewsky

Friederici

(2003) Contextual information modulates initial processes of syntactic integration: The role of inter- vs. intrasentential predictions. Journal of Experimental Psychology: Learning, Memory, and Cognition 29: 871–82.

Borovsky

Elman

Fernald

(2012) Knowing a lot for one’s age: Vocabulary skill and not age is associated with anticipatory incremental sentence interpretation in children and adults. Journal of Experimental Child Psychology 112: 417–36.

Chen

Braun

(2006) Prosodic realization in information structure categories in standard Chinese. In: Hoffmann

Mixdorff

(eds) Proceedings of Speech prosody 2006. Dresden: TUD Press, pp. 313–16.

Clahsen

Felser

(2006) Grammatical processing in language learners. Applied Psycholinguistics 27: 3–42.

10.

Cowles

Kluender

Kutas

Polinsky

(2007) Violations of information structure: An electrophysiological study of answers to wh-questions. Brain and Language 102: 228–42.

11.

Curcic

Andringa

Kuiken

(2019) The role of awareness and cognitive aptitudes in L2 predictive language processing. Language Learning 69: 42–71.

12.

Davies

(2008–) The Corpus of Contemporary American English (COCA): 520 million words, 1990–present. Available at: https://www.english-corpora.org/coca (accessed September 2019).

13.

Dahan

Tanenhaus

Chambers

(2002) Accent and reference resolution in spoken-language comprehension. Journal of Memory and Language 47: 292–314.

14.

Daneman

Carpenter

(1980) Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior 19: 450–66.

15.

Dimitrova

(2012) Neural correlates of prosody and information structure. Unpublished PhD, Rijksuniversiteit Groningen, Groningen, Netherlands.

16.

Dussias

Valdés Kroff

Guzzardo Tamargo

Gerfen

(2013) When gender and looking go hand in hand: Grammatical gender processing in L2 Spanish. Studies in Second Language Acquisition 35: 353–87.

17.

Federmeier

McLennan

De Ochoa

Kutas

(2002) The impact of semantic memory organization and sentence context information on spoken language processing by younger and older adults: An ERP study. Psychophysiology 39: 133–46.

18.

Foucart

Martin

Moreno

Costa

(2014) Can bilinguals see it coming? Word anticipation in L2 sentence reading. Journal of Experimental Psychology: Learning, Memory, and Cognition 40: 1461–69.

19.

Grüter

Rohde

(2013) L2 processing is affected by RAGE: Evidence from reference resolution. Paper presented at the 12th conference on Generative Approaches to Second Language Acquisition (GASLA), University of Florida, Gainsville, FL.

20.

Grüter

Lew-Williams

Fernald

(2012) Grammatical gender in L2: A production or a real-time processing problem? Second Language Research 28: 191–215.

21.

Hallett

(1986) Eye movements. In: Boff

Kaufman

Thomas

(eds) Handbook of Perception and Human Performance: Volume 1. New York: Wiley, pp. 99–106.

22.

Halliday

MAK

(1967) Notes on transitivity and theme in English, part 2. Journal of Linguistics 3: 199–244.

23.

Heim

Alter

(2006) Prosodic pitch accents in language comprehension and production: ERP data and acoustic analyses. Acta Neurobiologiae Experimentalis 66: 55–68.

24.

Hopp

(2009) The syntax–discourse interface in near-native L2 acquisition: Off-line and on-line performance. Bilingualism: Language and Cognition 12: 463–83.

25.

Hopp

(2010) Ultimate attainment in L2 inflection: Performance similarities between non-native and native speakers. Lingua 120: 901–31.

26.

Hopp

(2013) Grammatical gender in adult L2 acquisition: Relations between lexical and syntactic variability. Second Language Research 29: 33–56.

27.

Hruska

Alter

(2004) Prosody in dialogues and single sentences: How prosody can influence speech perception. In: Steube

(ed) Information structure: Theoretical and empirical aspects. Berlin / New York: Walter de Gruyter, pp. 211–36.

28.

Huettig

Janse

(2012) Anticipatory eye movements are modulated by working memory capacity: Evidence from older adults. Paper presented at the 18th Architectures and Mechanisms for Language Processing Conference, Riva del Garda, Italy.

29.

Huettig

Rommers

Meyer

(2011) Using the visual world paradigm to study language processing: A review and critical evaluation. Acta Psychologica 137: 151–71.

30.

Ito

Speer

(2008) Anticipatory effects of intonation: Eye movements during instructed visual search. Journal of Memory and Language 58: 541–73.

31.

Ito

Jincho

Minai

Yamane

Mazuka

(2012) Intonation facilitates contrast resolution: Evidence from Japanese adults and 6-year-olds. Journal of Memory and Language 66: 265–84.

32.

Jaeger

(2008) Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language 59: 434–46.

33.

Jin

(1996) An acoustic study of sentence stress in Mandarin Chinese. PhD Dissertation, Ohio State University, Columbus, OH, USA.

34.

Johnson

Clifton

Breen

Martin

Morris

(2003) ERP Investigation of prosodic and semantic focus. Paper presented at the Annual Meeting of the Cognitive Neuroscience Society, New York, USA.

35.

Kaan

(2014) Predictive sentence processing in L2 and L1: What is different? Linguistic Approaches to Bilingualism 4: 257–82.

36.

Kuznetsova

Brockhoff

Christensen

RHB

(2016) lmerTest: Tests in linear mixed effects models. R package version 2.0-33 [online]. Available at: https://CRAN.R-project.org/package=lmerTest (accessed September 2019).

37.

Lee

Perdomo

Kaan

(2019) Native and second-language processing of contrastive pitch accent: An ERP study. Second Language Research. Electronic publication ahead of print version. Published online: 15 April 2019 (doi: 10.1177/0267658319838300).

38.

Lemhöfer

Broersma

(2012) Introducing LexTALE: A quick and valid Lexical Test for Advanced Learners of English. Behavior Research Methods 44: 325–43.

39.

Lew-Williams

Fernald

(2010) Real-time processing of gender-marked articles by native and non-native Spanish speakers. Journal of Memory and Language 63: 447–64.

40.

Marian

Blumenfeld

Kaushanskaya

(2007) The Language Experience and Proficiency Questionnaire (LEAP-Q): Assessing language profiles in bilinguals and multilinguals. Journal of Speech, Language, and Hearing Research 50: 940–67.

41.

Martin

Thierry

Kuipers

et al. (2013) Bilinguals reading in their second language do not predict upcoming words as native readers do. Journal of Memory and Language 69: 574–88.

42.

Mennen

de Leeuw

(2014) Beyond segments. Studies in Second Language Acquisition 36: 183–94.

43.

Nakamura

Arai

Mazuka

(2012) Immediate use of prosody and context in predicting a syntactic structure. Cognition 125: 317–23.

44.

Namjoshi

Tremblay

(2014) The processing of prosodic focus in French. Paper presented at the 27th annual CUNY conference on Human Sentence Processing, Columbus, OH, USA.

45.

Otten

Van Berkum

JJA

(2009) Does working memory capacity affect the ability to predict upcoming words in discourse? Brain Research 1291: 92–101.

46.

Patterson

Esaulova

Felser

(2017) The impact of focus on pronoun resolution in native and non-native sentence comprehension. Second Language Research 33: 403–29.

47.

Pierrehumbert

Hirschberg

(1990) The meaning of intonational contours in the interpretation of discourse. In: Cohen

Morgan

Pollack

(eds) Intentions in communication. Cambridge: MIT Press, pp. 342–65.

48.

R Core Team (2017) R: A language and environment for statistical computing [online]. Vienna: R Foundation for Statistical Computing. Available at: https://www.R-project.org (accessed September 2019).

49.

Reichle

(2010) Near-nativelike processing of contrastive focus in L2 French. In: VanPatten

Jegerski

(eds) Research on second language processing and parsing. Amsterdam: John Benjamins, pp. 321–44.

50.

Reichle

Birdsong

(2014) Processing focus structure in L1 and L2 French. Studies in Second Language Acquisition 36: 535–64.

51.

Sedivy

Tanenhaus

Spivey-Knowlton

Eberhard

Carlson

(1995) Using intonationally marked presuppositional information in on-line language processing: Evidence from eye movements to a visual model. In: Moore

Fain Lehman

(eds) Proceedings of the 17th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum, pp. 375–80.

52.

Sorace

(2011) Pinning down the concept of “interface” in bilingualism. Linguistic Approaches to Bilingualism 1: 1–33.

53.

Sorace

Serratrice

(2009) Internal and external interfaces in bilingual language development: Beyond structural overlap. International Journal of Bilingualism 13: 195–210.

54.

Takahashi

Kao

Baek

et al. (2018) Native and nonnative speaker processing and production of contrastive focus prosody. Proceedings of the Linguistic Society of America 3: 1–13.

55.

Tanenhaus

Spivey-Knowlton

Eberhard

Sedivy

(1995) Integration of visual and linguistic information in spoken language comprehension. Science 268: 1632–34.

56.

Terken

JMB

Nooteboom

(1987) Opposite effects of accentuation and deaccentuation on verification latencies for given and new information. Language and Cognitive Processes 2: 145–63.

57.

University of Michigan (2001) Michigan English Language Institute College English Test: Grammar, cloze, vocabulary, reading. Ann Arbor, MI: University of Michigan.

58.

Viviani

(1990) Eye movements in visual search: Cognitive, perceptual, and motor control aspects. In: Kowler

(ed) Eye movements and their role in visual and cognitive processes. Amsterdam: Elsevier, pp. 353–93.

59.

Wagner

Watson

(2010) Experimental and theoretical advances in prosody: A review. Language and Cognitive Processes 25: 905–45.

60.

Wang

Kadier

(2011) Prosodic encoding of focus in six languages in China. Proceedings of the 17th International Congress of Phonetic Sciences. Hong Kong, pp. 144–147.

61.

Watson

Tanenhaus

Gunlogson

(2008) Interpreting pitch accents in online comprehension: H* vs. L H*. Cognitive Science 32: 1232–44.