Impact of language familiarity on abstract pattern recognition in 9- to 12-month-old infants

Introduction

Infants need to develop a knowledge base that allows them to predict and optimally respond to future events occurring in their environment. To do so, they must be able to maintain and eventually retrieve the knowledge of their previous experiences. Crucially, infants’ memory system enables the infants to benefit from their past experiences rather than having to constantly re-learn with each encounter.

It is now established that infants have some basic form of memory during their first year of life (Bauer et al., 2010), even if there is still a debate on how this form of memory relates to the one present in adults (Jabès & Nelson, 2015). For instance, Infants show long-term memory (days) for odor (Delaunay-El Allam et al., 2010), operant conditioning (Rovee-Collier, 1997), and pictures of faces or objects (Dupierrix et al., 2015; Fagan, 1972).

The visual paired-comparison task (VPC) has been especially effective for examining the development of visual memory processing in infancy, and more specifically, recognition memory. Recognition memory is a basic form of information retrieval that involves judging whether a stimulus or event has been experienced before. The VPC test indexes the participant’s level of interest for one stimulus in a pair, after one of the two stimuli has been learned during a prior familiarization or habituation period. Recognition memory is therefore inferred from the participant’s tendency to fixate toward the novel stimulus. To increase the memory demand, a delay (e.g., minutes, days, months) can be added between the end of familiarization and the VPC test phase. Using this procedure, infants as young as 3 days of age show evidence of recognition memory, that is, a novelty preference, even when a 2-minute delay is imposed between familiarization and the VPC test (Pascalis & de Schonen, 1994). Retention over longer delays, from days to months, has also been demonstrated by 3–6 months of age (Bahrick & Pickens, 1995; Fagan, 1973; Pascalis et al., 1998).

Crucially, recognition memory has been found to be impacted by various factors. For instance, visual recognition memory retrieval for objects or faces fails if the color of the background is changed between the habituation and test phase before 18 months of age (Robinson & Pascalis, 2004) or if it happens in different rooms before 12 months of age (Jones et al., 2010). These studies suggest that infants are sensitive to the visual context when storing faces and objects in memory but do not provide information about context as a multimodal factor. Humans live, indeed, in a multimodal environment; hence, the process of learning about faces and objects is often accompanied by an auditory background (sound, language). Face are usually concomitant with language: The faces that infant usually see also talk to them. As such, one might expect that language interacts with face processing in infancy.

To study this question, Hillairet de Boisferon et al. (Experiment 2, 2021) measured the impact of language familiarity on face recognition in 9- and 12-month-old infants. During familiarization, Caucasian French-learning infants were exposed to a picture of an own-race Caucasian face. The face was associated with the auditory soundtrack of a nursery rhyme told either in French (native language) or in German (non-native language). Face recognition was measured in a subsequent VPC test phase where a novel Caucasian face was presented alongside the face infants had been previously familiarized with. In the native language condition, only 9- and 12-month-olds showed a preference for the novel face. This suggests that infants only memorized the faces of their own race when presented together with their native language. However, when presented with a non-native language, infants did not show evidence of face recognition (Hillairet de Boisferon et al., Experiment 2, 2021). Thus, language familiarity was shown to influence how infants learn and recognize the faces that surround them. This effect could be due to the fact that, in the studied infant population, the native language is usually spoken by (and thus associated with) own-race faces. As such, infants might specifically expect that each own-race face talks in their native language. Following this rationale, this effect should be specific to face-recognition memory. On the other hand, however, this result could also be explained as reflecting a general familiarity effect of the native language soundtrack. The unfamiliar non-native language could simply have distracted them during the familiarization, or overloaded their attentional resources, preventing them from memorizing the face. Under this assumption, this effect should not be specific to faces but also affect the recognition of other visual stimuli. One could expect that ambient familiar speech context could also influence how infants learn and memorize different stimuli of their visual environment. Indeed, it has been shown that ambient sound influences infant memory for an operant conditioning (Fagen et al., 1997). Furthermore, Senju and Csibra (2008) have shown that language has a strong communicative and referential value. From 6 months of age, infants seem to understand that native and non-native language can refer to and communicate about one object/pattern (real or not) (Colomer & Sebastian-Galles, 2020; Vouloumanos et al., 2014; Yamashiro & Vouloumanos, 2018).

However, to date, no study has explored the impact of the familiarity of ambient (when the sound is not coming directly from a face) language on visual stimuli recognition.

The goal of the present article is to test whether the type of auditory speech context (familiar-native or unfamiliar—non-native) influences learning and recognition memory of unknown visual patterns. To do so, we used the same abstract visual patterns as those used by Dupierrix et al. (2015) which are easily recognized by 6- and 9-month-olds when presented in silence during familiarization in a VPC task.

We tested 9- and 12-month-old French monolingual infants with the same paradigm as that used by Hillairet de Boisferon et al. (2021) (Experiment 2). However, instead of using faces, infants were presented with an image of an abstract pattern (Figure 1) during familiarization. For half of the infants, the abstract pattern was associated with an auditory soundtrack of a speaker reciting a story either in their familiar native language (French) or in an unfamiliar non-native language (German). Immediately after the familiarization, the familiar and a new abstract pattern were displayed side by side for the recognition test.

OPEN IN VIEWER

Figure 1.

Visual Patterns Used. Stimuli Were Displayed on a 22-inch Computer Screen. The soundtracks can be found here: https://osf.io/snr6u/?view_only=cd0383950d0a4b898ed9be8227552918.

If language familiarity has an impact on pattern-recognition memory, we expect 9- and 12-month-old to exhibit a preference for the novel pattern at test, but only in the familiar native condition. Alternatively, we should observe memory recognition after either the native or the non-native familiarization. Last, if we expect that the mere presence of an auditory speech soundtrack somewhat prevents visual recognition memory processes for non-social visual abstract patterns, we should not observe recognition memory at test, in any condition.

Method

Results

Each statistical test was accompanied by a detection of potential outliers according to the recommendations of Judd et al. (2017). No outliers were detected, and therefore, no other participant was rejected from the final sample. To check that there were no differences between the age groups, we performed a two-factor analysis of variance (ANOVA). For the familiarization and the test phase, no interaction was found between age and language, familiarization: F(7) = 3.31; p = .083; test: F < 1.

First, results showed that infants’ total looking time for the object during familiarization did not vary as a function of language familiarity, t(39) = −1.36, p = .18, d = .43, 95% CI [−0.20, 1.04], M_familiar = 25.5 s, SD = 3 s; M_unfamiliar = 23.9 s, SD_unfamiliar = 3 s.

Then the proportion of looking time toward the novel pattern was computed for both recognition tests by dividing the time infants spent looking at the novel stimulus by the total time they spent looking at both patterns (the novel and the familiar one). The preference for the novel pattern was used as an indicator of the recognition of the familiar visual pattern presented in the familiarization. We first conducted an independent Student t-test with language familiarity (familiar native vs. unfamiliar non-native) as a between-participants factor. The results showed no main effect of language familiarity: t(39) = −1.26, p = .21, d = .40, 95% CI [−0.23, 1.01], M_familiar = 54.8%, SD_familiar = 8.3%; M_unfamiliar = 51.7%, SD_unfamilar = 6.9%. Next, we compared the novelty preference scores against the 50% chance level using one-sample t-tests. Statistical analyses revealed a novelty preference only in the familiar native condition—familiar: t(20) = 2.67, p < .001, d = .58, 95% CI [0.11, 1.04]; unfamiliar: t(19) = 1.17, p = .26 (Figure 2).

OPEN IN VIEWER

Figure 2.

Novelty Preference Score as a Function of Language Familiarity. The hatched line represents the chance level (50%). In each condition, the number of participants is 21. For the familiar native condition: 95% CI [0.11, 1.04], and for the unfamiliar non-native condition: 95% CI [−0.13, 0.77].

Discussion

The present study aimed at determining if language familiarity impacts non-social visual stimuli recognition in a similar way it influences face recognition (Clerc et al., 2022; Hillairet de Boisferon et al., 2021, Experiment 2). Our results support this assumption. Both 9- and 12-month-old infants show recognition memory for abstract visual pattern, but only when it was simultaneously presented at familiarization with a nursery rhyme in their native language. There was no sign of recognition when the nursery rhyme was in an unfamiliar language. Importantly, participants spent the same amount of time looking at the visual pattern in both conditions during familiarization, suggesting that the amount of visual attention was equivalent.

First, these results could be due to a competition between visual and auditory processing combined with familiarity-novelty effects. Indeed, hearing a novel language could attract infants’ auditory attention and distract them from learning the visual stimulus, while still looking at it. Processing a novel auditory speech stream could overload their attentional resources, preventing them from processing and memorizing the visual pattern. Alternatively, hearing a native language would not affect their attention at the same level, as it is more familiar.

This hypothesis is supported by the previous literature, showing that infants’ vision and audition do not follow the same developmental trend. In humans, hearing begins around 27–28 weeks in utero (Galambos & Hecox, 1978; Litovsky, 2015), whereas vision is the longest sense to mature (Graven & Browne, 2008). Indeed, several studies have shown that the visual acuity of newborns was about 40 times worse than that of adults (Brown & Yamamoto, 1986; Maurer & Lewis, 2001; Mayer & Dobson, 1982; Miranda, 1970; Van Hof-Van Duin & Mohn, 1986). The visual acuity reaches an adult level around the age of 4/6 years (Maurer & Lewis, 2001; Mayer & Dobson, 1982; Van Hof-Van Duin & Mohn, 1986).

Auditory information could be then more salient than visual information at 9 and 12 months of age, making the impact of language novelty more important in these age groups. Further studies are required to assess whether the impact of the auditory modality over the visual modality decreases in older children recognition memory, while the visual system matures.

Interestingly, instead of reflecting an attentional limit, these results could also be triggered by memorization issue: They could show a direct influence of ambient language familiarity, which is not especially directed or related to the object (visual pattern), on object recognition memory. Being in a certain auditory environment could have triggered (in the familiar native language) or overwhelmed (in the unfamiliar non-native language) the memory system. Such interpretation needs also to be considered carefully, as very little is known in the adult or the infant literature on the effect of context familiarity on learning. Indeed, classical faciliatory effects of same contexts between encoding of the information and test are due to memory-retrieval mechanisms rather than learning processes per se (Hupbach et al., 2008). Here, we hypothesize that ambient language familiarity context could influence memory encoding mechanisms during the learning process, not just the recuperation of the newly encoded information (Schroers et al., 2007). In line with this idea, infants exposed to silent familiarization conditions can learn and recognize as efficiently as the one learning with their native language (Dupierrix et al., 2015), not better or worse. It is thus reasonable to suggest that the context of a non-native language at familiarization could prevent the memorization of the object rather than its simple retrieval at test.

Indeed, Borghini and Hazan (2018) have shown in adults a difference in pupillary diameter when listening to an unfamiliar language, reflecting a difference in cognitive processing compared to listening to a native language.

Infants may also use different language-processing mechanisms depending on the familiarity of the language they are listening to. Studies in infants show that when confronted with a non-native language, they seem to focus more on the mouth of the speaker than when listening to a native language (Lewkowicz & Hansen-Tift, 2012; Barenholtz et al., 2016; Ter Schure et al., 2016). However, as we are not using faces but abstract patterns, we obviously cannot validate the hypothesis that they would look more at one part of the pattern when they are listening. However, if they have similar language-processing mechanisms to adults from infancy, it would be interesting in future studies to add oculometric or pupil measurements to confirm or not whether, in the case of abstract pattern memorization, similar mechanisms could justify our behavioral observations.

Furthermore, contrary to our findings, in which native language does not seem to hinder the recognition of unfamiliar abstract patterns, Clerc et al. (2022) have shown that the presence of a native language hinders the recognition of unfamiliar faces. Infants are initially able to recognize individuals from any type of own-race and other-race faces. However, from 9 months, infants maintain the ability to discriminate own-race faces but have increased difficulty discriminating other-race faces. Familiar faces being usually associated with native (familiar) language when infants see an unfamiliar face, they might expect, or not be surprised, to speak an unfamiliar language. The expectation for patterns might be different.

Hence, further studies with infants should tease apart the role of language familiarity context on memory encoding and memory retrieval. One possibility would be to use a similar VPC task and manipulate the congruency of the language familiarity context between encoding at familiarization and retrieval at test. If, for instance, playing the unfamiliar language both at familiarization and test allowed infants to recover memory recognition for abstract patterns, then this would suggest that the present results are due to memory retrieval processes. Conversely, if infants were still unable to recognize the pattern at test, it would suggest that the present results are indeed due to the effect of ambient language on the learning processes at play during the memory encoding process.

Another potential explanation of these results is the specific linguistic nature of the speech stream. Prior studies suggest that language conveys a strong communicative and referential value (Senju & Csibra, 2008). Therefore, it is possible that only the infants in the native condition inferred that the speech stream was somewhat referring to the visual pattern. This could have allowed them to direct most of their attentional resources at the visual stimulus to memorize it. In the non-native unfamiliar condition, however, infants might not have made the same inference, preventing them from allocating their attentional resources to the abstract pattern and favoring the deciphering of the non-native speech stream, while still looking at the screen (where the auditory soundtrack comes from). Importantly, such interpretation needs to be considered with caution as it appears less coherent with previous literature: As early as 6 months and up to 12–14 months of age, infants seem to understand that both native and non-native speech can refer to and communicate about objects (Colomer & Sebastian-Galles, 2020; Vouloumanos et al., 2014; Yamashiro & Vouloumanos, 2018). Yet, in all these studies, the native and non-native speech streams were always presented audiovisually, depicting two social agents interacting with each other, transferring information about third parties. In the present study, neither social agents nor referential communicative cues (e.g., gaze direction) were presented visually. The communicative intent was only presented auditorily, by the use of infant-directed speech (Fernald & Kuhl, 1987; Fernald & Mazzie, 1991; Gleitman et al., 1984).

Further studies should thus explore whether language familiarity influences how infants conceive the referential nature of language, when the referential cues are scarcer (e.g., when the speech signal in only presented auditorily). To test whether the present results are due to language-specific mechanisms, one possibility is to use the same VPC task but present the abstract pattern at familiarization along with a familiar music soundtrack or an unfamiliar music soundtrack (instead of a nursery rhyme in a familiar native or unfamiliar non-native language). If this is a language-specific phenomenon, infants should memorize equally well regardless of the familiarization condition. Conversely, if memorization of an object is not triggered by language-specific mechanisms, we should observe poorer memorization with unfamiliar music.

Conclusion/Impact

These results bring new data in the understanding of the effects of the environment of infants on their development. Indeed, it is the first to show that language does not only impact face recognition/memorization but more generally, the mechanisms of memorization of everything in the infant’s environment (faces, objects, and so on).

It also paves the way to a completely novel thread of research studying the impact of ambient sound environment on infant learning and memory. Indeed, little is known about the temporality of the effects of language during memorization processes: Does it occur during the memorization (familiarization) or the memory retrieval (test)? What would happen if the language during the test was different from the language during the familiarization during a VPC task?