Music intervals in speech: Psychological disposition modulates ratio precision among interlocutors’ nonlocal f0 production in real-time dyadic conversation

Music in speech

Dating back to Darwin (1871), important connections have been observed at a scientific level between human vocal communication and musical phenomena, suggesting that musical components of speech have influenced the evolution of human communication (Brown, 2000; Cross & Woodruff, 2009). At an ontogenetic level, researchers have investigated musical features in infant–caregiver interactions (Trainor, Austin, & Desjardins, 2000; Trainor & Desjardins, 2002; Trehub, 2003). In this context, infant-directed speech (IDS) has been described as a communicative device with specific acoustic features (exaggerated pitch contours, larger pitch range, and slower tempo) that resembles music rather than language (Fernald, 1992). Trainor et al. (2000) have argued that the same features can be found in adult-directed speech (ADS) whenever people express emotion, being otherwise not noticeable due to an inhibitory process that is typically present in adult–adult interactions. Malloch and Trevarthen (2009) have also emphasised that IDS’s musical features are present in interactions among adults in the form of discrete vocal events (Malloch & Trevarthen, 2009). Many dimensions of meaning can not only be situated but also conveyed throughout phonetic means (Cruttenden, 1997). Among these dimensions, the “attitudinal” or “affective stance” (Ochs, 1996) consists of how speakers convey their attitudes toward what is being said or the person to whom it is said. Trainor et al. (2000) have proposed that the introduction of language in infants corresponds to the end of prosody as the exclusive vocal vehicle for conveying meaning. This transition would mark the beginning of a progressive social restraint of the prosodic expression of emotion. Such restraint, they hypothesised, would allow more cognitive and reflective reactions to prevail over immediate emotional ones.

Pitch and musical intervals

Most of the work on musical aspects of vocal pitch describes it in terms of melodic contour (Fernald, 1989, 1991; Malloch & Trevarthen, 2009; Stern, Spieker, & MacKain, 1982; Trehub, Trainor, & Unyk, 1993). A limitation of this type of analysis, however, is that it describes pitch behaviour in a rather global way – in terms of, for example, “rising”, “lower” contours – overlooking more precise relations between frequencies. To assess pitch and pitch relations in vocal research, the notion of musical intervals (see Table 1) seems to provide an advantage, since intervals have been the main means for accurately addressing pitch distances (Barker, 1984). In fact, there is evidence that musical intervals have an impact at a psychological level from an early age and throughout life (Masataka, 2006; Oelmann & Laeng, 2008; Schellenberg & Trehub, 1999; Smith & Williams, 1999; Trainor, Tsang, & Cheung, 2002). Based on this evidence, recent studies have tested the role of musical intervals in speech.

OPEN IN VIEWER

Table 1.

Musical intervals, expressed through their distance in semitones (s/t) from a given tone, their sizes in cents for 12-tone equal temperament (12-TET), and the corresponding mathematical ratios between two frequencies.

Interval name	Semitone distance	Cents (12-TET)	Frequency ratio
Unison (Un)	0	0	1:1
Minor 2nd (m2)	1	100	16:15
Major 2nd (M2)	2	200	9:8
Minor 3rd (m3)	3	300	6:5
Major 3rd (M3)	4	400	5:4
Perfect 4th (P4)	5	500	4:3
Tritone (TT)	6	600	7:5
Perfect 5th (P5)	7	700	3:2
Minor 6th (m6)	8	800	8:5
Major 6th (M6)	9	900	5:3
Minor 7th (m7)	10	1000	9:5
Major 7th (M7)	11	1100	15:8
Octave (Oct)	12	1200	2:1

Although of course not perceptually noticeable in speech, musical intervals and the tonal hierarchy can be found not exclusively in the domain of music, but are also subtly present in human daily vocalisation. By analysing the distribution of amplitude-frequency combinations in a large human speech corpus, Schwartz et al. (2003) were able to predict both the structure of the chromatic scale and consonance/dissonance ordering – that is, they predicted the way people across several populations tend to provide similar judgement of a given interval in terms of consonance and dissonance. Schwartz et al. argue that this subconscious capacity would allow listeners to respond appropriately to significant sources of information embedded in human vocalisation. Such information would concern not only language itself but also cues such as the sex, age, and emotional state of the speaker.

Focusing on fundamental frequency and formant relationships, Bowling et al. (2010) found that spectral analysis parallels the distinction between major and minor music tonal features. Because the physiological differences between excited (i.e., happy, bright) and subdued (i.e., sad, dark) affective states alter the spectral content of voiced speech (Scherer, 2003), the spectra of major intervals are more similar to the spectra found in excited speech, while the spectra of particular minor intervals resemble the spectra of the subdued speech counterpart. In a second study concerning musical intervals in different languages and cultures, Bowling et al. (2012) found stronger similarities to music intervals in f0 than in spectral ratio analyses. It is worth mentioning that the studies so far presented in this section share the use of “musical intervals” as a categorisation matrix that allows any possible given ratio to be assigned to an interval.

A different case is that of Curtis and Barucha (2010), who found strong relationships between affective states and f0 ratios contained in speech contour: vocally generated intervals seemed to encode emotion, most noticeably in the case of sadness and the minor third. Here, “music intervals” exclusively refer to ratios precise enough as to be considered what is traditionally conveyed by the term in the musical domain. Curtis and Barucha’s (2010) interpretation of these findings implicates causality: the arrangement of frequencies in music would, to some extent, mirror natural human vocalisation, placing language as a mapping source for the use of such parameters in music and hence underlying its affective impact. Along similar lines, Bowling et al. (2010) have suggested that the characteristic sentimental impressions of major and minor intervals would arise from routine associations between specific musical intervals and voiced speech. What all of the previous studies have in common is the construction of ratios out of the pitches of adjacent syllables. However, intervals in tonal music not only entail representations of single events and local relationships on short time scales, but also nested hierarchical structures spanning longer time scales up to the entire length of a piece (Salzer, 1962; Schenker, 1956). Such nonlocal dependencies require working memory, do not require explicit structural knowledge, cannot be explained by Markov models (Koelsch, Rohrmeier, Torrecuso, & Jentschke, 2013) and remain to be explored in the context of speech.

Approaching interpersonality

All of these studies on musical intervals in speech address the inter-domain presence of musical intervals, but also share the methodological feature of approaching the phenomenon at a single person level of analysis, revealing a transmissional paradigm of communication (Kashkin, 2012), and ignoring the notions of feedback (Wiener, 1961) or dialogicity (Bakhtin, 1981) that the interactional paradigm implies (Kashkin, 2012). At the same time, musical intervals are not only generated within an instrument but also among instruments (allowing duet, ensemble, or choir performances). Considering these facts, a methodological focus on a single person may be a pitfall when studying musical intervals in human speech. In a conversational context, it is indeed reasonable to expect that pitch relations will be found not only within a single speaker’s f0 but also when juxtaposed to the f0 produced by the speaker’s interlocutor.

Following this line of reasoning, diverse prosodic phenomena have been reported in real dyadic interactions, particularly some that are focused on either fundamental frequency or f0 (Johnson, 2003; Kramer, 1964). By performing conversational and phonetic analysis on whole sequences of two-party conversations, Ogden (2006) qualitatively describes the way “speakers match their own tone production to that of another speaker and manipulate the relation between their co-participant’s production and their own in ways that have implications for meaning” (p. 1773). The phenomenon known as phonetic convergence (Gregory, Dagan, & Webster, 1997; Kim & Horton, 2011) is also of special interest. It is usually defined as the progressive accommodation of the distance (in Hz) between a speaker’s and his interlocutor’s f0s. Recently, a close relationship between the level of empathy shared by two interacting parties and the corresponding level of phonetic convergence unfolding during interaction has been reported (Quezada, Robledo, Román, & Cornejo, 2012). In other words, a psychological interactional disposition such as empathy influences the ratios between frequencies in speech prosody.

However, given that these studies do not accurately describe such ratios, it remains unknown whether musical intervals play a role in interactional situations. Hence, considering that the distance between two f0s can be modelled through mathematical ratios in the same form as the relation between two tones of a musical interval, given the evidence of musical intervals within a single person’s speech, and the fact that nonlocal dependencies are a key component both in language – at a syntactic level – (Nevins, Pesetsky, & Rodrigues, 2009) and music (Koelsch et al., 2013), we hypothesised reciprocal nonlocal f0 accommodations from a musical perspective between persons during a conversational interaction. By manipulating psychological disposition, we expected to see differences among conditions in terms of interpersonally generated musical interval distribution and/or precision.

Methods

Acoustic analysis

For investigation within a single person, there are several possible ways of contrasting discrete pitch measures of a melodic contour (Bowling et al., 2012; Curtis & Barucha, 2010). When approaching vocalisations between two persons, however, logical possibilities multiply, and theoretical ones are lacking. Although previous authors may have managed to construct criteria for pairing frequencies (and thus generating intervals) within a single person’s speech, it is less evident how one should pair frequencies between two interacting speakers (see Figure 1).

OPEN IN VIEWER

Figure 1.

Mode pairing and assimilation to musical interval categorisation matrix. Grouped by colour (different shades of grey), pitch modes corresponding to a particular dyad’s two participants (e.g., Control Condition dyad number 1, participants A and B) were paired in each question (q1, q2, and q9). Each pairing generated a mathematical ratio assimilated to a particular 12-TET interval. (e.g., Aq1 and Bq1’s ratio is 1.506, corresponding to the interval of a Perfect Fifth or P5).

Using the software Praat (Boersma, 2001), each section’s f0 mode was calculated in Hertz using a custom-made script. The analysis parameters were based on the f0 range of adult speech, corresponding to an f0 calculation range of 90 to 800 Hz in the case of women and 65 to 600 for men. The f0 contour of each speech sample was calculated using the Praat autocorrelation method, with a voicing threshold of 0.45 and a frame period of 0.005 seconds. Mode was selected as an important and representative parameter because it consists of the most repeated and stable frequency and is stronger when considering vast amounts of values, as is the case for a whole section of utterances. Mode was also selected because, unlike the mean, it displays a real f0 frequency value that participants actually produced.

Because the f0 mode corresponded to each dyad participant’s series of utterances during a particular question, it was possible to pair them (e.g., Participant A’s f0 mode corresponding to question 9 utterances paired with Participant B’s f0 mode corresponding to question 9 utterances). Out of each f0 mode pair, a mathematical ratio was calculated in cents following the formula Cents = 1,200 x (log[f1/f2]/log2). Because musical intervals are categorized by number of semitones in Western cultures, with each semitone comprising 100 cents, such mathematical ratios were then categorised into 100-cent bins. Intervals larger than an octave (1 in the CC, 5 in the LTC) were normalised to within octave range for all categorical analyses, as is customary in the conceptualisation of pitch class hierarchies in tonal music. The literature identifies perceptual discrimination boundaries at 50 cents between musical interval category prototypes (Burns & Campbell, 1994; Burns & Ward, 1978; Siegel & Siegel, 1977). Because such boundaries were recently corroborated in the specific field of vocal musical intervals (Curtis & Barucha, 2010), 50-cent bin categorisation was also performed. Therefore, the main pitch distance between interacting partners’ vocalisations could be analysed through musical logic.

Results

By using 100-cent bins as a categorisation matrix, differences were found among conditions. As shown in Figure 2, dyads generated different frequency ratios. In the case of the LTC a large percentage of pitch mode ratios corresponded to small intervals, in particular, minor and major seconds. Few larger intervals were found in this group. In the CC, small intervals were also present, but these appeared along with larger intervals, such as sixths and the minor seventh. Therefore, a larger and more even distribution was observed in the CC. When clustered into two groups, namely, smaller than a tritone and equal or greater than a tritone, a chi-square analysis confirmed such distribution differences, χ2 (1) = 3.887, p = .049.

OPEN IN VIEWER

Figure 2.

Differences among conditions on interval distributions using a 100-cent binning (smaller or equal to/larger than a tritone) were statistically significant, χ2(1) = 3.887, p = .049. LTC: Low Trust Condition; CC: Control Condition.

Given such differences, it seemed reasonable to ask whether they may have arisen from participants’ individual vocal characteristics. CC dyads may have comprised deep-voiced men talking to shrill-voiced women, thus generating larger intervals, or LTC dyads comprising shrill-voiced men talking to deep-voiced women may have generated smaller intervals. Consequently, the pitch range was calculated for participants in both conditions. These values were compared using Student’s t-tests for independent samples, assuming equal variance. There was no evidence of a significant between-group difference in tonal range, t(22) = 0.558, p = .582. This result indicated that individuals’ physiological/vocal characteristics did not explain these findings.

By utilising stricter categorisation for mode ratios such as 50-cent binning, the results can be better understood. As presented in Figure 3 (a and b sections), for mode ratios that did not satisfy such binning and have thus qualified as perceptually ambiguous in the literature, very slight differences among conditions can be observed (Figure 3b). Therefore, these ratios are henceforth tentatively called “un-tuned”. Conversely, when considering only those ratios that can actually be labelled as “musical intervals”, or “tuned” (Figure 3a), the condition dissimilarities are enhanced. The distribution of “tuned” and “un-tuned” evidently showed no significant difference among conditions (Figure 4).

OPEN IN VIEWER

Figure 3.

Differences among conditions on interval distributions using a 50-cent binning (smaller or equal to/larger than a tritone) were statistically significant. (a) comprises ratios within the 50-cent binning, henceforth labelled as “tuned”, χ2(1) = 26.337, p < .001. (b) comprises ratios outside the 50-cent binning, labelled as “un-tuned”, χ2(1) = 3.689, p = .055. LTC: Low Trust Condition; CC: Control Condition.

OPEN IN VIEWER

Figure 4.

Percentage of intervals that satisfied (“tuned”) and did not satisfy (“un-tuned”) the perceptual discrimination boundary at 50 cents for the musical interval category. No significant differences between conditions were found. LTC: Low Trust Condition; CC: Control Condition.

Pearson’s chi-square tests with Yates’ continuity correction were performed separately for tuned and un-tuned intervals to establish whether the proportion of smaller vs. equal or greater than tritone intervals was different for the two experimental conditions (CC and LTC). In fact, a very strong between-condition difference was found in tuned interval data, χ2(1) = 26.337, p < .001, whereas a less strong difference was found in the un-tuned interval data, χ2(1) = 3.689, p = .055. To express meaningful effect sizes, odds for the larger than tritone intervals were computed. Doing so yielded an odds ratio of 5.14 for tuned intervals and 1.85 for un-tuned. In other words, if we look only at tuned intervals, the odds of larger than tritone intervals are more than five times higher in CC than in LTC. When looking at tuned data, the odds are still higher for CC but are less than twice of those for LTC. As a result, the pattern of more frequent large intervals in CC than in LTC was statistically significant and very clear within tuned intervals data, but not to the same extent within “un-tuned” intervals.

By restricting generated musical intervals exclusively to those within a 50-cent binning, thus considering ratios susceptible of being perceptually discriminated as recognisable or tuned, some similarities were found when comparing the results to those of Figure 2. Nevertheless, there were also interesting differences. First, among the CC’s tuned intervals, none qualified as a minor or major second (see Figure 3a). This strongly contrasts with LTC’s clear preponderance of such intervals. Second, the CC’s tuned intervals showed a marked and sudden peak of minor thirds and then remained on a lower plateau until the major sixth, where it picked up again. The opposite behaviour was observed in the LTC. The average pitch distance among the interlocutors’ f0 modes, per section, showed no significant difference between LTC’s first (1.41) and last (1.44) sections. On the other hand, there was a significant reduction of the average pitch ratio in the CC (1.61 and 1.22), t(10) = 12.95, p = < .001.

Discussion

Our results demonstrate a relationship between psychological disposition and the distribution of musical intervals between persons participating in a real conversation. We found that tones corresponding to interlocutors’ f0s spontaneously generate ratios that can be categorised as musical intervals. This finding does not necessarily imply that modal frequency ratios in speech are perceived as musical intervals, but it does strongly suggest that conversational frequency patterns are related to Western music intervals in an inter-domain manner. Such information highlights novel relationships between psychological disposition and vocalisation because the interval distribution differed among conditions.

The main finding of this work concerns categorical precision or “tuning”, which stands as a key element of the differences found between conditions. The interval distributions of the LTC displayed in Figures 2 and 3 reveal a similar pattern: a greater presence of smaller intervals. This predominance has also been observed by Bowling et al. (2012), who compared intervals – regardless of their ratio precision – smaller and larger than a major second, which was chosen as a cut-off because it maximised the differences between positive/excited and negative/subdued vocalisations. The present data suggest that, in the case of interpersonally-generated ratios and their relationship to trust, the tritone stands as a suitable cut-off. It can therefore be hypothesised that smaller intervals are a predominant feature of regular vocal productions, both in melodic contour and interpersonally juxtaposed pitches.

However, the same cannot be said for the CC. When contrasting the CC’s 50-cent binning (tuned) interval distribution (Figure 3a) with the 100-cent and 50-cent “un-tuned” distributions (Figures 2 and 3b, respectively), the patterns differ importantly. Whereas major and minor seconds were present in the 100-cent binning, as were intervals larger than a tritone (more similar to the LTC in all of its versions), there was a remarkable absence of seconds in the 50-cent binning. Instead, a preponderance of larger intervals was not only maintained but also enhanced. Furthermore, the un-tuned version of the 50-cent distribution shows a regularity that diverges very little from all the binning versions of the LTC. It is worth mentioning that without the octave normalisation, the raw size of the intervals in the CC shows smaller intervals in general when compared to the LTC. Also, the significant pitch ratio reduction along CC conversations implies phonetic convergence, as would have been expected.

These findings demonstrate that psychological dispositions modulate musical intervals between conversational partners. As the LTC’s manipulation shook interpersonal confidence, it inhibited the interactants’ expressions, inducing a reflective disposition toward the other person (Trainor et al., 2000). This would not have occurred in the CC, where less constrained prosody could arise from both speakers, thereby generating the unique 50-cent tuned distribution.

Because previous research from Bowling et al. (2012) and Curtis and Barucha (2010) had an exclusively single-person scope, it is rather improbable that these results can be explained by single-person patterns that are reflected merely by chance at the interpersonal level. Thus, though the intervals studied here were generated by comparing two different persons’ pitches, the possibility that the predominance of small intervals (primarily seconds) could still be explained as a consequence of physiological phenomena is improbable, although this possibility cannot be discarded and should therefore be further explored. However, our findings instead suggest a cognitive connection between the individual physiological characteristics of human sounds and the interpersonal psychophysiological level. In particular, our results stand as evidence of how the motivational-structural dimension of sound might afford its socio-intentional role (Cross & Woodruff, 2009). According to Cross & Woodruff, music’s socio-intentional dimension is oriented towards the interpretation of human agency and intentionality. It arises from sound structures that could be interpreted as affording cues about shared intentionality that direct attention in interaction. Such sound structures may be construed as disclosural or dissimulative, denoting distinct and different communicative intentions and hence establishing the pragmatic contexts of utterances. Although originally conceived in engagement with music, a socio-intentional dimension can also be considered in vocal pitch interplay. In this particular case, the same sound structures (pitch ratio patterns) contained in melodic contour as an involuntary physiological “honest signal” can be found within two persons’ vocalisations. However, at an interactional level, they are no longer exclusively physiological but are socially constructed structures. Only mutual vocal accommodations that require complex online cognitive processing allow interpersonally generated musical intervals to happen. The fact that the socio-intentional dimension of vocal sound conveys a disclosural or dissimulative character is also important. Because the conditions differed through the introduction of lies, it is sensible to hypothesise that interpersonal pitch distances played a role in that matter and thus also differed among conditions.

When considering interpersonal pitch phenomena, another topic of discussion is how it is possible to coordinate such relations. Among Hockett’s (1960) design features of animal communication, two features are particularly interesting in this context. The first is Interchangeability, by which a speaker can both transmit and receive at the same time. The second is Total Feedback, which means that the speaker hears what he himself says. Hockett highlights the importance of the latter because it allows for the internalisation of communicative behaviour, which constitutes a major portion of what we call “thinking”. Therefore, it is possible that by obtaining a certain notion of the position of one’s own sounds within the larger sound plot of an interaction, it would be possible to insert one’s own vocal/oral/communicative activity within interpersonal social dynamics. Also highlighting our capacity to vocalise within acoustic contexts, Bannan (2008) claims that:

Harnessed to a “theory of mind”, a theory of harmony acts as a bridge between musical vocalisation and the properties of categorical discrimination that are demanded by unambiguous language … The theory of mind that permits unison- and harmony-singing comprises a unique cognitive adaptation that could be seen as the candidate human equivalent to the capacity of birds instinctively to fly in flocks and fish to shoal for their collective protection. (Bannan, 2008, p. 285)

Although thinking of choirs rather than interpersonally generated intervals, Bannan stresses the psychological and social implications of the human ability for perceiving not only the tones we emit but also their relation with other people’s tones – namely, musical intervals in the manner of a dynamical system (Kelso, 1995). Bannan’s notion of complementary pitches becomes crucial because it implicates that both interactants’ own tonal behaviour as “speakers” is not lost, but rather registered along with the one they listen to as “hearers”. This idea is congruent with Ogden’s (2006) statements. After his sequential analysis of agreement and disagreement conversations, he concludes that meaning conveyed through phonetic features is a local and collaborative achievement. The implication of this claim is that some linguistic features should be considered in their context – in his and in our case, a dialogical one. This type of “situated” (Malloch & Trevarthen, 2009) or “attitudinal” (Cruttenden, 1997) meaning would then not reside solely in one of the interactants’ minds, as “trust” and “mistrust” only have meaning in relation to at least two persons.

One additional consideration with regard to the present findings is pitch discrimination precision. As stated above, our results neither imply nor deny that mode frequency ratios in speech are consciously perceived as musical intervals. In fact, it seems difficult to imagine that interactants would be able to consciously relate each other’s vocal sounds to whole supra-segmental sections, as in this case. Still, the differences among conditions are consistent and confirm the idea that nested nonlocal dependencies is a multi-domain capacity of human cognition that relies on working memory (Koelsch et al., 2013). As it has already been demonstrated to be present both in tonality and speech syntax, it is not surprising to also find it in prosody (f0). Zatorre & Baum (2012) affirmed that despite their shared cognitive processes, the way in which pitch information is handled differs between speech and music. They propose two mechanisms: one that is “coarse-grained”, focused on contour, and another that is “fine-grained”, involving more precise pitch relations such as musical intervals. The first one might overlap across domains; the second is hypothesised to be specifically music-directed. The present findings provide relevant contributions to examine such propositions. Differences among the LTC and CC were only conclusive when considering 50-cent “fine-grained” pitch ratios. The presence of this mechanism – thought to be exclusively musical – becomes a necessary element for the unfolding of the adjacent pitch interplay reported in this study. Because Zatorre and Baum’s evidence considers only melody and sentence-level intonation, their differing hypothesis might be accurate at such a local level, but the present evidence demonstrates otherwise at a nonlocal one. This finding highlights the importance of paying attention not only to rule-based, grammatical sub-segmental aspects of language but also to its holistic aspects (Wray & Grace, 2007). In particular, it is important to consider that pitch interplay (both on a single or dyadic approach) may concern not only what can be noticed through melodic contour or adjacent pairs but also that which is beyond the structure that grammatical rules suggest. It is worth mentioning that all the potential local intervallic information contained within a single sentence, although beyond the scope of this study, is far from unimportant. Future research should scrutinise both nonlocal and local levels, and assess any possible patterned relationships between them. Finally, as the socio-intentional dimension of music arises not only from sound but from several multi-modal sources, it would also be interesting to investigate what other pragmatic aspects of language (gesture, body movement) may interrelate to the vocal phenomena presented in this work.