The effect of the typicality of song lyrics on song popularity: A natural language processing analysis of the British top singles chart

Typicality, lyrics, and song popularity

To understand the concept of typicality, one must first understand the concept of canon. In the field of arts, and more specifically in musicology, a canon is a

standard repertoire, [. . .] a loosely codified organism, broadly accepted, with some degree of flexibility on small exchanges or new members [that does however] exhibit recalcitrant behavior on wholesale changes, for these reflect major shifts in the aesthetic viewpoint that tend to evolve over a period of time (Citron, 1990, p. 102).

Canons can be shaped by organizations or artists who are sufficiently influential within an artistic movement or field to impose the domination of certain aesthetic preferences (Farnsworth, 1969; Kopiez et al., 2009; Mueller, 1951), but also by the fact that the more a piece is played, the greater the likelihood that it will be played again (Gilmore, 1993). Thus, canons are constantly torn between tradition and novelty. On the one hand, certain factors can contribute to their homogenization over time. For example, the classical music canon tended to homogenize as royal patronage for artists declined (Weber, 1984). Conversely, it has tended to become more heterogeneous as new university music programmes have emerged and as the performance capabilities of orchestras have improved (Dowd et al., 2002). Research in historiometry applied to empirical aesthetics has shown that there is a curvilinear relationship between the originality of a melody with regard to the canon and its popularity in both classical and popular music (Hass, 2016; Simonton, 1980, 1994, 1998). This relationship was confirmed by experimental studies, which showed that extremely typical and atypical melodies were preferred to moderately typical ones (Martindale & Moore, 1989; North et al., 2017, 2019).

However, the relative importance of lyrics in the formation of musical preferences has largely been ignored, let alone their typicality. Although composers are known to contribute more to the success of an opera than their librettists, even though they both share the credit (Simonton, 2000), it still seems crucial to take lyrics into consideration to predict the popularity of a piece of music and especially that of songs, since they are an assemblage of words and music (Encyclopædia Britannica, n.d.) and are by far the most popular musical style today (Billboard, 2022; Official Charts Company, 2022). Furthermore, given that song lyrics are vectors of nostalgia, cultural, ethnic and self-identity construction, and even feelings of connectedness (Batcho et al., 2008; Kennedy & Gadpaille, 2017), their role in the formation of musical preferences can hardly be ignored. Besides, understanding the effects of lyrics’ lexical typicality on song popularity could inform recommendation algorithms and marketing strategies, ultimately helping industry stakeholders anticipate musical trends.

Most research on lyrics-related determinants of song popularity has focused on how listener characteristics shape lyrical preferences, showing that personality traits can predict preferences for certain lyrical cues or features (Qiu et al., 2019; Rentfrow & Gosling, 2003). By contrast, studies examining how the intrinsic characteristics of lyrics themselves affect song popularity remain scant. A representative example of such research is the work of Nunes et al. (2014), which showed that songs with lyrics containing repetitions were more likely to reach the top of the charts. The typicality of lyrics is another one of those determinants. However, to the best of our knowledge, only two studies have looked into its impact on song popularity and were conducted by North et al. (2020) and Berger and Packard (2018). The latter have shown that the more the themes explored by the lyrics of a song deviate from those typically addressed by songs of the same musical genre, the more popular that song will tend to become. In addition, this effect is moderated by the relative importance that song lyrics may have within different genres. North et al. (2020) measured the typicality of the lyrics of songs within – and in comparison to – a corpus of the 1,414 songs that reached the top five spots of British charts between 1999 and 2013, which constitutes a canon of popular music during that specific period. They showed that the less typical the lyrics of a song are within it, the higher it reaches in the rankings, while the more typical the lyrics are, the longer the song stays in the top five spots of the charts. These two articles show that typicality can be analysed in relation to a canon formed within a given musical genre, or to a canon defined by an objective criterion, such as record sales. They also show that typicality can be analysed at different lexical scales, by choosing the word as a unit or the theme to which these words refer.

However, while these articles make important contributions to the field, both have methodological limitations. Indeed, Berger and Packard (2018) have reduced the lexical richness of the lyrics to the themes they explore and thus do not actually look into lyrics’ typicality as a whole, but only into one of its sub-dimensions, namely thematic typicality. As for the work of North et al. (2020), although it proposes a word-level measure of the typicality of song lyrics and presents it as being correlated with different measures of popularity, it stems from a larger model that makes it dependent on a measure of music typicality, which renders its statistical interpretation problematic. Furthermore, this research relies mainly on the use of text analysis software based on a 10,000-word dictionary. These word dictionaries are limitative because they cannot account for the use of certain specific words, which might be typical of a given musical genre, or of a social group, time period or geographical area, and that may be relevant to consider to better understand the popularity of a given song. This could be the case of certain slang words (e.g., homies) or regional variants of a given word (e.g., y’all), but also of brand names, not to mention words invented by musical artists, such as ‘mullet’, coined by the Beastie Boys, or ‘yolo’, created by Drake A. Side (2019). Methods stemming from NLP might, in this case, be more suited since they allow the analysis of the typicality of lyrics in relation to a given canon at the word level, without being limited by the content of dictionaries.

NLP and lyrics’ typicality

The notion of typicality was first introduced in psychological and linguistic research to account for the internal structure of perceptual and semantic categories. Rosch (1973) moved away from previous studies that treated categories as composed of undifferentiated equivalent instances and demonstrated that some category members were better examples than others. For example, ‘carrot’ was deemed more typical of the category ‘vegetable’ than ‘pickle’. She defined the internal structure of categories as a ‘core meaning’ constituted of the most typical category examples, surrounded by the less typical members.

This graded structure led to a prototype view – also known as the family resemblance mode – first introduced by Wittgenstein (1953) before being refined by Rosch’s (1973, 1975) studies. In this view, a category can be described as a set of features or properties. Some properties are privileged properties, in the sense that they are necessary to grant membership. But they are not enough to define the boundaries of a category: a family description is therefore required. The family resemblance of a category member is calculated as the weighted sum of the properties it possesses, the weight of each property being determined by the number of category members that share it. In this model, prototypical members have all or most privileged properties, whereas atypical members of a category only share a few. In other words, classification is based on similarity to the central tendency; the prototypical member of the category is the one that best reflects the structure of the category as a whole (Rosch et al., 1976).

However, the prototype theory has been shown to have a few shortcomings; for example, it is easier to find the prototype features of a category than to identify sufficient ones, and such theory complicates the lexical and compositional meaning of concepts (Armstrong et al., 1983). Furthermore, whereas the family resemblance model treats properties as additive and independent features, studies have demonstrated that such properties tend to occur together in clusters (Malt & Smith, 1984; Rosch et al., 1976).

A contrasting view on modelling typicality is the exemplar view. In the exemplar theory, typicality is based on the summed similarity of an item to all exemplars of the target category. Indeed, the theory originates from the hypothesis that people categorize items by comparing them to examples stored in memory (Nosofsky et al., 2011). This way of defining typicality has been shown as less ‘efficient’ than the prototype view but is more flexible since it can better account for atypical examples (Nosofsky et al., 2011; Richard-Bollans et al., 2020). The adoption of both theories has been suggested as an alternative: multiple prototypes may be used to represent a category, and multiple examples may be averaged to form a prototype (Nosofsky et al., 2011). Furthermore, depending on the task at hand, one theory may be more appropriate than another. For example, Nosofsky et al. (2011) showed that whereas classification was linked to the prototypical view, recognition depended on the exemplar view.

But whereas both approaches to modelling typicality rely on computing a semantic distance to either a central prototype or multiple exemplars, few works discuss how to put such theories into practice. Connell and Ramscar (2001), inspired by the prototype theory, successfully simulated typicality effects with a co-occurrence model of language: latent semantic analysis (LSA). Without explicitly coding categories or semantic features, they were able to compute the typicality score for each item in their dataset with respect to the superordinate canonical category. The success of this measure in predicting typicality shows that typicality cannot be considered without accounting for the linguistic environment surrounding such concepts. More recently, in an attempt to develop a model for detecting typical comparable entity pairs from two sets of entities, Jain and Pantel (2009) leveraged the exemplar view to propose an integer linear programming formulation to detect typical entities. Finally, Richard-Bollans et al. (2020) introduced various models to automatically measure typicality and generate prototypes in the context of spatial language by using both similarity-based methods. We build upon previous work and define typicality as the distance to a prototypical instance. To generate such a prototype, we leverage NLP techniques to extract multidimensional features from the lyrics. For each of them, we define the corpus average as the ‘most typical’ member of the category.

Sample

As North et al.’s (2020) article did not disclose the exhaustive list of songs within their corpus, we recreated it by selecting all the songs that fit the criteria they mentioned, namely, all those that reached one of the top five spots in the UK Official Singles Chart Top 100 between January 1999 and December 2013. We used the Official Charts Company’s website (www.officialcharts.com) to find the songs matching these criteria, as the organization is in charge of maintaining the official records of the British charts. The lyrics of those songs were collected using the PyLyrics-Extractor package, then processed by expanding contractions and deleting interjections and special characters. Songs with unavailable lyrics were excluded from the corpus. Our final corpus includes 1,457 songs, while that of North et al. (2020) consists of 1,414 songs. It shall be noted that although North et al. (2020) describe their sample as being constituted of pop music, the word ‘pop’ should not be understood here as referring to the ‘pop genre’ but to music that is, since it appears in the top British charts, inherently ‘popular’. These charts thus include songs that may belong to the pop genre (e.g., Domino by Jessie J, Stronger by Kelly Clarkson, etc.), but also songs that belong to other music genre, such as Elbow’s One Day Like This (indie rock), Calvin Harris’s We’ll Be Coming Back (EDM), or Public Enemy’s Harder Than You Think (hip-hop).

Popularity

Of the four popularity measures used by North et al. (2020), two were not replicable because they included data from a set of national and regional charts, of which the exact composition was not disclosed. However, we were able to collect data for their other two popularity measures, namely the maximum rank (max rank) reached by a song in the chart (from 1 to 5) and the number of weeks spent by a song in one of the top five positions of the chart (weeks in top 5). These measures have also been used in other articles studying music trends (e.g., Askin & Mauskapf, 2014, 2017; Interiano et al., 2018). We also added two measures of popularity related to the trajectory of songs in the rankings, namely skewness and kurtosis of the ranking scores (Lee & Lee, 2015, 2018). In addition to making an additional theoretical contribution to the typicality literature, these two measures are useful from a managerial perspective as they allow us to describe the speed at which a song gains or loses popularity. For example, a song with a high skewness score achieved rapid popularity and declined slowly, and a low kurtosis score signifies slow growth in popularity.

Variables

Once the songs and lyrics were collected, we proceeded to create popularity metrics and extract relevant lexical features. From the literature on popularity and typicality, we created features related to four dimensions to assess the typicality of our corpus: (1) lexical repetition, (2) complexity, (3) topics, and (4) emotions.

Topics

Berger and Packard (2018) showed that there was a relationship between a song’s lyrical topics’ typicality in regard to its own genre and its popularity. We focus on extracting the different topics covered in a song. We use latent Dirichlet allocation (LDA), a generative probabilistic model that allows a set of documents to be explained through thematic structures to extract the main themes of our corpus. More specifically, each document is seen as a mix of different topics, and each topic is a mix of words that frequently appear together. As such, each word has a probability of belonging to every topic; in that case, it is not surprising to see the most occurring words as probable in multiple topics occurring several times. Therefore, to understand the topics in an LDA approach, looking at all the words and their co-occurrences is essential. To provide a concrete example, the model looks at all the words across all documents and, based on their co-occurrences, tries to group them into topics. In our case, topic 1 is constituted of words such as ‘night’, ‘tonight’, ‘feel’, ‘girl’, and ‘let’, which means that these words tend to appear frequently together, while topic 3 is composed of ‘love’, ‘heart’, ‘never’, ‘feel’, and ‘let’. In that case, ‘let’ can be seen as a word occurring so many times that it is probable that it co-occurs with the other words twice. The interpretation of the meaning of each topic based on these co-occurrences remains different and subjective afterwards. To establish the best number of topics, we compare the perplexity score (Wallach et al., 2009) of different models with varying numbers of topics. The perplexity score is used to evaluate how well a model describes a set of documents by measuring its success in predicting new data. We also implement Green et al. (2014) stability analysis to confirm our chosen number. Green et al. (2014) proposed a term-centric stability analysis strategy to determine the appropriate number of topics given a corpus. They based their technique on the idea that a model with an appropriate number of topics is more robust to changes in the data. With these two approaches, we chose four as the number of topics. Table 1 shows the most important words associated with each topic.

OPEN IN VIEWER

Table 1.

Description of the Topics Identified Using LDA.

Topics	Words
1	night, tonight, feel, girl, go, good, let, get, ready, fire
2	get, let, put, come, hand, make, dance, turn, body, move
3	love, heart, never, feel, let, life, find, day, could, would
4	know, go, baby, get, say, want, make, think, see, tell

We then proceed to create two typicality metrics: the Kullback-Leibler divergence (KL divergence) – a statistical distance that measures how a probability distribution P is different from a second – and the distance between each song and the most recurrent topic in the whole corpus.

KL divergence: We first compute the topics’ distribution for each song. We then define the topic distribution for the whole corpus as the average of the topics for all songs. For each song, we are then able to compute the KL divergence between its own topics’ distribution and the corpus’ average. With this measure, we look for the extent to which the distribution of topics addressed in a song differs from the distribution of topics addressed on average in the corpus.

Distance to the main topic: With the LDA model, each document is assigned a probability of belonging to a certain topic. For each song, we identify the main topic with the highest probability of appearing in that given song. Therefore, we can define the main topic of the whole corpus of songs as the topic that is prominent in most songs. This topic represents the typical theme addressed. We can then define a distance for each song to this typical theme to measure the extent to which the song evolves around the most typical theme in the corpus. Since each topic is represented as a vector of the probability of words belonging to that topic, we can define the distance to the main topic of the corpus as the sum of the cosine similarity between each topic of a song and the main topic multiplied by the weight of each topic in the selected song.

Typicality score

Using the methodology introduced above, we created eight variables representing different potential aspects of a song’s typicality. However, since we needed to obtain a total typicality score encompassing all these aspects (as analysed in North et al., 2020), we created the variable ‘Typicality score’ defined as the sum of all the typicality variables extracted from the lyrics according to the formula:

A lower typicality score indicates that a song is more typical of its canon, whereas a higher score suggests that the song has more unique lyrics (North et al., 2020).

Limitations and future research

One of the main limitations of this study is its limited potential for generalization to other song canons. We solely examined a corpus of songs that appeared in the top five spots in the British charts between 1999 and 2013. This study thus investigates what has constituted a canon of popular music for the British public during this given period. In addition, this chart compiles songs from various musical genres, and Berger and Packard (2018) demonstrated that the impact of lyrics’ typicality varies from one music genre to another. According to their findings, we could hypothesize that within a rock music canon, songs that score higher in lyrics’ typicality would be less popular. Unfortunately, unlike Berger and Packard (2018), we did not have access to control variables such as radio airplay, artist popularity, and song release date, and it might impact the generalization of our results.

Furthermore, the canon we studied only spans from 1999 to 2013. Although streaming platforms existed in 2013, it is worth questioning the extent to which their recommendation agents are able to influence consumers’ preferences for typical lyrics, and how far they expose consumers to songs that are not typical of those they listen to the most. If these agents consider the similarity of lyrics to those of their users’ preferred songs when creating recommendations, it could reinforce their dislike of atypical songs.

Moreover, by choosing to focus on lexical lyrics’ typicality, albeit in an integrative manner, we have not included features of lyrics’ typicality that might be structural in nature, such as narrative arcs. We therefore invite researchers to develop natural language processing metrics that could detect the presence and articulation of storytelling components (e.g., conflict, plot, etc.) in order to investigate the effect of the typicality of the narrative structure of lyrics on song popularity.

Lastly, we must also raise the question of who decides what is included in a canon. Our study focused on a canon based on song popularity; however, our literature review revealed that this is not always the case. The creation of canons and their exposure often rely on the judgements of critics, music writers, record companies, and independent labels on what holds value, as well as the preferences of a dominant class with the resources to establish these canons as having greater cultural significance (Bourdieu, 1979; Farnsworth, 1969; Kopiez et al., 2009; Mueller, 1951). Further research could thus be conducted to analyse how lyrics’ typicality influences the popularity of songs that belong to canons that were not created solely on the basis of popularity.