Musical blending and creativity: An empirical evaluation of the CHAMELEON melodic harmonisation assistant

Abstract

This article presents the CHAMELEON melodic harmonisation assistant that utilises the principles of conceptual blending theory as a means for the invention of hybrid or novel harmonic idioms and an empirical evaluation of a number of computer-generated melodic harmonisation blends. Melodies originating from various idioms were harmonised either according to the harmonic rules of the original idiom, according to the rules of a different idiom (melody–harmony blends), or by blending idioms, modes and transported versions of the same idiom (harmony–harmony blends). In two similar experimental set ups, the task of the listeners was to i) perform idiom, mode or type of chromaticism classification, ii) report their preference, and iii) rate the degree of expectancy characterising each harmonisation. The results show that harmonic blending (either melody–harmony or harmony–harmony) influences the identification of idiom, mode and type of chromaticism. This suggests that the harmonic blending system has indeed succeeded in producing perceivable blends under various conditions that were unexpected and also equally preferred compared to non-blends.

Keywords

computational creativity conceptual blending empirical evaluation harmony style classification

The work by Margaret Boden (2004) suggests that the creative processes leading to new concepts and ideas, such as a new musical work or musical style, fall into three distinct categories. According to this approach, a new concept may emerge either through the “exploration” of previously unexplored areas of an existing conceptual space (exploratory creativity), or by transforming aspects of a given conceptual space (transformational creativity), or by aligning and combining elements of different concepts into a new consistent whole (combinational creativity). Aspects of combinational creativity are described by the cognitive theory of Conceptual Blending developed by Fauconnier and Turner (2003). Conceptual Blending Theory (CBT) maintains that even most ordinary concepts in everyday thought are constructed through a subconscious combination of different elements from diverse, but structurally-related, concepts giving rise to new blended conceptual spaces. The properties of such spaces may provide a better insight into existing concepts or may even be utilised towards the emergence of novel concepts altogether.

Conceptual integration allows the creation of meaning by correlating elements and structures of diverse conceptual spaces. In music, CBT has been primarily employed in the interpretation of blends involving the integration of musical elements (rhythm, melody, harmony, texture, etc.) with extra-musical domains such as text, narration, image, or with physical, embodied, emotional and “personality-related” responses to music; blending of musical structures per se (structural blends) has received less attention (see discussion in the introduction and other articles in this issue). The work presented in this paper is based on the notion that structural blending is probably the most appropriate way to highlight the creative potential of conceptual blending in the musical domain; more specifically, it demonstrates a computational system that applies structural blending processes in the domain of musical harmony.

Creative music systems often fall into two major categories (Pearce & Wiggins, 2001): systems whose goal is to imitate a particular musical genre or the style of a certain composer, and systems that aim to generate novel musical styles; Ames (1992) refers to these two approaches as “empirical style modelling” and “active style synthesis”. Synthesis of novel musical styles is more demanding than the mere modelling of certain musical characteristics of a given style, as its major challenge is to transform or combine existing rules of music making (cf. transformational and combinational creativity) into new principles that generate new meaningful musical material departing from mere creation within a given style (exploratory creativity); according to Boden (2004) combinational creativity is the hardest to describe formally. The current study presents the CHAMELEON melodic harmonisation assistant, which on the one hand learns and reproduces harmonies in a given style, and on the other hand utilises the principles of conceptual blending theory towards active harmonic style synthesis.

Structural blending in music exists in various forms such as grafting harmonic, melodic, rhythmic or timbral elements from one musical idiom to another, or integrating such elements from at least two different idioms into novel idioms. The CHAMELEON melodic harmonisation assistant has been developed in the context of the COINVENT project framework (Schorlemmer et al., 2014) and is capable of blending different harmonic idioms (Kaliakatsos-Papakostas, Makris, Tsougras, & Cambouropoulos, 2016; Kaliakatsos-Papakostas, Queiroz, Tsougras, & Cambouropoulos, 2017). Harmonic blending, as performed by CHAMELEON, involves two different processes. The first process, computationally simpler, is melody–harmony blending whereby a melody originating from a given musical idiom (with certain implied harmonic qualities) is harmonised based on a harmonic space (chord types, chord transitions, cadences, basic voice-leading) derived via machine learning from a different harmonic idiom. The second process is harmony–harmony blending whereby the harmonic space that is used to harmonise a given melody is, itself, the product of blending between two different harmonic idioms. In CHAMELEON, the user selects two harmonic idioms from which the system learns chord types, chord transitions, cadences and basic voice-leading from two corresponding annotated harmonic corpora, and then presents an (annotated) melody to the system to be harmonised in either of the two idioms or employing harmonic blends between the two idioms; an overview of the system is given in Figure 1.

Figure 1.

Schematic diagram of the CHAMELEON system.

An important question arises: how can such a creative system be evaluated in regard to its melodic harmonisation potential and, more specifically, its creative blending capabilities? Evaluating creativity – human or computational – is a non-trivial task. A prominent debate in the field of computational creativity evaluation is whether the assessment of creative systems should be based solely on their products or also on the process through which they are generated. In a short literature overview, Jordanous (2016) presents the rationale of both approaches. While Ritchie (2007) supports that humans judge the creativity of others mainly based on what they produce, Colton (2008) argues that process may be equally important for art appreciation, giving the example of conceptual art. However, although knowledge of the context in which the work of art is placed can be proven informative regarding lower-level creative processes, contextual information is (arguably) not necessarily identical to the process per se. Additionally, in their FACE/IDEA evaluating framework, Colton, Charnley, and Pease (2011) have accompanied process assessment with audience appreciation to measure the impact of a creative act.

The process of evaluation is complicated further in the case of artistic creativity, as the assessment of aesthetic quality has to be taken into account as well (e.g., Boden, 2004; Jordanous, 2012; Wiggins, 2006). As a result, creativity is often broken down into partial constituent dimensions (e.g., novelty, value, surprise, problem-solving ability, originality, divergence, etc.). For instance, Jordanous (2012) has suggested a system-specific approach whereby the researcher is provided with a set of 14 evaluation parameters in order to select the most appropriate ones according to the context of the assessed system. In a follow-up work, Jordanous (2016) proposed that a good strategy in computational creativity evaluation may be to make an assessment based on the pair value–novelty, the importance of which has also been highlighted by various researchers (Jordanous, 2012; Mayer, 1999; Ritchie, 2007; Wiggins, 2006), under the perspective of the four Ps (i.e., Person/Producer, Process, Product, Press/Environment) (MacKinnon, 1970; Rhodes, 1961).

A precursor study (Zacharakis, Kaliakatsos-Papakostas, Tsougras, & Cambouropoulos, in press) examined harmonic conceptual blending by focusing on the – arguably – most fundamental harmonic concept: the cadential progression. Following Zbikowski’s pragmatic approach to defining concepts (Zbikowski, 2002, 2006, this issue) we assume that a (music) concept is not necessarily tied to language but is rather based on the formation of cognitive categories stored in memory that can be related to any other sort of concepts. Musical cadences, that signify the endings of musical phrases/sections, constitute well-established categories in the minds of listeners and may be considered as harmonic concepts (Zbikowski, 2002, pp. 60–61). In Zacharakis et al. (in press), a number of blends between the perfect and the Phrygian cadential progressions (i.e., transitions between two chords ending phrases) were produced based on a model of chord transitions. A selection of these blends was then subjectively evaluated through both verbal and non-verbal means in an effort to assess the blending methodology. The empirical evaluation of cadence blending showed that applying the blending methodology to pairs of chord sequences typical of cadences can result in the production of interesting and meaningful blends (new cadential transitions such as the tritone substitution progression).

The current study extends the evaluation methodology of the above study to entire harmonic spaces and blends between them. Creativity evaluation within the context of the present harmonic blending system is broken down into three components, the first two being value (relating to aesthetic preference) and novelty (expectancy) of the produced harmonisation, and the third being the perception of “blendedness” (i.e., whether a given CHAMELEON harmonisation is recognised as a harmonisation in either of the two input harmonic idioms or as a cross-idiom blend between the two or neither of these options).

In a first experiment, six melodies were harmonised in two different harmonic styles (e.g., Bach chorale and classical jazz harmonies, or diatonic major and minor mode harmonies) plus a number of in-between blended harmonies. Listeners were asked to place each harmonisation on an axis having each style at its two ends (plus an “other” option if they felt it belonged to neither of the input harmonic spaces or in-between the two) and also to indicate their preference for each harmonisation. The scope of this experiment was to examine whether CHAMELEON harmonisations in the input styles and blends between them were perceived as such, or if blends were perceived as not relating to the initial spaces (“other” category). Additionally, listeners’ preference indications were sought in order to see whether the harmonic output produced by the system is more, less or equally appreciated in comparison to harmonisations in the established input harmonic styles. We were also interested to examine the potential influence of melody on this process (e.g., whether a jazz melody harmonised in Bach chorale style would be considered as a Bach chorale harmonisation or be placed somewhere in-between the Bach chorale and classical jazz idioms).

In a second experiment, one melody was harmonised in the Bach chorale idiom (major tonality) and various blends between different major tonalities (e.g., G major and B♭ major). Listeners were asked to judge whether the resultant harmonisations were diatonic, chromatic, atonal or “other”, to indicate their preference and also to indicate how expected the harmonisations were. This experiment’s scope was to study whether blending the same harmonic space in two different tonalities generates blends that are perceived as going beyond the diatonic input spaces (e.g., chromatic or atonal or “other”); if this was the case, then we could argue that CHAMELEON is creative in the sense that it can generate new harmonisations that are identified by listeners as belonging to a subsequent historically established idiom (e.g., chromaticism), without any knowledge of that idiom by the system (having merely diatonic harmonic spaces as input). Additionally, the pairs of indications by listeners for preference and expectancy are useful in judging whether the output of the system is not only unexpected (signifying originality) but also preferred (unexpectedness alone can be misleading as a harmonisation may be unexpected but considered aesthetically bad/poor).

In the next section the overall CHAMELEON melodic harmonisation assistant will be described with special emphasis on its harmonic blending core. Then, the two empirical studies that were set up for evaluating the creativity of the proposed system will be presented. Finally, the main findings of this empirical study will be discussed and future directions of research will be outlined.

The CHAMELEON melodic harmoniser assistant that blends harmonic spaces

Harmonic diversity in different musical styles/idioms is established by independent harmonic spaces that involve numerous concepts such as chord type, root, scale hierarchy, tonality, harmonic rhythm, harmonic progression, voice-leading, implied harmony, reduction, prolongation, and so on. Conceptual blending aims to exploit the rich background of concepts that is available in diverse input idioms and to construct new harmonic spaces that creatively combine elements of the concepts in the input harmonic spaces. The combination of concepts from different idioms injects novelty and creativity to the melodic harmonisation process.

Even though chord progressions and harmonic structure in tonal and jazz music have been effectively described by models related to grammar structures (Granroth-Wilding & Steedman, 2014; Koops, Magalhães, & de Haas, 2013; Rohrmeier, 2011), for the purposes of blending, more musical styles that are substantially different from the aforementioned ones need to be represented. The melodic harmoniser presented in this study follows a modular, hierarchical representation of harmonic structure and is able to learn from data of practically any musical idiom through statistical learning. Furthermore, this system employs conceptual blending to combine learned, diverse harmonic styles and generate new “meaningful” harmonic idioms that can be used to harmonise given melodies. The blending methodology is based on the framework developed in the COINVENT project while it is applied on the level of chord transitions, leading to the construction of Markov transition probability matrices that blend the elements of the respective matrices from learned initial idioms. Markov matrices are square matrices that include transition probabilities between chords found in a dataset, with each row and column corresponding to a chord. The next subsection briefly describes the idiom-independent harmonic learning and generating methodology while the following subsection includes a short overview of the harmonic blending methodology; finally, the last subsection gives some examples of the way the system works and its output. More details for both methodologies can be found in Kaliakatsos-Papakostas et al. (2016) and Kaliakatsos-Papakostas et al. (2017) respectively.

Statistical learning of harmonies and melodic harmonisation in diverse idioms

The melodic harmoniser used for producing the material of this study is based on a statistical approach that combines different learning modules concerning different aspects of harmony. The proposed probabilistic algorithms allow for diverse harmonic idioms to be learned, generating harmonisations that reflect the characteristics of learned idioms in terms of chord types, chord transitions, cadences and bass-line voice leading. The system learns the harmonic content of an idiom through annotated training data, while it produces new harmonisations according to guidelines provided in the annotated melody input file. The harmonic training pieces have been manually annotated by music experts in terms of the following structural aspects: (a) harmonic reduction(s) of each musical work/excerpt are created so that structural harmonic/non-harmonic notes are explicitly marked; (b) local scale/key changes are determined so that harmonic concepts relating to modulations can be learned; and (c) grouping structure is given so that cadential patterns at various hierarchic levels can be inferred.

More specifically, the system learns independently a number of harmonic aspects of a given idiom:

Chords and chord types are learned in the form of the General Chord Type (GCT) (Cambouropoulos, 2015; Cambouropoulos, Kaliakatsos-Papakostas, & Tsougras, 2014), followed by a grouping stage based on the relations between learned chord types (Kaliakatsos-Papakostas, Zacharakis, Tsougras, & Cambouropoulos, 2015). The GCT describes a simultaneity of notes in the form of a chord with a root note and a generic type. The computation of root and type is based on the definition of consonant intervals and the tonality of the examined piece.

Chord progressions are learned via a model based on hidden Markov Models (HMMs), namely the constraint HMM (cHMM) (Kaliakatsos-Papakostas & Cambouropoulos, 2014), which allows the generation of chord sequences that comply with given chord constraints (either the imposed cadences by the following module or user-defined chord constraints). The HMMs (and the cHMMs) combine probabilities from a Markov matrix that describes transitions of a hidden variable (chord transitions) and conditional probabilities of hidden variables given some observations (melody notes), for constructing the most probable hidden variable (chord) sequences given a set of observations (melody).

Cadences, considered as the final pair of chords on phrase endings (Kaliakatsos-Papakostas et al., 2016), are learned in the form of simple statistics regarding their number of occurrence in the training corpus.

Bass-line voice leading is learned by combining three statistical models: (a) an HMM learning the bass contour (hidden states) transitions, given the melody contour (observations), (b) distributions of the distance/range between the bass and the melody voice and (c) statistics regarding the inversions of chords (Makris, Kaliakatsos-Papakostas, & Cambouropoulos, 2015).

After the system is trained, it can harmonise a given melody that, at this stage, is manually annotated in terms of harmonic rhythm, harmonically important notes, key and phrase structure. Learned cadences are placed at positions indicated as phrase endings in the melody input files and, then, chord sequences in the GCT representation are generated with the cHMM methodology. The bass voice is determined by combining the bass voice related statistical models and, finally, the inner voices (between the bass and the melody) are placed according to criteria concerning attraction to a given intermediate pitch height, evenness with neighbouring note distances and movement distances of inner voices between successive chords (Kaliakatsos-Papakostas et al., 2016). The output of this system is a harmonic realisation with actual chord notes (not only chord symbols). Two examples of melodic harmonisation are presented in Figure 5 (Mj & Mn) and Figure 8 (Mn); for further examples (scores and audio) see the CHAMELEON¹ website.

Chord transition blending for blending harmonic spaces

The COINVENT framework (Schorlemmer et al., 2014) for conceptual blending extends Goguen’s (2006) generative model, according to which input spaces are described as algebraic specifications and their blended space is computed following a certain partial generalisation process that meets specific criteria of consistency; the generic space (i.e., the set of common properties between the input spaces) is found by generalising (removing) all input property values that are not shared between the two spaces.

The proposed melodic harmonisation system incorporates the above conceptual blending mechanism focusing on chord transitions. A central component of the harmonic description of an idiom is the first-order Markov matrix of GCT chord transitions; blending is realised at the level of chord transitions, i.e., chord transitions from one idiom are blended with ones from the other to generate blended transitions that reinforce the connections between the two chord transition spaces. The outcome of this process is a set of novel transitions (may include new chord types) that preserve salient features of the input transitions.

The conceptual blending technique (based on the COINVENT core model) was initially applied to a specialised problem involving well-defined harmonic concepts, namely cadence progressions (Eppe et al., 2015; Zacharakis, Kaliakatsos-Papakostas, & Cambouropoulos, 2015; Zacharakis et al., in press). In Figure 2 the perfect and the renaissance modal Phrygian cadences are used as input spaces. These have been chosen as they are both final cadences to the tonic and, at the same time, they are very different (i.e., the Phrygian mode does not have an upward leading note to the tonic but rather a downward “leading note” from II♭ to I). Assuming that the final chord is always a common tonic chord, blending takes place by combining pitches of the penultimate chords between different cadential progressions. Each chord transition is described as a set of weighted property values that reflect relative prominence, such as the chord type of the penultimate chord, interval between chord roots, bassline interval, upwards or downwards leading note, dissonant note that requires resolution (the thickness of lines in Figure 2 illustrates the relative prominence of voice-leading properties). The “blended” penultimate chord is additionally constrained to comply with a certain chord type such as, in this instance, the characteristic major chord with minor seventh. Among the accepted blends, one of the most highly rated ones (based on prominence values) is the tritone substitution progression (II♭7 – I) of jazz harmony. This simple blending mechanism “invents” a chord progression that embodies characteristics of the Phrygian cadence (root/bass downward motion by a semitone) and the dominant seventh chord (resolution of tritone, upward leading note). Thus, it creates a new harmonic “concept” that was actually introduced in jazz, centuries after the original input cadences.

Figure 2.

Cadence blending schematic diagram.

Transition matrix blending can be considered as a generalisation of cadence blending where the final chord is not fixed. A richer and more general representation of transitions is used for transition blending in comparison to the simpler description of cadences. The new blended harmonic idiom comprises the transitions of the initial idioms, along with a set of novel chords and transitions that provide creative and meaningful harmonic connections between the chords of the initial spaces.

The transition matrix blending process is now briefly described; the reader is referred to Kaliakatsos-Papakostas et al. (2017) for a detailed description. The above chord transition blending mechanism is applied to the most frequent chord transitions of each chord transition matrix (we select the 10 most frequent transitions). Each chord transition of the first space is blended with each of the most frequent transitions of the second space; this way, a large number of new transition blends is generated. These blends may contain new connections between existing chords of the two spaces, or new chords (not in the input spaces) that are compatible with background knowledge regarding the two input harmonic systems, i.e., belong to the chord types that are learned from the input idioms. Through this process, new chords are most likely to emerge as first or second chords in newly invented transitions. Transitions that incorporate newly invented chords that are not reachable from the two input spaces are filtered out (Kaliakatsos-Papakostas et al., 2017). A new chord is reachable if there are at least two transitions including this chord as the first and second chord respectively.

The new potential blends are ranked by summing the quantified salience of the features each blend has inherited from the input spaces. The salience value of a feature in a transition depends on the idiom that this transition belongs to and reflects how “characteristic” or “unique” this feature is for the examined transition in the corresponding idiom (property values that are shared by few transitions, i.e., are relatively rare, are more characteristic). Therefore, blends that are ranked high are the ones that inherit more features from the inputs that have greater salience values (are more characteristic; Kaliakatsos-Papakostas et al., 2017).

A compound chord Markov transition matrix is constructed, providing musically meaningful chord transitions between learned chords of the initial idioms, along with new chord transitions that connect the two spaces via new chords (chords not in the original spaces). Initially, before blending, same/similar chord transitions of the initial idioms are identified, enabling musically “natural” connections from chords of one initial idiom to chords of the other. Figure 3 illustrates the general form of a compound chord Markov transition matrix that extends two initial idioms, I₁ and I₂. Sections A_i-j of the matrix include chord transitions that are shared between the two input spaces (this is a pre-blending process) or have been created through blending as explained above. Transitions in these sections lead directly from idiom i to idiom j. Sections B_i-x include transitions that lead from idiom I_i to a new chord generated with transition blending, while B_x-i transitions lead from a new chord back to idiom I_i. Section C incorporates transitions between new chords but these are not considered herein, since the harmoniser in this study works under the assumption that a new chord can only be used as an intermediate “node” for transitioning from I_i to I_j.

Figure 3.

Graphical description of a compound matrix that includes chords and transition probabilities of both input idioms, along with new chords that are generated through transition blending.

Chord transition blending can create new transitions that preserve important features of the input transitions, while the number of topmost selected transitions among the blends for further processing is determined by the user of the system. A larger number leads to a compound matrix that is more populated, including more transitions and chords. Additionally, the user can select the intensity of the probabilities in the transitions that result from blending; higher probabilities force the system to move more freely between chords of the compound space, regardless of the initial space they belong to, creating a space with radical differences compared to the initial harmonic spaces (Kaliakatsos-Papakostas et al., 2017).

Examples of chord transition matrix blending

To illustrate the functionality of the proposed blending approach, we use the example in Figure 4, taken from Kaliakatsos-Papakostas et al. (2017), where the transitions of two purely diatonic but distant harmonic spaces are blended. These spaces are assumed to consist only of three basic chords, namely the tonic, the subdominant and the dominant; they do not overlap since they have no common chords and there is zero probability for transitions between chords of one space to chords of the other (as illustrated by the white colour squares in Figure 4 (a)). In the case where the system is requested to harmonise a melody that begins in C major and then modulates to F# major, the Markov-based model of the non-blended harmonic spaces reaches an impasse since there is no possible transition (with probability greater than 0) that leads to F# tonality chords. The role of transition blending between the matrices representing the two distant tonalities in this example is to generate chord transitions that allow creative and meaningful transposition “paths” from one tonality to the other. The new transitions illustrated in Figure 4 (b) are the topmost blends according to the ranking produced by a rating process that takes into account the number of common features between the blend and the input chord transitions (features include common pitch classes in the first and second chords of the blend in relation to the two input transitions and ascending/descending semitone movements to the root of the final chord of each transition). For instance, the chord transition methodology finds that the best way to connect the two spaces using only chords of the two spaces is when the first chord is a major seventh chord a semitone above the tonic of each space (e.g., 1 5 8 11) – this is a kind of tritone substitution progression (see new grey cells within the empty square submatrices of Figure 4 (b)). This new transition is preferred to all other blends between the chord transitions of the two spaces as it preserves many of the properties shared by the perfect cadences of both spaces (C#7 to F# and G7 to C) such as semitone movement to tonic, interval transition content, chord types, and the two common pitch classes of the tritone in the major seventh chords. Allowing the introduction of new chord types (from a certain background palette of chords such as minor, minor seventh, diminished seventh chords, etc.), a chord transition blend that ranks highly is a transition in which the first chord is a diminished seventh (pcs: 2 5 8 11) and the second chord is either of C or F# (i.e., the system “invents” the use of diminished seventh chords as a way to modulate to remote keys) – see external new column and row in Figure 4 (b). The diminished chord transition is invented by blending transitions related to the perfect cadences of both harmonic spaces. For instance, blending the perfect cadences, as in the tritone substitution example, the diminished chord is preferred to other chords as again it preserves the property of the leading note to the tonic, interval transition content and the common pitch classes of the tritone in the major seventh chords (see more details in Kaliakatsos et al., 2017).

Figure 4.

(a) A compound harmonic space of C and F# major diatonic transition spaces that does not include blends, and (b) with some of the highest rating blends.

Two examples of melodic harmonisation are illustrated in Figures 5 and 8. In the first example, a purpose-made melody, in which the 3rd and 6th melodic degrees of the diatonic scales have been omitted, was harmonised in the major and minor modes, and in three hybrid major–minor blends. In the second example, different harmonisations of the traditional Scottish melody Ye Banks and Braes are presented (one tonal harmonisation in G major, one “wrong” harmonisation in Bb major, three different blends between the “correct” tonality of the Bach chorales idiom and its transposition by two, three and four semitones respectively, and finally, a “peculiar” blend between the style of Hindemith and a transposition of The Beatles style by three semitones). Voice leading in these examples is rather primitive (but better than closed chords in root position); Roman numeral harmonic analysis has been added by a human expert in order to assist readability. These two examples (among others) were used as stimuli in the empirical study reported in the next section. For further blending examples (scores and audio) see the CHAMELEON² website.

Figure 5.

The major–minor harmonisations. Mj: Major, Mn: Minor, L, M, H: low, medium, and high blends respectively. Roman numeral harmonic analysis has been added manually by a human expert to facilitate readability of the score.

Empirical evaluation of harmonic blending performed by CHAMELEON on selected melodies

Since the developed system for harmonic blending falls into the scope of computational creativity, a comprehensive assessment of its value would require an attempt to “measure” its creativity. The current work adopts a product-oriented evaluation of the harmonic blending system through empirical testing. It focuses both on the perception of “blendedness” per se – that is, whether listeners recognise a blend as distinct from, but related to, the initial input idioms – and on the value (aesthetic preference) and novelty (expectancy) assigned to the produced harmonic blends by subjects. Annotated melody files were used as inputs to CHAMELEON for generating the stimuli of the two experiments described in this section. The idioms employed in the experiments were learned from sets of Bach chorales and Jazz standards, while some learned idioms were acquired from sets of songs by The Beatles and pieces by Hindemith. The choice to incorporate mainly the Bach chorale and Jazz idioms in the experiments is based on the assumption that participants would be familiar with them and could distinguish between the original idioms and/or variations of them.

The first experiment is designed to assess the effect of blending between these idioms through perceptual tests on categorising the produced melodic harmonisations. An additional inquiry examined in this experiment is the effect of the style/character of the melody in the melodic–harmonic blend. To this end, tonal and jazz melodies were harmonised with the Bach chorale and Jazz idioms interchangeably, as well as with blended versions of these two.

The Bach chorales are among the most characteristic paradigms of tonal music, making them perfectly suitable for examining whether the tonal character of this idiom can be drastically altered through blending-based techniques Specifically, the learned Bach chorales harmonic idiom was transposed in several keys and blending between these transposed spaces and the original key created new spaces that introduced harmonic elements that extended the initial tonal idiom. This effect and the extent of this diversification of the tonal spaces is assessed in the second experiment in which a traditional melody is harmonised in the idiom of the Bach chorales, and transposition-related blends.

Experiment 1: Idiom and mode classification

Stimuli

Six blocks of stimuli were presented in the idiom blending experiment. Each of the first five consisted of a different known tonal or jazz melody that was harmonised by the system according to the tonal idiom (learned through a set of Bach Chorales), the jazz idiom (learned through a set of jazz standards) and some of their blends. In a couple of cases some harmonisations were obtained either by blending between two other idioms (Beatles and Hindemith) or according to a third idiom (Hindemith). Two of the five melodies featured tonal implied harmony (the “Ode to Joy” theme in L. W. Beethoven’s 9th Symphony and “Ah vous dirai-je, maman”, which is a French children’s song used as the theme in W. A. Mozart’s Piano Variations K265), the other two featured Jazz implied harmony (“Summertime” by G. Gershwin and “Someday my prince will come” by F. Churchill, soundtrack from Disney’s Snow White and the Seven Dwarfs (1937)), while the last one was a Greek folk song melody (“Του Κίτσου η μάνα”). The last block consisted of a melody that was especially composed for the needs of the experiment and deliberately avoided the 3rd and 6th melodic degrees of the C scale, so that harmonisation in either the major or minor mode was equally possible. This melody was harmonised according to both modes and their blends as shown in Figure 5. The overall number of stimuli (presented in Table 1) in all six blocks was 25.

Table 1.

Excess kurtosis values of the classification and preference distributions for each stimulus.

	Ah vous dirai-je, maman				Summertime					Greek folk song					Ode to Joy			Someday my prince will come			major–minor
	Bc	Jz	Bl_L	Bl_M	Bc	Jz	Bl_M	Bl_H	Hm	Bc	Jz	Bl_M	Bl_H	other	Bc	Jz	Bl_L	Bc	Jz	Bl_M	Mj	Mn	Bl_L	Bl_M	Bl_H
Classification	4.79	−.23	.08	.54	−.68	−.38	.87	−.37	−.28	2.9	−.51	.55	−.94	−.64	−.04	1.7	.67	−.92	−.24	−.6	6.7	1.55	1.2	2.5	−.25
Preference	−1.0	−.94	−1.1	−.6	.39	−.45	−.33	−.2	.28	−0.4	−.29	−.12	.47	−.51	−.17	−59	.54	.10	2.0	−.25	.09	.08	.07	−.54	.10

Note. Bc: Tonal harmonisation, Jz: Jazz harmonisation, Bl_L_H_M: blended harmonisation with low, medium or high blending rate respectively, Hm: Hindemith harmonisation, other: Beatles–Hindemith blend).

Procedure

The experiment took place in three different sessions with simultaneous stimuli presentation through loudspeakers to a group of subjects following the experimental design by Antovic, Stamenkovic, and Figar (2016). Each listening session featured between 10 and 20 participants. The presentation order of the blocks, as well as the presentation order of the different harmonisations within each block, was different for each experimental session. Listeners were provided with a questionnaire asking them to classify each stimulus on a 5-point likert scale between Tonal and Jazz. Apart from the three in-between positions, which implied that the stimulus could not be classified as purely Tonal or Jazz but rather as somewhere in-between, the option “other” idiom was also provided. In addition to classification, participants were asked to note their preference for each stimulus in a scale ranging from 1 to 10. As a familiarisation stage, participants first listened to one tonal and one jazz harmonisation of the traditional Scottish melody Ye Banks and Braes and were asked informally to classify them before proceeding to the main experiment. Multiple playbacks of the stimuli were offered in the case that a listener was unsure of the classification he/she should assign. Overall, including instructions, each session lasted about 30 minutes.

Participants

Forty listeners (mean age 22, age range = 18–45, 18 male) volunteered to participate in the first listening experiment. Participants were students from the School of Music Studies at the Aristotle University of Thessaloniki. All of them reported normal hearing and long-term music practice (12.8 years on average, ranging from 5 to 30). All participants were naive about the purpose of the test and especially about the fact that the creative agent under consideration was computational rather than human.

Results

Figure 6 presents the histograms for the six categories provided for classification together with the preference ratings on the 10-point scale. Table 1 shows the excess kurtosis values (kurtosis – 3) for the preference ratings and for the ratings on the Tonal vs. Jazz classification (where the “other” bin was omitted). Lower kurtosis values signify the existence of frequent but mediocre outliers (in terms of deviation from the mean) while higher values represent either a distribution strongly concentrated around the mean with only occasional but strong outliers or a distribution with stronger concentration in the tails compared to the normal distribution (Moors, 1986). In our case, kurtosis is used to quantify the agreement among participants, where higher kurtosis values signify a strong concentration around the mean and therefore higher agreement. It can be observed that the agreement regarding classification is generally greater when melody and harmonisation come from the same idiom (i.e., when no blending of any form takes place). Also, the mode classification (major vs. minor) distributions feature the highest kurtosis values compared to all other distributions. Preference ratings lead to somewhat flatter distributions (but not too far away from normal) for the majority of the stimuli.

Figure 6.

Histograms of the participants’ responses regarding classification (left) and preference (right) for the different melodies and harmonisations. Bc: Tonal harmonisation, Jz: Jazz harmonisation, Bl_L_H_M: blended harmonisation with low, medium or high blending rate respectively, Hm: Hindemith harmonisation, other: Beatles–Hindemith blend.

Effect of harmony

Since most of the distributions failed to pass the Shapiro-Wilk test of normality, a non-parametric approach was adopted for the analysis of the data. Friedman’s ANOVA tests were applied to the Tonal vs. Jazz discrete variable (once again excluding the “other” field) to reveal a potential effect of harmonisation on idiom classification and/or preference for the examined melodies. The results shown in Table 2 indicate that while the different harmonisations indeed affected the idiom classification of every single melody, preference was affected only for Ode to Joy and major–minor at the p < .001 significance level.

Table 2.

Friedman’s ANOVA for classification (without the ‘other’ field) and preference.

		Ode to Joy	Ah vous dirai-je, maman	Greek folk song	Summer time	Someday my prince will come	major-minor
Classification	χ²	70.12	56.87	48.95	46.58	56.853	75.908
	df	4	3	4	2	2	4
	p	<.001	<.001	<.001	<.001	<.001	<.001
Preference	χ²	20.33	8.130	8.153	8.153	7.986	19.532
	df	4	3	4	2	2	4
	p	.000	.043	.086	.079	.018	.001

As a post-hoc analysis, Wilcoxon signed-rank tests for all possible pairs of harmonisations within each melody were applied to identify the exact pairs that were classified as significantly different. Bonferroni correction (p/number of comparisons) was also applied to correct for multiple comparisons. The analysis identifying the significantly different pairs for each melody together with their effect sizes is presented in Appendix S1 in the Supplemental Material Online section (Tables S1 to S6). Figure 7 (a) and (b) shows the median of each stimulus considering the Tonal–Jazz and the minor–major modes as continuous variables to provide a qualitative picture of the above statistical findings. Overall, this analysis shows that, with the exception of Ode to Joy where the success of the blending system was partial, the harmonisations produced by the system for the rest of the melodies seemed to have generated distinguishable harmonic idioms.

Figure 7.

(a) Medians (triangles) and means (circles) of all stimuli for idiom classification on the Tonal–Jazz continuum. The column on the right depicts the percentage of participants who selected the “other” field, and (b) medians (triangles) and means (circles) of all stimuli for mode classification on the Minor–Major continuum. The column on the right depicts the percentage of participants who selected the “other” field.

In the same manner as above, significant differences in preference as a result of harmonisation for each separate melody were further examined in a post-hoc analysis. Since Friedman’s ANOVA showed a significant effect (at the level of p < .001) only for Ode to Joy and major–minor, these were the only two melodies that were tested through a Wilcoxon signed-rank test. Table S7 (see Appendix S1 in the Supplemental Material Online section) shows the pairs that featured a significant difference in preference along with the effect sizes. The “other” harmonisation of Ode to Joy was significantly less preferred compared to the Bach, Jazz and medium blend and also the major harmonisation of the major–minor melody was less preferred compared to the minor one. Based on these results, it can be maintained that, with very few exceptions, there were no significant differences in preference for the different harmonisations of a given melody.

Effect of melody

Since we showed that the different harmonisations affected idiom classification, we were additionally interested to examine whether each of the melodies under study was affected by a different harmonisation in the same way. To this end, we considered only the harmonisations according to Bach’s chorales (Tonal) and Jazz that were shared by all blocks of melodies. The assumption here was that if idiom perception was merely based on harmonic style then idiom classification would be the same for these two harmonisations regardless of the harmonised melody. A Friedman’s ANOVA for Tonal and Jazz harmonisations across the five tested melodies did not confirm this hypothesis. Table 3 shows that the classification of both Tonal and Jazz harmonisations was affected by the melody itself despite not considering the “other” field. More specifically, the post-hoc Wilcoxon signed-rank tests presented in Table 4 showed that the tonal harmonisation of the jazz melody Summertime was classified as less tonal than the ones of the tonal melodies Ode to Joy and Ah vous dirai-je, maman. Additionally, the Jazz harmonisation of the jazz melody Someday my prince will come was classified as more jazz than the jazz harmonisation of the tonal melodies Ode to Joy and Ah vous dirai-je, maman. Finally, even the jazz harmonisation of the modal melody of the Greek folk song was classified as more jazz than the jazz harmonisation of the tonal Ode to Joy.

Table 3.

Friedman’s ANOVA (without the “other” field) for comparison between the same harmonisation style across the five melodies.

	Tonal	Jazz
χ²	14.92	29.52
df	4	4
p	.005	<.001

Table 4.

Significantly different pairs identified through Wilcoxon signed-rank tests with Bonferroni correction (p/10).

	Tonal		Jazz
	Ode to Joy vs. Summertime	Ah vous dirai-je, maman vs. Summertime	Ode to Joy vs. Greek folk song	Ode to Joy vs. Someday my prince will come	Ah vous dirai-je, maman vs. Someday my prince will come
z	−3.3	−3.09	−3.42	−4.29	−3.29
p	.001	.002	.001	<.001	.001
Effect size	−.37	.35	.39	−.49	−.39

Note. Effect size: ( $r = \frac{z}{\sqrt{N}}$ ).

Experiment 2: Type of chromaticism classification

Stimuli

This additional experiment featured one traditional Scottish pentatonic melody (Ye Banks and Braes) harmonised using the following harmonic spaces:

A tonal idiom as learned from a set of Bach chorales (indicated by “Tn”).

A “wrong” harmonisation obtained by harmonising the melody in the wrong key, i.e., B♭ major instead of G major (“Wr”).

A peculiar blend between the style of Hindemith and a transposition of The Beatles harmony by three semitones (“HB”).

Three different blends between the “correct” tonality of the Bach chorales idiom and its transposition by two, three and four semitones (“BL_2”, “BL_3” and “BL_4” respectively).

The total number of stimuli was 7 since the tonal harmonisation was presented twice to test consistency of the responses. The inclusion of the “wrong” and “peculiar” blends in the stimulus set was made in order to use them as potential low references for preference and expectedness ratings and as potential candidates for the Atonal and Other categories. Figure 8 illustrates the six different harmonisations of Ye Banks and Braes.

Figure 8.

The different harmonisations of the traditional Scottish melody Ye Banks and Braes. Tn: tonal idiom as learned from a set of Bach chorales (where -1 and -2 indicate the order of appearance of the same stimulus in the set), BL_2, BL_3 and BL_4: blends between the “correct” tonality of the Bach chorales idiom and its transposition by two, three and four semitones respectively, HB: peculiar blend between the style of Hindemith and a transposition of The Beatles style by three semitones, Wr: “wrong” harmonisation obtained by harmonising the melody in the wrong key (Bach chorales idiom transposed by three semitones, i.e., B♭ major instead of G major). Roman numeral harmonic analysis has been added manually by a human expert to facilitate readability of the score.

Procedure

The experiment took place in two different sessions, each one featuring 10 to 20 participants. Listeners were provided with a questionnaire asking them to classify each stimulus in one out of four categories: diatonic, chromatic, atonal, and other. The presentation order for the different harmonisations was kept the same for all sessions as it was assumed that it would not affect the judgements. The smaller number of stimuli (5 vs. 25) compared to the style classification experiment (translated into shorter experimental time) allowed us to request one additional rating apart from chromaticism class and preference for each stimulus: the degree of expectancy characterising each harmonisation. The scale of the preference and expectancy ratings ranged from 1 to 5. Multiple playbacks of the stimuli were offered in the case that a listener was unsure of the classification he/she should assign. Overall, including instructions, each session lasted about 15 minutes.

Participants

Thirty listeners (mean age 22.2, age range = 19–29, 13 male) volunteered to participate in the listening experiment. Participants were students from the School of Music Studies at the Aristotle University of Thessaloniki. All of them reported normal hearing and long-term music practice (12 years on average, ranging from 6 to 20). All participants were naive about the purpose of the test and especially about the fact that the creative agent under consideration was computational rather than human.

Results

Figure 9 presents the histograms for the four categories provided for type of chromaticism classification together with the preference and expectancy ratings on the 5-point scale. Similarly to the previous analysis, Table 5 presents the excess kurtosis values (kurtosis – 3) for all three distributions (treating the style classification as a discrete rather than a categorical variable for depiction purposes) in order to get a quantification of the outlying values. The results of a Friedman’s ANOVA on the preference and expectancy distributions (shown in Table 6) revealed that there was an effect of harmonisation on both properties. Tables 7 and 8 present the significantly different pairs of harmonisations that resulted from a post-hoc analysis (i.e., Wilcoxon signed-rank tests).

Figure 9.

Histograms of the responses regarding type of chromaticism classification (left), preference (middle) and expectancy (right) for the different harmonisations of Ye Banks and Braes. Dtn: Diatonic, Chr: Chromatic, Atn: Atonal, Oth: Other. Tn: tonal idiom as learned from a set of Bach chorales (where -1 and -2 indicate the order of appearance of the same stimulus in the set), BL_2, BL_3 and BL_4: blends between the “correct” tonality of the Bach chorales idiom and its transposition by two, three and four semitones respectively, HB: peculiar blend between the style of Hindemith and a transposition of The Beatles by three semitones, Wr: “wrong” harmonisation obtained by harmonising the melody in the wrong key (Bach chorales idiom transposed by three semitones, i.e., B♭ major instead of G major).

Table 5.

Excess kurtosis values for type of chromaticism classification, preference and expectancy responses for the different harmonisations of Ye Banks and Braes.

	Excess kurtosis
	Tn-1	BL_4	BC_3	BL_2	Tn-2	BH	BL_3
Style classification	∞	−1.44	.55	−.09	8.88	6.25	−.63
Preference	−.077	−.03	3.52	1.26	−1.07	1.38	−.77
Expectancy	.35	.05	.26	−1.26	1.87	2.85	−.14

Table 6.

Friedman’s ANOVA for expectancy and preference between the various harmonisations of Ye Banks and Braes (N = 30).

	Expectancy	Preference
χ²	118.05	85.40
df	6	6
p	< .001	< .001

Table 7.

Significant difference for expectancy from post-hoc analysis (Wilcoxon signed-rank test) of all pairs (Bonferroni correction p/21).

	Tn-1	BL_4	BC_3	BL_2	Tn-2	BH	BL_3
Tn-1	–
BL_4	X	–
BC_3	X		–
BL_2	X		X	–
Tn-2		X	X	X	–
BH	X	X		X	X	–
BL_3	X		X		X	X	–

Table 8.

Significant difference for preference from post-hoc analysis (Wilcoxon signed-rank test) of all pairs (Bonferroni correction p/21).

	Tn-1	BL_4	BC_3	BL_2	Tn-2	BH	BL_3
Tn-1	–
BL_4		–
BC_3	X	X	–
BL_2			X	–
Tn-2			X		–
BH	X	X		X	X	–
BL_3			X			X	–

According to Figure 9 and Table 5, the Tn harmonisation was unanimously classified as diatonic in the first presentation and the same with very high agreement in the second exposure where it was mistaken for chromatic, atonal or other from only four out of 30 participants. In addition, it was rated as the most expected harmonisation and it was more preferred compared to Wr and HB. The Wr and the HB harmonisations were classified as Atonal with high agreement and were attributed the highest unexpectedness (with an exception between Wr and BL_4 where the difference was not significant) and the least preference among the rest of the harmonisations. The BL_2, BL_3 and BL_4 were mostly rated as chromatic but with some percentage of participants rating them as diatonic or even as other. The BL_4 was the harmonisation that featured the least agreement among participants for chromaticism classification. These three harmonisations were rated as less expected than Tn but more expected than Wr and HB. The difference of expectancy among them was insignificant. Regarding preference, BL_4 was significantly less preferred than BL_3 but no significant difference was found among any of these blends and Tn.

Discussion and future work

As discussed in the introduction of the previous section, a creative agent may be evaluated both in terms of the processes it incorporates and in terms of the artefacts it generates. This piece of research has focused on the empirical evaluation of a number of computer-generated melodic harmonisations (i.e., products of the harmonic blending system). The proposed computational system has a well-defined target, which is the creation of hybrid harmonic idioms through harmonic blending. In this sense, there was a clear criterion for measuring success: the extent to which the produced harmonisations are perceived by listeners as being instances of the input idioms, hybrids between them, or new idioms not related to either of the original idioms. Value and novelty of the generated harmonisations, which are indicators of creativity, are also assessed by requesting judgements on preference and expectancy.

The empirical experiments have shown that the different harmonisations produced by the system have indeed influenced idiom perception. The harmonisations according to purely tonal or jazz transition matrices were mostly classified as belonging to the original harmonic styles, whereas harmonic blends between classical tonal and jazz harmony were mostly perceived as belonging to a hybrid jazz-tonal idiom or, occasionally, to an unidentifiable “other” idiom. The “other” option was, as expected, most prominent in the two cases where the harmonisation style was neither of the tonal or jazz styles nor of their blends (i.e., Hindemith and a blend between Beatles and Hindemith). This shows that participants were mostly able to discriminate between a tonal-jazz blend and a harmonisation that was based on a totally different idiom.

Inspection of the kurtosis values of the classification distributions showed that the harmonisation of a melody by an idiom other than its original (either blended or not) generally lowered agreement among participants regarding the perceived musical style. That is, blending (either melody–harmony or harmony–harmony) seems to introduce some uncertainty for idiom identification. The identified effect of harmony on idiom perception was also melody dependent, meaning that idiom classification was made considering both harmonic and melodic characteristics. This was tested on harmonisations based on purely tonal or jazz styles so as to have the same reference for all melodies. This finding implies that even a melody–harmony blend (i.e., harmonisation of a melody according to an idiom other than its original) may constitute a perceivable type of blend. More thorough testing for this result with a dedicated experiment using a wider range of melodies and idioms is required. Finally, the differently harmonised versions of each melody were generally found to be equally preferred with the exception of the “other” harmonisation of Ode to Joy, which received significantly lower preference ratings.

The mode classification (major–minor) experiment showed that listeners were able to almost perfectly classify the major mode and were also very successful regarding the minor mode. The slight confusion observed in the minor mode classification may be due to the fact that the short excerpt was concluded with a Picardy third (major tonic chord) that is common in Bach’s minor mode chorales. This has expectedly shaped the judgements of the listeners slightly towards the major mode. Other than that, the blends were mostly classified as in-between the major and minor mode or, less commonly, they were rated as belonging to the “other” option. In addition, the preference ratings were significantly different only between the purely minor and major harmonisations (with a preference for minor). The blended harmonic versions were equally preferred to the major mode harmonisation, indicating that listeners valued the system’s major–minor creative blends positively.

Finally, the type of chromaticism classification experiment demonstrated that judgements on the type of chromaticism were quite consistent except for the BL_4 blended harmonisation, the ratings of which were almost equally distributed between tonal, chromatic and other. This shows that there was high agreement among listeners regarding the classification of the blended harmonisations produced by CHAMELEON into the three broad categories of diatonic tonality, chromatic tonality and atonality. More specifically, the tonal harmonisation according to Bach’s chorale style received a unanimous diatonic tonal rating in the first presentation (Tn1) with very slight differentiation in the second one (Tn2), showing consistency in listeners’ responses to the same stimulus. The “wrong” (Wr) harmonisation was mostly judged to be atonal, similarly to the HB “peculiar” harmonisation. The two diatonic blended harmonisations, BL_2 and BL_3 (two and three semitones apart), were generally considered to be chromatic (small leakage towards tonal and “other” in both cases).

The preference and expectancy ratings give interesting results regarding the creative aspects of the CHAMELEON harmonisation system. The diatonic tonal harmonisation (Tn) along with all the chromatic blended harmonisations (BL_2, BL_3, BL_4) received equally high preference ratings, indicating that listeners liked both the original tonal harmonisation and the chromatic blends. The standard tonal harmonisation received significantly higher ratings for expectancy compared to all the rest. This means that the blending system produced some novel harmonisations (i.e., BL_4, BL_2 and BL_3) that were both recognised as being stylistically rather unexpected or original and, at the same time, they were equally appreciated in comparison to a conventional tonal harmonisation. The combination of originality (i.e., something being unexpected) and relatively high preference is an indication of positively valued creativity. On the other hand, the “Wr” and HB harmonisations received significantly lower preference ratings accompanied by significantly lower expectation ratings; this constitutes a confirmation of the fact that excessive novelty often is not highly appreciated (Margulis & Beatty, 2008) and of our hypothesis that harmonisations deliberately produced to sound “wrong” are identified as such.

Overall, based on the above it can be supported that the harmonic blending system has indeed succeeded in producing perceivable blends – across idioms, modes and types of chromaticism – that are equally preferred compared to non-blends.

Future work will aim to quantify the potential influence of the harmonisation assistant on human creative processes. Such an evaluation would not be possible through the passive listening protocol adopted in this work; some active interaction with the system would be required instead. In this direction, a pilot study that aimed to assess the enhancement of human creativity through interaction with the system was conducted. The objective was to examine whether (and in what way) the harmonisation of a certain melody by humans would be affected as a result of participants’ exposure to a number of different harmonisations of the same melody produced by the system. This would, in turn, require a reliable measure of harmonic dissimilarity that could be used as a metric of harmonic divergence (i.e., inspiration offered as a result of an influence by the products of the system). This piece of research is still in progress and results will be reported in future work. The obvious next step would be to allow people to make use of the system without any type of mediation to evaluate both user experience and artefact production in such a scenario.

Finally, in the context of an active evaluation of the system, a small compositional “exercise” was set to seven composers or student composers in Thessaloniki. The composers were given three different traditional Greek melodies and 40 harmonisations in diverse styles and blends between them for each melody. They were asked to choose one melody and one or more harmonisations in order to compose a miniature piece for piano employing any compositional elaboration/variation techniques they wanted. The seven miniature pieces were presented on 19 October 2016 in a live concert at the Museum of Contemporary Art in Thessaloniki (recordings of the seven pieces are available at the CHAMELEON website).³ Most composers reported that they found the whole exercise very stimulating and that they thought that some of the harmonisations were particularly inspiring; some stated that they would have never come up with one or more of the harmonisations they used. Overall, there was a positive response regarding at least some of the creative products of this system. Hopefully, in the near future a user-friendly implementation of the CHAMELEON system will be developed so that it can be used more widely and more extended evaluation studies can be performed.

Conclusions

In this paper, Conceptual Blending has been explored in the context of a creative music system. A computational model of Conceptual Blending Theory has formed the core of a music generation system that creates actual melodic harmonisations combining properties from different harmonic input spaces. The CHAMELEON melodic harmonisation assistant, on the one hand, learns harmonies (chord types, chord progressions, cadences, bass-line voice leading) from human annotated harmonic data in diverse harmonic idioms and then harmonises a given melody in different harmonic styles, and, on the other hand, combines two different harmonic idioms into a new blended harmonic space that preserves characteristics from both input spaces in a consistent manner.

The proposed harmonic blending system generates novel blended harmonies and the current study attempts to evaluate its creative output following an empirical methodology, whereby listeners rate different versions of plain or blended harmonisations of different melodies in terms of “blendedness” (i.e., how they perceive a certain harmonisation as a blend relating to the two input source harmonic idioms), of “value” (relating to aesthetic preference) and “novelty” (expectancy). The experimental data show that the harmonic blending system has indeed succeeded in producing perceivable blends across idioms, modes and types of chromaticism. In most cases, the produced blends were equally preferred to the original non-blended harmonisations, but blended harmonisations were perceived as being more unexpected, i.e., having a higher degree of novelty. The fact that harmonic blends were considered to be both novel (unexpected) and aesthetically pleasing (preferred) indicates that the proposed harmonic blending system captures aspects of creativity, and supports the hypothesis that conceptual blending can be used as a generative mechanism to create novel concepts and artistic artefacts. Based on the empirical results, it will be interesting to study further whether the CHAMELEON melodic harmonisation assistant may be used creatively by composers as a source of harmonisation ideas that can provide novel harmonic material for the free development of musical textures.

Footnotes

Funding

This work was funded by the COINVENT project. The project COINVENT acknowledges the financial support of the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission, under FET-Open grant number: 611553.

Supplementary Material

Tables and figures/audio files with the index “S” are available as Supplemental Online Material, which can be found attached to the online version of this article at . Click on the hyperlink “Supplemental material” to view the additional files.

Notes

References

Ames

(1992). Quantifying musical merit. Journal of New Music Research, 21(1), 53–93.

Antovic

Stamenkovic

Figar

(2016). Association of meaning in program music: On denotation, inherence, and onomatopoeia. Music Perception, 34, 243–248.

Boden

M. A.

(2004). The creative mind: Myths and mechanisms (2nd ed.). London, UK: Routledge.

Cambouropoulos

(2015). The harmonic musical surface and two novel chord representation schemes. In Meredith

(Ed.), Computational music analysis (pp. 31–56). Berlin, Germany: Springer.

Cambouropoulos

Kaliakatsos-Papakostas

Tsougras

(2014, September). An idiom-independent representation of chords for computational music analysis and generation. In Georgaki

Kouroupetroglou

(Eds.), Proceedings of the joint 11th Sound and Music Computing Conference (SMC) and 40th International Computer Music Conference (ICMC-SMC 2014) (pp. 1002–1009). Athens, Greece. Retrieved from http://speech.di.uoa.gr/ICMC-SMC-2014/images/VOL_2/1002.pdf

Colton

(2008, March). Creativity versus the perception of creativity in computational systems. In Proceedings of AAAI Symposium on Creative Systems (pp. 14–20). Stanford, CA.

Colton

Charnley

Pease

(2011, April). Computational creativity theory: The FACE and IDEA descriptive models. In Proceedings of the 2nd International Conference on Computational Creativity (pp. 90–95). Mexico City, Mexico.

Eppe

Confalonieri

Maclean

Kaliakatsos-Papakostas

Cambouropoulos

Schorlemmer

Codescu

Kühnberger

K.-U.

(2015). Computational invention of cadences and chord progressions by conceptual chord-blending. In International Joint Conference on Artificial Intelligence (IJCAI) 2015.

Fauconnier

Turner

(2003). The way we think: Conceptual blending and the mind’s hidden complexities (Rep. ed.). New York, NY: Basic Books.

10.

Goguen

J. A.

(2006). Mathematical models of cognitive space and time. In Andler

Ogawa

Okada

Watanabe

(Eds.), Reasoning and cognition: Interdisciplinary conference series on reasoning studies (Vol. 2, pp. 125–128). Tokyo, Japan: Keio University Press.

11.

Granroth-Wilding

Steedman

(2014). A robust parser-interpreter for jazz chord sequences. Journal of New Music Research, 43, 355–374.

12.

Jordanous

(2012). Evaluating computational creativity: A standardised procedure for evaluating creative systems and its application (Unpublished doctoral dissertation). University of Sussex, Brighton, UK.

13.

Jordanous

(2016). Four PPPPerspectives on computational creativity in theory and in practice. Connection Science, 28, 194–216.

14.

Kaliakatsos-Papakostas

Cambouropoulos

(2014, September). Probabilistic harmonization with fixed intermediate chord constraints. In Georgaki

Kouroupetroglou

(Eds.), Proceedings of the joint 11th Sound and Music Computing Conference (SMC) and 40th International Computer Music Conference (ICMC-SMC 2014) (pp. 1083–1090). Athens, Greece. Retrieved from http://speech.di.uoa.gr/ICMC-SMC-2014/images/VOL_2/1083.pdf

15.

Kaliakatsos-Papakostas

Makris

Tsougras

Cambouropoulos

(2016). Learning and creating novel harmonies in diverse musical idioms: An adaptive modular melodic harmonisation system. Journal of Creative Music Systems, 1(1). Retrieved from http://openmusiclibrary.org/article/982966

16.

Kaliakatsos-Papakostas

Queiroz

Tsougras

Cambouropoulos

(2017). Conceptual blending of harmonic spaces for creative melodic harmonisation. In press.

17.

Kaliakatsos-Papakostas

Zacharakis

Tsougras

Cambouropoulos

(2015, October). Evaluating the general chord type representation in tonal music and organising GCT chord labels in functional chord categories. In Müller

Wiering

(Eds.), Proceedings of the International Society of Music Information Retrieval Conference (ISMIR 2015) (pp. 427– 433). Malaga, Spain.

18.

Koops

H. V.

Magalhães

J. P.

de Haas

W. B.

(2013). A functional approach to automatic melody harmonisation. Proceedings of the First ACM SIGPLAN Workshop on Functional Art, Music, Modelling and Design (pp 47–58). New York, NY: ACM.

19.

MacKinnon

D. W.

(1970). Creativity: A multi-faceted phenomenon. In Roslansky

J. D.

(Ed.), Creativity: A discussion at the nobel conference (pp. 17–32). Amsterdam, the Netherlands: North-Holland Publishing Company.

20.

Margulis

E. H.

Beatty

A. P.

(2008). Musical style, psychoaesthetics, and prospects for entropy as an analytic tool. Computer Music Journal, 32, 64–78.

21.

Makris

Kaliakatsos-Papakostas

Cambouropoulos

(2015). A probabilistic approach to determining bass voice leading in melodic harmonisation. Proceedings of the 5th International Conference on Mathematics and Computation in Music (MCM 2015). Berlin, Germany: Springer.

22.

Mayer

R. E.

(1999). Fifty years of creativity research. In Sternberg

R. J.

(Ed.), Handbook of creativity (pp. 449–460). Cambridge, UK: Cambridge University Press.

23.

Moors

J. J. A.

(1986). The meaning of kurtosis: Darlington reexamined. The American Statistician, 40, 283–284.

24.

Pearce

Wiggins

(2001, March). Towards a framework for the evaluation of machine compositions. Proceedings of the AISB’01 Symposium on Artificial Intelligence and Creativity in the Arts and Sciences (pp. 22–32). York, UK.

25.

Ritchie

(2007). Some empirical criteria for attributing creativity to a computer program. Minds and Machines, 17, 67–99.

26.

Rhodes

(1961). An analysis of creativity. Phi Delta Kappan, 42, 305–310.

27.

Rohrmeier

(2011). Towards a generative syntax of tonal harmony. Journal of Mathematics and Music, 5, 35–53.

28.

Schorlemmer

Smaill

Kühnberger

K.-U.

Kutz

Colton

Cambouropoulos

Pease

(2014, June). COINVENT: Towards a computational concept invention theory. Paper presented at the Fifth International Conference on Computational Creativity (ICCC), Lubljana, Slovenia. Retrieved from http://computationalcreativity.net/iccc2014/wp-content/uploads/2014/06//14.1_Schorlemmer.pdf

29.

Wiggins

G. A.

(2006). A preliminary framework for description, analysis and comparison of creative systems. Knowledge-Based Systems, 19, 449–458.

30.

Zacharakis

Kaliakatsos-Papakostas

Cambouropoulos

(2015). Conceptual blending in music cadences: A formal model and subjective evaluation. In Müller

Wiering

(Eds.), Proceedings of the International Society of Music Information Retrieval Conference (ISMIR 2015) (pp. 141–147). Malaga, Spain.

31.

Zacharakis

Kaliakatsos-Papakostas

Tsougras

Cambouropoulos

(in press). Creating musical cadences via conceptual blending: Empirical evaluation and enhancement of a formal model. Music Perception.

32.

Zbikowski

L. M.

(2002). Conceptualizing music: Cognitive structure, theory, and analysis. Oxford, UK: Oxford University Press.

33.

Zbikowski

L. M.

(2006). The cognitive tango. In Turner

(Ed.), The artful mind: Cognitive science and the riddle of human creativity (pp. 115–137). Oxford, UK: Oxford University Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.42 MB