Processing Abstract Language Modulates Motor System Activity

Abstract

Embodiment theory proposes that neural systems for perception and action are also engaged during language comprehension. Previous neuroimaging and neurophysiological studies have only been able to demonstrate modulation of action systems during comprehension of concrete language. We provide neurophysiological evidence for modulation of motor system activity during the comprehension of both concrete and abstract language. In Experiment 1, when the described direction of object transfer or information transfer (e.g., away from the reader to another) matched the literal direction of a hand movement used to make a response, speed of responding was faster than when the two directions mismatched (an action–sentence compatibility effect). In Experiment 2, we used single-pulse transcranial magnetic stimulation to study changes in the corticospinal motor pathways to hand muscles while reading the same sentences. Relative to sentences that do not describe transfer, there is greater modulation of activity in the hand muscles when reading sentences describing transfer of both concrete objects and abstract information. These findings are discussed in relation to the human mirror neuron system.

Keywords

Language Embodiment Abstraction Mirror neurons TMS

Embodied approaches to language (e.g., Barsalou, 1999; Glenberg & Robertson, 1999; Lakoff, 1987; Zwaan, 2004) propose that language comprehension makes use of neural systems ordinarily used for perception, action, and emotion. Empirical work from the behavioural and neuroscience communities has strongly supported embodied accounts of the comprehension of language about concrete situations (for reviews, see Glenberg, 2007; Pulvermüller, in press). For example, using brain imaging techniques, it has been shown that during processing of language material with content related to different effectors, effector-specific sectors of the premotor and motor areas become active (Hauk, Johnsrude, & Pulvermüller, 2004; Tettamanti et al., 2005). Similarly, behavioural and neurophysiological studies have shown a modulation of motor responses related to the content of the language material (Boulenger et al., 2006; Buccino et al., 2005). The neurophysiological basis for this modulation of the motor system is most likely related to the properties of a set of neurons, the so-called mirror neurons, first discovered in the monkey premotor cortex (DiPellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992). These neurons discharge when the animal performs an object-related action with the hand or the mouth and when it observes the same or a similar action done by another individual. More recently, it has been shown, also in humans, that the observation of actions done with different effectors (hand, foot, mouth) recruits the same motor representations active during the actual execution of those same actions (Bucinno et al., 2001). These findings strongly support the existence of mirror neurons in the human motor system and have led to the notion of a mirror neuron system involving areas in the frontal and parietal lobes. Even more interesting for the present study is the fact that the mirror neuron system can be activated by the typical sound of an action and even when actions are described verbally (for reviews, see Buccino, Binkofski, & Riggio, 2004; Buccino, Solodkin, & Small, 2006; Rizzolatti & Craighero, 2004).

As reviewed next, there is also a growing literature demonstrating that the comprehension of language about abstract ideas may be embodied. The research reported here, however, is the first neurophysiological study to demonstrate the embodied understanding of abstract language. We briefly review three approaches to the embodiment of abstract language and some of the evidence consistent with each. We then report two experiments, one behavioural and one using transcranial magnetic stimulation (TMS), which converge on the conclusion that processing some abstract language modulates activity in the motor system.

Three approaches to the embodiment of abstract language

The first approach, strongly associated with Lakoff (1987) and Gibbs (e.g., Gibbs & Steen, 1999), is based on metaphor. Lakoff proposes that bodily structure induces a consistent structure to experiences, and those experiences become represented as image schemas. For example, interactions with containers induce an image schema, and the schema is structured to represent the knowledge that things can be inside the container or outside the container, but not both at the same time. This image schema can then serve to ground abstract ideas that have a similar structure, such as the logical proposition “p or not-p, but not both”. Or, experience with journeys induces an image schema structured to have a beginning, a middle, and an end. The journey image schema can then serve to ground abstract concepts such as a relationship. Primary evidence consistent with this approach comes from an analysis of language use. For example, when people describe a love affair as “just beginning”, or “smooth sailing”, or “coming to an end”, it is presumed that this language reflects the use of the journey schema to understand the idea of a love affair.

Closely related to the Lakoff approach is the metaphorical use of space to represent abstract concepts. For example, Boroditsky and Ramscar (2002) review evidence that people understand time as movement in space. Richardson, Spivey, Barsalou, and McRae (2003) noted that people will consistently choose a vertically oriented image (e.g., of a block above a circle) as consistent with some verbs (e.g., “respect”) and choose a horizontally oriented image for other verbs (e.g., “argue”). To test the claim that these choices reflect processes important during language comprehension, participants read sentences such as “The man respects his father” followed by a target shape presented on the vertical or horizontal axis. As might be expected if people were simulating events along the vertical or horizontal axis, the time to detect the shape depended on the relation between the axis of presentation and the presumed axis of simulation.

A second approach to the embodiment of abstract knowledge was developed by Barsalou (1999; Barsalou & Wiemer-Hastings, 2005). On this approach, at least some abstract concepts arise from simulation processes. For example, to verify the truth of a proposition such as “The cup is on the table”, one creates an image of what would be expected if the proposition were true, and the image is compared to the perceptual situation. If the two correspond in relevant ways, then it is concluded that the proposition is true. Multiple experiences of the set of processes (image creation, comparison) become the grounding for the concept of truth. Evidence consistent with this approach comes from property listing experiments. For example, people should focus on processes or events more when considering the meaning of abstract concepts than when considering the meaning of concrete concepts. Barsalou and Wiemer-Hastings (2005) report just such a difference between properties listed for abstract concepts such as “truth” and “freedom” and properties listed for concrete concepts such as “car” and “bird”.

A third approach relates abstract events to actions. For example, the indexical hypothesis (Glenberg & Robertson, 1999) asserts that sentences are understood by creating a simulation of the actions that underlie them. Glenberg and Kaschak (2002) tested this proposal in a task in which participants judged the sensibility of sentences describing the transfer of concrete objects such as “Andy delivered the pizza to you/You delivered the pizza to Andy” and abstract information, such as “Liz told you the story/You told Liz the story”. As in these examples, half of the sensible sentences described transfer toward the reader, and half described transfer away. Participants responded using a three-button box held in the lap so that the buttons were aligned on the front/back axis. Participants viewed a sentence by holding down the middle button with the preferred hand. In one condition, the “sensible” response was made by moving the preferred hand to the far button, thus requiring a movement consistent with a simulation of transfer to another person. In the other condition, the “sensible” response was made by pressing the near button, thus requiring a movement consistent with transfer from another person to the reader. Consistent with the indexical hypothesis, there was an interaction in the time needed to judge the sentences: Judgements were faster when the action implied by the sentence matched the action required to make the response, and this was true for both the concrete and the abstract transfer sentences. Glenberg and Kaschak refer to this sort of interaction as an action–sentence compatibility effect, or ACE. De Vega (in press) has reported an ACE-type of interaction in understanding counterfactual sentences such as, “If the jeweller had been a good friend of mine he would had shown me the imperial diamond”. Note that the sentence is abstract in that the precondition does not exist (i.e., the jeweller is not a good friend) nor did the event occur (i.e., the jeweller did not show the diamond).

The action approach can be subdivided. As noted above, comprehension of transfer might involve a simulation process in which the motor system is used to simulate the specific actions used to accomplish the transfer. For example, one might simulate moving the arm in understanding, “You give the pizza to Andy”. But, simulation of this sort does not provide a convincing explanation for the ACE interaction found for abstract transfer as in, “You tell Liz the story”. For this sentence, simulation would involve the neural substrates for moving the lips and other speech articulators, not the arm and hand, and hence it is difficult to understand why this sort of simulation would affect time to move the hand to a response button.

We refer to the second action approach as an action schema approach (see also the discussion of motor resonance in Fischer & Zwaan, 2008 this issue, and Zwaan & Taylor, 2006). In this approach, the linguistic material is grounded in motor processes, but not necessarily by direct simulation. For example, consider Glenberg and Kaschak's (2002) hypothesis for how motor processes come to be used in understanding abstract transfer. Initially, children learn the linguistic encoding of transfer actions almost exclusively with the verb “to give” (Goldberg, Casenhiser, & Sethuraman, 2004; Tomasello, 2000). In this context, the meaning of transfer is encoded as an action having as parameters a type of grasp (e.g., power grip), force related to the object being transferred, and a direction of movement specified by the location of the object (e.g., the self) and location of the recipient (e.g., the mother). Repetition of actions of this sort leads to development of an action schema in anterior portions of premotor cortex (Aziz-Zadeh, Wilson, Rizzolatti, & Iacoboni, 2006), which becomes the meaning of the verb “to give”. Furthermore, this schema is grammaticized (in English, at least) as the double object construction (Goldberg, 1995) consisting of a subject (source of transfer), verb (means of transfer), recipient, and object being transferred. Examples of the construction are “Marco gives you the papers” and “You give Art the money”.

The basic action schema can be associated with other verbs of transfer by generalizing the grasp and force (i.e., the means of transfer) parameters. Thus, “Marco hands/throws/sends the papers” can all be understood using the same schema with a change in the specification of the means of transfer. The action schema is generalized for abstract transfer by using communication as the means of transfer. This step may well be facilitated because Broca's area participates both in control of the hand and in control of the speech articulators (Fadiga, Craighero, & Roy, 2006). Thus, an action schema developed for control of the arm and hand during transfer is, through successive generalization of action parameters, also used for (a) the understanding of observations of transfer using mirror neurons, (b) the understanding of concrete language about transfer of objects, and (c) the understanding of abstract language about the transfer of information.

In control of literal transfer, a transfer schema would require specification of what is being transferred (to control orientation toward the object and aspects of kinematics such as grip type and force) and the recipient of the object (to set additional kinematic parameters). We propose that the action schema is used in understanding language in an analogous manner. Individual words (e.g., “give”) and phrases (e.g., “delegate responsibility”) activate the action schema. The action schema then supplies the expectation that there will be a specification of the recipient and the object. These expectations may be satisfied by words or through gesture (e.g., pointing to the object or the location). Once the object and recipient are identified, the action schema provides an organization and coherent interpretation for the whole event. That is, the action schema is used to ground the meaning of both concrete and abstract transfer in the motor system.

The notion that the same action schema is used for understanding both concrete and abstract transfer is consistent with analyses of various signed languages. For example, in al-Sayyid Bedouin Sign Language (Sandler, Meir, Padden, & Aronoff, 2005) both concrete (e.g., give, throw) and abstract (e.g., inform, announce) verbs of transfer involve a hand motion that begins near the body and moves outward. The use of arm movement to represent both concrete and abstract transfer is also typical of American Sign Language and several other signed languages (K. Emmorey, personal communication, September 16, 2006).

If such an action schema is utilized during comprehension of both concrete and abstract transfer, then we would expect an ACE interaction for both. Furthermore, when using TMS to measure muscle activity in the hand (as in Experiment 2), we would expect to find activity for both the concrete sentences (consistent with both the simulation and action schema view) and the abstract sentences (consistent with the action schema view).

The various approaches to the embodiment of abstract language are not mutually exclusive; in fact they may all be emphasizing different aspects of the same phenomenon. Thus, simulation of action is the simulation of an event, as proposed by Barsalou and Wiemer-Hastings (2005), and events generally are extended in time and space, as related to Boroditsky and Ramscar (2002) and Richardson et al. (2003). Furthermore, to the extent that transfer of information is understood much as physical transfer (Glenberg & Kaschak, 2002), that understanding is metaphorical, as suggested by Lakoff (1987).

These theoretical ideas and empirical findings are consistent in demonstrating how abstract language may be grounded in bodily experiences of perception and action. However, given the absence of neuroimaging or neurophysiological evidence for the involvement of the sensorimotor system during abstract language comprehension, important questions remain. For example, it may be the case that the language is understood using neural systems unrelated to perception and action, and only later (e.g., in anticipation of action) are these systems activated (e.g., Mahon & Caramazza, 2005).

The present study had two aims. The first aim was to assess in two experiments, one using a behavioural paradigm and one using TMS, whether the motor system is modulated during the processing of language material expressing transfer of concrete objects. The second aim was to assess whether the motor system is also modulated during the processing of language expressing abstract transfer (e.g., “Anna delegates the responsibilities to you”). In general, the data are consistent with the action schema approach outlined above. We relate this finding to the mirror neuron system, which has already been implicated in the processing of concrete language (e.g., Buccino et al., 2005).

Experiment 1

The purpose of this behavioural experiment was to replicate and modestly extend the results of Glenberg and Kaschak (2002), as well as to test the linguistic materials for inclusion in Experiment 2. The stimulus materials were written in colloquial Italian. Participants judged as sensible or nonsense sentences describing transfer toward the reader (e.g., “Andrea ti porta la pizza”/”Andrea carries the pizza to you”), away from the reader (e.g., “Tu porti la pizza ad Andrea”/”You carry the pizza to Andrea”), or no-transfer sentences using the same character names and direct objects (e.g., “Tu annusi la pizza con Andrea”/”You smell the pizza with Andrea”).¹ Within each of these categories, half the sentences referred to a concrete object, and half referred to an abstract object or information, such as “Arturo ti presenta l'argomento”/”Arthur presents the argument to you”. Other examples are presented in Table 1. As in Glenberg and Kaschak (2002), for half of the sentences, the “sensible” response was made by moving to a response button situated farther away from the participant than the start button, and for half of the sentences, the “sensible” response was made by moving to a response button closer than the start button. We expected an ACE interaction for both the concrete and the abstract sentences. Such a finding would validate the stimuli for Experiment 2.

It is worth underlining that in Italian the sentence “Andrea ti porta la pizza”, referred to as toward transfer (see Table 1), may be expressed also as “Andrea porta la pizza a te” where the personal pronoun is postponed to the object. Although this last construction is more parallel to the sentence “Tu porti la pizza ad Andrea” used as away transfer, we preferred the first one because it is more common in colloquial Italian.

Table 1.

Example of stimuli used in Experiments 1 and 2

Sentence type	Sentence direction	Stimuli	Approximate English translations
Triad of concrete sentences	Toward	Marco/ ti/ da’ /le carte.	Marco gives you the papers.
	Away	Tu/ dai /le carte/ a Marco.	You give the papers to Marco.
	No transfer	Tu/ leggi/ le carte/ con Marco.	You read the papers with Marco.
Triad of abstract sentences	Toward	Anna/ ti/ delega/ le responsabilità.	Anna delegates the responsibilities to you.
	Away	Tu/ deleghi/ le responsabilità/ ad Anna.	You delegate the responsibilities to Anna.
	No transfer	Tu/ discuti/ delle responsabilità/ con Anna.	You discuss the responsibilities with Anna.
Nonsense sentences		Giovanni/ canta/ le carte/ con te.	Giovanni sings the papers with you.
		Daniele/ ti/ pulisce/ la responsabilità.	Daniele cleans the responsibilities to you.

Note: The slashes indicate the segments used in Experiment 2.

Method

Participants

A total of 22 native Italian speakers participated in the experiment. All were right-handed according to a standard handedness inventory (Oldfield, 1971), and all had normal or corrected-to-normal vision. Half of the participants were randomly assigned to begin the experiment using the far key to indicate sensible and the near key to indicate nonsense, and the other participants received the reverse assignment. Midway through the experiment, the response assignments were reversed.

Materials

A total of 40 triads of sensible sentences were created. Each triad consisted of a toward sentence, an away sentence, and a no-transfer sentence in which the verb did not express any transfer action. Half of the triads described concrete objects, and half described abstract events such as the transfer of information. Half of the no-transfer sentences began with “Tu”, and half began with a proper noun (as did the away and toward sentences, respectively). In addition, 120 nonsense sentences were created. Of these, half mentioned concrete objects and half abstract objects or events, and an equal number used the syntax of toward, away, and no-transfer sentences.

A different randomization of the 240 stimulus sentences was created for each participant within the following constraints. Each block of 24 sentences contained an equal number of sensible and nonsense sentences and, orthogonally, an equal number of toward, away, and no-transfer sentences and an equal number of concrete and abstract sentences. The toward and away sentences in a block never came from the same triads.

Participants responded using a modified keyboard. The keyboard was oriented so that the long axis projected away from the participant. The “h” key was designated as the start key. The stimulus sentence was visible only while the start key was depressed. The keys symmetric to the “h” but five removed (e.g., “a” on a standard American keyboard) were modified by attaching cardboard squares, approximately two centimetres on each side, to facilitate movement to the keys. The squares were labelled “Yes” (i.e., sensible) and “No” (i.e., nonsense).

In summary, the independent variables were sentence direction (toward, away, or no-transfer), concreteness (concrete and abstract), sensibility (sensible and nonsense), and response direction (yes-is-near and yes-is-far). All of the variables were manipulated within subjects.

Procedure

Participants were instructed to consider the sentences as about themselves. In addition, they were told to read quickly because the computer would record an error if a response was not made within three seconds from the time at which the stimulus sentence was presented. Before beginning the first block of trials, participants received two sets of practice. The first practice set consisted of 20 trials in which the participants (a) depressed the start key using the index finger of the preferred hand, (b) observed the word “Yes” or “No” on the computer screen while the start key was depressed and moved the index finger to the corresponding “Yes” and “No” keys as quickly and accurately as possible. Errors were indicated by a feedback tone. The second practice set consisted of 10 trials in which the participants (a) depressed the start key, (b) read a sentence, and (c) responded “Yes” or “No” as quickly as possible. After these practice trials, the participant received the first five blocks of trials. Following the fifth block, the response keys were reversed, and the participant engaged in the two practice sets using the new response assignment. This practice was followed by the remaining five blocks of trials. The computer registered the amount of time the start key was depressed (judgement time—that is, the time to read, comprehend, and judge the sensibility of the sentence), the time between lifting the finger and moving to the “Yes” or “No” keys (movement time), and the identity of the key depressed.

Results

The data from 2 participants were eliminated because they made more than 10% errors in judging the sensible sentences. For the 20 remaining participants, the judgement time data for the sensible sentences were analysed as follows (the movement times were not analysed). First, considering all 4,800 responses (240 trials by 20 participants), the longest 1% and shortest 1% were eliminated. Second, Trials 1–12 and 121–132 (the first 12 trials with each response assignment) were considered additional practice and were eliminated. Third, errors were eliminated. Fourth, for each participant in each of the six conditions defined by the two levels of concreteness and the three levels of sentence direction, a mean and standard deviation were computed, and any judgement time more than 2.5 standard deviations from the mean was eliminated. In total, 4,238 trials were analysed to obtain error rates, and after eliminating errors and data from the nonsensical sentences, 2,078 trials were used to determine the judgement times. Analyses focused on the data from the sensible sentences in the toward and away conditions, and they were conducted with a Type I error rate set to .05.

In an analysis of the error rates for the sensible transfer sentences, there was a significant effect of concreteness. The concrete sentences were responded to correctly on 97% of the trials, and the abstract sentences were responded to correctly on 99% of the trials, F(1, 19) = 4.39, MSE = 0.002, p = .05. No other main effects or interactions were significant.

To focus on the possible ACE interaction, we eliminated the data from the no-transfer sentences. There was an ACE interaction of sentence direction and response direction, as illustrated in Figure 1, F(1, 19) = 5.02, MSE = 3,222. The judgement time for toward sentences, relative to away sentences, was shorter when the response required movement toward the participant than when the response required movement away from the participant. The size of the ACE interaction can be quantified as the average difference between incongruent and congruent sentence direction by response direction pairings. Using this measure, the ACE for concrete sentences (31.63 ms) was numerically smaller than that for the abstract sentences (48.77 ms), but the three-factor interaction of concreteness by sentence direction by response direction was not statistically significant, F < 1.²

Separate analyses were conducted on the concrete and abstract sentences (although with reduced power). For the concrete sentences, the ACE interaction was not significant, F(1, 19) = 1.54, MSE = 3,253. However, there was an ACE interaction in the error rates, F(1, 19) = 4.14, MSE = 0.002 (4% errors when sentence direction mismatched response direction and 2% error when they matched). For the abstract sentences, the ACE interaction approached significance, F(1, 19) = 2.45, MSE = 4,853, p = .13, and there was no interaction in the analysis of error rates, F < 1.

Figure 1.

Judgement time data from Experiment 1. Error bars illustrate one standard error.

There were two other significant effects. First, there was a main effect of concreteness (M for concrete = 1,099 ms; M for abstract = 1,178 ms), F(1, 19) = 34.97, MSE = 7,055. Given that concrete sentences were read faster and had more errors than abstract sentences, there is the possibility of a speed–accuracy trade-off. This possibility does not appear to compromise any of the conclusions based on the ACE interaction, however. We think that it is likely that much of the effect of concreteness can be attributed to differences in word frequency—that is, the lower frequency words used in the abstract sentences were read slowly.³ Second, there was an interaction of concreteness and sentence direction, F(1, 19) = 10.04, MSE = 3,414 (the corresponding interaction in the analysis of errors was not significant, F < 1). The concrete toward sentences were read faster than the concrete away sentences, but no difference was apparent for the abstract toward and away sentences.³

Based on the Laudanna, Thornton, Brown, Burani, and Marconi (1995) database, the mean frequencies for the verbs are 124 and 27 for the sensible concrete transfer sentences and sensible abstract transfer sentences, respectively. The difference between the means should be treated with some caution, however, because it is greatly affected by the very high frequency of the Italian verb “dare” (“to give”).

Discussion

The results are a conceptual replication of the ACE found by Glenberg and Kaschak (2002). That is, when judging a sentence describing action toward the reader, judgements are faster when the response is made by literally moving the hand toward the reader, whereas when judging a sentence describing action away from the reader, judgements are faster when the response is made by literally moving the hand away from the reader. Importantly, the size of the ACE interaction was not statistically affected by the concreteness factor.

The results are consistent with the action schema account that sentence comprehension requires the use of an action schema representing transfer. Combining the action schema and simulation variants of the indexical hypothesis may also explain the finding (both here and in Glenberg & Kaschak, 2002) that the ACE interaction is slightly (but not significantly) larger for the abstract sentences.⁴ For both concrete and abstract sentences, a transfer action schema is contacted. In addition, there may be a motor simulation, particularly for the concrete sentences. For some of the concrete sentences, however, the motor simulation will mismatch action parameters used in responding. For example, for the sentence “You give the pizza to Andy” might involve a motor simulation with the hand palm up. However, the response requires that the hand be palm down. This type of interference would be rare for the abstract sentences: As noted before, any simulation for abstract sentences would be more likely to involve the speech articulators than the arm and hand.

We thank Rolf Zwaan for suggesting this account.

Nonetheless, at least one alternative explanation is viable. Perhaps sentence comprehension does not require any action simulation. However, after a sentence is understood, the meaning is used to prepare for action, and it is at this point that the ACE interaction arises. We were able to test this alternative in the following experiment.

Experiment 2

Single-pulse TMS can be used to index activity in the motor system. The TMS pulse evokes a motor response (motor evoked potential, MEP), and the question of interest is whether that MEP is differentially modulated by a simultaneously presented nonmotor stimulus. For example, Fadiga, Craighero, Buccino, and Rizzolatti (2002) applied a TMS pulse over the motor area controlling tongue muscles while participants listened to Italian words and pseudowords containing a tongue-trilled, double-r sound or a nontrilled, double-f sound. They found that MEPs recorded from tongue muscles were significantly larger when listening to double-r stimuli than double-f stimuli. Thus, listening to linguistic stimuli resulted in the phoneme-specific activation of speech motor centres. Using a similar methodology, Buccino et al. (2005) presented acoustically sentences describing hand action (“he turned the key”), leg action (“he kicked the ball”), or no concrete action (“he loved his wife”). The TMS pulse was delivered during presentation of the verb. MEPs measured at the hand and foot were modulated by sentence content, leading to the conclusion that sentence content differentially modulates the motor regions where the effectors implied by the sentence are represented.

In Experiment 2, we used single-pulse TMS to study changes in the corticospinal motor pathways to hand muscles during the presentation of concrete and abstract sentences. The pulse was applied either shortly after presentation of the verb or shortly after the last segment of the sentence was presented. MEPs were recorded from the opponens pollicis (OP) muscle of the hand. The sentences were the concrete and abstract transfer sentences (both toward and away) and the no-transfer sentences from Experiment 1. If sentence comprehension invokes a motor simulation or action schema, we expect to see greater motor system modulation (e.g., a larger MEP) for the transfer sentences than the no-transfer sentence. Furthermore, if language understanding modulates motor activity during comprehension, then we would expect to see a difference between transfer and no-transfer sentences when the pulse occurs shortly after the verb—that is, while the participant is in the process of understanding the sentence. In contrast, if the difference is found only at the end of the sentence, it might be due to some form of motor imagery invoked after sentence comprehension. Finally, the questions of major interest concern the variable of sentence concreteness. Will sentences describing the transfer of abstract information invoke MEPs at the hand? Will the effect be equivalent to that found for sentences describing the transfer of concrete objects?

Note that our predictions are in terms of modulation of motor system activity. The direction of modulation (possibly related to the task used and the time at which the response is required) is difficult to predict in behavioural experiments (Glenberg & Kaschak, 2002; Kaschak et al., 2005), TMS experiments (Buccino et al., 2005, Fadiga et al., 2002) and kinematics experiments (Boulenger et al., 2006; Gentilucci, Bernardis, Crisi, & Dalla Volta, 2006).

Method

Participants

A total of 11 participants (2 males and 9 females who had not participated in Experiment 1; mean age ± SD, 22 ± 3 years) participated in the experiment. All were right-handed, according to a standard handedness inventory (Oldfield, 1971) and had normal or corrected-to-normal vision. Participants were screened for neurological, psychiatric, and other medical problems and contraindications to TMS (Wassermann, 1998). Informed consent was obtained for all participants, and they were paid for their participation. The protocol was approved by the Parma University Ethical Committee and was carried out in accordance with the ethical standards of the 1964 Declaration of Helsinki.

Stimuli

The sentence stimuli were from those used in Experiment 1. We used 80 of the nonsense sentences, 40 no-transfer sentences (half concrete and half abstract), and half of the toward and away sentences for a total of 40 transfer sentences (half concrete and half abstract). We collapsed across the toward/away distinction for several reasons. Because it is difficult to record reliable MEPs from proximal muscles, such as the biceps and triceps which are generally related to toward and away arm actions, we recorded MEPs from a distal muscle, OP. This muscle is involved in grasping action, and thus it may be considered a component of both toward and away transfer actions. Moreover, by collapsing across the distinction between toward and away, we could reduce the number of TMS pulses.

A different randomization of the 160 stimulus sentences was created for each participant within the following constraints. Each block of 32 sentences contained an equal number of sensible and nonsense sentences and, orthogonally, equal numbers of transfer (half toward and half away) and no-transfer sentences, equal numbers of concrete and abstract sentences, and equal numbers of TMS pulses on the verb and at the end of the sentence. The toward and away sentences in a block never came from the same triads.

For purposes of sentence presentation, each stimulus sentence was divided into four segments as illustrated in Table 1. The segments were presented using a modified moving-window technique (cf. Just, Carpenter, & Woolley, 1982). First, a series of dashes and spaces was presented. Each space corresponded to a space between words, and the number of dashes corresponded to the number of letters in each word. Next, the first segment was presented for 250 ms, and it was followed by each successive segment also presented for 250 ms each. Unlike in the standard moving-window technique, segments remained on the screen after they were presented. Although the sentences differed in number of words (five or six), each sentence contained four segments, so the total time of presentation was the same.

Participants indicated that the sentence was sensible by pressing the “j” key on a keyboard using the left index finger, and they indicated that the sentence was nonsense by pressing the “g” key with the left middle finger.

Electromyography

Continuous electromyography (EMG) recordings from the right-hand OP muscle were acquired with a CED Micro 1401 analogue-to-digital converting unit (Cambridge Electronic Design, Cambridge, UK). The EMG signal was amplified (1,000 times) and digitized (sampling rate: 5 kHz, band-pass filter: 5–4000 Hz) and stored on a computer for offline analysis. Ag/AgCl surface electrodes with a bipolar montage were used. The active electrode was placed on the muscle belly and the reference electrode on the corresponding tendon.

Transcranial magnetic stimulation

The left hemisphere was magnetically stimulated by means of monophasic single pulses delivered through a figure-of-eight coil connected to a transcranial magnetic stimulator (ESAOTE Biomedica, Italy). The coil was moved over the scalp in order to determine the optimal site from which maximal-amplitude MEPs were elicited in the OP muscle. For optimal stimulation of the hand motor cortex, the intersection of the coil was placed tangentially to the scalp with the handle pointing backward and laterally at a 45° angle away from the midline (Mills, Boniface, & Schubert, 1992). The coil handle was fixed to a mechanical arm to suppress movements of the coil itself from the original position on the scalp. The resting motor threshold of the OP muscle was determined according to standard methods as the minimal intensity capable of evoking MEPs (Rossini et al., 1994) in 5 out of 10 consecutive trials from the relaxed muscle with an amplitude of at least 50 µV. The output of the stimulator was set to 120% of the resting motor threshold for the stimulations applied during the experimental session.

Procedure

The experiment was programmed using Matlab (The Mathworks Inc., Natick, MA), Cogent (Functional Imaging Laboratory, Queen Square, London, UK), and Signal (Cambridge Electronic Design, Cambridge, UK) software to control the stimulus presentation and to trigger the TMS and EMG recordings.

Participants were comfortably seated on an armchair with the right elbow flexed at 90° and the right hand half-pronated in a relaxed position. The participant's head was supported by a headrest to maintain a comfortable and stable position. The left hand was positioned on a keyboard oriented to facilitate comfortable responding with the index and middle fingers.

For each sentence, the TMS pulse was delivered either 200 ms after the onset of the verb or 200 ms after the onset of the last segment. The delay of 200 ms was chosen in light of previous studies indicating that lexical and semantic processes during word recognition entail activation in the frontal cortex as early as 150–200 ms after onset of written word stimuli (e.g., Hauk, Davis, Ford, Pulvermüller, & Marslen-Wilson, 2006; Pulvermüller, Shtyrov, & Ilmoniemi, 2005).

The experimental session began with the determination of magnetic coil location and pulse strength. Next, the participant received 20 practice trials. These trials were followed by the 160 experimental trials.

Results

The data from 2 participants were eliminated because technical problems resulted in incomplete data files. For the remaining participants, data were processed offline. The mean percentage of errors was 5.42% (SD = 2.5). No participants exceeded the limit of 10% of errors. Error trials and trials with EMG activity before TMS (less than 5%) were discarded from the analysis. For each participant and each trial, the EMG trace was rectified, and the area under the curve corresponding to the MEP was measured. Data were then z-score normalized to the grand average of all MEP areas (evoked by sensible sentences) within the same subject, and means were calculated for the eight conditions resulting from the factorial combination of pulse time (verb and end of sentence), concreteness (concrete and abstract), and sentence type (transfer and no-transfer). The means are illustrated in Figure 2. Note that zero on the ordinate corresponds to the grand average MEP, not the absence of activity. Analyses focused on the data from the sensible sentences. All analyses were conducted using a Type I error rate of .05.

Figure 2.

The z-scores of the motor evoked potentials (MEPs) from Experiment 2. Error bars illustrate one standard error.

The three-factor, within-subject analysis of variance produced a main effect of pulse time, F(1, 8) = 18.03, MSE = 0.12, in that the MEPs at the verb (M = 0.17) were larger than those at the end of the sentence (M = –0.18). In addition, there was a significant effect of sentence type, F(1, 8) = 8.02, MSE = 0.07, in that the MEPs to the transfer sentences (M = 0.08) were greater than the MEPs to the no-transfer sentences (M = –0.09). These effects are illustrated in Figure 2. There was little evidence for a difference between concrete and abstract sentences (F < 1) and little evidence that the size of the difference between transfer and no-transfer sentences depended on concreteness (F < 1 for the interaction).⁵ Similarly, there was little evidence that the size of the difference between transfer and no-transfer sentences differed for the pulse times (p = .15 for the interaction).

Separate analyses were conducted on the concrete and abstract sentences (although with reduced power). For the concrete sentences, the effect of sentence type approached significance, F(1, 8) = 3.90, MSE = 0.09, p = .08. For the abstract sentences, the effect of sentence type approached significance, F(1, 8) = 3.84, MSE = 0.05, p = .09.

Discussion

The data from Experiment 2 are consistent with several conclusions. First, the larger MEPs for the transfer sentences compared to the no-transfer sentences is consistent with the action schema view. That is, during language comprehension, motor areas of the brain are differentially activated depending on the content of the language. Second, the size of the sentence type effect is comparable for sentences that describe concrete and abstract transfer. Third, the size of the effect is comparable when the pulse is at the verb (during sentence comprehension) and when the pulse is at the end of the sentence. Apparently, the motor system is modulated during comprehension of both concrete and abstract language.

A reviewer of an earlier version of this article noted that when the pulse is at the verb, the reader may not yet know that the sentence will describe transfer. Consequently, it is difficult to understand why there should be any transfer-related activity in the OP muscle. We think that there is a relatively simple (but untested) explanation for this early activation. Many of the verbs used in the experiment (e.g., “to give”) are highly associated with transfer. Consequently, the putative action schema will be primed by the verb, and this priming would then lead to activity in the motor system. In fact, such an account is very close to what is meant by our claim that the motor system contributes to language comprehension: As described in the Introduction, words and phrases invoke the action schema, which is then used to help organize the components of the sentence into a coherent meaning.

General Discussion

The data from the two experiments converge on the conclusion that the motor system is modulated during the understanding of both concrete and abstract language comprehension (see also, Taylor & Zwaan, 2008 this issue, and Zwaan & Taylor, 2006). In Experiment 1, the direction of movement implied by the sentence (toward the reader or away from the reader) determined whether responding was easier by literally moving the hand toward the reader or away from the reader. The effect was observed on the time needed to read and understand the sentence. Nonetheless, the data from Experiment 1 can be questioned on several grounds. First, the response was measured only after the sentence was fully presented, and thus the finding might reflect a postcomprehension process such as preparing for action. Second, the effect was observed in a context requiring hand movement, and thus the effect might only be found when participants pay particular attention to hand movement. Both of these suggestions can be ruled out by the data from Experiment 2. First, the effect of transfer versus no-transfer was found at the verb—that is, during sentence comprehension, as well as at the end of the sentence. Second, there was no need for overt hand movements (other than left-hand finger movements to press the response key). Importantly, no difference in motor system modulation was found comparing the concrete (transfer of objects) and abstract (transfer of information) sentences. These data support the notion of a relatively general, transfer action schema in motor cortex, probably in the mirror neuron system. That is, the schema is used to control transfer, recognize acts of transfer, and understand language about transfer.

At first glance, our effects appear to be in contrast with some other studies. In Experiment 1, language consistent with overt action speeded that action, and in Experiment 2, language describing action increased MEPs. In contrast, Buccino et al. (2005), Boulenger et al. (2006), and Gentilucci et al. (2006) report that language appears to interfere with the motor system. However, these studies differ from the present studies in several respects. First, in Experiment 1, the response was made after the meaning of the sentence was understood. Boulenger et al. (2006) have demonstrated that interference effects are greatly reduced under these conditions. Also, the task of moving to near and far buttons may exert a different modulation on the motor system.

The difference between the findings of Buccino et al. (2005) and the TMS findings from Experiment 2 could relate to the difference in syntactic person. In Buccino et al., the sentences were in the third person (e.g., “He turned the key”) and never referred to the participant. In the current experiments, the sentences always referred to the participant as either agent or recipient of the action. In this regard, Schütz-Bosbach, Mancini, Aglioti, and Haggard (2006) report that observations of (what one interprets) as one's own movements (first person) and observations of (what one interprets) as another's movements (third person), have opposite effects on the mirror neuron system thus allowing for a differentiation between self and others. Thus, it might be that language about third-person events and language about first-person events have dissimilar effects on the mirror neuron system just as observation of self and others have dissimilar effects.

We have argued that the activity in motor cortex during language comprehension is not simply a postcomprehension process used to prepare for action, but then what is the function of that activity? One alternative is that the activity is epiphenomenal rather than intrinsic to language comprehension. For example, perhaps it is a vestigial habit based on early sensorimotor associations with language. This explanation cannot be ruled out by our data (although it would be ruled out if repetitive TMS, rTMS, were to selectively disrupt comprehension of transfer versus no-transfer sentences), but it seems unlikely for several reasons. First, the effects are found with the abstract transfer sentences that do not refer to hand or arm movement. Second, as reviewed earlier, data consistent with an embodied interpretation of abstract language can be found using a variety of procedures and a variety of types of abstractions. Thus the simplest explanation of the set of data is that embodied mechanisms, such as using an action schema, are an integral part of the comprehension process.

How is activity in motor cortex used in language comprehension? We answer this question in three steps. First, motor activity, and the mirror neuron system in particular, play an important role in producing and understanding observed actions. Second, the same mirror neuron system is implicated in the understanding of language about concrete actions. That is, we are suggesting that the action schema used for controlling action and comprehending language about action is implemented in the mirror neuron system. Third, understanding of abstract actions is derived from the understanding of concrete actions.

The mirror neuron system has been implicated in the understanding of language. Using functional magnetic resonance imaging (fMRI), Tettamanti et al. (2005) observed activation of the mirror neuron system when participants listened to sentences describing mouth, hand, and leg actions. Similarly, Aziz-Zadeh et al. (2006) demonstrated congruence in brain areas activated when participants either observed mouth, hand, or foot actions or read short descriptions of those same actions. Thus, observed actions are understood by engaging in motor simulation, and language about actions is understood by engaging in a substantially similar type of activity. An important fact supporting this interpretation is that a part of the human mirror neuron system is located in Broca's area, which has long been associated with language (e.g., Fadiga, Craighero, & Roy, 2006).

Demonstrations of the relation between the mirror neuron system and language understanding have used language about concrete actions, whereas we observed motor activity for both concrete and abstract transfer. Why should language about the transfer of information call upon a motor simulation involving the hand? As described in the Introduction, one possibility invokes a developmental scenario. Infants first represent concrete transfer events using the motor system. Experience with different sorts of transfer (e.g., handing, throwing) leads to an action schema generalized over action parameters such as source location, recipient location, object, and mode of transfer (i.e., kinematics). This action schema can then be used to ground other events with a similar set of parameters and a similar outcome structure—that is, something moves from the source location to the recipient location. Note that telling a story or delegating responsibilities also involves a source, a recipient, something that is transferred, and a mode of transfer. Hence the action schema can also ground the understanding of abstract transfer. Given that the action schema initially developed to control the arm and hand during literal, concrete transfer, when the action schema is contacted during the comprehension of abstract transfer sentences, areas of motor cortex controlling the hand also become active, as found in Experiment 2.

References

Aziz-Zadeh

Wilson

S. M.

Rizzolatti

Iacoboni

(2006). Embodied semantics and the premotor cortex: Congruent representations for visually presented actions and linguistic phrases describing actions. Current Biology 16, 1818–1823.

Barsalou

L. W.

(1999). Perceptual symbol systems. Behavioral and Brain Sciences 22, 577–660.

Barsalou

L. W.

Wiemer-Hastings

(2005). Situating abstract concepts. Grounding cognition: The role of perception and action in memory, language, and thought. Pecher

Zwaan

New York: Cambridge University Press 129–163.

Boroditsky

Ramscar

(2002). The roles of body and mind in abstract thought. Psychological Science 13, 185–188.

Boulenger

Roy

A. C.

Paulignan

Deprez

Jeannerod

Nazir

T. A.

(2006). Cross-talk between language processes and overt motor behavior in the first 200 ms of processing. Journal of Cognitive Neuroscience 18, 1606–1615.

Buccino

Binkofski

Fink

G. R.

Fadiga

Fogassi

Gallese

(2001). Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study. European Journal of Neuroscience 13, 400–404.

Buccino

Binkofski

Riggio

(2004). The mirror neuron system and action recognition. Brain and Language 89, 370–376.

Buccino

Riggio

Melli

Binkofski

Gallese

Rizzolatti

(2005). Listening to action-related sentences modulates the activity of the motor system: A combined TMS and behavioral study. Cognitive Brain Research 24, 355–363.

Buccino

Solodkin

Small

(2006). Functions of the mirror neuron system: Implications for neurorehabilitation. Cognitive Behavioral Neurology 19, 55–63.

10.

De Vega

(in press). Levels of embodied meaning: From pointing to counterfactuals. Symbol, embodiment, and meaning. de Vega

Glenberg

A. M.

Graesser

A. C.

Oxford, UK: Oxford University Press.

11.

DiPellegrino

Fadiga

Fogassi

Gallese

Rizzolatti

(1992). Understanding motor events: A neurophysiological study. Experimental Brain Research 91, 176–180.

12.

Fadiga

Craighero

Buccino

Rizzolatti

(2002). Speech listening specifically modulates the excitability of tongue muscles: A TMS study. European Journal of Neuroscience 15, 399–402.

13.

Fadiga

Craighero

Roy

(2006). Broca's region: A speech area?. Broca's region. Grodzinsky

Oxford, UK: Oxford University Press.

14.

Fischer

M. H.

Zwaan

R. A.

(2008). Embodied language: A review of the role of the motor system in language comprehension. Quarterly Journal of Experimental Psychology 61, 825–850.

15.

Gentilucci

Bernardis

Crisi

Dalla Volta

(2006). Repetitive transcranial magnetic stimulation of Broca's area affects verbal responses to gesture observation. Journal of Cognitive Neuroscience 18, 1059–1074.

16.

Gibbs

Steen

(1999). Metaphor in cognitive linguistics Amsterdam: John Benjamins.

17.

Glenberg

A. M.

(2007). Language and action: Creating sensible combinations of ideas. Oxford handbook of psycholinguistics. Gaskell

Oxford, UK: Oxford University Press 361–370.

18.

Glenberg

A. M.

Kaschak

M. P.

(2002). Grounding language in action. Psychonomic Bulletin & Review 9, 558–565.

19.

Glenberg

A. M.

Robertson

D. A.

(1999). Indexical understanding of instructions. Discourse Processes 28, 1–26.

20.

Goldberg

A. E.

(1995). Constructions: A construction grammar approach to argument structure Chicago: University of Chicago Press.

21.

Goldberg

A. E.

Casenhiser

D. M.

Sethuraman

(2004). Learning argument structure generalizations. Cognitive Linguistics 15, 289–316.

22.

Hauk

Davis

M. H.

Ford

Pulvermüller

Marslen-Wilson

W. D.

(2006). The time course of visual word recognition as revealed by linear regression analysis of ERP data. NeuroImage 30, 1383–1400.

23.

Hauk

Johnsrude

Pulvermüller

(2004). Somatotopic representation of action words in human motor and premotor cortex. Neuron 41, 301–307.

24.

Just

M. A.

Carpenter

P. A.

Woolley

J. D.

(1982). Paradigms and processes in reading comprehension. Journal of Experimental Psychology: General 111, 228–238.

25.

Kaschak

M. P.

Madden

C. J.

Therriault

D. J.

Yaxley

R. H.

Aveyard

Blanchard

(2005). Perception of motion affects language processing. Cognition 94, B79–B89.

26.

Lakoff

(1987). Women, fire, and dangerous things: What categories reveal about the mind Chicago: University of Chicago Press.

27.

Laudanna

Thornton

A. M.

Brown

Burani

Marconi

(1995). Un corpus dell'italiano scritto contemporaneo dalla parte del ricevente [A corpus of contemporary written Italian language from the recipient's side]. III Giornate internazionali di analisi statistica dei dati testuali. Bolasco

Lebart

Salem

Rome: Cisu I, 103–109[III International Meeting on Statistical Analysis of Textual Data].

28.

Mahon

B. Z.

Caramazza

(2005). The orchestration of sensory-motor systems: Clues from neuropsychology. Cognitive Neuropsychology 22, 480–494.

29.

Mills

K. R.

Boniface

S. J.

Schubert

(1992). Magnetic brain stimulation with a double coil: The importance of coil orientation. Electroencephalography and Clinical Neurophysiology 85, 17–21.

30.

Oldfield

R. C.

(1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia 9, 97–114.

31.

Pulvermüller

(in press). Grounding language in the brain. Symbol, embodiment, and meaning. de Vega

Glenberg

A. M.

Graesser

A. C.

Oxford, UK: Oxford University Press.

32.

Pulvermüller

Shtyrov

Ilmoniemi

(2005). Brain signatures of meaning access in action word recognition. Journal of Cognitive Neuroscience 17, 884–892.

33.

Richardson

D. C.

Spivey

M. J.

Barsalou

L. W.

McRae

(2003). Spatial representations activated during real-time comprehension of verbs. Cognitive Science 27, 767–780.

34.

Rizzolatti

Craighero

(2004). The mirror-neuron system. Annual Review of Neuroscience 27, 169–192.

35.

Rossini

P. M.

Barker

A. T.

Berardelli

Caramia

M. D.

Caruso

Cracco

R. Q.

(1994). Non-invasive electrical and magnetic stimulation of the brain, spinal cord and roots: Basic principles and procedures for routine clinical application. Electroencephalography and Clinical Neurophysiology 91, 79–92Report of an IFCN committee.

36.

Sandler

Meir

Padden

Aronoff

(2005). The emergence of grammar: Systematic structure in a new language. Proceedings of the National Academy of Sciences. 102, 2661–2665.

37.

Schütz-Bosbach

Mancini

Aglioti

S. M.

Haggard

(2006). Self and other in the human motor system. Current Biology 16, 1830–1834.

38.

Taylor

L. J.

Zwaan

R. A.

(2008). Motor resonance and linguistic focus. Quarterly Journal of Experimental Psychology 61, 896–904.

39.

Tettamanti

Buccino

Saccuman

M. C.

Gallese

Danna

Scifo

(2005). Listening to action-related sentences activates fronto-parietal motor circuits. Journal of Cognitive Neuroscience 17, 273–281.

40.

Tomasello

(2000). Do young children have adult syntactic competence?. Cognition 74, 209–253.

41.

Wassermann

E. M.

(1998). Risk and safety of repetitive transcranial magnetic stimulation: Report and suggested guidelines from the International Workshop on the Safety of Repetitive Transcranial Magnetic Stimulation, June 5–7, 1996. Electroencephalography and Clinical Neurophysiology 108, 1–16.

42.

Zwaan

R. A.

(2004). The immersed experiencer. Toward an embodied theory of language comprehension. The psychology of learning and motivation. Ross

San Diego: Academic Press 44, 35–62.

43.

Zwaan

R. A.

Taylor

L. J.

(2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology: General 135, 1–11.