Actions,Words,and Numbers

UNDERSTANDING ACTION WORDS

Words, especially verbs, are often used to describe one's own actions or to understand the actions performed by others. Following the influential work of Pulvermuller and colleagues (Hauk, Johnsrude, & Pulvermuller, 2004; Pulvermuller, Harle, & Hummel, 2001), several authors capitalized on the somatotopic organization of the motor cortex—that is, the maintenance of the same spatial organization between different body parts and their cortical representation—to investigate the relationship between the processing of action words and motor activity. For example, actions involving the lower and upper limbs are controlled, respectively, by the median and lateral parts of the primary motor cortex. Using functional magnetic resonance imaging (fMRI), Tettamanti et al. (2005) showed that, when subjects listened to sentences describing leg movements (e.g., “I kick the ball”), the activation of the motor circuits controlling the leg increased. Comparable results were found in the arm and face motor circuits when listening to action words involving, respectively, the upper limbs and face. Because no movements were permitted during word listening, the authors concluded that the motor cortex is directly involved in the understanding of action words. However, a precise measurement of the muscle activity would be necessary to strengthen this conclusion.

The use of transcranial magnetic stimulation (TMS) has brought further support to this conclusion. TMS is a noninvasive stimulation technique that consists of generating a brief, high-intensity magnetic field next to the scalp using an electrical current; this magnetic field induces an electric field that activates the neuronal elements in the underlying cortex. Buccino et al. (2005) asked healthy participants to listen to sentences related to foot or hand actions while TMS was applied over the cortical motor representation of the leg or of the hand. In this experiment, TMS was used to assess the influence of this cognitive task on the excitability of motor circuits by measuring the amplitude of the muscle twitch observed when TMS was delivered over the motor cortex. As predicted by fMRI results, changes in motor excitability were exclusively found in the muscles potentially involved in the action described by the sentence. In another study, Pulvermuller, Hauk, Nikulin, and Ilmoniemi (2005) examined whether the interference induced by TMS applied over the motor cortex influenced linguistic judgement. TMS was delivered over different zones of the motor cortex in healthy participants who had to judge whether a letter string corresponded to an existing word. When TMS targeted the leg area, words related to foot actions were recognised more rapidly than words related to hand actions; the opposite results were observed when TMS was delivered over the upper limb representation. These findings demonstrated a facilitatory effect of the TMS of the motor cortex on the processing of action words, further supporting the essential contribution of the motor system to semantic processing.

GRASPING AND MAGNITUDE PROCESSING

In everyday activities, humans are able to grasp precisely a wide variety of objects of different shape and size (Olivier, Davare, Andres, & Fadiga, 2007). During the transport of the hand toward an object, the grip aperture (i.e., the distance between the thumb and index finger) is known to be correlated with the size of the object to be grasped, revealing a transformation of the visual representation of the object into a magnitude estimate used by the motor system to program the appropriate hand posture. If the motor system is involved in the representation of semantic knowledge of actions, it is sensible to postulate that word processing could influence the programming of the grip aperture if words refer to a graspable object.

To investigate these possible interactions, Glover, Rosenbaum, Graham, and Dixon (2004) used a task in which participants had to reach and grasp a wooden block after reading the name of different graspable objects. The grip aperture was measured by means of a camera that recorded the position of infrared-light-emitting diodes placed on both fingertips. Kinematic recordings revealed that the initial grip aperture was smaller when participants read the name of small (e.g., “grape”) rather than large objects (e.g., “apple”), irrespective of the actual size of the wooden block. These results were extended to number processing in another study in which participants had to grasp wooden blocks with small (i.e., “1” or “2”) or large numbers (i.e., “8” or “9”) printed on their visible face (Andres, Ostry, Nicol, & Paus, 2008). The grip aperture was larger in movements performed toward blocks displaying large numbers than it was in movements performed toward the same blocks displaying small numbers (see Fig. 1).

OPEN IN VIEWER

Fig. 1.

Interaction between finger movements and numbers in precision grasping. Subjects had to grasp a wooden block with either a small or a large digit (e.g., 2 or 8) printed on its visible face; the actual block size was identical in both conditions. The X-axis represents the time interval during which the hand was transported towards the object to be grasped, with the time expressed as a percentage of the total duration of the reaching movement (i.e., 0% indicates the time the hand started moving and 100% the time the fingers contacted the object); the Y-axis shows the grip aperture (i.e., the distance between the two fingertips) for movements directed toward bocks with small (white rectangles) and large (black rectangles) digits. This figure illustrates representative results from one individual. The experiment demonstrated the influence of number magnitude on grip aperture, at least during the early phase of the reaching movement: When compared to small digits, the presentation of large digits significantly increased grip aperture in the first half of the movement (Andres, Ostry, Nicol, & Paus, 2008).

An intriguing question remains as to whether magnitude processing interferes directly with the grip aperture or whether it biases the estimate of the object size. This issue was addressed by investigating the effect of number magnitude on the capacity to judge a potential grasping action without performing it (Badets, Andres, Di Luca, & Pesenti, 2007). Participants had to decide whether they felt able to grasp rectangles of different lengths; the display of each rectangle on the computer screen was preceded by the presentation of a small or a large digit. This study showed that the average size of the objects that the participants found impossible to grasp was larger when small digits were presented before the object display. In other words, participants overestimated the maximal size of graspable objects following the display of small digits, presumably because those objects were perceived as smaller than their actual size. The opposite results were found for large numbers, suggesting that the interference between grip aperture and number magnitude originates from the interaction between the numbers and object-size estimates. A control experiment showed that number magnitude did not affect the object-size estimate when it was processed to perform a comparison with a reference object, indicating that the interference between number magnitude and object size is specific to action. Altogether, these results suggest that the motor system responsible for grasping is endowed with processes that automatically take into account the magnitude estimates conveyed by the names of graspable objects or by numbers.

COUNTING ON FINGERS

Finger-counting strategies are used spontaneously by children when learning to count. Developmental studies have shown that the ability of 6-year-old children to identify the fingers touched by the experimenter, in the absence of visual feedback, is a better predictor of their mathematical skills than standard developmental tests (Noel, 2005), supporting the existence of interactions between finger use and number processing in the first years of primary school.

The question arises as to whether these finger-counting strategies still influence numerical representations in adults. Evidence that they do comes from an experiment in which participants had to press a key on a computer keyboard in response to Arabic numerals from 1 to 10 using a different finger for each number. Results showed that the responses were faster when the finger assigned to each number matched the finger-counting strategy of the participant (i.e., from the thumb to the little finger of the right hand for 1 through 5 and from the thumb to the little finger of the left hand for 6 through 10), relative to other key assignments (Di Luca, Grana, Semenza, Seron, & Pesenti, 2006). This finding was corroborated by an electrophysiological study showing that the amplitude of the motor twitch induced by TMS in a right-hand muscle was larger during verbal judgements performed on the odd/even status of numbers ranging from 1 to 4, whereas no increase was found during the processing of numbers between 6 and 9 (Sato, Cattaneo, Rizzolatti, & Gallese, 2007). Because all participants reported starting to count on their right hand, these excitability changes have been regarded as reflecting the existence of overlapping representations for small numbers and fingers of the right hand.

The relationship between finger movements and counting was further investigated in a study in which TMS was used to measure changes in the excitability of the right-hand motor representation in participants counting visual stimuli using either numbers or letters (Andres, Seron, & Olivier, 2007). In the letter task, each stimulus had to be associated with a letter according to its alphabetical order and the final letter was produced verbally as the result of the “count.” Results showed that the motor cortex excitability increased equally in both the number and letter tasks, when compared with a control task (see Fig. 2). This finding suggests that hand motor circuits may assist the counting process by keeping track of the association between a given stimulus and the elements of an ordered series, irrespective of its nature.

OPEN IN VIEWER

Fig. 2.

Changes in motor cortex excitability in counting tasks. Transcranial magnetic stimulation (TMS) was applied over the hand area of the primary motor cortex in order to elicit a small muscle contraction in the controlateral hand (A). The amplitude of this response was measured and used to assess quantitatively the excitability of the motor system during various tasks. In the counting tasks (B), subjects were instructed to give either the number of dots displayed on the screen (“NUMBER”) or the corresponding letter of the alphabet (“LETTER”), whereas in the control task (“COLOR”) they had to decide whether two contiguous dots had the same color. Electromyographical recordings (C) from a representative subject illustrate the increase in motor system excitability in the counting tasks, as evidenced by larger muscle twitches (i.e., peak-to-peak amplitude) in the number and letter tasks as compared with the control task. The two counting tasks did not differ from each other (Andres, Seron, & Olivier, 2007).

Future research should explain how numbers are mapped to finger representations as well as to other mental representations. Indeed, it is well known, for example, that numbers can also interact with space processing in hand-selection or attention-orientation tasks (Hubbard, Piazza, Pinel, & Dehaene, 2005). One possible explanation is that the practice of finger-counting strategies gives rise to conventional associations between hands, numbers, and space, the hand being typically used to guide visual search strategies or to indicate the left or right side of space. However, the influence of counting habits on interactions between number and space is still largely speculative.

FUTURE DIRECTIONS

Altogether, the results reviewed above agree with the view that the motor system interacts with semantic processes. Indeed, it has been shown that motor circuits may be recruited to represent the body movements described by action-related words. The involvement of the motor system in semantic processing is also supported by the interactions found between finger movements and concepts as abstract as numbers. However, it is noteworthy that the semantic processing of words and numbers remained largely implicit in the aforementioned studies. In order to define more precisely the nature of the relationship between action and cognition, future research should investigate the involvement of the motor system in a broader range of linguistic tasks (e.g., discriminating object names based on the object function) and numerical tasks (e.g., number comparison or addition).

In order to assess the claim of sensorimotor theories that the motor system is crucial for semantic processing, further TMS studies are necessary to determine whether a “virtual” lesion of the motor system could impair linguistic or numerical skills. The finding that TMS applied over the primary motor cortex facilitates judgment on action words suggests that linguistic performance can be enhanced by the activation of motor circuits, but it can be argued that such facilitation concerns processes that are not causally linked to word understanding.

Another interesting matter would be to investigate the semantic knowledge of actions in brain-damaged patients with ideomotor apraxia, a disorder affecting the mental representation of gestures. Recent investigations show that some apraxic patients are able to recognize pantomimes of actions despite having difficulty performing the actions themselves (Negri et al., 2007). Although this finding indicates that the integrity of gesture representations is not necessary for successful action recognition, the question remains as to whether patients with impaired gesture representations have impoverished concepts of actions. Because ideomotor apraxia includes difficulties with symbolic gestures, the capacity to perform arithmetical operations using finger configurations should also be explored in these patients.

Is cognition built on actions or on action consequences? An interesting viewpoint is that gestures are represented not only in terms of specific body movements but also in terms of the consequences they are intended to produce. One may therefore speculate that the semantic representation of action words is partially determined by the perceived effects of the designated action. Similarly, the small or large magnitude assigned to a given number may depend on the predictive value of this number regarding the goals of object-directed actions (e.g., how large is an object to be grasped?). Investigating how the perceived effects of a given action can shape the semantic representation of action words and numbers should provide further insight into the interactions between actions and cognition.