Preferential Biases for Texts That Include Neuroscientific Jargon

Abstract

The results of an experiment of preferential biases for texts that include neuroscientific jargon are presented. Such preferential bias has been reported even when the presented jargon is meaningless. In a variation of the well-known Weisberg et al. experiment, a group of undergraduate students (N = 150; females 48%, males 52%, other 0%; M age = 22.4 year, SD = 2.6) chose between two possible explanations for a psychological phenomenon: a correct explanation or a circular restatement of facts. Unrelated neuroscientific terms were added to one of the explanations. Participants were asked to choose the correct explanation. There was a statistically significant preference for the explanation without neuroscientific terms. These findings differ from Weisberg et al.’s experiment and a number of others. The implications of this discrepancy are discussed.

Keywords

preferential bias neuroscience jargon

Introduction

Neuroscience assumes that understanding people’s behavior is possible using techniques such as brain scans of cerebral activity. Neuroscientific research and its applications are peculiarly fascinating to the general public (Thornton, 2011), despite the limits intrinsic to every field of science (Weisberg, 2008). Neuroscience seems to provide explanations of psychological phenomena in unequivocal terms; the label “neuromania” has been coined to describe the expectations it evokes in the public (Legrenzi & Umiltà, 2011). In recent years, this has been at the center of a lively debate (Tallis, 2011), started by an article skeptical of the role of neuroscientific discoveries as a basis for explanations of human behavior (Legrenzi & Umiltà, 2011). Responses varied from a critique of the presumed conservatism in the discipline of psychology (Parisi, 2009), to a defense of the experimental nature of cognitive research (Aglioti, 2013), and of the technical approach of neuro-imaging (Girotto, 2009).

A good example of neuromania in action comes from a well-known set of experiments by Weisberg, Keil, Goodstein, Rawson, and Gray (2007), showing how possible explanations of psychological phenomena chosen by naïve participants were rated as more satisfying when they included neuroscientific information, even if such information was completely irrelevant to the phenomenon being examined. In their experiment, Weisberg et al. described to participants a number of well-established psychological phenomena and their possible explanations. Their descriptions of the phenomena were accurate, but the offered explanations were either correct or circular restatements of the premise (as such not informative). Further, neuroscientific jargon with no relationship to the phenomena was added to some explanations. Each participant was asked to rate how satisfactory the explanation was on a 7-point scale. Results showed a significant effect of the inclusion of neuroscience jargon: explanations with neuroscientific jargon were rated as significantly more satisfying than explanations that did not include it. Building on the work of Weisberg et al. (2008), Rhodes, Rodriguez, and Shah (2014) examined the influence of neuroscience information on evaluations of flawed scientific studies, finding that irrelevant neuroscience information led people to believe these studies provided a better understanding of the mechanisms underlying the psychological phenomenon.

Neuroimaging can be considered a sort of iconic representation of neuroscience in popular culture, but replication of the effect of the original Weisberg et al. (2008) experiment using neuroimaging is controversial. McCabe and Castel (2008) found that presenting brain images with articles summarizing cognitive neuroscience research resulted in higher satisfaction ratings, but Michaels, Newman, Vuorre, Cumming, and Garry (2013) failed to reproduce such findings in an experiment with a larger number of participants and in a meta-analysis combining their experimental results with those obtained by McCabe and Castel. In a comprehensive follow-up, Fernandez-Duque, Evans, Christian, and Hodges (2015) presented a set of experiments aimed at ruling out the effects of other factors in the original Weisberg et al. experiment. The authors confirmed the findings by Michael et al. (2013) that brain images did not have effect satisfaction ratings above and beyond written text, and ruled out the difference in text lengths between the explanations with and without neuroscience information. They documented the specificity of the effect of neuroscience information by showing that the addition of irrelevant terms from hard sciences and social sciences did not elicit the same effect.

Weisberg revisited her original experiment in 2015 (Weisberg, Taylor, & Hopkins, 2015) in order to deconstruct and explore the causes of the effect of neuroscience information. Results shown that length of explanation and neuroscientific terminology made independent contributions to the effect (Study 1), and that technical neuroscientific terms and general terms related to the brain elicited the effect in comparable ways (Study 3). In Study 2, which is of specific interest here, each participant was presented with two explanations at the same time: one good, and one faulty. Neuroscientific terms were in turn added to none, both, or only to the faulty explanation. Participant were asked to choose the explanation that they found more satisfying. Result showed that adding neuroscientific terminology to the faulty explanation made the participants less likely to prefer the good explanation. These experiments indicate that humans are, to use Weisberg et al.’s phrasing, “seduced by the allure of neuroscience information.” It is important to understand that the judgments expressed by the participants were mainly of an aesthetic nature, as they usually expressed their agreement or satisfaction with the explanations, and an eventual persuasive effect of the neuroscientific language is generally not explicitly explored.

Given that the previous work indicates the existence of a seductive effect on neuroscience, it is of interest if the inclusion of irrelevant neuroscientific terminology can alter, in addition to satisfaction with the quality of the explanation, the judgment of the correctness of the explanation. The research question in this study was whether using the language of neuroscience, even without any meaningful content to speak of, can affect people’s choice between a good and a faulty explanation of a psychological phenomenon, and the valence of this effect. With this aim, a slightly modified version of Weisberg et al.’s original study was devised.

Method

Participants

There were 150 participants (76 female and 74 male; M age = 22.4 year, SD = 2.6, range = 19–26). All participants were undergraduate students with no specific neuroscience background and received no compensation. Before the real experiment, participants were asked to fill a questionnaire to ascertain their self-reported knowledge of neuroscience, in order to assess their naïve status. The self-report questionnaire tested their previous experience with neuroscientific topics in education, their specific interest in reading or watching neuroscientific topics in the media, and their involvement with neuroscience in their personal lives. Nine participants had to be excused, as they have professed themselves somehow proficient in the field.

Design

There were four randomized trials per participant, each of which used a different item. Each item was accompanied by both a “Good” and a “Bad” (faulty) explanation. This study was carried out with three between-subjects conditions to prevent participants from direct comparison of stimuli. Participants were assigned to a condition at random. In the Without Neuroscience (n = 50) condition, neither explanation contained any neuroscience information. This condition was necessary to establish a baseline; in the original Weisberg et al. (2008) experiment, satisfaction as a measure was considered absolute, and while in the current experiment, the effect of neuroscience was considered relative to a ground-truth situation, representing the ability of the participant to discriminate between objectively good and bad explanations. The other two conditions were Good with, Bad without (n = 50) and Good without, Bad with (n = 50), where the neuroscientific terms were associated, respectively, with the Good and Bad explanations. A statistically significant difference in the results from these two groups would suggest that participants evaluation of the veridicality of a psychological phenomenon can be modified by the addition of irrelevant information, on condition that such information deals with people’s relationship with science in general and neuroscience in particular.

Materials

Four of the items used in the original experiment were selected for this study: object labeling, goal-directed action, attentional blink, mental imagery (Table 1). All the items were pretested by asking a further 50 participants (with the same background and specifications of the experiment’s participants, but not part of the experiment itself) to read them and then try to explain them back. The above items were those for which the explanations in a qualitative analysis more closely resembled the original items (Table 1). A similar reduction in the number of stimuli from the original paper was carried out in Rhodes et al. (2014) and Weisberg et al. (2015), although using different criteria and resulting in the selection of different items.

Table 1.

The items chosen for our experiment.

Item 1

A group of 3-year-old children were shown a set of five objects, four of which were familiar to them and one of which was new. An experimenter asked them to point to each of the familiar objects, which they did correctly. Then the experimenter asked them to point to the “blicket,” a name that the children had not heard before. Results: The children reliably pointed to the novel object and not to any of the familiar objects.

Item 2

Subjects watched two short videos involving a pair of animated balls. In one video, the balls moved randomly around the screen. In the other video, one ball appeared to chase the other around the screen. Results: The researchers found that the subjects couldn’t stop referring to intentions, desires, and goals in their descriptions of the “chasing” video. The subjects used different types of words, usually related to motion, when describing the other video.

Item 3

Subjects sat at a computer screen. They saw a rapidly flashing series of pictures of faces. Somewhere in this series of faces, there were two pictures of houses. Subjects had to press a button each time they saw a house. When the two houses were far apart in the sequence, the subjects were very good at this task. But when the houses were presented close together in the sequence, subjects failed to press the button for the second house. The researchers call this phenomenon “attentional blink.”

Item 4

Subjects were asked to imagine a series of objects that were make-believe (e.g., a unicorn) or that were real but not present in the room (e.g., a mountain). As the subjects created mental images of the various objects, they were asked questions about their images and told to respond as quickly as possible, without reflecting on their answers. They were also asked the same questions about objects they could actually see in the room (e.g., a pen). From an analysis of the responses to these questions and of the times it took subjects to respond, the researchers found a similar pattern of responses and response times for all three types of objects.

Note. Presented here in their original English wording, the items were translated into Italian for the experiment.

As in the original experiment: the “good” explanations were the real explanations that research has provided for each phenomenon, while the “bad” explanations are circular restatements of the phenomenon, hence not explanatory at all. The neuroscience information was always identical in both good and bad explanations for a given phenomenon; the neuroscience information did not alter the underlying logic of the explanations. An example is given in Table 2.

Table 2.

Good and bad explanations to Item 1.

Good explanation

Researchers examined children’s brain activation as they performed this task and concluded that children applied the new name to the unfamiliar object because of the way brain’s language area in the left temporal lobe assigns labels. Children assume that a single object can have only one name. All of the other objects already had names, so they applied the word “blicket” to the new object.

Bad explanation

Researchers examined children’s brain activation as they performed this task and concluded that children applied the new name to the unfamiliar object because of the way brain’s language area in the left temporal lobe assigns labels. The word “blicket” was new to the children. They needed to pick one of the objects to which to apply this name and, therefore, chose the new object.

Note. The explanation in the without condition is the roman text, the one in the with condition also included the bold text.

Procedure

Participants were briefed about the experiment, highlighting the fact that these are actual studies considered to have valid and replicable results, and that one of the proposed explanations is the real one. They were then introduced to a paragraph describing the first item. They read the description of the item in their own time, then were shown the two possible explanations and asked to choose one. The same procedure was repeated for the other items. At the end, some demographic and general interest questions not directly related to the experiment were asked. The order of the items and the position of the explanations (left, right) were randomized. The questionnaires were administered and all the data gathered with the help of a professional CAWI system, under the supervision of a researcher for the whole duration of the test. Data were treated according to current Italian privacy law. After completion of the experiment, participants were fully debriefed about the purpose of the experiment and the correct explanations of the phenomena.

Analysis

The results from the Without Neuroscience condition was used as baseline, in order to evaluate the direction of the effect of the neuroscientific addition to the explanations. To attest if a statistically significant difference between the Good with, Bad without and the Good without, Bad with condition exists, the Pearson’s chi-square test with Yates correction for binary classifications (Yates’ chi-square test) was carried out on the data. Preliminary analyses revealed no gender effect on performance, so results were collapsed.

In order to verify that the results were not influenced by a single item, analyses were repeated item-by-item.

Results

In the Without Neuroscience condition (n = 50 × 4, Good explanations = 111, Bad Explanations = 89), participants were not able to consistently separate a correct explanation from a circular restatement of the facts. These values have been considered as the experiment baseline, and as such determine the direction of the neuroscience effect in the other conditions. This is necessary as there is no way to determine the a priori expertise of the homogeneous group of participant.

Answers from the Good with, Bad without condition (n = 50 × 4, Good explanations with Neurological terms = 73, Bad explanations without = 127) were compared with the baseline condition using Yates’ chi-squared test. Results (χ²= 29.15, df = 1, p < .05) indicated a statistically significant difference between the two conditions, i.e., that the addition of the neurological terms had a measurable effect on the explanation choice. Since the direction of the effect was away from the Good with and toward the Bad without, the addition of irrelevant neurological information influenced participants in this manner. The same result has been obtained in the Good without, Bad with condition (n = 50 × 4, Good explanations without Neurological terms = 146, Bad explanations with = 54, χ²= 24.88, df = 1, p < .05), indicating that the effect is not linked to the Bad or the Good explanations per se, but to the addition of neurological terms.

Results of the item-by-item comparison (Table 3) are generally consistent with the global findings, and the null hypothesis was rejected for all items, except Item 4. For Item 4, the chi-squared test could not rule out the hypothesis that choice was not influenced by neuroscientific terms at 95% significance, but the absolute value still indicated a preference for the explanation without them.

Table 3.

Frequencies of response types by condition with chi-squared comparisons.

Item	Explanation type	Frequency			χ²
Item	Explanation type	Good without, bad with (1)	Good with, bad without (2)	Without neuroscience (3)	3 vs. 1	3 vs. 2
Item 1: Object labeling	Good explanation	37	17	26	4.24	3.47
	Bad explanation	13	33	24	5.51	3.01
					9.75*	3.48*
Item 2: Goal-directed action	Good explanation	37	18	28	2.58	3.94
	Bad explanation	13	32	22	4.10	4.10
					4.16*	4.20*
Item 3: Attentional blink	Good explanation	39	17	30	2.41	6.08
	Bad explanation	11	33	20	4.51	7.81
					4.56*	7.83*
Item 4: Mental imagery	Good explanation	33	21	27	1.12	1.56
	Bad explanation	17	29	23	1.84	1.32
					1.85^a	1.38^a
Items 1–4	Good explanation	146	73	111	10.72	13.35
	Bad explanation	54	127	89	14.16	15.80
					14.24*	15.85*

Note. Chi square applied with Yates correction for binary data, all df = 1. *p < .05; ^ap < .10.

Discussion

Results obtained from comparison between the Without Neuroscience condition and both the Good with, Bad without and Good without, Bad with conditions differ from the conclusions from the original paper (Weisberg et al., 2007) and from Study 2 in (Weisberg et al., 2015). In the present study, the addition of neuroscientific terms made participants less willing to choose an explanation compared to another explanation that had no neuroscientific terms, regardless of the explanation’s correctness. In the original study, the addition of neuroscientific jargon improved the ratings of explanations, while in Study 2 (Weisberg et al., 2015) a preference for a Good explanation without neuroscience jargon was less likely than for the Bad explanation with neuroscience language. A plausible explanation from the difference in results between the present study and the original study may be due to the difference between tasks. In the original study, participants are asked to rate their satisfaction with the explanation, which is an aesthetic judgment, while the task in the present study had to do with selection of a correct explanation. Participants had to choose the correct explanation from two alternatives, rather than declaring how much they were satisfied by one of them.

The same line of reasoning is less likely to hold for Study 2 of Weisberg et al. (2015). The question was still worded in terms of satisfaction, but the task was a choice as in the current study: participants could be more satisfied by the Good or the Bad explanation (or express indifference), thus implicitly selecting among competing choices. Results of the current study and Weisberg et al.’s Study 2 are more directly comparable, and differences are worthy of further investigation. Some elements that attract aesthetic judgment could serve as a deterrent when a concept has to be decoded and used as a meaningful carrier of information. Testing this hypothesis could give insight on the role of neuromania and neurophobia¹ as the evaluation and production sides of the same underlying mechanism.

An important peculiarity in the results of the current study relative to the baseline data should be pointed out. The self-reportedly neuroscience-naïve participants, who nonetheless had a university background and some technical proficiency, were unable to distinguish consistently between a correct explanation and a circular reinstatement. This contrasts with some previous literature: Weisberg et al. (2008) reported that participants in all groups could tell the difference between Good and Bad explanations, regardless of the presence of neuroscience language, based on their satisfaction ratings. In Weisberg et al. (2015), results from Studies 1 and 2 demonstrated that participants understood the difference between Good and Bad explanations. This factor may have influenced performance in the With Neuroscience conditions in the current study and must be further investigated.

Especially in comparison with the other instance in which the effect of neuroscience information was tested in a selection task (Weisberg et al., 2015, Study 2), results from the present study suggest that effect may exist, but the direction and magnitude may vary. Differences in cultural background, geographical provenence, and personal disposition toward technology may elicit different choices as well as influencing the perceived correctness of a statement.

Building on Weisberg et al.’s (2015) Study 3, in which alternative versions of the explanations were tested to single out the contribution of neuroscience language vs. generic scientific terms, and from the work of Fernandez-Duque et al. (2015), in which neuroscience language was compared to social and hard science terminology, it would be interesting to test the effect using other specific instances of neuromania (such as neuroeconomics, neuromarketing, neuroaesthetics, neuroquantistics). As many of these subdisciplines have gained recent popularity, the effects could be compared and contrasted with results from neuroscience in general (and especially neuro-imaging), which has a more substantial history.

A potential limitation of the current study stems from the fact that all participants share similar backgrounds (specifically, the sample was drawn entirely from a Western, educated, industrialized, rich, and democratic society; Henrich, Heine, & Norenzayan, 2010). It would be interesting to choose a random sample, e.g., using Mechanical Turk, and by sourcing a specific sample from people with a broader range of educational backgrounds, to gather data on the penetration of the effect in people who are exposed to neuroscience only through mass media. This could also give new insights on how the effect of neuroscience can interact with prior beliefs. Scurich and Shniderman (2014) demonstrated how neuroscience information is subject to confirmation bias, and it would be interesting to see if the converse is also true, that neuroscience terminology can actually modify beliefs.

Further work should be also be devoted to define a precise and replicable protocol that could be administered in a more comprehensive way to different classes of participants, to ascertain how the presence of neuroscientific jargon would affect them. Following Weisberg et al. (2015, Study 2), such a protocol should also include a debriefing to discuss the motivations and beliefs that are the basis of their choices. This in turn would allow testing the idea that neuromania and neurophobia are different sides of the same phenomenon, and that the emergence of one or the other in response to the use of neurological terms may be linked to the fascination that attracts members of the younger generations to cutting edge scientific topics but also the fear that such topics are too complex to be fully grasped.

Footnotes

Article Notes

References

Aglioti

S. M.

Berlucchi

(2013) Neurofobia: chi ha paura del cervello? [Neurofobia: Who's afraid of brain?], New York, NY: R. Cortina.

Fernandez-Duque

Evans

Christian

Hodges

S. D.

(2015) Superfluous neuroscience information makes explanations of psychological phenomena more appealing. Journal of Cognitive Neuroscience 27(5): 926–944.

Girotto

(2009) Un’immagine del cervello vale più di mille parole (anche se dedicate al cervello) [An image of the brain is worth a thousand words (although dedicated to the brain)]. Giornale italiano di psicologia 2(2009): doi:10.1421/29756.

Henrich

Heine

S. J.

Norenzayan

(2010) The weirdest people in the world? Behavioral and Brain Sciences 33: 61–83.

Legrenzi

Umiltà

(2011) Neuromania: On the limits of brain science, Oxford, England: Oxford University Press.

McCabe

D. P.

Castel

A. D.

(2008) Seeing is believing: The effect of brain images on judgments of scientific reasoning. Cognition 107(1): 343–352.

Michael

R. B.

Newman

E. J.

Vuorre

Cumming

Garry

(2013) On the (non)persuasive power of a brain image. Psychonomic Bulletin and Review 20(4): 720–725.

Parisi

(2009) Non si può capire la mente senza studiare il cervello [You can not understand the mind without studying the brain]. Giornale italiano di psicologia 2(2009): doi:10.1421/29748.

Rhodes

R. E.

Rodriguez

Shah

(2014) Explaining the alluring influence of neuroscience information on scientific reasoning. Journal of Experimental Psychology: Learning, Memory, and Cognition 40(5): 1432.

10.

Scurich

Shniderman

(2014) The selective allure of neuroscientific explanations. PloS One 9(9): e107529.

11.

Tallis

(2011) Aping mankind: Neuromania, darwinitis and the misrepresentation of humanity, Durham, England: Acumen Publishing.

12.

Thornton

(2011) Brain culture: Neuroscience and popular media, New Brunswick, NJ: Rutgers University Press.

13.

Weisberg

D. S.

Keil

F. C.

Goodstein

Rawson

Gray

J. R.

(2007) The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience 20(3): 470–477.

14.

Weisberg

D. S.

(2008) Caveat lector: The presentation of neuroscience information in the popular media. The Scientific Review of Mental Health Practice 6(1): 51–56.

15.

Weisberg

D. S.

Taylor

J. C.

Hopkins

E. J.

(2015) Deconstructing the seductive allure of neuroscience explanations. Judgment and Decision Making 10(5): 429.