Abstract
This article proposes a duplex theory for understanding the scientific impact of contributions to psychological science. I argue that articles that we “love” can be understood in terms of (a) triangular elements of intimacy, passion, and commitment and (b) types of stories that characterize high-impact articles. Certain kinds of stories (e.g., review articles) are more likely to have lasting impact, on average, than other kinds of stories (e.g., data-driven empirical articles).
Some scientific work, we love; other work, we just like; and still other work, we don’t care one whit about. All the articles in this symposium are “loved” enough for them to be among the most-cited articles published in journals of the Association for Psychological Science in the 30-year history of the organization. Why these articles rather than thousands of others? That is the question I will address in this concluding essay.
When I first read these highly cited articles, my reaction—and, I suspect, the reaction of many others—was “Hey, these articles have nothing in common.” That is pretty close to the truth, or so it seemed when I first read them. But then I started to think about people we love, and I came to the conclusion that the scientific work we love can be understood in terms somewhat related to the way we love other things, whether people or ideas. I argue in this article that our reactions to high-impact articles bear some resemblance to our reactions to the people in our lives whom we come to like or even love.
We have a relationship with the articles that can be characterized by a duplex theory (see also Sternberg, 2006). The theory has two components—a triangular component and a story component—which I discuss here in turn. (The use of the concepts of “triangle” and “story” here are metaphorical, of course; see Sternberg, 1985a.)
Triangular Component
The triangular element of the theory has three components: intimacy, passion, and commitment (see also Sternberg, 1986b, 1998a). The argument here is that high-impact works—the works that the field comes to “love”—generate these three elements (through the kinds of stories to be discussed later).
For the sake of broad familiarity on the part of readers, I will introduce the elements with reference to research with which virtually everyone is familiar—Milgram’s work on obedience to authority, which has been very highly cited. According to Google Scholar, the book Obedience to Authority (Milgram, 1974) has been cited 8,962 times, and Milgram’s article in the Journal of Abnormal and Social Psychology has been cited 5,461 times (Milgram, 1963). So just these two works on the same problem have been cited more than 14,000 times. That would qualify as extremely high impact. Of course, few of us (if any) will ever do work with the impact of Milgram’s, but the work serves as a limiting case of a high-impact set of studies with which virtually every psychological scientist is familiar.
Table 1 summarizes the aspects of the triangular theory applied to high-impact scientific work. Not all highly cited work will rate highly for each individual on each of the three elements, but the more people who experience these elements, and the more of the elements they experience, the more likely the work is to have a lasting impact.
Triangular Theory Applied to High-Impact Work in Psychological Science
Intimacy
One characteristic of scientific work that we love is that we may feel a sort of intimate connection to it:
Trust. We believe that the empirical findings are real and not fly-by-night ones that could never be replicated. We believe a scientific method can be used and give reliable results. Or we believe that a scientific theory is valid, or at least credible. For example, Stanley Milgram’s work is as trustworthy today as when Milgram first did it half a century ago. And it has been replicated. The findings are about as trustworthy as any in psychological science.
Understanding. We feel we understand the work. It makes sense to us. Although a given theory, set of empirical findings, or new method may surprise us at first, once we learn about it, we can assimilate it with existing cognitive structures or accommodate it by creating new cognitive structures. Almost no one would have predicted Milgram’s findings—not even psychiatrists did—but people understood the results once they learned about them: They had seen, in their lives, how easily people capitulate to authority. I remember doing something I should not have done when I was young, just because I was pressured into doing it by particular others who thought they knew what I should do. The Milgram work helps me understand why I did it.
Connection/attachment. We feel a connection with, or attachment to, the work. It clicks. It is somehow personally meaningful to us. We are drawn to it. For example, I find it horrific to think that if my own country, the United States, were taken over by a ruthless despot with no conscience, roughly two thirds of the population might do pretty much anything he or she asked. I lost half my family of my grandparents’ generation as a result of such a despot, so I know what the consequences can be.
Familiarity/Comfort. The work somehow becomes a part of us—we are comfortable with it. It is readily accessible to us to use to interpret phenomena in our life and the lives of others. Most scientific work is easily forgotten (or at least becomes inaccessible in memory), but high-impact work can easily be accessed when we need it. The Milgram results are familiar to almost everyone who has studied psychology. We all can call on them to understand the ease with which people can behave in ways that they never would have thought possible.
Passion
A second characteristic of work we love is that we may feel passionate about it. (Passion also can arise from ideas or people we hate—see Sternberg & Sternberg, 2008). People differ widely in the extent to which they become passionate (Tennov, 1979), but all of us feel passion under some circumstances and to at least some extent.
Excitement. The work excites us. It arouses our enthusiasm—not just for the specific work, but for what is possible in all scientific work. Many teachers of introductory psychology, even after teaching about the Milgram work any number of times, remain excited about it. When I teach about the work, I always ask my students how many of them believe they personally would have administered the highest level of shock supposedly administered by the apparatus. Typically, at most one or two people raise their hands, even in a very large class. I tell them that chances are roughly two thirds of them would go to the top level of shock. To me, that is an exciting demonstration. How wrong we are about ourselves! The Milgram experiments help us understand how easily an authoritarian leader could destroy democracy, here or anywhere.
Intense focus. Most scientific work we learn about remains in our focus of attention only briefly and superficially. It does not draw us in so that we keep thinking about it. I learned about the Milgram experiments when I took an introductory psychology class in 1968—50 years ago! Yet I can’t let go of the work. It draws me in. How did the world let the Holocaust happen? Why are so many politicians today utterly spineless and feckless? Why are people so quick to praise or condemn individuals just because others do? Why do scientists jump so quickly and completely into one fad after another, each time believing that the fad will last (Sternberg, 1997a, 2017)? I keep coming back to Milgram.
Transformation. Passion transforms us. We feel like different people because of it. We feel an intense attraction to whatever arouses our passions. Collectors know the feeling when they add one more highly sought-after possession to their collection. The Milgram experiments were transformational for the field of psychology—which is why they are still so widely cited today—and for many students and psychological scientists individually. They made us realize how easily we can become tools of someone else and his or her desires.
Exclusivity. When we feel intense passion, it has a certain kind of exclusivity; the passion is reserved for the object of the passion. This is not to say that one cannot be passionate about other people or ideas. But the object of passion occupies a unique place in our minds and in our “hearts.” Many of the studies we read are very similar to other studies we read—they feel almost interchangeable. Work like Milgram’s is unique. Others of course can replicate it. But they are replicators—they did not come up with the idea.
Commitment
Enduringness. Commitment implies enduringness—a relationship with a person or an idea that lasts. Some works continue to have impact year after year, decade after decade, whereas other works arouse passions for a few years (or perhaps only a few months), and then pass into the dustbin of psychological science—they prove to be flashes in the pan. The Milgram results have had about as much enduringness as any in psychological science. Whereas some data may tell us something about people at one point in history—for example, some of Freud’s results may have been more telling during Victorian times than they are today—other data, such as Milgram’s, continue to help us understand human nature.
Lasting engagement. When we are committed to a person or an idea, we engage with it over a long period of time. We stick with the individual or the idea. The field of psychological science has stuck with Milgram. Not only has the work undergone replications of various kinds over a period of many years, but the work continues to be cited in virtually all introductory psychology and social psychology texts. The field is committed to the work, the data, and the inferences that can be drawn from it.
Meaningfulness. A truly committed relationship has a certain kind of unique meaning. It has a purpose in our lives. In the case of scientific work, the work is not merely something one has read about and remembers. High-impact work has a meaning and perhaps even a purpose for our lives that endures over time.
Resilience. A truly committed relationship is for better or worse. People in a committed relationship grow stronger through the challenges they face together. Likewise, truly significant work in science survives challenges and may even be strengthened by them. The results of Milgram’s work, for example, have been challenged for their generalization across time and culture. The results have endured. They have not endured ethical challenges all that well, but when the study is replicated ethically, the results survive the challenge (e.g., Burger, 2007).
The work that will have the highest impact and receive the most citations is work that can appeal to all three of these elements—that is, work that engenders what I have called consummate or complete love (Sternberg, 1986b). Very few endeavors will produce that kind of effect. To produce such work, a scientist needs to be not only analytically intelligent (Sternberg, 1985b, 1986a, 1997c; Sternberg & Grigorenko, 2002), but also, and more importantly, highly creative (Sternberg, 2016a; Sternberg & Davidson, 1982; Sternberg & Lubart, 1995.) A scientist needs to be socially as well as practically smart—to have a sense of how to produce work that can reach people and possibly change their minds (Kihlstrom & Cantor, 2011; Sternberg, 1997b; Sternberg & Hedlund, 2002; Sternberg & Smith, 1985; Wagner, 2011). It is perhaps ironic, therefore, that graduate schools place so much emphasis on choosing students for analytical skills, because these are probably not the ones that, alone, will propel students to do research that is highly cited (Sternberg & Sternberg, 2017). The tests may identify the best consumers of information but not necessarily the best producers of it (Spear-Swerling & Sternberg, 1994; Sternberg, 1986a). Scientists who reach the top levels of creative work (what has been called “Big C”—Kaufman & Beghetto, 2009) inevitably complement analytical skills with creative and practical ones—analytical skills are probably necessary but far from sufficient for high levels of scientific success (Sternberg, 2016a, 2016b, 2016c). Most of all, perhaps, the scientist needs luck—that his or her work is timed just right to reach the needs and wants of his or her audience (Gaughan, 2010; Merton & Barber, 2004). And if one has the timing right, an echo chamber can result, whereby a few people citing work can lead to others citing it and then to still others citing it. Creativity always involves a match that continues over some period of time between the work of the individual, the state of the domain of work, and the field of people pursuing work in that domain (Csikszentmihalyi, 1988, 2013; Gardner, 2011).
Which articles appeal to individual readers—and which articles individual readers will love—will depend in large part on the interests of the readers and what connects with them. But one can be assured that many readers resonated—through intimacy, passion, and commitment—to articles in the symposium, because those elements are, I believe, a large part of what produced the citations that got the articles into the symposium in the first place.
Story Component
A second part of the duplex theory of love is an understanding of love as a story (see also Sternberg, 1995, 1998b). Whereas the triangular component characterizes general elements of love and attraction (to people, objects, or ideas), the story component characterizes the particular stories we all create that produce our feelings of love. In my previous work, I have suggested roughly two dozen stories that are common in loving relationships between people, but I believe the number of stories is potentially far greater when it comes to scientific work. The story component is the message one brings to or takes from a relationship—between people or between a person and an idea or a thing.
I have summarized in Table 2 what I believe to be the five types of compelling stories behind each of the articles in the 30th anniversary symposium. In order from most to least frequent, they are:
Story Theory Applied to High-Impact Work in Psychological Science
Theory/review that creatively integrates existing findings. These reviews take existing data, sometimes data that do not even seem to bear much relation to each other, and integrate them into a coherent and unified theoretical or other framework. They provide what Sternberg, Kaufman, and Pretz (2002) refer to as syntheses.
Review that tells a new story about old data (debunking). These reviews also are integrative, but are distinguished primarily not by their synthetic quality but rather by their new interpretations of old data. They may be redirections, in the sense that they try to move the field in a different direction, or reinitiations, which try to start a field anew (Sternberg, Kaufman, & Pretz, 2002).
Implications of psychological science for policy decisions. These articles are important because the theories and data they contain have fairly immediate implications for social, educational, or even political policies.
Interesting and important empirical result. These articles are empirical ones—they contain novel and compelling results.
Description of methodological innovation. These articles describe, not necessarily for the first time, methodological innovations. They bring to the field either new methods or directions for use of new methods that can facilitate the research enterprise.
In Table 3, for each article in the symposium, I describe the type of story on which it is based (in my view) and provide a brief quotation that seems to highlight a main idea of the article. Obviously, one could quibble over which articles are assigned to which kinds of stories, and some articles may tell multiple stories. But I believe the table gives a reasonable characterization of where each article makes its greatest contribution. Citations to the original, most-cited articles can be found in the articles cited here.
The Stories Behind 23 High-Impact Articles in APS Journals
The frequency of each kind of story is unequal, ranging from seven down to one. On the one hand, it is hard to know what to make of these numbers, because they may reflect only my own classifications, or they may reflect the kinds of articles APS journals publish. On the other hand, the data do suggest that effective review articles—those that either synthesize a field or debunk previous interpretations of data—can be particularly useful to other scientists as they go about their work. Pure empirical articles seem, on average, to be somewhat less likely to be cited extensively over long periods of time, perhaps because empirical results accumulate and new data add to or even supersede old data. Put another way, if one wishes to write an article that will be cited over long periods of time, a review might be the preferred path. That said, as discussed earlier, stunning results such as Milgram’s continue to be cited decade after decade.
Conclusion
I have argued that impact in psychological science can be understood, in part, in terms of a duplex theory with two components—a triangular theory and a story theory. The triangular theory seeks to explain how the work attracts us; the story theory seeks to explain what attracts us. The triangular theory consists of three elements—intimacy, passion, and commitment—that can lead to scientific impact; the story theory consists of (at least) five kinds of narratives that, if done extremely well, can lead to success of the work.
One might ask why citations are even important. A distinguished psychological scientist recently commented to me that, in his or her opinion, citations are not correlated with either the scientific merit or importance of scientific work. I disagree.
Indeed, there may be excellent scientific work—comprehensive theories, clever experiments, or whatever—that are not much cited. But I would argue that if work is not much cited, or in the extreme is largely ignored, its scientific impact will be limited and transitory. As both of my scientific advisers (Endel Tulving and then Gordon Bower) pointed out to me many times, science is public—scientists need an audience. Research that is not public is not science (and is reminiscent of the philosophical conundrum of whether a tree that falls in the forest makes a sound if no living organism hears it). The scientists whose work most endures, through textbooks or encyclopedias or handbooks or websites or whatever, tend to be those who are most cited over time, through these works and otherwise. Mozart and Salieri both wrote popular and presumably high-quality music during their time, but Mozart’s endured, having been played over and over again countless more times than Salieri’s. Probably few of us are cited as much as we believe we deserve to be, and we may create accounts for why citations really do not matter for much. And they certainly do not matter for everything. But in the end, citations are a main vehicle through which research lives on—an important vehicle through which scientific work has heuristic value for future generations of scientists.
Citation counts and related metrics (see the “Special Section on Scholarly Merit in Psychological Science” in the November 2016 issue of Perspectives on Psychological Science, volume 11, number 6) are certainly not sufficient for high and positive scientific impact. Trofim Lysenko’s work is often talked about, but virtually no one today would call it good science. And sometimes work simply is not appreciated in its time, such as the work of Gauguin in art. All fields operate in echo chambers. The echo chambers may be small or large at any given time. But I would argue that the most scientifically important work reverberates not only for small groups of like-minded aficionados, or even for larger groups in a given time, but also across time as well as space. In the end, the work that has heuristic value to many groups of scientists and others in many places and times is the work that survives.
Our current citation metrics are far from perfect (see the “Special Section on Scholarly Merit in Psychological Science, Part 2” in the November 2017 issue of Perspectives on Psychological Science, volume 12, issue 6). Citation counts are imperfect, as are derivative indices, and are best supplemented with other kinds of measures—for example, outside letters (for a discussion of such criteria, see Sternberg, 2016a, 2016b). When I came up for promotion earlier in my career, I, like so many others, was judged largely on the basis of letters written by outside experts. But from a scientific point of view, there is much to be said for having some kinds of quantitative indices of impact in addition to scholars’ qualitative opinions, no matter how rational or heart-felt those opinions may seem to those who have them. It is useful to have measures that represent the views of the scientific population over time and place of possible readers and users of research, in addition to the opinions of small samples of experts, even very elite ones, whose opinions may (and almost inevitably do) reflect the particular zeitgeist of their scientific socialization at a given time and place. Citations over time can provide one way of distinguishing what is faddish from what endures. Work that is not cited over time, like Salieri’s music, has a short life span, no matter how important it may seem at the time it is done.
If one wishes, in one’s career, to do high-impact work, what can one learn from this symposium? I believe there are perhaps three important take-away lessons. You need to write an article that:
Connects deeply with readers (intimacy), that readers feel passionate about (passion), and that readers believe has a message that will endure over time (commitment);
Tells a compelling story—perhaps a literature review that definitively integrates findings that previously have not fit together or an empirical finding that has clear implications for educational or health policy; and
Reflects your own abiding interests and enthusiasm: If there is one thing that comes through in all the articles, it is that the authors cared deeply about their work and were not writing just to get their next promotion.
I hope you have found the symposium interesting and that one of your articles will have a shot at being included in some future reflection on highly cited articles.
Footnotes
Acknowledgements
I am grateful to Ellen Berscheid, Brad Bushman, Stephen Ceci, Alexandra Freund, James Kaufman, and Timothy McNamara for comments on an earlier draft of this manuscript.
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.
