Naive and Obvious Questions

BEHAVIORAL OBSERVATION IN THE SERVICE OF THEORY

Research that designs and gathers behavioral measurements to test theories of how people think is useful, important, and sometimes intellectually brilliant. Indeed, it represents a gold standard for psychological research: Predict variation in a behavioral dependent variable across experimental conditions on the basis of a theory of cognitive process, and thereby give the theory a nice, clean, clear empirical test. Journal editors and grant reviewers love this kind of study—often, they require it.

But notice how the behaviors measured in such a study are not chosen because anybody thinks that they are particularly interesting or important in themselves. Rather, they are chosen because of what they are believed to potentially reveal about the cognitive process in question. Secondarily, and often necessarily, they typically are chosen to be feasibly measured within tight resource constraints, which is probably the main reason why psychological research so often relies on behaviors that are reduced to questionnaire responses or keyboard presses.

Where does 75 years or so of research following this model leave us? Just where you would expect: with a literature containing many studies that test specific predictions of easily measured behavioral outcomes based on a wide variety of mini-theories designed to yield just such predictions, and with an extremely uneven empirical map of the behavioral terrain, in which a few areas are represented in exquisite detail (e.g., self-description, reaction time, memory recall) and many others are left almost completely blank. As examples of behaviors lying within the relatively unexplored territory, Baumeister et al. (2007) listed “helping, hurting, playing, working, taking, eating, risking, waiting, flirting, goofing off, showing off, giving up, screwing up, compromising, selling, persevering, pleading, tricking, outhustling, sandbagging, refusing, and the rest” (p. 399).

TOWARD A MORE REPRESENTATIVE BEHAVIORAL MAP

I propose two reasons why it might be worthwhile for psychological research to put more effort into developing a more evenly representative map of the behavioral terrain:

The availability of such a map would promote integrative science by showing how behaviors relate with each other and thus how the many diverse minitheories that have been used to predict those currently in the literature overlap or conflict in their empirical support.

More specifically, I propose it would be useful to design more of our research toward two ends: First, we should observe and measure behaviors that matter. Ideally, this would be done not haphazardly, but guided by some kind of taxonomy. “Behaviorism,” despite its name, never offered one. 1 The Riverside Behavioral Q-Sort (Funder, Furr, & Colvin, 2000) offers one potential foundation by providing a list of (in its current version) 67 observable, meaningful behaviors. 2 Second, we should measure these behaviors in studies designed to uncover their determinants. There are two prominent candidates for what these determinants might be: persons and situations. In short, I believe we need more studies that measure interesting and important behaviors and include a range of person variables, situational variables, or, ideally, both.

For person variables, the agenda is relatively clear, once behaviors of interest have been identified. Many individual difference variables have been developed along with technologies for measuring them, including a rich catalog of personality traits, individual differences in cognitive skills, biological variables such as hormone and neurotransmitter levels, and even brain structure. It is important to note that the personality variables that receive the most attention in current research are very different from the kind of behavioral and situational variables that dominate the literature. For the most part, personality variables are chosen and constructed to capture psychological characteristics that are intrinsically important (e.g., depression, Machiavellianism, self-monitoring, authoritarianism), and major efforts have been made to categorize and identify their core dimensions (e.g., the Big Five). Moreover, personality research typically employs correlational designs in which the levels of the personality variables are measured as they vary naturally in the participant population at hand and not experimentally manipulated to arbitrary levels. 3

For situational variables, the current position is very different and the way forward is less clearly marked. Just like behaviors, situational variables typically have been chosen on the basis of their relevance to the theory being tested, provided that they can be feasibly manipulated within in an experimental hour. 4 They were not chosen because anybody thought that they were the most common, interesting, or important stimuli or situations encountered in human life. The result of decades of research designed in this manner is a map of situational variables that might be even more unevenly drawn than the map of behavioral variables mentioned earlier.

An alternative strategy would be to experimentally manipulate major aspects of the kinds of situations that are consequential, in the classic tradition of 1970s social psychology wherein people in a hurry were induced to pass by someone in apparent need or were ordered to give electric shocks to a protesting victim. But such research runs into formidable practical and logistical issues, as well as raising ethical concerns that go beyond the typical concerns of finicky Institutional Review Boards. Because, like person variables, the important dimensions of situations probably cannot be readily manipulated in experimental contexts, correlational research tapping behavior in real life situations may be required (again, like person variables). People do many different things in many different situations every day. My suggestion is that we do more research that measures the elements of these situations and correlates them with the behaviors that occur in them.

Even if researchers agreed to this, a further methodological complication would remain. Psychology lacks a well-developed, comprehensive, widely accepted set of variables for conceptualizing and assessing the essential ingredients of psychological situations. The Riverside Situational Q-Sort (RSQ) has recently been introduced as an attempt to begin to remedy this lack, but more work on the RSQ and alternative schemes are needed (see Wagerman & Funder, in press).

A SIMPLE RESEARCH AGENDA

With taxonomies and measurement instruments for behaviors and situations in hand, the research agenda is, in principle, simple. First, researchers should conduct studies that examine how and to what degree personality variables are associated with behavior, or (to paraphrase Lewin, 1951) B = f (P). This requires measuring personality variables along with observing meaningful behaviors as directly as possible. This goal is not well served by merely correlating questionnaires with one another.

Second, researchers should conduct studies that examine how and to what degree situational variables are associated with behavior or B = f (S). This goal will require that investigators design experimental studies in which individuals' behaviors are observed in situations that vary along identifiable dimensions that have been experimentally manipulated or, as mentioned above, correlational studies in which the elements of naturally occurring situations are measured along with the associated behaviors. The goal will be aided to the extent that comprehensive lists of variables for describing situations become available, such as the RSQ and, hopefully, other instruments.

Later, studies might seek to combine both kinds of variables in pursuit of the interactions between these determinants, as in the classic Lewinian B = f (P, S). The effect of situational variables might well turn out to depend on the level of the personality variables, and vice versa. But in the enthusiasm to embrace interactionism, I hope psychologists remember that main effects come first, in more ways than one. Interactions are what are left after the main effects soak up much, if not most or all, of the variance, which is why, as every practicing researcher knows, they are so difficult to replicate (Chaplin, 1991). And, interactions aside, we still know much less about the main effects of persons and situations than we should after 75 years of busy research.

OBSTACLES TO THE AGENDA

This research agenda faces two formidable obstacles that together probably explain why it has not been regularly pursued. First, such research is difficult and expensive. As was noted by Baumeister et al. (2007), questionnaire research is much easier and much cheaper than research that attempts to observe and measure behavior more directly, and very often, given the relative poverty of psychological science compared with some other sciences, it is all that is possible. This is why social psychology relies on self-report methods almost as much as personality psychology does. As regards situational variation, it is procedurally and ethically feasible to manipulate only relatively nonimpactful and inconsequential stimuli rather than the range found in real life, and moving research into the wider environments of our research participants, where important situational events happen on a daily basis, faces its own procedural and ethical hurdles.

The second obstacle to this research agenda would still exist even if all the procedural difficulties were somehow solved: The research I am proposing is descriptive. It does not entail thinking up clever minitheories of behavioral determination and then designing studies to test them. It does not even entail seeking ingenious ways to obtain seemingly counterintuitive findings (another gold standard for social psychological research). Instead, it envisions choosing the situations and behaviors to assess on the grounds that they are intrinsically important—a very different ground for choosing what to study. Decades ago, Stanley Milgram and John Darley studied obedience and helping behaviors in powerful situations not because they had developed precisely detailed theories, 5 but because they believed that these behaviors are important and that almost anything that could learned about the circumstances that evoke them would be bound to be important as well. They were right.

AN ANECDOTE OF THE OBVIOUS

Research psychologists have a license—perhaps even an obligation—to study many of the most fascinating and useful questions a scientist could hope to address. Surprisingly often, we manage to avoid doing that. The world is full of interesting topics that are neglected when research prioritizes exact tests of specific hypotheses over broader description and experimental feasibility over representativeness and realism.

Psychology's knack for avoiding obvious questions was vividly demonstrated to me a long time ago, while I was completing my dissertation. I was doing a study to follow up on the research by me and my graduate school mentor, Daryl Bem, on the template-matching technique, in which predictions concerning the personality profiles of people who respond to experiments in different ways are used to test psychological theories (Bem & Funder, 1978). To make a long story very short, my own subsequent study used self-descriptions of personality (rendered using the California Q-Sort), and it didn't work. Nothing correlated with anything. The academic year had not yet ended, so in desperation I recontacted my college student research participants. I persuaded each of them to help me recruit two of their acquaintances to provide descriptions of their personalities. Happily, these peer descriptions yielded much better results than the self-reports had; not perfect, but good enough to complete my dissertation and be published, eventually, in JPSP (Funder, 1982).

Here is where the story becomes relevant to present concerns. One day, I realized that, almost accidentally, I had gathered peer descriptions as well as self-reports of personality by the very same people. How did they compare? I ran a few simple analyses and found that although average self and others' descriptions differed on many items, by and large the correlations between self and others' descriptions of personality were remarkably large. In several cases, impressive self–other correlations were found on the very same items that showed large self–other mean differences. I was fascinated by these results and began to wonder if they might be publishable.

It seemed doubtful to me that they would be, because the study, if you could even call it that, was so obvious. All I did was gather self-descriptions of personality and peer descriptions of the same people and compare them. Surely this had been done many times before. I asked several faculty members for advice and they all assured me that they were sure such an obvious study had been done. But when I sought a specific reference, nobody could quite pin one down. I went looking in the literature myself and to my surprise was also unable to find even one study that simply compared self and others' personality descriptions on a range of variables. I did come across one large project that appeared to have gathered the necessary data, but all the published results I could find compared the factor structure of the two sets of ratings (both yielded about five factors) but never reported the simple correlations between self and others' ratings.

So, with some trepidation, I wrote up my results and submitted them to the Journal of Personality. I was thrilled when the paper was accepted (it was only my second independent publication) but also worried. The odds seemed pretty good that the papers I had failed to find and cite would now emerge, and my sad ignorance of the literature would be publicly exposed.

It never happened. There still might be an earlier study out there somewhere that I failed to find. There certainly ought to be. I never did anything more obvious. But that article (Funder, 1980) has been cited almost 100 times (according to ISI), and none of these citations claims that what I found had been reported earlier. 6

The moral of this story for all aspiring researchers: Never, ever assume that a study or a finding is so obvious that somebody “must have” done or found it already. Many of the most interesting and even important studies you can do may be hidden in plain sight. My own subsequent research program addressed the naive question of when judgments of personality are likely to be accurate, and when they are not (Funder, 2001). For researchers wondering what other obvious questions might be out there, awaiting a researcher naive enough to study them, I cannot do better than repeat Baumeister et al.'s (2007, p. 399) list of “helping, hurting, playing, working, taking, eating, risking, waiting, flirting, goofing off, showing off, giving up, screwing up, compromising, selling, persevering, pleading, tricking, outhustling, sandbagging, refusing, and the rest.” There has to be an interesting—and obvious—study or two in there someplace.

CONCLUSION

Referring to seemingly naive questions laypersons often have about human behavior, my long-time mentor Jack Block once wrote the following:

Questions such as these – responding to persistent human wonderings – lie behind the wide and demanding lay interest in psychology. Academic and research psychologists sometimes forget, or do not recognize, that these lay questions are fair questions ultimately, even if sometimes ingenuously framed. They deserve far more serious scientific attention than has yet been granted them by the busily preoccupied field of scientific psychology. (Block, 1993, p. 10)

Elegantly designed, tight little studies that measure one behavior in two behavioral conditions in pursuit of the test of a focused hypothesis are the current state of the art, and every psychologist is proud to be able to design and to publish one. But more studies like these are not what we need right now. We need a map of the broader behavioral terrain (Funder, 2009). To allow this map to be drawn, journal reviewers and granting agencies will need to give higher priority to descriptive—and mostly correlational—research that measures interesting and consequential behaviors across a realistic range of situational variables. Careful methodology and appropriate data analysis remain essential, but perhaps the requirement that every study must test a tightly specified theory can be relaxed for while.

Why not give it a shot? Psychology has tried putting theory ahead of data for a long time, and not without some success. But a deep-seated frustration with the final products of a research tradition that always puts theory first might be what lies behind the often-voiced complaints that articles in psychology's best journals are so often boring. Perhaps it is time to try the reverse strategy and in that way become a bit more like other sciences such as biology. In the end, the result might be better theories—theories that come from data (Haig, 2005). In other words, I suggest a style of research that values theory but shapes it to data rather than the other way around. The theories that will emerge will naturally become more interesting and widely relevant because they will be based on observations of interesting and important behaviors observed in interesting and important situations.