A Multitheoretical Approach to Big Text Data: Comparing Expressive and Rhetorical Logics in Yelp Reviews

Abstract

This article uses a multitheoretical approach to investigate the relationship between language use and opinion expression on Yelp. Using review metadata (e.g., star rating) to observe variation in reviewer feelings and motivations, we test for the strength of different message design logics: expressive logics, where language reflects a reviewer’s underlying opinion, and rhetorical logics, where language reflects a reviewer’s desire to make his or her opinion credible and acceptable to their audience. Results suggest that emotional language is motivated by expression as higher rated businesses are reviewed with more positive and fewer negative emotion terms. Rhetorical logics are associated with the use of abstract and self-focused language, with analysis suggesting this may result from the reviewer’s decision to write either narratively or formally.

Keywords

online reviews crowdsourcing text mining message design LIWC social media automated text analysis

Communication behavior is now observable to an unprecedented degree (Lazer et al., 2009). Because the vast majority of these observations are raw text (Rudder, 2014), research has increasingly focused on developing methods for analyzing these data (see Brook O’Donnell & Falk, 2015; Hopkins & King, 2010; Mehl & Gill, 2010). Less attention has been paid, however, to the theoretical study of textual production: the mechanisms and properties that observed discourse might reflect (Barnett, Wigand, Harrison, Woelfel, & Cohen, 1981; Brook O’Donnell & Falk, 2015; Lin, Margolin, & Lazer, 2016). Instead of systematically considering the possible ways a text may have been produced, research tends to assume a plausible theory consistent with a study’s aims. For example, research using sentiment analysis generally treats a text’s emotional content as a direct reflection of the speaker’s attitude (Himelboim, 2010; O’Connor, Balasubramanyan, Routledge, & Smith, 2010). Conversely, framing research typically evaluates differences in language use as choices in how to communicate rather than reflections of underlying intentions (Borah, 2014).

These assumptions distort analysis. If researchers assume that emotions expressed in discourse are authentic, they will be misled when it is used rhetorically (e.g., to provoke reactions in others). Similarly, if individuals use hostile or ideological frames because they view their opponents with hatred or disdain, their communication behavior is a symptom rather than a strategy (Borah, 2014). More broadly, failing to consider different processes of discourse bypasses the opportunity to use “big” text data to advance research into new domains.

This study aims to provide theoretical and methodological guidance for exploring different discourse processes by explicitly comparing them within a single dataset. We use the communication theoretic distinction between message design logics (O’Keefe, 1988)—expressive, conventional, and rhetorical—to make predictions for three kinds of discourse variables: emotion words, abstract language, and immediacy words. This approach, adapted from Monge and Contractor’s (2003) multitheoretic, multilevel (MTML) rationale for network study, provides a first step in distinguishing discourse production processes in real-world text.

Message Design Logics and the Challenge of Big Text Data

“Big” text data differ importantly from data used in prior communication research. Traditionally, text data were produced by unidentified individuals in controlled experiments (e.g., Hancock & Dunham, 2001) or surveys (e.g., Barnett et al., 1981; Rice & Danowski, 1993). In these contexts, topics, audiences, and incentives could either be manipulated or fixed in advance. When researchers gathered text from the messier real world, they were restricted to those produced by or within institutions (Lammers & Barbour, 2006), such as specific organizations (e.g., Kleinnijenhuis, van den Hooff, Utz, Vermeulen, & Huysman, 2010), fields (e.g., Doerfel & Barnett, 1999), or media (e.g., Corman, Kuhn, McPhee, & Dooley, 2002). While these institutions could not be controlled, they could be studied exhaustively through complementary means such as interviews or archival research. New media allows the observation of discourse produced by individuals in relatively uncontrolled settings (Himelboim, 2010). With these data, researchers can neither control why individuals are writing nor easily gather complementary, extratextual data about them. This uncertainty encourages researchers to assume that the text is produced via a plausible rationale without considering alternatives.

Monge and Contractor (2003) found a similar pattern in network research. Network scientists often assumed the network they observed was formed via a particular theoretical process when in fact multiple processes were plausible. In their MTML model, Monge and Contractor synthesized and catalogued the predictions made by different theories about how observed networks would form. They suggested that researchers should look for the signature of a theory, cases where the theory made a prediction that was distinct from those made by other theories. Identifying signatures of existing theories helped network researchers to identify dominant mechanisms (e.g., homophily) from rarer, more contingent ones (e.g., generalized exchange; Snijders, van de Bunt, & Steglich, 2010). The search for signatures also informed theory development by uncovering patterns not expected from known mechanisms.

We propose a similar multitheoretical approach to the study of observational text data. In particular, we suggest communication researchers should begin by exploring the textual signatures of different message design logics (Hart & Burks, 1972; O’Keefe, 1988). There are three basic design logics that govern the production of individuals’ statements. A message can be constructed expressively, meaning it reflects the underlying thoughts or feelings of the speaker; conventionally, meaning it is built in accordance with a set of rules appropriate to that situation or context; or rhetorically, meaning it is built in cognizance of feelings and rules but also with an awareness of the most persuasive way to present the message to achieve the speaker’s goal. A recent study reporting an association between the amount of anger in tweets sent throughout a county and the rate of heart disease in that county illustrates this difference (Eichstaedt et al., 2015). The implications of this correlation vary substantially depending on whether this observed anger reflects (a) self-expression: people feel (more or less) angry in some counties, (b) convention: people live in an environment where they are expected to sound (more or less) angry in some counties, or (c) rhetoric: people believe that to have their voice heeded, they must express (more or less) anger in some counties.

The difference between these logics is evident when considering their empirical implications for nontextual data. In an expressive logic, the discourse produced reflects the speaker as “an authentic being, . . . capable of imparting himself ‘as he really is’” (Hart & Burks, 1972, p. 76). The discursive signature of an expressive logic thus corresponds to an extratextual measure of the speaker’s self, such as physiological measures of their cognitive or emotional state (Brook O’Donnell & Falk, 2015). Conventional and rhetorical logics intervene between the speaker’s authentic state and their stated messages, as “sometimes the proposition expressed has nothing to do with one’s own actual salient beliefs” (O’Keefe, 1988, p. 86). The emphasis in these logics shifts to the speaker’s relationship to the audience. In conventional speech, this relationship is cooperative (O’Keefe, 1988). Speakers communicate by following the rules of a game and the audience infers the speaker’s intent within these rules. Rhetorical logics are extensions of conventional logics applied by speakers who have strategic incentives (O’Keefe, 1988). Rhetorical logics rely on, but also transform, conventions for strategic gain. Following the MTML rationale, we seek to first catalog and understand the extent to which well-theorized mechanisms are present in existing discourse.

The Current Study

The Setting: Online Review Sites

Online review sites have become increasingly prominent in commerce and research (Dellarocas, 2006; Li & Hitt, 2008; Walther & Jang, 2012). They are also an important site for competition between different message design logics. It is often assumed that the text of reviews is expressive. For example, the review “I’ve had excellent luck with the pizza from this place”¹ might be interpreted to reflect a positive experience. Yet reviews are also produced rhetorically, such as when an individual is paid to write a fraudulent review (Luca & Zervas, 2016; Ott, Choi, Cardie, & Hancock, 2011). Rhetorical practices also extend beyond fabrication through strategies such as framing (Entman, 1993) or self-presentation (Kacewicz, Pennebaker, Davis, Jeon, & Graesser, 2014).

Analytical Approach

We focus on expressive (emotional expression, Construal Level Theory) and rhetorical (self-presentation, rhetorical justification) mechanisms that prior work suggests should have signatures in reviews, and the dependent variables: emotion terms, linguistic abstraction, and words related to psychological immediacy—that characterize these signatures in other studies. We do not test conventional mechanisms in this article because of the difficulty associated with identifying the specific conventions reviewers are subject to (though we control for them statistically). To capture variation in logics, we focus on two kinds of review metadata—star rating and location. As we argue below, each of these should impact (a) how a reviewer thinks or feels about the object under review, altering the ideas they might express, and (b) the reviewer’s motivations relative to their audience, altering their rhetorical incentives.

Research Questions and Hypotheses

Expressive Uses of Emotion

When emotional language reflects an expressive logic, the emotion words in the text reflect the true feelings of the writer. In Yelp, reviewer satisfaction is represented by the number of “stars” given to a business (more stars meaning greater satisfaction). While positive emotions are often associated with satisfaction (Pennebaker, 2011), negative emotions are associated with failed expectations. Thus, if reviewers express their emotions in the review text, star rating should be positively correlated with the use of positive emotion words and negatively correlated with the use of negative emotion words.

Importantly, this seemingly obvious relationship is not an empirical certainty in real-world statements. While research has established that positive and negative emotion words can accurately capture a speaker’s inner emotional states (for a review, see Pennebaker, 2011), these studies intentionally suppress the influence of conventional and rhetorical logics that might have interfered with this relationship. For example, a participant might be encouraged to write only for himself or herself (e.g., via a diary) or the researcher (see Mehl & Gill, 2010). These studies thus confirm that when an individual intends to express their emotions, he or she uses emotional words. However, they do not indicate whether actual uses of emotional language, in cases where many nonexpressive incentives are present, are primarily expressive. Yelp reviews offer the possibility of testing this proposition via the following expressive predictions:

Hypothesis 1a (H1a): Reviews with higher ratings (e.g., 5-star reviews) will have more positive emotion words (e.g., love, benefit, awesome) than reviews with lower ratings (e.g., 1-star reviews).

Hypothesis 1b (H1b): Reviews with lower ratings (e.g., 1-star reviews) will have more negative emotion words (e.g., hate, dislike, rotten) than reviews with higher ratings (e.g., 5-star reviews).

Rhetorical Uses of Emotion: Self-Presentation

Rhetorical motivations may induce the writer to modify the emotional content of a message so that it is distinct from their true feelings. A well-known reason for this modified emotional content is offered by the theory of self-presentation (see S. W. Smith, Yoo, & Walther, 2008). Individuals use self-presentation strategies to achieve social goals such as to appear attractive to others or maintain a high status with their community (Kacewicz et al., 2014). Specifically, research suggests that people have incentives to present an overly positive image, as positive emotions are considered more socially facilitative and publicly appropriate (Bazarova, Taft, Choi, & Cosley, 2012; Turner, 2007). This leads to a tendency to express more positive emotion to others than one is privately feeling.

The motivation to self-present can vary across situations (see Walther, 1996). Individuals should be more motivated to engage in selective self-presentation when they believe that their audience is one in which their reputation is tracked and evaluated (Bazarova et al., 2012; Sommerfeld, Krambeck, & Milinski, 2008). In online review sites, this tracking and evaluation is performed largely through the text of reviews, such as seller feedback on eBay (Cabral, 2012). It follows that users manipulate this text to manage the impressions of others (see Ellison, Heino, & Gibbs, 2006). The consequences of earning a positive or negative reputation depend, however, on the individual’s relationship to the social community in which that reputation resides. People can afford to risk earning poorer reputations with audiences with which they share few social ties and/or little likelihood of future interaction (Burt, 2005; Coleman, 1988).

Social ties tend to cluster around their geographic home (Krackhardt, 1994; Preciado et al., 2011), even in virtual spaces (Huang, Shen, & Contractor, 2013). Thus, when individuals are writing about businesses located in their own geographic community, their reviews are more likely to be seen by people they know and to impact their neighbors. As such, self-presentation strategies are called for more when an individual is engaging in this more local social interaction for which they feel greater accountability (see Ellison, Hancock, & Toma, 2012; Lerner & Tetlock, 1999; Toma & Hancock, 2010). By contrast, when individuals write reviews for distant communities, they are ostensibly anonymous, and thus their incentives for self-presenting are lower (Anthony, Smith, & Williamson, 2009). Thus, the following hypothesis is proposed:

Hypothesis 2 (H2): Controlling for rating, reviews written by local reviewers will contain more positive emotion than reviews written by visiting reviewers.

Expressive Uses of Abstraction and Immediacy: Construal Level Theory (CLT)

Neuroimaging has revealed that, beyond emotion, expression involves many subprocesses that can indicate why a particular message was produced (see Weber, Eden, Huskey, Mangus, & Falk, 2015). One of these subprocesses is how people think about current objects or life experiences in terms of psychological distance from the self.

CLT is a widely known theory of psychological distance that has been demonstrated to relate to language use (Trope & Liberman, 2010). CLT addresses the level of abstraction with which objects and experiences are conceptualized:

Moving from a concrete representation of an object to a more abstract representation involves retaining central features and omitting features that by the very act of abstraction are deemed incidental. For example, by moving from representing an object as a “cellular phone” to representing it as a “communication device.” (Trope & Liberman, 2010, p. 441)

A key tenet of CLT is that abstraction is associated with psychological distance. Distant objects are conceived of abstractly while close objects are conceived of more concretely (Trope & Liberman, 2010). For example, when anticipating a trip three months in the future (temporally distant), an individual may think about how the vacation will be enjoyable. When the trip is 1 week away (temporally proximate), he or she is likely to think about concrete items such as clothes to pack.

Construal is affected by social distance in addition to physical and temporal distance. When people are exposed to others who they perceive are socially similar, they achieve more concrete conceptual outlooks on ensuing tasks (Liviatan, Trope, & Liberman, 2008). When individuals feel a high level of power, they feel more social distance from others and produce more general and abstract categorizations (P. K. Smith & Trope, 2006). Social distance from an audience can also be important. Individuals focus more on the abstract features of a decision when giving advice to socially distant others (Kray & Gonzalez, 1999).

The conceptual differences predicted by CLT also modify word patterns. Magee, Milliken, and Lurie (2010) found that quotations from authority figures that were more focused on the future (rather than the present) contained a more abstract, less concrete language style. In an experiment by Fujita, Henderson, Eng, Trope, and Liberman (2006), individuals who viewed a video supposedly filmed at a distant location gave a more abstract description of the scene compared with those who viewed a video supposedly filmed nearby.

The social distance between a reviewer and a business should vary consistently with the reviewer’s relationship to the business’s location. Social identification with the local area as “home” can alter construal directly (Liberman & Förster, 2009). Local businesses are also more likely to be frequented by local people who the reviewer will consider similar (Christakis & Fowler, 2009). The reviewer’s location at the time of writing may have some impact, with some reviewers writing while physically at an establishment and others after they have returned home. Nonetheless, the systematic variance in social distance should persist because, according to CLT, variation in any distance has an impact (Katz & Byrne, 2013). Thus, we hypothesize the following:

Hypothesis 3 (H3): Reviews written by local patrons will be less abstract (more concrete) than reviews written by visiting patrons.

Language patterns can also indicate construal through the use of immediacy words—language patterns that approximate the “here and now” (Cohn, Mehl, & Pennebaker, 2004). These words indicate where the speaker is focusing their attention. Immediate language expresses psychological closeness. High immediacy words, including first-person singular pronouns (self-references: I, me, my) and present-tense language, suggest that the speaker is focusing inward and on the self, rather than on the outside world (Pennebaker, 2011). This pattern is demonstrated in cases where people experience psychological distress. People facing distress reflect intensely on their current experiences and state of mind (Beck, 1967). Consistent with this tendency, suicidal writers show more self-focused communication patterns, including the increased use of self-references, than people who die of other causes (Stirman & Pennebaker, 2001). An increase in first-person singular pronouns can thus suggest where an individual’s attention is directed (e.g., inward and on the present; see Pennebaker, 2011). Therefore, if a review expresses the reviewer’s thoughts, text that contains more first-person singular pronouns suggests the reviewer is more focused on the self.

We predict that local patrons will feel more psychological closeness to the object of their review, and their writing style will reflect more verbal immediacy. If they write expressively, this will result in a higher rate of self-references in their reviews. Thus, we predict the following hypothesis²:

Hypothesis 4 (H4): Reviews written by local patrons will contain more first-person singular pronouns (e.g., I, me) than reviews written by visiting patrons.

The valence of an experience can also impact construal. When people reflect on themselves and their own behavior, positive mood is associated with abstract construal (Labroo & Patrick, 2009). When reflecting on external circumstances and events, however, people are more likely to feel personally attached and closer to positive experiences because positive experiences often have an immersive effect of being present. Furthermore, negative experiences make people feel less personally and psychologically attached (see Carrera, Muñoz, Caballero, Fernández, & Albarracín, 2012). One way to cope with a negative experience, such as an unforeseen trauma, is to psychologically distance the self from the event. For example, Cohn et al. (2004) compared LiveJournal blogs before and after the terrorist attacks of September 11. Entries after 9/11 contained more psychological distancing (less immediacy) than before the attacks, suggesting psychological changes prompted by the need to manage the trauma.

If language is used expressively, then first-person singular pronouns should be used at a higher rate in the description of positive experiences. For example, Salovey (1992) induced participants into a positive, negative, or neutral mood and then had each person finish a sentence completion task with pronouns (e.g., I, she, or we). Those induced into a positive mood completed sentences with more first-person singular pronouns than those in the neutral mood (see also Silvia & Abele, 2002). We thus predict the following hypothesis:

Hypothesis 5 (H5): Reviews with higher ratings (e.g., 5-star) will contain a greater rate of first-person singular pronouns than reviews with lower ratings (e.g., 1-star).

Rhetorical Uses of Abstract and Immediate Language: Ambiguity and Justification

Expressive uses of abstraction and immediacy reflect how an individual conceives of an object. People may also choose to write reviews using abstract concepts or make more self-references because of their persuasive effects on an audience.

Abstraction can have rhetorical advantages by making it easier for the writer to escape the consequences of unpopular or controversial statements. More abstract, less concrete language creates greater opportunities for strategic ambiguity (Eisenberg, 1984), as vague, general claims are harder to evaluate and scrutinize (Margolin & Monge, 2013). For example, fraudulent scientific papers tend to have a more abstract writing style than legitimate papers (Markowitz & Hancock, 2016). Similarly, individuals concerned about their reputation on social media often make ambiguous statements to satisfy their audience (Marwick & boyd, 2011).

This argument suggests a prediction contrary to the prior argument for the expression of construal. CLT suggests that local reviewers will conceive of objects in more concrete terms. However, as local reviewers are more likely to be concerned about maintaining their reputations, they may also have a greater incentive to write abstractly. Thus, when reviewing a local object, individuals may edit their statements to be fuzzier or more abstract and thus more palatable to their local audience. Visiting reviewers, by contrast, should feel unimpeded to share their concrete evaluations expressively.

Local reviewers may also use language that appears vague, ambiguous, and abstract to outsiders but is well understood within their community (Margolin & Monge, 2013). For example, one reviewer wrote of a pizza restaurant in Lafayette, Indiana, “If you are from Lafayette, you don’t need to read this review . . . pizza is a thin crust Pizza King style pizza.”³ This statement is conceptually abstract to outsiders (“Pizza King style”) but might be concrete to other locals. If individuals use community jargon in their reviews, their language may appear more abstract when it is not, because these reviews achieve specificity in a manner that is not recognized by standard coding dictionaries (Eisenstein et al., 2010). Thus, rhetorical logics predict the following hypothesis:

Hypothesis 6 (H6): Reviews written by local patrons will use more abstract (less concrete) language than reviews written by visiting patrons.

A related rhetorical motivation is the desire to justify nonnormative views (Hart & Burks, 1972). Individuals do not need to justify ideas that are already acceptable to a community (Green, Li, & Nohria, 2009). However, when statements are controversial or out of the norm, they require warrant and backing (Toulmin, 1958). “Extreme” reviews, or those whose star ratings deviate from the typical review, would thus be more likely to attract abstract writing. While a reviewer may write more concretely to reflect a positive experience, rhetorical logics suggest that as a rating increases toward either extreme, reviewers will muddle or “fuzz up” their claims (making them more abstract) so they are less controversial:

Hypothesis 7a (H7a): Reviews with more extreme ratings (e.g., 5-star, 1-star) will contain a greater rate of abstract language.

People can use first-person singular pronouns similarly. Claims about the self are harder for an audience to test than claims about the broader world. For example, it is harder to challenge the statement, “I called and was asked to hold. Two minutes went by and it was obvious I’d been forgotten about”⁴ [1 star] than the statement, “The sushi here is decent, but the service is just too slow”⁵ [2 stars]. The former refers to a now unobservable event in the past, something no one can challenge. The latter makes two testable claims that could be inconsistent with others’ experiences, casting doubt on the credibility of the reviewer.

Thus, self-references may justify opinions in a way that puts less credibility at risk. For example, Ott et al. (2011) found that, even though liars often use fewer self-references than truth-tellers (also Toma & Hancock, 2012), fraudulent hotel reviews contain more self-references than genuine reviews. Increased self-references may reflect a method to increase review credibility.

Local reviews should contain more self-references (H4), but for a different reason than predicted above. CLT suggests local reviewers perceive an object as psychologically closer to the self than visiting reviewers. Rhetorical logics suggest local reviewers reference the self to add credibility and defend their claims.

This rhetorical rationale leads to a different prediction when considering review valence. Reviewers should construe extremely positive reviews as close but extremely negative reviews as distant. Rhetorically, regardless of how the objects of review are viewed, both extremely high and extremely low-rated reviews should contain high rates of self-references to add defensibility:

Hypothesis 7b (H7b): Reviews with more extreme ratings (e.g., 5-star, 1-star) will contain a greater rate of first-person singular pronouns.

Method

The Yelp Academic Dataset

We collected Yelp reviews from the Yelp Academic Dataset from 2014, which consists of information about businesses located near 30 major U.S. universities. Each review contains several attributes, and includes the review text and the user’s rating of the business on a 1-star (worst) to 5-star (best) rating. Other metadata, such as a nonidentifiable code for the specific reviewer as well as the city in which the business is located, are also provided. The city/town information provided by Yelp is very specific. Many towns contained very few businesses or reviews and were geographically very close to much larger cities or towns. Some locations are also nested within others. This inconsistency creates a challenge which we address in our operationalization of “local reviewers.”

Measures

Ratings

Each review is accompanied by a rating on a scale from 1 (a typically poor experience) to 5 (a typically great experience). No formal criteria or instructions are given by Yelp, so reviewers interpret these ratings for themselves.

Language analysis

Each review was analyzed with Linguistic Inquiry and Word Count (LIWC) software, an automated tool to calculate relative frequencies of words that reflect psychological categories (Pennebaker, Booth, & Francis, 2007). LIWC has become a primary means of automated text analysis in large datasets (Boyd & Pennebaker, 2015). Each LIWC category contains a dictionary of words that has been empirically validated to reflect that psychological dimension across different contexts (Pennebaker, 2011). Because LIWC relies on a standard dictionary that is independent of context, it does not account for slang or other domain-specific vocabulary that might also convey these dimensions. However, when data are drawn from a wide variety of locales for which the researchers lack detailed domain knowledge, LIWC’s dictionary ensures comparability across social communities (e.g., Coviello et al., 2014; Kramer, Guillory, & Hancock, 2014; Pennebaker, Chung, Frazee, Lavergne, & Beaver, 2014).

The text of each review was scored as a percentage of its words that match a particular language dimension. For example, the four-word phrase, “The best burger ever,” has one word that reflects positive affect (i.e., best), and it therefore receives a score of 25% for positive emotion terms. For transparency, we offer the following examples that are typical of Yelp reviews for the categories used in this study (words tapped by LIWC are italicized).

Positive emotion terms

“Food is priced well, tastes good, and the owner is really nice, fun to talk to him about Hawaii.”

Negative emotion terms

“The coffee is great but the service is so horrendous I am going back to Starbucks. The owner is so rude to her customers! PASS.”

Abstraction

Linguistic abstraction refers to the absence of concreteness or specificity of the words used. There are typically two approaches to operationalizing abstraction. The first considers the abstraction level of content words. For instance, the words evolution and mortal are more abstract than house and tiger because the mental images they induce are likely to be fuzzier (see Feng, Cai, Crossley, & McNamara, 2011; Paivio, 1991).

A second approach considers the rates of function words (Chung & Pennebaker, 2007). The use of articles (e.g., a, the) indicates the presence of concrete nouns, and the use of prepositions (e.g., over, under) indicates precise relationships. Quantifiers (e.g., any, every, more, less) express explicit differences between objects. Text that contains a high rate of articles, prepositions, and quantifiers is more concretely written than text with a low rate of articles, prepositions, and quantifiers (Larrimore, Jiang, Larrimore, Markowitz, & Gorski, 2011; Markowitz & Hancock, 2016).

Because our text is drawn from a particular domain (business reviews), we are unsure whether generic content dictionaries would appropriately capture abstraction. Function word use should be less sensitive to domains, however (Chung & Pennebaker, 2007; Pennebaker, 2011). Based on prior work (Larrimore et al., 2011; Markowitz & Hancock, 2016), we created an abstraction index that combines the standardized rates of articles, prepositions, and quantifiers into an index and multiplies that sum by −1. A higher score on this index suggests that a piece of text is more abstractly written compared with a piece of text with a low score on this index.

To demonstrate the index in practice, consider two pieces of text describing loan requests in a peer-to-peer lending network: (A) “It was tough getting my finances back together and I was relying on my credit card” and (B) “I need a loan to pay off my credit card debt.” Text B is more specific and concrete than Text A, and this is reflected in the abstraction index. Text A contains a lower rate of articles (0 words = 0%) than Text B (one word = 9.09% of the total word count), indicating a reduced presence of concrete nouns, and contains fewer prepositions (Text A: one word = 6.25%; Text B : two words = 18.18%), suggesting that A specifies fewer specific relationships between its referents than B. Neither text has any quantifiers (Text A: 0 words = 0%, Text B: 0 words = 0%). When combined, A will be scored as more abstract than B.⁶

First-person singular pronouns

These include references to the self. For example, “I feel ill whenever I eat here. I noticed I tend to review the places I love and not the places I hate. This was the first spot I thought of under the ‘hate’ category” (1-star).

Local reviewer

The local reviewer category is designed to capture whether the business being reviewed is in the “home” local area of the reviewer. While in these anonymized data we do not have the reviewer’s address, we can observe the locales of the businesses for which they typically write their reviews. As mentioned above, the challenge is that the raw business locations provided by Yelp are at inconsistent levels. Some businesses are given generic cities (e.g., Albany, NY; Amherst, MA) and others small, adjacent communities (e.g., Watervliet, NY) or more specific neighborhoods within those broader cities (e.g., North Amherst, MA).

As the goal of capturing location is to identify meaningful differences in the reviewer’s conception of the object or their audience, we combined reviews into “local areas” that included all named locations that were adjacent to or subsumed by another location in our data. This process left us with 26 distinct, noncontiguous local areas. We then checked whether these 26 areas were meaningfully coherent and distinct. Research indicates that people spend the majority of their daily time within one hour of home (see Marchetti, 1994; Schafer & Victor, 2000). Examination of the distances between the businesses and the local areas to which they were assigned indicated that our local areas capture distinct communities. First, we observe that 77% of businesses reviewed were assigned to their original location (i.e., were not reassigned to an adjoining location). An additional 14% (91% cumulative) were assigned to a local area that was 5 miles or fewer from their original location with an additional 8% (99%) cumulative were within 11 miles. The remaining 1% were assigned to local areas no further than 23 miles from their original location. At the same time, no distinct local areas were this close to one another, with only three (out of 325) pairs of local areas even within one hour of one another according to Google Maps navigation (Los Angeles, CA-Claremont, CA = 36 minutes; Palo Alto-Berkeley = 50 minutes; Boston, MA-Providence, RI = 55 minutes). This method thus separates objects of review into those that are well within an individual’s egocentric area of identification and interaction from those that are outside of this area (Liberman & Förster, 2009).

To assign specific local areas to reviewers, we examine the frequency with which they review businesses in an area. Specifically, if 75% or more of a reviewer’s reviews are written about businesses in a particular local area, we consider this local area to be the reviewer’s home area (see explanation below). Each individual review where the business reviewed is in the reviewer’s home area was dummy-coded as a local review.

Threshold Selection and Robustness Checks

As is often the case with datasets of real-world behavior, our data require us to make a number of choices regarding thresholds for measurement (González-Bailón, 2013). The full dataset also brings substantial statistical power, as it contains over 300,000 reviews to perform our analyses. However, our goal is not to understand or characterize Yelp reviews in particular but to understand general processes of discourse production that extend across platforms. For this, the full bulk of Yelp data is not needed, because it does not afford observable variation in important dimensions for the specific message design logics in which we are interested. To test for these logics, we substantially subset our data to those cases where each theoretical mechanism would reasonably be expected to have an observable influence. In other words, we ask the following question: With these criteria for inclusion in our dataset, if this mechanism were to have a strong influence, should we expect to see that influence within this set of cases, and if it were to have a weak influence, should we expect to see little or no effect?

Focal dataset

The primary logic of our analysis is based on within-reviewer differences: how reviewers’ language changes for good experiences versus bad experiences, and local businesses versus distant businesses. We are also interested in reviewers who have enough investment in their reputation on Yelp to be subject to reputational concerns (Anthony et al., 2009). Thus, we restrict the dataset to reviews by individuals who have written at least 10 reviews including at least one local review and one visiting review and then set the threshold for local reviews at 75%. These decisions should give us reasonable precision for identifying local reviewers, as each local reviewer must have written at least eight reviews within one area, and no more than 25% of their reviews in all other areas, to be considered local. Importantly, because our data do not comprise all Yelp reviews, but only those from specific areas provided by Yelp, lower thresholds carry the risk that the reviewer’s true home is unobserved. For example, an individual could write five reviews in New York, three in Los Angeles, two in Houston, and six in Denver (which is not included in the academic dataset). A 50% threshold would mark this person as a New Yorker, when it is more likely they do not clearly identify with a particular community. While setting the threshold at 75% does not eliminate such cases, it makes them much less likely, because the observed data would show a clear tendency for one location.

Robustness check datasets

Based on the preceding rationale, we expect that if expressive and rhetorical logics meaningfully influence language choices in reviews, there should be significant differences observable in the focal dataset. We thus report analyses of our focal dataset in detail. Nonetheless it is possible that these decisions, though reasonable, lead to misleading conclusions. We also run each of our analyses at the following levels—five or more reviews (rather than 10), 50% local threshold and 90% local threshold, and reviews with 50 or more words—and summarize the main findings from these robustness checks.

Models

As our data contain multiple levels with repeat observations at several levels (reviewers, businesses, local areas), we implement multilevel modeling using the lme4 package (Version 1.1-10) in R. Each model contains a random intercept for reviewer, business, and local area. We apply these random effects because specific reviewers, businesses, and local areas appear numerous times in our data and might have specific effects that influence the results.

Results

Descriptive statistics for the focal dataset are provided in Table 1. Bivariate correlations for each variable of interest are provided in Table 2.

Table 1.

Focal Dataset Descriptive Information.

Dataset-level descriptives		Count	Review-level descriptive
Reviews		35,065	Average star rating		3.56 (1.08)
Reviewers		1,543	Median star rating		4.00
Businesses reviewed		7,148
Local areas		26
Language-level descriptives	5-star(n = 6,950)	4-star(n = 13,173)	3-star(n = 9,193)	2-star(n = 4,056)	1-star(n = 1,693)
Positive emotion	6.11% (4.19%)	5.85% (3.64%)	5.21% (3.46%)	4.00% (2.72%)	2.95% (2.50%)
Negative emotion	0.96% (1.41%)	0.97% (1.29%)	1.25% (1.57%)	1.79% (2.36%)	2.99% (4.75%)
Abstraction	−0.10 (1.68)	−0.16 (1.61)	−0.13 (1.63)	0.04 (1.59)	0.25 (1.71)
First-person singular	4.32% (3.07%)	4.12% (2.78%)	4.12% (2.80%)	4.48% (3.03%)	4.71% (3.22%)
CDI	13.76 (11.64)	13.82 (11.12)	12.15 (11.30)	10.30 (11.27)	10.51 (12.19)

Note. The focal dataset was comprised of approximately 20% 5-star reviews while less than 5% received only 1 star. Standard deviations are represented in parentheses adjacent to means. CDI = Categorical-Dynamic Index.

Table 2.

Correlations Between Independent and Dependent Variables (N = 35,065).

	Local	Stars	Stars deviation	PE	NE	Abstraction	I-words	CDI
Local	—	−.057***	.006	−.024***	.01	.016**	−.001	−.028***
Stars	−.057***	—	−.319***	.218***	−.215***	−.041***	−.026***	.103***
Stars deviation	.006	−.319***	—	−.107***	.132***	.050***	.063***	−.064***
PE	−.024***	.218***	−.107***	—	−.135***	.170***	−.097***	.001
NE	.01	−.215***	.132***	−.135***	—	.110***	.030***	−.076***
Abstraction	.016**	−.041***	.050***	.170***	.110***	—	.183***	−.610***
I-words	−.001	−.026***	.063***	−.097***	.030***	.183***	—	−.379***
CDI	−.028***	.103***	−.064***	.001	−.076***	−.610***	−.379***	—

Note. Local is dichotomously coded. PE = positive emotion terms; NE = negative emotion terms; I-words = first-person singular pronouns; CDI = Categorical-Dynamic Index.

p < .01. ***p < .001 (all correlations are two-tailed).

Hypothesis Tests

Results for emotion variables can be observed in Table 3, with abstraction and immediacy findings displayed in Table 4. A general summary of our findings with their corresponding empirical and theoretical motivations can be found in Table 5.

Table 3.

Regressions on Use of Emotional Language.

	Dependent variable
	Positive emotion			Negative emotion
	(1)	(2)	(3)	(4)	(5)	(6)
Constant	2.767***	2.882***	3.252***	2.538***	2.541***	2.245***
Constant	[2.582, 2.953]	[2.665, 3.099]	[3.017, 3.487]	[2.461, 2.616]	[2.441, 2.640]	[2.134, 2.356]
Stars	0.748***	0.747***	0.700***	−0.374***	−0.374***	−0.337***
Stars	[0.713, 0.783]	[0.712, 0.781]	[0.663, 0.737]	[−0.392, −0.355]	[−0.392, −0.355]	[−0.356, −0.317]
Stars deviation			−0.265***			0.213***
Stars deviation			[−0.331, −0.200]			[0.177, 0.248]
Reviewer is local		−0.124*	−0.135*		−0.002	0.007
Reviewer is local		[−0.247, −0.002]	[−0.258, −0.013]		[−0.068, 0.064]	[−0.059, 0.073]
Observations	35,065	35,065	35,065	35,065	35,065	35,065
Log likelihood	−93,041.78	−93,039.81	−93,008.64	−71,199.16	−71,199.15	−71,129.93
Akaike information criterion	186,095.60	186,093.60	186,033.30	142,410.30	142,412.30	142,275.90
Bayesian information criterion	186,146.30	186,152.90	186,101.00	142,461.10	142,471.60	142,343.60

Note. 95% confidence intervals are below coefficients in brackets.

p < .05. ***p < .001.

Table 4.

Regressions on Use of Abstract and Immediate Language.

	Dependent variable
	Abstraction			First-person singular
	(1)	(2)	(3)	(4)	(5)	(6)
Constant	−0.193***	−0.015	−0.143**	4.262***	4.667***	4.262***
Constant	[−0.263, −0.122]	[−0.107, 0.078]	[−0.245, −0.041]	[4.129, 4.396]	[4.500, 4.835]	[4.080, 4.444]
Stars		−0.048***	−0.032***		−0.108***	−0.057***
Stars		[−0.064, −0.032]	[−0.049, −0.015]		[−0.136, −0.080]	[−0.086, −0.027]
Stars deviation			0.092***			0.291***
Stars deviation			[0.062, 0.123]			[0.238, 0.344]
Reviewer is local	0.103***	0.095**	0.099***	−0.026	−0.043	−0.031
Reviewer is local	[0.046, 0.159]	[0.038, 0.152]	[0.042, 0.156]	[−0.124, 0.072]	[−0.141, 0.055]	[−0.129, 0.067]
Observations	35,065	35,065	35,065	35,065	35,065	35,065
Log likelihood	−66,052.70	−66,039.84	−66,025.59	−85,362.58	−85,337.33	−85,281.73
Akaike information criterion	132,117.40	132,093.70	132,067.20	170,737.20	170,688.70	170,579.50
Bayesian information criterion	132,168.20	132,152.90	132,134.90	170,787.90	170,747.90	170,647.20

Note. 95% confidence intervals are below coefficients in brackets.

**p < .01. ***p < .001.

Table 5.

Summary of Hypotheses and Results.

	Hypothesis		IV	DV	Model result	Robustness models
Justification	Expressive uses of emotional language
Emotional expression	1a	Reviews with higher ratings (e.g., 5-star reviews) will have more positive emotion words than reviews with lower ratings (e.g., 1-star reviews)	Stars	Emotion	Supported	Supported
Emotional expression	1b	Reviews with lower ratings (e.g., 1-star reviews) will have more negative emotion words than reviews with higher ratings (e.g., 5-star reviews)	Stars	Emotion	Supported	Supported
	Rhetorical uses of emotional language
Self-presentation	2	Reviews written by local reviewers will contain more positive emotion than reviews written by visiting reviewers	Local reviewer	Emotion	Opposite	Not significant
	Expressive uses of abstraction and immediacy
Construal level theory	3	Reviews written by local patrons will be less abstract (more concrete) than reviews written by visiting patrons	Local reviewer	Abstraction	Opposite	Opposite
	4	Reviews written by local patrons will contain more first-person singular pronouns (e.g., I, me) than reviews written by visiting patrons	Local reviewer	Immediacy	Not significant	Mixed
	5	Reviews with higher ratings (e.g., 5-star) will contain a greater rate of first-person singular pronouns than reviews with lower ratings (e.g., 1-star)	Stars	Immediacy	Opposite	Opposite
	Rhetorical uses of abstraction and immediacy
Ambiguity	6	Reviews written by local patrons will use more abstract (less concrete) language than reviews written by visiting patrons	Local reviewer	Abstraction	Supported	Supported
	7a	Reviews with more extreme ratings (e.g., 5-star, 1-star) will contain a greater rate of abstract language	Stars	Abstraction	Supported	Supported
	7b	Reviews with more extreme ratings (e.g., 5-star, 1-star) will contain a greater rate of first-person singular pronouns	Stars	Immediacy	Supported	Supported

Note. IV = independent variable; DV = dependent variable.

H1a

H1a predicted that as review star rating increases, the rate of positive emotion terms in the review text would increase as well. As shown in Table 3, column 1 (tested in isolation) and column 2 (with covariates included), the result is statistically significant at the p < .001 level. The estimates suggest that a 1-star review would contain about 3.6% positive emotion terms, with each increase in star adding about 0.75% (B = 0.747). In our focal dataset, the average length of a review was 138 words, suggesting that an incremental increase in star rating adds approximately one positive emotion word to the review. Overall, this result was very robust, as it was statistically significant in each model regardless of thresholds for local reviewer (50% or 90%), number of reviews (5 or 10), or number of words in the reviews (at least 1 or at least 50), and holds when controlling for stars deviation (column 3). Together, H1a is supported.

H1b

H1b predicted that as review star rating increases, the rate of negative emotion terms in the review text would decrease. As shown in Table 3, column 4 (tested in isolation) and column 5 (when covariates included), the result is statistically significant at the p < .001 level. The estimates suggest that a 1-star review would contain about 2.2% negative emotion terms, with each increase in star removing less than 0.4% (B = −0.374). This suggests that based on the average length of a Yelp review from our database, half of one negative emotion term would decrease as star rating increases. This result was very robust, as it was statistically significant in each model regardless of thresholds for local reviewer, number of reviews, or number of words in the reviews, and holds when controlling for stars deviation (column 6). Thus, H1b is supported.

H2

H2 predicted that for businesses rated with the same number of stars, reviewers would use more positive emotion terms when writing about businesses in their home area relative to a visiting locale. As shown in Table 3, column 2, the result is not in the predicted direction (B = −0.124) and statistically significant (p < .05). That is, when writing about businesses in their local community versus a nonlocal community, reviewers used an average of 0.12% fewer positive emotion terms. Considering the length of an average Yelp review from our sample, local reviewers use close to one fewer positive emotion term per five reviews than visiting reviewers. This result is not robust, however, as it is not significant in any of our other models. Thus, H2 is not supported.

H3 and H6

H3 predicted that reviewers would communicate with a less abstract (more concrete) style when writing about local businesses while H6 predicted that reviewers would communicate with a more abstract (less concrete) style when writing about local businesses. As shown in Table 4, column 1 (and consistently in columns 2 and 3 where stars are included as a covariate), the result is consistent with H6 (B = 0.103, p < .001). This suggests that, on average, a local business review contains approximately one fewer article, preposition, and quantifier per seven reviews compared with a visiting business review. This result was very robust, as it was statistically significant in each model regardless of thresholds for local reviewer, number of reviews, or number of words in the reviews. We thus find support for H6 and no support for H3.

H4

H4 predicted that reviewers will use more first-person singular pronouns when writing about local businesses. As shown in Table 4, column 4 (and consistently in columns 5 and 6 where stars are included as a covariate), the result is not statistically significant. Inspection of the robustness check models suggests a slightly more complicated pattern. In models with the local threshold, at 90% the negative coefficient was significant (p < .05). Thus, there is some evidence that locale influences first-person singular pronoun use, nonetheless, in no cases were results consistent with H4, which is not supported.

H5

H5 predicted that reviewers will use more first-person singular pronouns when writing about businesses to which they assigned a higher star rating. A shown in Table 4, column 5 (and also column 6), the coefficient is in the opposite predicted direction and significant (B = −0.108, p < .001). On average, more highly rated reviews contain approximately one fewer self-reference (e.g., I, me, my) for every seven reviews. This result was very robust, as it was statistically significant in each robustness check model except one (where only five reviews were required and no minimum word limit was required). Thus, the pattern appears consistent but contradicts H5, which is not supported.

H7a

H7a predicted that reviewers will use a more abstract style when writing about businesses to which they assigned a more extreme rating. To test this hypothesis, we include both a term for stars and a term for stars deviation, the number of stars that a particular review’s rating is above or below the average for that business. H7a is supported if there is a significant, positive relationship between stars deviation and abstraction. As shown in Table 4, column 3, the stars coefficient is significant and negative (B = −0.032, p < .001) and stars-deviation coefficient is positive (B = 0.092, p < .001). This result was very robust as it holds in each robustness check model. Thus, H7a is supported.

H7b

H7b predicted that reviewers will use more first-person singular pronouns when writing about businesses to which they assigned a more extreme rating. H7b is supported if there is a significant, positive relationship between stars deviation and the use of self-references. As shown in Table 4, column 6, the stars coefficient is significant and negative (B = −0.057, p < .001) and stars-deviation coefficient is positive (B = 0.291, p < .001). This result was very robust as it holds in each robustness check model. Thus, H7b is supported.

Post Hoc Analysis

An advantage of the multitheoretical approach is its ability to identify specific cases that appear to contradict established theories and thus require further attention. Contrary to predictions derived from CLT, we observe that first-person singular pronouns decline as stars increase (Table 4, column 5). This effect is reduced, but not eliminated, by the inclusion of the rhetorical effect for extreme reviews (Table 4, column 6). We also observe that stars have a similar, negative relationship for the use of abstraction (Table 4, columns 2 and 3). This suggests that, contrary to the logic of CLT, which suggests that objects at greater psychological distance should be associated with both more abstraction and fewer self-references, abstraction and self-references are positively correlated, which examination of the bivariate correlation confirms (r = .18, p < .001). Having identified this anomaly, we now inspect the raw text of the reviews to see if other theoretical processes, beyond those we have considered, are leading abstract writing style to be joined to self-references in the production of text.

We began our investigation by identifying cases where such processes would likely be visible to the naked eye. We constructed several relevant categories of review: reviews that were high (top 15%) in both abstraction and self-references, those that were low (bottom 15%) in both abstraction and self-references, and those that were high in one category and low in the other.⁷ We then inspected the texts in each of these groups to see if any pattern or intuition emerged regarding how or why abstraction and self-references were joined.

Our observations lead us to hypothesize that their co-occurrence may be caused by reviewers’ choosing or avoiding narratives. Specifically, we observe that many of the reviews that score high on both categories take the form of informally depicted stories. For example,

I learnt a lesson. I had dismissed this place, merely because their food is not very nice. One Saturday morning, having entered the realm of the unwell, I had to leave S&S (for the last time, odious place). I had still not eaten nor had coffee, so I ducked in here in desperation.

They looked after me very well. They saw I was none too lively and it seemed mere humane hospitality to them.

Bless them. [4-stars]⁸

Similarly, we observe that many reviews that score low on both abstraction and self-references are written in a more formal, analytical style such as

Pagliacci is good for a slice on the go. They usually have a few different varieties available for by-the-slice action, ranging from the fancy to the unhealthy. All seem to be in the $2.50 range, which is quite reasonable if you’re going for a quick meal.

I personally enjoy the double pepperoni. Sometimes you just need some grease. The crust is great—soft inside and crunchy on the outside—and it feels like a classic pairing with tasty-but-simple cheese and sauce. The pepperoni can get greasy, but it’s delicious. [4-stars]⁹

Rather than telling a personal story, this reviewer provides a more formal, analytical description of the typical experience at the restaurant.

Having observed this pattern, we considered available theories that might explain this difference. This distinction between narratives and analytical descriptions has both a theoretical and methodological background. Theoretically, the difference can be interpreted as one of “genre” (Yates & Orlikowski, 1992) or “register” (Biber & Jones, 2005). Yates and Orlikowski (1992) defined a genre as “a typified communicative action invoked in response to a recurrent situation” (p. 301). Genres are deployed for rhetorical reasons, to meet the needs of a situation, but carry with them rules, or conventions, for their execution. Similarly, registers are “varieties of language that are defined by their situational (i.e. non-linguistic) characteristics . . . e.g., conversation, spontaneous speeches, newspaper editorials, academic prose” (Biber & Jones, 2005, pp. 164-165). Typically, genres and registers are not considered individual-level choices. Rather, they are given by the norms of a situation. However, one of the unique features of online communication is the speaker’s freedom to define the institutional nature of the forum (Marwick & boyd, 2011). Thus, while restaurant reviews written for a newspaper might be required to provide a formal description, Yelpers are free to choose whether to write formally or tell a story.

Pennebaker et al. (2014) observed writing style differences between genres in college applications. They measure the distinction using the Categorical-Dynamic Index (CDI). This measure evaluates text along a continuum, which on the high end is formal and analytical (the categorical end of the CDI) and on the low end that is more free-flowing and less formal (the dynamic end of the CDI). The more dynamic a piece text, the more it is associated with a more narrative style of explication (see Pennebaker et al., 2014, for the CDI formula). In the examples above, the first, more narrative review receives a lower, more dynamic CDI score (−2.89) when compared with the second, more categorical and formal review (33.30).

Our inspection of the text thus leads us to a testable hypothesis that might explain the otherwise anomalous co-occurrence of abstraction and self-references: a dynamic, narrative writing style. We first test whether our independent variables predict this choice. We predict CDI scores based on the location of the reviewer, stars, and the extremity of the stars in the rating. We find a clear effect for all three variables. First, local reviewers show a significant tendency to use a less categorical, more dynamic style (B = −0.840, p < .001), indicating that they are more likely to write narratively. Second, we find that categorical language increases with stars, as the stars coefficient is positive (B = 0.923, p < .001), but decreases with extremity (B = −0.545, p < .001). Thus, consistent with the rhetorical rational provided for H7a and H7b, extreme reviews are more likely to be couched in a more narrative writing style and thus more difficult to challenge. However, we also see that the willingness to write categorically increases as the review becomes more positive.

Next, we predict both the level of abstraction and the level of first-person singular pronouns in reviews based on both our hypothesized variables and the CDI as a covariate. Results show that, contrary to the results for H6, local reviewers are no longer more linguistically abstract once their choice to write narratively or analytically is accounted for (B = 0.022, p = .17). Similarly, local reviewers are now less likely to use first-person singular pronouns, once their choice to write narratively or analytically is accounted for (B = −0.105, p < .05). At the same time, consistent with the predictions of CLT, stars are now positively associated with self-references (B = 0.029, p < .05), though the coefficient is very small.

Taken together, these results suggest that rhetorical choices occur in a reviewer’s selection of genre or register, rather than in their selection of specific words. Local reviewers, and reviewers with extreme ratings, often tell stories rather than provide formal assessments. Controlling for this genre-level choice, however, there is some evidence of typical CLT effects.

Discussion

Review and Implications of Findings

Our results help to advance the theoretical understanding of the mechanisms that produce real-world text. First, we find support for the presence of both expressive and rhetorical design logics in Yelp reviews, with each operating on the production of a different language feature. Expressive logics were dominant in the production of emotion terms. More highly rated businesses were described with more positive emotion and less negative emotion. Rhetorical logics appeared to operate more strongly on the use of an abstract writing style and self-references. Local reviews, where reviewers have more reputation at risk, and extreme reviews, where reviewers are more likely to draw skepticism or criticism, were written with a more abstract, less specific style. More extreme reviews also contained more self-references, suggesting claims that are harder to challenge. There was little evidence that linguistic choices directly reflected construal in the observed discourse. These results do not suggest that these more muted mechanisms do not operate but rather that, in the production of discourse in this context, they are drowned out by competing theoretical processes.

The results also reveal interesting findings about how these more dominant logics may work. First, our results indicate that expressiveness is not equal across these emotional categories. Inspection of the regression coefficients indicates that the use of positive emotion words varies almost twice as much with star ratings (B = 0.748 in Table 3, column 1) as the use of negative emotion words (B = −0.374 in Table 3, column 3). In other words, the use of positive emotion words appears to be more sensitive to the speaker’s underlying feelings. As we discuss below, this may be an important topic for further research.

Results of the post hoc analysis also suggest the possible need for a shift in the theoretical understanding of these logics. In the approach taken at the start of this article, rhetorical logics are assumed to act after conceptualization through self-editing. The post hoc analysis suggests, however, that rhetorical logics may act more strongly earlier in the process, when individuals choose to write with a particular style (story, formal description, argument). Specifically, we find that local reviewers, and reviewers writing extreme views, are more likely to use language in a manner consistent with narrative or storytelling (indicated by lower CDI scores). Controlling for this tendency, local reviewers are indistinguishable from visiting reviewers in terms of abstraction. This theoretical model suggests that rhetorical and expressive logics may both operate but at different levels, with rhetorical motivations influencing genre choice and expressive logics influencing word choice. Future work might focus specifically on more complex models of discourse production, including relationships between multiple LIWC dimensions, using path analysis.¹⁰

Limitations and Future Directions

By design, this study focuses exclusively on theoretically expected associations between a small set of variables in textual data observed in a messy, real-world setting. Its strength lies in its ability to untangle the influences that operate in real-world discourse, but its weakness is that it can only go so far in identifying and isolating mechanisms. Below we delineate the main limitations of the study and describe how we hope our theoretical approach can help to engender complementary research to address them.

First, now that the most important effects observable in real-world data have been identified, controlled studies could go much further to measure and manipulate emotions, construal, and sensitivities to audiences. For example, a more precise measure of emotions to be expressed could be obtained using functional magnetic resonsance imaging and other physiological measures of emotion, which could then be compared with word choice and review ratings (Brook O’Donnell & Falk, 2015). Such research might investigate how sensitive the use of positive or negative emotion words is to a reviewer’s underlying feelings.

Experimenters may also choose to more precisely manipulate self-presentational concerns to see what prompts shifts in choices for genre or language. An important limitation of the current study is that “localness” may only weakly capture the social community to which the reviewer feels most accountable. Reputational concerns could be manipulated directly, however. For example, subjects could be asked to write reviews that would vary in whether the reviews were shown to friends. These reviews could be evaluated in terms of genre (narrative vs. analytical) as well as the specific word choices. Construal could also be manipulated in terms of hypothetical distance, the probability with which an individual anticipates an experience. This manipulation may be less likely to be confounded with self-presentational concerns and may provide insight into how construal affects discourse.

A second limitation concerns the small effect sizes resulting in the limited change in language use we observe at the review level. For example, with each incremental star rating increase, there would be approximately one positive emotion term added to a review. Conversely, negative emotion terms decrease by approximately half of one word with a reduced star rating as well. While the scale of these changes is consistent with manipulations in many priming and framing studies (Aarts & Dijksterhuis, 2003; Greene, McDaniel, Buksa, & Ravizza, 1993; Schuldt, 2013), it is likely that on the individual review level these sentiment changes are unnoticeable to the average Yelp user—a trend that has been acknowledged in prior large-scale experiments evaluating discourse patterns (Coviello et al., 2014; Kramer et al., 2014). Changes for other effects were even smaller (approximately one word in every five to 10 reviews).

This is an important limitation for certain applied uses of this research for understanding review sites. Our interest in this research, however, is less in the effects of these changes on Yelpers per se, but on the kinds of theoretical conclusions that researchers might draw from textual data. In that sense, statistically significant patterns in large textual datasets, and the relative strength between them, are the relevant research subject, not just patterns that are influential on or detectable by humans.

Another limitation is the restrictiveness of Yelp, particularly its subject matter. We use Yelp because review metadata allow us to observe variation in important extratextual variables that are more difficult to measure in typical social media. Considering a broader array of review sites, such as Airbnb or similar sites where users rate one another’s personal property or services, may afford a greater opportunity to observe certain theoretical dynamics. For example, the effects of self-presentation, which is primarily activated in interpersonal domains where communication is dyadic and face-saving techniques are important, may be easier to observe in this setting (Brown & Levinson, 1987). We encourage future research to explore dynamics of the online sharing economy in a systematic manner.

Conclusion

This study has provided an example of a multitheoretical approach to studying observable text data. Its aim was to encourage further research on the mechanisms through which discourse is produced, that is, the processes that stand between the text that researchers observe and the underlying thoughts, feelings, and other attributes of the text’s authors. The results of the study indicate that while some aspects of this process are already well understood, others merit, and hopefully will draw, further interest.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This material is based upon work that is supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture, Hatch under 1004600.

Notes

Author Biographies

Drew Margolin is an Assistant Professor in the Department of Communication at Cornell University. His research focuses on the mechanisms that shape the production of individual and collective discourse on the Internet and social media.

David M. Markowitz is a PhD Candidate in the Department of Communication at Stanford University. He uses automated methods to collect and analyze language data to reveal communication and psychological dynamics, such a deception and persuasion.

References

Aarts

Dijksterhuis

(2003). The silence of the library: Environment, situational norm, and social behavior. Journal of Personality and Social Psychology, 84, 18-28. doi:10.1037/0022-3514.84.1.18

Anthony

Smith

S. W.

Williamson

(2009). Reputation and reliability in collective goods: The case of the online encyclopedia Wikipedia. Rationality and Society, 21, 283-306. doi:10.1177/1043463109336804

Barnett

G. A.

Wigand

R. T.

Harrison

R. P.

Woelfel

Cohen

A. A.

(1981). Communication and cultural development: A multidimensional analysis. Human Organization, 40, 330-337. doi:10.17730/humo.40.4.dq72w082g1p4u6x0

Bazarova

N. N.

Taft

J. G.

Choi

Y. H.

Cosley

(2012). Managing impressions and relationships on Facebook: Self-presentational and relational concerns revealed through the analysis of language style. Journal of Language and Social Psychology, 32, 121-141. doi:10.1177/0261927X12456384

Beck

A. T.

(1967). Depression: Clinical, experimental, and theoretical aspects. New York, NY: Harper & Row.

Biber

Jones

J. K.

(2005). Merging corpus linguistic and discourse analytic research goals: Discourse units in biology research articles. Corpus Linguistics and Linguistic Theory, 1, 151-182. doi:10.1515/cllt.2005.1.2.151

Borah

(2014). Does it matter where you read the news story? Interaction of incivility and news frames in the political blogosphere. Communication Research, 41, 809-827. doi:10.1177/0093650212449353

Boyd

R. L.

Pennebaker

J. W.

(2015). A way with words: Using language for psychological science in the modern era. In Dimofte

Haugtvedt

Yalch

(Eds.), Consumer psychology in a social media world (pp. 222-236). New York, NY: Routledge.

Brook O’Donnell

Falk

E. B

. (2015). Linking neuroimaging with functional linguistic analysis to understand processes of successful communication. Communication Methods and Measures, 9, 55-77. doi:10.1080/19312458.2014.999751

10.

Brown

Levinson

S. C.

(1987). Politeness: Some universals in language usage. Cambridge, UK: Cambridge University Press.

11.

Burt

R. S.

(2005). Brokerage and closure: An introduction to social capital. New York, NY: Oxford University Press.

12.

Cabral

(2012). Reputation on the Internet. In Peitz

Waldfogel

(Eds.), The Oxford handbook of the digital economy (pp. 343-354). New York, NY: Oxford University Press.

13.

Carrera

Muñoz

Caballero

Fernández

Albarracín

(2012). The present projects past behavior into the future while the past projects attitudes into the future: How verb tense moderates predictors of drinking intentions. Journal of Experimental Social Psychology, 48, 1196-1200. doi:10.1016/j.jesp.2012.04.001

14.

Christakis

N. A.

Fowler

J. H.

(2009). Connected: The surprising power of our social networks and how they shape our lives. New York, NY: Back Bay Books.

15.

Chung

Pennebaker

J. W.

(2007). The psychological functions of function words. In Fiedler

(Ed.), Social communication (pp. 343-359). New York, NY: Psychology Press.

16.

Cohn

M. A.

Mehl

M. R.

Pennebaker

J. W.

(2004). Linguistic markers of psychological change surrounding September 11, 2001. Psychological Science, 15, 687-693. doi:10.1111/j.0956-7976.2004.00741.x

17.

Coleman

J. S.

(1988). Social capital in the creation of human capital. American Journal of Sociology, 94, S95-S120.

18.

Corman

S. R.

Kuhn

McPhee

R. D.

Dooley

K. J.

(2002). Studying complex discursive systems. Human Communication Research, 28, 157-206. doi:10.1111/j.1468-2958.2002.tb00802.x

19.

Coviello

Sohn

Kramer

A. D. I.

Marlow

Franceschetti

Christakis

N. A.

Fowler

J. H.

(2014). Detecting emotional contagion in massive social networks. PLoS ONE, 9, e90315. doi:10.1371/journal.pone.0090315

20.

Dellarocas

(2006). Strategic manipulation of Internet opinion forums: Implications for consumers and firms. Management Science, 52, 1577-1593. doi:10.1287/mnsc.1060.0567

21.

Doerfel

M. L.

Barnett

G. A.

(1999). A semantic network analysis of the International Communication Association. Human Communication Research, 25, 589-603. doi:10.1111/j.1468-2958.1999.tb00463.x

22.

Eichstaedt

J. C.

Schwartz

H. A.

Kern

M. L.

Park

Labarthe

D. R.

Merchant

R. M.

. . . Seligman

M. E. P.

(2015). Psychological language on Twitter predicts county-level heart disease mortality. Psychological Science, 26, 159-169. doi:10.1177/0956797614557867

23.

Eisenberg

E. M.

(1984). Ambiguity as strategy in organizational communication. Communication Monographs, 51, 227-242. doi:10.1080/03637758409390197

24.

Eisenstein

O’Connor

Smith

N. A.

Xing

E. P

. (2010). A latent variable model for geographic lexical variation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (pp. 1277–1287). Association for Computational Linguistics.

25.

Ellison

N. B.

Hancock

J. T.

Toma

C. L.

(2012). Profile as promise: A framework for conceptualizing veracity in online dating self-presentations. New Media & Society, 14, 45-62. doi:10.1177/1461444811410395

26.

Ellison

N. B.

Heino

Gibbs

(2006). Managing impressions online: Self-presentation processes in the online dating environment. Journal of Computer-Mediated Communication, 11, 415-441. doi:10.1111/j.1083-6101.2006.00020.x

27.

Entman

R. M.

(1993). Framing: Toward clarification of a fractured paradigm. Journal of Communication, 43, 51-58. doi:10.1111/j.1460-2466.1993.tb01304.x

28.

Feng

Cai

Crossley

McNamara

D. S.

(2011, May). Simulating human ratings on word concreteness. Paper presented at the Florida Artificial Intelligence Research Society Conference, Palm Beach, FL.

29.

Fujita

Henderson

M. D.

Eng

Trope

Liberman

(2006). Spatial distance and mental construal of social events. Psychological Science, 17, 278-282. doi:10.1111/j.1467-9280.2006.01698.x

30.

González-Bailón

(2013). Social science in the era of big data. Policy & Internet, 5, 147-160. doi:10.1002/1944-2866.POI328

31.

Green

S. E.

Jr. Li

Nohria

(2009). Suspended in self-spun webs of significance: A rhetorical model of institutionalization and institutionally embedded agency. Academy of Management Journal, 52, 11-36. doi:10.5465/AMJ.2009.36461725

32.

Greene

J. O.

McDaniel

T. L.

Buksa

Ravizza

S. M.

(1993). Cognitive processes in the production of multiple-goal messages: Evidence from the temporal characteristics of speech. Western Journal of Communication, 57, 65-86. doi:10.1080/10570319309374431

33.

Hancock

J. T.

Dunham

(2001). Impression formation in computer-mediated communication revisited: An analysis of the breadth and intensity of impression. Communication Research, 28, 325-332. doi:10.1177/009365001028003004

34.

Hart

R. P.

Burks

(1972). Rhetorical sensitivity and social interaction. Speech Monographs, 39, 75-91. doi:10.1080/03637757209375742

35.

Himelboim

(2010). Civil society and online political discourse: The network structure of unrestricted discussions. Communication Research, 38, 634-659. doi:10.1177/0093650210384853

36.

Hopkins

D. J.

King

(2010). A method of automated nonparametric content analysis for social science. American Journal of Political Science, 54, 229-247. doi:10.1111/j.1540-5907.2009.00428.x

37.

Huang

Shen

Contractor

N. S.

(2013). Distance matters: Exploring proximity and homophily in virtual world networks. Decision Support Systems, 55(4), 969–977. https://doi.org/10.1016/j.dss.2013.01.006

38.

Kacewicz

Pennebaker

J. W.

Davis

Jeon

Graesser

A. C.

(2014). Pronoun use reflects standings in social hierarchies. Journal of Language and Social Psychology, 33, 125-143. doi:10.1177/0261927X13502654

39.

Katz

S. J.

Byrne

(2013). Construal level theory of mobile persuasion. Media Psychology, 16, 245-271. doi:10.1080/15213269.2013.798853

40.

Kleinnijenhuis

van den Hooff

Utz

Vermeulen

Huysman

(2010). Social influence in networks of practice: An analysis of organizational communication content. Communication Research, 38, 587-612. doi:10.1177/0093650210385225

41.

Krackhardt

(1994). Constraints on the interactive organization as an ideal type. In Heckscher

Donnellon

(Eds.), The Post-Bureaucratic Organization: New Perspectives on Organizational Change (pp. 211–222). Thousand Oaks, CA: Sage Publications.

42.

Kramer

A. D. I.

Guillory

J. E.

Hancock

J. T.

(2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111, 8788-8790. doi:10.1073/pnas.1320040111

43.

Kray

Gonzalez

(1999). Differential weighting in choice versus advice: I’ll do this, you do that. Journal of Behavioral Decision Making, 12, 201-217. doi:10.1002/(SICI)1099-0771(199909)12:3<207::AID-BDM322>3.0.CO;2-P

44.

Labroo

A. A.

Patrick

V. M.

(2009). Psychological distancing: Why happiness helps you see the big picture. Journal of Consumer Research, 35, 800-809. doi:10.1086/593683

45.

Lammers

J. C.

Barbour

J. B.

(2006). An institutional theory of organizational communication. Communication Theory, 16, 356-377. doi:10.1111/j.1468-2885.2006.00274.x

46.

Larrimore

Jiang

Larrimore

Markowitz

D. M.

Gorski

(2011). Peer to peer lending: The relationship between language features, trustworthiness, and persuasion success. Journal of Applied Communication Research, 39, 19-37. doi:10.1080/00909882.2010.536844

47.

Lazer

Pentland

Adamic

Aral

Barabasi

A. L.

Brewer

. . . Van Alstyne

(2009). Computational social science. Science, 323, 721-723. doi:10.1126/science.1167742

48.

Lerner

J. S.

Tetlock

P. E.

(1999). Accounting for the effects of accountability. Psychological Bulletin, 125, 255-275. doi:10.1037/0033-2909.125.2.255

49.

Hitt

(2008). Self-Selection and information role of online product reviews. Information Systems Research, 19, 456-474. doi:10.1287/isre.1070.0154

50.

Liberman

Förster

(2009). The effect of psychological distance on perceptual level of construal. Cognitive Science, 33, 1330-1341. doi:10.1111/j.1551-6709.2009.01061.x

51.

Lin

Y.-R.

Margolin

Lazer

(2016). Uncovering social semantics from textual traces: A theory-driven approach and evidence from public statements of U.S. Members of Congress. Journal of the Association for Information Science and Technology, 67, 2072-2089. doi:10.1002/asi.23540

52.

Liviatan

Trope

Liberman

(2008). Interpersonal similarity as a social distance dimension: Implications for perception of others’ actions. Journal of Experimental Psychology, 44, 1256-1269. doi:10.1016/j.jesp.2008.04.007

53.

Luca

Zervas

(2016). Fake it till you make it: Reputation, competition, and Yelp review fraud. Management Science, 62, 3412-3427. doi:015.210.1287/mnsc.2304

54.

Magee

J. C.

Milliken

F. J.

Lurie

A. R.

(2010). Power differences in the construal of a crisis: The immediate aftermath of September 11, 2001. Personality and Social Psychology Bulletin, 36, 354-370. doi:10.1177/0146167209360418

55.

Marchetti

(1994). Anthropological invariants in travel behavior. Technological Forecasting & Social Change, 47, 75-88. doi:10.1016/0040-1625(94)90041-8

56.

Margolin

D. B.

Monge

P. R.

(2013). Conceptual retention in epistemic communities: Examining the role of social structure. In Moy

(Ed.), Communication and community (pp. 1-22). New York, NY: Hampton Press.

57.

Markowitz

D. M.

Hancock

J. T.

(2016). Linguistic obfuscation in fraudulent science. Journal of Language and Social Psychology, 35, 435-445. doi:10.1177/0261927x15614605

58.

Marwick

A. E.

boyd

(2011). I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience. New Media & Society, 13, 114-133. doi:10.1177/1461444810365313

59.

Mehl

M. R.

Gill

A. J.

(2010). Automatic text analysis. In Gosling

S. D.

Johnson

J. A.

(Eds.), Advanced methods for conducting online behavioral research (pp. 109-127). Washington, DC: American Psychological Association.

60.

Monge

Contractor

(2003). Theories of communication networks. New York, NY: Oxford University Press.

61.

O’Connor

Balasubramanyan

Routledge

B. R.

Smith

N. A.

(2010, May). From tweets to polls: Linking text sentiment to public opinion time series. Proceedings of the International AAAI Conference on Weblogs and Social Media, Washington, DC.

62.

O’Keefe

B. J.

(1988). The logic of message design: Individual differences in reasoning about communication. Communication Monographs, 55, 80-103. doi:10.1080/03637758809376159

63.

Ott

Choi

Cardie

Hancock

J. T.

(2011, June). Finding deceptive online spam by any stretch of the imagination. Paper presented at the 49th Annual Meeting of the Association for Computational Linguistics, Portland, OR.

64.

Paivio

(1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255-287. doi:10.1037/h0084295

65.

Pennebaker

J. W.

(2011). The secret life of pronouns: What our words say about us. London, England: Bloomsbury Press.

66.

Pennebaker

J. W.

Booth

R. J.

Francis

M. E.

(2007). Linguistic Inquiry and Word Count. Austin, TX. Available from www.liwc.net

67.

Pennebaker

J. W.

Chung

C. K.

Frazee

Lavergne

G. M.

Beaver

D. I.

(2014). When small words foretell academic success: The case of college admissions essays. PLoS ONE, 9, e115844. doi:10.1371/journal.pone.0115844

68.

Preciado

Snijders

T. A. B.

Burk

W. J.

Stattin

Kerr

(2012). Does proximity matter? Distance dependence of adolescent friendships. Social Networks, 34, 18-31. doi: http://dx.doi.org/10.1016/j.socnet.2011.01.002

69.

Rice

R. E.

Danowski

J. A.

(1993). Is it really just like a fancy answering machine? Comparing semantic networks of different types of voice mail users. Journal of Business Communication, 3, 369-397. doi:10.1177/002194369303000401

70.

Rudder

(2014). Dataclysm: Who we are (When we think no one’s looking). New York, NY: Crown Publishers.

71.

Salovey

(1992). Mood-induced self-focused attention. Journal of Personality and Social Psychology, 62, 699-707. doi:10.1037/0022-3514.62.4.699

72.

Schafer

Victor

D. G.

(2000). The future mobility of the world population. Transportation Research Part A: Policy and Practice, 34, 171-205. doi:10.1016/S0965-8564(98)00071-8

73.

Schuldt

J. P.

(2013). Does green mean healthy? Nutrition label color affects perceptions of healthfulness. Health Communication, 28, 814-821. doi:10.1080/10410236.2012.725270

74.

Silvia

P. J.

Abele

A. E.

(2002). Can positive affect induce self-focused attention? Methodological and measurement issues. Cognition and Emotion, 16, 845-853. doi:10.1080/02699930143000671

75.

Smith

P. K.

Trope

(2006). You focus on the forest when you’re in charge of the trees: Power priming and abstract information processing. Journal of Personality and Social Psychology, 90, 578-596. doi:10.1037/0022-3514.90.4.578

76.

Smith

S. W.

Yoo

J. H.

Walther

(2008). Self-presentation. In Donsbach

(Ed.), The international encyclopedia of communication (Vol. 10, pp. 4555-4559). Oxford, UK: Wiley-Blackwell.

77.

Snijders

T. A. B.

van de Bunt

G. G.

Steglich

C. E. G.

(2010). Introduction to stochastic actor-based models for network dynamics. Social Networks, 32, 44-60. doi:10.1016/j.socnet.2009.02.004

78.

Sommerfeld

R. D.

Krambeck

H.-J.

Milinski

(2008). Multiple gossip statements and their effect on reputation and trustworthiness. Proceedings of the Royal Society B: Biological Sciences, 275, 2529-2536. doi:10.1098/rspb.2008.0762

79.

Stirman

S. W.

Pennebaker

J. W.

(2001). Word use in the poetry of suicidal and nonsuicidal poets. Psychosomatic Medicine, 63, 517-522. doi:10.1097/00006842-200107000-00001

80.

Toma

C. L.

Hancock

J. T.

(2010). Looks and lies: The role of physical attractiveness in online dating self-presentation and deception. Communication Research, 37, 335-351. doi:10.1177/0093650209356437

81.

Toma

C. L.

Hancock

J. T.

(2012). What lies beneath: The linguistic traces of deception in online dating profiles. Journal of Communication, 62, 78-97. doi:10.1111/j.1460-2466.2011.01619.x

82.

Toulmin

(1958). The uses of argument. New York, NY: Cambridge University Press.

83.

Trope

Liberman

(2010). Construal level theory of psychological distance. Psychological Review, 117, 440-463. doi:10.1037/a0018963

84.

Turner

J. H.

(2007). Human emotions. New York, NY: Routledge.

85.

Walther

J. B.

(1996). Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal interaction. Communication Research, 1, 3-43. doi:10.1177/009365096023001001

86.

Walther

J. B.

Jang

J.-w.

(2012). Communication processes in participatory websites. Journal of Computer-Mediated Communication, 18, 2-15. doi:10.1111/j.1083-6101.2012.01592.x

87.

Weber

Eden

Huskey

Mangus

J. M.

Falk

(2015). Bridging media psychology and cognitive neuroscience: Challenges and opportunities. Journal of Media Psychology, 27, 146-156. doi:10.1027/1864-1105/a000163

88.

Yates

Orlikowski

W. J.

(1992). Genres of organizational communication: A structurational approach to studying communication and media. The Academy of Management Review, 17, 299-326. doi:10.5465/AMR.1992.4279545