Abstract
The relevance and recall of information sources, particularly those used in the prepurchase hotel search process, have not been fully explored. As they are critical to inform the final hotel booking, it is essential to examine the relevance and recall of information sources used by customers. Thus, the main aim of this article is to investigate both relevance, in this case measured by the pages viewed, and recall, gauged by both aided and unaided recall of keywords used and websites visited during an online session. An exploratory, observation method that is “goal-directed” is the main research instrument with both qualitative and quantitative data collected. This method is supplemented with questionnaires to measure both implicit and explicit recall. Results show that online travel agencies are more relevant than both search engines and travel-related social media, based on the number of pages viewed. The recall of both keywords and websites was low, both for aided recall and unaided recall.
Introduction
The Internet and empowered consumers have created both marketing opportunities and challenges for the hotel sector. The opportunities include the exposure to global customers, instant reservations, and the development of customized services, while the challenges include working with third-party intermediaries and channel management (O’Connor and Frew 2002; Murphy, Mascardo, and Benckendorff 2007; Pan, Zhang, and Law 2013), managing online reputations on review sites, managing user-generated content on social media sites (Sparks and Browning 2011; Vermeulen and Seegers 2009), and search engine marketing (Paraskevas et al. 2011; Fesenmaier et al. 2011).
The Internet has spawned a vast volume of information and content that consumers can now exploit when purchasing products and services (Beritelli, Bieger, and Laesser 2007). Hotel customers use this vast pool of information through a range of channels, for example, search engines, comparison websites, online travel agencies (OTAs), and the hotel’s own/brand sites not only to book but also to search for information. This dual role of the channels means that not only do they facilitate a booking but also provide information that influences purchase (O’Connor and Frew 2002). Recent research has focused heavily on the role of online reviews, user-generated content, and social media in general (Filieri and McLeay 2014; Xiang et al. 2014). Though some research has been conducted on the information communication role of websites, they tend to focus specifically on travelers and destinations (Jun and Vogt 2013; Tang, Jang, and Morrison 2012). No specific investigation on the range of information sources has been conducted for the hotel sector, which involves “complex decision making that requires an extensive informational search” (Filieri and McLeay 2014, p. 46).
Relevance is defined as a measure of the effectiveness of a contact between a source and a receiver in a communication process (Saracevic 1975, 2007). It is subjective and is related to contexts, goals, and tasks (Sheinin, Varki, and Ashley 2011; Pan et al. 2007). While researchers have investigated information sources, and some of the dynamics between information sources, in the prepurchase search stage, the relevance of information sources has not been fully examined. Evidence already exists that there is some contribution from social media to hotel bookings (Anderson 2012; Filieri and McLeay 2014), but social media have yet to provide a volume of direct, qualified traffic to hotels. This is an important factor to investigate as marketing investment in social media increases while hotels struggle to manage their social presence and, at the same time, optimize multiple managed and unmanaged distribution channels to connect to consumers. Scarce resources must be invested in these channels and information sources that yield a return, however challenging this may be to measure.
Furthermore, bookings are rarely completed in one online session. Hoon (2014) reports that Expedia customers conduct 48 searches before a hotel is finally booked, while ThinkwithGoogle (2013) reports that travelers plan travel in micro-increments throughout the purchase process and switch between devices. Hence, it is important to understand how customers recall and reuse their information search findings. Furthermore, it is critical for hotel marketers that information sources are easily recalled and are relevant, and thus likely to be used again in future information search and worth the investment. Recall may also minimize the need for search engine optimization (SEO) and keyword purchase from the search engines, on the part of the hoteliers, to redirect searchers, every time, to preferred information sources.
Thus, the focus of this study is to strengthen the understanding of the use of information sources, albeit within a simulated situation, and address the overarching research question, What is the relevance and recall of key information sources that inform the hotel booking process?
As the nature of the human–computer interface is constantly evolving and hotel customers have access to an increasing number of external sources of information, it is critical to scrutinize how these consumers navigate through this expanse of information. By examining the relevance and recall of information sources, this study will contribute not only to the literature but also to methodologies and practice. The main contribution to the literature is in the examination of both recall and relevance. Recall is rarely examined and relevance is only studied selectively. Though we use the information sources identified by previous researchers (Xiang et al. 2014; Choi et al. 2012), we also introduce an additional layer of investigation that attempts to measure the relevance and recall of these information sources. We focus on a specific product, that is, accommodation, while most of the literature, till now, has focused on the wider tourism sector (Xiang et al. 2014; Fesenmaier et al. 2011; Pan et al. 2011; Grønflaten 2009). We also attempt to contextualize the search process within the parameters of the prepurchase stage of booking a hotel room. Previous researchers have measured reported data (Xiang et al. 2014; Filieri and McLeay 2014; Fesenmaier et al. 2011), whereas our contribution, in methodological terms, is in adopting a more experimental, goal-directed, and investigative approach to discover actual online information usage behaviors, rather than reported or intended ones. This answers the call from researchers in this domain (Bieger and Laesser 2004; Fesenmaier et al. 2011) to adopt a more pragmatic research design. This is achieved by simulating the online search process for a hotel room booking and a combination of observing online hotel search behaviors and questionnaire surveys, which will then be used to identify information sources used to evaluate the relevance of information sources and measure recall. To our knowledge, this is the only study that has attempted to measure relevance and recall of information sources in a specific hotel booking setting. As such, and as a contribution to practice, it provides managers with a deeper insight into how consumers engage with a range of hotel-related websites.
This article is organized in five sections. Following the introduction, related literature in information sources, search, relevance, and recall is reviewed to provide the theoretical foundation of this research. The method section explains the two-phase research design, data collection, and data analysis procedure. In the results section, we present the findings. Finally, the discussion, implications of the findings, and research limitations are elaborated.
Literature Review
In our study, we draw on a number of previous studies that investigate the booking process, mostly rooted in consumer behavior theory and more recently in online consumer behavior. Here, we will review the relevant literature on information search and sources, relevance, recall, and keywords in a travel-related context.
Information Search and Sources
When making a product or service purchase, a buyer usually follows certain well-established steps. These steps have been modeled by various authors, including Solomon (2010) as shown in Figure 1.

The stages of the purchase process.
Customers consult information sources, at the information search stage, prior to buying and these can be broadly categorized into internal and external sources (Lee and Cranage 2010; Jacobsen and Munar 2012; Hyde 2008; Pan and Fesenmaier, 2006). Internal sources are dependent on personal experiences and knowledge, passive reactions, and recall from memory. External sources are dominated by commercial marketing actions, for example, advertising, brochures, websites, and newspapers (Kim, Lehto, and Morrison 2007; Ramkissoon and Nunkoo 2008). Generally, internal resources are used first and, if insufficient, consumers resort to external sources for information (Lee and Cranage 2010; Chen and Gursoy 2000; Fodness and Murray 1997, 1999).
Previous research reports that the Internet is now the most important information source for travel planning and hotel booking (Xiang et al. 2014; Xiang and Gretzel 2010; Toh, DeKay, and Raven 2011; Jun, Vogt, and MacKay 2010). Indeed, more travelers search both online and offline for accommodation than for other travel-related products such as flights and car rental (Jun, Vogt, and MacKay 2007). Based on the perceived usefulness and need, Fesenmaier et al. (2011) identify and rank travel-related websites in the following order; general search engines, suppliers’ sites, OTAs, destination sites, general travel sites, travel search engines, travel guidebook sites, community sites, newspaper/magazine sites, consumer content-generated sites, and social networking sites. For American travelers, OTAs’ and suppliers’ sites remain the most used information sources, with lodging, air tickets, and car rental dominating travel purchases (Xiang et al. 2014). Verma, Stock, and McCarthy (2012) state that travelers use different websites during different stages. The search engines are important at the beginning of information search, while review sites, suppliers’ sites, and OTA sites are essential during the comparing and considering stages. The suppliers’ and OTA sites are important at the booking stage. Anderson (2011, 2012) reported that when customers book on the supplier’s site, 75% of them have had previously visited an OTA and 83% a search engine before making reservations.
Gretzel and Yoo (2008) report that travel-related user-generated content (UGC), and particularly reviews, do affect the purchase decisions for hotels at the information search stage. They emphasize their impact on search and report that 10% of the first 10 pages of Google search results are social media sites and the 20 top social media sites contributed to 50% of the search results, which further emphasize the influence of social media sites. Recent research by Xiang and Gretzel (2010) classified travel-related social media into six types, consumer review sites (e.g., TripAdvisor), social networking sites (e.g., Facebook), media-sharing sites (e.g.YouTube or Flickr), virtual community sites (e.g., LonelyPlanet and IgoUgo), personal blogs and blog aggregators (blogspot.com), and “others.” The intention to use social media (Parra-López et al. 2011; Xiang et al. 2014; Schroeder and Pennington-Gray 2014), the motivation to read online hotel reviews (Kim, Mattila, and Baloglu 2011), the characteristics of helpful TripAdvisor reviewers (Lee, Law, and Murphy 2011), and the economic benefit of positive e-word-of-mouth (Ye et al. 2011) have all been addressed by researchers in the context of travel.
While globalization and the increasing individualism of society continue to fuel the rise of online social networks with UGC and with Facebook now the most visited website (Experian Hitwise 2013), hotels are also investing time and money into their social media presence, in preference to other channels of communication (HotelMarketing 2012).
Mangold and Faulds (2009) argue that social media plays a hybrid role on integrated marketing communication, and this moderating role is supported and reemphasized by Chu and Kim (2011), who state that “social media relationships must be taken into account” (p. 67). Yet, other researchers argue that social media are used more at the post-purchase stage of consumption than at the prepurchase stage (Litvin and Hoffman 2012). Though there is an increased use of social media, there remains uncertainty regarding the specific role and influence of these channels (Jordan, Norman, and Vogt 2013). Filieri and McLeay (2014) more recently reported that social media, particularly online reviews, are strong predictors of adoption of information and are relevant to both peripheral and central routes of persuasion in accommodation bookings. The complexity of these layers is investigated by Rihova et al. (2013), suggesting a more complex relationship between social connections and consumption, that is, “socially immersive co-creation layers”, that are proving difficult to unravel, let alone measure.
Grønflaten (2009) advocates that researchers should distinguish between information sources (travel agents vs. suppliers) and information channels (face-to-face vs. the Internet) and consider travel style, age, and nationality to predict travelers’ preferred communication channels. Wu, Law, and Jiang (2013) advocate using the weight-of-evidence (WOE) data mining technique to investigate hotel website browsers, that is, information seekers, and purchasers.
Existing research reveals that travelers tend to use a range of information sources, including social media, and examines the interactions between a limited number of information sources (Litvin and Hoffman 2012; Verma, Stock, and McCarthy 2012). They differ in information sources addressed (Filieri and McLeay 2014; Fesenmaier et al. 2011; Grønflaten 2009) and tend to focus on one or two information sources (Xiang et al. 2014; Pan, Zhang, and Law 2013; Park and Chung 2009; Grønflaten 2009), but rarely address the relevance and recall of information sources.
Relevance and Recall
Relevance is a key notion in information retrieval. Researchers define relevance as a measure of the effectiveness of a contact between a source and a receiver in a communication process (Saracevic 1975, 2007). It represents subjective judgments of the likelihood that the information is related to the answer of the question or the goal of a search task (Sheinin, Varki, and Ashley 2011; Pan et al. 2007; Filieri and McLeay 2014).
Relevance tends to be subjective and situational (Sheinin, Varki, and Ashley 2011; Pan et al. 2007). Instead of measuring relevance per se, researchers have used other evidence online, that is, frequency of visits, time spent viewing, pages viewed, recall, recommendation and repurchase, to infer relevance (Saracevic 2007; Jung, Herlocker, and Webster 2007; Danaher and Mullarkey 2003). To measure relevance, researchers have employed both implicit and explicit feedback techniques (Jung, Herlocker, and Webster 2007). Explicit techniques are more verbal, conscious, contextual, and holistic, for example, using a questionnaire or interview after exposure (Filieri and McLeay 2014; Jansen and Spink 2003), whereas implicit techniques are more nonverbal and procedural, for example, observation, automatic tracking/logging frequency of visits, and time spent viewing (Jung, Herlocker, and Webster 2007; Danaher and Mullarkey 2003; Couture et al. 2013). Kelly and Teevan (2003) select the implicit techniques over the explicit techniques because of the advantages of low (time and effort) costs to the subjects and the possibility of collecting a large quantity of data. Kelly and Teevan (2003) also identify several indicators of relevance for online behaviors. For example, to retain relevant information, the observable implicit behaviors might include printing, bookmarking, saving, deleting, purchasing, e-mailing, or subscribing to the information source. Jung, Herlocker, and Webster (2007) state that clickstream data that track clicks, page views, and websites visited in a search session can be considered as indicators of relevance. Bilenko and White (2008) argue that users’ browsing activity reflects implicit topical relevancy of the visited pages. More recently, Montesi (2010) states that user relevance judgments can be inferred through the time spent reading a webpage, or actions such as clicking, printing, saving, or selecting. The relevance factor is highlighted by Taylor, Lewin, and Strutton (2011), who caution that the relevance of advertisements on social media sites cannot simply be measured in isolation and should be carefully aligned to the interests and motivations of individuals.
Potential customers rarely complete their search in one session nor do they spend the same amount of time on every website (Verma, Stock, and McCarthy 2012; Choi et al. 2012; Hoon 2014). Anderson (2011, 2012) reports that customers who book on the suppliers’ sites tend to have multiple visits to OTAs and search engines before booking. Therefore, it is important to understand how hotel customers recall their previous searches to inform ongoing search, even when the hotel secures the final booking.
Recall (or retrieval) is the process whereby information is recovered from long-term memory. Recall, especially the process of recovering information by mental effort, is essential in marketing to establish product characteristics and strategic positioning. Nonetheless, “mental effort” is not the only means of recall. Sparrow, Liu, and Wegner (2011) state that when people expect to have future access to information, they have a lower rate of recall of the information and an enhanced recall instead for where to access it. Sparrow, Liu, and Wegner (2011) also report that the Internet has become the “external memory” for many people. Danaher and Mullarkey (2003) report specifically on recall of web advertising and that when participants are in “goal-directed” mode, recall is less likely than in general browsing/surfing. The recall-based approach is widely used in post-test marketing and has a range of metrics associated with it (Wells 2000) but is often simply measured, for example, as a percentage of total actual recall events/total possible recall events (PwC 2013).
Keywords and Information Search
Of course, self-efficacy, context, interest, and motivation play a role in the recall of both information sources and keywords (Kim and Kim 2005; Wu, Lin, and Lin 2011). Even when consumers do recall the exact web address (uniform resource locator [URL]), they may also use the search function for expediency (Blankenbaker and Mishra 2009). Pan and Fesenmaier (2006) discovered that customers also search in clusters to minimize intercluster information-seeking costs and classified e-travelers into two categories at the prepurchase stage, that is, “direct entry,” those who type in URLs and “referring entry,” those from other websites. They noted different consequent search behaviors, such as length of stay.
Keyword searching is the common start point when looking for information (Xiang and Gretzel 2010; ETC 2013). In the travel sector, the most common keyword search terms identified by Xiang, Wöber, and Fesenmaier (2008) are; “accommodation,” “hotel,” “activities,” “attractions,” “park,” “events,” “tourism,” “restaurant,” “shopping,” and “nightlife.” Xiang and Gretzel (2010) report that keywords such as “nightlife” and “restaurant” generated the most social media–related results, 21.5% and 14.3% respectively, with “hotel” and “accommodation” at less than 7%. More recently, the ETC (2013) states that half of all online travel search is driven by 13 words, with search terms dominated by “cheap” and “discount,” with review sites such as TripAdvisor more closely related to keywords such as “hotels,” not social network sites. Xiang and Pan (2011) report that, at least in travel, keyword searches tend to be short, consisting of fewer than four keywords.
Consumers may believe that they have less need to recall websites, relying on keyword searches to find sites and content. Thus, hotel customers may fail to take explicit action to recall websites, for example, bookmark. The practical boundaries of what can be searched online and the way it is represented is dictated largely by the search engines, particularly the dominant influence of Google (Xiang and Gretzel 2010). Pan, Zhang, and Law (2013, p. 82), in the context of hotel choice, “demonstrate the importance of search . . . for online consumers.” Search engine optimization (SEO) is the process of using keywords in the text, heading, and content of a website that impacts on “natural” or un-paid (“organic”) search engine return pages (SERPs) and thus visibility (Park and Chung 2009). Research shows that customers do not understand the impact of SEO and the changing Google algorithm, which means that the same search keywords may result in different search results (Xiang and Gretzel 2010; Pan, Zhang, and Law 2013; Park and Chung 2009). Yet, while previous studies focus on identifying keywords used in the search process they fail to consider the fact that customers conduct many searches along a time continuum that may last days or months, that it is an iterative process and earlier SERPs inform consequent search terms. As keywords are generally used at the start of search, and in different formats throughout the booking process, the recall of keywords is critical to explore.
In summary, though researchers have identified information sources and frequently used keywords used in the travel search process, the relevance and recall of these information sources and keywords has not been fully investigated for the hotel booking process. Hence, one of the research objectives of this research is to investigate the relevance of information sources by using, as a surrogate for relevancy, the number of pages viewed and the descriptive statistics of keywords used in search sessions. Furthermore, customers use a range of information sources, over a period of time; therefore, which information sources and keywords customers are able to recall is critical and has not been explored by previous researchers. This exploratory research will utilize both implicit and explicit measures to examine relevance and recall, employing observation techniques and questionnaires. The research objectives are more specifically to
identify what are the key sources of information that inform hotel booking;
investigate the relevance of information sources, particularly travel-related social media websites, in the hotel room booking process;
examine the recall of information sources and keywords in the hotel room booking process; and
discover if social media merits the marketing investment, particularly at the prepurchase stage.
Methods
Research Design
Data collection was conducted in two main phases for this investigation. The first phase used a laboratory setting where subjects complete an initial questionnaire and read a brief scenario. Next, they were observed, and their screens recorded, in the process of searching and selecting a hotel room. The second phase took place one week later where a further two questionnaires were administered to the same group to measure, firstly, unaided recall and, secondly, aided recall of the previous week’s hotel room search. Protocols were designed by the research team to ensure that there is consistency in the way the experiment and recordings are conducted, the purpose of the research is explicit, written instructions for participants are clear and unambiguous, and informed written consent is acquired from participants. The subjects are informed that the session would be recorded but their anonymity and confidentiality would be maintained and that they are free to withdraw from the study at any point in time. Subjects received no incentive for their participation. Observation data were recorded during the online search by Desktop Screen Record 5, which subjects installed on their own computers before commencement of the study. This software records the desktop screen as a video file that can be used to track the entire process of the directed online search, including websites visited, the time spent, pages viewed, the sequence of website visits, the keywords typed, and mouse movements. A nonparticipant observation method that is “goal-directed” in a laboratory setting is employed here, with two researchers and one assistant present at all times to observe and maintain the integrity of the observations. Specifically, our observations were overt, structured, undisguised, and time constrained, as advocated by Churchill and Iacobucci (2009), Bruce, Jones, and Dumais (2004), and Loonam and O’Loughlin (2008). Both qualitative and quantitative data are collected through these observations, which are supported by the two follow-up questionnaire surveys one week later. The two-phase process and one-week interval in the research design ensured the continuity of subject participation (Collis and Hussey 2009).
Observation and Surveys
Observation is the most appropriate method when the following conditions are present: the research problem is concerned with human meanings and interactions, the research problem can be addressed by qualitative methods, the data gathered are observable, the researchers can gain access to an appropriate setting, and the event is sufficiently limited in size (Jorgensen 1989). The survey methodology is one of the most widely used in tourism research to investigate travelers’ behavior and is used here to supplement the observations (Pan 2010).
Fesenmaier et al. (2011) and Bieger and Laesser (2004) advocate that research about information sources and search behavior should employ both survey and real-time observations of search behavior. Observation researchers have used cookie tracking or actual response (Danaher and Mullarkey 2003), click-stream log data (Lee and Cranage 2010; Park and Chung 2009; Pan, Litvin, and O’Donnell 2007), and eye-tracking observations (Pan, Zhang, and Law 2013) to collect qualitative and/or quantitative data. Here, both methods are applied with observation to record the actualities of hotel search behavior and surveys to supplement and complete the research design with the respondents reporting data on demographics and recall behaviors (Collis and Hussey 2009).
The Two Phases of the Data Collection Process
The first phase recruited 19 participants who were final-year undergraduate students. Instructions to subjects commences with setting the starting screen to record the search process using Desktop Screen Record 5, and to read a scenario description. There are two components at this stage, a preliminary questionnaire and the recording of the search process.
Phase 1: As suggested by Loonam and O’Loughlin (2008), a questionnaire before the observations can reduce the initial anxiety for observation participants. The participants completed a questionnaire, which included their demographic profile and Internet usage and proficiency. Next, in order to simulate a booking, the following scenario, as recommended by Lee and Cranage (2010) and Pan and Fesenmaier (2006), was given out and students then carried out the search which was recorded.
Your grandparents gave you a graduation gift—one week vacation in Tokyo! They gave you enough loyalty points to cover two round-trip tickets between Geneva and Tokyo. Your grandparents will cover the hotel charges, but will not be glad to see you in the most expensive hotel in town. On the other hand, they don’t expect you to stay in a dangerous or dirty place to save money. You decide to go with your best friend. You will search for one room for two people in Tokyo for 4 nights from July 16th–20th. You will make the final decision with your friend, so you will only search for a group of choices. Stop when you find a small group of hotels, exit without making reservations. Your grandparents will book this for you in 1 week time, and they will need your information. You have 30 minutes to complete the search.
It is critical in this scenario that the travel element and previous destination experiences are minimized, thus focusing attention on the hotel booking process. The subjects are also informed that the final information gathered will be requested a week later to make the booking.
Phase 2 (one week later): There are two components at this stage. One week after the observation task, we distributed a written “unaided” recall questionnaire to elicit the web sites visited, the final group of hotels, and keywords used. After subjects completed the “unaided” recall questionnaire, we then manually distributed an “aided” recall questionnaire. The “aided” recall questionnaire listed all the actual sites visited by all subjects from the recordings, and each respondent indicated the sites s/he recalled during the hotel search task.
Data Analysis
Each recording was scrutinized by both researchers, and a written transcript was completed for every video file. These transcripts detailed the sequence of web sites visited, pages viewed, the keywords typed, and the presence of any bookmarking or activities that may facilitate information retrieval (Pan and Fesenmaier 2006). The transcripts were next verified by the second researcher to ensure for completeness and accuracy. Next, the researchers aggregated all transcripts and identified the number of sites visited, the number of pages viewed, the number of keywords typed, and the final hotel choices (Pan and Fesenmaier 2006).
For every participant, responses from the unaided recall and aided recall questionnaires were compared with his/her actual behaviors identified from the relevant transcript and video recording. Statistics such as frequencies and percentages, along with subject demographic profile and Internet usage and proficiency, were calculated. The quantitative data are reported in graphical format and the qualitative data are used for illustrative purposes.
Results
Participants Profile
The sample comprises of 68% female respondents with an average age of 21.8 years. The proportion of females is higher than 50%, but comparable to other observational studies, for example, Pan, Zhang, and Law (2013). The sample size of 19 is similar to previous observational studies, such as 22 (Pan, Litvin, and O’Donnell 2007) and 15 (Pan and Fesenmaier 2006). These subjects spend 5.4 hours per week online for leisure purposes, which is in line with other studies that report 225.3 minutes a week for leisure purposes (Vandelanotte et al. 2009). In addition, 89% had booked a hotel room online in the last 12 months and they had an average of 10 years’ Internet experience. All, but one, belong to a social network site and use them daily, with most, 61%, having used these sites for 4 years or more. However, few post comments on online communities, 21%, but 68% do consult specific travel review sites such as TripAdvisor.
Information Sources, Sites Visited, and Pages Viewed
Table 1 lists “No. of Participants Visited,” “No. of Sites Visited,” and “No. of Pages Viewed.” The “No. of Participants Visited” column shows the usage of different categories of information sources, and indicates the popularity of these sites. The most visited information source category is the general search engines, followed by OTAs and consumer review sites. On average, during the tracked session, participants visited 5 websites, with a range of 2 to 17 websites overall. This finding supports previous research where travelers use multiple information sources, and further justifies the need to understand the influence of information sources.
Sites Visited and Pages Viewed.
Note: From phase 1 tracked recordings. OTA = online travel agencies.
Though participants used a generic search engine, predominately Google, to start the research, most “anchored” (i.e., reverted continuously) their search to the OTAs or a consumer review site. This result confirms the findings from Pan and Fesenmaier (2006) that users tended to look for information in clusters to minimize time spent on “inter-cluster seeking costs.”
None of the participants used travel community sites (i.e., LonelyPlanet or IgoUgo) or travel-specific search engines (i.e., Kayak). Regarding other travel-related social media sites, 11 participants used consumer review sites (i.e., TripAdvisor), 2 used airBnB and 1 used WikiTravel, 1 used YouTube, and 1 used a blog site. For presentation purposes, media-sharing sites, blog sites, and social networking sites are combined under “travel-related social media.” Furthermore, currency exchange sites (used by 3 participants) and general news media (used by 1 participant) are combined under “others.”
“No. of Sites Visited” shows the number of accessed websites per information source category, while “No. of Pages Viewed” shows the total number of pages viewed per information source category. A total of 53 websites and 599 pages were visited by 19 participants. In terms of all 53 sites, participants visited a total of 2 search engines, 1 consumer review site, 21 OTAs, and 15 suppliers’ sites. In terms of pages viewed, participants collectively viewed 284 pages from OTA sites, 105 pages from general search engines, and 104 pages from consumer review sites. We also present the share of pages viewed, with OTAs at 47%.
As demonstrated in Table 1 and Figure 2, there is a dominance of search engines and review sites, as well as a multiplicity of suppliers’ sites and OTA sites. Except for 2 subjects, most participants used a generic search engine. From the observed screen data, 1 participant used Bing and the rest used Google. Similar dominance has been observed for the consumer review sites, with 11 participants visiting one commercial review site, TripAdvisor. These observations suggest that the search engine sites and customer review sites are dominated by the two well-known major players in these areas, Google and TripAdvisor. On the contrary, there were no similar dominant OTA or supplier sites. Of the 16 participants who visited a total of 21 OTA sites, 5 used only a single, consistent OTA site while the rest spread their search among two to six OTA sites. This finding is similar to that of Anderson (2011), who reported that 22% of Expedia customers only visit Expedia, which implies that the remaining 78% visit other OTAs as well. Eight participants visited a total of 15 suppliers’ sites, with an average of 2 sites and a range of 1 to 5 sites, with no single supplier dominating during search mode.

Comparison of the proportion of sites visited and pages viewed.
The numbers of pages viewed are treated as a measure of the relevance of the website; that is, the more pages viewed, the more relevant the information source. Based on the numbers of pages viewed, the OTAs are the most relevant with 284 pages viewed. In addition, participants viewed 104 pages on TripAdvisor and 105 search engine pages. Although most content viewed by participants are from OTAs, these were dispersed among 21 OTAs, indicating the range of choice of OTAs and that participants do not rely on one OTA for information.
Travel-Related Social Media
Table 1 and Figure 2 show, overall, that the total number of all travel-related social media sites, excluding TripAdvisor, is 10% of all websites visited, and 19% of total pages viewed. The most relevant travel social media site is TripAdvisor, as reported by Xiang and Gretzel (2010), with 104 pages viewed. Given the dominant contribution of TripAdvisor to the total pages viewed for travel social media sites, and the relative low contribution from other travel social media sites, we are cautious of the overall “relevance” of all travel-related social media at the prepurchase stage of hotel booking, as most are explained by TripAdvisor. Similar to the position Google has in the search domain, TripAdvisor may have a more relevant role at the prepurchase stage as a trusted information source. Yet, the relevance of TripAdvisor cannot be generalized to other travel-related social media, based on this research.
Recall of Sites
Figure 3 reports the overall sites used for both unaided recall and aided recall results. The results show that many participants do not recall the sites visited in phase 1. The only successful recall, both unaided and aided, is the official destination website, hosted by the Tokyo Government, and recalled by two participants. Participants had a high unaided recall rate for TripAdvisor, categorized as consumer review sites, with 10 participants recalling this site. Participants used 16 OTAs, but only recalled 14 unaided. Additionally, two participants identified the wrong OTAs in the unaided recall, which may indicate that participants have some difficulty differentiating between OTAs.

Unaided and aided recall results.
Furthermore, the results reveal that aided recall did not generally outperform the unaided recall and participants only improved their recall, when aided, for general search engines. Seventeen participants used search engines, with only eight able to recall unaided, but 14 identified search engines in aided recall. The failure to recall the search engine, in the unaided survey, may indicate that using a search engine is almost becoming automatic or unconscious. Only when given the aided list, did some participants remember they had used a search engine.
Given the listed sites in the aided recall, five participants who used OTA sites and three participants who used suppliers’ sites failed to identify them. Conversely, six participants identified OTA sites that they had not used, while five participants identified suppliers’ sites they had not used, that is, false recall. These results might again demonstrate a perception of the OTA and supplier market that is homogenous and undifferentiated, packed by a large number of players that creates confusion among hotel customers.
Recall of Hotels
Participants were also asked to recall their specific hotel selection. Only two subjects remembered their final hotel choices, indicating that participants seem to have better recall of most other websites than the hotels finally selected! This may confirm the findings from Sparrow, Liu, and Wegner (2011) that when people expect to have future access to information, they have a lower rate of recall on the information and instead have enhanced recall for where to access it. This lack of recall also supports the results of Danaher and Mullarkey (2003), who found that goal-directed activities diminish recall.
Keywords
Participants conducted a total of 71 keyword search queries, with an average of 4.7 searches and a range of between 1 and 17 searches. These queries range from one word to a maximum of six words, with an average of 2.7 words, which is comparable to the findings of Xiang and Pan (2011). Seven of the 11 one-word queries are OTAs, hotel, and travel-specific website brand names. This may confirm the research by Blankenbaker and Mishra (2009) that reports that some web surfers use Google navigationally; that is, they type into the Google toolbar instead of typing the URL directly into a browser. The researchers also observed that the low keyword usage is related to those participants who copied and pasted into the Google search box, instead of typing them in. In terms of frequencies of keywords, the most popular word is the destination, “Tokyo,” which appeared in 62% of search terms. The second most popular word is “hotel,” which appeared in 51% of search terms used. The most popular phrases are “hotel(s) in Tokyo,” “hotel(s) Tokyo,” and “Tokyo hotel(s).”
Other Findings
Our participants did search for specific, branded hotel names. These keyword searches contained hotel names but were mostly derived from a previous search on OTAs or TripAdvisor or online travel magazines, where the hotel names were first found and then cut and pasted in the search box to use in consequent searches in Google. No participants went straight to any hotel sites at the very beginning of search process. This suggests that prospective customers will revert to OTAs as a the primary information source and OTAs may be perceived as a reliable, quick source of hotel information in preference to information on hotel brands’ sites. This may be explained by the advanced filtering functionalities offered on these websites or because of the wider choices offered in most OTAs.
Except for TripAdvisor, participants did not recall they had used any travel-related social media sites in either aided or unaided recall survey. Because of the low number, five, of participants who actually used any travel-related social media sites, this finding cannot infer that the low recall rate was related to individual participant demographics or any general behavior patterns. Implicit recall actions, such as bookmarks, e-mail, printing, forwarding, or saving have not been observed in any of the 19 video recordings.
Discussion, Conclusions, and Limitations
Discussion
This research aimed to investigate the relevance of information sources by observing the actual usage behaviors and to understand the recall behavior in the hotel booking information search process. The results shown in Table 1 and Figure 2 suggest there is no simple answer to relevancy. The most used category of information sources is the search engine, with all but two participants using a search engine. Though Google dominates in terms of visitors, their visits are inevitably of short duration and are aligned to the “referring entry” mode, providing a gateway to more detailed information sources. OTAs and TripAdvisor are the critical information sources, that is, 21 OTAs with 284 pages viewed and TripAdvisor with 104 pages viewed. This indicates that TripAdvisor may have a wider role in informing the purchase process beyond reviews, that is, pricing and channel information. Participants exhibit no real “functional difference” between OTAs and TripAdvisor as information sources, at least during the initial information search stage.
Even though some researchers define TripAdvisor as a form of social media, here, when we exclude TripAdvisor, then the main results indicate that search engines, OTAs, and suppliers’ sites are more relevant than travel-related social media sites in the hotel booking search process. This finding reinforces the low relevance of social media in the travel/hotel prepurchase search context, as reported by Fesenmaier et al. (2011) and Litvin and Hoffman (2012), who report that social media sites are more useful at the postpurchase stage.
Destinations are memorable and have a high recall, even if only a small percentage of this sample used the “official” destination website, and this study confirms that destination + hotel is the most popular keyword search (Fesenmaier et al. 2011; ETC 2013)
TripAdvisor appears to be highly relevant and, given the exploratory nature and small sample, more research is needed to determine if other commercial review sites (epinions, holidaychek, etc.) are as relevant as TripAdvisor in the hotel search process. Additionally, TripAdvisor now displays hotel prices from OTAs on one screen, instead of multiple pop-up screens. By making price information more visible, and thus price comparisons much easier, TripAdvisor is becoming more than a consumer review site and may use its prepurchase relevancy to encroach on bookings with their recently launched TripConnect. The positioning and functionality of TripAdvisor is now challenging the existing OTAs.
A further aim is to examine the recall of websites in hotel booking decisions. Subjects in this research have higher recall of TripAdvisor and OTAs in comparison to hotel suppliers’ sites. This could be explained by the fact hotel rooms are now viewed as commodities with the hotel brand now less relevant and less recalled. Brand names were only searched for specifically when triggered by other sources, for example, OTAs or TripAdvisor. This could imply that information capacity is a critical function of OTAs, which benefit from high relevance and recall.
This study reinforces the relevance of the OTAs, particularly the landing page from SERPs where the participants “anchor” their search, and search terms radiate and return to this point as subjects cut and paste from the featured hotels. However, OTAs also face challenges of differentiation in that participants remembered they had visited OTAs, but could not recall the specificities, as reported by Burgess et al. (2011). While participants at least remembered they have been to OTAs and search engines, they failed to remember the hotel choices, which to some extent confirm the findings of Sparrow, Liu, and Wegner (2011) in that those who expect to re-search do not actively recall the websites they visit.
Conclusions and Managerial Implications
As “search” retains its importance, there is little indication here that “social search” will replace it, rather that purely social network sites are just that—places of social interaction that are not routinely used as information sources. The search engines remain dominant as an entry and referral point for information, and hotels must ensure that they are visible, if not on SERPS, at least on the second level of search, such as on OTAs and TripAdvisor, for visibility (Anderson 2011, 2012). Hotels must also ensure that they take a consistent approach to marketing communication strategy, on all channels, as customers sift, picking up keywords, through several websites to arrive at a final booking. This also highlights the necessity of vigilant content management across all channels. The OTAs/TripAdvisor will, no doubt, exploit their unique role in marketing of hotels and attempt to monetize the pivotal role they play in hotel room search, even when they are not ultimately used for purchase transactions.
Previous, related research by Fesenmaier et al. (2011) classified the nature of “keywords” into eight mutually exclusive categories, that is, destination, OTA, a travel-related company, a website, a travel magazine, advertising slogan, and others. This research, in contrast, allows the researchers to infer more about the categorization of keywords. It reveals that keyword searches are potentially recursive and even conditional on content and text found in earlier search sessions and then further refined. However, keywords are not easily recalled, at least not in this research. To some extent, this finding confirms the finding of Pan and Fesenmaier (2006) that travel planning is an adaptive and iterative process that mostly follows a contingent structure. Fesenmaier et al. (2011) also stated that search queries tend to be short, consisting of less than four keywords. This is particularly true of hotel booking keyword searches that are significantly shorter, often only ONE word. Given the observed behavior in this study in terms of recall and keyword searches, that is, less typing and more copy-and-paste, it might be anticipated that the length of search queries may be longer in the future, with searchers pasting more lengthy keywords into search toolbars, but starting off with very short keyword searches. Given that users copy and paste the hotel name and descriptions into the Google search box, it is therefore important to confirm the accuracy of the hotel name on major OTAs, TripAdvisor, and other review sites. Additionally important, and a major implication for hotel marketers, is to carefully monitor all additional information returned on strategic partners’ pages, as searchers appear to use all available, visible text resources to create new search criteria.
Should hotel marketers invest in buying keywords from Google, Bing, and other search engines though it seems that this will not generate qualified traffic? Or should hotel marketers rely on their distribution channels for visibility? Certainly, with the development of Google Hotel Finder and Google Carousel, hoteliers need to make sure their hotels are, at least, search engine and OTA “friendly.”
The relevance of OTAs in the hotel customers’ search process implies improved presence is needed in OTAs, which are becoming more like search engines. Withiam (2011) reports that year-round room availability, conversion rate, margin, price, quality, and location are some variables Expedia now use to arrange search results. Hoteliers should leverage these criteria to move up on search placement on these sites.
It is clear from previous research that social sites are used more often in the postpurchase stage (sharing), and therefore investing in social sites such as Facebook at the prepurchase stage may be somewhat redundant. Whereas Blankenbaker and Mishra (2009) refer to the search funnel as a major notion, one of their underlying arguments suggests that searchers also use Google as a navigational tool. This research supports this argument in that search terms seem to have “lengthy” and “networked” associations throughout the search session and are not used in a manner that can be described as “mutually exclusive.” This has implications for hotel marketers in their choice of keyword selection for search engine optimization, making it more complex and difficult to manage. Importantly, hotel marketers must realize that keywords must be visible not memorable for potential customers, particularly at the second level of search that takes place on the OTA and TripAdvisor sites. This research reveals that internal sources of information, for example, recall, are perhaps being rapidly replaced by easily available, online external sources of information, which are now ubiquitous. Recall is poor and would require a major investment for hotel marketers to drive direct, and referral, traffic to the desired sites.
Nonetheless, the boundaries between search (i.e., Google), reviews (i.e., TripAdvisor), and OTA bookings (i.e., Expedia, booking.com) are blurring! As these sites evolve, they will continue to encroach on the functionalities of the others sites and look to take a share of the booking commissions. It is clear that customers will continue to use multiple sites to seek information that lead to bookings and that no one site dominates the entire hotel booking process.
Limitations
Although observation reveals “what people do” and may capture unexpected data that other methods can miss, reliability remains an issue, with the interpretation of data being subject to the biases of the observers. The second limitation is the low number of participants resulting from the requirement to participate in two phases of the research design. Nevertheless, 19 observations are comparable to the number of participants in some previous observation studies, which included 15-24 participants (Pan et al. 2013; Pan and Fesenmaier 2006; Bruce, Jones, and Dumais 2004; Loonam and O’Loughlin 2008). The third limitation is that this research was “goal-directed” and time constrained, which impacts participant behavior. Further generalization of this research must be proceeded with caution, given the potential observers’ bias, small-scale student sample, the “unnatural” setting and scenario, and the possible moderating effects of context, motivation, and interest.
Additionally, there is debate as to whether recall is a reliable measure of effectiveness. Certainly, recall does not equate to response, which tends to be a key focus of marketing. Based on our findings, hotel bookers cannot recall most sites they have used, which is similar to previous research on website recall. Regardless, the recall of the sites does not always align with the relevance of sites, and we need to further consider other measures to further approximate relevance.
In this research, participants cannot seem to explicitly recall accurately their site visits under aided and unaided recall, and more research should be conducted by observations and linked to click stream data within a variety of hotel booking settings and over a number of sessions to elicit more in-depth insights. As the purchase influence of social media remains vague and the traffic on these sites continue to increase, questions remain whether the amount of time, money, and attention invested in some social media marketing by hotels yields significant returns. Certainly, as an information source, channels other than social media are currently preferred by potential hotel customers and, if hotels wish to promote their own sites as information sources, they need to adapt to the changing behavior of their customers, making their sites more relevant and easier to recall.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the HES-SO (University of Applied Sciences and Arts, Western Switzerland), funding reference 26781.
