Abstract
Beginning with the August 2003 issue, Journal of Marketing Research (JMR) has published one or more comments on the lead article, followed by a rejoinder. I have asked experts to provide commentary on one article in each issue that I believe has especially relevant content for researchers and managers. In the current issue, the lead article is an invited paper for which I also asked several experts to provide comments. In all cases, I provide an opportunity for the author(s) of the original article to prepare a rejoinder in the same issue. My hope is that such related reflections and commentaries on a current topic will enhance the value of JMR to readers. Although I see no reason to entice authors to express strong disagreements about specific issues, I expect that such collections of articles will enable readers to become more informed about differences in perspectives that researchers with substantial expertise and experience have on important issues.
Econometric Models
The invited article in this issue focuses on the use of models of historical data for policy simulations. Marketing journals have published applied econometric papers for more than four decades. The first article in JMR on regression results was by Buzzell (1964). The title of Buzzell's article indicates that his results are related to short-term changes in the criterion variable. Intuitively, researchers know that models of historical data are much more likely to yield useful insights in the short run than in the long run. A reason for this is that economic agents have more opportunity to adjust their behaviors strategically in the long run. It is also well-known that such models are more likely to provide accurate conditional forecasts for minor shifts than for major shifts in marketing activities. The debate in the first four articles of this issue focuses especially on modeling issues relevant to major shifts.
Researchers with extensive experience in econometric modeling know that substantive insights often depend greatly on which variables are included (some experts insist that researchers show the robustness of the results to alternative specifications), the functional form used and the allowance for interaction effects, the measurement of variables, the treatment of missing data, the accommodation of endogeneity, allowance for cross-sectional heterogeneity in effects, allowance for variation in effects over time, avoidance of aggregation biases, and so on. Franses (2005) provides an inventory of commonly used diagnostic tests. Franses (see Table 2 for 1998–2000 and Table 3 for 2001– 2003) also shows that there is a paucity of actual use of diagnostic tests in articles published in JMR. Such paucity may suggest that researchers have a disincentive to conduct all relevant tests. For example, it might be imagined that though researchers prefer to publish a valid model rather than one that is invalid, publishing an invalid model is still preferred to not publishing one, especially if it is difficult for readers to detect model deficiencies.
Given data constraints, it is virtually impossible for researchers to accommodate all possible nuances. Thus, researchers rely on theories and experience to decide which aspects are the most critical to include in a model. With accumulating empirical evidence in the literature, the expectation is that future modeling efforts will be more informed and thus likely to provide increasingly useful (i.e., valid and reliable) results. However, I urge researchers to consult the checklist that Franses provides and to conduct all diagnostic tests when appropriate. In applied econometrics, there are three possible reasons for specific tests not to be used. First, researchers may argue convincingly that a test does not apply in the model's context. For example, testing the null hypothesis of zero autocorrelation in the error term in a model of purely cross-sectional data is irrelevant. (Separately, I note that the use of generalized least squares rather than ordinary least squares to accommodate serial correlation in time series data is a technical correction that is not convincing unless the researcher can justify how serial correction logically arises for an otherwise correctly specified model.) Second, some tests may not yet be available for cases other than linear models and normally distributed errors. Third, researchers may argue that the violation of a particular assumption does not invalidate the substantive results. For example, consistency of ordinary least squares does not require normality of the error term. In all other cases, it is the researchers' responsibility to conduct and show appropriate diagnostic tests. The model cannot be assumed to be valid unless proper diagnostic tests fail to reject the assumptions.
Because all models are incomplete representations of reality, a central question is: How can a model that is superior to a meaningful alternative (e.g., judgment or a simpler representation than the proposed model) be obtained? If the interest of the researcher is to discover how marketing activities affect purchases or other responses (as in a “causal” model), any comparison to a model without marketing variables seems useless. Still, it might be argued that noncausal models (e.g., purely autoregressive models) can serve as benchmarks. If the causal model fails to outperform a noncausal one, how much confidence can a researcher have in the conditional forecasts of the preferred model? Another question is: How much understanding is obtained from noncausal models? A pertinent issue is that forecast errors represent a combination of bias and variance. The best performing models are likely to be ones that exploit the bias–variance trade-off (see Bronnenberg, Rossi, and Vilcassim 2005). I advocate that researchers distinguish between these components in the comparison of alternative models.
A current perspective on comparisons of alternative modeling approaches is to determine the benefit of adding additional components to what used to be, often exclusively, models of demand. Bronnenberg, Rossi, and Vilcassim (2005) emphasize the opportunity to add the supply side (see also Villas-Boas and Zhao 2005; for a review of the first marketing article to include the supply side, see Kadiyali 1996), but they propagate a viewpoint that recognizes important trade-offs. Errors in the supply-side specification can cause the demand-side estimates to be biased. Thus, although inclusion of the supply side is theoretically correct, the additional dependencies may actually reduce the quality of the demand-side results. This suggests that it is important that researchers conduct empirical comparisons of models that differ in the extent to which additional complexities are captured. For example, models that present a “fix” to endogeneity by directly modeling the implied correlations between the error term and marketing-mix variables could outperform structural models. Yet I do not advocate specific approaches for the development of models. My aim is to suggest that advanced models should not only be diagnosed extensively but also be compared against plausible alternatives, including reasonable simplifications.
Van Heerde, Dekimpe, and Putsis (2005) focus especially on the Lucas critique, and they argue that multiple approaches are available that differ in the manner in which aspects inherent in the Lucas critique are handled. Empirical studies of the magnitudes of problems in marketing models implied by the Lucas critique as well as comparisons of alternative approaches to accommodate the critique are welcome. To highlight elements of the Lucas critique, I discuss a few simple examples. The first addresses models of the effects of marketing activities on sales of prescription drugs. The second focuses on marketing strategies in the supermarket industry.
Prescription drugs
Suppose that a pharmaceutical firm asks the Food and Drug Administration (FDA) for permission to change a prescription drug to over-the-counter (OTC) status. The FDA may be favorably disposed if the drug is effective and has minimal side effects. If the drug's status becomes OTC (which recently occurred for the allergy drug Claritin), patients who continue to use the drug might have to pay the full cost, because health maintenance organizations (HMOs) tend not to reimburse the cost of OTC products. In that case, patients have an incentive to ask their physicians to prescribe an alternative prescription drug included in the formulary. Physicians are likely to comply if there are such alternatives with comparable efficacy and side effects. In turn, the HMO has an incentive to remove prescription drugs with comparable OTC drugs from the formulary or to increase the co-payment. If there is a difference among HMOs in OTC coverage, the patient also has an incentive to consider switching HMOs (switching typically may occur only once a year, and most patients have few alternatives). Even if there are no HMOs that currently cover OTC drugs, some insurers may consider offering limited coverage of OTC drugs, as the number of OTC drugs increases. As the total co-payment of drugs to patients increases, patients have more incentive to shop around both for the lowest cost (co-payment) of a given drug and for alternative drugs, including generic versions. Thus, to capture the strategic role that patients may increasingly play, it is important that pharmaceutical firms understand how patients “choose” between alternative drugs and HMOs.
This example is intended to show how economic agents might strategically react to changes in market conditions. It suggests that any model in which sales of prescription drugs are made simply a function of marketing activities, such as detailing, physician meetings and events, medical journal of advertising, and direct-to-consumer advertising, is necessarily incomplete. For example, suppose that a pharmaceutical firm finds that the return on investment of detailing for a high-priced drug is such that detailing should be expanded. If there are less costly alternatives, an HMO will have a greater incentive to remove the drug from its formulary. Thus, a high return-on-investment estimate may cause the firm to increase detailing, and the higher sales may cause the HMO to place further constraints on the drug's availability for reimbursement. Unless researchers properly capture the behavior of physicians, patients, HMOs, and the FDA, the estimated effects of marketing activities from otherwise sophisticated econometric models will be misleading.
Supermarket items
Consider a hi-lo operator: The retailer uses the results of a model to estimate the effects of discounts on sales of individual products. Suppose that a descriptive model shows, for different supports and discount percentages, how much of a given item's unit sales increase in a store is attributable to various sources, such as other items in the store and other weeks (see Van Heerde, Leeflang, and Wittink 2004), but that the retailer wants to use the results as a basis for changes in marketing activities to try to improve profit. For example, the retailer may conclude from the model's results and relevant cost data that an increase in the use of feature-supported discounts but a reduction in the average magnitude of price discounts is predicted to be profitable. Suppose also that households tend to use the weekly retailer fliers to decide where to purchase most of a given week's grocery items. If the retailer reduces the average discount (and regular prices remain constant), many households may switch to other retailers. Thus, if the model results are used for policy decisions when the model fails to capture how households choose between alternative stores, strategic decisions may fail to realize the expected outcomes.
Implications
The two examples suggest that many models are, at best, descriptive (and sometimes are intended to be only descriptive). In that case, the estimated effects may properly describe how consumers act under prior conditions. The conditions may or may not be explicit, but as long as researchers do not capture how consumers act strategically or how other relevant parties may change behavior as a function of changes in market conditions, the models will fail to make correct predictions of marketplace outcomes. I hope that this issue of JMR stimulates new research to determine the validity of marketing policy recommendations.
Update on JMR
Separately, I am pleased to report that the journal received 329 new submissions from July 1, 2003, to July 1, 2004. Cindi Privitera is doing a magnificent job handling the processing and other administrative matters. I have also been impressed with the quality of the reviewers' reports. The 2/2/2 process (see Wittink 2004) appears to be working well, though I deviate from it more often than I had intended. My first year as editor has been a very positive experience, despite severe and unexpected difficulties. I am also pleased with the addition of 21 researchers to the editorial board.