Abstract
We present an analysis of the Manufacturing Business Opinion Survey carried out by Mexico’s national statistical agency. We describe first the survey and employ exploratory statistical analyses based on coincidences and cross-correlations. We also consider forecasting models for the indices of industrial production and the Mexican global economic activity, including opinion indicators as predictors as well as lags of the quantitative variable to be predicted, so that the net contribution of the opinion indicators can be best appreciated in a forecasting experiment. The forecasting models employed are statistically adequate in the sense that they satisfy the underlying assumptions, so that statistical inferences and conclusions are validated by the data at hand. Our results lend empirical support to the intuition that this survey provides information that anticipates the behavior of important macroeconomic variables, such as the Mexican index of global economic activity and the index of industrial production. We include a section where some original tables and figures covering data up to year 2009 are updated to include data up to October 2017. These new tables show that the original conclusions remain valid even though the data were subjected in 2011 to some modifications due to a three-fold increase of the sample size and an extended coverage of economic activities.
Keywords
Introduction
Economic agents tend to make rational decisions based on the largest amount of available information. Such information could be based on opinions and expectations from producers about the behavior of relevant variables of the economic system. On the basis of this belief and the assumption that the responses to opinion surveys provide reliable information, it makes sense to aggregate such responses to get an index of the behavior of such variables as production and global economic activity. This is the basic idea behind the calculation of diffusion indices, as defined by the manual of the Organisation for Economic Co-operation and Development (OECD), see [13, Section 5]. The information provided by the respondents are qualitative since they simply answer “Higher”, “Equal” or “Lower” to questions related to, for instance, production in a month relative to the previous one. Those answers are used to calculate quantitative indices that summarize the trend of a group of variables and that supposedly anticipate the behavior of relevant macroeconomic variables in the short run. At the international level, there are several studies that validate such an assumption, e.g., [1, 2, 3, 4, 5, 10, 15]. In this work, we aim at validating that assumption for Mexico, with data from the monthly Manufacturing Business Opinion Survey (MBOS).
The advantage of a business opinion survey is that it generates timely information that may prove useful in short term forecasting. The official Mexican statistical agency (INEGI) calculates several indices from the MBOS; two of the most important are the Aggregate Tendency Index (ATI) and the Manufacturing Orders Index (MOI). Here, we want to answer the question of whether the ATI and/or the MOI anticipate the monthly figure of industrial production and that of the Mexican Index of Global Economic Activity (IGEA). The latter index achieves almost 90% coverage of the quarterly Mexican Gross Domestic Product (GDP) and its figure is released 57 days after the end of the month of reference, while the GDP figure is published 52 days after the quarter of reference.
The following section describes the MBOS and the two main indices used in this work. Section 3 presents an exploratory analysis aimed basically at detecting potential relationships between qualitative variables from the survey and quantitative macroeconomic variables. Section 4 complements the previous analysis by using some quantitative data provided by the MBOS, even though such data are not true measurements but numerical opinions expressed by the respondents as quantitative beliefs. The fact that the MBOS provides this kind of data makes it different from similar opinion surveys and enriches the present analysis. This kind of information can be gathered by the MBOS because the National Law of Statistics [11] establishes as a legal duty that respondents must answer the questionnaires required by INEGI. Section 5 describes a forecasting experiment and shows the models employed to predict some relevant quantitative variables that incorporate the timely opinion indicators as predictors. The experiment was carried out to select the best models and validate their predictive ability with data for years 2009 and 2010. The last section concludes and makes some recommendations about the usage of the techniques and models employed.
Monthly manufacturing business opinion survey
The MBOS follows the OECD recommenda-tions [13]. Details on the sampling design and some other technicalities of the MBOS can be found in [12]. The summary of answers to questions that ask for an assessment of the change of one month with respect to the previous one is expressed in differences. Such is the case for most questions of the MBOS questionnaire, for instance, the question For the indicated periods, how was the volume of production of your firm? admits one of the following answers: Much higher, Higher, Equal, Lower and Much lower.
The opinion indices obtained from the MBOS are supposed to anticipate the economic facts, because the producers express expectations that tend to become plans, whose realizations are registered at the end as facts that can be measured with traditional quantitative surveys. The diffusion indices summarize the responses and assign weights in accordance to the firm’s relative importance, i.e. sales, fixed assets or occupied personnel of the firm, depending on the variable under consideration. The response combination leading to the respective index makes use of the following weights: 1.00 for Much higher or Much better; 0.75 for Higher or Better; 0.50 for Equal; 0.25 for Lower or Worse; and 0.00 for Much lower or Much worse. The central value is 0.5, which is considered the borderline between expansion of manufacturing activities (above 0.5) and contraction (below 0.5), although we should keep in mind that the weights are assigned arbitrarily.
We concentrate on ATI referred to the historical (previous month) behavior of the firm and MOI referred to its expected (current) behavior. The ATI combines data from five questions: Volume of Production; Use of Plant and Equipment; Domestic Demand; Volume of Exports; and Occupied Personnel. Responses to each question lead to obtaining the corresponding tendency subindices: Production (PROTI); Plant and Equipment (UPETI); Demand (DEMTI); Exports (EXPTI); and Occupied Personnel (OCPTI). To calculate these subindices in month t, the following percentage of responses are obtained, for each question and option
where
with
MOI is derived from expectations of the producers about such variables as: Orders; Production; Occupied Personnel; Timely Supply Delivery; and level of Supply Inventory. The subindices generated are: Volume of Orders (VOMOI); Volume of Production (VPMOI); Occupied Personnel (OPMOI); Supply Delivery (SDMOI); and Supply Inventory (SIMOI). The percentages of responses are weighted as with the ATI except for SDMOI that is weighted as: Much faster, 0.00; Faster, 0.25; Equal, 0.50; Slower, 0.75; and Much slower, 1.00. Again, calculation of weighted percentages is done as in Eqs (1) and (2) where now
Another subindex that will be used here corresponds to expected demand (EDEMI). Historical data of the MBOS can be found on the following websites:
ATI (
MOI (
Due to the type of questions involved, ATI and MOI can only be compared with variables expressed as monthly differences or changes. For instance, Fig. 1 shows plots of ATI and the (quantitative) Volume of Production Index (VPI) expressed as monthly percent changes VVPI
Standard errors and critical values for cross-correlations with
72
Standard errors and critical values for cross-correlations with
Percentage of coincidences and cross-correlations between ATI and some quantitative variables
Significance:
Percentage of coincidences and cross-correlations between the production and demand subindices of ATI with some quantitative variables
Significance:
Left panel: original series ATI (left scale) and VVPI (right scale). Right panel: seasonally adjusted series ATI and VVPI. January 2004–December 2009.
When two time series change from one month to the next one in the same direction we say that a coincidence occurred. The percentage of coincidences measures contemporary affinity between the two series, but no predictive ability can be claimed by getting a large percentage. It is the timeliness of the opinion indicators that makes them useful tools for the analysis of the current state of the economy. Another measure that quantifies association between two series is the cross-correlation coefficient for different lags and leads.
We do not attempt to establish any formal statistical comparison for the percentage of coincidences, nor indicate that a given percentage may be significantly different from zero, but just to observe if they are similar for different pairs of variables. On the contrary, the statistical significance of a cross-correlation can be established by referring it to a standard Normal distribution whose asymptotic variance is obtained as follows. If
where
Table 2 presents the percentage of coincidences and some cross-correlation coefficients between ATI and some relevant quantitative variables, whose significance is indicated in order to decide if each of them (individually) can be considered different from zero. There we see that the contemporaneous correlations are the highest in the three cases and are significant at the 1% level. IGEA, expressed as a monthly variation is indicated with a V at the beginning of its name. The percentage of coincidences shown in Table 2 indicates that ATI has more affinity with VVPI than with VIGEA. Moreover, the contemporaneous correlation of ATI with VVPI is higher than that with VIGEA. Besides, the cross-correlations of order two point in the direction that ATI can be used to anticipate VIGEA and VVPI. Of course, this is just an exploratory analysis and we shall validate our findings in the next section.
In addition, we investigate the predictive ability of the subindices PROTI (production) on VVPI and VIGEA, and DEMTI (demand) on VIGEA. The plots of the series under comparison with PROTI appear in Fig. 2. The corresponding percentages of coincidences and cross-correlations appear in Table 3. It is clear that PROTI has lower percentages of coincidence but higher contemporaneous correlations with the respective quantitative variables than ATI. This apparently contradictory fact emphasizes the need of carrying out a confirmatory analysis. We see that cross-correlations in absolute value are higher for PROTI than for ATI and even the lead one cross-correlation is significant. Besides, we consider DEMTI because demand has a clear anticipatory content and the comparison is with VIGEA, the most relevant quantitative economic indicator. The results shown at the bottom of Table 3 are similar to those between PROTI and VIGEA.
Percentage of coincidences and cross-correlations between MOI and some quantitative variables
Significance:
Left panel: PROTI (left scale) and VVPI (right scale). Right panel: PROTI (left scale) and VIGEA (right scale).
An aspect to be considered when analyzing time series is whether seasonal effects should be removed to obtain appropriate conclusions. For time series generated from opinion surveys, several studies have shown that they should be adjusted only when there is strong evidence of seasonality (e.g. [9] and the references therein). This recommendation arises because (i) there is no clear evidence of seasonality in opinion data and (ii) seasonal adjustment procedures are biased towards finding seasonality in series that do not have it. The OECD manual (see [13]) recommends analyzing each series individually to detect the ones that deserve being deseasonalized. We decided to compare the results previously obtained with those for the series adjusted for seasonality. To adjust the series we applied the Census Bureau X-12-ARIMA method (
Percentage of coincidences and cross-correlations between expected and historical indices about production and demand
Significance:
Monthly mean values and standard deviations of approximate variations for Production, with and without trimming.
Another aspect worth of study is whether the data provided by the MBOS are internally coherent; for example, whether the subindex of expected production (VPMOI) anticipates the behavior of current production (PROTI). If so, we may say that the responses provide reliable information, otherwise the information may be suspect of lack of coherency. We should recall that this study involves diffusion indices (aggregated responses) and that fact did not allow us to study the internal coherency of the survey at the questionnaire level. Nevertheless, we found strong associations between expected and historical indicators for several variables considered by the MBOS. For example, Table 5 shows the percentage of coincidences and cross-correlations between expected and historical production and demand. Therefore, we may say that the historical opinion indices not only anticipate the economic fact, but are also contemporaneously correlated with expected opinion indicators, though the expected indicator is released one month before its historical counterpart. This result is important when trying to anticipate quantitative economic variables and gives support to the belief that the opinion indicators provide valid information to do that.
Aside from qualitative responses, the MBOS questionnaire asks the respondent for a numerical value that reflects his/her belief on the outcome of a variable by providing an Approximate Percent Variation, e.g. for Volume of production. These questions are usually considered sensitive, but according to the people in charge of the MBOS there is no particularly higher nonresponse for them. These questions can be asked because INEGI is an official statistical agency, so that the law enforces responding the questionnaire and protects confidentiality of the data. Perhaps a private organization would not get responses to these questions and that might be the reason for not including them in the questionnaire of that organization. It remains to show that the responses to such questions have predictive ability on economic facts.
We emphasize the questions referred to production and demand. The approximate variations are expressed as percentages and are related to the qualitative responses considered in Section 2, with positive content Much higher (
Descriptive statistics of the approximate variation series with and without trimming
Percentage of coincidences and cross-correlations of MVPRO and MVDEM with some quantitative variables
Significance:
Left panel: MVPRO and VVPI series plotted in the same scale. Right panel: MVPRO and VIGEA series plotted in the same scale.
where
At first sight, there is very large variability in the mean values and variances attributable to an exaggerate perception of the producers about growth (either positive or negative) in production and demand. In order to mitigate that effect we used trimmed means (we trimmed 5% at each end of the range, so that only the central 90% of the observations was employed). To appreciate the trimming effect on the production variable, in Fig. 3 (left panel) we show the series of mean values of the approximate variation for production with and without trimming. We can see that the level and dynamics of both series are similar, except for the extreme values. As a complement, the plots in the right panel show standard deviations, obtained as the square root of Eq. (7), for the original and trimmed approximate variations. The standard deviation for trimmed data is evidently lower than that of the original data, particularly for months with extreme observations.
Estimation results of the forecasting models for VVPI. Estimation period: 2004:05*–2009:12
The trimming effect can also be seen in Table 6. Both the center of the data (measured as mean or median) and the dispersion (measured as standard deviation about the mean or as mean deviation around the median) get closer to zero for the trimmed data than for the original data, whereas the skewness and kurtosis coefficients fluctuate around zero and do not show a clear pattern. Thus, trimming does not alter the dynamics of the series and essentially reduces the variability associated to extreme values, thus allowing us to see the underlying signal more clearly. Hence, we decided to work with the series of trimmed means and from here on we refer only to this kind of data.
The exploratory analysis starts by looking at the coincidences and cross-correlations of the series under comparison. In Table 7 we present the results obtained for approximate variations against the quantitative series VVPI and VIGEA. This table is akin to Table 3, since the variables involved measure essentially the same thing. The same can be said about Fig. 4 as compared with Fig. 2. A comparison of the two tables indicates that the percentage of coincidences and the contemporaneous correlation increase for approximate variations. Not only affinity increased, but magnitudes become comparable, as can be seen in Fig. 4 where two pairs of series are plotted in the same scales.
Using opinion indices to improve prediction of a given variable is one of the main objectives to carry out opinion surveys. Several works (e.g. [1, 14]) have shown that such indices increase predictive ability of turning points. In order to check that such an objective is achieved by the MBOS we consider the information of the previous section to propose several forecasting models for VVPI and VIGEA.
We specified the forecasting models starting with an Auto-Regressive (AR) model for the quantitative variable of interest and incorporating opinion indices as exogenous variables. Such models will be referred to as ARX models. A typical ARX equation is
Estimation results of the forecasting models for VIGEA. Estimation period: 2004:05*–2009:12
where
In all cases the AR model was augmented with one or at most two opinion indices or approximate variations previously found with predictive ability. A summary of the estimation and validation results of the models for VVPI is shown in Table 8, where the variables appear with their respective significant lags.
In Table 8 we also see the adjusted coefficient of determination
Similarly, in Table 9 we show the estimation results of the models for VIGEA. All models are valid in data according to the Ljung – Box statistic and DEMTI is the variable that considered alone best explains VIGEA (
To take advantage of the available data we must be aware of the releasing dates of the variables to be forecasted and their predictors. Figure 5 shows that the opinion indices on expectations OI(E) are released almost 40 days ahead of VPI and about 50 days ahead of IGEA. The anticipation of the opinion indices on historical performance OI(H) is smaller (about 10 and 22 days) but even so, its timing is relevant.
Releasing dates of VVPI and VIGEA relative to the opinion indices provided by the MBOS, with reference to month t of a given quarter.
Summary of the one-step-ahead forecasting results for VVPI
Summary of the one-step-ahead forecasting results for VIGEA
The forecasting experiment is designed to compare the performance of one-step-ahead predictions for VVPI and VIGEA with the different ARX models in the post sample period 2009:01 to 2010:02 and each model in Table 8 produced monthly forecasts of VVPI for such a period. The models were reestimated each month, excluding the current and future values of VVPI and the one-step-ahead prediction was recorded. We calculated the one-step-ahead prediction errors for the 14 post sample values and Table 10 presents the following summary measures: (i) Mean Error (ME) to check for potential prediction biases, (ii) Root Mean Square Error (RMSE) as a measure of precision expressed in the same units of the variable and (iii) Theil’s U statistic as a measure of relative precision against that of a benchmark model (the AR model in this case). The interested reader is referred to [6] for details on these measures. We do not present the Mean Absolute Error because it provides basically the same information as the RMSE (see [8]).
Further empirical support to the forecasting models is provided by the Mincer-Zarnowitz equation (see [6, Ch. 11]) which is useful to verify that all the information available was efficiently employed by the model in turn, that is,
where {
In Table 10 we can see that none of the models shows significant non-zero bias. This fact indicates that the models make efficient use of their corresponding information sets. Moreover, the Theil’s U statistics show that the inclusion of an exogenous variable to the AR model increases predictive ability over VVPI. By looking at the RMSEs and Theil’s U statistics it is clear that the best model is the ARX-9 that incorporates MVPRO as predictor. It should be stressed that this model was among those with best goodness of fit.
The explicit expression of the recommended ARX-9 model to get one-step-ahead predictions of VVPI is
where
and a 95% prediction interval is given by
Table 11 shows a summary of the prediction errors for several models employed to forecast VIGEA. We present only the results for models with higher forecast precision and higher
With regard to precision, Table 11 allows us to see that the best model is the ARX-12 (that includes DEMTI and MVPRO) which also has one of the highest
where
The coincidence and cross-correlation analyses are very simple and easy-to-use exploratory tools that helped us to select a few opinion indicators from a large list of potential predictors for relevant macroeconomic variables. An opinion indicator may be considered a good predictor because it indeed shows anticipatory power on the quantitative variable of interest. However, it suffices that both the indicator and the variable move contemporaneously to select the respective indicator as a useful predictor, due to its timely publication. Thus, we suggest applying this type of analysis routinely with the opinion indices generated from opinion surveys.
With respect to the forecasting models, this work lends support to the use of opinion indices from the MBOS to improve the prediction of VVPI and VIGEA, since these variables can be best predicted when the contemporaneous values of MVPRO and DEMTI are used, correspondingly. We should stress that MVPRO is an opinion variable that had never been used before in Mexico to improve the predictive ability on important macroeconomic variables. We strongly recommend the use of this kind of variables, whenever the corresponding question can be asked in the questionnaire of an opinion survey, because the data generated this way was seen here to have high predictive power on the corresponding quantitative variable (to signal both direction and size of change).
The main conclusion is that opinion indices are useful to generate timely nowcasts and improve the forecasting ability of time series models for some important Mexican macroeconomic variables.
Footnotes
Acknowledgments
This work was carried out in collaboration with the INEGI personnel. Víctor M. Guerrero acknowledges financial support from Asociación Mexicana de Cultura, A. C. to carry out this work.
A Partial Update with Some Data up to October 2017
The basic aim of this section is to show that the original patterns seen in the graphs obtained in the previous version of this paper remain valid, even though the variability of the ATI indicator diminishes notably due to the increase of sample size starting in 2011. In fact, we can summarize the new results by saying that the previously observed (graphical and numerical) patterns remain valid, and they even become more statistically significant than before.
Standard errors and critical values for cross-correlations with N
Lags
0
Standard error
0.079
0.078
0.078
0.078
0.078
Significance
Critical values
10%
0.130
0.129
0.128
0.128
0.128
5%
0.155
0.154
0.153
0.153
0.152
1%
0.204
0.202
0.201
0.201
0.200
Percentage of coincidences and cross-correlations between ATI and some quantitative variables Significance:
Coincidences
Cross-correlations: ATI
i
6
3
2
1
0
1
81.2%
r(i)
0.23
0.08
0.22
0.29
0.59
0.20
Cross-correlations: ATI
79.4%
r(i)
0.17
0.17
0.27
0.13
0.52
0.20
Cross-correlations: ATI
46.7%
r(i)
0.10
0.04
0.20
0.25
0.21
0.22
Left panel: original series ATI (left scale) and VVPI (right scale). Right panel: seasonally adjusted series ATI and VVPI. January 2004–October 2017.
Percentage of coincidences and cross-correlations between the production and demand subindices of ATI with some quantitative variables Significance:
Coincidences
Cross-correlations: PROTI
i
0
1
84.2%
r(i)
0.29
0.11
0.29
0.36
0.65
0.30
Cross-correlations: PROTI
vs. VIGEA
76.4%
r(i)
0.25
0.15
0.34
0.22
0.53
0.28
Cross-correlations: DEMTI
vs. VIGEA
79.4%
r(i)
0.14
0.14
0.18
0.17
0.60
0.24
Percentage of coincidences and cross-correlations between MOI and some quantitative variables Significance:
Coincidences
Cross-correlations: MOI
i
0
1
80.6%
r(i)
0.24
0.11
0.18
0.21
0.51
0.16
Cross-correlations: MOI
76.4%
r(i)
0.15
0.15
0.22
0.08
0.46
0.16
Cross-correlations: MOI
56.4%
r(i)
0.07
0.01
0.20
0.24
0.23
0.20
Percentage of coincidences and cross-correlations between expected and historical indices about production Significance:
Coincidences
Cross-correlations: VPMOI
vs. PROTI
i
0
1
83.0%
r(i)
0.03
0.31
0.85
Left panel: PROTI (left scale) and VVPI (right scale). Right panel: PROTI (left scale) and VIGEA (right scale).
