Abstract
The motion picture industry is one of the largest industries worldwide and has significant importance in the global economy. Considering the high stakes and high risks in the industry, forecast models and decision support systems are gaining importance. Several attempts have been made to estimate the theatrical performance of a movie before or at the early stages of its release. Nevertheless, these models are mostly used for predicting domestic performances and the industry still struggles to predict box office performances in overseas markets. In this study, the aim is to design a forecast model using different machine learning algorithms to estimate the theatrical success of US movies in Turkey. From various sources, a dataset of 1559 movies is constructed. Firstly, independent variables are grouped as pre-release, distributor type, and international distribution based on their characteristic. The number of attendances is discretized into three classes. Four popular machine learning algorithms, artificial neural networks, decision tree regression and gradient boosting tree and random forest are employed, and the impact of each group is observed by compared by the performance models. Then the number of target classes is increased into five and eight and results are compared with the previously developed models in the literature.
Introduction
Thousands of movies are released around the world every year. Given that the average budget spent on a film ranges from 100 to 150 million, the motion picture industry contributes extensively to the development of not only the national economies but also to the global economy [29]. According to THEME Report (2018), the home and theatrical entertainment market reached $ 98.6 billion with a 25 percent increase from five years ago, where the global box office of the released movies all over the world is $41.1 billion and the total number of screens approximately 190 Globally, total annual spending in entertainment industry is approaching one trillion dollars [23]. Among music, publishing and video games, the film industry is a key driver of the entertainment industry.
Although it is such a large industry, it cannot be said that it is equally profitable. Each year, there is a considerable number of movies that fail to break even and lose a lot of money. Large-scale investments are required to produce cinema films. In general, it is estimated that studios averagely spent around $100 million to produce and market a single film [62]. Based on the used computer-generated imagery (CGI), or high stars, a movie can cost even several hundred million dollars. From a financial point of view, these increased costs and low possibility of success make the movie industry one of the riskiest businesses.
The riskiness of the sector and the availability of large data have increased the interest of academics in the film industry too. From sociology to computer science, many researchers from different disciplines have worked on film marketing. The studies investigating the success of a movie can be grouped under two main topics: psychological perspective and economical perspective. The psychological perspective mainly concerns subjects like why people choose cinema as a leisure activity, or what factors affect the choice of a specific movie, impact of personality traits and mood on film selection or satisfaction, and most importantly how to segment the audiences. Economical perspective, on the other hand, examines the impact of movie or audience related attributes on financial indicators such as ratings, box office, market share or profit [5].
Generally, from an economical perspective, forecast models and decision support systems are developed by taking weekly or cumulative theatrical success as the independent and various production, distribution related variables or external attributes as the dependent variables. One of the earliest approaches used to forecast theatrical success was multiple linear regression. Due to their ease of use and ability to show which factors contribute to the success of a film multiple linear regression models have been quite popular [5]. However, some researchers argue that the assumption of linearity between box office performance and its determinants do not represent the complex dynamics of the industry, where multiple regressions do not properly model the interrelated relationship between supply and demand or word of mouth (WOM) and the film success, thus using simultaneous equation models, such as two or three stage least square models are suggested [14]. On the other hand, some researchers argued to use behavioral models such as are diffusion or probabilistic models. These models were efficient because they are capable of representing the whole life cycle, but to perform better they need the initial figures like the opening week data [65].
The fact that production costs of a movie can be astronomical, that they are experimental hedonic products, and that they have a short life cycle, makes forecasting accuracy of a movie before its release an important issue. In recent years, machine learning models have begun to attract attention in prediction studies [35]. Especially for pre-release models, it has been observed that nonlinear machine learning algorithms perform better in forecasting box office performance [65].
Other than increasing the accuracy of the forecast performance, another challenge is to estimate international performance. Movies are cultural and experiential products and their performance can vary significantly across countries and in general 70% of worldwide revenue comes from the international markets, which become worthwhile sources to compensate for the increasing costs [25]. Reflections of the rising importance of international markets have started to be seen in academia too, where there are successful models developed for some Far Eastern or European countries, yet there is much more to be explored.
Although international markets have been important for Hollywood since its emergence, film export revenues have become particularly important since the early 90 s, where they have become a vital source of income to cover up the continuously increasing production costs [20]. When the marketing and production trends in film industries are examined, it is seen that America has a strong position in the international market due to its financial power and systematic structure [60]. However, for international movie marketing the concept “one recipe for all” is not applicable, therefore country specific studies are needed, there are only a limited number of studies determining country and product specific factors contributing to the international success of a movie.
This study is conducted to be a small step to fill this gap in the international motion picture literature by examining American movies’ performances in the Turkish market, where no previous study developed an estimation model neither foreign nor local domestics films. The main purpose of the study is to create a forecasting model for American films screened in Turkey. To do this, the model with the most accuracy was investigated by comparing the performances of different groups of variables and different machine learning algorithms such as random forest (RF), gradient boosting tree (GBT), multilayer perceptron (MLP) and decision tree (DT). Variables found significance in previous studies are categorized into groups according to their stage in a movie’s value chain in other words, as production related, distribution related and international release related, and their influences on movies’ performances are analyzed. Though many studies have investigated influential factors on movie performances, there have been only a few studies examining the performances of international movies in a regional market.
The remaining part of the paper proceeds as follows: Sub-section 1.1 briefly reviews the literature on forecasting the box-office performances of movies. Section 2 gives the details of the variables that are used in the model and the methodology. Section 3 presents the findings of the research. Section 4 includes a discussion of the findings, and finally in section 5 limitations and implications of the findings to future research into this area are mentioned.
Literature review
Producers and executives begin to make box office predictions while the film is in the concept idea stage and from its production to consumption all the managerial decisions are usually made based on these estimates [45]. Therefore, it has always been very important for studios to make accurate predictions. How to assess the success of a film has been a moot point for the cinema industry. Since cinema is both an art and an industry, how to evaluate a movie’s success has been a debated issue. From the “cinema as an art branch” perspective, the artistic value may be more important than the financial success of the film. Also, the popularity of some movies may increase over time, even though their box office numbers can be low. Yet from the financial point of view, a movie is generally evaluated by its box office figures [11].
Due to their importance in the movie business and ease of obtaining, in most of the studies in the literature researchers attempt to estimate box office revenues or the number of audiences [5]. Since the 80 s there have been many proposed models that forecast weekly or cumulative box office performances. The most commonly used technique in the economic perspective domain is multiple linear regression models. Even though some researchers argue that, due to the uncertain and unpredictable nature of the market, more complex models must be used, the studies, where early forms of multiple linear regression models are employed to assess the various factors and their influences on the theatrical success are very important and should be examined as they lay the basis for the later work [14]. With the development of more sophisticated regression techniques, which conceptualize the dynamics of the industry better, and due to its simple use, regression models are still very popular. Yet, some researches still consider that the assumption of linearity between the box office performance and its determinants can cause these models not to perform well [65].
Generally, in these models weekly or cumulative box office revenues are taken as the dependent variable, where various production, distribution related or external attributes such as budget, star power, MPAA ratings, critics, number of screens, critics as the independent variables [14]. For instance, in one of the pioneering studies on forecasting theatrical success of a movie, Barry Litman examined the “ingredients of a successful motion picture” using a multiple regression model. Many attributes such as genre, MPAA rating, cost, cast and the type of distributor company were tested to examine their impact on theatrical rentals [13]. Since then, many studies have been carried out by trying different determinants, such as country of origin, sequels, marketing, and advertising efforts and user generated content [12, 70].
Based on the premise that each movie is a new product and the relationship between the box office performance and its determinants are more complicated than the linear relationship accepted in the regression models some researchers focus on more elaborate behavioral models like probabilistic or diffusional models [14]. For example, in one of the earliest examples of behavioral models for forecasting box office, Jones and Ritz used a diffusion model with various differential equations. They assumed that for any new product, the adoption process was not just only for the consumer but also for the retailer and these processes interacted with each other. This study is significant because the previous diffusion models lack to emphasize the interdependency between the supply, which is the number of screens and demand, which is box office receipts [32]. Behavioral models are useful in terms of covering all the stages in a movie’s life cycle and conceptualizing the dynamic behavior of consumers and their impact on theatrical success, but to perform more accurately they need initial data.
In recent years, machine learning models have begun to attract attention in prediction studies and became a serious competitor to classical statistical models. The reflections of this increasing interest have also been seen in the studies attempting to estimate the box office revenue. With the increasing amount of available data and developing machine learning applications, nonlinear algorithms such as artificial neural networked (ANN), support vector machine (SVM), classification and regression trees (C&RT) started to be used in forecasting box office performance. The fact that these models give good results with pre-release data without the need for initial data has increased their popularity.
Some of these studies aimed to identify the determinants that affect the performance of a movie and select the algorithm that makes the most accurate prediction. Sharda and Delen were the first to attempt the use of artificial neural networks to forecast the box office performance before the theatrical release. Instead of forecasting the point estimate, they approached the subject from a classification perspective. They classified the movies into 9 categories from flop to blockbuster based on their cumulative revenues and compared accuracies of various models. [52] After this study, many studies have been conducted with different variables and different algorithms, and It has been observed that in terms of accuracy, these models can make better predictions. [37, 46].
The success of a good marketing forecasting model is not only distinguished by its accuracy but also by the insights it provides [17]. Seonyeong and Kim attempted to emphasize the significance and relative contribution of the distribution power and competition in the theatrical success of the movies. In addition to the movie related features, they added the variable competition, which is devised by the number of competitors as of the previous day of the opening day and distribution power of competitors, which is determined as the sum of the screens allocated to competitors on the opening day. They grouped the variables based on their characteristics and found that the number of competitors was found to be more effective compared to their distribution [49].
Although international markets have been important for Hollywood since its emergence, film export revenues have become particularly important since the early 90 s, where they have become a vital source of income to cover up the continuously increasing production costs [20]. Nor many studies One of the earliest forecast models conceptualizing international markets was developed by Neelamegham and Chintagunta. who proposed a hierarchical Bayesian model consisting of three stages forecasting viewership at different stages of a movie for the US and 13 foreign markets [48]. Elberse and Eliashberg developed an econometric model using 3SLS least for the US and four European countries using weekly revenues. Their findings provide strong evidence for the interdependence between the supply and the demand within and across markets. In an international context, their findings reveal that there is an interaction between domestic performance and foreign performance, but interestingly this association weakens as the release gap increases [4].
Some studies have tested the effect of different variables specific to the international market in addition to the previously described variables in the literature. For example, Gaenssle, Budzinski, and Astakhova examined the determinants of the office success of international movies in Russia. As independent variables, in addition to the frequently used variables in other markets, variables like Google-hits of actors and movies, attendance of international stars at Russian movie premieres and title adaptation into the Russian language were included in their model [57]. In another study, evaluating the factors influencing the box office performance in China Kwak and Zhang also added a country-specific variable to their model. Besides the commonly used variables such as genre, director and actor, release time, and distribution, the impact of the existence of Chinese cultural elements in the content and the existence of Chinese actors were also investigated and found that movies containing Chinese cultural elements or cast have a higher box office [31].
Methodology
The value chain of a movie consists of 3 stages: “Production”, “International Release” and “Local Distribution”. “Production” refers to the movie related attributes of a film. Decisions in the process of exporting (or importing) a film are made based on the evaluations of characteristics of the film, as well as the film’s local success and the time lag between domestic and international market releases. The stage “International Release” implicates these decisions. And finally, “Local distribution” conveys all the distribution and exhibition-related activities. The purpose of the first experiment is to analyze the impact of each stage in the value chain on the number of attendances. Overall, the first part study was conducted by comparing the performance of each group. At the outset, Model 1, which includes only movie-related variables, becomes the baseline. To compare the performance of each variable group, other variable groups were added to the baseline model incrementally. With this structure, the degree of contribution of each variable group can be captured by the difference between the performances of each model. For example, the performance difference between model 1 and Model 2 can be interpreted as the effect of variable group 2. Pruning or additive modeling approach is a frequently used method in studies with machine learning algorithms, to assess the contribution of a variable to forecasting accuracy. Like in some previous examples, the subject is approached from a classification point of view and the dependent variable is discretized [18, 52]. According to the literature, as a dependent variable, both revenue and number of attendances can be used as an indicator. Due to the changing ticket prices and high inflation rates in Turkey, it was thought that the number of attendances will be a better indicator. So, the number of attendances is discretized into three and Model 1, Model 2, Model 3, and Model 4 are tested by applying four machine learning multilayer perceptron, decision tree, gradient boosting tree, random forest, support vector machine and their performances are compared. In the second part of the experiment, independent variables are discretized first into five and then into eight to develop a more elaborate model and to compare its accuracy performance with the previous models in the literature.
Variables
Our dataset consists of 1559 US movies released in Turkey between 2005– 2017. In this section, variables used in the model and how they are devised will be explained.
MPAA ratings
In 1968, MPAA introduced a rating system to inform the parents about content of obscenity, violence and profanity in the movies. Impact of MPAA rating has been investigated for many years both in economic and psychological models [1, 23]. Despite to some contradictory examples [12, 40], in general MPAA ratings have been identified as a significant predictor of box-office performance [33, 58]. As some previous studies internet movie database (IMDB) is used to gain MPAA Ratings.
In our data, there are five categories G, PG, PG-13, R and Not Rated. G stands for the general audience, which means that the content of the movie is appropriate for audience for all ages. Movies that should be watched with parental supervision are classified as PG. PG-13 category indicates that parental guidance suggested for audience under 13, because some material in the film might be not suitable for children. Movies containing profanity, nudity, sex or violence are rated as R- restricted audience, where viewers under the age of 17 cannot watch these films unless accompanied by their parents or guardians. Some R-rated movies have recut versions, where inappropriate content is removed. Not rated or unrated movies are the ones which are not submitted to MPAA.
The classification system implemented in Turkey is slightly different though very close to MPAA ratings. Since in this study, it is attempted to forecast American movies’ performances, MPAA system was used. Like many previous studies IMDB is used as the source of information for MPAA ratings. The majority of American films released in Turkey between 2005– 2012 are R and PG-13.
Genre
The most convenient and accessible method used to segment, and target audience is to use genre preference [64]. Therefore, the content or genre of a movie is often examined as an independent variable in box office studies [19, 28]. In this study, genre is also represented as one of the independent variables, but movies often have more than one genre. First three genres named at IMDB are taken and devised genre variable in a 1-0 binary fashion. Overall, obtained genres are action, adventure, animation, biography, comedy, crime, documentary, drama, family, horror, romance, mystery, thriller, war, sci-fi, and history. Instead of showing each genre as a dummy variable, each genre is designed as a binary variable that is assigned the value 1 if said genre is within the list taken from IMDB and 0 if not.
Budget
The issue of budget is perhaps the most controversial of all the determinants of film success because of the many examples of both big and small budgeted films becoming financially successful [13]. Not for the profits but for box-office performance, the budget of a film has been generally found as a strong indicator of box-office performance [13, 58]. Although low budget films are known to be more advantageous in terms of profit and require less investments, studios believe that big budget films are less risky, because there is a common understanding in the industry that larger budget means higher quality, where big budget indicates extravagant sets and costumes, expensive special effects, and more popular stars [55].
While the impact of budget is examined in many studies, it is one of the most difficult data to obtain, because studios do not share them publicly and only estimated budgets can be found. Another drawback of using budget is that the publicly available budgets are usually represent the aggregate budget data, where specific budgetary components like wages of the casts, amount of the money spend of special effects and marketing budgets are not specifically indicated [44]. As with other studies that have had to use the aggerate budget, this imposes a limitation on our study, as on what it was spent can be as important as the size of the budget itself.
In this study, estimated budgets are used as a continuous variable like in many other studies before. As a source of information, IMDB, Box Office Mojo, and Wikipedia are used in that order. For the movies, with missing budget values even after IMDB was scraped, first, Box Office Mojo and finally Wikipedia was checked. Nevertheless, budget is the variable with the most missing value. Considering the imbalanced distribution of the data as missing value treatment approach, substituting with median is chosen.
Technical merit
Only a few studies used the technical merit of a movie as an independent variable [18, 52]. Three categories to identify technical effects are used. Movies like science fiction or animation are given the highest rating, where the amount of special effects is high. Trailers of the remaining films were watched to understand the premise and the general atmosphere the trailers reflect. Those, who have many special effects are given the highest rating. Films, of which trailers include moderate special effects (e.g fire, gun shots, car chases) are given medium and films with few or no special effects are given 0.
Ghiassi, Lio and Moon removed this variable from their model due to the difficulty of measuring it objectively [44]. Gunther suggests that technical merit of a movie can be measured by the number of stunt actors and the number of sound or visual effect crew [11], but in our case, since such data was unobtainable, the trailers are evaluated.
Star power & director power
Star value is a widely discussed topic, in which no consensus was reached so far. For example, some studies argue that stars play no role in the financial success of a film [54]. On the other hand, there are many evidences that having a well-known star in the cast can influence motion picture revenues [6, 36]. Another issue with star power is that there is not a clear definition of how to represent and quantify star value [65]. IMDB starmeter, lists of magazines, net worth of actors/actresses, revenue of their recent films are some known measures used in various studies.
Star power can be particularly important for foreign consumers because they have less access to reviews compared to US consumers, so they can rely more on star power as a sign of films quality, thus improving the relationship between star power and financial success [50]. However, how to define star value for international markets is more complex, because an actor/actress can be famous in the domestic market but not known in the international arena, therefore in this study, a local website, sinemalar.com, is used where ratings for any actor and director was available.
Many films usually employ three lead actors or actresses. In most of the posters or in movie review sites the names of the top three actors stand out. Focusing on only the first lead actor may not fully represent the star power, because possible synergy created by two or three actors may be ignored [50]. Therefore, the top three actors/actresses are included in the analysis.
Taking into account the “ensemble effect” the variable Star Power is computed by taking the average of the normalized ratings of the top three actors. There is a need to normalize the data, as the rating on the sinemalar.com was calculated by taking the simple average, which is easiest to implement but has an important limitation because it does not take into account the number of voters. In other words, two actors can have a rating of 9 points, but one being voted by 1 and the other voted by 1000 people which makes a difference and simple average does not reflect this difference by applying Bayesian average ratings are normalized.
Majority of the films are released in Turkey with subtitles however animation films are released as dubbed. In these films generally local voice actors are more prominent, where their names are written on the posters or promoted. So, only for animation films, star power is calculated with the ratings of the local voice actors.
The reputation of the director can also have an impact on the success of movie. Who directs the film can be an important factor for moviegoers when they decide whether to see a film or not [4, 50]. In this study director power is represented as a separate variable.
Familiarity
The subject familiarity has been studied in the literature as a brand extension in the motion picture industry in several studies [9, 55], and it is stated that remakes, adaptations or sequels differ from original concept movies [22]. Literature suggests that these movies need to be handled differently than the original concepts movies [10, 58].
To this end three binary variables are included to specify if the movie is the continuation of a movie or adaptation/remake. The variable “one shot” indicates whether the movie is a part of a film series or not. Sequel indicates if it’s a continuation of a movie and with the “adaptation/remake” variable it is captured if a movie is adapted from literature or tv series or remake of an old movie.
Instead of measuring familiarity with a single variable, it is preferred to use three separate variables, because the impact of a movie being a continuation, or an adaptation/remake can be different and also a movie can be both an adaptation/remake and a sequel which will differ its familiarity. These data were gathered from Wikipedia.
Studio
As it is explained previously, since the very beginning, Hollywood studios carry out the producing and distribution activities together. In the American Motion Picture industry there are six, more recently five major studios, mini majors or independent studios. Major studios are a part of larger conglomerates and unlike the independent studios, conglomeration provides an economic stability, security, and financial flexibility to major studios. This financial power impacts the revenues as well.
In the studies for the US market, no distinction is needed between studio and distributor, since production and distribution are done by the same company. Usually distributor power is measured with a simple indicator reflecting whether a movie is distributed by one of the major distribution companies or not [13, 51].
However, since in this study the distributor is referred to the local distributor, the power of the company in which the film is produced and internationally distributed is represented as Studio Power. To do so three variables denoting high, medium and low are used. When the distribution structure of the American cinema industry is examined, it can be seen that there are 6 major studios, a few mini major studios, which are not as powerful as the major ones but can be considered as large, and independent or small studios. The distinction is made according to the conglomerate or the parent studio. For example, since TriStar is a brand of Sony Pictures, it was classified as high. Mini major studios like MGM, Lionsgate, STX are shown as medium and low is assigned to independents and mini studios, which are not a part of a conglomerate.
Timing of release
Timing of release, as in some studies called seasonality or competition, usually indicates whether or not a film has been released on a particular period based on total demand [52, 54]. In the majority of the studies, dummy variables were used to show whether the film was released during summer or holiday season [30, 69]. A number of researchers have reported that the time of release to be a significant contributor of a movie’s box-office performance [12, 59].
Seasonal fluctuations occur in the movie industry and seasonality is an important factor in determining business strategy. Using movies’ weekly box office revenues Einav analyzed the effect of seasonality, movie decay pattern, and movies’ quality on movies’ box office revenue for the US market and found that the number of audiences varies significantly during the year [38].
Foreign films and domestic films might differ in terms of seasonality trends. In Turkey, the distributors usually think that viewers do not go to the cinema in summer [24]. According to the list of 100 films with highest box office revenue in Turkey, it is seen that the domestic blockbuster movies are screened in theaters usually between November and April.
However, a movie’s country of origin has an effect on its theatrical demand [26]. A similar impact exists in the Turkish market. It is observed that there are different trends when only the total number of audiences of American films is examined. Based on the assumption that US movies have a separate market, competition is represented by using three variables (High, Medium, and Low) in regard to the found seasonality in total number of the US movie audience of movies released between 2005 and 2017. So, movies released in November, December, May or June are given ‘High Competition’; movies that are released in July, August, March and April are given ‘Medium Competition’ and movies released in the months of January, February, September and October represented by ‘Low Competition.
Number of screens
Previous research efforts show that there is a high correlation between the number of screens, on which a film is shown and a movies’ financial success. [3, 63]. Based on the type of the dependent variable and subject of the research, this variable is measured by the number of screens on a daily [40] or weekly basis [4, 70].
In this study, cumulative box office performance is taken as a dependent variable. As in consistence with the literature, where the number of screens a movie scheduled to be shown at its opening day was shown to have a strong impact on the total box office performance [1, 68]. In our models the initial screen numbers on the launch day are used and represented with a continuous variable. Relevant data is obtained from Box Office Turkiye.
Distributor
As it is mentioned before, there are two foreign distribution companies operating in Turkey, Warner Bros and UIP. Neelamegham and Chintagunta showed that there is an association between the use of local distributors and the number of audiences [48]. Therefore, an independent variable indicating whether the movie using local or international distributor is included as an independent variable indicating whether the movie using local or international distributor. (1 = international, 0 = national) 50% of films in our dataset are distributed by international companies.
Domestic performance & release gap
Most U.S. movies do not launch simultaneously in US and foreign markets. Due to the various factors like changing seasonality of demand, varying competition from domestic and imported movies, market entry costs, most movies are distributed sequentially to foreign markets [25]. Although entering a foreign market after the local market is beneficial in observing the performance of the film in the domestic market and developing marketing strategies for the new market, accordingly, increasing release gap may also be risky due to reduced interest in the film or piracy. One way that Hollywood studios try to combat with piracy is shortening the release gap or releasing the movies simultaneously [34]. However, by doing so, they lose the chance to use the information from the previous market.
In this study, the release gap is measured in days as the difference between the US Release and TR release dates. There are some movies that are released in Turkey before US. Like Elliot and Simmons negative ones are shown as 0 in this study. For both domestic performance and US release dates Box Office Mojo is the main source of information [16].
Awareness (word of mouth)
Awareness is associated with the popularity of a movie within the public. With the advent of internet technology, researchers are now able to observe the consumer activities on online platforms such as blogs, social media or websites. In many studies ratings or numbers of user reviews are used to reflect the public awareness [15, 67]. Sinemalar.com was used as the source of the awareness.
In this study, instead of using the total number of comments it is preferred to divide awareness into pre-release and during release. Pre-release awareness is measured by the total number of the comments before a movie is theatrically released in Turkey and during the release is the number of comments made during the film’s theatrical lifetime. Based on the assumption that viewers comment shortly after watching the movie, the comments made after the theatrical screening of the film is over were presumably made by people who watched this film in other platforms and not in cinema, so these comments are excluded. When considered high piracy rates in Turkey, it is believed that all the comments made during release are not done by the people who watched that film at the cinema. However, still it as an accurate indicator of the film’s awareness. In a further research with applying text mining techniques this distinction can be made more clearly.
Accuracy metrics
Overall accuracy also called average percentage hit rate (APHR) to measure the predictive performance of the machine learning methods is used, which is the ratio of the number correctly classified classifications to total number of samples. Like Sharda and Delen or Zhou, Zhang and Yi a relative accuracy metric is also used. This metric not only considers the exact classification into the same class, but also the classification results in adjacent classes (1-Away) [52, 71].
Results
Research framework
Research framework
The list of variables
When looking at the results, the production variables were only predictive of box office performance at 60– 67% accuracy. Adding international variables to these movie related variables in model 2 increased the accuracy using FR, MLP, and GBT. Adding distribution variables to the two other variable groups increased the accuracy even further. Adding distribution variables to model 1 (production variables) was not as accurate as model 3 results, thus showing all three variable groups (movie related, international release and distribution) independently influence the accuracy of predicting box office performances. The models where all variables were added gave the highest performance. This demonstrates the importance of both local and international distribution decisions in predicting the success of an American film in Turkey.
Results of experiment 1
When algorithms are evaluated in terms of their performance accuracies, among the five algorithms random forest yields the best results with 82.4% accuracy. The accuracy of the neural network and gradient boosting tree algorithms are close to each other. (79.2% – 79.8%) and the decision tree was able to predict 72.4 % of the classes correctly.
In the second part of the experiment, the number of target classes was increased to five and then into eight to compare performances with the previous models of different markets in the literature. Quader, Gani, and Chaki use profits as the dependent variable and discretized it into five [46] and Sharda and Delen discretized box office revenues into nine classes [18]. Based on the distribution of data, our dependent variable is divided into eight classes. The models, where the target value is discretized into five and six, our models can outperform the models from the previous studies. With 8 classes, we obtained not better but very close results. (Table 4) Possible reasons for this will be discussed in the next section.
Results of experiment 2
In this study, initially, an experimental investigation was conducted to explore how each variable group such as production, international and local distribution affects the performance of the models. Three variable groups all helped to improve the prediction performance all algorithms Here it was specifically aimed to observe the impact of the international release variable group, as in some studies using econometric models these variables were found effective, but to our knowledge, they have never been tested on models using machine learning algorithms and it is observed that they contribute to the accuracy performance of the models but found that local distribution variables are more crucial.
Among the algorithms employed in this study, random forest gives the best results. Due to its high predictive power, random forest has been highly favored in recent years. In addition to its predictive power, being easy-to-implement, fast and its ability to provide explanatory insights makes this method quite popular. The results of this study prove the success of random forest in addressing difficult problems.
When we compare our model with similar models in the literature, it is seen that our model obtains promising results, where it outperforms other models with 5 target categories and achieved close results to the model with nine target categories. It must be stated that although similar studies as ours are chosen for experiment 2, the algorithms were not applied exactly under the same conditions. The number of movies Quader, Gani & Chaki’s study is less than ours where Sharda & Delen were able to work with a larger database [46, 52]. In the future this study can be repeated with a larger database, to gain higher accuracies. For example, when Sharda and Delen repeated their study in 2009 with 2632 films instead of 834, the accuracy of the neural network increased from to 36.39% to 52.50%. [18, 52].
Also in this study, we preferred to apply split percentage and not to interfere with the parameters to make models easier and faster, however in the further studies cross-validation or hyperparameter optimization techniques can be used to improve model performance. Also ranges for classes are classes are determined arbitrarily based on trial and error methods, we first used equal-frequency binning, and then tried different ranges according to the estimation results and the distribution of the data, but in Sharda and Delen’s study classes are determined based on the advice of the business executives.
Conclusion
To best of our knowledge, this is the first attempt trying to estimate the box office performances in Turkey, though it has several limitations that need to be addressed. First of all, numbers and ranges for target classes are determined arbitrarily and are set based on the distribution of the independent variable. However, in the future, consulting experts in the industry such as Sharda and Delen determining the interval intervals based on the expertise’ advice can help to achieve more satisfactory results and would help to represent the market more correctly.
Another important limitation of this study is that only volume, number of comments, as an indicator of word of mouth is used as an indicator of awareness. Although volume is found that has more explanatory than valence (Liu,2006), illegal film consumption is very common in Turkey. To minimize the impact of it we excluded comments after the movie is no longer on screens. However, the assumption that all pre-release comments show expectations may be incorrect. Considering the high piracy rates in Turkey, some pre-release comments are posted by people, who watch the film on an illegal movie streaming site. Although excluding the comments after the movie is no longer on screens can minimize the impact of illegal consumption, still, a finer separation of the comments type might increase the performance of the model. So, in future studies, both pre- and post-release comments can be distinguished as to whether they indicate expectations or evaluations.
In further research, the performance accuracy of the model can be improved by adding more data or applying hyperparameter optimization techniques or by using fusion models. Besides, other forecasting methods such as fuzzy linear regression, ARIMAX can be also used to improve the results.
Once accuracy has been improved, a GUI can be added to the model to develop a DSS model where decision-makers or executives can enter parameters such as different screen number week numbers themselves and forecast the box office performance.
There are many other issues to be explored in this domain. This work can also be done for the Turkish movie market and it can be observed what parameters are important and how much they differ from American films. The effects of determinants on the box office can be determined using different models and finally, this model can be applied to other cultural or entertainment products.
As managerial implications, since our model is developed to predict the expected box office of a movie before its theatrical release in Turkey, it can help studios distributors and exhibitors in their decisions about market entry, the timing of entry or distribution strategies.