Abstract
Rank-based groupings are uninformative signals of quality when the exact rank is known; therefore, they should be ignored in decision making. However, evidence is presented that rank-based groupings are used to determine the compensation of rookie players in the National Football League (NFL). The NFL draft, which largely determines rookie compensation, provides two signals of player quality, selection number, and round. However, the rounds are simply groupings based on selection number; thus, the rounds should not affect subsequent decisions. Sharp regression discontinuity design (RDD) estimates of discontinuities in rookie compensation at the round cutoffs are shown to be very large and robust. The first to second round discontinuity is −US$240,000 to −US$250,000, or 36% of the average salary of the first selection in the second round. The second to third round discontinuity is −US$60,000 to −US$70,000, or 17% of the average salary of the first selection in the third round. The results show rookie compensation, which comprises a large share of career earnings, is subject to heuristic thinking.
Introduction
Often when making decisions people consider rankings as a signal of quality. For example, Salganik, Dodds, and Watts (2006) show how rankings affect behavior in experimental music download markets. However, many times we are presented with not only rankings but rank-based groupings. For example, we see the top 10 songs, books, movies, or college football teams. Such groupings are only informative signals in the absence of the underlying or rank. All information contained in the grouping is provided by the specific rank, which also provides more information.
A simple example of college football rankings helps to illustrate the situation. During the season, college football teams are ranked according to quality. It is very common for people to discuss the top 10 teams. In this case, the grouping is being in the top 10. The underlying characteristic is a team’s exact rank. It is clear if the exact rank is unknown, the grouping is informative about the quality of the team. However, if the ranking is known, the grouping is an uninformative signal.
Similarly, the National Football League (NFL) draft, which consists of seven rounds of selections, provides two signals of expected productivity, selection number, and round. However, several studies have shown draft information to be very noisy signals for productivity (Berri & Simmons, 2011; Hendricks, DeBrock, & Koenker, 2003). Regardless of the noisiness of selection number and round, the round signal is dominated by selection number. Rounds are simply groupings of players based on selection number. Therefore, the rounds are uninformative signals of player quality, given the selection number is known. Since there are 32 NFL teams and each team has the right to one selection per round, we automatically know that selection numbers 1 through 32 are in the first round and selections 33 through 64 are in the second round. 1
If all teams exercised their rights in each round, the round number would indicate a player’s within-team selection number. However, teams use their selections as assets that can be traded for other selections in the same year, selections in future years, or to obtain nonrookie players. As a result, it is very common for teams to select multiple times in a given round and for teams to not select a player in a given round (see Appendix for the number of teams with multiple selections in a given round).
As the rounds are simply groupings based on selection number, they should be ignored in decision making. However, we present evidence that in the presence of the noisy draft signals the NFL rookie labor market relies on heuristic thinking. 2 The rounds are very influential in the evaluation of expected productivity for rookies. Players are even referred to by the media, or members of a franchise, as first round or second round picks. The heuristic thinking manifests itself as discontinuities in the compensation of drafted NFL rookies precisely and uniquely at the round cutoffs. Discontinuities in compensation are very important because initial compensation has a large impact on the total career earnings of NFL players. As the average NFL career is very short, rookie compensation comprises a large portion of career earnings.
Model
A player’s compensation, w, is dependent on his expected productivity. When the compensation of rookies is determined, productivity is unknown, since these players have not participated in any NFL games or practices.
Where n indexes players, θ is the random variable for a player’s productivity, and X is the vector other factors determining compensation.
Teams use selection number, S, and round, R, as signals of expected productivity.
However, since the round is simply a rank-based grouping of selection numbers we have.
Thus, the round signal should have no impact on rookie compensation. Evidence that the rounds are a significant determinant of rookie compensation suggests the use of heuristic decision making in the market for drafted rookies. 3
Sharp Regression Discontinuity Design (RDD) Method
We exploit the structure of the NFL draft to conduct a sharp RDD for the first 3 rounds. The presence of compensatory picks, designed to compensate teams who lost many players to free agency, causes there not to be consistent cutoffs between later rounds (Karimian, 2011). 4 A sharp RDD requires a cutoff in the data at which point observations move, discretely, from untreated to treated. Let Ri be a binary variable for players selected in the ith round, where i = 1, 2,…,7, and all other variables retain their previously mentioned meanings. If S < ci , then Ri = 0, and if ci ≤ S < ci+1 , then Ri = 1, where ci is the cutoff point between rounds i − 1 and i. For example, if we examine the effect between the first and second rounds R 2 is a binary variable for second round selections and c 2 is selection number 33.
To estimate the size of the round discontinuities, we first estimate a pooled regression using third order polynomials of selection number (Lee & Lemieux, 2010).
The coefficient, τ, on the binary round variable is the average treatment effect (ATE) and X is a vector control variables including year, team, and position fixed effects. By including the interaction terms, R*(S − c), we allow for the effect of selection number to be different between the rounds.
The drawback to the pooled regression approach is that it weighs all observations equally in the calculation of the ATE. Therefore, we estimate the local average treatment effect (LATE) with local linear regression, which minimizes bias (Fan & Gijbels, 1996; Imbens & Lemieux, 2008; Lee & Lemieux, 2010). We use the triangle kernel because it is the optimal weighting function for boundary cases, which directly applies to RDD (Cheng, Fan, & Marron, 1997). 5 Also, we use the method proposed by Imbens and Kalyanaraman (2012) of calculating the optimal bandwidth. 6 The method is completely data dependent and derived specifically for RDD to minimize root mean squared error. 7 Since the selection of bandwidth can have a large impact on results, we use the method proposed by Nichols (2007) and report results for the optimal bandwidth, half the optimal bandwidth, and double the optimal bandwidth.
RDD requires the inability of individuals to perfectly manipulate assignment, called exchangeability, which allows treatment around the cutoffs to be randomized (Lee, 2008; Lee & Lemieux, 2010; Nichols, 2007). This condition is satisfied since prospective rookies cannot be perfectly manipulated into one round or another. The standard tests for manipulation are to examine the density and individual characteristics near the cutoff. In the NFL draft, there is a single player chosen at a given selection number creating equal densities on either side of the cutoff. We present further justification of the exchangeability condition in the Results section.
However, due to the structure of the NFL draft, there are two concerns. First, the average within-team selection number changes between rounds. As previously mentioned, NFL teams trade selections so the change is not discrete, but the average within-team selection number does change. Since selection order within each round, before trades are considered, is the reverse order of previous season team success, team quality is also a concern. Again, due to trades the change is not discrete, but the average team success varies between rounds. As a result, we conduct various robustness checks for these issues, which are further detailed in the Results section.
Data
The data cover every player drafted in the first three rounds of the NFL draft from 2002 to 2009. The data begin in 2002 because the NFL expanded in 2002 from 31 to 32 teams. The data end in 2009, as it was the final year of the collective bargaining agreement (CBA) between the NFL owners and the National Football League Players Association (NFLPA; National Football League, 2008). The CBA originated in 1993, but 2009 was the final year under these guidelines. As compensation in the NFL is dictated by a salary cap, which is specified in the CBA, the data end in 2009 for consistency.
The NFL also has a salary cap for rookies, a cap within a cap. Prior to the draft, the NFL uses a secret formula to determine each team’s specific dollar amount available to sign rookie players. The equation is kept very private to the NFL and is not disclosed to teams, players, players’ agents, or the public (Mirabile, 2007). The NFL makes no attempt, as stated in the CBA, to suggest or restrict how the money is allocated (Florio, 2011). Teams are simply told the amount of money they have to sign their rookie players. A player’s salary cap value is the sum of his base salary, incentive bonuses, and signing bonus prorated for the life of the contract. Due to the amortization of signing bonuses, we use salary cap value as the measure of rookie compensation.
The data were collected from two sources. The USA Today (2011) maintains a database of professional athletes’ salaries. We use drafted rookie salary cap values between 2002 and 2009. The second source, Pro Football Reference (2011), was used to collect all draft and team-related information: selection number, round, year, position, team, and previous season team wins. Descriptive statistics are reported in Tables 1 and 2.
Descriptive Statistics—Mean Compensation by Round and Year.
Descriptive Statistics—Compensation and Pick by Position.
Results
We first examine the exchangeability condition by analyzing the number of games started in a player’s third season and Pro Bowl selections for players selected near the cutoff between the first and second rounds. We use games started and Pro Bowl selections as productivity measures due to the presence of multiple positions. We calculate both measures for selections 30 through 32 and 33 through 35. The average number of games started for the first round players is 9.96 with a standard deviation of 6.12. For the second round players, the average number of games started in their third season is 10.25 with a standard deviation of 6.12. For the specified first round selections, there are two players who have been selected to the Pro Bowl, through the 2012 season. The total number of Pro Bowls for the first round players is 9. Three second round players were selected to the Pro Bowl, totaling eight selections. Both measures point to the fact that players selected close to the Round 1 cutoff are equal in productivity.
One benefit of RDD is the discontinuity is easily seen on a graph. Figures 1 and 2 display the mean compensation by selection number for the first two rounds and the second and third rounds, respectively. 8 It is evident that there are indeed discontinuities in salary cap value between rounds.

First and second rounds mean compensation by selection.

Second and third rounds mean compensation by selection.
Cubic Regression Results
Pooled cubic regression results including year, team, and position fixed effects are reported in Table 3 for first and second round selections and Table 4 for second and third round selections. Since selection number is discrete, we follow Lee and Card (2008) and correct the standard errors for clustering on selection numbers. The ATE, or round effect, is statistically significant at 1% for all estimations. The first to second round baseline estimate is −US$255,300. 9 This is even more surprising when considering the mean salary for the 33rd pick, the first selection of the second round, is US$702,680. Therefore, the first to second round effect is 36.3% of the average compensation of the first selection in the second round. The baseline second to third round ATE is −US$71,810. Since the mean salary cap value of pick number 65 is US$430,527, the estimated round effect is 16.7% of the average salary of the first pick in the third round.
Pooled Results First and Second Rounds.
Note. ATE = average treatment effect. Robust standard errors in parentheses. Standard errors are corrected for clustering on selection number. Estimations exclude the outlier. Wins and Wins-squared are for the previous season. Fixed effects are estimated using binary variables.
*Significant at 10%. **Significant at 5%. ***Significant at 1%.
Pooled Results Second and Third Rounds.
Note. ATE = average treatment effect. Robust standard errors in parentheses. Standard errors are corrected for clustering on selection number. Wins and Wins-squared are for the previous season. Fixed effects are estimated using binary variables.
*Significant at 10%. **Significant at 5%. ***Significant at 1%.
As previously mentioned, average team success and average within team selection number shift at the round cutoffs. Therefore, we include the number of previous season wins and the number of wins squared, to allow for a nonlinear effect of team success, and a vector of binary variables indicating a player’s within team selection number. The first to second round effect is −US$242,400 and the second to third round effect is −US$64,570. 10 The results are robust to changes in team success and within-team selection number and support our RDD identification strategy.
Local Linear Regression Results
We estimate the LATE using the triangular kernel and the Imbens and Kalyanaraman (2012) optimal bandwidth. We analyze the round effects both with and without year fixed effects. Local linear regression is normally estimated without additional covariates; however, year fixed effects are included for comparison, as there is clear impact of the year on salary cap value which is exogenous from the effect of selection number. The results are presented in Table 5. The first to second round discontinuity is robustly significant at 1%. It is robust to the selection of bandwidth and the inclusion of year fixed effects. Similar to the pooled regression results, the first to second round discontinuity is very large. The optimal bandwidth displays an LATE of −US$237,000 to −US$251,000. 11 The second to third round LATE using the optimal bandwidth is −US$60,000 to −US$62,000.
Local Linear Regression Triangle Kernel Wald Estimates
Note. Standard errors in parentheses. Optimal bandwidth calculated using Imbens and Kalyanaraman (2012) method. Fixed effects listed as regression titles.
*Significant at 10%. **Significant at 5%. ***Significant at 1%.
Possible Mechanisms
Our study is similar to Lacetera, Pope, and Sydnor (2012), who find evidence of the left-digit bias in used vehicle prices, in that there is no actual treatment being administered at the cutoff points. 12 Therefore, we explore possible psychological mechanisms that may cause people to use rank-based groupings in decision making, thus creating the round effects. It is possible the round discontinuities are the result of the rookie salary cap. As previously mentioned, NFL teams are allotted a specific dollar amount to sign rookie players. Although the equation determining each team’s allotment is kept secret to the NFL, NFL general managers and players’ agents may have uncovered the mechanism to some degree. In fact, the CBA, which was supposed to run until 2012, explicitly states the rounds are considered in the rookie salary cap (National Football League, 2006). However, in the CBA discussion of rookie compensation, for purposes of incentive bonuses the first three rounds were grouped together. Therefore, it is possible the round discontinuities are an artifact of the secret equation, but unlikely.
A more plausible explanation is the theory of anchoring and adjustment, which has been shown in many situations (Ariely, Lowenstein, & Prelec, 2003; Northcraft & Neale, 1987; Tversky & Kahneman, 1974). Anchoring takes place when individuals focus on a particular, often irrelevant, piece of information. Given the anchor, additional information is used to make adjustments and decisions. If there is not substantial adjustment, the anchor will have a significant impact on decision making. For example, Ariely, Lowenstein, and Prelec (2003) asked individuals whether or not they would purchase several ordinary consumer products, one of which was a bottle of wine, for a price equal to the last two digits of their social security number. Individuals then stated their willingness to pay (WTP) for each good. The results show an individual’s social security number had a significant impact on their WTP. Teams and agents may be using the round as an anchor. Expected productivity is then adjusted based on selection number. Round discontinuities are created due to a lack of adjustment and the anchor being changed at the round cutoffs.
Another mechanism, which is very similar to anchoring and adjustment, is the use of focal points to facilitate coordination (Schelling, 1957, 1960). Schelling discusses focal points in situations where agents cannot communicate, but are required to coordinate. The classic example is being told you must meet a person in New York City and the two of you must arrive at the same location at the same time. Here, people often coordinate using focal points, which are common meeting places at common times. Rookie compensation is a coordination situation, as teams bargain with players’ agents. Teams and agents may use the round as a focal point or common starting point for negotiations. At the round cutoffs, the focal point shifts, which may result in the observed discontinuities. It is important to note that any possible alternate explanation must generate discontinuities uniquely at the round cutoffs.
Conclusion
The NFL draft is analyzed to examine the effect of rank-based groupings on decision making. The rounds of the draft are simply groupings of players based on selection number. As rank-based groupings are uninformative signals given the rank is observed, the rounds should be ignored in decision making. Discontinuities in rookie compensation at the round cutoffs are evidence of a causal effect of the draft rounds on compensation. Thus, they are evidence of the use of rank-based groupings in high-stakes decision making.
A sharp RDD analysis of the round cutoffs reveals very large and robustly significant round effects on rookie salary cap value. The analysis uses both pooled cubic and local linear regressions. The estimated first to second round effect is −US$240,000 to −US$250,000. The estimated second to third round effect is −US$60,000 to −US$70,000. Both effects are robust to team success and within-team selection number, which vary between rounds. Later round effects could not be estimated using RDD because there are no consistent cutoffs between rounds throughout years.
Evidence of the effect of rank-based groupings on decision making in the NFL is important for several reasons. First, rookie compensation comprises a large portion of career earnings for players. Heuristic thinking in the compensation of rookies has a major impact on lifetime earnings, especially for those near the round cutoffs. Second, the use of a sharp RDD quasi-experimentally estimates the causal relationship. Third, the results are derived from real-world decision making. Finally, the use of rank-based groupings is shown in a market with very high stakes.
Footnotes
Appendix
Number of Teams With Multiple Selections in a Given Round.
| Round | |||
|---|---|---|---|
| Year | 1 | 2 | 3 |
| 2002 | 1 | 7 | 4 |
| 2003 | 4 | 3 | 4 (1) |
| 2004 | 5 | 8 (1) | 3 |
| 2005 | 4 | 3 | 7 |
| 2006 | 3 | 3 (1) | 6 (1) |
| 2007 | 2 | 6 (1) | 7 (1) |
| 2008 | 5 | 4 (2) | 9 (4) |
| 2009 | 4 | 7 (2) [1] | 5 |
Note. The number of teams with two selections in a given round. Numbers in parentheses indicate the number of teams with three selections in the given round. Numbers in square brackets indicate the number of teams with four selections in the given round.
Author’s Note
I am grateful for the feedback of Joshua Tasoff, Thomas Kniesner, and Serkan Ozbeklik. All errors are mine alone.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
