Alex Egol – RSAC Writer
The basketball season is back, folks, which means it is time to start thinking about prediction and how the season is going to pan out. Around this time every year, ESPN and other sports media sites begin generating “power rankings” that try to evaluate how strong each of the NBA’s teams will be going forward (this can be in the week between the next power ranking, but also across the entire season). ESPN is not very clear about how it creates its power rankings. To create its recent power rankings, ESPN surveyed “more than 40 ESPN NBA reporters, insiders and editors”, but in the past they have left their methods undisclosed. The natural question, for anyone interested in analytics, is: do they work?
Last year, many of us expected a lot from the Los Angeles Lakers with Lebron’s free agency move there. At the beginning of that 2018-19 season, ESPN power rankings rated the Lakers at 8 overall. In Figure I, we can see how their season panned out, besides teams such as the Golden State Warriors and New York Knicks, whose seasons, sadly for me (a Knicks fan), went as planned. Looking at the chart, we see that, although the Lakers’ win percentage remained relatively consistent throughout the season, the Lakers’ power ranking fluctuated, reaching a maximum of ranking=7 before steadily declining to an end-of-season minimum of ranking=22. Unlike Los Angeles, Golden State and New York’s power rankings and win percentages were relatively parallel to each other, and remained decently consistent throughout the season. The rest of this post will be dedicated to understanding just how predictive Power Rankings are.
Table I (2006-2019)
Figure II displays a scatter plot of the opening week Power Rankings and end of season standings, for 14 seasons (’06-’19, excluding ’12) worth of data. A linear regression of final standings on initial power ranking has a 0.58 R^2 value, a reasonably strong correlation. The tightness of the scatter plot backs that up. As another way of looking at this, Table I shows that beginning-of-season power rankings seem to be OK predictors. Looking at the top-left to bottom-right diagonal in Table I, we see that a majority of teams power-ranked in the top 10 ended in the top 10, as did middle 10 and bottom 10 teams for their respective categories. However, power rankings were far better at predicting results for top 10 and bottom 10 teams, logging 73.85% and 66.92% for those categories, as opposed to the 46.15% for middle 10. From ESPN’s perspective, this is cause for a sigh of relief. As a power ranking consumer myself, I know that the most interesting and memorable aspects of a power ranking are the tops and bottoms. ESPN clearly knows that too, judging by the marketing in their Power ranking headlines (see Headline A and Headline B).
Before we make a judgement on Power Ranking predictiveness, let’s look at Power Rankings from one more angle. If the ultimate goal of basketball prediction is to predict an NBA champion, then for anything to be called “predictive”, it hopefully should be good at predicting champions. Although it is not explicitly stated, when a Power Ranking ranks a team at number one, it is suggesting that that team is the strongest team in the NBA, and therefore the most likely to win the title. How often did power rankings rank champions at number one over the past 14 years?
Figure III tells us the power rankings across the regular season for each NBA champion since 2006, excluding the shortened season of 2012. Champions such as the 2006 Miami Heat and 2011 Dallas Mavericks present Power Rankings’ imperfection, but generally speaking, and especially for dominant franchises such as the Big-3 Miami Heat and the Golden State Warriors, power rankings are good when it comes to champions, as shown by the small number of dots concentrated towards the x-axis. Also, overall, the grand average of all champion power rankings is 3.0375, or ~3, meaning that NBA Champions (league number ones) were on average power ranked as league number threes, or, in playoff terms, as conference finalists/losing NBA finalists. Again, this is not bad.
So far, we have seen that Power Rankings are mediocre. They seem to predict good teams’ final records and bad teams’ final records fairly well. However, they often miss the mark (e.g. 2019 Lakers, the “Middle 10”, 2006 Heat, 2011 Mavericks). No ranking system is perfect, and this one surely is not. I think we will need more analysis to draw a conclusion about just how imperfect these rankings are. Answering that question will be the endeavor of our next post.
(Cover photo credit to Andy Lyons/Getty Images)