This raises the question of whether Champion Data are rewarding or penalizing players in a fair and accurate manner. They claim that their formula has been devised using ‘research into winning and losing factors in AFL games’. This could mean that, the higher your Champion Data score, the more likely your team’s score will be higher relative to the other team (i.e. your team’s percentage will be higher) or that the higher your score, the more likely it is that your team will win, whether it be by one point or 100 points. I’m going to assume the former explanation is true, since it seems to make more sense. But how well does it work in practice?
Not that surprisingly, it tends to work pretty well. The figure below compares the actual percentage of each AFL team over the first 21 rounds of the 2007 season to their estimated percentage using their cumulative Champion Data scores. Apart from West Coast, the results are all pretty close.
Team Percentages and Cumulative Champion Data Scores Over 2007 AFL Season
Importantly, the Champion Data scores appear to perform better at predicting a team’s success than simply looking at that team’s number of possessions. The root mean squared error using the Champion Data scores (which is basically just a method of summarizing the difference between a team’s actual percentage and its predicted percentage) is about half of that which you would get if you used the number of possessions. Incidentally, the formula for the Dream Team competition on the AFL website doesn’t do much better at predicting a team’s success than possessions do.
Root Mean Squared Error
|AFL Dream Team|
Readers of my post Win Score and the Productivity of Basketball Players will know that post also talked about a formula that did pretty well at predicting the success of a (basketball) team, but that I had reservations about how well it did at attributing this success to particular players. So how well does the Champion Data formula do in this respect? My guess is that similar types of problems apply, namely how do you account for the value of defenders who aim to prevent other players from collecting possessions rather than gathering possessions themselves, and how much credit should be given to the player who ultimately puts the score on the board? But I’m fairly satisfied that the Champion Data formula does a better job than simply looking at the raw numbers. On the other hand, if my Supercoach team loses its Grand Final this weekend, well…