Trying to fix the QB Rating System
ESPN’s recent release of a new quarterback rating system started me thinking of my own design to improve the method of rating quarterbacks.
Unfortunately for the ESPN system, it is not just based on statistics, but rather film study and a subjective analysis of game tape. I find that hard to agree with as a statistical method of grading quarterbacks. I’d like to find something easier and based off relatively easy data to find in a box score.
The current quarterback (or passer) rating is a fairly simple mathematical formula and only based on the following four statistics:
1. Completion Percentage
2. Yards per Attempt
3. Touchdown Percentage
4. Interception Percentage
These four statistics are multiplied by constants to yield a linear scale between approximately 1.0 (bad) to 2.0 (exceptionally good). For example, a completion percentage of 50% would give you 1.0 point for that category, while a 70% completion percentage would give you 2.0 points.
The current passer rating system just adds up the points you gain in each category and multiplies the total using another constant (in this case 16.6667) and that is your passer rating. The system further caps how much you can earn in each category (a very arbitrary 2.375 points) which yields the very awkward maximum rating of 158.3.
The flaws with this system are fairly apparent. First, it only accounts for the four categories listed. No rushing yards, no sacks, no big plays, no fumbles lost, no care in the world how good the offense is in other ways. Second it gives each of these categories EQUAL WEIGHT towards your total rating. The reality of NFL football does not agree with that statement. Historically, Yards per Attempt is a much more important statistic towards winning and losing games than Completion Percentage.
So how should we tweak the system?
First, by picking more relevant categories and weighting them properly in relation to each other and second, to eliminate the arbitrary maximum value.
After some thought and brainstorming ideas, I have decided to choose the following statistics be involved in a quarterback rating system:
1. Completion Percentage (Comp%)
I still believe completion percentage and accuracy is an important statistic, but a change in its importance and calculation is needed. When the current rating system was designed 50-55% completion percentage was typical. Now it is 60-65%. That’s a big flaw with the current system and needs to be fixed.
2. Total Yards per Pass Play (YPP)
I am trying to incorporate yards per attempt to including scrambling yards as well as pass yards since a mobile quarterback should get credit for positive plays using his legs. What I do not want to include is quarterback sneaks (which is a rush attack method) or sack yardage (I don’t want negative sack yardage to skew how aggressive a quarterback is downfield). What I have decided is to do is analyze all positive yards on “pass play calls”. If a quarterback gains them with his arm or leg, I don’t care. This is a very important statistic in historic success and will be weighted accordingly.
3. Offensive Points per Possession (PPP)
The goal of every offense is to score and it is the quarterback’s job to make sure that happens. I struggle to understand why a quarterback should be penalized or rewarded based on how the touchdown is actually scored (pass vs. run) when almost every drive is a combination of successes both ways. This statistic is representative of offensive team success since I strongly believe a quarterback is vital to their success, even in conservative or run-oriented offenses. This also rewards “situational” success like 3rd down conversions and red zone success.
4. Turnovers per Pass Play (TO%)
This is actually a simple fix to the INT% statistic used in the old formula. Fumbles lost by a quarterback should be just as costly as interceptions (in fact, likely more since some interceptions act as “punts”).
5. (20+ Yard Pass Plays minus Sacks) per Pass Play (Risk/Reward %)
I have created a minor 5th category (this will be weighted approximately half as much as the others) to take into account the risk/reward of quarterback play. I have no problem with quarterbacks who take a lot of sacks but do so at the reward of making big plays downfield. Nor do I want to rewards a quick-triggered quarterback who avoids sacks with constant throw-aways or checkdowns at the expense of ever going downfield. I thought by analyzing these two totals, thus creating a +/- or ratio it would best indicate this risk/reward behavior.
The next step is to create formulas that allow for much of what we are used to with the current quarterback rating system to be reflective in the new numbers. An average quarterback should have a rating of around 75.0 and a good/great quarterback season should be 100.0. I could also create the formulas to make a new “maximum” grade, which I think should be 200.0. This allows for some truly stellar performances to show through a little better than the old "158.3" way.
So let’s try this example of an “average” quarterback and their resulting baseline score in each category:
A. Completion % = 60% 1.00
B. YPP = 6.0 1.20
C. PPP = 1.80 0.80
D. TO% = 3.5% 0.80
E. R/R % = +2% 0.66
This total baseline score (4.4666) is multiplied by the same constant the old formula used (16.667) for a new Quarterback Rating of 77.78. I think that’s a rating fans would correlate to an “average” player. So our first math “check” seems to work out.
Let’s look at what the exceptional scores would be during a season. These are the approximate averages of the top-5 in each category:
A. Completion % = 66% 1.24 (Brees, P. Manning, Rivers, Brady, Rogers)
B. YPP = 7.50 1.80 (Rivers, Rogers, Roethlisberger, Brady, Vick)
C. PPP = 2.40 1.40 (Brady, Rivers, Manning, Schaub, Vick)
D. TO% = 1.6% 1.64 (Brady, Cassell, Freeman, Roethlsiberger, Vick)
E. R/R % = 5% 1.02 (E. Manning, Brady, River, Vick, Schaub)
This baseline total, for a quarterback that would rank around 3rd in all major statistical categories, would be 7.1 and yield a new QB rating of 118.3. Again, I think that is appropriate. As you can see from the list above, Tom Brady actually did accomplish this feat. He also led an offense (New England) that scored considerably more points than its nearest competitor.
Tom Brady’s new quarterback rating is an astonishing 129.5 but when you consider what he and the Patriots did during the regular season, it should be that high:
1. Brady completed 66% of his passes (4th in the league).
2. Brady averaged around 7.4 yards per pass play, good for 4th in the league.
3. The Patriots scored 458 points on offense (no returns/defensive TD’s) and averaged 2.90 points per possession. This was light years ahead of the league and ranks New England in the top-10 most prolific offenses ever.
4. Brady set an NFL record for fewest turnovers, committing only 5 in roughly 527 drop backs.
5. Brady had 53 pass plays of 20 yards or greater vs. only 25 sacks. This +28 was second in the league behind only Eli Manning (who was +41).
When I use my new quarterback rating formula for the rest of the top quarterbacks in the league, seven players reach the 100.0 milestone (Brady, Rivers, Vick, Rogers, Roethlisberger, Schaub and P. Manning). Again, I think this “checks out”.
Our last “check” is to resolve game-day passer ratings and what constitutes a “perfect rating”. As stated before we can cap each category to yield a maximum score and thus create the necessary parameters to yield a quarterback rating of 200.0 (which I will consider “perfect”).
In order to accomplish a game-day perfect rating, the following has to happen:
1. A quarterback has a minimum 75% completion percentage.
2. A quarterback gains a minimum 12.0 yard per pass play.
3. The quarterback-led offense scores a minimum 3.5 points per possession.
4. The quarterback can not commit a turnover (either interception or lost fumble).
5. The quarterback has to have a +10% risk/reward ratio (ex. have +3 big plays vs. sacks over a 30 pass play day).
If all these conditions exist, the quarterback will attain a perfect 200.0 rating for the game.
In conclusion, the new ESPN formula frustrated me so much and left me so disheartened, I just decided to fix the old formula myself. As I did research for this article, I think the old formula has merit and a certain level of comfort with the fans. We know what certain scores mean. But the old formula was flawed and out-of-date and did not weight categories correctly with their impact on winning games.
I hope my new system is more realistic and plan on testing it during the season.
As always, thanks again for reading and I look forward to your comments.
Appendix A:
Here are the actual formulas I have created for each category:
A. (Completion % - 35%) * .04 max Score = 1.65
B. (Yards per Pass Play - 4.0) * 0.5 max Score = 4.00
C. Points per Possession - 1.0 max Score = 2.50
D. 2.2 - (TO/PP * 35) max Score = 2.20
E. ((20+ yard pass plays - Sacks)/PP + .035) * 12 max Score = 1.65
Key:
Completion % = Pass Completions/Pass Attempts
Yards = Passing Yards + Scrambling Yards
PP (Pass Plays) = Pass Attempts + Sacks + Scrambles
Points Per Possession (PPP) = Points Scored (TD’s, XP’s and FG’s) divided by Possessions. Kneel down drives are not included.
TO = Interceptions + Fumbles Lost
20+ Yard Pass Plays = Pass Completions for 20+ yards
Sacks = Official Sacks Against
Good job, lot of work was put into that.
ReplyDeleteVery nice work. I actually like how it is more of an offense rating than strictly a QB rating, and if anything, it might be interesting to tweak your system to reflect the running attack, making it a true team statistic. After all, dropped passes and wrong routes really aren't the QB's fault. While the QB gets all the press, it is a team game.
ReplyDeleteOne question though: you say that the PPP accounts for third down conversion and red zone success. I can see how it addresses the latter, but 3rd down % seems tangental at best.
nm