Predicting the top performing teams
The absolute predictions of these algorithms are not the most important metric to judge them by, it is the relative performance of the teams that matters. We really need to have the ranking of performances correctly, so we will test these algorithms by seeing how accurately we can predict the top 3 teams.
For a selection of three teams out of 32 in the NFL, the likelihood of randomly drawing one of the top three in a given metric is 9.4%. To randomly guess all three correctly, the likelihood is 0.020%. These serve as a baseline to compare our precision and recall metric with, though there is not a 1 to 1 comparison we can make between the values. There are more than three teams that can appear in the top teams, as say, if no team fumbles the ball in a given week, you would have 32 teams in the "top 3" for fumbles for the week. We will however use the precision and recall metric for "team in the top three" for a week, for our test data set over 2020-2022.
| Field Name | N in top 3 | N predicted in top 3 | N correct | Precision | Recall |
|---|---|---|---|---|---|
| Complete Passes | 254 | 965 | 220 | 0.228 | 0.866 |
| Defensive Points Allowed | 628 | 1354 | 628 | 0.464 | 1.000 |
| Field goal successes close | 810 | 1354 | 810 | 0.598 | 1.000 |
| Field goal successes far | 1013 | 1354 | 1013 | 0.748 | 1.000 |
| Sacks | 364 | 1354 | 364 | 0.269 | 1.000 |
| Touchdown Passing | 584 | 1257 | 546 | 0.434 | 0.934 |
| Touchdown Rushing | 705 | 1354 | 705 | 0.521 | 1.000 |
| Touchdown Extra Points | 470 | 1175 | 395 | 0.336 | 0.840 |
| Yards Passing/Receiving | 189 | 372 | 82 | 0.220 | 0.434 |
| Yards Rushing | 184 | 656 | 128 | 0.195 | 0.696 |
Some of the fields only make sense to compare the top 1, or best performing. In cases like fumbles, where many teams may only be at 0, this category will be crowded and useless as a top 3. We provide that result below, and recommend comparing against a simple 3.12% chance of guessing randomly the top performing team.
| Field Name | N top | N predicted in top | N correct | Precision | Recall |
|---|---|---|---|---|---|
| Fumbles | 443 | 471 | 165 | 0.350 | 0.372 |
| Fumbles Forced | 443 | 471 | 165 | 0.350 | 0.372 |
| Interceptions caught | 108 | 1207 | 89 | 0.074 | 0.824 |
| Interceptions thrown | 634 | 1349 | 632 | 0.469 | 0.997 |
Comparing our metrics against the random samples, it's clear most of our forecasts at least approximate better than random guessing. We are capturing a signal, and able to predict based on it.
However, there is a big difference between being able to predict values, and the predictions being useful. In some cases the top performers are predicted at twice the rate as they should be. While not a terrible performance on paper, someone having a pen and pencil following players and teams can likely perform on a similar or better level than this algorithm with simple assumptions.
This project is an ongoing one, and I've already outlined places for improvements. I will update these pages when appropriate, and when results improve.