We all want to know an answer to a question. Why? Or we want to know some form of a question. Is there….? We read, search, listen, wanting to educate and enrich ourselves daily.
As a recurring guest on Locked On Bills, Joe Marino of The Draft Network gets a lot of those questions and addresses them on Herd Mentality weekly.
Joe came to me with a question that he received recently and asked me to dig into the question. In the medical world, that is known as referring out. This became a deep dive into how NFL teams use questionable and doubtful designations. This research looked at the frequency of the usage, who played and who didn’t, and how this compared to the Buffalo Bills.
QUESTION
(Lightly edited for clarity and grammar)
Submitted following Week 11 of the 2021 NFL season.
Are the Bills too conservative with injuries? I heard a lot this week about how Darius Leonard had been playing through a high ankle sprain all season, and sure enough, he was out there again against the Bills. It got me thinking about how earlier this year, Milano was pulled out of a game late for his hamstring after the lightest of pushes from behind against the Texans, and I remember thinking there was no way he was missing time. He was held out the next week – Chiefs game, I believe. He was walking around fine all week at practice, and also on the sidelines of that game.
Then Edmunds was pulled out two weeks ago for hammy concerns, also seemingly a minor injury at the time. He’s held out against the Jets – but I was at the game, and he was horsing around in pregame warmups (impossible to miss, he’s a monster even in street clothes) and certainly didn’t look hobbled. Missing this week was unfathomable to me after watching that, but nope – inactive. Even an 80% Edmunds would’ve been better than none against Taylor today.
It’s a recurring point with Kyle how they always ebb on the side of caution with injuries – is it too much so, and possibly even to a point of detriment? This question comes to mind in a context where there’s a lot of chatter happening that questions other areas of McDermott’s conservatism on game days lately too.
PARAMETERS OF THE ANALYSIS
- All information was gathered from Pro-Football-Reference.com
- Timeline for assessment was 2017-2020, this coincides with the start of the Sean McDermott era
- Research was focused on the frequency of the questionable/doubtful designation used
- Team records were noted
- Coaching changes were noted
- Injured reserve designations were excluded
- Man games lost were excluded
- Team Averages
- League Averages
- Bills injury breakdown by soft tissue, sprain, concussion, or other
The official NFL Personnel Injury Report Policy does list designations of Out, Doubtful, and Questionable, but does not list an odds or chance as to what that designation has to be. You can read the full policy here from 2017.
Pro Football Reference breaks the designations down into color codes and percentages for easy understanding.
-Questionable (Yellow): has a 50 percent chance of playing
-Doubtful (Orange): has a 25 percent chance of playing
-Out (Pink): the player did not play that week
-//////: the player did not play that week, this is seen with Questionable, Doubtful, and Out designations
*Probable (Green): this was eliminated in 2016 due to the overwhelming number of players deemed probable that ultimately played.
RESULTS
Overall, the league average from 2017-2020 for questionable designations that ultimately played was 66.67%.
The league average for doubtful players that ultimately played was 1.77%
The three teams with the highest frequency instance of the questionable designation were:
New England Patriots: 487
Baltimore Ravens: 330
Arizona Cardinals: 257
The three teams with the lowest frequency instance of the questionable designation were:
Atlanta Falcons: 52
Los Angeles Rams: 65
Pittsburgh Steelers: 73
The three teams with the highest frequency of questionable players playing:
New York Jets: 89.5%
Tampa Bay Buccaneers: 89%
Baltimore Ravens: 77.8%
The three teams with the lowest frequency of questionable players playing:
Houston Texans: 41.4%
Pittsburgh Steelers: 46.58%
New Orleans Saints: 46.9%
The three highest single-season frequency with questionable designations playing were:
2017 Tampa Bay Buccaneers (5-11): 96.3%
2019 Tampa Bay Buccaneers (7-9): 96%
2019 New York Jets (7-9): 93.2%
The three lowest single-season frequencies with questionable designations playing were:
2018 New York Giants (4-12): 12.5%
2020 Pittsburgh Steelers (12-4): 20%
2018 New Orleans Saints (13-3): 27.2%
To note
Not one team in the AFC had a player listed as doubtful and had them play. All instances were found in the NFC.
DISCUSSION
Looking at the usage of the questionable designations, this appears to be team-dependent.
I did not include doubtful designations into this as less than 2 percent of players suit up during this time frame, making this rather insignificant. It is important to note how teams use the doubtful designation. The team with the most doubtful designations over the 4-year period is the New York Jets with 78 and the least being New Orleans Saints with 0.
I am not seeing a strong correlation between team record and questionable designation usage. The Pittsburgh Steelers and New Orleans Saints have had wildly successful seasons yet had less than 30 percent of their questionable players play that season. We have seen subpar teams in the Tampa Bay Buccaneers and New York Jets have nearly every questionable player suit up.
We are seeing highly successful teams be conservative and still win. We have also seen aggressive teams still be successful as seen below.
The last four Super Bowl Champions by questionable designation playing percentage:
2017 Philadelphia Eagles (13-3): 38.89%
2018 New England Patriots (12-4) 65.69%
2019 Kansas City Chiefs (12-4) 53.13%
2020 Tampa Bay Buccaneers (11-5) 70.83%
Training staff turnover was assessed to see if there was any connection between coaching staff changes and the injury management approach. Most of the training staff stayed the same or had some level of continuity despite coaching changes.
There were coaching staffs that brought their trainer from their previous team to their new team immediately such as Ron Rivera going from the Carolina Panthers to the Washington Football Team or the Jacksonville Jaguars bringing in a new athletic trainer with the hiring of Urban Meyer.
There were other athletic trainers moving around during this time period, but I did not see the shake-up of the training staffs that I had expected to see with coaching changes.
I did see some changes in training staff use of questionable designations with the Chicago Bears being more aggressive going from 33.9 percent of questionable players playing in 2017 up to 70 percent the next year after hiring a new head athletic trainer.
However, most of the rates stayed relatively steady. Whether this was the approach of the training staff or whether this was the incidence of injury, that’s not clear. We saw the Miami Dolphins drop from 80.6 percent in 2017 down 65 percent in 2018 with a new head athletic trainer.
The Philadelphia Eagles had a wholesale change in 2018, one year removed following their Super Bowl win, most of their athletic trainers were new since 2018.
The New Orleans Saints also had some turnover in 2018 with the firing of several orthopedic doctors, but there was continuity with one assistant athletic trainer being there since 1981 and the other in 2014.
However, injuries are random and the frequency of these injuries varies greatly between teams. We do see a correlation in how injuries are managed from team to team, but it doesn’t appear that changes in the coaching staff, injury frequency, or record have a notable say in questionable players suiting up.
Getting back to the original question, are the Bills too conservative with injuries, or is it to their detriment?
Breaking down the Bills’ questionable designations by year, record, and designations.
2017
9-7 record
Total Questionable designations/Questionable played: 47/33 = 70% play
Soft tissue questionable/played: 9/5 = 55%
Back questionable/played: 6/4 = 67%
All other questionable/played: 32/24 = 75%
OUT (28 total): Soft tissue (5) 18%, Concussion (6) 21%, Sprain/Tear/Contusion (17) 61%
2018
6-10 record
Total Questionable designations/Questionable played: 32/17 = 53%
Soft tissue questionable/played: 4/1 = 25%
Back questionable/played: 1/1 = 100%
Concussion questionable/played: 3/1 = 33%
All other questionable/played: 24/14 = 58%
Doubtful/played: 2/0 = 0%
OUT (21 total): Soft tissue (4) 19%, Concussion (6) 28.5 % Sprain/Tear/Contusion (11) 52.3%
2019
10-6 record
Total Questionable designations/Questionable played: 38/22 = 58%
Soft tissue questionable/played: 14/5 = 36%
Back questionable/played: 2/2 = 100%
Concussion questionable/played: 3/3 = 100%
All other questionable/played: 18/12 = 66.6%
Doubtful/played: 1/0 = 0%
OUT (19 total): Soft tissue (6) 30%, Concussion (1) 5.2% Sprain/Tear/Contusion (12) 63.1%
2020
13-3 record
Total Questionable designations/Questionable played: 40/26 = 65%
Soft tissue questionable/played: 12/8 = 67%
Back questionable/played: 0/0 = 0%
Concussion questionable/played: 1/0 = 0%
All other questionable/played: 27/18 = 66.6%
Doubtful: 0/0 = 0%
OUT (27 total): Soft tissue (12) 44%, Back/Neck (1) 3.7% Concussion (2) 7.4% Sprain/Tear/Contusion (12) 44%
2021* (Through Week 12)
7-4 record
Total Questionable designations/Questionable played: 15/9 = 60%
Soft tissue questionable/played: 5/3 = 60%
Back questionable/played: 0/0 = 0%
Concussion questionable/played: 1/1 = 100%
All other questionable/played: 9/5 = 55%
Doubtful: 1/0 = 0%
OUT (7 total): Soft tissue (1) 14.2% Back/Neck (1) 14.2% Concussion (1) 14.2% Sprain/Tear/Contusion (4) 57.1%
Digesting all this information, breaking everything down, the Bills initially got more conservative with injuries, especially since the head athletic trainer changed after 2017. However, they have since improved in getting guys back onto the field more frequently, demonstrating a reverse bell curve.
They continue to be below the league average as a whole with overall questionable designations playing at 62 percent, good for 21st overall out of 32 teams.
With regards to soft tissue, they once again showed that reverse bell curve in how many players suit up with soft tissue injuries, hitting just over the league average with soft tissue injury rates alone in 2020. They are below in 2021, but this number continues to change.
The Bills have 3 athletic trainers on staff along with a physical therapist. They did hire a new head athletic trainer in 2018 with Nate Breske and Tabani Richards and had one holdover from 2017 in Danny Kelligton along with Joe Micca as a physical therapist since 2015 that demonstrated continuity for the players.
The new practice facility and wellness center opened in 2019. We’ve seen injuries as a whole appear to stay suppressed with questionable and out designations and the rates at which players play with a designation or are ruled out.
CONCLUSION
So the question is, have the Bills gotten more conservative with injury management this season as we have seen Sean McDermott get conservative with his performance?
It has varied over the years, initially conservative and then more progressive, inching closer to the league average in the coming years.
In 2021, they have been more conservative this year but the use of injured reserve last year and this year likely heavily influences this. This metric was difficult to account for on Pro Football Reference due to their confusing record keeping.
It’s important to note that they have not been significantly more conservative than in past years. The Bills continue to be conservative with soft tissue injuries but on par with the overall league average when dealing with general injuries such as sprains 3 out of the 4 total years.
LIMITATIONS OF THIS ANALYSIS
-All data is collected from Pro Football Reference. This is the most comprehensive source of information I have found to perform this assessment. I have at times found inaccuracies or incomplete information. However, using the same source does even the playing field.
-How effective were these players if they played through a questionable designation looking at snap counts or statistics?
-How do the rates of soft tissue injuries, concussions, strains in terms of playing for the Bills compare against the league when breaking their specific injuries down?
-More statistical analysis to evenly compare years against each other and the incidence of injuries as some teams either deal with a lot of injuries or report everything or teams avoid injury and don’t have a lot to report. The overall sample size must be evenly weighted. (If anyone has a statistical background and would like to dig into this further, please reach out.)
This is the first known in-depth look at the questionable rates since 2018 when fellow colleague Dr. David Chao looked at the wide variance of questionable designations following the 2017 season. You can read his analysis here.
This was fascinating to perform the research and I believe this is something that can be further addressed with more time and information.
