What does the Department of Analytics mean?

When Russ Brandon announced the formation of a Department of Analytics a couple of weeks ago it left many fans wondering what that meant and what possible applications there were to the Bills talent evaluation process. Well you can count us among those fans who were interested in the answers to those questions. We are not experts in analytics nor do we  have any intention of trying to act like it, so we turned to people who we consider experts.

Two websites currently take statistical study and tape breakdown to a level many people have never even considered. Both Pro Football Focus and Football Outsiders are amazing sites that provide a service to their subscribers that you cannot find anywhere else. We wanted to reach out to people at both those sites to help get some insight of what the Bills may be implementing.

Steve Palazzolo, Analyst, Pro Football Focus

1. What are analytics or the study of them when it comes to football in your opinion?
There are a number of ways to look at the term “analytics.” Some sites have tried to become predictive in nature, similar to the trend in baseball toward sabermetrics, but football is such a complicated, team sport that it’s often a difficult practice. Other analysts have done numerous studies on things like “how often should a team go for it on 4th down,” and while there is certainly value in those studies, I don’t think a team of analysts is needed to reap the benefits. So this one part of “analytics” is taking the raw data that is available and breaking it down further to find the numbers most commonly associated with winning football.

At PFF, it’s a little different as we have a plethora of numbers, but they’re all derived from film study of every single play of the season. We are able to provide an evaluation on every player in the NFL, all through a similar set of eyes. Perhaps our biggest contribution to the football industry is our quantifying of pressure, both for pass rushers and offensive linemen. Our ability to dig beyond sack numbers has changed the game in my opinion, and it’s that type of analysis that we continue to improve upon on a yearly basis.

2. What are the possible benefits of analytical evaluation? Can it really help tell the difference between two players?
There are a number of benefits and it absolutely helps us discover the differences between two players. The benefits are endless, whether we’re talking NFL teams, agents, media, or fans. Every sport is being inundated with new stats and analysis, but I think the proper analytics, when applied to football, can change the way we view the game for the better. Some of the data that we supply, whether it’s pass rush numbers, coverage stats, Time in the Pocket numbers, or even our in-depth special teams analysis all bring new perspective to the game.

As far as differentiating between two players, again I think the proper numbers can be very useful. There will never be one magic number that completely distinguishes one player from another, though the closest thing we have is overall PFF grade. It’s a good way to evaluate a player’s performance, though scheme, surrounding talent, and other factors always need to be considered within the context of the grades. But even beyond that, our film study breaks players down in all aspects of their game, and it does so against players who play the same position, so it’s a big advantage when trying to decipher a player’s strengths and weaknesses.

3. Working with Pro Football Focus what are some of the key aspects that you find useful in your evaluations that are applicable to NFL teams?
There is a lot. First, we can effectively act as a scouting department for all 32 teams and we do so in a timely manner. Usually within 24 hours after a game is completed, we have grades for every player, detailed reports on snap count and alignment, formation and personnel data, as well as strategic tendencies. This is the type of stuff that may take hours on end for a team’s coaching and scouting staff to accumulate, but we have it for every team very quickly.

Of course, in addition to the strategic data, I think the grading is of value to NFL teams as well. Since every NFL player is essentially viewed through the same lens, it’s a lot easier to compare player performance.

4. Is this move to form a Department of Analytics are innovative and forward thinking move or one that may be seen as a waste of time?
For all of the reasons mentioned above, I think the move toward Analytics Departments is essential over the next few years. There are a number of teams already using our data either as a paid service or simply through our subscriptions that are available to the public.  The paid service is essentially what I previously mentioned, where a team can have access to an extremely detailed scouting report on future opponents. To this point, we’ve had very positive feedback from the front offices that use our data and as we continue to grow and improve our processes, I expect more teams will follow suit.

5. How difficult will it be for the average fan to understand and comprehend the analytical data that a site like PFF produces?
This may be one of my favorite things about PFF. In addition to our detailed information that is sent to NFL teams, the site is extremely intuitive and user-friendly. When looking at the grades, there’s not much easier than seeing “green is good” and “red is bad,” while our more detailed metrics make enough sense that the average fan can gain understanding quickly. All of the grades are broken up to distinguish between the passing game, running game, penalties, etc. Again, it’s very intuitive and easy to understand.

Rivers McCown, Assistant Editor, Football Outsiders

1. What are analytics or the study of them when it comes to football in your opinion?
Analytics in football is basically a three-branch path. The first two are math-based: play-by-play analysis and situational analysis. Our main stat at FO, DVOA, is a percentage-based stat that shows how much better or worse a team is at something than every other team has been in this season. For instance, when we say the Jaguars had a -18.3% DVOA on offense, that means that they performed, on a play-by-play basis, 18.3% worse than the average offense this season. Situational analysis is the kind of stuff that you’re more likely to find with Brian Burke at Advanced NFL Stats: specific breakdowns of when it makes sense to go for it on fourth down, for example. Finally, there is tape study. That’s much harder to quantify in our opinion, because even when you see the tape, you don’t know the actual play calls. Despite that, tape-watching is very important for picking up on habits and tendencies, as well as individual counts of things like broken tackles or blown blocks.

2. What are the possible benefits of analytical evaluation? Can it really help tell the difference between two players?
Objective analysis in general is a fail-safe against flawed common logic. Common logic will tell you that Kyle Boller’s got one of the best arms of all-time and should be a first-round pick. Statistical analysis could tell you that his college statistics didn’t portend good things in the NFL. I think you’re speaking more to mathematical analysis than tape analysis here, so I’ll kind of assume that for the second part of this question since anyone can tell you that tape study could help tell the difference between two players. I think in football, statistical analysis runs into more problems with context than in any other sport. That said, there are important things to be gleaned from regression analyses and there are ways of weeding out what you don’t want from what you do. Once you hit a certain baseline, that’s when it’s time to rely on scouts and tape study.

3. Working with Football Outsiders what are some of the key aspects that you find useful in your evaluations that are applicable to NFL teams?
I can’t really say that there is *one* key aspect that stands out. Because of how complicated a ballet of 22 players moving at the same time is, I feel like most of what we are doing is trying to find the most correct analysis possible after we build up a complete stash of information. For example, one of the things we wrote about for last weekend’s Bengals-Texans game was that despite having Geno Atkins (a deserving All-Pro), the Bengals weren’t very good on runs up the middle. Does that mean that Atkins is overrated? Probably not. Does it mean Domata Peko is having an off-year? Maybe. But if you asked me the real crux of the problem, it was probably poor linebacker play from Rey Maualuga. Looking at his advanced statistics, his average tackle came 4.6 yards past the line of scrimmage. I believe some of our competitors had him come out fairly poorly as well. So in a sense, football analysis runs a lot like that on the micro level: surprising problems, and testing a lot of hypotheses with the numbers.

4. Is this move to form a Department of Analytics an innovative and forward thinking move or one that may be seen as a waste of time?
Well that’s a layup question for me: of course I think it’s an innovative and forward-thinking move. To people who would see it as a waste of time, the way I’d present it to them is to think of it kind of like spellcheck. There are some glaringly wrong things that happen on a football field — both from a coaching and player personnel standpoint — and having a department of analytics will help the Bills explore the best ways to fix those problems. It doesn’t necessarily mean that they’ll come up with the right answers, or that they will actually be heard and validated (as the numbers are oft-ignored by coaches, though that too is beginning to change), but it does at least light up the problem area and present some options on how to fix it.

5. How difficult will it be for the average fan to understand and comprehend the analytical data that a site like FO produces?
That really depends on where you draw the line for the average fan. I think we’ll be the first to admit that advanced statistics aren’t everyone’s cup of tea. If your idea of being a football fan is pounding beers and cheering on your favorite team until you’re out of energy, then it probably isn’t going to be very helpful for your enjoyment of the game. If you’re interested in the truth of what happened in a football game, the details of why a team lost or what makes one team better than another — and are actually willing to put in some work on creating your positions on these subjects rather than just accepting conventional wisdom or a talking head opinion as the whole truth — then these statistics are the tools that will help you build a hypothesis. As daunting as it may seem on paper, our mathematics aren’t high-end trigonometry or anything. Heck, I majored in Creative Writing in college. Aaron Schatz, our founder and editor-in-chief, was a radio DJ for several years before starting the site. We’re not pushing a bunch of highly complex subjects and formulas here; this is math founded on a common-sense approach first.

We want to thank both Steve and Rivers for their thoughts and contributions to this article. Take in what they said and if you are more interested in their services then please go here to subscribe;

Pro Football Focus

Football Outsiders

One Response to “What does the Department of Analytics mean?”

  1. Buddy Nixon says:

    Interesting to hear what two guys working in established places have to say about the state of analytics.

    To me, the exciting part is that the field is in it’s relative infancy. This type of rigor hasn’t really been applied to football, and now that the teams are taking the trend seriously, it won’t be long before we start to see some groundbreaking insights change the game.

    The best part is, the Bills should be ahead of the curve on this one.




  • Categories

  • Search BBD Archives

  • Archives