August 31, 2020

Next Gen Stats: Intro to new Route Recognition model

Conventional counting stats like receptions and receiving yards provide a way to measure an individual player's ability to catch and move the football, but they only tell part of the story. Advanced stats like depth of target, separation window and completion probability provide greater insight, but they still leave out an important factor. Namely, which route did the pass catcher run to get open before catching the ball?

With the help of player-tracking technology, the Next Gen Stats Analytics team set out to answer that exact question, decoding one of the key elements of an offensive play call by using player-tracking data to measure which routes pass catchers are running on any given pass play.

Last month, they revealed a new set of rushing metrics derived from the ability to calculate Expected Rushing Yards. They are now introducing another new machine-learning tool: the Route Recognition model, which classifies routes by type, in real time, with the help of player-tracking data.

The methodology behind the Route Recognition model:

How the model works

The Next Gen Stats player-tracking system records the x-y location, speed, acceleration, direction and orientation of all 22 players on the field in real time. The new Route Recognition model leverages this data as inputs into a model that assigns a route type to every eligible receiver on every pass play, including tight ends and running backs. The architectural approach uses a combination of convolutional neural networks (CNNs) and long short-term memory (LSTM) networks trained on Amazon's SageMaker platform. CNNs allow them to engage with the spatial nature of our dataset (that is, where each player is on the field in a given play), while LSTM networks allows them to engage with the temporal nature of our dataset (what happens as the play develops over time).

They approached routes run by players aligned in the backfield separately from routes run by players aligned out wide, in the slot or tight, because of clear differences in route archetypes. Below are the 15 unique route types assigned to all route runners, based on their location when the ball is snapped. Note that while NFL playbooks have hundreds of variations of routes, so they narrowed it down to these high-level categories, including 10 routes for those in typical wideout alignments and five for those aligned in the backfield:

Wideout Routes (10): Screen, flat, slant, crossing, out, in, hitch, corner, post, go

Backfield Routes (5): Screen, flat, angle, out, wheel

The model was trained and validated against all routes from every passing play from 2018 and '19, including both the regular season and the postseason. All route runners were included, regardless of whether or not they were targeted; given that they could not find a difference between the shapes of targeted routes and non-targeted routes, they saw no reason to train on only targeted routes. In total, the wideout model was trained on over 100,000 routes, while the backfield model was trained on over 15,000 routes.

To avoid noise in the data from broken plays (during which pass catchers often stop running their assigned routes) and player movement after the catch (which would not tell us much about the efficacy of any given route), all routes were capped at either the moment the ball was passed forward or at a given time (4.6 seconds after the snap for wideout routes and 4 seconds after the snap for backfield routes) -- whichever came first. The optimal time limits for these route types were determined by analyzing how pass attempts played out over the past two seasons; for reference, 4.4 seconds accounted for the 75th percentile of all pass attempts by time to throw in that span.