Noise from Signal: On Matthew Coller’s “Football Is a Numbers Game”

IN THE FALL of 2022, some friends and I took the PATH train from Manhattan out to MetLife Stadium in East Rutherford, New Jersey, to see the New York Jets host my beloved Buffalo Bills. The idea was that seeing it live would fill in some details that you can’t glean from watching the NFL on TV: the sheer speed of the game, the noise of the crowd, the full picture of the complex plays on the field as seen from above. Instead, it was hard to follow and sort of flat. The firehose of information I’m used to—updates from other games, stats flashing across the screen, each team’s tendencies and plays analyzed by announcers in real time—was missing. I couldn’t even approximate those on my phone, because the 80,000-plus other phones in the stadium were crowding the cell signal. I left deflated, and not just because the Bills took a hard loss in which their quarterback, Josh Allen, injured his throwing arm.

As I was drafting this essay, the Bills had just fired their offensive coordinator after a similarly frustrating loss. NFL Twitter was exploding with takes; many of these are indistinguishable from the reactions, on sports talk radio and in print newspaper columns, that have accompanied these firings since the dawn of spectator sports. Timo Riske of Pro Football Focus—PFF, the football analytics company that Minnesota Vikings beat reporter Matthew Coller has profiled in his new book, Football Is a Numbers Game: Pro Football Focus and How a Data-Driven Approach Shook Up the Sport (2023)—slipped into a different syntax. He called the move “homeopathy.” The Bills offense at the time of the firing ranked in the NFL’s top five in every conceivable statistical category: yards per game, touchdowns in the red zone, and especially EPA, “expected points added,” which measures each play by its variance from the league average for a given situation. Another wing of analysts and fans, though, disagreed: watch a game, they said, and you could see this offense sputtering, undisciplined, scratching and clawing their way to make games belatedly competitive.

After the firing, the Bills went on a winning streak that ended only a few weeks ago in a heartbreaking loss in the divisional round of the playoffs. But that inflection point in their season illustrates a typical debate in the modern NFL over whether data reveals a picture that the human eye cannot. Coller writes that “signs are everywhere that data is beginning to drive decisions in the NFL.” But it’s doing more than that: it’s changing the face of football as it exists in our culture.

The tension between that data and the naked eye is inescapable today. From finance to public policy to elections, a group that is sometimes called “the Quants” tells us not to believe our eyes. Using sophisticated statistical techniques, algorithmic modeling, and the ravenous collection of unprecedented amounts of data, they purport to reveal a world hidden beneath ours. The truth for the Quants is not “out there” but in there, underneath what we see in patterns that we cannot surface on our own. These patterns—movements in lithium markets, faltering confidence in the president on the part of Wisconsin’s suburban moms, the role of the tight end in run blocking—are buried in data and revealed by algorithms. The story of PFF is the story of the rise of the Quants in the least likely sector of American society, the National Football League.

PFF was the brainchild of a British megafan named Neil Hornsby, and Coller’s book is based on extensive interviews with him and his colleagues. Hornsby started what became a megamillion-dollar company on his computer in Britain, obsessively tracking game tape on videocassettes as early as the 1990s. He slowly formed an ad hoc team that worked feverishly, with little pay, to track, simply, who did what during every play in the NFL, on every week of the season. It seems bizarre now that such data could be lacking, but PFF was the first to gather it—painstakingly and without a clear goal at the beginning. By about 2010, they had by far the largest such database in existence. (The NFL itself did not start to release PFF’s version of this information until 2017, in the form of “Next Gen Stats,” a big part of what I was missing in East Rutherford.) Coaches and teams guarded their own analytic techniques fiercely, so there was huge skepticism about an outside company run by an “English Dilettante”—a phrase Hornsby put in his Twitter profile after an NFL insider used it to describe him.

Today the company is ubiquitous in the NFL, with “tentacles in every facet of the game,” Coller writes, “from shaping the way coaches game plan to building models that help teams better understand the draft to whittling down a player’s exact worth to dollars and cents to influencing the way football fans across the globe understand the sport.” Every team uses, and pays for, its own data and statistical tools.

PFF has also expanded to cover college football, fantasy football, and traditional betting, along with other sports. “Football has gotten smarter over the last decade,” sportswriter Kevin Clark writes in the introduction to the book. That’s the usual take on the introduction of stats into sports—and, really, the rise of statistical prediction more generally. Everything is getting “smarter.” But the story Coller tells is more complicated.

¤

Super Bowl LII, played in early February 2018, was a major cultural event, as all Super Bowls are, drawing over 100 million viewers (with more for the Pepsi halftime show with Justin Timberlake); a 30-second ad spot cost a cool five million dollars. (There are always people at a Super Bowl party who watch the commercials but not the game.) But the real cultural tension was specific to this contest. The Philadelphia Eagles ultimately defeated the New England Patriots, whose world-famous quarterback, Tom Brady, had recently made headlines for his chumminess with then-president Donald Trump—who was also openly supported by the owner of the team, Robert Kraft. The Eagles, by contrast, had made a playoff run with their backup quarterback, Nick Foles. In the first half, Foles caught a touchdown pass on a play that seemed to mock Brady, who had dropped his own attempt at a trick quarterback-as-receiver play earlier in the game. With just over two minutes to play, Foles led the Eagles down the field and threw what turned out to be a game-sealing touchdown to tight end Zach Ertz. Few watching could have known it, but this was the moment that PFF sealed its new status as a major influence in the modern NFL.

It turned out that Eagles head coach Doug Pederson (now of the Jacksonville Jaguars) had asked his staff to pore over PFF data the team had purchased, looking for unseen weaknesses in the New England defense. The data helped them formulate a plan: bunch three wide receivers on one side of the field and have the tight end run a crossing route. Foles completed the pass, Ertz scored, the Eagles hoisted the Lombardi trophy. It was the first Super Bowl won by analytics.

Coller’s book includes many anecdotes like this one, although its backbone is reportage on the rise of the company, the personalities that led to its success, and its influence on the different groups that make up the complex multibillion-dollar industry that is the NFL. And still, it’s clear that the Eagles’ win was an inflection point. PFF started selling data to teams in 2011, and Pederson’s Super Bowl–sealing play was drawn up using a tool called “PFF Ultimate,” which allows coaches to produce video supercuts of almost anything suggested by the data. In the old days, assistant coaches had to actually splice film together to show opposing teams’ tendencies, a cumbersome process with almost no flexibility. Ultimate allowed Pederson’s staff not only to find the plays where the Pats defense had failed but also to create, nearly instantly, a package of footage that included all those plays for coaches, and then players, to study.

This part of Coller’s story is all about efficiency combined with insight, language often used to tout the application of data to “productivity” in general. Experts estimate, he reports, that more than two-thirds of the league employs the PFF stats in-game, with data analysts consulting coaches in real time. When you see a quarterback come off the field and immediately turn to a tablet, he’s probably reading numbers attached to quick-cut film produced during that very game. I’m not sure if that means football has gotten “smarter,” but it seems pretty clear that you have to have a specific kind of intelligence, or at least fluency, to coach or play in that atmosphere.

Today there is a mind-bending clot of acronyms describing heavy stats applications: EPA, DVOA, CPOE, aDOT, YPRR. Even the plain-English terms, like “win rate,” hide complex statistical functions. But as with any data-driven innovation, it’s pretty hard to quantify what has changed, because the quantitative techniques that would allow comparison didn’t exist just a few decades ago. That leaves Coller’s story in need of quite a lot of context, which the book largely elides in favor of solid reporting on PFF and its history. But anyone paying attention to the NFL today will find hints that stats are not just shaping the game but also making it more entertaining by flooding an already heated fandom with data science. The game isn’t exactly “smarter”; it’s just more filled with data, for managers, coaches, players, and fans.

¤

If you go back and listen to the Super Bowl LVII broadcast, you’ll hear Cris Collinsworth discussing the PFF-inspired touchdown. A decorated former wide receiver for the Cincinnati Bengals, Collinsworth is one of the most recognizable voices in the football world. He has a sort of folksy way of supplying commentary, mixing enthusiasm and knowledgeable player perspective with awe for the physical feats performed on the field. He’s also the owner of PFF, having purchased the company from Hornsby in 2014.

Coller reports that when Collinsworth told Fred Gaudelli, who has been in charge of the Sunday Night Football broadcast for decades, about this purchase, Gaudelli did a double take. Collinsworth’s move was even more significant because it doesn’t track with his public personality, but his gravity in the culture of the sport couldn’t be denied. Stats was already clearly coming, but Gaudelli saw this as the moment to act.

By the 2016 season, PFF’s “player grades”—a controversial single-number rating for each player on a scale of one to 100—were being used on NBC’s broadcasts. That seems almost quaint now, as we not only see but also hear breakdowns of highly abstruse and unexpected stats during every football game, in college as well as the pros. The single score is both reductive in a way that strains against the “advanced analytics insights” these techniques promise and also a ubiquitous feature of data culture. If you can’t compress a complicated process, action, or person into a single number, you can’t do data science at all.

It’s worth noting that Hornsby’s original grading system was plus or minus, and Collinsworth’s kids convinced him to use the 100-point scale to mimic player ratings in the ultra-popular Madden video games. Such numbers are so baked into the game today that the spectacle my friends and I experienced at MetLife, replete with a field-size American flag, military-jet flyovers, and fireworks, couldn’t replace them. Amazon purchased the rights to Thursday Night Football in 2021, integrating so-called “Next-Gen” stats into the actual game interface on some devices. The data-driven presentation of football on television is in the bones of the sport now.

Chuck Klosterman famously claimed that football is a conservative-presenting sport with a liberal backbone—it changes and adapts far faster than any other major professional sport. Adopting data science has exaggerated this contradiction: coaches are nerdier, quarterbacks are more cerebral, and fans have a new vector for endless fights on the radio and Twitter. But the promised efficiency, insight, and smartness has become what it is in the rest of our society: an arms race that produces more surplus entertainment value—and profit—than any real knowledge.

¤

Football Is a Numbers Game is trying to be, for football, what Michael Lewis’s Moneyball: The Art of Winning an Unfair Game (2003) was for baseball: an epic story about a radical change in a traditional American pastime. But it does not succeed in its ambitions, partly because Coller keeps his focus narrowly on PFF, with too little explanation of what it’s like to play, coach, or cheer for a data-driven NFL.

But the difficulty of crafting a Moneyball for football is not entirely Coller’s fault. As he points out, there’s no single team or season that serves as a tidy encapsulation of PFF’s rise. The 2002 Oakland Athletics campaign that Lewis described was indeed remarkable, a David-versus-Goliath story driven by unconventional, data-based ingenuity. Lewis’s book spawned a 2011 feature film, written by Aaron Sorkin and Steven Zaillian and starring Brad Pitt, but not before its very title had become a kind of shorthand for shrewd approaches to markets across different sports as well as different industries entirely.

Questions about the practice of moneyball (and the book Moneyball) have lingered. Sportswriter Allen Barra has raised doubts about whether it was really stats that fueled the Athletics’ turnaround. These sorts of questions dog all stats-driven logic: it’s not really possible to prove that the analytics were responsible for the change. And once every team is incorporating analytics into its approach, the question itself becomes meaningless. The Eagles definitely used PFF stats for the play that won Super Bowl LVII. But today, when all 32 teams are doing this, we can definitely say that “football is a numbers game” without having any idea what exactly that means. Coller doesn’t help with this conundrum, because his reporting is about the rise of a company, not about the broader context in which the Quants went from gathering data from videotapes in their own homes with an uncertain purpose to being on the sidelines with the coaching staff.

Numbers have become a fact about the game, rather than a way to see the game more clearly. Ravens quarterback Lamar Jackson was asked at a recent press conference about his success record—an absurd 17–1—against the NFC (the National Football Conference; the league is divided, for historical reasons, into two conferences, the other being the American Football Conference). Jackson responded, “Bro, we’re playing football. It’s not about NFC or AFC—I’m trying to win regardless.” Jackson is right: there’s no signal in that noise. But for a fan—much less for a coach whose seat may be getting hot or a bettor risking his savings on a game—it is impossible to ignore anyone who claims they can pinpoint what your team is doing wrong. Coller insists that the resistance to numbers in the NFL isn’t totally over but also that its influence is never going to be totally clear. I think that’s right, and that it’s part of our data culture more generally. Sure, you can point to some things that are relatively certain—coaches are risking fourth-down conversions more frequently now, running backs are not getting drafted as high, and so on—but the real impact is more diffuse.

Coller writes that it wasn’t just that PFF was “Johnny on the Spot when the NFL was finally ready to make some of the same advancements that had previously only been seen in baseball and basketball.” It was also that fans were “intrigued by the idea of hidden truths in sports.” By 2019, the NFL was hosting something it called the “Big Data Bowl,” a kind of incubator competition for data nerds to explain studies they had done and get some visibility in the league.

You read that right: studies. Coller provides examples of studies done by PFF and other data scientists, although he fails to give the reader a full sense of what kind of data science is being done, what techniques it uses, and how that relates to the larger issue of sports and numbers. The result is dissatisfying for those fluent in the data-science world and for in-the-weeds football fans alike. He highlights studies that conclude things like “offensive control at the running back’s expected point of intersection with the line of scrimmage was the most important predictor of run yardage,” which sounds like a conclusion you could only think was significant if you had never actually watched a football game. Coller says you “could fill an entire book looking at all the angles and mathematics on display from only the finalists in only one year,” which made me wish he had.

But Coller is reporting before the true numbers deluge, he thinks. “[I]t feels like we are only at the beginning of football’s acceptance of analytics,” he writes, pointing out that most teams have just a few data analysts on staff while the New York Yankees have more than 20. The NFL itself has gone all-in on these numbers, both for entertainment and for game-strategy reasons. That opens up the question of whether moneyball is indeed about to happen in the NFL: will there be a repeat of the 2002 Oakland Athletics in the coming years? Will football experience some seismic shift because of data science?

I think the answer is no, but that doesn’t mean that football isn’t a numbers game now. Every February, the NFL holds its annual scouting combine, where college players work out and run through specific drills for scouts and front-office members from every team in the league. Coller reports that it’s no longer possible for scouts to convince general managers they should draft a player based on that scout’s personal expertise. You have to make a case using the data.

Sportswriter Mike Florio has argued that the draft is “anti-American” because it doesn’t allow the players to negotiate their eventual place of employment. But the combine reeks of something worse than that: a well-funded spectacle where most of the focus goes to measuring Black men’s hands, wingspan, and speed. Disappointingly, Coller’s book completely misses the ways in which data science can evoke the practice of eugenics, a connection that takes on a particularly American twist when the slim chance of becoming a multimillionaire is dangled in front of ambitious young people who line up to play a sport that—as has been extensively documented—can have catastrophic effects on your long-term health. It wasn’t until 2021 that the NFL pledged to stop using different cognitive scales for Black and white players in evaluating long-term brain damage, and payouts for that damage, a practice that placed a lower value on Black players’ brains and lives.

But even at the level of pure efficiency, the story of the stats is mixed at best. One of the statisticians Coller interviews says that front offices will soon achieve an 80 to 90 percent success rate in predicting the future NFL performance of college players. I’m not a stats guy, but that strikes me as absurd. San Francisco quarterback Brock Purdy, who has led his team to the NFC Championship in each of his first two seasons—and who will be under center in next week’s Super Bowl—was drafted last overall two years ago, earning him the title “Mr. Irrelevant.” Tom Brady, generally considered the Best to Ever Do It, was famously drafted in the sixth round. Beyond the fallibility of old-school scouting, and the limits of data or physical measurements as barometers for football-specific success, there are innumerable unpredictable variables in play once a player makes the pros: his health, the coaching he receives, the role and system in which he is asked to play, and the sorts of psychological and intangible qualities that are impossible to test.

Compressing all that into a single number might just not work. But maybe stats isn’t about what works; when we say a process is “data-driven,” it’s the driving that we should be paying attention to. The NFL illuminates something about data culture more generally: statistical “insight” has become a material condition in our society, unmoored from the truth that quantitative techniques promise.

¤

Coller spends considerable time exploring every group affected by this shift in football—except, bizarrely, the players. All the changes he documents, all the analytical insights, every tweak done by a coach—it all has to be implemented by “execution,” as the league’s jargon has it.

It’s the quarterback who leads the way in that implementation. The rise of data in the NFL has changed the intellectual complexion of the quarterback position more than any other. Gone are the freewheeling days of Jim Kelly and Brett Favre, men who were certainly not dumb but who presented themselves a bit like Robert Duvall’s Lieutenant Colonel Kilgore in Apocalypse Now.

Josh Dobbs, recently traded to the Minnesota Vikings to replace the injured franchise quarterback Kirk Cousins, majored in aerospace engineering at the University of Tennessee. To listen to him talk about football is to get a master class that is clearly informed by analytical trends. That’s hardly the type of thing that the UNC scandal over fake classes for athletes prepared us for. C. J. Stroud, the rookie phenom for the Houston Texans, is making film for his receivers ahead of matchups, using tools like PFF Ultimate.

The quarterback is football today: he represents his team, and the sport, far more than any other player. When you listen to quarterbacks talk, you hear the analytics conversations as a basso continuo. The results are palpable. Cousins, whom Dobbs replaced, says that it’s not enough to just learn the playbook: “It has to be locked in. You have to get to a place where it’s just instinctual.” To produce intellectually at the level required, he uses a “neurofeedback” tool called “Myndlift,” which deploys electrodes to measure your brain’s focus levels, fading or blacking out the app’s screen if your attention strays. This didn’t start yesterday; quarterbacking has always been an intellectual task, even overlapping with older analytics approaches. But the cognitive pressure on the position has never been higher. The same data-produced chaos that leads to conspiracies like QAnon gives us Aaron Rodgers, the vaccine denialist New York Jets quarterback most recently seen accusing talk-show host Jimmy Kimmel of being on Jeffrey Epstein’s flight logs. If all you ever do is analyze data and win football games, it makes sense that you would want to “do your own research.”

For its repeated insistence that football has changed, Coller’s book never makes clear exactly what that change has entailed beyond the deluge of information that has been unleashed. The book left me with the impression that that change is less about the “hidden truth” the data analysts have uncovered and more about what it takes for an outsider to engage with football today. Quarterbacks and coaches have to ingest a ton of numbers and make serious decisions, many in fractions of a second with millions of dollars riding on them, while for fans, the tsunami of data increases the drama, as well as the ammunition needed to spout off on Twitter or talk radio. In light of all this, it’s clear that football runs on numbers. So do most things in our data-driven society. But whether it has made any of us smarter remains highly debatable.