Why Is It So Hard to Predict the Next Great Quarterback?

This week we have the NFL draft, which is an annual exercise in failure. Every year, some NFL team makes a disastrous quarterback decision but also overlooks a potential star. Why is it so damn hard to predict QB play in football? Are scouts stupid, or is the future just unknowable, or is hiring fundamentally chaotic, or is there something specific about quarterbacking that makes it uniquely difficult to forecast? The economist David Berri joins to share his research on why scouts are terrible at evaluating quarterbacks. His ideas shed light on larger questions like “What is talent, exactly?” and “Does anybody know what they’re doing when they’re hiring somebody for a new role?” Part of their conversation is excerpted below.

If you have questions, observations, or ideas for future episodes, email us at PlainEnglish@Spotify.com. You can find us on TikTok at www.tiktok.com/@plainenglish_.

Derek Thompson: So let’s jump right into it. What is the relationship between a quarterback’s draft position and his subsequent performance in the NFL? Are the top quarterbacks drafted the best NFL quarterbacks? Is the answer “yes,” “no,” or “hell no”?

David Berri: If you are trying to predict who’s going to play, then the quarterbacks taken first will play more than the quarterbacks taken later. That is definitely true. If you’re trying to predict who is gonna play better on a per-play basis, not a chance in hell. They do not know that. There is no statistical relationship between per-play performance and where they’re actually drafted. We do know that if you’re taken earlier, you’re gonna get a lot of chances, and if you’re taken later, you are not gonna get a lot of chances. But of those who end up getting chances, their performance and their draft position are not actually related.

DT: So draft position predicts opportunity, but it does not predict performance. Let’s go right through this paper. For those following at home, the name of the paper is “Catching a Draft: On the Process of Selecting Quarterbacks in the National Football League Amateur Draft.” So in a nutshell, David, your thesis is that where people draft quarterbacks basically is unrelated to future NFL performance. Take me through everything that you looked at in order to come to that conclusion.

DB: We looked at a wide variety of performance measures, some that I’ve created. So I’ve gone through and looked at the data and connected it to wins and looked at how many wins a quarterback produces. Some of it is the measures that the NFL has created, such as the quarterback rating, and then the individual stats: completion percentage, touchdowns per attempt, interceptions per attempt, fumbles per rushing attempt. And there just simply is nothing that’s very predictable. You can’t predict any of this stuff, knowing where they’re drafted. There are quarterbacks who do play better than others. It’s not related to where they were drafted, though.

DT: So this is a quote from your paper: “On a per-play basis, quarterbacks chosen with picks 11 through 50, as well as picks 51 through 90, outperform quarterbacks chosen in the top 10.” Top 10 quarterbacks really don’t offer more. They just get to play more. One explanation of this is that this is a reverse-order draft. That means that the quarterbacks chosen in the top 10 tend to be going to really sucky teams. So maybe the quality of the teams that are hiring the top 10 picks is lowering their numbers. You looked into that potential explanation—Is that it? Is it because of the reverse-order draft that we’re seeing this phenomenon?

DB: Part of that is the story, right? Quarterback play entirely depends on the context of who you put around them. So when we look at a top quarterback, this is the mythology the media creates: A top quarterback makes their team so amazingly great that it doesn’t make a difference what team they play on. They’re just winners. They’re all winners. That’s the story. They’re like, “We gotta draft a winner,” and that’s ridiculous. That’s not how it works. The quarterback’s play depends on who you put around them.

This excerpt has been lightly edited for clarity.

Host: Derek Thompson
Guest: David Berri
Producer: Devon Manze

Subscribe: Spotify