Lehrer on measurement
Jonah Lehrer has expanded his recent focus on measurement and grit (on which I recently posted) in an article on the usefulness of the Wonderlic test, a quasi-IQ test, in predicting quarterback performance. Lehrer cites a paper by David Berri and Rob Simmons which suggests that some metrics, including the Wonderlic test, are influencing draft position even though they are not predictive of performance. Lehrer writes:
While they found that Wonderlic scores play a large role in determining when QBs are selected in the draft – the only equally important variables are height and the 40-yard dash – the metric proved all but useless in predicting performance. The only correlation the researchers could find suggested that higher Wonderlic scores actually led to slightly worse QB performance, at least during rookie years.
Unlike Lehrer’s piece on which I previously posted, I’m sympathetic to the argument that this suggests that some people are acting on some not particularly useful measurements. However, I’m not as convinced when Lehrer (again) moves into the idea that the missing element is grit. Lehrer closes the article with the following:
So where is all this heading? How will grit become a bigger part of the scouting equation? The first step is to finally acknowledge that maximal tests aren’t effective. “I really see the Wonderlic as a reading test,” says former NFL executive Michael Lombardi, now with the NFL Network. “Until we get a better test, teams are just going to have to evaluate players the old-fashioned way, by watching them play in actual games. It takes good instincts to be a QB. Maybe it takes good instincts to find one, too.”
Hasselbeck suggests that teams pay more attention to the fundamentals of college quarterbacks, since their passing mechanics are often a window into how much grit they possess. “You know these guys have been coached for years,” he says. “So if you see a QB with flawed fundamentals, you gotta wonder what’s wrong. Is he coachable? Will he work to improve? Because that’s important. You can teach a kid to throw the ball, but only if he wants to learn.”
After all, deliberate practice makes perfect.
This first paragraph is almost the opposite of Michael Lewis’s Moneyball, where he suggested that there was too much faith in instinct and not enough in measured performance. Having years of college performance at hand, I’d be sceptical if some measures of “grit”, assuming it was the important missing variable, do not already exist.
On that note, the second paragraph suggests an opportunity. Maybe we should see some predictions in the lead-up to the next few drafts, where some of these grit loving experts could assess “passing mechanics” as a measure of grit, state who the teams should draft where and see if their performance measure is a better indicator of future success than actual draft position. That is what was impressive about Moneyball. Rather than being a story about someone complaining that teams should do a difficult task better, it was a story about someone taking their belief and acting on it (and with the presence of Michael Lewis, putting one season’s draft selections on the record for an assessment of those beliefs).