Pro Football Focus: "Broncos O-line Is NFL's Best"

[Joel]

http://insider.espn.go.com/nfl/story/_/id/9964660/nfl-fortunately-battered-peyton-manning-denver-broncos-offensive-line-nfl-best?addata=2009_insdr_mod_nfl_xxx_xxx

This, tucked inside the ESPN article Carol linked on PFM missing practice after a second MRI in two months, when BOTH ankles already injured in previous games got separate wicked shots Sunday. For those unclear on why some (e.g. me) don't take PFF seriously: Because of this kind of oversimplified cursory crap. For the first month, our line was truly phenomenal: Just 5 sacks in as many games, Moreno averaged 5.1 yards per attempt and Mannings ONE pick was Game 5 @Dallas. Then we lost Clady for the year in Week 2, and Franklin joined him on the bench halfway through Week 6s game; our line's been AWFUL since. HOW awful?

Morenos average dropped to 3.3 in those four games, matching Balls season average. We've given up 8 more sacks, including strip-sacks for TDs (in Indys case, safety and TD) EACH of our last THREE games. Manning's gone from 1 Int in 5 games to 5 Ints in 4; a couple against Washington were on WRs, but several were hits as he threw. Our current line's only "the NFLs best" if we scan totals/averages without looking closely enough to see they were an impenetrable juggernaut the first month but have been Swiss cheese and noodles since. Unfortunately, that seems SOP for PFF, undermining their general credibility.

Sorry, ya'll, but there's more to Sabermetrics than Hoovering up everything with a number attached, adding total TDs, penalties and cheerleaders, then dividing by hot dogs sold to tease out an insight.

It's great to bring scientific statistical analysis to football; Virgil Carter's weak arm but sharp mind has influenced the NFL more than anyone since Jim Thorpe by prompting offensive coordinator Bill Walsh to develop the West Coast Offense to maximize his limited skills, then co-publishing "Operations Research on Football" with his Northwestern professor: http://pubsonline.informs.org/doi/pdf/10.1287/opre.19.2.541 Nearly two decades later, The Hidden Game of Football took that a step further by compiling three whole seasons of NFL stats for computer analysis in an era when most athletes had never heard of databases.

If we're going to do that though, we must be aware of all the traditional pitfalls of scientific statistical studies.

Cherry-picking the stats that support our argument from far more numerous ones that don't (the "370 Rule" for RBs is a favorite bone of contention here.)
The similar Texas Sharpshooter Falacy. http://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy
Fitting data to curves rather than the reverse (a statistical form of Assuming the Consequent.)
Perhaps most importantly, the eagerness for the fame and impact of a revolutionary counterintuitive discovery encouraging people to defy common sense and call it defying conventional wisdom.

As a general rule, it's a huge red flag when a "paradigm-shifting" "insight" doesn't pass the BS test. I hear some people who recall my statistical arguments for a T QB renaissance chuckling now, but that was just defying WALSHS defiance of conventional wisdom: Carter was a weak armed QB with little mobility but great accuracy; since Esiason was still in elementary school, inventing the West Coast Offense was Walshs only option, but that didn't make it the ideal, and Montana would excel in ANY offense. Steve Youngs 50 yd TD scrambles and errant bombs were more typical of Arnie Herber than of PFM.

Not to pile on PFF, but anyone who says the Broncos offensive line in its CURRENT form is the NFLs best should check with PFM and Knowshon before publishing an argument they'd probably dispute.

[MOtorboat]

Funny.

[TXBRONC]

http://insider.espn.go.com/nfl/story/_/id/9964660/nfl-fortunately-battered-peyton-manning-denver-broncos-offensive-line-nfl-best?addata=2009_insdr_mod_nfl_xxx_xxx

This, tucked inside the ESPN article Carol linked on PFM missing practice after a second MRI in two months, when BOTH ankles already injured in previous games got separate wicked shots Sunday. For those unclear on why some (e.g. me) don't take PFF seriously: Because of this kind of oversimplified cursory crap. For the first month, our line was truly phenomenal: Just 5 sacks in as many games, Moreno averaged 5.1 yards per attempt and Mannings ONE pick was Game 5 @Dallas. Then we lost Clady for the year in Week 2, and Franklin joined him on the bench halfway through Week 6s game; our line's been AWFUL since. HOW awful?

Morenos average dropped to 3.3 in those four games, matching Balls season average. We've given up 8 more sacks, including strip-sacks for TDs (in Indys case, safety and TD) EACH of our last THREE games. Manning's gone from 1 Int in 5 games to 5 Ints in 4; a couple against Washington were on WRs, but several were hits as he threw. Our current line's only "the NFLs best" if we scan totals/averages without looking closely enough to see they were an impenetrable juggernaut the first month but have been Swiss cheese and noodles since. Unfortunately, that seems SOP for PFF, undermining their general credibility.

Sorry, ya'll, but there's more to Sabermetrics than Hoovering up everything with a number attached, adding total TDs, penalties and cheerleaders, then dividing by hot dogs sold to tease out an insight.

It's great to bring scientific statistical analysis to football; Virgil Carter's weak arm but sharp mind has influenced the NFL more than anyone since Jim Thorpe by prompting offensive coordinator Bill Walsh to develop the West Coast Offense to maximize his limited skills, then co-publishing "Operations Research on Football" with his Northwestern professor: http://pubsonline.informs.org/doi/pdf/10.1287/opre.19.2.541 Nearly two decades later, The Hidden Game of Football took that a step further by compiling three whole seasons of NFL stats for computer analysis in an era when most athletes had never heard of databases.

If we're going to do that though, we must be aware of all the traditional pitfalls of scientific statistical studies.

Cherry-picking the stats that support our argument from far more numerous ones that don't (the "370 Rule" for RBs is a favorite bone of contention here.)
The similar Texas Sharpshooter Falacy. http://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy
Fitting data to curves rather than the reverse (a statistical form of Assuming the Consequent.)
Perhaps most importantly, the eagerness for the fame and impact of a revolutionary counterintuitive discovery encouraging people to defy common sense and call it defying conventional wisdom.

As a general rule, it's a huge red flag when a "paradigm-shifting" "insight" doesn't pass the BS test. I hear some people who recall my statistical arguments for a T QB renaissance chuckling now, but that was just defying WALSHS defiance of conventional wisdom: Carter was a weak armed QB with little mobility but great accuracy; since Esiason was still in elementary school, inventing the West Coast Offense was Walshs only option, but that didn't make it the ideal, and Montana would excel in ANY offense. Steve Youngs 50 yd TD scrambles and errant bombs were more typical of Arnie Herber than of PFM.

Not to pile on PFF, but anyone who says the Broncos offensive line in its CURRENT form is the NFLs best should check with PFM and Knowshon before publishing an argument they'd probably dispute.

Ironic how you can complain about PFF cherry picking stats and yet you do it all the time.

[Simple Jaded]

The Football Scientist is generally an idiot too.

[Joel]

The Football Scientist is generally an idiot too.
Never heard of 'im, but not surprised with a name like that. The problem with a lot of sports stats analysts is they aren't. Guys like Bill James and Virgil Carter are worth hearing because they actually know what they're talking about, not just in sports terms (though there, too,) but in math terms. Doing that job takes a statistician, by which I don't mean someone who jots down names and tally marks after each play. Guys who've not only heard of least squares regression but can EXPLAIN it.

[Simple Jaded]

Never heard of 'im, but not surprised with a name like that. The problem with a lot of sports stats analysts is they aren't. Guys like Bill James and Virgil Carter are worth hearing because they actually know what they're talking about, not just in sports terms (though there, too,) but in math terms. Doing that job takes a statistician, by which I don't mean someone who jots down names and tally marks after each play. Guys who've not only heard of least squares regression but can EXPLAIN it.

He's on Espn, the first I remember of these stat geeks. I stopped reading when he came up with complex algorithms that told me how David Garrard was better than Ben Roethlisberger.

[Joel]

He's on Espn, the first I remember of these stat geeks. I stopped reading when he came up with complex algorithms that told me how David Garrard was better than Ben Roethlisberger.
Maybe as a runner. I dunno, Big Ben's kind of careless with the ball (and in general, by all accounts.) I'm still a little embarrassed at only finding out a few weeks ago that ESPN was founded by Bill Rasmussen and his son Scott, the latter now better known (at least to me) as founder of Rasmussen Reports. So I don't know ESPNs pick in next weeks game, but do know whoever it is is favored by ~4 pts too many. :tongue:

[Lancane]

Ironic how you can complain about PFF cherry picking stats and yet you do it all the time.

As do we all TX, it is the nature of anyone to try and come out ahead in an argument and three-quarters of the time in regarding sports it's based on some sort of statistical evidence.

And other then gathered rumor mongering and tidbits, or piggybacking on columnists PFFT is a complete joke.

[Simple Jaded]

I wonder where the Broncos OL will rank after this game?

[Joel]

I wonder where the Broncos OL will rank after this game?
Probably about the same in most season categories. They were evidently #1 in season pass blocking stats already; 0 sacks and 0 hits won't change that. They were also 20th in total and... 24th...? in average rushing; averaging 2.9/carry won't (positively) change THAT either. They were stellar in pass blocking and good enough run blocking; no complaints, though we may need a bit more from the run blocking in KC.

Hopefully we don't spend another game insisting on running straight at the best run stuffer any good 3-4 has, because neither Beadles nor Ramirez can't push back a 350 lb. clogger: Run counters at the DEs until those overpursuit and having to make tackles in traffic leaves those speedy LBs bruised and gassed. That said, I couldn't help wondering if we kept running up the gut to keep those LBs from congregating on the edges where they could overwhelm our tackles. If that was the goal, it succeeded (either that or happy coincidence,) and I'm satisfied with that.

[Simple Jaded]

Probably about the same in most season categories. They were evidently #1 in season pass blocking stats already; 0 sacks and 0 hits won't change that. They were also 20th in total and... 24th...? in average rushing; averaging 2.9/carry won't (positively) change THAT either. They were stellar in pass blocking and good enough run blocking; no complaints, though we may need a bit more from the run blocking in KC.

Hopefully we don't spend another game insisting on running straight at the best run stuffer any good 3-4 has, because neither Beadles nor Ramirez can't push back a 350 lb. clogger: Run counters at the DEs until those overpursuit and having to make tackles in traffic leaves those speedy LBs bruised and gassed. That said, I couldn't help wondering if we kept running up the gut to keep those LBs from congregating on the edges where they could overwhelm our tackles. If that was the goal, it succeeded (either that or happy coincidence,) and I'm satisfied with that.

I think this game shows how these stat geeks are flawed, they're ranked overall but the OL (or any other position) has to be evaluated on a game by game basis, individually and as a unit, taking into account level of competition on a play-by-play basis. This is why I hate judging teams by opponents win/loss, the Jacksonville team that Denver beat is a far better team than the one KC beat. Is the WR that beat Revis this week the same as the WR that beat him week 1? And what route vs what coverage?

I'm not statistician but the amount of work you'd have to put in to really rank teams position by position must be staggering. I don't read those sites at all but I'm guessing that power rankings are relatively as accurate for doing far less work.

[Joel]

I think this game shows how these stat geeks are flawed, they're ranked overall but the OL (or any other position) has to be evaluated on a game by game basis, individually and as a unit, taking into account level of competition on a play-by-play basis. This is why I hate judging teams by opponents win/loss, the Jacksonville team that Denver beat is a far better team than the one KC beat. Is the WR that beat Revis this week the same as the WR that beat him week 1? And what route vs what coverage?

I'm not statistician but the amount of work you'd have to put in to really rank teams position by position must be staggering. I don't read those sites at all but I'm guessing that power rankings are relatively as accurate for doing far less work.
It depends on the stat geek; the big pioneers did their homework, and well, in an age when both stats and the means to process them were very hard to obtain, but their success spawned a lot of pale imitations (and fantasy football, of course.) The big reason I like Football Outsiders is because they consciously and conscientiously built on The Hidden Game of Footballs groundbreaking work and openly admit it. They do a LOT of good position and team ranking, and normalizing play-by-play for relative opponent quality is Step 1. It's only tedious if you do it RIGHT (and rarely difficult) but there's the rub: Most don't.

A good example of the difference is the conflation of power ratings (i.e. point spread predictions mathematically computed from all of each teams previous games, as explained in THGoFs chapter on betting,) with power rankings (i.e. pulled out of some analyst or commentators rear.) The latter rarely do the same work as the former, for a very compelling (but no good) reason:

It's not enough to just rank, normalize and re-rank once, because that shuffles ALL values, so it must be done repeatedly until the numbers near equilibrium. Next week everyone will have one more game, requiring the whole process be repeated. Even then, even after a whole season, we STILL can't hang our hat on it, because no team plays even HALF the rest. Power RATINGS are easily calculated:

1) Average the teams victory margin,
2) Average 1) for all its past opponents
3) Subtract 2) from 1)
4) Repeat until 3) stops changing (much.)

It's kind of like Linear Regression for Dummies. :tongue: Power RATINGS can be used as power RANKINGs, it's just a lengthy boring process, with many steps where an error will make the final result less accurate. It won't be completely accurate or precise ANYWAY, because no team plays even a majority of the NFL. That also doesn't factor homefield advantage, but we can easily calculate that by subtracting all road from all home victory margins and dividing by 256 games (or 718 if we use three seasons, as THGoF did to get ~2.2 for average NFL HFA 1984-1986.)

Yet AVERAGE NFL HFA, though accurate and easy to calculate, isn't terribly precise. Some stadiums are loud, some are cold, some are both and some are just weird (e.g. the dustdevils in Candlesticks end zones should be reported to the Competition Committee, and I've heard Soldier Field is like running on concrete, because that's what's underneath the thin sod.) We can power rate THAT, too, and it's as simple as any other power rating—it's just a lot more work on top of what we've already done.

Serious statisticians do that work, even enjoy it. Football Outsiders does, and dutifully calculates Defense-adjusted Value over Average along with many similar stats every year. http://www.footballoutsiders.com/dvoa-ratings The ubiquity and increased power of computers has made it MUCH easier than when Bob Carroll, Pete Palmer and John Thorn published THGoF was back in '88. The internet's even made stats FAR easier to get: They had to write to the Elias Sports Bureau; we can just google NFL.com (though its stats are alarmingly less reliable than one would expect.)

The guys just trying to pump ratings, blog hits and/or pills promising to pump anatomy don't do their homework though, because they don't have to be accurate or precise: Just convincing, or at least interesting. If you can't dazzle them with dexterity, baffle them with BS. ;) Let's face it, a site falsely claiming statistical proof Garrard is better than Roethlisberger gets a lot more attention than one promising to demonstrate just how MUCH Elway was better than Flutie. Win Probability is instructive enough most stat geeks have picked it up since THGoF introduced it, but its uselessness for anything but post mortems limits interest.

[DenBronx]

It might be the best with a healthy Clady. Not so sure other than that.

Clark did really good last night though and Manning stayed clean. They dont have to be the "best" but as long as they just keep Manning from getting hit then that's all we will need.

[Joel]

It might be the best with a healthy Clady. Not so sure other than that.

Clark did really good last night though and Manning stayed clean. They dont have to be the "best" but as long as they just keep Manning from getting hit then that's all we will need.
Maybe with Clady; again, I think our first month (when he had him the first two games and Franklin didn't get hurt at all) are padding our season line stats. 5+ yds per carry, 5 sacks, 1 Int and... many TDs were every bit as breathtaking as the press portrayed them. I still don't think Beadles or Ramirez get enough push in the run game (our biggest enduring problem there,) but they're good pass blockers the vast majority of the time (Beadles blew a block that got Manning leveled with a shoulder square in the chest on JTs first career TD, and Ramirez allowed the shot to his knees at the end against SD, but no one's perfect.)

We must run better though, not force the D to protect fourth quarter leads. A BIG lead and bad offense allowed it yesterday, and the D stepped up in SD, but in the playoffs that'll eventually burn us (as last year.)