NFL Gameplan Ratings:
Part I -- Playcalling Predictability

The gameplans put together by coaching staffs go a long way in telling us which team will win a given matchup (you only have to look at the Oakland-Tampa Bay Super Bowl for an example of what happens between even very good teams when one side has a definite upper hand on the playcalling), and yet you would be hard pressed to find any statistical analysis of this subject outside of the deep dark interiors of NFL team headquarters.

We aim to change that however, and this is the start of a series looking at coaches, co-ordinators, and the correlations between winning the playcalling war and winning games.

A good place to start is by examining how predictable coaches are in their playcalling between rushing and passing plays. Now we've tracked the NFL league averages over the 1998 to 2002 seasons (five years) to establish the baselines for compiling 2003 playcalling ratings.

Basically we broke down plays by down, to-go yardage, quarter of the game, score of the game, and field position and calculated how often teams ran versus passed under these different circumstances. These numbers can range from say 53% rushes for a team with 1st and 10 at its own twenty during the first quarter of a tied game, to 90%+ passes for a team facing 3rd and long in the fourth quarter down by a bunch.

Simply comparing the playcall for each team on every play during the 2003 season to the expected "predictability rate" of the league as a whole in that situation during the previous five years gives you one look at the data:

NFL Rush/Pass Predictability 2003 Offense Defense Net Team Rush Pass All Rush Pass All O-D PHI .535 .596 .570 .499 .678 .602 -.032 NE .556 .617 .590 .501 .689 .619 -.029 DEN .550 .629 .589 .490 .693 .612 -.023 GB .560 .611 .586 .488 .684 .607 -.021 SEA .540 .609 .579 .496 .668 .597 -.018 CIN .526 .638 .588 .532 .662 .603 -.015 MIN .534 .638 .590 .506 .665 .603 -.013 DAL .535 .641 .591 .516 .667 .603 -.012 SF .518 .638 .581 .508 .655 .593 -.012 JAX .494 .661 .584 .534 .644 .595 -.011 BAL .511 .684 .591 .506 .670 .600 -.009 NYJ .482 .670 .589 .558 .639 .598 -.009 NO .494 .643 .580 .523 .643 .587 -.007 TEN .555 .630 .595 .472 .671 .600 -.005 MIA .535 .643 .589 .494 .664 .593 -.004 CAR .513 .650 .580 .488 .654 .583 -.003 KAN .581 .603 .594 .484 .682 .597 -.003 IND .552 .614 .588 .502 .657 .584 .004 OAK .484 .682 .598 .544 .646 .594 .004 HOU .471 .674 .582 .544 .609 .577 .005 WAS .505 .658 .595 .542 .627 .586 .009 ATL .517 .667 .600 .542 .626 .587 .013 CLE .515 .664 .601 .554 .616 .588 .013 STL .592 .612 .604 .479 .673 .591 .013 PIT .524 .675 .610 .534 .646 .595 .015 CHI .517 .671 .604 .538 .628 .588 .016 SD .481 .704 .609 .553 .622 .590 .019 DET .478 .689 .610 .563 .611 .590 .020 NYG .507 .672 .612 .564 .614 .592 .020 TB .562 .645 .612 .523 .650 .592 .020 BUF .527 .678 .613 .528 .636 .588 .025 ARI .482 .703 .614 .587 .588 .588 .026

Analysis: The higher the number in the various categories to the left, the more predictable it was that the team would elect to run or pass (or have the opposing team run or pass against their defense). For example, the Eagles on offense had an average predictability of 57.0% for the run/pass decision for the specific situation based on league averages, while 61.4% of the time the Cardinals run/pass play call was in keeping with the NFL five-year trends. Arizona was thus more predictable. At a glance it might seem like these are tremendously good stats, since most of the best teams are showing negative nets (meaning that their offense is less predictable than what their opponent does against their defense), and the worst teams are showing positive nets. However, the read is partly false in that good teams put their opponent in many situations where the foe 'must pass' to try and come from behind.

Astute observers will note that passing play calls are significantly more predictable on the whole than rushing plays. This is because there are many more obvious "passing down" situations than obvious rushing calls. 3rd and Long will almost invariably lead to a passing play, while 3rd and 1 even is still iffy whether the team elects to go 'by ground or by air' for that last crucial yard.

A second look that might be somewhat more revealing of true play calling tendencies, by reducing the number of extremely predictable plays for a team (i.e. take out those rushes to run out the clock in the final seconds, or the passes against a prevent defense), is to limit the playcalling view to those plays in the season where neither team was ahead by more than eight points (eg. one score could tie it), and to cut off the last two minutes of the fourth quarter (but include any overtime action) since the final minutes see a lot of clock management plays.

NFL Rush/Pass Predictability: close score, with no plays inside the two minute warning 2003 Offense Defense Net Team Rush Pass All Rush Pass All O-D CIN .531 .620 .579 .528 .655 .597 -.018 GB .537 .604 .570 .519 .641 .588 -.018 MIN .534 .607 .572 .531 .627 .588 -.016 JAX .499 .637 .567 .511 .636 .582 -.015 SEA .523 .595 .566 .525 .632 .581 -.015 DAL .519 .618 .571 .498 .650 .585 -.014 PHI .518 .593 .560 .501 .639 .573 -.013 BAL .501 .674 .577 .525 .643 .589 -.012 NE .542 .609 .580 .502 .654 .591 -.011 OAK .507 .645 .579 .540 .640 .589 -.010 DEN .546 .614 .580 .503 .656 .588 -.008 NO .506 .633 .575 .521 .633 .581 -.006 NYJ .488 .646 .574 .550 .613 .580 -.006 SF .510 .624 .570 .522 .623 .575 -.005 HOU .490 .648 .570 .528 .615 .571 -.001 CAR .500 .653 .571 .505 .627 .570 .001 TEN .539 .608 .577 .494 .626 .575 .002 WAS .517 .620 .576 .533 .611 .573 .003 ATL .529 .627 .581 .528 .617 .576 .005 IND .535 .603 .575 .509 .629 .568 .007 CHI .521 .642 .586 .537 .613 .578 .008 NYG .527 .623 .583 .528 .614 .575 .008 DET .525 .641 .593 .547 .610 .584 .009 BUF .517 .647 .590 .540 .612 .579 .011 CLE .525 .646 .593 .549 .609 .582 .011 MIA .536 .633 .586 .508 .627 .574 .012 SD .520 .656 .589 .539 .613 .577 .012 PIT .516 .660 .593 .534 .615 .578 .015 KAN .562 .613 .590 .511 .626 .572 .018 STL .577 .599 .591 .491 .643 .573 .018 ARI .521 .658 .591 .537 .595 .570 .021 TB .557 .621 .596 .521 .624 .574 .022

Analysis: This more focused look shifts things around a little, with the .500 Bengals winding up as the team with the biggest discrepancy between their own predictability on defense to the plays called against their defense. Some good teams show in the upper half of the list, some show up near the bottom! For example, the 13-3 Chiefs are #29, while the 12-4 Rams are #30. This kind of haphazard hierarchy might leave you thinking these play call numbers won't be able to tell us a lot, at least at this fairly comprehensive level of counting most plays. To answer that charge, we'll need to review the correlations of the various factors to a team's wins on the season...

Before we jump to the "heavy math" let's run through the top five and bottom five teams in the different areas:

MOST PREDICTABLE

Offense			Defense
Rush	Pass	Full	Rush	Pass	Full
Stl Kan TB Den NE	Bal Pit Ari SD Car	TB Pit Cle Det Ari	Nyj Cle Det Buf Oak	Den Cin NE Dal Bal	Cin NE Bal Oak Den

LEAST PREDICTABLE

Offense			Defense
Rush	Pass	Full	Rush	Pass	Full
Nyj Hou Jac Car Bal	Phi Sea Stl Ind GB	Phi Sea Jac GB SF	Stl Ten Dal Phi NE	Ari Cle Det Was Buf	Ind Ari Car Hou Kan

Well, enough of the fun and games, time for the serious data crunching...

Correlation of Play Calling Predictability factors

Factor	Correlation	95% CI
Offense-Rush	.48	.16 to .71
Offense-Pass	-.60	-.78 to -.31
Offense-Full	-.27	-.57 to +.18
Defense-Rush	-.79	-.89 to -.61
Defense-Pass	.63	.36 to .80
Defense-Full	.03	-.32 to +.38

So in order of significance it's defensive rushing (the less predictable the better) that comes out as the best single factor for last season, although with wide open 95% confidence intervals that go juggle the ordering around quite quickly. Perhaps when opposing teams are electing to pass in seemingly good rushing situations it's because you have proven you can stuff the run (alternatively it's still a holdover of the team behind passing mode).

Putting our number machine through its paces, let's finish up today's effort with a look at a simple regression on the four main factors:

Regression of Play Calling Predictability

Factor	Coefficient	95% CI
Intercept	37.2	-40 to +115
Offense-Rush	18.4	-25 to +62
Offense-Pass	-17.0	-63 to +29
Defense-Rush	-100.8	-161 to -40
Defense-Pass	39.1	-20 to +98

Again, the defensive rushing predictability is the dominant force, coming in with a coefficient 5x stronger than either offensive measure, and nearly 3x stronger than defensive passing. No shock to this -- the top five most predictable teams in terms of defensive rush play calling were a combined 26-54 on the season, the five least predictable defensive rush squads were 60-20.

The conclusion? If you can make teams pass in situations where the NFL as a whole favors rushing then you have a healthy advantage.

We would be the first to admit that even by cutting the data set to close scores and excluding the post-two minute warning plays we still seem to have a bias to teams playing with a lead. Consequently part two of this series will look at the splits in much more depth, and follow not just the decisions to run or pass, but also the outcomes when doing so!