Quality Competition Splits

If you asked me to vote for a ranking every week, my ballot would be incredibly boring. Teams sort of fall in a certain range, (as I illustrated the day after New Years), and it takes multiple games for me to change my mind. I’ve liked Kentucky, North Carolina, and Ohio St. from the start of the year, and nothing has changed that. Thanks to Baylor’s sudden ability to play solid man-to-man defense, after years of playing incredibly poor 2-3 zone, I am now a big believer in Baylor. And thanks to Fab Melo’s development, I am now a big believer in Syracuse. But if you asked me to rank teams every week, my poll would almost never change.

But fans want the polls to change. When Illinois upsets Ohio St., Illinois fans want to see the team pick up more AP votes.  Illinois fans want to log onto kenpom.com and discover that the season expectation has changed from 8-10 in the Big Ten to 10-8 in the Big Ten. Fans want to see that the games are meaningful, even if each game is only a small part of an overall evaluation.

And that’s why fans find it so frustrating when something they love (the Pomeroy Rankings) seems detached from their perception. Wisconsin is currently 3^rd in Ken Pomeroy’s ratings, a spot that is so high that Ken felt the need to defend his ratings on Monday. Kansas’ second place spot might puzzle some people too.

Today I had planned to run some splits showing various teams’ records against teams ranked 1-100 and 100+.  And given the discussion on Wisconsin, it seems particularly relevant.

Crushing Bad Teams

Wisconsin has absolutely been pummeling bad teams this year. Even counting the home loss to Iowa, Wisconsin has absolutely been running up the score on the little guys. (I’d say it is bad sportsmanship, except that at Wisconsin’s slow pace, it doesn’t look like it.) And since Ken’s rankings are primarily derived from margin-of-victory, that causes his rankings to love Wisconsin.  Here are the teams with the best ratings against bad teams: 

Now, it is tempting to argue that we should throw out the results against teams rated 100+. But running up the score on the little guys is generally an important indicator that you have a good team. In particular, Syracuse, Kentucky, North Carolina, and Ohio St. are all on this list, and they tend to be considered among the top teams by most experts. The key is that all of these teams have also been good against quality competition. Meanwhile Wisconsin has been much worse against quality teams: 

The Badgers haven’t been horrible against Top 100 teams. Remember, they lost a close game at North Carolina, lost in OT to Michigan St., and beat BYU and UNLV pretty convincingly. But the Badgers have been nowhere near as good against elite teams as the have been against the Nebraskas of the world.

Interestingly, Wisconsin is far from the worst culprit in this rating inflation. Until Florida St.’s win against Virginia Tech this weekend, the Seminoles hadn’t done anything against a quality team: 

On the flip side, here are some teams that have elevated their play against elite competition: 

In Baylor’s case, they were without Perry Jones for the early part of the season, and that explains why they were worse against team 100+. But in other cases, I honestly think it can come down to style of play. Michigan doesn’t have the type of athletes that can just pummel a team from the Patriot League. But their offensive system is so difficult to defend, that elite teams usually cannot stop them from scoring.

As followers of college basketball, we tend to judge teams mostly on how they do against quality competition. To some extent I agree with this. Everyone in the Big 12 should beat Texas Tech, so I don’t think we should read too much into a win against them. But there is a cost to ignoring this information. Baylor beat Texas Tech by 13, while Kansas beat Texas Tech by 35. And when trying to set a line for the Baylor vs Kansas game, I think we should be concerned about Baylor’s struggles against a mediocre team. But for those of you who believe games against teams 100+ are useless, the next table shows how the Pomeroy Rankings would look if we only looked at games against teams rated 1-100.

What you see are some pretty unrealistic rankings. Just because Middle Tennessee has looked great against elite competition, pounding UCLA, crushing Akron, and splitting with Belmont (with the loss coming in 2 OTs), I don’t think anyone believes for a second Middle Tennessee is a Top 10 team. The fact that their Sun Belt games are “competitive” tells us that they are not elite. And it should convince us that ignoring games against weaker teams is not a good way to build a ranking system.

Still, it is interesting to note that Missouri has actually performed better against elite competition than any of the other contenders, and that Virginia has also been surprisingly dominant against good teams this year.

But while it is ridiculous to throw out games, let’s take it one step further and apply the Jay Bilas test. Who has gone on the road and beaten quality teams in true away games (i.e. neutral sites don’t count)? At this point almost no one has multiple true road wins against quality competition:

By the end of the season, true road wins might be a nice metric, but right now there haven’t been enough of these games to learn anything.  (I mainly bring this up because a lot of people have been criticizing Indiana for doing all their damage on their home floor, but that’s just silly.  Let’s see how the Hoosiers are doing in February before we make that pronouncement.)

Realistically, there are a lot of ways to solve the “Wisconsin problem” without throwing away information. Systems that put less weight on margin-of-victory get around this problem. If you want more weight on outcomes and less weight on margin-of-victory, look no further than the LRMC system Ken linked to, or the Sagarin ELO chess.  Another solution is to cap margin-of-victory at a certain level so that Middle Tennessee’s close games count, but Wisconsin's blowouts get less weight.

But if you want to calculate Offensive and Defensive ratings and truly understand “why” teams are winning or losing, then you can’t cap margin of victory. And that’s why Ken’s model won’t and shouldn’t change.  His numbers are as much about understanding why teams are good or bad as finding the perfect ranking.

Overall, remember that margin-of-victory is just one piece of information that should be used to evaluate teams. I happen to think it is among the most important pieces of information, but it isn’t the only piece of information. I’ve said this for years and I’ll continue to say it. The beauty of the NCAA tournament is that the selection committee is made up of people. And unlike a formula, people can adapt and consider different pieces of information that may be important to evaluating a team. The NCAA committee may be less consistent than a formula, but it is more flexible.  And since odd results crop up every season, that flexibility is incredibly valuable.