What Cheer, Andrew Gibson?

When I first started out writing a blog, I was largely unfocused and unsure why exactly I was writing a blog.  It was shortly after I found out that there was a blog about the Orioles called Cookin' With Gas.  It mingled my more in depth interest (and perhaps too strong devotion at the time) to utilizing statistics alternative to those frequently used by more traditional scribes.  That is, why rely on problems with fielding percentage when you can use new things like total zone.  As far as I could tell, it was an oasis in the statistics barren field of Oriole writing.  What was disappointing to me was that the site had been recently defunct. So, I took to periodically writing in 2007 to fill the void. The hope was that my presence would somehow trigger others to write and then I could read them instead of writing my own work. 

It is now 2010 and I think there are several bloggers out there who are interested in the numbers.  Not merely what the numbers say, but how they are crafted, how they are calculated, and how effective they are.  People that seem to share that joy of mine in sitting down and re-engineering statistics.  Yet . . . I still blog.

Today, I would like to focus on a writer who has lately emerged and has written several interesting items.  Andrew Gibson has been posting over at Camden Chat.  He is someone to which I think you all should pay some attention.  In today's BORT, I am going to go over his piece on the Orioles Defensive Efficiency right after the jump cut.


In baseball, defensive efficiency can be measured in many ways, but there is only one statistic called Defensive Efficiency.  That is a good lesson to learn if one is green to the numbers game.  A catchy title and an article do not necessarily mean that a statistic is a good one in that it says what it means to say and does it better than other available statistics.  A good example of this would be the Save.  The Save appears to be a metric that basically connotes that the last inning or three are the most important of the game.  That is a weak situational argument.  We often see in a one or two run game that the most important moment to save that game is when the starter gets in trouble in the 6th with guys on second and third.  The manager will often bring in his third or fourth reliever to diffuse the situation.  The rest of that game might be a breeze with no one getting on base, but the run expectancy of 2nd and 3rd with however many outs may surely be the most consequential moment for the bullpen in that game.  The best that reliever can do though is the lowly Hold.  Both Saves and Holds are faulty statistics. 

Andrew uses Defensive Efficiency though, so what is that?  It is an out efficiency metric that tells you how often a batted ball that does not leave the playing field (no homers) turns into an out.  If we would multiply the number by a 100 and add a percent sign, you could call it out percentage.  The equation is such:

1 - ((H + ROE - HR) / (PA - BB - SO - HBP - HR))

However, Andrew goes beyond this step.  He wants to see how well the Orioles Defensive Efficiency has been over the past few years between the infield and outfield.  So, he decides to use the Baseball Reference splits data to further divide this into ground ball and fly ball scenarios.  Ground balls are essentially infield opportunities for outs and fly balls are largely outfield opportunities for outs.  As an aside, you can approximate BABIP (Batting Average Balls in Play) by multiply a batter's hit types with almost universal coefficients.  Line drives are basically defenseless.  They need to be hit incredibly close to a fielder to turn into an out, so the batting average for line drives is about .700.  Outfield flies are more easily caught, resulting in a .160 batting average.  Infield flies are much more easily caught with a .018 hit rate.   Ground balls come in at .230.

Getting back to Andrew's study (go and read it now if you have not yet), he shows that last year's O's had one of the worst years for ground ball defensive efficiency and the outfield its best over their last five years.  He then comes up with these thoughts.

• Sample size

Sample size is often a great consideration for a defensive metric.  That is one problem I have with WAR.  It uses batting and fielding runs over the same time span when their need for sample size differs.  When batting, having four plate appearances over 130 games will result in a rather decent number to use.  Fielding is more difficult to measure and has more dynamic categories than batting does.  As such, you really need 250-400 games (depending on the position) to have as consistent of a result.  It is a great point.

I do have an issue here though.  I think that just because a defensive metric needs a couple years to define a player's defense accurately enough, it does not therefore mean that the metric needs the same amount of time to assess the entire defense.  I think it is plausible (I have not tested it) that just as three seasons will cancel out the noise for an individual that a whole team can cancel out the noise of an individual.  It really is a nature of sample size and evening out of unique occurrences/players within that sample.

That said . . . I still feel pretty shaky about how to assess whether or not defensive metrics are accurately portraying what they are attempting to portray.  It certainly is less straight forward than batting metrics.  The fact that many defensive systems seem to relate well to one another does not necessarily mean they are correct in what they are measuring.  This is an exciting time.

• Defense Efficiency is affected by the pitching tendencies of your staff.

This is a great point and should be recognized when thinking about defense.  Ground ball and fly ball pitchers are dependent upon different sets of fielders on a team.

I like Defensive Efficiency and think it is a great stat to report and to look at.  I do think that this statistic should be used on its way to a second objective.  How many runs are being lost or saved?  Using Baseball Reference's splits, you do not know how many ground balls up the middle and how many on the sides.  Likewise, you do not know where the fly balls are being hit in the outfield.  This can be relatively troubling to come to terms with because a ground ball at first or third is more likely to result in a double.  A fly ball on the edge of each outfielder's range is more likely to be a double.  These positions have effects on run expectancy of an event.  This is a decent set of numbers to use.

Po. Runs/play
1B 0.798
2B 0.754
3B 0.800
SS 0.753
LF 0.831
CF 0.842
RF 0.843

 Baseball Reference's data is a bit restrictive, so I often try to use weighted averages between positions to determine the cost of missed plays.  You can turn that into runs gained or lost depending on your point of view and framework.  I go through that and incorporate Defensive Runs Saved (DRS; John Dewan's system), Ultimate Zone Rating (UZR), and Total Zone (TZ) over at Camden Depot in a companion piece to this one on BORT.

I find Andrew's conclusion to be right on the money . . . the change in defense is just not going to matter much.  Based on my own calculations, the Orioles defense was worth about -23 to -13 runs last year.  That means at worst, the defense cost the Orioles about 2.6 wins in comparison to an average defensive team.  I have the expected 2011 roster to improve defensively by about one game.  My estimate would be a little kind if the team could push a few more ground balls out there.  No one in the Orioles outfield now has decent range.  Pie was the only one. 

Now, I want to reiterate that Andrew's piece over at Camden Chat was a great post on defensive metrics.  We need more of this in this Orioles blog community.  This post provided explanation and some constructive criticism to shoulder Andrew's own piece as well as provide inspiration to write something of my own over at Camden Depot.  We need more of these voices for Daniel, Heath, myself, and others to communicate with.  No statistics are perfect and we can have a much better conversation about them if more of us get up to speed and learn what they mean.  We no longer live in our parent's basement.  Ideas about the futility of using RBIs or batting average as dependable statistics is fading.  I hope BORT provides this forum for this type of communication.  I greatly enjoy it and am wishing we get some interesting comments.

So, long live Andrew Gibson and long live the revolution!