One frequent complaint of baseball fans attempting to quantify the performance of relievers is the lack of a metric that provides an adequate representation of how a reliever impacts the game. ERA is faulty for players who frequently enter with runners on base, and other stats like inherited runners scored and WHIP tend to give only a partial picture. The save is the least helpful indicator, as it does not differentiate between a save with a 3-run lead and a much more difficult 4 or 5 out save in a one run contest. Attempting to remedy this situation, Tom Tango and Fangraphs have created a new metric for evaluating relievers, based on WPA. For those who are not familiar with WPA, here is a quick primer
WE (win expectancy): The percent chance a particular team will win based on the score, inning, outs, runners on base, and the run environment.
WPA (win probability added): WPA is the difference in win expectancy (WE) between the start of the play and the end of the play. That difference is then credited/debited to the batter and the pitcher. Over the course of the season, each players’ WPA for individual plays is added up to get his season total WPA.
Using WPA, they have devised a way to credit relievers for particularly good performances, called shutdowns, and give demerits for poor performances, called meltdowns:
A Shutdown is when a reliever accumulates greater than or equal to 0.06 WPA in any individual game.
A Meltdown is when a reliever’s WPA is less than or equal to -0.06 in any individual game.
This is simply a more precise way to evaluate the impact the reliever had on the game, and considers the context of the situation, such that 2 runs allowed in a 12-2 game are weighed differently than the same 2 runs in a 4-3 game. In the comments, Tango notes that the ratio is 1.6 Shutdowns for every Meltdown, and that a poor reliever would have a ratio of 1 to 1. Let’s take a look at the Yankees bullpen and how it has performed in this area thus far:
The last two columns represent Shutdowns (SD) and Meltdowns (MD). It all seems fairly intuitive to me. Alfredo Aceves has been used in a number of tight spots, and his ability to eat 2-3 innings in an outing contributes to his good showing. Mariano and Joba have also been excellent, with the two of them combining for just one poor outing. Meanwhile, Boone Logan has been neither great nor poor, while Chan Ho Park and David Robertson have shown wild inconsistency. Finally, Damaso Marte has been atrocious, allowing a number of inherited runners to score. The metric seems to fit with the observed performances, and gives a quick and dirty way to measure impact on the game. I hope it catches on.