What makes a bullpen successful? Whether you are a college coach trying to determine what to develop and how to compose your staff, or an MLB team trying to determine what to do for the looming trade deadline, it only makes sense to answer this question first.
Evaluating relievers is a little more difficult than starters for many reasons and deciding on a performance metric to use is not exactly a no-brainer. I decided to use 2019 overall bullpen xFIP to evaluate performance because it is more indicative of performance than ERA.
Some quick notes: I evaluated MLB bullpens as a whole, not relievers on an individual basis. In order for a pitcher to qualify for their respective bullpen, they needed to have pitched at least 15 innings in relief, and were disqualified if more than 50% of their appearances came as a starting pitcher. If a pitcher was traded mid-season and only threw 14 innings of relief for his new team, he would only be included in his former team’s bullpen numbers. I also classified four-seam fastballs, two-seam fastballs, sinkers and cutters as a "Fastball," considered splitters as "Changeups", and knuckle-curves and "Curveballs."
Here are the 2019 Bullpen xFIP rankings, courtesy of FanGraphs.
The first thing that jumps out at me is your previous World Series contestants rank 1 and 29. Guess it's not how you start, it's how you finish. But what separates the Astros bullpen from the Nationals? Here's something that stands out...
The Nationals bullpen ranks first in fastball vertical break and they share almost exactly the same release height on all pitches. How can the Nationals bullpen perform so badly with that type of vertical movement on their fastball? This has to be due to mis-usage, which is something I wrote about specifically a few months ago and why "Tanner Rainey should be better than Nick Anderson." While the relationship is not super strong (-.18 correlation), the Nats certainly stick out in the upper right of the chart below.
The next thing that stood out was how good the Padres bullpen was. Had to be because of the flamethrowers in the backend of their pen, right? Wrong...the Padres were dead last in average bullpen fastball velocity. This was really surprising to me considering they ranked second in xFIP. (It's important to mention once again that I only included relievers with at least 15 innings pitched and less than 50% of their games were started. The Padres ranked 14th on this list where all relievers were included. Even in that list, there is just a .06 correlation between fastball velocity and xFIP.)
So what does the Padres bullpen do well? Three things stand out to me. First, they rank fourth in fastball vertical break, combined with a lower release height of 5.8 feet (ranked 21st). Why is this important? Vertical approach angle. I did not want to provide vertical approach angle numbers because my model is slightly off, but a low release height paired with "rising" fastballs is a recipe for whiffs. Second and third, their breaking balls...
While the Padres relievers do not exactly light up the gun on their fastballs, they let it eat on their sliders. Is this correlated with xFIP? No, but I am a big believer that the harder a slider is thrown, the more it tunnels with a fastball. Opposing hitters can sit on a 90 mph fastball all day until it ends up breaking at the last second. Curveballs are a little more interesting in how they relate to xFIP...
Curveball velocity has a correlation of .3 with xFIP (meaning xFIP increases as CB velo increases--no bueno) and a correlation of -.21 with curveball horizontal movement and xFIP. These were the two most correlated measures I found with performance and are not as significant as I would have liked, but not terrible in this context (so many contributors to xFIP beyond what is solely in the formula).
The Twins also had some characteristics that stood out. They rank dead last in fastball vertical movement and 28th in fastball velocity, yet they rank 6th in xFIP. Whether their bullpen was structured in what is depicted below is one question, but there's no denying they are outliers here...
I typically (and I think most others do, as well) associate sliders and changeups as moving primarily horizontally. As you can see, most teams have relievers that do just that, but check out the "MIN" Twins. Their bullpen threw curveballs 16% of the time, 2nd most in baseball (they rank 9th in CB horizontal movement). Maybe they have found the secret sauce: know what you're good at and exploit it.
One last notable observation: The Blue Jays, Mets, Rockies and Marlins make up 4 of the 5 top average fastball velocities, but all of their xFIP's were ranked 22nd or worse. This was surprising and makes me think teams should look to build their bullpens around relievers with above average secondary offerings rather than focusing on fastball velocity. That leads me to...
So What?
With everything I mentioned, the formula for building a successful bullpen is not exactly clear-cut. It looks like it's a combination of a few things, but here are my main takeaways.
No matter what your bullpen is composed of, exploit the strengths of the pitchers within it. This seems way too obvious, but there is no reason the Nationals should be 29th in xFIP. The Twins on the other hand seem to recognize that their fastballs do not stand out, but they have identified their strong secondary pitches. Up the usage rate when you have something special.
If you had to pick pitch qualities to build a bullpen around, pick slower curveballs with more horizontal movement and fastballs with high carry. (While I have doubts about MLB potentially moving the mound back, this would bode well for pitchers with already slow curveballs because long-form movement is only going to increase.) Maybe this is where the Nationals lack, their bullpen threw just 2% curveballs, and none of their relievers that throw a curveball qualified for my list.
Limitations and Further Research Ideas
Since I did not use weighted averages for both innings pitched and usage rate, some of my assumptions could be off here. For example, if Aroldis Chapman's average fastball velocity last year was 100 mph and David Hale's was 93 mph, the average of the two is 96.5 mph. This is not completely fair, however, because Chapman threw 57 innings and Hale pitched 37.2.
It's also possible that "Openers" could throw off some things in here. They would not be included if they started more than 50% of games.
Expanding the dataset to previous seasons could also help, and would make classifying into more than just four pitch types more sensible.
Someone could also make the case that there is a better metric to use than xFIP. I mentioned before why I used it and xFIP was more correlated with the pitch flight data I looked into than FIP and ERA were.
I touched on this briefly, but incorporating all usage rates could turn up some interesting results. The Astros threw fastballs 46% of the time, only the Red Sox bullpen threw it less. The Nationals ranked 5th in fastball usage with above average fastball characteristics, so maybe they struggle with command or are not throwing to optimal spots.
Lastly, this idea stems from a post by Jacob Foster where he looked into entire pitching staffs. Here is a link to his post.
Comments