 |
|
|
Frequently Asked Questions about Plus/Minus and Runs Saved What is the Range & Positioning System? The Range & Positioning System, formerly known as the Plus/Minus System, is the key technique that we use to study defense in baseball. We introduced the system in the first volume of The Fielding Bible. Here’s the question that we try to answer with the Range & Positioning System: how many plays* did a fielder make above or below an average player at his position? (*With an adjustment, we can actually also estimate the number of bases saved above or below average.) That’s what you should think to yourself when you’re looking at all those plus and minus numbers. The average is zero. If a player makes one play more than the average, that’s +1. Every MLB play is entered into the computer where we record the direction, distance, speed, and type of every batted ball. Direction and distance are recorded on a computer screen by simply clicking the location of the ball on a replica of the field shown on the screen. Speed is scored as either soft, medium, or hard, while different batted ball types include bunt, groundball, liner, fly, and “fliner”. Fliners, introduced in 2006, are balls that are somewhere between flies and liners. Beginning in 2009, BIS video scouts began additionally tracking batted ball timer information. This data has allowed us to make objective assessments of the velocity and trajectory of each grounder or flyball. In the Range & Positioning system, the computer totals all groundballs hit by right handed batters to Vector 206 (Vector 206 is a line extending from home plate towards the hole between the normal shortstop and third base positions, 19 degrees off the third base foul line) with an average velocity between 65 and 75 miles per hour and determines that these types of batted balls are converted into outs by the shortstop only 23 percent of the time. Therefore, if the shortstop converts a slowly hit ball on Vector 206 into an out, that’s a heck of a play, and it scores at +.77. The credit for the play made, 1.00, minus the expectation that it should be made, which is 0.23. If the play isn’t made, it’s -.23. The key is that if a fielder makes a play on a specific type of batted ball, hit to a specific location on the field, and hit at a specific speed, he gets credit if at least one other player in MLB that season missed that exact ball sometime during the season. A fielder who misses a play on a specific type of batted ball, hit to a specific location on the field, and hit at a specific speed, loses credit if at least one other player made the same play some other time. Add up all the credits the fielder gets and loses based on each and every play when he’s on the field and you get his plus/minus number (rounded to the nearest integer). Let’s continue with the Vector 206 example. Shortstops fielded a 65-75 mph groundball there 23 percent of the time in over the past two years. Slightly softer groundballs (between 55 and 65 mph) were fielded more often—43 percent of the time to be exact. Even softer grounders (45-55 mph) were converted for an out 59 percent of the time. Overall, there are 90 vectors we use for fair territory on the field. For outfielders we also consider the distance of every batted ball. But we’re not done, are we? A shallow line drive single isn’t the same as an extra base hit in the gaps, so we don’t treat them the same. To account for this, we make the adjustment. What’s the difference between Plays Saved and Bases Saved While Plays Saved only takes into account the number of plays a player makes above or below average, Bases Saved measures the total bases saved. For example, an outfielder might have a basic plus/minus score of +15 (15 plays above average at his position), but saving 15 singles is very different than saving 15 extra-base hits. Often, we'll cite the Bases Saved number since it’s a more comprehensive description of a player’s value added or lost compared to an average fielder. Of course, in The Fielding Bible – Volume II we translated Bases Saved into Range & Positioning Runs Saved, which is the best indication of a player’s defensive value. What adjustments do you make in the Plus/Minus System? Every position has at least one special adjustment to improve accuracy. Let’s go through each position: First Base – There is a big difference between how a first baseman positions himself, depending on whether he’s holding the runner or not. To approximate this, we break down all plays involving first basemen into two categories, Holding and Not Holding. “Holding” is any situation where there’s a man on first with second base open. We may refine this in the future, but we found that since the outcomes are very different with runners being held this adjustment made an important difference in improving the accuracy of the first base plus/minus numbers. We also make the Bases Saved adjustment at first base, since balls down the line can potentially go for extra bases. In The Fielding Bible—Volume III, we began calculating a separate basis for infielders when facing left-handed and right-handed batters, beginning with the 2010 Plus/Minus calculations. Second Base and Shortstop – The key adjustment for these two positions is made on hit-and-run plays. We consider any play where the runner on first is breaking towards second a hit-and-run play. It may have been intended as a straight steal, but if the batter hits the ball, it becomes a hit-and-run in practice, at least from the standpoint of the defense. On these plays, either the second baseman or the shortstop is breaking towards second to cover a possible throw and the dynamics of the defense change completely. For the Range & Positioning System, we use Hit-and-Run as another variable. After Volume II of The Fielding Bible, we tweaked the system to look at adjacent vectors for the hit-and-run adjustment to give a better estimate of each play's difficulty. For middle infielders, we don’t make the Bases Saved adjustment, since almost all balls hit anywhere near them go for singles anyways. In The Fielding Bible—Volume III, we began calculating a separate basis for infielders when facing left-handed and right-handed batters, beginning with the 2010 Range & Positioning calculations. Third Base – At third base we make the same Bases Saved adjustment as first base, but not the Holding adjustment. In The Fielding Bible—Volume III, we began calculating a separate basis for infielders when facing left-handed and right-handed batters, beginning with the 2010 Range & Positioning calculations. Outfield – For outfielders, we also use both Plays Saved and Bases Saved, since balls not fielded by outfielders frequently wind up as extra-base hits. For outfielders, we do not make an adjustment based on the batter's hand. How do you handle the “ball-hogging problem”, where one fielder makes a play that another fielder could have also made? A play made by one fielder never counts against any other fielder. We remove those plays from the denominator. For example, say there are 100 balls hit into a certain “bucket” (balls with the same trajectory, location, velocity, etc.). Suppose the left fielder handles 40 of them, the center fielder handles 50 of them, and 10 fall for hits. It’s obviously a pretty easy play for both fielders, since it rarely fell for a hit. The difficulty level of the play (which we call the “Ratio”) for the center fielder is 50/(100-40) = .83; for the left fielder, it’s 40/(100-50) = .80. How do you handle the Green Monster in Boston and other strange park oddities? In 2007, we introduced the “Manny Adjustment.” In this adjustment we eliminate any ball that hits an outfield wall that is out of reach of the outfielder (i.e. too high on the wall). Basically, we’re treating a ball hitting a wall out of reach in the same way we treat a home run. They can’t be caught so they are left out of the universe of plays to consider. As for non-wall park peculiarities, we only compare balls that are in play in one park to other parks where the ball was also in play. In other words, a hard flyball hit 405 feet to Vector 183 in center field is only compared to other hard flyballs that stayed in play when hit 405 feet to Vector 183. The system doesn’t know that this particular ball was a home run in other ballparks; it only knows that when that particular ball was in play, it was caught X% of the time. This is a de facto park effect—because the plus/minus zones are so precise, it handles strange wall and park configurations pretty well. What about infield flies and pop-ups? We treat infield flies and pop-ups very similarly to outfield flies. We then add an infielder’s score on flies to his score for groundballs to arrive at his final Range & Positioning total. How do you account for a player’s positioning before the pitch? Positioning, reaction, and range all factor into a player’s defensive ability. Whether an outfielder made a routine catch because he was positioned well or he had to sprint twice as hard because he got a poor read on the ball off the bat, the out still counts the same. The Range & Positioning system doesn’t know why the player made the play; it just knows that he did (or that he didn’t). As a result, the best plays according to the Range & Positioning System aren’t always flashy Web Gem nominees. As with anything else, the best defensive players can make the hard plays look easy. The Range & Positioning system does make a few situation-specific adjustments: one for first basemen holding runners and another for middle infielders on hit-and-run type plays. Beginning in 2010, we also began splitting groundballs by left-handed batters from grounders by right-handed batters to account for the difference in positioning against opposite-handed batters. What do you think is the appropriate sample size to use when evaluating a player’s Plus/Minus? The higher, the better. When evaluating a player using any statistic, you should always be aware of the sample size you’re dealing with. As more and more data becomes available, we can make more definitive conclusions. This is especially true of defense. For example, if a player has Range & Positioning numbers of +2, -5, and +4 over the course of 3 seasons, it is a safe bet that he is a league average fielder. Just like with offensive stats, players can have “fluke” months or seasons which don’t reflect his actual ability. It’s important to remember that all statistics—whether it’s batting average, Runs Saved, or an election poll—aren’t always indicative of true value: they’re just estimates within some amount of measurement error. Additionally, players can have uncharacteristically good or bad days, weeks, or months which have little effect on what we think of them going forward. When Mark Teixeira is hitting under .200 at the end of April, you write it off as a bad month and hope that he recovers soon (if you’re a Yankees fan, that is). Players can have seemingly good and bad defensive months and seasons too. How do you come up with Runs Saved? A fielder’s Defensive Runs Saved (Runs Saved, for short) indicates how many runs a player saved or cost his team in the field compared to others at his position. There are nine components we use to total a player’s Runs Saved: - Range & Positioning Runs Saved (All Non-Catchers) - Catcher Adjusted Earned Runs Saved (Catchers) - Strike Zone Runs Saved (Catchers) - Catcher Stolen Base Runs Saved (Catchers) - Pitcher Stolen Base Runs Saved (Pitchers) - Outfielder Arm Runs Saved (Outfielders) - Bunt Runs Saved (Corner Infielders, Catchers, Pitchers) - Double Play Runs Saved (Middle Infielders and Corner Infielders) - Good Plays/Misplays Runs Saved (All Fielders) In order to translate each component to Runs Saved, we consulted the “24-States Run Matrix”. We compared the expected number of runs allowed before and after each play and calculated the average change in run expectancy for each event. We then apply these average run values to convert to Runs Saved. What's a good Defensive Runs Saved total for a season? The best fielders at each position typically finish with between 15 and 20 Defensive Runs Saved at the end of the year. In other words, we estimate that they saved their team 15-20 runs more than an average fielder at the position. Certain elite defensive players have approached or surpassed 30 Defensive Runs Saved in a full season. On the flip side, the worst fielders usually finish around -15 or -20 Defensive Runs Saved. Usually, teams realize these defenders are significantly costing the team runs, and the offending player is moved to another position or benched altogether. How do you measure an Outfield Arm Runs Saved? We account for the strength and accuracy of an outfielder’s arm by comparing the rates at which runners advance in potential extra-base situations. In extra base situations, the runner could either 1) advance safely, 2) get thrown out attempting to advance, or 3) hold at the previous base and not challenge the outfielder. Not all extra base advancements are created equal, so beginning with The Fielding Bible--Volume III, we made a few modifications to our calculation. First, we split grounders from fly balls. Then, we added hit locations as part of our calculation. For grounders, we also consider velocity. Based on average 24-states run expectancies of each outcome as well as the ball in play type, location, and velocity, we award the outfielder the appropriate credit/penalty for his contributions above/below the average fielder at his position. We also account for miscellaneous “kills”, where the outfielder directly guns down an opposing baserunner in a situation we haven’t already covered. We compare the outfielder's expected number of miscellaneous "kills" to his actual total and multiply the difference by 0.75, the run value of these miscellaneous plays. What if runners stop running on the outfielder based on his reputation? We account for this by doing more than just adding up outfield assists or baserunner kills. If a runner holds up at third on a single to right field based on the right fielder’s reputation, he gets credit for holding the runner at 3rd and not scoring. Just because a fielder doesn’t throw a runner out doesn’t mean he hasn’t saved his team a run. How do you measure Bunt Runs Saved? Part of a corner infielder’s defensive responsibility includes fielding bunts. We rate each fielder based on the outcome of each bunted ball hit to him, whether it’s an out, hit, error, sacrifice, or double play compared to the league average rates. How do you measure Double Play Runs Saved? Each middle infielder is rated on their double play rates compared to the league average. Beginning in 2009, we incorporated timer and location data into the estimation of the difficulty of turning a double play. We split the credit for double plays between the lead man and pivot man and adjust each for the league average conversion rate at each position. How do you measure Pitcher Stolen Base Runs Saved? As discussed in The Fielding Bible - Volume II, an often-overlooked part of a pitcher’s defensive contribution is his control over the running game. John Dewan estimated that 65 percent of the control is in the pitchers’ hands while the remaining 35 percent is up to the catcher. Since runners are less likely to even attempt stolen bases with Zack Greinke or Dallas Keuchel on the mound, we consider both stolen base success rates and stolen base attempt frequencies in our calculations. How do you measure Catcher Stolen Base Runs Saved? Catchers also have a hand in controlling the running game. We also control for the pitcher’s historical stolen base rates in order to isolate the catcher’s contributions. How do you measure Adjusted Earned Runs Saved? Much more than simple Catcher ERA, Adjusted Earned Runs Saved accounts for the quality of the pitching staff the catcher works with. There are still many other variables in play and some noise in the results, so we regress the final data to a more reasonable scale. How do you measure Strike Zone Runs Saved? Strike Zone Runs Saved ascertains the contribution the catcher, pitcher, batter and umpire is making in getting more or fewer called strikes than average, converted to a run value. To compute Strike Zone Plus/Minus, the pitch is categorized by its location, the count the pitch was thrown in, the proximity of the pitch to the catcher's target, and the batter's handedness. That allows the determination of the percent likelihood that each pitch is to be called a strike. The full array of these strike percentages represents our Strike Zone Plus/Minus Basis, i.e., the basis by which credit is assigned if the pitch is called a strike or debit if the pitch is called a ball. This uses a rolling basis of four years of data to determine how to bucket our pitches. If a pitch is called a strike, there is positive credit to be awarded (plus) to the players involved. If a pitch is called a ball, there is negative credit to be assigned (minus). The amount of positive or negative credit given depends on how likely that pitch was to be called a strike in the first place, which is known from the Basis that was previously calculated. For more information, read our paper that won an award at the Sloan Sports Analytics Conference at MIT in 2015. When you say a player’s Runs Saved total ranked 12th at his position, how many is that out of? When a player is ranked 12th at his position, it means he ranks 12th out of the 35 players who played the most innings at that position. For pitchers, we rank the top 175 in innings pitched. Matt Olson had 14 Runs Saved at first base in 2018, while Francisco Lindor had 14 Runs Saved at shortstop. Does that mean that Olson is as good as Lindor defensively? No- each Runs Saved total should be compared to other fielders at the same position. Obviously, shortstop is a much more difficult position than first base and requires a very different set of skills. Plus/Minus and our other Runs Saved components consider each position separately. In The Fielding Bible - Volume II, Bill James wrote an article, “Crossing Positions”, which outlines a process for putting each position on the same scale. Building off Bill’s article, we developed Total Runs. What is “Total Runs”? Total Runs is the total of a player’s contributions to his team when taking into account Runs Created on offense, Defensive Runs Saved, Baserunning Runs, and a positional adjustment. Roughly speaking, it measures the positive impact a player had on his team. As a frame of reference, the leaders in total runs are annually between about 160 and 180 Total Runs. Why is the Positional Adjustment sometimes different for people who play the same position? This occurs because the positional adjustment is based not just on what position a player plays, but for how many innings a player plays at that particular position. A fielder who plays shortstop the entire season will have a higher positional adjustment than someone who is more of a utility player and spends his time at various other positions. Where can I find the Plus/Minus and Runs Saved stats and analysis? Plus/Minus and Runs Saved stats are available at BillJamesOnline.com, along with a variety of other things that may interest you. Bill James Online features all of the stats you can find in the book, updated through the previous night’s games. Y ou can also find updated Runs Saved data on Fangraphs.com and Baseball-Reference.com. What’s the difference between Runs Saved and UZR? Both systems have the same goal- estimate a player’s defensive worth in units of “runs”, and both rely on hit location and play by play data from Baseball Info Solutions. The differences lie in the various adjustments and calculations that are made. For example, Defensive Runs Saved utilizes the batted ball timer data, while Lichtman relies on the type and velocity categorizations. Defensive Runs Saved also includes components to measure pitcher and catcher defense. BIS owner John Dewan has specialized in defensive evaluations for decades. While still with STATS, Inc. John developed the original “Zone Rating”, which rates each fielder on his ability to make plays on balls hit inside a pre-determined zone for each position. Shortly after leaving STATS, he began work on a more advanced defensive metric. metric called “Ultimate Zone Rating” was introduced in the STATS Scoreboard books shortly before Dewan left the company. Dewan’s ongoing research eventually produced the Plus/Minus System. Shortly thereafter, Mitchel Lichtman began calculating UZR on his own, with additional calculations and small adjustments of his own. Lichtman’s UZR is maintained on Fangraphs.com using the same BIS data. What are “Good Fielding Plays” and “Defensive Misplays”? To overly-simplify it, a Defensive Misplay is any play where the fielder screws up. More specifically, a misplay is recorded when a fielder does something identifiably wrong AND an opposing batter reaches base or takes an extra base as a result. There are currently 60 types of Defensive Misplays we record. A Good Fielding Play is recorded when a fielder does something to prevent an advancement or record an out that we wouldn't typically expect from a fielder at the position. It is a play that is made when, had the play not been made, no one would have faulted the fielder for not making it. There are currently about 30 different types of Good Fielding Plays. A player cannot be rewarded with a Good Fielding Play unless there’s a positive consequence - an unexpected out or outs are recorded or runners are prevented from advancing. Defensive Misplays work in the same way in that no matter how many times an infielder, for instance, kicks a grounder, if he is able to record the out at first and no other outs (such as a force) could have been recorded if he fielded the ball cleanly, it is NOT scored a Defensive Misplay. While it’s quite obvious that it is impossible to remove all subjectivity from awarding GFPs and DMs, Baseball Info Solutions strives to be as objective as possible. A baseball observer, understanding that a GFP is a play that is made in an environment where the outcome in unclear, can recognize this environment and select the correct GFP from the list. How do Good Fielding Plays and Defensive Misplays factor into your analysis? We have taken many of the Good Fielding Plays and Defensive Misplays and incorporated them into “Good Play/Misplay Runs Saved”. For example, we studied how catchers block pitches in the dirt and first basemen handle difficult throws at first base. The difference between the best and worst fielders in these regards can amount to several runs over a season. Good Play/Misplay Runs Saved and the seven other components make Defensive Runs Saved the most comprehensive defensive statistic available. Who determines whether the play is a Defensive Misplay? BIS video scouts, trained specifically to recognize and record Defensive Misplays and Good Fielding Plays, record DM and GFP information. Who are your video scouts and what do they do? BIS video scouts are traditionally former professional, college, and amateur players who help collect and maintain the tremendous volume of data at Baseball Info Solutions. Our video scouts track every MLB game using our specialized software to track hit location, hit type, and hit velocity. They also record pitch types, velocities, and locations. Starting in 2009, we also began recording objective batted ball timer data, which has begun to dramatically improve the accuracy of hitting, fielding, and pitching analysis. How do you train your video scouts? First of all, no one makes it past the first cut without baseball knowledge. Each video scout has to take a rigorous pitch charting test and earn a score high enough to meet our standards. After selecting the video scouts, we bring them in for two weeks of intense training with our full scoring and charting procedures. Between charting and scoring, BIS video scouts are watching an average of two games per day and get to know the whole league better than almost anyone. How do you ensure that you’re providing quality, unbiased data? BIS video scouts are assigned randomly to games. Scorers have a designated number (Ex. Scorer #1009) which are then rotated through different slots in the schedule. If scorers 1007 and 1008 are scoring the late (west coast) games one day, they’ll be rotated to early games the next time around. There’s some miscellaneous switching to accommodate vacation, etc. too. In the end, everyone’s getting a good mix of every team in every park. We also have several different quality control methods in place to make sure that scorers are consistent and accurate with hit locations and types. We’re constantly improving our data quality so that each year’s data is the best ever collected. How can I become a BIS Video Scout? We start reviewing candidates for our video scout team during the MLB Winter Meetings each December. If you can’t make it to the Winter Meetings, be sure to send us your résumé in early December. We also occasionally seek an intern to work in our R&D department. Contact information is available at SportsInfoSolutions.com. I can’t arrange my schedule to work a full-time internship in Pennsylvania. Where can I find out if you have any other positions available? We have a network of minor league scorers across the country covering the upper levels of the minor leagues. For more information on minor league scoring opportunities, check here. |
|
 | |
