ADVERTISEMENT
Science Talk

Take Me Out to the Run Expectancy Matrix Analysis

Smith College economics professor Andrew Zimbalist talks about his latest book, The Sabermetric Revolution: Assessing the Growth of Analytics in Baseball (co-authored with Benjamin Baumer), at the Bergino Baseball Clubhouse, with proprietor Jay Goldberg

Podcast Transcription

[music playing]

S. Mirsky:  Welcome to the Scientific American podcast’s “Science Talk”, posted on March 18th, 2014.  I’m Steve Mirsky. On this episode:

A. Zimbalist:      You know, you could confirm that by using run expectancy matrices, or you can use linear weights and confirm that.  The problem is that what’s true on average, isn’t necessarily true in every circumstance.

S. Mirsky:  That’s Andrew Zimbalist, and he is, of course, talking about baseball, which starts up its regular season March 22nd, with the Dodgers and Diamondbacks playing at the historic Sydney Cricket Ground in Australia.  Everybody else cranks it up; well, we can change later.  Andrew Zimbalist is a professor of economics at Smith College, and he’s probably the foremost sports economist in the country.  His latest book, written withBen Baumer, is The Sabermetric Revolution:  Assessing the Growth of Analytics in Baseball.  A game like no other in its capacity to generate data, meaningful and otherwise.  On January 28th, Zimbalist sat down with Jay Goldberg, proprietor of Bergino’s Baseball Clubhouse, a high-end baseball memorabilia gallery, at 67 East 11th Street, here in New York City.  Goldberg is a lawyer and former player agent.  What follows is an edited version of the conversation between Goldberg and Zimbalist.

J. Goldberg:       Define sabermetrics, mainly for the people who are listening to the podcast, who may not know.

A. Zimbalist:      So sabermetrics is a term that Bill James invented a couple of decades ago, and its origin is from the Society of American Baseball Research; SABR.  Bill put an e between the b and the r and added “metrics”, and that’s the word.  So what does it refer to?  It refers to the use of statistical analysis to understand and evaluate player performance, team strategy, and front-office strategy. 

J. Goldberg:       Your last paragraph of the book, I would like to start with, actually.

A. Zimbalist:      Okay. 

J. Goldberg:       Mainly because I think there’s this misconception out there, which ties in with Moneyball – which we’ll get to in a second – that people think sabermetrics, if you’re into sabermetrics, you’re completely against scouting, or vice versa.  So the final paragraph of Andrew’s book is:

         While Michael Lewis emphasizes the conflict between scouts and sabermetricians, smart baseball executives today know that there is no reason to tie one hand behind their backs.  There is no sense in arbitrarily limiting the amount of information you gather.  The trick is to parse and process the information effectively; a lesson that all companies have to learn. 

         And in the preface, you talk, you start right away about – and then getting into it in the first chapter – about Moneyball, and the, you set the record straight a little bit, on the book and the movie.

A. Zimbalist:      Sure.  So let me, if I could say that I didn’t write the book by myself; I wrote it with Benjamin Baumer – or Ben Baumer – who, for eight years was the chief sabermetricians for the New York Mets, and got a PhD while he was doing that, at, here in New York at CUNY, and got a job two years ago at Smith College, teaching math.  So he and I wrote the book together.  And so I appreciate your references to me as the author, but I’m just one of them.

J. Goldberg:       [laughs] Okay.

A. Zimbalist:      Alright.  So, the book Moneyball, I think is a very important book in baseball history.  It served the function of canalizing the use of sabermetrics in baseball.  It popularized and made acceptable the hiring of statistical analysts in baseball’s front offices, it dramatized the success of the Oakland A’s in 2002, and it basically got all of the te-, today there 26 teams, by my count, that are practicing sabermetrics. 

But I think that whatever else one can say about the book Moneyball, it catalyzed that, accelerated that development.  I’m not sure that, other than it being a wonderful story – and Michael Lewis is a great storyteller – other than those two things, I’m not sure that you can – or at least I would want to – say very much positive about the book.  ‘Cause I think it misrepresents the historical genesis of the use of statistical analysis in baseball.  

I think it misrepresents what was happening to the A’s in 2002, it misrepresents the draft, it misrepresents sabermetrics by having him characterize Billy Beane as somebody who thought that statistical analysis told us that when you do an amateur draft, you should only pick college players, you shouldn’t pick high school players.  It mischaracterizes Billy Beane when he says that you should never bunt or that you should never steal.  It mischaracterizes the players that he talks about; he talks about a pitcher by the name of Chad Bradford.  He has a sentence or two for Hudson and Zito and Mulder; he does, he has a sentence or two for each of – but there’s a whole paragraph on Chad Bradford. Bradford, you might remember, was a submarine pitcher, and he talks about Bradford’s upbringing and family situation, so on and so forth, and talks about how nobody would, believed he’d ever be a major-league pitcher.  And then he comes out and he says Bradford threw the ball 84 miles an hour.  But he says that Bradford’s delivery was such – because he was a submariner – that it actually appeared, to the batters, as a 94-mile-an-hour fastball.  He said because he was a submariner, when he delivered the ball, his hand was so much closer to home plate, that even though the speed was 84, in terms of the amount of time that the batters had to swing at it, it was a 94-mile-an-hour fastball. 

So think about this.  This is the guy who’s interpreting the introduction of statistics into major league baseball for us – Michael Lewis – who made this comment.  So think about what that comment means.  If the fastball goes from 84 miles an hour to 94 miles an hour, then it’s a 12-plus percent increase in speed.  That suggests that the pitcher’s mound, or his hand release must’ve been 12% closer than 60 feet, 6 inches.  12% of 60 feet, 6 inches, is longer than Shaq O’Neal’s body length; it’s about 7, almost 7 feet, 3 inches.  It’s patently absurd that that could happen.  And I threw that out with some detail, because I think it’s indicative of the way he butchers statistics and the way he butchers the history of the use of statistics in baseball. 

So the book begins with – kinda try to, clearing the debris out of the air – as we talk about the book, we also talk about the movie, which of course, as movies are wont to do, distorts reality still further.  And then we launch into the gists; we talk about how sabermetrics has proceeded through American front offices:  how many teams practiced it in 2003, how many in 2007, how many do it today, what is it that they do when they practice it?  Then we have a couple of chapters that introduces people; it’s kind of a basic primer on fundamental sabermetric principles.  Then we have a chapter called “The Moneyball Diaspora” that looks at the introduction of analytics or statistical analysis for the other sports:  basketball, football, hockey, soccer.  Then there’s a chapter on the use of statistical analytics for understanding the business of baseball.  And then we conclude with turning the table on sabermetricians. 

So sabermetricians, as know them, evaluate player performance; they’re evaluating others all the time.  And so we turn the tables on them, and we try to evaluate sabermetrics, using statistical indexes to see how sabermetrically-oriented – or what we call “saber-intensity” – each team has been, and then we correlate that index of saber-intensity with team performance, after subtracting out the impact that team payroll has on team performance.  So we look at those things together, at the end of the book, and make some conclusions about how productive or lack of productivity there is in sabermetrics.  So that’s, you only asked me to talk about the first chapter, but too bad, I’m a, [laughter] I’m a professor; as long as the door is closed, I talk for an hour and 20 minutes, and then I shut up.

J. Goldberg:       No, that was perfect, that was perfect.  There’s a great table in here, which touches upon something that you mentioned; that 26 of the 30 teams are practicing analytics in some method.  So I know people are gonna say, “Who are the four teams?”  So I just wanna pick those from your chart, just so people know, and then we can get off them.  But it’s a range of quality to dreg, as they say.  Atlanta, Colorado, Miami and Philadelphia are the four teams that, as you write, teams with no apparent analytical presence.  So now there’s this information out there; 26 teams, in theory, are dealing with information.  Of the top teams that are really heavily into it – Tampa, Cleveland, the Yankees, Boston, the Mets – again, there’s quite a difference of results.  And this is as a Mets fan.  What’s really distinguishing one from the other, at this point?

A. Zimbalist:      Well, I think one of the things that distinguishes them is, if you look at the Tampa Bay Rays, they’ve got probably eight, nine, or ten people who are in their analytics department.  And they’ve got some people who are doing programming, and they’ve got some people who are doing what most of us would regard as standard sabermetrics.  And they’ve got some people who are using videos, or analyzing videos.  So it’s a very large department, and they recognize that there’s a lot there, they recognize that the stuff is all integrated and related to each other.  They have a guy, Andrew Friedman, who’s the head of the baseball operations there, who comes from Wall Street and has a very analytic bent.  And the culture of the A’s is such that, when Andrew Friedman and his team have an insight and they think something should be done, then there’s a dedication to doing it.  I think –

Male:         The Rays, right?

A. Zimbalist:      Did I not say the Rays?

Audience: You said the A’s.

A. Zimbalist:      Rays, sorry, yeah.  So there’s a dedication to doing it, and it does get done.  I think it’s also important to – and the A’s are similar, the A’s are similar in that regard – but one of the things that’s characterized Billy Beane – and by the way, even though I critique how Lewis treats Beane, I’m a great admirer of his; I think he’s brilliant, and I think that he deserves to be singled out as a pioneer in this area – but if you talk to Billy, what he says to you is that the use of numbers to analyze performance and to analyze strategy is a relatively modest part of what they do, when they’re making those organizational decisions.  They use a lot of video, they spend a lot of time these days trying to understand the players’ makeups.  They have the video, now, analysts and the statistical analysts working side by side together, which they did, by the way, back in 2002 also. 

And 2002 is not – one of the things that’s interesting – is that 2002 is not the year that things began to change in the Oakland A’s, and it’s not the year, if you look at statistics, like on base percentage, that that goes highest.  In fact, it goes lower, substantially lower, in 2002, for the A’s, relative to 2001 or relative even to 1999.  So that’s another thing that’s misrepresented.  But I think that there’s a culture of saying, “How can we figure out something that nobody else has figured out?”  And it doesn’t have to be with a number.  It could be because they’re spending more time trying to understand the players’ emotional makeup than other teams are.  And indeed, one of the things that I think Billy Beane recognized early on, when all of this attention started getting lavished on on-base percentage – through movie, through the book, and later the movie – is that if all the teams start piling on, all the teams say, “Oh, we’ve gotta emphasize on-base percentage,” guess what happens? 

One of the great insights for Billy Beane was not just simply that on-base percentage was important.  Allan Roth, who worked for Branch Rickey in the 1940s in the Brooklyn Dodgers, he knew that too.  One of the things that was distinguishing about the way Beane applied that knowledge, was that not only was it important, but that it was a characteristic or an attribute that was undervalued.  Other teams were not appropriately valuing it.  But so along comes the book, and guess what?  Everybody says we gotta value on-base percentage, we gotta value the walk rate of a player, and the marketplace evaluation went up so fast, that now it’s overvalued.  So what did Billy Beane do?  Billy Beane didn’t stick around and say, “By golly, I’m gonna keep on emphasizing on-base percentage.”  He kinda threw that out and started finding something else.  And that’s the instruction that he gives. 
So I think that one of the things that characterizes the successful practitioners – the teams that are most successfully practicing sabermetrics – is an innovative and creative spirit, and it’s intelligence.  If you think about it in other walks of life, when you introduce a novel or creative idea, a novel way of doing things, people who are self-confident, who are intelligent enough and feel confident in their intelligence, they welcome that.  They say, “Oh, that’s interesting; let’s pursue it.  Let’s dig down.”  But if you’re not particularly intelligent, and you’re insecure in your knowledge of something and you see something new, right away it’s a threat, and you push it away.  So I think that, you know, one of the things that characterizes the successful front offices is you got a lot of smart people in those offices.

J. Goldberg:       So then the only, my remaining question on this particular table is:  Is the decision not to get into analytics really, is it from the owner down, or where does that come from, at this point?

A. Zimbalist:      I think it’s from the owner down.  Yeah, I mean Walt Jocketty was in St. Louis, and the DeWitts – Bill DeWitt and Bill DeWitt, Jr. – wanted to move in the sabermetric direction, and Jocketty didn’t get along with them and he left.  And they have a new guy there who’s terrific, very involved with statistical analytics.  So yes, I think it’s primarily the owner, but sometimes the GM can go to the owner and say, “We have to do things a little bit differently, and I wanna hire somebody in this area.”  By the way, it’s beginning to change now, but five years ago, you can hire a sabermetricians – somebody who maybe majored in statistics or economics and has a BA degree and nothing more – you could hire one of those people for $40,000.00 or $50,000.00, $60,000.00.  So you could hire a team, hire three of these guys, it’d cost you $150,000.00.  You don’t need sabermetrics to be very productive [laughs] if that’s all you’re, that’s one-third of the minimum, it’s less than one-third of the minimum salary, in major league baseball for a player now, which is $500,000.00. 

But I think, and one of the things that we find in the book is that, indeed, there is pretty strong statistical evidence that sabermetrics does – the application of sabermetric insights – does indeed enhance performance.  And at the end of the book, I’ll give you one of our punchlines.  I think one can reasonably construct a statistical argument that says that those teams that were 10% more sabermetrically oriented than the average in the league, between 2000 and 2012, 10% more would, on average, have 4.5 more wins per year.  So that’s, using sabermetric jingo, that’s – or, lingo – it’s a war of 4.5.  On average, each win is roughly worth – it varies a lot from city to city, win percentage to win percentage – but on average it’s about $5 million.  So you’re looking at a $20 million-plus benefit from the sabermetric orientation.  And until recently, that cost you $150,000.00; that’s a pretty good rate of return, right? [laughter] 

What’s happened to sabermetrics is that the amount of data that’s involved with big data – and I was with Bloomberg yesterday, with Bill Squadron, who runs their analytics department – it’s unbelievable how much data there is.  And it’s gonna multiply several-fold, in the coming months and years.  So you’ve got to hire people who understand how work with big data, and you gotta hire people who know how to parse through that data and pick out what’s important, and then, also, people who can analyze it.  And part of the problem today is that everybody’s doing – except four teams – everybody’s doing it.  So how do you get a leg up on the next guy?

J. Goldberg:       And one of the areas, or a couple of the areas where that is being used, which I thought was fascinating in the book, is in strategies.  And maybe where some of this perception has changed over time, with platooning, clutch hitting, and sacrifice bunting.

A. Zimbalist:      Yeah.

J. Goldberg:       If you could just speak a little about that.

A. Zimbalist:      So platooning is actually something that a statistical analyst who wrote about baseball in the 1940s and ‘50s, George Lindsey, he wrote some articles where he was looking at platooning and he found, indeed, that there’s something like a 20% boost in batting average when you face pitcher from the opposite side, throwing from the opposite side.  The numbers that Ben and I cranked out –looking at OPS, not batting average – were that, on average, lefties batting against right-handed pitchers had 60 points higher in OPS than lefties batting against left-handed pitchers.  And righties batting against left-handed pitchers had a 40% - I’m sorry, a 40-point higher OPS than righties batting against right-hand pitchers.

So it was very clear that anybody who bothered to look at the evidence –it’s been around for a long time; Lindsey wrote about it, some people knew about Lindsey – and then you had people like Earl Weaver, who I had the good fortune to talk to for a while, a few months before he passed away.  And Weaver explained that he was always a minor-league player; he never made it to the majors.  And he wasn’t terribly good as a minor leaguer, at least as he describes it.  He said he found himself to be very successful against some of the best pitchers in the minor leagues.  His best-hitting teammates wouldn’t be able to touch these pitchers, and Weaver would get up there, and he’d have a field day.

On the other hand, Weaver said, the weakest pitchers, very often he couldn’t hit at all, and most pitchers he couldn’t hit at all. [laughter] So what he finally said to himself was, “This is very idiosyncratic.  Arm angle matters, feeling comfortable matters, all these things matter.  And one of the things that he noticed was that this platooning effect was important.  And so he comes, and he’s manager of the Orioles, and he starts platooning players.  And he has people in the front office running down every inning with 3x5 cards saying that this person hits, this person on the Orioles bench hits that guy pretty well, doesn’t hit that guy well.  So he’s doing this.  It’s rudimentary, but it’s insightful and it’s creative and innovative; he’s doing this analysis. 

Charles Steinberg, who’s the PR director for the Red Sox, and has been for a long time – he’s a brilliant man – he’s been following Larry Lucchino around.  He started off as the dentist for the Baltimore Orioles in the 1980s, and they liked his comments so much about relating to the community, that they made him the PR director at the Orioles.  He went with Larry to San Diego, and he’s been in Boston with a very small interlude in between.  Anyway, Steinberg was one of the guys who’d run from the front office with his cards – 3x5 cards – down to the dugout and give ‘em to Earl Weaver.  So that’s the platooning effect.  I mean, it doesn’t work for all players, but on average, it works, and it works in a fairly dramatic way. 

Bunting.  The early sabermetric wisdom on bunting was that you don’t do it, because you’re giving up an out.  By the way, if it works, right – ‘cause sometimes you’re sacrifice bunting and you strike out, or you pop out or whatever and it doesn’t work at all – but when you sacrifice and it works, you’re givin’ up an out.  And giving up an out is bad news; you only get three per inning, and the advantage you get by having a player on second base rather than at first base isn’t greater than the disadvantage you get by yielding an out.  So on average, the sabermetricians said, “Don’t bunt.”  That’s fine, on average.  You could confirm that by using run expectancy matrices or you can use linear weights and confirm that. 

The problem is that what’s true on average isn’t necessarily true in every circumstance, right?  One of the problems with the early analysis of bunting was that they were looking at the average number of runs that were scored in an inning when you sacrifice somebody –with one out or with, with no outs or with one out – what happens to the average number of runs you score per inning.  And it goes down.  But if you’re in the eighth inning or the ninth inning in a close game, you’re not necessarily interested in maximi-, when you make a strategic decision, you’re not necessarily interested in maximizing the number of runs you’re gonna score for the whole game.  You’re interested in getting one more run across the plate, so you go up by a run and you can bring in Mariano Rivera.  Or you’re interested in getting one run across the plate so you can tie the game and go into extra innings.  You’re not really thinking about, “I’d rather have eight runs that seven runs,” or whatever. 

So that’s one of the areas in which one would have to modify the dictum that you don’t bunt.  But of course there are others, too.  Because pitchers – and if we’re talkin’ about the National League – pitchers probably aren’t gonna be successful when they hit.  And so it depends on who’s up; what’s the batting average of the person who’s up, and how fast is the runner on first base and if we didn’t sacrifice bunt, what’s the chance that he can steal a base, and who’s up after the guy who’s bunting?  I mean, there are all sorts of questions, like, do you have Mariano Rivera in the bullpen, or do you have David Robertson in the bullpen?  Or do you have somebody else in the bullpen, who’s gonna come in and shut the other team down?  So it turns out, when you start going through all of the permutations and the details, that bunting sometimes makes sense. 

And it’s roughly the same thing with stealing.  The early sabermetric wisdom was “don’t steal”, ‘cause you could potentially be giving up, you’re giving up an out when you go to second base.  But it turns out that if you can steal it roughly, successfully steal a base at roughly 67% rate of success – two-thirds of the time you’re successful – then that’s more or less break-even.  And if you can steal it 70% or 80%, then it’s more than break-even.  But again, it will depend upon the details of the circumstance of the game.  So a lot of the sabermetric wisdom that was around 10 years ago has been nuanced and made more adaptable to different game circumstances. 

J. Goldberg:       And some of these strategies, the flip side – for example, with bunting – and this is taking it to an extreme but – when I would watch Keith Hernandez play first base, he was so amazing defensively, he took away a manager, even if a manager wanted to bunt, sometimes he had to change his strategy, because Keith Hernandez was gonna take away that bunt from him.  So he wouldn’t even attempt it.

A. Zimbalist:      Sure, there you go, right.

J. Goldberg:       So where, defensively, where are we with sabermetrics?

A. Zimbalist:      I don’t think we’re anywhere. [laughter] We have a lot of measurements, primarily UZR, but when Derek Jeter is in the bottom 10% of UZR one year, and the next year he’s in the top 10%, you have to question what is UZR measuring?  Are you measuring a skill, when there’s so much variance from one year to the next year?

Male:         What are these initials, UZR?

A. Zimbalist:      What does it stand for?

Male: Yeah.

A. Zimbalist:      Ultimate Zone Rating.  So what they’re trying to do with UZR is to take advantage of an insight – that I guess we can attribute, in modern times, to Bill James – which is that the traditional, the historical measurement of fielding is fielding percentage.  And what that’s fundamentally measuring is, when a ball’s hit pretty much right at you, whether or not you catch it and throw it to first base and get the guy out or not.  And if either the ball hits your glove and bounces out – or you’re Steve Sax and it goes in your glove and you throw into the stands – [laughter] and the guy isn’t out, then that’s an error.  And that’s what’s being measured. 

And this is a concept that goes back, was introduced in the late 19th century; introduced at a time when the fielders basically had a golfer’s glove on their left hand.  Not much more than a golfer’s glove.  And so it was a big deal; it was a big deal if somebody hit a nice, hard line-, groundball at you, that you actually caught it and made the play.  But that doesn’t tell you anything at all – that measurement – or, basically very, very little, about the player’s ability to anticipate where the ball’s gonna be hit, get a good jump once the ball is hit, and then get to the ball to run fast enough to the place where it’s hit, and then to have enough balance when you’re running and you catch the ball, to convert the play. 

And so fielding percentage was increasingly seen as a very limited measurement of somebody’s fielding prowess, somebody’s fielding contribution.  And Bill James developed this notion of Defensive Efficiency Rating, which was basically not whether, when the ball was hit to you, did you convert it into an out, but when the ball was hit anywhere, was it converted into an out or not?  The DER rating  - the Defensive Efficiency Rating – is something that applies pretty nicely to a team, but it doesn’t apply to an individual so much.  And so some individuals, like Michael Lichtman and others, have tried to develop concepts like the Ultimate Zone Rating, or UZR, that attempts to measure these things, about a player’s ability to move laterally and move back and forth, to see what their range is. 

So I think that it’s a nice idea, but I think the way it’s been implemented is very unsatisfying, for a number of reasons.  One reason is that we haven’t had enough information available yet.  We’re starting to get it now, with the field effects, but we haven’t had enough information so that we can distinguish betw- other than saying a ball was hit slowly, medium, hard or fast –those are three categories – we don’t know what the muscle speed was, off the bat. We don’t know how many hops the ball took.  We don’t know if the infield was slippery or not.  We don’t know if the guy who’s playing shortstop was having to lead toward second, ‘cause there was a guy leading off second base or not.  We don’t know whether Elliot, or, rather Greg Maddux was pitching and he threw the ball exactly to where the catcher had his glove, which enabled the shortstop to get a jump on it or not. 

There’s – and I can go on and on – there’s a lot of stuff that we don’t know when we say, “Here’s a player’s UZR.”  And then they divide the field into these bins and you’re supposed to be able say, “Well, if a guy’s playing a typical shortstop position, and the ball’s hit into bin 73 – which, you know, might be whatever, 15 feet from there – 80% of the time, the shortstop can get that ball, if it’s hit into bin number 83 or whatever it is.”  But here again, we don’t know whether the ball was  line drive, whether it was a one-hopper, whether it was a five-hopper, whether they were playing on artificial turf or playing on regular grass, whether the grass was wet or not, whether the shortstop in the traditional position that shortstops play, or maybe there was a shift.  And one of the things that changes UZR is people are shifting all the time, these days.  Particularly the Rays, and I think that’s been a major benefit for them. 

So there are all sorts of issues about, we just don’t have enough information about what’s happened on a play, or the circumstances of the play, to be able to say that the UZR means very much.  If you look at – I threw out the Derek Jeter example; there are lots of others – but if you look at a player’s UZR in year 1, and then you compare it to his UZR in year 2, what you would want, if you were measuring a player’s skill, you would expect that year 1 and year 2 would be pretty close to each other, right?  Well, it turns out – and different people have done this – it turns out that the UZR from year 1 to year 2, for all of the players who have played over the last 20 years or so, a certain number of innings, is about .4; the correlation is about .4.  And what that means is that you explain 16% of a player’s UZR in year 2, when you look at his UZR in year 1.  Which suggests there’s a lot of randomness there. 

So that’s a problem with UZR, but to Ben and my way of thinking, there’s still a larger problem with UZR, and it’s a problem that adheres to many statistics, WAR is one of them.  WAR is W-A-R; it’s Wins Above Replacement player; it’s supposed to be kind of an ultimate comprehensive measure of how productive a player is.  And that problem is that these are now proprietary metrics.  The people who generate these are selling their metrics to teams, or they might be selling them to television stations.  They’re making money off of it.  And because they’re proprietary – and by the way, there are a lot charlatans out there who are practicing sabermetrics and consulting and selling their services, who always claim everything is proprietary and they can’t tell us how they’re doing it –[laughter] and that’s true, I think, of lots of fads. 

You always get, “Oh, there’s a fad that, we can make money doing this, so let’s dress ourselves up as doing this.”  But there’s a real problem, is that, since it’s proprietary, it’s a black box.  They’ll tell us, “Oh yeah, well we have bins and we can see if the ball’s hit slow, medium or fast.”  So they give us a few things, but at the end of the day, the numbers their using to feed into their algorithms and to their formulae, and how they weight all these different numbers, they don’t tell us.  ‘Cause it’s proprietary. 

And as long as it’s a black box, it doesn’t make a heck of a lot of sense.  It doesn’t make a heck of a lot of sense, and you can see, if you go down, if you go to FanGraphs and take a look at what happens to a player’s statistic over time, or go to Baseball-Reference or go to Baseball Prospectus, look at, these guys are smart.  They’re doing good work, but for us to say, “Okay, we embrace that idea,” this is not like on-base percentage; we all know what on-base percentage is, right?  We all know what a homerun is.  But we don’t know what’s getting, what is it that’s in that UZR?  Where did the numbers come from?  How were they put together? 

It’s the same thing with WAR; it’s proprietary.  FanGraphs has got a WAR, Baseball Prospectus has got a WAR, Baseball-Reference.com has got a WAR.  They all have different measurements.  You’re a Mets fan, so one of the __, you may have noticed, we show for David Wright, the third baseman for the Mets, we show the three measurements from these three outfits, about David Wright.  And it’s broken down into different components:  his batting, his fielding, his base running and so on.  And if you look at it, all of them come out, they all have Wright coming out somewhere, with a WAR of around 7; I think that that, for this particular year, some were 6.8, one was 7.1, but they’re all pretty close.  It looks like, “Oh, this is pretty consistent; they’re doing the same thing.”  But then if you break it down, you see that one of them has got him as a negative fielder, and the other one’s got him as a very positive fielder, and one of them’s got him as a negative base runner and the other’s got a very positive base runner. 

So you have to, you simply have to, if we’re gonna accept these estimates and these numbers and these concepts, we have to know what’s there.  And we don’t right now.  And that’s a problem.  One of the things that’s interesting, when you look across the sport, is that in baseball, we’ve had this development, this maturation, of statistical analysis that goes back to the 1970s and Bill James’ writing.  It goes back further, but the burgeoning of the internet and the free-agency era, which starts in 1977, with, salaries are going way up, so now it’s more important than ever to really understand what you’re paying for. 

We had this period, started with Bill James, where sabermetricians were communicating with each other, by telephone, by email, by blogs and so on, that went basically from the mid or late ‘70s up until around 2000.  And all of the people who were participating in this exchange were doing it because there was an intellectual love of the game and an intellectual love and curiosity about the information that they were sharing with each other.  And then when it gets incorporated into the front offices, gradually, after 2000 –although there was some incorporation earlier –when the momentum, when the snowball effect really starts going, after 2000, all of a sudden, there are people who want to make some money off of this – they’re producing valuable pieces of information, after all – and then it all turns in, and it becomes proprietary. 

If you think about statistical analy-, the introduction of statistical analytics in the other sports, they don’t benefit from a couple of decades of open, free exchange.  It’s almost immediately, ‘cause it really all happens after Moneyball, and it’s all at a time period when, and salaries are very high, and there’s a lot of big data processing ability out there.  So it all gets absorbed almost immediately – I’m exaggerating a little bit – almost immediately into this proprietary black-box environment.  And I think it’s one of the many reasons why statistical analytics is less developed in the other sports. 

The other major reason, of course, is those other sports are very continuous in their production function, whereas baseball’s very discrete.  When you’re a batter, might depend a little bit – evidence isn’t so clear – who’s batting after you and other things.  But pretty much it’s you against the pitcher, and if you’re a good batter, you’re gonna do pretty well.  It’s not true for a quarterback, in football, right?  Quarterback in football needs a good line, needs some good ends, maybe needs a coach to design the plays well.  So there’s all this interdependence that goes on, not only in football, but in all the other sports.  Makes it much harder to isolate meaningful statistics for how good each player is. 

But baseball, both because of this maturation period of a couple of decades of statistical analysis, and because of the nature of the sport, it has lent itself to the development of sabermetric analysis.

[music from 00:35:55 to 00:36:03]

S. Mirsky:  Zimbalist answered questions from the audience for another half hour or so.  Some of that interaction got pretty technical.  If you’re interested, the entire session is up at the Bergino Baseball Clubhouse website, at their podcast page.  You can go to bergino.com and navigate to it, or I’ve made a shortened URL that’ll take you directly to Zimbalist.  That’s at goo.gl/w8adI4. And if you’re into this stuff, I recommend visiting the website of the saber conference that ran from March 13th to the 15th in Phoenix.  The audio for a lot of the talks there is up at sabr.org –that’s S-A-B-R.org – /analytics [sabr.org/analytics].  And check out my column on sabermetrics and Zimbalist’s talk in the April issue of Scientific American, now available on our website. 

That’s it for this episode; get your science news at our website:  www.scientificamerican.com, where you can check out our coverage of the verification of the presence of gravitational waves, which gives us new information about the state of the universe in the first infinitesimally-short moments after the big bang, which is not Walter Johnson’s nickname, but it’s close.  And follow us on Twitter, where you’ll get a tweet whenever a new item hits the website.  Our Twitter name is @sciam.  For Scientific American’s “Science Talk,” I’m Steve Mirsky.  Thanks for clicking on us.

[music from 00:37:30 to 00:37:54]

[End of Audio]


This is an edited version of the discussion. For the complete version, including the subsequent and technical Q&A session, go to the Bergino Baseball Clubhouse

For audio of talks at the recent SABR conference in Phoenix, go to sabr.org/analytics

Share this Article:

Comments

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Scientific American Special Universe

Get the latest Special Collector's edition

Secrets of the Universe: Past, Present, Future

Order Now >

X

Email this Article

X