Q&A: Jay Jaffe on JAWS and the Hall of Fame

Barry Bonds, Roger Clemens and Sammy Sosa are the hot names on this year's Hall of Fame ballot. (Getty Images)

If you don't believe WAR should be looked at in MVP voting or think anyone who uses an acronym other than RBI is just some nerd who doesn't really enjoy baseball, then you probably don't have much use for JAWS.

JAWS is also known as the Jaffe WAR Score system. It's hosted on Baseball-Reference.com and defined by the site as: "a means to measure a player's Hall of Fame worthiness by comparing him to the players at his position who are already enshrined, using advanced metrics to account for the wide variations in offensive levels that have occurred throughout the game's history."

A player's JAWS uses their career WAR averaged with their seven-year peak WAR. For the defensive component, Jay Jaffe uses Defensive Runs Saved from 2003 on and Total Zone for the years before 2003. In the end, it gives a good measure of how different players at their positions stack up against the best of all time, and we're using it to rank all 37 players on this year's Hall of Fame ballot starting Monday. You can view Jaffe's rankings, by position, here.

Eye On Baseball caught up with Jaffe recently to discuss his system and the Hall of Fame. Jaffe is a writer for Sports Illustrated and is also a member of the Baseball Writers Association of America but not yet eligible to vote for the Hall of Fame.

Q: Jay, where did this come from? Have you always been fascinated with the Hall?

A: I started the Futility Infielder blog in the 2002 season and the first time the Hall of Fame ballot came up for examination in January of 2002, was right after Bill James' New Historical Abstract came out and the fall after it came out, I decided to do a series of blog posts about that. It was a fairly popular feature, and it got a huge response the next year, the 2003 ballot. The Baseball Prospectus guys had seen my work, and they invited me to contribute for what would have been the 2004 ballot.

At that point, I came up with a system that kind of derived from the way James had done his position-by-position players, but using BP's Wins Above Replacement Player stat and maybe simplify a little bit. At that time, I was using five hand-picked seasons with a caveat that you had to have, when it came to the World War II guys or Carlton Fisk missing two complete years back-to-back. I think it was two years later that I switched to using seven seasons. By that point, I'd named the system JAWS. It continued in that form until that day. This is the first year I've used Baseball-Reference's Wins Above Replacement instead of BP's WARP.

Q: So why seven years?

A: To me, I studied it, it becomes a bit of a sweet spot. It's kind of an arbitrary decision to use seven years, but it's one I was happy with the alternative view that it provided to looking to career lengths. Career lengths shouldn't be the only determinate. Back when James did the first Historic Abstract in the late '80s, he had ranked guys based on career and peak -- so that distinction was always in my mind. But seven just seemed to be a sweet spot. I don't remember exactly the correlations, but I got a reasonable correlation when I looked at best seven years. If you go to 10 years, you might as well go career lengths, based on some of those careers. I looked backwards and was pretty satisfied with the yields that I got in predicting whether a guy who had a good seven-year score was in the Hall of Fame.

Q: What kind of criticisms have you heard?

A: Mostly, the one I concede every year, is that JAWS doesn't account for everything. It doesn't account for awards and postseasons or historical factors that have to do with more subjective measure, Gold Gloves and things like that. That's why I still believe the numbers are a place to start, but I still think there's plenty of room for discussion beyond that. I think there are reasons to overlook if there are shortcomings.

I think there are some people who don't see WARP or WAR as a valid way to look at baseball. Well, we're never going agree on an advanced metric system for the Hall of Fame. Once you get past that, and if you acknowledge that advanced metrics are a reasonable way to look at Hall of Famers, I don't get a ton of resistance. It's the crowd that still thinks RBI are a good measure or pitcher wins are a good measure. The guys who would put [Miguel] Cabrera over [Mike] Trout are the people who are going to have trouble with this. I'm probably not going to reach them in the long run, because they're fundamentally resistant to the idea that we can measure player value.

Q: This system changes over time, doesn't it?

A: Tweaks to WAR and WARP have happened and change over time, and the averages change over time. Jim Rice's entry the other year lowered the bar for all left fielders, maybe only a point or so, but it does change the answer. But I don't think that's an invalidation of the system. We come to the question with the best available answer with the information we have at the time. But if that information changes over time, we have to live with it. This is science, our answer is never a final answer and we should be wary of anything that purports to be the final answer. To me, this is a more intellectually honest way to go. If it does change our view and we've elected someone who falls below the standard based on one version of WAR or WARP, well, there's still a lot of guys who are a whole lot worse and we know we applied stronger scrutiny than some of the veterans' committees have.

Q: There's a difference between those who have gotten in from the writers and those who are from the veterans' committee, too, right?

A: Absolutely. There's a huge split between the BBWAA guys and the VC guys in general. There's about a 15-point split. I've written about that before and probably will again on this cycle. That's part of the point of the metric.

Q: One of the things I like about JAWS on Baseball-Reference -- and when it first went up, I lost a day -- when you see it, you look at it, it passes the sniff test. Look at the eyeball test, and those lists look pretty much right. I know that's not scientific, but does that hold some weight with you?

A: I think so. That may have been a hinderance when I was at BP; there was no way to lay it out and see it. I would do it in the context of my pieces, but to do it every time when you're talking about Larry Walker or Todd Helton or someone at the end of their career and we want to put placement on him. It was a lot of extra labor. This saves me a lot of labor.

Q: There's something to me, that if I look at right fielders, I see Babe Ruth, Hank Aaron, Stan Musial, Mel Ott, Frank Robinson, Roberto Clemente, Al Kaline, Reggie Jackson, Larry Walker … to me that makes total sense.

A: I agree, and thank you for recognizing that. I will fall short of saying this is the one answer to any player ranking that you want to come up with, because there are a lot of subjective decisions to make if you want an all-time ranking. But if you want to talk about fitness for the Hall of Fame, I think it's more than up to the task.

Q: What are the factors built in here that say this guy is a Hall of Famer? Which numbers are the most important and weigh the most heavily?

A: It's Wins Above Replacement, which accounts for historical changes in scoring levels over time and adjusts for position, so that it's fairly supple when it comes to answering questions like, "Was this guy in the '30s worth more than this guy in the '60s or the '90s?" It answers the questions if historical guys, like a [Sandy] Koufax or Pedro Martinez are up to the task even if they didn't win 300 games. I think it's ideal for figuring out the tradeoffs. That said, when I do my evaluations and I see a guy above on peak but below on career standards, I think we can take those guys into consideration. For the most part, you're only talking about guys playing out the string at a less than satisfactory level. I think this answers the questions whether this guy in his prime was as good as someone else. …

When it comes to the steroid era, this is a valuable tool. There are many ways you can apply standards of who is worth a vote and who isn't. I think you should still start with the numbers. To me, you can justify not voting for Sammy Sosa or Rafael Palmeiro simply looking at the JAWS numbers and putting into context how a guy got to 500 or 600 home runs. Mark McGwire is below the JAWS standard by a couple of points in both career and peak. Palmeiro, above on career and below on peak. If you are someone who sticks to a value of peak over career or the other way around, I think you have your answer here or get a good answer here. If you want to say Fred McGriff's 493 home runs are worth more than Rafael Palmeiro's 569, well, I've got a response to that based on the numbers here. We've adjusted for the era, we've adjusted for these things, and they don't come out that way. There are a lot of things, and I believe I came up with a tool that I believe is fairly elegant. I may be surprised with myself that it's so versatile in that regard. I like when we can stick to numbers and not get into the shouting matches over subjective things. My hope is to cut the noise level a little bit.

Q: It seems like 10 years ago when you came up with this, the system could be used as an answer. But this vote, as much as any, the questions are different. They're not about the numbers.

A: Absolutely. It's not that the voters haven't had to face those questions with Palmeiro and McGwire, but this one is going to be further away from the numbers when you talk about guys like Bonds and Clemens. Any argument against them rests on something entirely different than the numbers.

Q: So where do you stand on the big three of Clemens, Bonds and Sosa?

A: I think Clemens and Bonds should absolutely be in. For me, with Clemens the fact that the substance of the Mitchell Report didn't stand up in court raises questions about applying guilt to any of those guys that were named in there without other corroborating evidence. And you can say more or less the same with Bonds. Even then, if you forget that at the moment, I think you have to distinguish between infractions that occurred before the game came up with a clear steroid policy that included punishment versus the wild west days of the '90s and early 2000s. I think those guys should both be in. Sosa, I'm satisfied that with the idea, simply based on JAWS, he's a bit short, despite the home runs and MVP awards and that four-year stretch there. I'm satisfied with that answer.

Q: In the end, is the Hall less because Andre Dawson or Jim Rice or whoever you want to name, is in there.

A: No. I'm comfortable with the Dawsons and Rices. I've written about them and argued against those guys. But I think to me, the Hall of Fame, as an institution, while sometimes flawed, does a pretty good job of capturing baseball history. I want to see it be more rounded out and the right guys are included. I don't want to see anyone kicked out. Let's just make sure the guys who deserve get in.

For more baseball news, rumors and analysis, follow @EyeOnBaseball on Twitter, subscribe to the RSS feed and "like" us on Facebook.

Our Latest Stories