An Arms Race of Model Building: How Analytics and Variance Are Changing Sports Analysis


Nate Silver has changed the way people look at sports and politics (Ian Hill/Thinkstock/Penguin)

 

Nate Silver has changed the way people look at sports and politics (Ian Hill/Thinkstock/Penguin)
Nate Silver has changed the way people look at sports and politics (Ian Hill/Thinkstock/Penguin)

In the first installment of this series, Joshua Meredith dove head first into the technological background behind Daily Fantasy Sports.

In Part two of this series Meredith looked at the intersection of Internet Poker, Gambling and Daily Fantasy Gaming.

Today in Part 3 of this series, Meredith looks at the links between Daily Fantasy Sports, Wall Street Analytics and the impact data analysis is having on the industry.

The Coders on Wall Street, Quantifying the Algorithm, Variance: Friend and Foe, The Coders on Wall Street

Somewhere in America at this exact moment, a computer server houses data, a code, that tries to predict the future of stocks on the NASDAQ. It doesn’t matter where you are or what the time is, it is happening. The code is constantly tweaked, in hopes of perfecting its results. It will never succeed in being 100% accurate, but it will be constantly tweaked until the largest brokerage agencies on Wall Street feel their predictions are the closest they can be to perfect. Then these men and women of finance will hope that the “human element,” never rears its ugly head.

In sports, the “human element” is on display almost every day of the year. But still the modeling that takes place on Wall Street and inside the head of Daryl Morey is now being used in Daily Fantasy Sports. The “stat heads” are in the game, and they are thriving.

The Sloan Conference has become the pre-eminent sports conference that covers the latest innovations in sports analytics. (Dashiell Bennett/Business Insider)
The Sloan Conference has become the pre-eminent sports conference that covers the latest innovations in sports analytics. (Dashiell Bennett/Business Insider)

If you have ever attended the Sloan Conference on Analytics or the annual SABR meetings, it is apparent that statistical analysis in sports is occurring both on a micro and macro level. Let’s say you have no knowledge of this math world, behold the great equalizer: Nate Silver. America’s most omnipresent statistician is taking his work at the New York Times, FiveThirtyEight.com blog, to ESPN. For every non-sports junkie, who just knows Silver as the math wiz from the last election, people don’t know that he created Player Empirical Comparison and Optimization Test Algorithm a.k.a. PECOTA, which determines how baseball players will actually perform.

A lesser-known fact is that Silver left his economic consultant career at KPMG, to play Internet poker full-time. Silver, used his statistical background and sound critical reasoning to enter the world of professional gambling. Daily fantasy sports is now courting Silver’s disciples under the theory that the same skills used to predict stocks and baseball player’s performance over the course of the season can be boiled down to one-day.

Joe Peta, renowned gambling expert and author of “Trading Bases: A Story About Wall Street, Gambling, and Baseball (Not Necessarily in That Order),” believes that critical reasoning and professionals in finance can be drawn into daily fantasy sports just like Internet poker was.

“I think the one thing the poker boom proved 10-15 years ago was that there is a huge appetite among intelligent individuals in the millennial generation…for making money through critical reasoning…that’s why people on Wall Street should be drawn to daily fantasy baseball and sports gambling.”

Peta added, “So it doesn’t surprise me, Wall Street doesn’t have a product they use to simply make money through critical reasoning.”

“This is something I tell Las Vegas executives: to my view sports gambling has evolved over the last 50-years from sin, and by sin I mean it was illegal; to vice, it was legal but it was still looked down upon, and then it went to entertainment which is where we are today. It is definitely looked upon as an acceptable form of entertainment. Where if you may have been looked down upon or you may have had to hide the fact that you bet on a football game 30 years ago, now my mother understands points spreads now. I think the next evolution is an investment vehicle, the way you get there is because of modeling and analytics.”

Mark Nerenberg, Chief Product Officer at DraftStreet.com, believes that courting Wall Street players who create complex mathematical algorithms and others with analytical backgrounds is a sound strategy.

“There are people doing it (modeling). I know for a fact that some of the very successful players thus far who do have a computer algorithm that they are using as a starting point,” Nerenberg added.

The existence of computer and mathematical modeling is a huge component of the daily fantasy sports rise in prominence. The next evolution of this industry is how do the mathematicians and causal player interact in the game. Is it possible the two sides are closer than we think?

Quantifying the Algorithm

What do the casual players look at it when selecting their players? This becomes one of the interesting questions related to the daily fantasy gaming industry. Since all players are dealing with a salary cap (in standard leagues), some randomness occurs in the selection of players but does this create an edge for the modelers? Unfortunately the statistics do not exist to quantify who is wining more, but what is evident is that the models and casual players on the surface are looking at the same metrics.

S.J. Laird, who considers himself a casual player (username: trapallday) has not invested in computer modeling but reviews a certain criteria before selecting his players. He credits looking at pitchers starting in their home ballparks, looking at favorable match-ups and historical pitcher-hitter data when it comes to baseball. Additionally, Laird goes outside the “traditional” stats to determine who he is going to select.

“I also like to look at hitters who are on fire! I tend to see for example, Alfonso Soriano right before the All-Star break, [he] had six home runs in four games, and if you pick up on the trend half way through, and you plug it into your lineup, even though he might not have favorable splits, that is something you want to look for as well.”

The New York Yankees hope Alfonso Soriano follows his career history and has a strong second half of the season. (Jim McIsaac/Getty Images North America)
The New York Yankees hope Alfonso Soriano follows his career history and has a strong second half of the season. (Jim McIsaac/Getty Images North America)

What is more interesting, is daily fantasy baseball turns its users into amateur meteorologists.

“Something else I look at is weather…it plays such a big factor because on the sites I play at, the rosters lock at the first game,” said Laird. “You can’t take a chance that a game is going to be rained out because then you won’t get any points for your hitters or pitcher.”

Additionally, Laird said that he gets wind reports, especially for Wrigley Field, where if the wind is blowing out on the North side, he is more inclined to take a Cubbie or one of their opponents.

The industry is very aware of some of the models that exist but even they see the issues with modeling.

“I think its tough to have a computer factor in every single variable, especially with injuries and things like that. You are essentially getting projections for one-day instead over the course of multiple games,” Nerenberg stated. “The match-ups are much more important in the formula if you are trying to project for one-game, baseball especially, lefty-lefty splits, some batters are better against sinkerballs, some hitters can hit the best fastballs but not curveballs, the stadium, the weather, all these things would be factors for one game, but not nearly as important if you are making predictions over the course of the season.”

What the industry faces on a daily level is altering their modeling for player’s salaries. Nerenberg admits that he tinkers with their formula in order to provide DraftStreet users the best possible product that he can. Speaking to his own customers, he summarized his thoughts on modeling: “Is it possible to be 100% accurate, no. It is possible to be more accurate than other people, yes! So that is the goal, to be more accurate than everybody else. The first goal is to beat the salaries.”

Laird also stated that any and all modeling must be supplemented: “If you’re not looking into the details like I am, than your not going to be successful over the long term.”

The next level for daily fantasy baseball is the incorporation of advanced metrics such as Expected Fielding Independent Pitching (xFIP) for pitching, calculating teams with players who have lower Ultimate Zone Ratings (UZR), and batting predictions based on pitch f/x.

With catches like this one, Jason Heuward of the Atlanta Braves had the top Ultimate Zone Rating (UZR) of the 2012 MLB Season. (Scott Cunningham/Getty Images North America)
With catches like this one, Jason Heyward of the Atlanta Braves had the top Ultimate Zone Rating (UZR) of the 2012 MLB Season. (Scott Cunningham/Getty Images North America)

As the sport grows and attracts players on both sides of the spectrum, it will be interesting to see how both types of users react. Laird understands the plight he is in, when dealing with competitors who are using sophisticated computer modeling.

“I wish I had as much time as they do to put in all that work. Obviously it’s the type of situation where it’s an up and coming industry and I am not really surprised that someone is taking the time to have all of these models and try to beat the system.”

Whether or not this fight becomes a full-fledged war as portrayed in Michael Lewis’s Moneyball or just a healthy rivalry is something the industry is monitoring closely, as both groups are needed to grow the game and revenue.

Variance: Friend and Foe?

Over the course of any fantasy season, one can survive the occasional bad bounce, missed call, or a player’s stint on the Disabled List. In daily fantasy sports, those events change who wins and who loses, and can move a significant amount of money in the process. In mathematics and analytics, random events are called: variance. These random events can affect pretty much every element of daily fantasy sports, no matter the sport. Baseball might be the toughest to predict to the every changing match-ups.

“There is a huge amount of variance, the smaller sample size the more variance there is going to be,” Peta explained. “Trying to project what a player, let alone nine hitters and pitchers, is going to do on a nightly basis is going to cause huge variance.”

The industry is dealing with a significant issue when we look at traditional variance. The first is that it affects the computer modelers, because they are playing a law of averages style, which may allow their non-analytical competition to benefit from things they have discerned from the ‘eye test.’ The industry tries to chalk this up to normal elements that might occur on Wall Street.

“It is variance, its calculated risk, anything can happen,” Nerenberg stated. “Some sort of fluke thing, you have to know that if the game was played out a thousand times, it would have only occurred in one or two of the thousand times. It is essentially a bad beat. Luck is variance in my mind, and there are things that can happen that are not probable, they were unlikely to occur in the first place.”

Nerenberg says that he feels that variance is not ruining his customers who have significantly better systems:

“If your going in as the favorite everyday you’re going to end up ahead, no doubt about it and it’s been proven. We have users who are playing day-after-day, in thousands and thousands of leagues that are ahead, and that wouldn’t happen if there was luck. Some of the best guys are dominating, to a point where no one wants to play them, so there is no doubt over the long run that this is a skill game, and just like anything else, flukes can happen and over one night anything can happen, but over time it is skill.”

This is one of the quintessential questions regarding daily fantasy sports, is it luck or skill? The old sports cliché, ‘on any given day’ the worst team can beat the best team, seems to apply in this version of sport as well. While that may not matter in a $10 buy-in match, it will matter when the stakes get higher. As with Internet poker or sports gambling, the element of a fluke exists and it’s hard for anyone to quantify it. As Nerenberg stated, he believes that skill is the determining factor in being successful daily fantasy sports. But there is another way to remove variance from the game, something that was often used by poker players and Vegas gamblers.

“Playing multiple games on multiple sites can decrease your volatility, because you will have different opponents…so I can see people who use the term ‘multi-table playing,’ which is possible in online poker happening for fantasy sports,” Peta advised.

Peta agreed with Nerenberg that the modelers and analytics community would have an edge dealing with variance:

“The nice thing is that you have 162 days a year to repeat the process, but the variance should not affect a person who is highly analytical or a model builder but effect the amount of capital they are putting down, not their calculated edge.”

Over time this becomes one of the greatest issues that the daily fantasy sports industry is going to deal with; will their customers stay on for the long-term even when things don’t break their way? Could the product be diluted by the casual fans who are just getting lucky and will modeling truly be a better system as more users begin using similar math? All these are issues that this multi-million dollar industry is going to have to solve or at least work around as they continue to grow.

Postscript

Sometimes when diving into an industry, every road we turn down just leads to even more scenarios and questions then when started with. This was truly the case in SportTechie’s investigation and profile of the daily fantasy sports industry. From its cushy position in the eyes of the law, to its customers, to its relation to other forms of gambling, it is a topic in which every branch leads to a series of new branches. One of the most interesting facts is that even for an industry with a laundry list of issues, it is incredibly profitable. So profitable that even the financiers in our countries most profitable brokerage and investment houses are taking notice. The largest companies in the fantasy world are even starting to poke around the idea of joining the marketplace.

Finally, maybe this is the time a game can be wagered on without significant heartbreak or controversy. The industry, which is growing by leaps and bounds can perhaps focus on fusing the worlds of Nate Silver and Warren Buffet. But what will measure their ultimate success is if they can keep the United States Department of Justice out of their backyard and keeping the Tony Soprano’s of the world from intersecting with their versions of Silver and Buffet. An interesting way we measure success when the “g” word is involved, isn’t it?