Big data: Giving people what they want

Tweet about this on Twitter42Share on Facebook9Share on Google+9Share on LinkedIn2Email this to someone

Netflix is using data to create original content for its subscribers, the first example of which was House of Cards. Three main data points for this show were that (1) People like David Fincher (because they watch The Social Network, like, all the time); (2) People like Kevin Spacey; and (3) People liked the British version of House of Cards. Netflix obviously has tons of other data, including when you stop, pause, rewind certain scenes in a movie or TV show.

Netflix has always used data to decide which shows to license, and now that expertise is extended to the first-run. And there was not one trailer for “House of Cards,” there were many. Fans of Mr. Spacey saw trailers featuring him, women watching “Thelma and Louise” saw trailers featuring the show’s female characters and serious film buffs saw trailers that reflected Mr. Fincher’s touch.

Using data to program television content is about as new as Bryl Cream, but Netflix has the Big Data and has direct interaction with its viewers (so does Amazon Prime, which apparently is also looking to create original content). So the question is, does it work? My personal opinion is that it's probably not any worse than previous methods, but may not be a lot better. But I would be delighted to be proven wrong. From my walks around the hallway here it seems House of Cards is in fact a good show (I haven't seen it). But one observation probably isn't enough to draw a conclusion here.

John Landgraf of FX Networks thinks Big Data won't help:

“Data can only tell you what people have liked before, not what they don’t know they are going to like in the future,” he said. “A good high-end programmer’s job is to find the white spaces in our collective psyche that aren’t filled by an existing television show,” adding, those choices were made “in a black box that data can never penetrate.”

I was a bit confused when I read this but the use of the word "programmer" here I'm pretty sure is in reference to television programmer. This quote is reminiscent of Steve Jobs' line about how it's not he consumer's job to know what he/she wants. It also reminds me of financial markets where all the data it the world can only tell you about the past.

In the end, can any of it help you predict the future? Or do some people just get lucky?


  • Hadi

    I think this is emblematic of the difference between predictive analytics without deep models and prediction of the more scientific variety. In science, a theory that truly works should be able to predict outside the data used to generate it. In statistical modeling, we could draw the analogy with being able to predict well outside the data range (not only passing cross-validation from similar data, but even doing well outside).

    I think it is wrong to assume that Netflix couldn't do this, though they may well fail. In any case, it will likely require domain knowledge and human insight (though I'm no machine learning expert) to break down movies into characteristics that are indicative 'behind the scenes'. Just as a theory of physics could predict particles that are unseen and a theory of economics can (or really, should be able to...) predict responses to various actions, so should a theory of movies predict response to various combinations of characteristics.