Predictive Analytics Unlocks Big Data

Predictive Analytics is the "Open Sesame" for the world of Big Data. It's the predictive technology that enables computers to learn how to predict the future behavior of individuals. In business, this ability to predict—which is based on surfacing patterns found in data—helps businesses make informed decisions and identify risks and opportunities.

It's the science that unleashes the power of Big Data. And the results affect everyone.

But it can seem inscrutable. Eric Siegel, a former Columbia University professor and founder of Predictive Analytics World lifts the veil on this often-arcane world in his new book "Predictive Analytics: The Power to Predict Who Will Click, Buy Lie or Die" (Wiley, 2013). In this primer he offers 147 examples of how predictive analytics is applied in various aspects of life and business, ranging from why early retirement decreases life expectancy to how companies ascertain untold, private truths—how Target figures out you're pregnant and how Hewlett-Packard deduces you're about to quit your job.

Siegel recently shared his thoughts on how this new technology with affect the way we live and work, and some cautionary advice on how we keep the genie from running amok.

BusinessNewsDaily: What is Predictive Analytics?

Eric Siegel: The shortest definition is my book's subtitle: The power to predict who will click, buy, lie, or die. Predictive analytics is the technology that learns from data to make predictions about what each individual will do—from thriving and donating to stealing and crashing your car. By doing so, organizations boost the success of marketing, auditing, law-enforcing, medically treating, educating, and even running a political campaign for president.

BND: What are the goals of Predictive Analytics?

E.S.: Prediction is the key to driving improved decisions, guiding millions of per-person actions. For healthcare, this saves lives. For law enforcement, it fights crime. For business, it decreases risk, lowers cost, improves customer service, and decreases unwanted postal mail and spam. It was a contributing factor to the reelection of the U.S. president.

BND: When did Predictive Analytics first become realistic? Was there any tipping point?

E.S.: With the underlying technology firmly established in the research lab, the major challenge to deploying predictive analytics was a kind of requisite culture shift. Beyond the technical endeavor of building a predictive model from data, the per-individual predictions it then generates must then be used by the organization, acted upon in order to drive operational activities. Integrating predictive analytics in this way and thereby changing (and improving) "business as usual" entails an organization change that doesn't happen with the snap of your fingers.

Although we've just reached a tipping point as far as critical mass of widespread use and general awareness, until now it kind of crept up on the world. There were niches were it became common, such as targeting massive direct mail marketing campaigns, predicting which cellphone customers are at risk of leaving to another wireless carrier, and determining the risk of a credit card applicant. These have been firmly in place for at least a couple decades. The broader use for marketing, fraud detection, customer cancellation in other businesses, online ad targeting, and much more has then grown organically from that base of success.

BND: How crucial was it for Predictive Analytics to develop tools and methodologies that deal with unstructured data such as text and other subjective material?

E.S.: In some projects, unstructured data is critical to predictive precision. For example, for some organizations, processing the customer service agents' typed notes are central to detecting customers more at risk for cancellation. In other cases, no pertinent unstructured data is available at all.

BND: What differentiates this from data mining and business intelligence?

E.S.: Predictive analytics fits squarely within the broad "data-driven" arena referred by terms like big data, data mining, business intelligence, and analytics (without the "predictive"). The excitement around how much data there is and its potential begs the question, what should we do with it, what's the specific value? The answer to this question is, learn from it how to predict. The thing that makes a direct difference for how organizations operate is prediction.

BND: Does Predictive Analytics deal primarily with correlation or with causation?

E.S.: Correlation. Causation is an elusive thing to establish, and you don't necessarily need it in order to predict well. If we see the correlation that early retirees have higher health risks, we'd like to know why – but  we don't actually need to know why in order to make use of that information. Instead, early retirement becomes one factor to consider when determining whether to prioritize a patient for additional screening or other prevention-oriented activities.

BND: Is Predictive Analytics something that can be implemented by small firms as well as large?

E.S.: Yes, and it often is. As long as there is a long enough customer list from which to learn, there's potential. For example, many small companies conduct direct mail (or online activities) across large numbers of customers.

BND: I have a small, consumer-facing company with several databases of customer information, competitive intelligence, etc. Where do I begin?

E.S.: The first thing to determine is what customer behavior to predict and how the predictions will provide value, i.e., what operations will be tweaked with the per-individual predictions. For example, predict which customer will purchase if mailed a brochure in order to decide who is worth investing the $2 to send the brochure to.

BND: Why are we so dataphobic?

E.S.: I think we're becoming much less dataphobic extremely quickly at this time. People who've never felt safe or comfortable with math may initially shy away from quantitatively-oriented concepts and assume they're arcane and difficult to understand. But the idea of deciding "yes versus no" for each individual as to whether to mail, approve, investigate, incarcerate, or set-up-on-a-date -- based on a predicted behavior for the individuals -- is not so elusive, as folks quickly discover. And the basic idea of how to form a prediction for individual based on all the factors known also turns out to be easy for anyone to grasp, even without getting into the math.

BND: You write that data is the world's most booming unnatural resource. Please explain.

E.S.: That's me being cute and humorous. Data is certainly a booming resource. "Unnatural resource" is a play on the well-known phrase "natural resource"—because, after all, the information on a disk drive (or millions of disk drives, for that matter!) would probably be considered artificial rather than part of nature. Hmm, not so funny when you have to explain it.

BND: Will there be something like Moore's Law that describes the growth of Predictive Analytics?

E.S.: Predictive analytics will continue to grow quickly like any emerging best practice that's not only a win to employ, but a competitive necessity. Moore's Law comes in because it tells us how quickly data will continue to grow, and the more data from which to learn, the better you can predict and the more types of behavior that can be predicted.

BND: You're fond of quoting from "Spiderman"—" With  great power comes great responsibility." What do you mean?

E.S.: With the advent of predictive analytics, organizations gain power by predicting potent yet—in some cases—sensitive  insights about individuals. The fact is, predictive technology reveals a future often considered private. These predictions are derived from existing data, almost as if creating new information out of thin air. Examples include Hewlett-Packard inferring an employee's intent to resign, retailer Target deducing a customer's pregnancy, and law enforcement in Oregon and Pennsylvania foretelling a convict's future repeat offense.

BND: Is there a dark side to Predictive Analytics? How can we control it?

E.S.: As with any marketing, law enforcement, or other activities, the needs and rights of the individual must become part of the equation. With any activities that operate en masse across many people, there is always the risk to lose site of the individuals. It is critical to increase public understanding of what predictive analytics is, how it is being used, and a sense of how it works in order to inform discussions, debates, and legislative activities.

BND: The algorithms of Predictive Analytics are getting increasingly better at figuring out what we like. Will this kill creativity and serendipity? Could Predictive Analytics ever produce an iPod?

E.S.: I strongly believe this powerful tool helps the world and elevates human activity. Predictive analytics helps tweak existing operations—it is a paradigm shift but it does not create new paradigm shifts like the iPod. Running things more intelligently and rendering operations more effective and efficient (e.g., decreasing junk mail and spam) only opens up additional resources and opportunities that in turn foster continued human creativity. There's nothing there to disincentivize human creativity, and I don't see entrepreneurs and scientists planning to slow down any time soon.

Reach BusinessNewsDaily senior writer Ned Smith at nsmith@techmedianetwork.com. Follow him on Twitter @nedbsmith.Follow us @BNDarticles, Facebook or Google +. Follow us @BNDarticles, Facebook or Google +.