Responding When It Matters: Creating Real-Time Unique Responses Using Analytics
To make real-time unique response a reality, you must focus on three core competencies.
By Venkat Rajan, Vice president, analytic services, Symphony Teleca Corp.
You're at a home improvement store when you get an offer for a patio furniture set on your cell phone. You think to yourself, "Interesting. This is exactly what I need! I should take a look and see if it's what I've been looking for."
How was an offer presented to you that seemed to practically read your mind? If you study the scenario shown in Figure 1 (below), you will see that no single activity indicates a need for a patio set. These activities by themselves don't provide enough insight to what kind of offer you should get, but there it is. How did it happen?
Figure 1: Patio furniture real-time promotion
Today, we have all the technology components and most of the data to make this happen. First, the home improvement retailer has data about your prior shopping activities. Through analysis of this structured, internally available, transaction data, the retailer can categorize you into the right behavior segment by evaluating characteristics such as basket size, frequency, price sensitivity, and style. It is then a matter of targeting and deciding what kind of offer to make. The retailer can stop at that point and make an offer that fits your behavior and how it wants to influence you. For example, if you are a frequent shopper who buys products in certain categories, the retailer may want you to buy more or buy a product in a related category.
That is not enough. You may not be in the market for buying products in those other categories. If the retailer can integrate unstructured data from Web logs, Facebook, Twitter, and other sources and find the relevant insight into your current behavior, the targeting can be more, well, "on target."
Additionally, by integrating location-based real-time data, the retailer can make that relevant offer to you now, while you have an expressed intent to buy, because you're currently in the home improvement store.
The challenge for many marketers has been knowing how to respond when it matters -- to create targeted, relevant, and current responses, real-time unique responses, and today's technology can make that a reality. Big data resides everywhere and tapping into that data applies to a variety of industries -- not just retail:
- Internet and social media: There is already a significant level of real-time unique response from Amazon's recommendation engine to Google's ad presentations. Much of this is driven by reacting to online actions and needs more integration of transactional and causal data.
- Finance: Financial markets have been using real-time unique response using structured data for many years, now they have the opportunity to integrate unstructured data to make it richer. Sales and marketing, compliance, and risk are other processes where real-time unique responses can have a significant impact.
- Health care: A rapidly emerging opportunity through electronic health records make it possible to contemplate real-time unique responses to patient behavior and needs.
- Media and telecom: Sales and marketing as in any B2C segment, but also operations such as surveillance, load balancing, and maintenance.
The opportunity exists in most of these B2C and B2B2C segments, but interesting applications of real-time unique response are also emerging in B2B segments, where the opportunity is greater on the operational side, to impact supply chain and maintenance activities. As M2M interaction becomes pervasive, the B2B opportunity will grow rapidly.
What kind of a technology infrastructure of data and analytics is required to make real-time unique response possible? The figure below provides an example in the retail market.
Figure 2: Real-time unique responses in a retail environment
Three Key Competencies for Real-Time Unique Response
To make real-time unique response a reality, companies need to focus on competencies that take raw data in its various forms and convert it into the targeted, relevant and current response.
Competency #1: Data Management
Given the dimensions of volume, variety and velocity, this is a big data problem: 80 - 90 percent of the data needed for an enterprise business process is outside the enterprise, on the Internet, or with partners. Most of it in is unstructured form. Big data is not just about unstructured data. There is a significant volume of unstructured data within the enterprise in call center data, e-mails, and Weblogs. More than 50 percent of the enterprise data is in a variety structured and unstructured forms and in various repositories outside of the main data warehouse.
With all this data, timely use becomes critical. Data relevance in some cases, and response in all cases, has a shelf life. Either the opportunity is completely lost or is significantly degraded as time passes. Companies need to focus on making data fluid, letting master data evolve as new attributes become available, and letting data flow quickly to analytic processes with minimal friction. This requires complementing the structured data warehouse with newer big data and semantic intelligence technologies to create integrated fluid data management capabilities.
Competency #2: Data Science Expertise
Generating insights from structured data using data mining techniques has been in use for many years in various enterprise functions. In the retail example, segmentation and targeting have used statistical models.
Data science has an even greater role when it comes to generating insights from unstructured data. We characterize this problem as "finding the golden needle in the haystack." As most of this data comes from sources that are not fully vetted, establishing confidence in any information is a critical requirement. For example, how reliable is a Twitter or Facebook posting? Can the information be validated using other data sources?
Combing through the massive data volumes and finding a valuable insight requires strong data science competency. The consulting firm McKinsey, in its 2011 big data report, said that "The United States alone faces a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts to analyze big data and make decisions based on their findings." Companies need to have a strategy for acquiring and developing this talent, or establishing partnerships with service providers to provide the required competency and capacity.
Competency #3: Analytic Processes
Analytics has at times been confused with business intelligence (BI). Although BI has enabled data consolidation and reporting, it has been mostly about users consuming data and analyzing the outcomes to understand causation using an ad hoc approach. Analytics, on the other hand, is about interacting with the data, creating and analyzing scenarios through a structured process, leveraging people and system competencies as needed.
Companies need to develop analytic processes that integrate people and technology from data ingestion to real-time, unique response generation. Offline analytic processes require data visualization, structured analysis, and data mining to establish the targeting rules. These processes utilize a combination of people and system competencies. Real-time analytic processes require complex event processing, machine learning, and mobile technologies to create and deliver relevant and current responses. These processes are completely dependent on system competencies because of the real-time response requirements. However, a closed-loop process needs to be established to monitor and tune the real-time systems.
We use the framework shown in Figure 3 as a reference model for building real-time unique response analytic processes.
Figure 3: A reference model for building real-time unique response analytic processes
Summary
Real-time unique response is currently an opportunity and will soon be a necessity for enterprises. Making this a reality will require building the right fluid data management infrastructure for structured, unstructured, and real-time streaming data, having the data science competencies to build data mining and machine-learning algorithms that can generate the right insight, and integrating people and systems into offline and real-time analytic processes.
The prize for making this happen: a highly differentiated business that delivers unparalleled financial results.
Venkat Rajan is vice president, analytic services at Symphony Teleca Corp.. You can contact the author at [email protected].