Teradata Magazine | Special Focus

Getting the most from your active data warehouse requires new approaches to your business.

A day in the life of active data warehousing

7:00 a.m.
En route to the office, Joyce Mason, vice president of operations at a telecommunications provider receives a notice on her handheld device that a potential service interruption requires additional analysis as to the possible impact on customers.

7:32 a.m.
Joyce arrives at her office and logs onto her computer. An auto-alert message from the company's active data warehouse reveals that the highway construction project on Route 53 has severed an underground cable.

7:33 a.m.
Her handheld shows a message from George in operations, saying that the active data warehouse has automatically rerouted the network and customers are receiving uninterrupted service. Funny, she thinks, events like this used to be so traumatic, what with potential customer loss, the cost of retaining those customers, not to mention the deluge of complaints at the call center. Now, the active data warehouse ensures smooth, uninterrupted operation.

10:30 a.m.
Joyce pings engineering on a high-priority project they need to complete by week's end. They're waiting for a package of special parts, they say. Logging into the new shipper's Web site, Joyce traces the location of the package, courtesy of the shipper's active data warehouse, and requests a status notice to be sent to engineering with the package's estimated time of arrival.

10:40 a.m.
It's her son's birthday, and gift and celebration arrangements still need to be made. Joyce goes online and orders the new game her son has been talking about. In the order, she requests that it be ready for store pickup over her lunch hour, noting that it's a birthday present.

Noon
As Joyce is getting out of the car at the store, she sees a package that needs to be returned, but she's lost the receipt. She'll take it to the service desk anyway, she decides, and take her chances. The store's active data warehouse not only allows the clerk to use Joyce's credit card to look up the transaction and successfully complete the return, but the game she'd ordered for pickup is gift-wrapped with a helium balloon attached to it.

12:50 p.m.
Joyce is back in her office, reflecting on how peaceful and successful her day has been. Multiple crises have been averted, she's prepared for her son's special day, her business is running smoothly and all the people within the business that she interacts with are happy business partners.

It's barely afternoon and Joyce's entire day has been improved by active data warehousing.

The moment of truth describes the instant at which a company either delivers—or fails to deliver—the product or service that’s expected. It may be an in-house logistics company carrying items for the parent corporation or a retailer managing inventory so that it remains fully stocked at all times. The important thing is that at the moment of truth, those needs are satisfied.

Leading companies know that such moments occur daily, hourly and even from minute to minute. That’s why they rely on timely information to drive key operational decisions.

Typically, operational decisions are more confined in scope than the strategic decisions normally associated with an enterprise data warehouse (EDW). When operated as an active data warehouse, however, the same EDW can support today’s new breed of operational analytical application.

As a single, integrated environment, the active data warehouse not only provides strategic decision support but also simultaneously pushes up-to-the-minute business intelligence (BI) to the front-line service representatives or applications that ultimately satisfy the client. This platform can serve an unlimited number of users with a mixed workload of operational queries and more strategic BI queries. It can support near real-time data loads, triggered events, analyses and actions that allow it to close the loop back to operational systems, eliminating the need for human intervention in some cases and suggesting actions in others.

An active data warehouse interfaces with a rules-driven engine linked to BI applications that can monitor, detect and take action the moment a business event is triggered. The system doesn’t just operate in real time—or in sufficient time to be able to answer the business question when it is asked (i.e., right time). Rather, it uses both current and historical data to put the focus on the corporate goals and facilitate better front-line decision making.

More powerful tools are only better if you know how to make them work for you, of course. “Companies are assembling huge amounts of information,” says Dr. Richard Hackathorn, president of Bolder Technology (BTI). “Some of it is relevant to the company; much of it isn’t. But you can’t put blinders on and simply ignore the vastness of this information. Show me a company where peripheral vision is not of business value. Huge stores of information constitute the environment of any global corporation in the world today.”

Typically, the approach within enterprises has been for each department to store massive amounts of data generated from day-to-day activities in its own separate data silo or data mart. Unfortunately, those silos do not interface with one another; moreover, the volume of data stored within them can be overwhelming and difficult to capture for effective operational decision making. An EDW solves that problem by integrating data and analysis capabilities across the enterprise; using the EDW to perform active data warehousing dramatically drops data latency.

Hackathorn likens the shift from static data to streaming data to getting water from a high-pressure hose. “Imagine trying to capture water coming from a fire hose with buckets,” he says. “You’d get a little water, but most of it would spill on the ground.” Such pointless activity benefits no one. “You need to be able to capture and look at streaming data on the fly, to detect patterns and pick out the most valuable pieces. You can’t keep blindly filling up buckets.”

Companies have begun to realize they have a store of information that can be used to enhance their performance and increase their profitability—provided they have the tools to do it. Active data warehousing can be one of those tools because it is about sharing corporate-level insights with front-line staffers to make a difference in how they do business, whether that means improving vendor relations, logistics and inventory management, or serving customers.

Highmark, one of the largest health insurance companies in the United States, needed to better understand its cost drivers that would allow it to hold the line on premium increases, thereby retaining customers, enhancing its competitive position and protecting its financial strength (see “Best practices in ROI” ). In response, Highmark created an active EDW capable of capturing claim information within one day of an event such as an office visit or hospital discharge; for comparison, the data warehouses of most health insurers suffer a time lag of three to six weeks.

Highmark’s system supplies it with near real-time analysis of business, clinical and operational opportunities, automated event/trigger-based intelligent applications and more actionable member interventions. Active data warehousing has allowed Highmark to better serve customers, enhancing its profitability.

Timing is everything
An active data warehouse reduces the latency between a business event and the time at which the resultant data is available (data latency; see “Active data warehouse: from nice to necessary”). The real value appears when that data latency corresponds to the desired timeframe, reducing overall action latency and enabling front-line users to influence the business process. Analyses performed by an EDW can help decision makers change marketing plans, track promotions and plan inventory levels. Active data warehousing pushes the capabilities of the EDW one step further, providing not only the ability to ask new questions and analyze cross-enterprise data but also to make decisions for the moment.

It’s not only about low data latency, however, experts caution. “A data warehouse may be refreshing in hour spans, but that doesn’t mean it’s an active data warehouse,” says Daniel Linstedt, CTO for Myers-Holum. “The active data warehouse is much more attuned to the physical occurrences of transaction changes.” In other words, it doesn’t just need to know, it needs to do. An active data warehouse can simultaneously be both a right-time and an archived environment—the business decides how often data needs to be refreshed. “You could align the data refresh rate to business requirements, but only if the business require-ment says it needs data every X number of minutes,” adds Linstedt.

Indeed, it all comes down to business requirements, agrees Claudia Imhoff, president and founder of Intelligent Solutions. “The further away from the event you get, the older the data itself gets,” she says. “It doesn’t mean you can’t make a good decision with data that’s not synchronous with the operational systems; the fact is, you absolutely can.” What’s important is that you leverage your active data warehouse as rapidly as you need to—not necessarily as rapidly as you are able.

“What people need to do is figure out where the data fits along a time continuum. For a particular application, does it really have to be real time, or can it be a few seconds or several minutes behind the actual event occurring? Can it even be a day, a week or a month? It depends on what you’re using the data for.” In some cases, an active data warehouse provides immediate data to facilitate a decision and response within seconds, such as in rerouting packages or airline passengers. In other cases, the analysis and decision strategy occurs offline, but appears to be real time as implemented to the party who benefits.

The cost of providing timely data also needs to be weighed against the resources and requirements of the business. Linstedt says the way to determine the value of right-time data is to back into it. “You have to answer the question: ‘What’s the cost of not having this data at the right time?’ Then figure out the single most important question you can answer with that data. What value does answering this question have for the company at this point?”

Making the connections
Instances at which moments of truth transform to moments of opportunity abound. Consider a recently graduated student who places $10,000 in her bank account. Based on her account history, the financial institution’s data warehouse flags it as an unusually large deposit, triggering a notice to the branch sales manager that identifies the graduate and provides relevant financial data. Because the graduate is a potentially profitable customer over the long term, the sales manager follows up by phone, as her profile indicates she prefers.

In their conversation, the graduate explains that she’s sold her old car and has her heart set on a new model that costs more than she’d originally intended to spend. After talking with her about her needs, the sales manager offers her a low-interest loan, one that the bank is currently promoting to help increase sales. The loan will allow her to buy her dream car and still hold onto some of the proceeds from the sale of her old vehicle. It’s the perfect solution for the graduate, who needs transportation in order to commute to her new job. The terms of the loan also allow her to save some money toward a down payment on her first home purchase. Not only did the bank assist the student with the current solution, but they also achieved their strategic goal of selling more loans and laid the groundwork for a long-term relationship with the customer.

In a different sector, a shipping company can use active data warehousing to reroute packages in transit. Operational controls can enable managers to address operational foul-ups, and radio frequency identification (RFID) technology interfacing with the data warehouse can help pinpoint package location on a minute-by-minute basis. When a truck en route to a shipping hub breaks down, for example, the company can check the list of packages on board. If the cargo includes packages belonging to priority customers, the company may elect to hold outbound trucks at the shipping center until those packages are transferred over.

Active data warehousing delivers enterprise intelligence which can impact behind-the-scenes operational decisions, as well. With access to critical information, a manufacturer or supplier may be able to rapidly manage production environments—and cut costs, says Linstedt. “The company can ask new questions: ‘Are my machines producing faulty parts?’ ‘Are my human laborers making mistakes?’ ‘Can we optimize this process in any way?’ Active data warehousing enables managers to get the right information off of their machines or work-stations as soon and as frequently as needed to optimize the process within their organization.”

The blending of strategic data with operational data can also help improve fraud detection. Some financial services firms use activity profiles to track credit card usage at automated points-of-sale (POS) such as gas stations, because thieves often use these touchpoints to determine whether a stolen card is valid before attempting larger transactions. Many legitimate customers make both gas station purchases and large purchases in a single day, however, diluting the effectiveness of such profiling.

Active data warehousing can help. Data shows that most people only frequent five or fewer gas stations, so sudden usage at an unusual location, coupled with abnormal buying patterns—purchases of jewelry or home electronics, say—could flag an anomaly worthy of further investigation. A system that correlates historical data with up-to-the-minute transactions offers a better opportunity for sifting through otherwise legitimate transactions to detect suspicious behavior.

Getting active
Capturing and leveraging timely information has become a key advantage for companies. “The driver is competitive pressure,” says Linstedt. “The ability to make decisions faster can increase profitability. Certain large retail chains have changed the competitive landscape in their marketplace.”

Of course, any effective data warehousing initiative requires a team effort across the organization. As companies move deeper into active data warehousing, the relationships between data sources become much more critical, as does their consistency. The risk, says Imhoff, is that without full buy-in from the organization, there is the potential of merely making bad decisions faster. “The biggest road-block has nothing to do with the technology,” says Imhoff. “It has more to do with whether the organization is ready to handle active data warehousing. You can speed up decision making but this doesn’t address your requirements if the people using the information aren’t ready, and if they don’t know what to do with the data or don’t have processes in place that allow them to use the data in a more right-time environment.”

The entire organization needs to be focused around the goal of achieving enterprise intelligence. “Cultural change is huge,” says Hackathorn. “You’ve got thousands of people and hundreds of managers in many different companies trying to run a single business, and everyone has different perspectives on where they need to go as a company. The company needs to say, ‘Here’s who we are, here’s the way we’re going to keep track of customers. Here’s when we make a sale and don’t make a sale.’” Active data warehousing helps expedite those operational decisions and keeps them in line with corporate goals and strategies, serving customers—whether the traditional consumer or other—in the best way possible.

Emerging technologies and methodologies also promise to help enterprises leverage the benefits of active data warehousing; for example, more companies are adopting service-oriented architecture (SOA) approaches in which data and application components are presented as reusable services that can be adapted to changing business requirements.

Luck, the saying goes, is where preparation meets opportunity. Active data warehousing allows companies to make enormous improvements to processes and the ability to serve the client, all the while remaining consistent with overarching corporate strategy. As with any tool, the platform requires understanding and proper use to achieve maximum benefit. With the right approach and the right company philosophy, active data warehousing can convert the moment of truth into the moment of success. T

Joe McKendrick has written for journals such as Database Trends & Applications, Enterprise Systems and others.

Active enterprises get intelligent

In a world in which businesses generate data at a stupefying rate, we’re faced with one inescapable truth: Data itself does not have intrinsic value—it‘s the intelligence you derive from it that makes it matter. The whole is greater than the sum of the parts. Turning data into intelligence is what data warehousing is all about. Turning data into active enterprise intelligence is what powers a company to success.

Strategic intelligence leverages historical data to create information for managing the business more effectively, like tracking compliance or analyzing customer and product profitability. Operational intelligence involves making decisions at critical points of contact, such as during a customer interaction.

A traditional enterprise data warehouse (EDW) analyzes data to extract strategic intelligence, producing reports and analysis to support corporate decision makers and back-office users. This big-picture view is important; in today’s business climate, however, responsiveness is even more so. Ratcheting up the speed, an EDW performing active data warehousing not only delivers strategic intelligence, but also turns near real-time data loads and analyses into operational intelligence. It is this operational intelligence that allows front-line users to make the minute-by-minute decisions that reduce cycle times to drive revenue, performance and competitive differentiation.

Combine both strategic and operational intelligence and you get active enterprise intelligence. With active enterprise intelligence, companies place the right information in the hands of decision makers throughout the organization. With active enterprise intelligence, a company has the tools to respond to customers and the market in the most efficient, effective way possible.

Enterprise intelligence is what is delivered; active data warehousing is how it’s delivered. —Kristin Lewotsky