When it comes to business intelligence (BI) and data warehousing (DW), the marketing-spin factory spews out an endless supply of new names and concepts: Real-time data warehousing, near real-time data warehousing, zero-latency warehousing, and active data warehousing (ADW).

It would be easy to dismiss ADW as a marketing concept. But active data warehousing does represent an important approach: blending tactical (operational) and strategic (warehouse) data to support on-demand decision-making. Seasoned architects are familiar with the need to blend these kinds of data (and have likely heard the demand for it from user communities, which need both kinds of data to support customer relationship management and other efforts).

What exactly is an active data warehouse? Good question. I recently enjoyed a dinner with a data architect where we discussed the ADW concept. The architect's take on the matter was that any warehouse that goes live is active. That's a good point and a concise definition. But nothing's ever that simple, is it?

I'll provide several common descriptions of an ADW and clarify the original intent behind the concept. Then, I'll show you that there's more than one way to achieve the intended result.

What It Is — And Isn't

First and foremost, ADW isn't a single technology or product; it's an approach, a process, and an overall architecture that includes many technologies.

Here's a definition from Capgemini Consulting Technology Outsourcing (published in its report "Power and Simplicity, The Active Data Warehouse Evolution," available at us.capgemini.com):

"The Active Data Warehouse combines deep, detailed historical data with current operational data and events, providing the full context needed to allow sophisticated analytics and automated business rules that operate in real time to give front line employees the intelligence needed to drive improvements in customer relationships, supply chain efficiency, and financial operations."

Compare that to this description from NCR Teradata, a BI vendor:

"A key capability of an active data warehouse is to reduce the time between critical business events and the actions taken as a result of such events. It is essential that the data analysis that takes place in an active data warehouse be translated to actionable decisions to maximize the value proposition from its deployment. The underlying philosophy for implementation of an active data warehouse is to increase the speed and accuracy of business decisions. The goal is to allow decision-making to take place as near real-time as necessary to deliver on a maximized value proposition ... an active data warehouse enables whatever service levels are appropriate." (From "The Five Stages of an Active Data Warehouse evolution," by Stephen Brobst and Joe Rarey, Teradata magazine.)

And a third definition from an independent data architect:

"The ADW provides an integrated, near real-time information store to drive both strategic and tactical decision-making processes.... The active warehouse extends the traditional data warehouse architecture into the area of tactical decision-making and allows the decision makers to close the loop between the operation effect and the data warehouse analysis."

Each of these examples describes an ADW, but falls short of a definition. Within these descriptions, however, you can identify characteristics that reveal the intent of an ADW, including:

• Real-time or near real-time refresh cycles
• Blending of tactical (operational) and strategic (warehouse) data.

(An important side discussion is the concept of real-time, near real-time, and zero-latency. The terms real-time and even near real-time don't properly describe the refresh cycles needed for BI. Instead, both suggest that every piece of data is pouring into the warehouse at the same refresh cycle. For many organizations, this may not be desirable or possible. I prefer the term zero-latency, which translates to getting the data to the users when they need it — in other words, on demand.)

These common ADW characteristics reveal its fundamental intent: to blend tactical and strategic data to support on-demand decision-making.

These characteristics shouldn't be new to BI architects. The best BI environments address the needs of the following two major functional areas and their respective data (see Figure 1):

• Business operations. Decision makers focus on efficiently developing, managing, and executing business processes.
• Business strategy and planning. Decision makers focus on strategic issues such as what products to produce or what services to offer.

When it comes to BI, these two audiences have been treated as independent. Until recently, you could read volumes on how to support either, but not necessarily both.

Blending via ADW

ADW is only one option to provide this blended tactical and strategic data on demand. There are other approaches that are just as valid and compelling. A service-oriented architecture (SOA) based on XML and Web services, for example, is a valid architectural option to consider.

The architecture most often proposed for an ADW builds on a traditional, centralized data warehouse to expand the warehouse's uses (see Figure 2). This approach to creating an ADW is very similar to the Corporate Information Factory (CIF) architecture developed by Bill Inmon.

The CIF takes into account strategy and tactical data by establishing an atomic-level data warehouse and operational data store (ODS), respectively. The only difference between the CIF and an ADW is the implementation of both data stores in a single environment (as described in "Active Data Warehousing — the Ultimate Fulfillment of the Operational Data Store," by Claudia Imhoff). Inmon has even defined refresh cycles into the ODS that address most ADW applications.

A centralized, enterprisewide warehouse is easy to describe and affords tangible benefits. It also presents real challenges for many organizations. The hub-and-spoke architecture cornerstone to a centralized repository may be the hardest to achieve and scale. That's not to say it's not the right choice; however, keep in mind that vendors that try to talk you into (or out of) an ADW have a vested interest.

Blending with an SOA

Few organizations have a single, enterprise-wide data warehouse for both operational and strategic reporting. But that doesn't mean they can't achieve the same blend of data promised by ADW. SOAs provide an alternate means to the same end.

Managing and integrating multiple data sources, data structures, and tools in a modern BI environment becomes far more feasible if you look at the capabilities provided by each as a service. SOAs for application, data, and processing integration can include a collection of capabilities such as traditional transformation, data enhancement, OLAP, forecasting, and spatial visualization. And each service is implemented to source its respective data, whether operational or strategic. All of the capabilities can be invoked by BI applications in batch, as messaged events, and on demand.

SOAs afford integration at several layers, including application, data, and processing. Each layer represents a collection of capabilities that can be invoked by a batch or messaged event. The combination of these individual services forms the corporatewide BI environment. The specific technology implemented and services provided are less important than the fact they are bound together by a central metadata management function.

The industry trend is to support Web services as the technology for building a flexible framework for delivering advanced analytics, content enrichment, data quality, and visualization across the enterprise. These services can be provided at the data, process, and application levels. At the data level, Web services can be invoked to ensure data quality via standardization. Processes can invoke Web services for such things as information enhancement. And, applications can invoke one Web service to deliver tactical data, while another is called to provide strategic — all in the same application. Web services provide broad access to business processes, applications, operational data sources, OLAP cubes, atomic-level warehouse sources, ad hoc query and reporting, and so on.

As Figure 3 shows, all Web services provide an interface, accessible via XML messages, that describes their operations. Web service consumers locate the desired service by querying the Universal Description, Discovery, Integration (UDDI) registry. After locating a service, the consumer dynamically connects to it using an XML message known as a Simple Object Access Protocol (SOAP) request. The service then communicates back to the consumer using XML.

At the application level, users have the ability to invoke BI-centric Web services on-demand via applications (both tactical and strategic) as well as portlets. Using Web Service Remote Portal (WSRP), analytic applications can publish reports, OLAP cubes, data mining models, and spatial analysis directly to the portal itself. Processes such as extract, transform, and load packages can be invoked from other Web services so that data integration can be initiated on demand. Leading vendors already provide this XML access.

Focus on the Ends, Not the Means

Providing a blend of strategic and tactical information on demand isn't a question of a single product or technology. It requires a mosaic of techniques and technologies and affords BI architects and strategists a choice of methods to achieve the desired goal.

Building an ADW in a traditional hub-and-spoke manner is one approach to establishing an architecture and process that achieves the intent. An SOA based on Web services and XML is another means to the same end. Thinking of BI functions (such as ETL, query and reporting, OLAP, and data mining) and the data they source as services expands the ways that all these applications and infrastructure components fit into the corporate information processing architecture.

A Question of Choice

Whether to implement an ADW or SOA architecture to deliver integrated tactical and strategic data is something to consider carefully. Of course, these architectures aren't mutually exclusive. A company might build an ADW to integrate tactical and strategic data and still use Web services to enhance BI applications and information delivery. Properly planned, the SOA can also deliver integrated tactical and strategic data into advanced BI applications, achieving the intent of an ADW without requiring a single warehouse environment to handle both types of data.

Research both architectures thoroughly before making a decision. But keep this in mind: If you've had limited success using a traditional hub-and-spoke data warehouse, blending in tactical data isn't going to make matters easier.

Whatever approach you choose, integrating tactical and strategic data into advanced BI applications that can be invoked on demand is a requirement. As an architect, I always appreciate having choices in how to address that requirement.

Michael L. Gonzales has been a BI data architecture and solutions strategist for more than a decade. He teaches a series of courses internationally through HandsOn-BI, LLC and is the author of the recent book IBM Data Warehousing (Wiley, 2003).

Resources

DB2 Business Intelligence
ibm.com/bi