Poor data quality and the lack of integration make it difficult for business users to obtain a consistent view of the business process.

—Colin White
BI Research

Accurate business information is required for effective decision making, and it will always take time to collect, deliver and act on this information.

That was really smart!

Technology plus strategy yeilds intelligent real-time decisions

by Colin White

WHEN I FIRST STARTED DISCUSSING real-time decision support several years ago, most IT managers considered the real-time concept to be about improving the performance of data warehouse queries and analyses. Today, with increased publicity about the business benefits of real-time technologies, both technical and business users have begun to realize that real-time processing is not about performance but about making organizations more responsive to solving business issues and satisfying business needs.

Despite this increased awareness of the business benefits, there is still considerable confusion and debate about what the term "real time" really means or what it provides to the enterprise. Real-time decision support provides four key capabilities:
1. Data-on-demand for rapid access to information about current business operations;
2. Business activity monitoring for quickly generating business metrics to optimize daily business operations;
3. Automated business alerts for notifying users about critical business issues;
4. Instant recommendations that help you make better, faster business decisions.

Together, these capabilities enable an organization to improve the overall efficiency of business operations and activities, identify business performance problems and satisfy customer requirements. The results are reduced costs, increased profits and improved customer satisfaction.

Integrate data, improve quality
Batch and online operational applications have evolved over many years. Most organizations disperse operational data across different corporate systems and manage it with a variety of homegrown applications, resulting in inconsistent and often redundant data.

Poor data quality and the lack of integration make it difficult for business users to obtain a consistent view of business processes in areas such as finance, customer interaction and supply chain operations.

Enterprise information integration (EII) and near real-time data integration are two technologies that can help improve this situation. EII, a rapidly evolving technology, allows interactive tools and operational applications to issue queries (often called federated queries) that can retrieve data from multiple heterogeneous data stores.

EII can access current, real-time operational data. However, it only offers limited data cleanup and transformation. So while this approach helps with data integration, it is only useful for solving minor data-quality problems.

A better solution for dealing with operational data consistency and redundancy issues is near real-time data integration, which captures and transforms data from multiple operational systems and integrates it into a low-latency data store. Applications and interactive tools can then use the low-latency store to display, report on and summarize information about the status of business operations.

This approach is frequently used to create a consolidated view of master reference data, to create integrated data sources for new operational applications and to propagate data to downstream applications, e.g., from front office to back office.

Easy and fast access to a low-latency data store enables business users to gain a consistent view of business data and operations. There will always be a delay, however, between capturing operational data and making it available.

How fresh the information needs to be in such an environment will vary by company and application. Some applications require as close to zero latency as possible, whereas a few minutes or hours might be sufficient for others.

"Based on discussions with business users, we chose to update the real-time component of our Teradata Warehouse (system) every two hours," says Phillip Gollhofer, manager of business intelligence at Burlington Northern Santa Fe Railway. "This is a trade-off between IT costs, making rapid decisions and keeping the data stable long enough to enable issues such as train delays to be analyzed by users."

Gather metrics, reach goals
A business intelligence system is useful not only for querying and reporting, but also for gathering metrics of business performance, comparing those metrics against business plans and goals, and alerting users when business objectives are not being met. This capability is known as corporate or business performance management.

Until recently, performance management has been primarily used for strategic planning and tactical analysis, but new real-time technologies have enabled it to manage day-to-day business operations. Gartner Inc. has coined the term business activity monitoring (BAM) to signify real-time and event-driven performance management.

BAM is capturing considerable attention these days because it can monitor specific business processes and rapidly identify business problems before they can have a major impact on business performance. Examples include fraud detection, risk management, just-in-time inventory, real-time product promotions and pricing, dynamic portfolio analysis, programmatic trading and so forth.

Examine BAM, analyze vendors
A variety of vendors are upgrading and targeting their products at the BAM marketplace. These include application integration vendors, new BAM vendors, and existing data warehousing and business intelligence vendors. Key distinguishing factors between these products include scalability, transformational power and the ability to put performance metrics into a business context by supporting access to data warehousing and planning systems.

Application integration products are ideal for BAM usage since they already handle event-driven business processes. Supporting BAM involves adding a monitoring and reporting facility to a product's business event-handling capabilities.

The main issue with this approach is that it only enables events passing through the integration facility to be monitored. However, application integration is especially useful where near real-time performance management is required.

New BAM vendors are building performance management products from the ground up. These products can tap into the event flow of application integration products, but they can also receive events and information from other sources such as hardware devices, Web click streams, data warehouses (for putting performance metrics into a historical context) and so forth. The issue for new BAM vendors is that these performance management products lack market visibility and penetration. As with all start-ups, some will be acquired, many will go out of business and a few will become key players in the market.

Data warehousing and business intelligence tool vendors are also beginning to offer BAM solutions. In most cases, these solutions are being built on top of low-latency data stores. The benefit of this approach is better integration with the existing business intelligence environment. Disadvantages include the need to construct and maintain the low-latency store and the inherent data latency caused by creating such a store.

This approach, however, is ideally suited to BAM applications where split-second performance management is not required and where other business intelligence applications use the low-latency data store.

Continental Airlines, for example, combines both customer reservation data and operational flight data into its Teradata Warehouse.

The low-latency data in the warehouse is used in a variety of business intelligence applications, including monitoring flight delays and automatically rescheduling delayed passengers.

Predict outcomes, take action
The types of decision processing discussed so far are reactive in nature-that is, they involve processing business events either as they occur or after they have happened.

Predictive analysis, on the other hand, tries to predict the outcome of a business event, such as the risk of granting someone a loan or the propensity for someone to purchase a product. This style of processing in a real-time environment involves sending information about the business event to a predictive analysis application and requesting a recommended action. The application must respond in a timely manner because a customer might be on the Web or a phone waiting for a response.

A predictive analysis application uses business rules to determine probable outcomes and make recommendations. Business users can define these rules or a decision support application such as a data-mining tool can define them.

Collect data, time correctly
Regardless of the technology used, it is not possible for organizations to react in real time to resolve business issues and satisfy business needs. Accurate business information is required for effective decision making, and it will always take a certain amount of time to collect and deliver this information to business users-and for users to act on this information.

"People should not think in terms of real-time, but should consider how responsive each business process needs to be to satisfy a specific business goal," says Teradata CTO Stephen Brobst. "Organizations should think in terms of right time, rather than real time."

It is important to recognize that a real-time enterprise is also not just about technology. For a real-time enterprise and real-time decision support to be successful, organizations must modify their business practices and educate business users about real-time solutions in order to exploit and gain maximum business benefit from real-time enterprise initiatives.

"When we implemented our real-time BI system, we had to convince our field staff that we weren't going to just increase their workloads," says Alicia Acebo, data warehouse director at Continental Airlines. "These people were already very busy, and we had to demonstrate that the alerts and information we were delivering to them were going to make it easier for them to handle customers who were delayed. The result was we reduced employee workloads and also increased customer satisfaction."  T