Creating the Strategic Intelligence Framework

Traditional business intelligence (BI) reflects the analytic technologies and architectures born from the original data warehousing efforts of the 1980s. Standard reporting, ad hoc and online analytic processing (OLAP) are all part of the conventional tool chest that has become synonymous with the phrase "business intelligence." But while these tools contribute to the discovery of insight, by themselves they are hardly sufficient for today's competitive climate.

Modern organizations insist on getting information on demand and uncovering actionable insight using a broad array of tools and techniques. Spatial and unstructured text analytics, advanced visualization techniques and predictive and exploratory mining are some essential competing techniques.

In today's environment, traditional BI is not enough. Teradata offers "strategic intelligence" as the new phrase to describe the modernized architecture that raises BI to a new level. "Strategic intelligence" is defined as the broad analytic landscape that's matched with computing scale and performance to achieve faster refresh cycles. The integrated and optimized strategic intelligence framework spans the entire system stack, offering deeper, more concise and more precise analytical insight—all within an enterprise data warehouse (EDW).

Representing the marrow of modern BI platforms, strategic intelligence can be summed up in three distinct characteristics:

1. Integration. Functionality and enablers are built into the database, leveraging the technology and maximizing return on investment (ROI).

2. Accessibility. Open, standard interfaces on an integrated platform make enterprise data and insights available to all users.

3. Collaboration. Partnership with BI and data warehouse vendors provides the best practices.

Stepping up to BI challenges
Companies worldwide face challenges that dictate a broader and more robust vision than what traditional BI techniques and architectures can deliver. Reduced cycle times and faster response times are driven by the rapid growth of Internet commerce and other business needs. As data volumes continue to explode and the number and complexity of business decisions increase, CIOs must provide accurate analytics within days or hours in near real time, while minimizing cost.

Now businesses empower their enterprise users with a variety of tools and applications to give them deeper insight into their organization while hiding the complexity of data and analytics. Data visualization, canned reports, simplified query capabilities, OLAP and exploratory analytic techniques are some tools that enable strategic intelligence.

Most BI implementations are little more than disparate technologies loosely cobbled together. Many of these BI tools and applications use simple SQL syntax to extract data from the EDW in bulk, stage it within the application tier, filter it and aggregate it to produce the desired results. Only then do they apply the functionality for which the BI tools were purchased.

So why do so many tools and applications do more of the data preparation work as opposed to leveraging the relational database platform? The answer is found in traditional development practices.

The processes often implemented by developers today were cultivated at a time when SQL couldn't even do simple "top n" rankings. Now, modern SQL functions in relational database management systems (RDBMSs) can perform everything from regression analysis for customer churn to a smoothed moving average of stock prices.

Furthermore, today's RDBMS platforms provide functionality from multi-dimensional (aggregation-centric) analysis to data mining. And these aren't just basic features, but sufficiently robust, built-in functionality accessible via application programming interfaces (APIs), user-defined functions (UDFs) and stored procedures.

Raising intelligence in item deletion

Item deletion is a complex problem many category managers face. A slow-moving product can affect the bottom line, so the manager may need to consider replacing it with a more profitable product. The first step is to analyze the product's sales financial data to determine its margin and profitability.

However, it's not as simple as ranking the sales and revenue data. Many other factors must be considered before yanking it off the shelf:

> How well does the product sell with a special promotion?

> Does the item sell well during a particular time of year?

> How much product inventory is in various stores?

> Who buys this product? Does this item attract high-value customers?

> What is the market basket of this item? Would removing this item affect the salability of other items in the store?

> Could promotions from nearby competing stores affect the sale of this item? Conversely, does this product pull in customers from other stores?

The Teradata approach to strategic intelligence is to integrate not only the detailed data required to make these decisions but also the analytic results of key performance indicators, scores, segmentations and promotional data. By applying analytic capabilities in the enterprise data warehouse, businesses can enhance their item deletion application by taking the guesswork out of decision making. Based on detailed data, category managers can make informed decisions regarding which items should be pulled from the shelves. These faster, smarter decisions will lead to greater profitability.

—A.Z.

Teradata provides in-database functionality to support the discovery of actionable business insight. Significant benefits can be realized by pushing this functionality into the core of the architecture—the EDW. For starters, the heavy-lifting is shifted onto the database server where data is sorted, joined, transformed, merged and aggregated.

With the data in the RDBMS, data propagation and duplication are minimized, which leads to greater efficiency and reduced inconsistency. Because the algorithms for common business measures are performed on the same data and in the same RDBMS environment, a single view is achieved across the enterprise, regardless of user tools or applications. The result is a database platform that can deliver refined data to strategic intelligence endpoints, such as users, applications and tools, at the desired level of granularity.

Strategic intelligence functionality
Using technology like UDFs and stored procedures, the modern strategic intelligence framework is based on leveraging as much analytic functionality as possible on the same data within the same data warehouse.

Leveraging common warehouse data with open, standard interfaces such as SQL, multi-dimensional expressions (MDX), extensible markup language (XML) and predictive model markup language (PMML) serves two purposes. First, it makes for a more efficient and effective environment by simplifying data movement and providing seamless integration of technology. Second, the strategic intelligence framework provides an environment where the data-intensive analytic tasks are performed inside the EDW, allowing BI and analytic tool providers to focus on adding value to the user discovery effort.

Of course, the best strategic intelligence framework not only integrates a broad analytic landscape but also provides data management and administration with tools that enable scalability and maximum performance. For Teradata users, these tools include performance optimization, automated data management, workload management, availability and reliability, and scalability and parallelism.

The figure above illustrates the core components of an advanced strategic intelligence framework. Based on the underlying RDBMS—the cornerstone of the framework—the modern, flexible platform offers a broad analytic landscape at zero-latency service levels. Leading RDBMS vendors have been moving steadily down a path of supporting strategic intelligence functions, such as predictive and exploratory analytics, text mining, OLAP, spatial analysis, advanced statistics and business rules engine support, inside the database.

This integrated platform provides an array of analytic functionality accessible through standard, open interfaces or via extensions in SQL. The SQL extensions expose the strategic intelligence functions with the most ubiquitous standard interface—SQL—and can be complemented with XML/PMML to support non-tabular data (for example, data mining models) and MDX to support multi-dimensional analysis for both metadata and partner tools.

Stored procedures and UDFs provide added flexibility to invoke analytic functions in the database without requiring proprietary languages or APIs. Consequently, advanced analytic techniques such as data mining can be accomplished entirely within the relational database using SQL exclusively. By leveraging common warehouse data with open interfaces exposing spatial analysis, OLAP, mining and statistical functions, BI tool partners exploit the back end for greater efficiency and lower total cost of ownership.

Beyond BI
For those who practice strategic intelligence on a daily basis, traditional BI tools and architectures are woefully inadequate. While providing an important means for analysis in some circumstances, standard reporting and OLAP tools represent only a fraction of the techniques and technologies forward-thinking architects consider.

Instead, text mining, exploratory and predictive analytics, advanced visualization and spatial analysis have become necessary components of the strategic intelligence tool chest. These expansive analytic capabilities must be matched with optimum computing power in terms of scalability and performance. This will provide the efficiency and effectiveness modern, competitive organizations demand. T

How intelligent is your data warehouse?

Strategic intelligence must provide integrated analytic and data management capabilities while it efficiently and effectively processes user requests. An optimum database must deliver all of the data in a centralized data warehouse, while providing user and data scalability, mixed workload management and optimal performance. Teradata provides a strategic intelligence framework through:

> Centralized integrated data. Contained in a centralized data repository—the enterprise data warehouse—integrated data from across the organization is used to perform business analyses.

> Performance optimization. The Teradata Optimizer provides parallel performance and ensures all requests take the optimal query path to leverage in-database analytic features—part of a wide range of analytic and multi-dimensional capabilities in the database—to pre-calculate summaries optimizing online analytical processing.

> User scalability. All enterprise users can apply analytic insight in their decision making while enabling the data warehouse to become a self-service strategic intelligence foundation.

> Scalability and parallelism. The scalability and parallel performance required to power strategic intelligence are enabled through the massively parallel processing architecture. This shared-nothing architecture is made of independent nodes. As these nodes become available, tasks are balanced across each of them for parallel processing, thus delivering linear scalability.

> Mixed workload management. Strategic intelligence adds a variety of different workloads to the data warehouse, from ad hoc queries to complex analytics. These mixed workloads benefit from built-in workload management capabilities, which prioritize queries based on user, number of tables, rows and databases, and estimated processing time.

> Availability and reliability. Up-to-the-minute data availability and data reliability for analytic intelligence are ensured:

Built-in redundant features guarantee continuous operation when one component goes down.

Hot-swap functionalities replace failed components, without system downtime.

Standby nodes minimize system disruption.

Fallback guarantees data is not lost in the event of a large, localized disaster.

> Flexibility to integrate in-database analytics. Dynamic user-defined functions allow users to smoothly encapsulate their analytic technique to run in the database.

—M.L.G. and A.Z.

Creating the strategic intelligence framework

enlarge

To establish an environment for strategic intelligence, these high-level steps are necessary:
1. Set up a strategic intelligence foundation. Establish a centralized data repository containing integrated data that is required to address business analytic projects from across the enterprise.
2. Create reports. Generate reports to help executives understand, interpret and monitor the state of their businesses. Organizations can generate simple or complex reports and effective executive dashboards using data derived from integrated data in the strategic intelligence foundation.
3. Analyze. Use data in the reports to identify the reason a particular condition occurred. Use online analytical processing tools for a multi-dimensional view of the data, and ad hoc query tools to drill down on insight discovered through reports or analyses. All discoveries and calculations are stored in a collaborative environment for user access to ensure a single view of the data and a common definition of calculations.
4. Predict. Use predictive and exploratory analytics to forecast next events or occurrences and advanced analytic techniques to uncover data patterns. The results of these analyses are stored and shared through the strategic intelligence foundation.
5. Act. Put strategy and insights into action. Strategic applications can be created and campaigns developed from the acquired knowledge. Front-line workers can use this intelligence to respond to customers in near real time. To support putting insight into action, businesses can either implement custom applications or apply ready-made tools such as business rule engines.

—M.L.G. and A.Z.

Michael L. Gonzales specializes in BI technologies and techniques and is a principal at Claraview, Inc., where he leads the education department. He is a doctoral candidate at the University of Texas at El Paso and a successful author. His latest book is "BI Strategy: How to Create and Document."

Arlene Zaima is the Teradata Strategic Intelligence optimization program manager and has more than eight years of experience in advanced analytics.