Most companies have to piece "master" or "reference" data together to get a consolidated and accurate view of their business because this data is shared across operational and analytic systems.

The challenge is maintaining consistency in this data, which includes information about customers, suppliers, accounts or organizational units and is used to classify and define transactional data. It's especially difficult to keep master data consistent, complete and controlled across the enterprise when disparate, decentralized data marts define and handle master data in different ways.

Misaligned and inaccurate master data can cause costly data redundancies and misleading analytics, limiting an organization's ability to answer questions such as: "How can I market opportunities to my customers that best fit their needs?"

Data mart consolidation is one way to address the issues raised by poor-quality master data. However, if master data differences are not reconciled when the data marts are consolidated, the business will still be plagued by data inaccuracies, redundancies and discrepancies, all of which compromise the decision makers' view of the enterprise.

The answer to these and related issues is a master data management (MDM) strategy with a set of processes that creates and maintains an accurate, consistent view of reference data in a single, integrated enterprise data warehouse (EDW) that the entire organization can access for decision making.

enlarge The cost to maintain a data mart is between $1 million and $2 million. These costs include data quality processes, software licenses and maintenance, storage and server hardware, and personnel. The summation of the individual costs to support each data mart results in "big dollars."

enlarge Data mart consolidation delivers a consolidated, single view of the business for increased accuracy and speed in decision making.

Merging data to save money
A data mart was once the data storage architecture of choice for most businesses. It was seen as a quick, easy and inexpensive data management solution that could address the needs of a specific business unit. Additional data marts could be built as the business grew or as other business units were formed.

Although decentralized data marts may appear to be quick and easy, they are actually quite labor-intensive and expensive to maintain. And while they do meet the needs of specific business units, they usually fail to meet the needs of the business as a whole.

This failure occurs because independent data marts do not reconcile information with other data marts. The business rules of a particular department may require that data be defined and stored one way, while another department might define and store the same type of information in a completely different manner. As a result, business leaders have difficulty reconciling disparate data from various data marts to achieve a consolidated or single view of the business.

The solution to the problem—and a compelling value proposition—is data mart consolidation.

The basic business concerns addressed by this process are illustrated in figure 1, above. Over time, multiple data marts were created to solve analytical problems within different business units. For example, DM-1, DM-2 and DM-3 all analyze orders by customer.

Maintaining these data marts is expensive, with costs related to database software licenses, hardware and the labor required to support a variety of data mart-enabling technologies. Also, 35% to 70% of these costs are redundant as a result of expenses associated with the data redundancy, data inaccuracy and maintenance of multiple data marts.

Data mart consolidation is represented in figure 2, above. All of the data that originally went through extract, transform and load (ETL) to various data marts is now consolidated and loaded directly into an EDW. This reduces costs, allowing enterprise summaries, such as total orders for all business units, to be calculated and reported with greater accuracy and ease.

MDM is key for data mart consolidation success
A "best practices" solution for MDM consists of a set of processes, software, templates and services that create and maintain an accurate, consistent view of reference data that the entire organization can access for decision making. It also needs to be designed to create a single environment where master data can be consistently described, used, synchronized and stored within an organization. This approach has two key functions:

	Data management creates a framework through which data is authored, monitored and maintained.
	Data synchronization ensures the coordination of business systems that touch the data.

Data mart consolidation is an ideal time to implement this type of MDM solution, which can create and maintain application-specific data; consolidate data from disparate systems; cleanse, transform and validate data; and filter unwanted data.

Consider a data mart consolidation without the use of MDM. When data is loaded into the EDW without resolving the master data obtained from the transactional systems, the EDW does not contain "enterprise information"— it contains the same decentralized information that resided in the business-unit data marts. Consequently, any business intelligence (BI) application that requires enterprise information (e.g., customer data) must create the enterprise master data as a part of its individual reporting process, and each BI application will incur the cost of correcting the master data.

But it's possible to correct the master data once so it can be reused by every BI application that requires it. Each successive BI application that uses the master data drives down the unit cost of creating that business unit's master data and increases the economies of scale for the enterprise master data.

As illustrated in figure 3, below, the ETL process moves transaction data from the various data marts to the EDW, just like in a conventional data mart consolidation. At the same time, MDM provides the ability to extract, transform and load master data (e.g., customer name) from the transactional systems—independent of the transaction data—and resolves any differences in the master data that may have been created as a result of the decentralized nature of the transactional systems.

enlarge Combining data mart consolidation with master data management ensures complete and accurate reference data for "customer" throughout the enterprise.

With this additional step, MDM ensures that the data mart consolidation results in the cost benefits of consolidating data on a single enterprise platform. Moreover, there is an "informational benefit" of having a truly complete and accurate view of the business. The cost of maintaining decentralized data marts is eliminated, as would be expected, but companies also experience the benefits of new business insights and resulting business opportunities that were not possible with decentralized data marts.

In the long run, this informational benefit may be even more valuable to the business than the creation of the new enterprise information itself—the company now has a better understanding of how many customers it serves, as well as the profitability of each customer. That intelligence can lead to more targeted and effective marketing campaigns, which can result in increased revenue and business profitability over time.

Good data leads to great value
Some companies awaken slowly to the importance of MDM, while others are jolted into awareness when a major project is sidelined by poor master data quality. Data mart consolidation is an important step toward developing a single, enterprise-wide data repository that can facilitate an integrated view of critical business data. But MDM is critical to ensuring that the view is clear and accurate.

Data mart consolidation with MDM provides the right solution for companies that wish to lower costs yet increase business agility by improving enterprise decision making. T

Mark Shainman, a senior program manager for Teradata, manages the deployment of text analytics across Teradata. He also works on the company's strategy, market analysis and MDM teams.

Photograph by Maki Strunc

Teradata Magazine-June 2007