|
Playing to win
Data integration scores big points for your enterprise.
by Dan Higgins
The most popular team sport in the world is soccer, also called football. Scenarios like this one are played out on hundreds of open fields nearly every weekend:
Anticipating the opposing player's pass, the defensive back intercepts the ball and sends it skimming across the turf to a midfielder. Before even receiving the ball, the midfielder is already aware of the movements of his teammates. He places a long pass over the defense and just ahead of his left midfielder sprinting down the sideline. The left midfielder anticipates the defense's shift; aware of the movement of his teammates, he lifts a high pass across the field. His attacking teammate, expecting the pass, slips between two defenders, leaps into the
air and heads the ball into the upper right corner of the net. Goooaaaaallllll!
Anyone who has played or watched soccer recognizes that the best teams are more than just a collection of skilled players. To compete effectively and win, the players must work hard to integrate their play. They must constantly adapt their action and positioning to complement that of their teammates. They must maintain the "shape" of their formation, rotate back in support of an advancing player, link passes through the midfield or time a "through ball" with the run of a striker. All this must be done in a highly competitive and highly dynamic environment. And when it is done well, soccer becomes "the beautiful game."
In many ways, corporate business is like soccer. Business is a team endeavor in a competitive and dynamic environment. Just as integrated play is important in soccer, the effective corporate team must operate as an integrated whole. And sharing of integrated information is critical to the success of the corporate team.
Nearly five years ago, Teradata initiated a data mart consolidation program to help companies realize considerable cost savings by consolidating data marts onto a single data warehousing platform. Numerous companies have achieved significant benefits through these efforts. Yet in some cases, data mart consolidation has simply been server consolidation. Though there are
cost savings to be realized by consolidating onto a single platform, the greatest opportunity for business benefit is available to those who move beyond simple data mart (server) consolidation to cross-functional data sharing and potentially
to data integration.
Consolidation, sharing
and integration
For the sake of this discussion, let's first explain what we mean by consolidation, sharing and integration.
Consolidation: By consolidation we mean rehosting previously independent
or dependent physical data marts onto a single system platform. This is the simplest form of data mart consolidation. There are few, if any, schema changes, and the data marts continue in their prior role.
Sharing: Data sharing is allowing a
variety of end-user groups or applications to access data that belongs to a different user group or a different data mart or
subject area. It does not require that all data be integrated into a single, common data model. Data sharing may occur on an infrequent and limited basis, in which data is copied or extracted from the originating source and placed in the target data mart or database. Or it may be required on a more constant basis, in which individual queries will join across "virtual data marts" which reside on the same platform. In this case, database views can be used to manage access by one user group to another user group's data.
Integration—integrated view: Integration can occur on two levels. The first is an integrated view. By that, we mean gathering selected data, often as a summary, drawn from various sources to present a picture of the enterprise. This occurs in executive or financial reporting (sometimes referred to as "consolidated reporting"). Although useful for reporting purposes, an integrated view does not allow the deep data interrogation or in-depth analysis of detailed data that is necessary for business scenarios such as enterprise risk management, customer relationship management or integrated supply chain or demand chain management. Nor does it address the issues associated with data redundancy.
Integration—integrated data: With this approach, data from a variety
of subject areas (and sometimes business units or functional areas) is combined
into a single, integrated repository with shared access. This pinnacle of data integration provides the greatest opportunities for business benefit and overall cost savings. The business units and IT cooperatively manage this shared repository. Data is managed as a corporate asset and leveraged across a wide range of applications and end-user groups.
Benefits and challenges
There are benefits and challenges with all
of these approaches. Understanding the differences in benefits and challenges will help manage expectations (see figure 1).
|
Figure 1
|
|
The benefits and challenges for business and IT regarding data mart (server) consolidation, data sharing and data integration.
|
With simple data mart consolidation or rehosting, there are few schema changes and possibly very few ETL (extract, transform and load) changes. Therefore this is a common first step among Teradata customers. The benefits are primarily for the IT organization: easier platform administration and management, reduced IT support and system costs, and more efficient system utilization. From an end-user perspective, there may be benefits around performance and available data space. But this simple rehosting will have little, if any, impact on reducing data redundancy and disk storage use. And with multiple end-user groups sharing the same hardware, careful attention will need to be paid to data warehouse governance, service-level agreements, funding and charge back processes.
With data sharing, we begin to see new business benefits by expanding the potential scope and depth of business questions. For example, call-center data regarding customer problems can enrich analysis of products or suppliers. Customer-oriented marketing analysis can be enhanced through the use of channel capacity data. Incorporating data from across the enterprise can significantly improve fraud detection or enterprise risk analysis.
But data sharing is not easy, especially when multiple data marts are involved. Organizations that want to share data may have different priorities, agendas and processes. Once there is an organizational agreement to share, it is necessary to agree on the data structures and the domains for the primary and foreign keys that will be used to "link" the shared data. It will be necessary to modify ETL or develop "mapping" tables. Clearly most of the challenges are not technical, especially if you are using Teradata solutions. Rather, they are organizational and process challenges. In fact, the data
sharing approach has all of the challenges of an integrated data approach, without the benefits of significantly reduced data redundancy and application development costs and timelines. To reduce or eliminate data redundancy and the associated redundancy in ETL processes, the organization will need to evolve to an integrated data approach.
|
Figure 2a
|
|
The value realized by evolving to an integrated enterprise data warehouse (EDW) increases exponentially while the costs of data redundancy and application development time decrease.
|
An integrated approach offers the
benefits of the consolidated and sharing approaches, while also reducing application development time and data redundancy. When an enterprise commits to a strategy of integrated data, the realized value of the data warehouse will increase exponentially while the cost of decision support development will fall
off over time (see figure 2a). Application development time is reduced through a "load once, use many times" approach.
As the integrated data warehouse evolves, future application development will find that most of the difficult and time-consuming data preparation work is already done. And as data redundancy is reduced or eliminated there will be significant savings in (or "re-purposing of")
disk space. There will also be a significant reduction in the "ripple effect" resulting from changes to data elements which can plague an environment with high data redundancy. In contrast a data mart-centric approach will see spiraling costs, while the initial value of data marts tend to diminish over time (see figure 2b). We refer to this as the "half-life of data mart value."
Of course, an integrated data repository is also not without challenges. There are challenges of data stewardship and governance. In addition, a new, integrated data model will evolve. Over time, existing applications may need to be modified to leverage the new data model as the older, redundant structures are removed.
|
Figure 2b
|
|
The value of independent data marts and operational data stores diminishes over time while the costs of data redundancy and application support exponentially increase.
|
Making it happen
Not everyone is ready for a broad, cross-functional, integrated approach to data management. Those organizations that do commit to a long-term strategy of integrated data face a unique set of challenges that depend on their current systems, the state of their data, their organizational structure, their business needs, and their existing staffing and skill sets. But there are some common principles that will encourage success.
Vision: It all begins with a vision. The goal of a soccer team is not simply to play, but to win. Likewise, with data warehousing, the enterprise must establish a business-driven vision. This vision must define a goal state for the data warehouse and decision-support environment. And it must be agreed upon by the business units and sponsored by executive management. It should look beyond immediate, tactical needs and current business processes. Although the technology architecture and strategy are important, they should be secondary to and supportive of the business vision for the data warehouse. If the vision cannot be achieved, it should never be because of short-sighted or inadequate technology architecture.
Executive leadership: A good coach
can integrate a collection of skilled
players together into a superb soccer team. The coach determines the team's formation and alignment of skills and roles. The coach holds the players accountable for integrating their play with the other players and into the overall team strategy. Similarly, the executive leadership of an enterprise must take responsibility for determining the strategy, aligning the business units and holding them accountable for the continuous evolution and exploitation of the integrated data warehouse.
Iterative approach: A "big bang" approach to establishing an integrated, cross-functional data warehouse is rarely effective. The more common approach is to evolve the data warehouse in phases. Enterprises should lay out a plan that aggressively expands and enhances the data warehouse. The source systems and subject areas to be addressed should be determined by the organization's ability to execute (data availability, technology, skill sets), readiness (ability of the business to adapt and exploit the new data) and the strategic and tactical value to the business. This iterative approach applies to the integrated data model, which, again, will evolve with experience and with the phased reengineering of data sourcing and ETL processes. The iterative evolution of applications will coincide with the evolution of the data model.
Normalized, detailed data foundation: Although many companies will employ a range of structures in order to meet the needs of the business, at its foundation the integrated data in the data warehouse should take the form of a normalized schema incorporating a very low or "atomic" level of detailed data. This iterative approach will retain the greatest number of relationships between business data elements and will afford the greatest flexibility to serve current and future needs of a wide range of applications and end-user groups. That said, a normalized data model, which may eventually include much of the data in the enterprise, can be overwhelming and confusing to business users whose scope of responsibility is much narrower than the enterprise. This can be alleviated through the effective use of database views. End users need only "view" what is relevant to them. Where these views are accessed dozens of times each day and require especially good performance, they can be instantiated in a physical form.
Strong, cooperative data warehouse governance: As with any shared infrastructure, there must be a cross-functional team that will oversee the evolution and exploitation of the data warehouse. The team should report to executive management. Members should establish "rules of the road" for use of the data warehouse. And they should set the priorities for development, enhancement and use with a goal of maximizing the benefit to the business.
Cross-functional data stewardship team: Along with a team to govern the data warehouse, there should also be a cross-functional team whose responsibility is defining the shared data model, resolving data structure conflicts and possibly resolving data quality issues. Without common agreement on the structure and domain of shared data elements, there can be little effective data sharing.
Persistence and hard work: The bottom line is that becoming a great soccer team requires a tremendous amount of hard work. The same is true with an integrated data warehouse. There is no technological quick fix or magic formula. Yet there is the potential for huge benefits. Companies are able to answer business questions that previously were not possible. They drastically reduce data redundancy while increasing the amount of useful and relevant data in the data warehouse. In some cases they can reduce application development times from months to days. If you are committed to making it happen, it will happen. T
Dan Higgins is the director of Teradata Warehouse Sales Support and the co-author of Evolving Through Action. He leads a team of subject-matter experts who assist Teradata's worldwide sales force.
| Teradata enables integration
|
Teradata has always provided a platform that
lets customers choose the data warehousing approach that will be most effective for them
while letting them evolve at their own pace and without technical limits. Whether a company begins with a data mart that addresses an
immediate need or launches a full-fledged integrated data warehouse, Teradata offers technical capabilities and professional services to help it move forward.
 | Foremost is Teradata's historically strong support for the complex, normalized data models associated with cross-functional integration and the often complex queries necessary to fully exploit
a rich, normalized model. |
|
This is complemented by Teradata's powerful support of database views to provide application or department-specific logical views of the shared data and to hide unnecessary complexity from particular users. |
| Teradata's ability to create multiple distinct
and hierarchical (nested) databases on a single platform facilitates data management while
allowing users to query across these databases when appropriate. |
| Teradata's multiple, parallel utilities help
simplify reengineered ETL (extract, transform
and load) processes. |
|
To help a company get off to a running start Teradata has a number of industry-specific
logical data models (LDMs). |
|
When a company is determining how to evolve through a phased implementation while delivering business value, Teradata Professional Services can help. Industry-specific enterprise data warehouse roadmap engagements begin with business improvement opportunities (BIOs), which lead to the queries that will support those BIOs and then to the data elements from the industry-specific LDM required to support those queries. |
|
© Teradata Magazine-March 2006
back to top
|
Quick Links
Print
Send to colleague
PDF
E-mail us
