|
|
|
|
|
|
The active advantage
Active data warehousing brings sophisticated insights to day-to-day business decisions.
|
The moment of truth
Getting the most from your active data warehouse requires new approaches to your business.
|
Extending the use of your enterprise data warehouse to active data warehousing requires a focus on integration and interoperability.
|
The sky is the limit
Applebee's, Continental Airlines, Hudson's Bay Company and Travelocity share secrets of their success.
|
| View full document > PDF (1818 kb) |
|
|
|
|
|
Putting theory
into practice
Extending the use of your enterprise data warehouse to active data warehousing requires a focus on integration and interoperability.
by Jim Utsler
The basis of active data warehousing is the enterprise data warehouse (EDW), a timely, integrated store of detailed data available for analytic or strategic business decision making. An active data warehouse is merely an augmented EDW; active data warehousing is as much about how you use your EDW as it is about what capabilities you add to it.
The active data warehouse extends the functionality of the EDW into the realm of operational decision making. It integrates current transactional data and near-time analysis with historical data and enables timely deployment of the information to decision makers throughout the organization. An operational decision could involve the rerouting of trains in the aftermath of a derailment, the choice by a customer service representative as
to whether a fee should be waived for a caller,
or any of the multitude of decisions that get made daily in the course of running a business.
“The ideal is for organizations to use the data warehouse as a core engine and analytical platform for managing and exploiting information, so that it may be delivered in an actionable and intelligible form across all business operations and strategic environments,” says Helena Schwenk, senior analyst with the IT advisory firm Ovum. “It’s about closing the gap between a business event
occurring and the ability of the organization to act upon that event in a more proactive
and timely manner.” The reach and impact
of that intelligence can positively affect almost
all aspects of customer service, supply chain management, logistics and beyond.
Bringing the elements together
In order for organizations to take full advantage of analytics for both operational and strategic decisions, the technology infrastructure of the data warehouse needs to support the demands of this “mixed workload” environment. It must be able to support the elements of active load, active access and active events with the enablers
of active enterprise integration, active work-load management and active availability (see "Getting it done" below).
Active load is the ability to load data twice
a day or more without impacting the decision-making workload on the data warehouse. These active loads can occur multiple times daily, hourly or even in near real-time continuous feeds, driven primarily by the business need to have current data to support operational decision making.
As corporations become more global, many now load data continuously throughout the
day to eliminate the need for batch windows, which are constantly shrinking. Others move
to active load simply to anticipate the future. As one vice president of applications at a large retailer says, “I don’t have the business need today to justify active loads but I know it’s only a matter of time before our business will change and require that I do. I might as well do it right the first time.”
Active access is the ability to access data
in timeframes of 10 seconds or less. This low latency allows the active data warehouse to support operational business processes with service level agreements (SLAs) for response-time consistency. Many times we refer to these active-access queries as tactical queries because they frequently support the tactical or operational decisions executed at the front lines or
in the operations of the business. Active access queries may consist of a query to retrieve information to support a customer interaction, or they may include some analytics on the data
to facilitate the decision-making process.
Examples of information retrieval queries include shipping company applications that allow customers to monitor package delivery,
or call center applications that display the last 20 transactions of a given customer within seconds of identification. Analytical queries tend to be more complex, such as matching a customer’s historical behavior with current sale offers to provide a personalized offering only moments after contact. Many of these operational or customer-facing active-access queries have a business SLA of two seconds or less.
Active events push events and actions from the data warehouse to systems or users supporting business processes, depending on rules and context. As analytics move into enterprise operations, the ability to detect an event in real time, analyze it in the context of business rules and take operational action as appropriate becomes a competitive differentiator. Such events are referred to as “push” events because the event process (data acquisition, analysis, action) is pushed out to the enterprise upon event detection. In contrast, “pull” events are detected by batch jobs or SQL for events that have previously occurred. The primary differentiator between the two is that push events are targeted toward impacting the event while it is in process, while pull events drive operational execution after the fact.
Both types of events are valuable, but push events provide companies with a competitive advantage. Consider the supplier who detects
a large customer order at 2:00 p.m. on Tuesday. Their active data warehouse analyzes the current inventory to determine that filling the order will impact the next day’s delivery for
30 customers. In a push event, the system alerts operations so that personnel can notify a vendor by 2:30 p.m. to ship an overnight delivery
of a required material; as a result, all 30 orders can be filled by the next day. Compare this to the supplier who finds out Wednesday morning that, because of a large order input the previous afternoon, they only have enough inventory to fulfill 80% of the customer shipments that day. The consequences can be serious, at best generating customer dissatisfaction and, at worst, impacting SLAs and incurring penalties.
Active enterprise integration enables the data warehouse to span the organization using
a real-time enterprise reference architecture to ensure interoperability and optimal performance (see “The conductor’s score” below). Such an architecture can help you determine which of your current integration tools are required
to allow your data warehouse to operate and interface with operational data, business processes and applications.
An open approach to integration generates the most effective results. Supporting open standards such as .NET allows a seamless integration into the existing enterprise architecture, providing operational applications with access to analytical information.
Just as integration is key, so are ease-of-use, reusability and interoperability. An effective development infrastructure and an application platform will allow you to build applications that easily fit into your existing computing environment. The managed object framework in an application platform allows you to build your own analytic components and store them for future use (see “Making the job easy” below). Consider credit checks, which are used for customers opening checking accounts or applying for mortgages or employment. If you build a credit check component, you can use it in a variety of applications, saving money and development time.
Active workload management is the ability to dynamically manage system resources in the mixed workload environment of active load, active access and traditional enterprise-data-warehouse business analysis. The active load and active access workloads by themselves are not a challenge. Add complex strategic business analysis, however, and managing this real-time enterprise decision support environment becomes a real juggling act. To support the SLAs set by the organization, an agile resource and performance management system
is a requirement.
Recall our shipping company illustration. Customers want the answer to their package inquiries in one or two seconds, which is easy enough under general operating conditions. When the fleet operations department submits its route-optimization query, however, things change. The route-optimization query uses the same transaction or data that supports the Web tracking application, so the one- to two-second Web response time for customers can easily increase by a factor of ten, or even stretch to more than a minute. That may not seem like
a long time unless you’re the customer hitting “submit” over and over again trying to find
out where your package is. Active workload management eliminates this problem.
Active availability requires that the technology/platform selected meet the availability requirements of a business- or mission-critical system. An operational or transactional system delivers a predetermined service level
to users who depend on it to support critical business processes. Traditionally, data warehouses have not been considered business critical and have not had the same expectation levels placed upon them. An active data warehouse must achieve the operational
service level defined by the organization in order to satisfy the “near real-time” feeds of active load, the 10-second-or-less SLAs of
active access and the requirement of accessibility for both informational and analytic retrieval of data to support operational
decision making.
It is important to remember that simply having the hardware and software running
does not constitute availability for an active
data warehouse—the data must also be fully accessible to users so they can do their jobs. Traditional data warehouse operations often assume that data can be taken offline for data load, backup, data maintenance or model changes. In an active data warehouse, data
can never be offline.
The extension of an EDW to an active data warehouse can be straightforward, provided
you work to develop an integrated, interoperable system. Combine the capabilities
we’ve discussed and you have an active data warehouse that can drive your business success.
Beyond technology
Implemented properly, an active data warehouse can dramatically improve the quality of day-to-day business transactions. It creates real differences in serving customers, delivering products, manufacturing goods and securing supplies across the entire value chain. These differences will not be realized if the company is not prepared to leverage it, however.
An active data warehouse project demands that business and IT work in concert to set and achieve corporate objectives to derive optimum business value from the data warehouse investment. Culture, processes, procedures and skill sets need to change.
Unique to an active data warehouse
project are the shifts required by the IT staff. Data warehousing organizations are sometimes established as small islands within the larger IT department. As the data warehouse transforms to a mission-critical system, tighter integration between the data warehouse, the data warehouse team and the rest of the IT organization becomes essential. It may involve simple remedies, such as data warehouse team members being included in meetings, or it may involve expanding enterprise systems management processes to include the data warehouse.
The migration from an EDW to a service-based data warehouse calls for specific skill
sets. The move requires that the data warehouse team take on a few more roles and learn new technologies with which their application developer counterparts may already be quite competent. Initial service-oriented architecture (SOA) implementations may be a struggle;
for example, integrating roles, defining responsibilities and formulating standards are not easy tasks. After all, building Web services components that use technologies for the delivery and exchange of information provides
a mechanism to expose the true business value of the information stored in the data warehouse. Embrace this time of change to showcase your active data warehouse’s ability to provide intelligence to your enterprise. T
Jim Utsler is a senior writer for MSP TechMedia.
Illustration by Jean Tuttle.
|
|
An application platform simplifies the development of components for your computing environment. Teradata Application Platform supports both standard enterprise data warehouse (EDW) applications and applications for EDWs being used as active data warehouses. Designed as a set
of six services, it includes:
1. Application Development—Teradata Application Platform provides extensions to Eclipse-based tools for development and environment integration into Java 2 Platform, Enterprise Edition (J2EE)-based Enterprise Frameworks like IBM WebSphere or SAP NetWeaver.
2. Presentation Services—If you do not have a portal or other standard front-end, this supplies
a common GUI framework and navigation tools for Web applications. Presentation services allow users to create a consistent GUI look and feel for their applications, as well as provide navigation tools for finding and reusing components.
3. Managed Object Framework—This framework delivers base-class implementations as well
as guidelines to build J2EE component and object patterns. This service is the basis for building
a Teradata Application Platform Component Library of reusable analytic components.
4. Application Services—This set of services includes event handling, security and scheduling.
5. Data Services—Providing common services for constructing and reusing SQL and business objects/components within a Teradata Application Platform, Data Services facilitates easy accessibility and reuse.
6. Administration Services—This toolset consists of a collection of services for configuring, managing and monitoring the application platform. Teradata Application Platform allows administrators to monitor application use and performance for capacity-planning purposes.
Building on common J2EE application servers like JBoss, IBM WebSphere and SAP NetWeaver, Teradata Application Platform provides base services so your application developers can focus on the business logic, not the environment details. Teradata Application Platform gives you a library framework for capturing your Teradata-based components, allowing you to increase component reuse. It increases the ease of deploying Teradata-based applications to your user base, based on roles, presentation services or tools. —J. U.
Back to supporting paragraph |
|
|
Teradata offers many options to equip your enterprise data warehouse to perform active data warehousing. The following is a representative list of products and features for your reference:
| Active load:
|
TPump, FastLoad, Multiload |
| Active access: |
Aggregate Join Index (AJI); Atomic UPSERT; Partitioned Primary
Index (PPI) and dynamic partition elimination; value list compression;
unique secondary index (USI); read access dictionary locks; ARC with JI;
collect stats locking; VM&F, PUT; scan-disk time reduction; prioritized file
system; snapshot dump; restart time reduction; Microsoft’s Open Database
Connectivity (ODBC), Java Database Connectivity (JDBC), Object Linking
and Embedding Database (OLE DB) |
| Active events: |
Stored procedures and external stored procedures, triggers, triggers calling
stored procedures, queue tables, external table functions |
| Active enteprise integration: |
Teradata’s Real-Time Enterprise Reference Architecture, Teradata integration: Application Platform and other software and tools that work with open standards and adapters. Enterprise tools supported include IBM WebSphere, BEA WebLogic, SAP Netweaver, TIBCO, JBoss, .NET and others compatible with service-oriented architectures (SOAs) |
| Active workload management: |
Teradata Priority Scheduler, Teradata Dynamic Query Manager, Reserve AMP Worker Tasks, Teradata Active System Management |
| Active availability: |
Redundant hardware components, single-system availability with
implementation of large cliques, Hot Standby Nodes, Fallback,
BAR, Teradata Dual Active Solution, etc. |
Back to supporting paragraph |
|
|
|
To realize a composer’s vision, an orchestra and its conductor need to know when each individual instrument should play, what they should play and how to get the instruments to play together in harmony. They need a score, a blueprint for the symphony. That’s the same role Teradata’s Real-Time Enterprise Reference Architecture plays in active data warehousing.
At first look, the architecture may be daunting, but it is most useful if you keep some basic thoughts in mind:
1. The transactional, front-office applications and data lie on the left.
2. The strategic, tactical and business intelligence (BI) applications and data lie on the right.
3. Data acquisition for the Teradata Warehouse (bottom) can be performed in a number of ways, including using Teradata Standalone Utilities, Teradata Parallel Transporter, scripts or database triggers, TPump, Enterprise Application Integration (EAI) or replication, or via Teradata partner products such as Informatica, GoldenGate, Ab Initio, ETI.
4. The big pipe in the middle represents all the middleware required to get the data from point A
to point B. That middleware may include complete integration stacks (like IBM WebSphere, SAP NetWeaver, BEA WebLogic, AquaLogic
or Microsoft BizTalk), or it may include best-of-breed message
brokers, enterprise service buses, portals, and/or Web services management tools.
An active data warehouse will usually involve the sequencing of applications or components into a business process, as shown by the three teal boxes below the middleware pipe. This may be done by something as simple as Teradata Database triggers or as sophisticated as packaged Business Process Management (BPM) solutions from Hyperion or rules engines such as Fair Isaac.
You can evolve, when you’re ready, to a service-oriented architecture (SOA), and Teradata’s Real-Time Enterprise Reference Architecture can be used with both Web services-based middleware and more traditional middleware products. —J. U.
Back to supporting paragraph |
|
© Teradata Magazine-June 2006
back to top
|
Quick Links
Print
Send to colleague
PDF
E-mail us
