Data warehouse governance

A focal point of direction, accountability and authority.

by Betty Kight

Are you getting the most out of your data warehouse investment? If the system is working efficiently and lots of work is going on, but the impact on the bottom line you anticipated is not being realized, chances are the cause is not technical. What your data warehouse needs is a process that ensures the right work is being done at the right time and in the right way. More than any other technology solution, an enterprise data warehouse (EDW) project demands that business and IT work in concert to achieve corporate objectives, ensuring the right projects are done at the right time in order to drive optimum business value from the data warehouse investment. A very effective mechanism in meeting that objective is the formation of formal data warehouse governance policies, guided by a data warehouse governance committee.

Once a data warehouse's implementation phase ends and production begins, it becomes an ongoing program. The EDW program requires the same kind of discipline used in the EDW project phase—on an ongoing basis—such as planning, monitoring, schedules, communication, and clear roles and responsibilities. Governance is an ongoing process that ensures that data warehouse initiatives align with corporate business objectives today and into the future.

"Enterprise data warehouses don't get built with a single project," says Claudia Imhoff, president of Intelligent Solutions, Inc. "They're built through an ongoing number of iterations and projects. Adding new functionality, data, users and tools to the data warehouse is a continuous process made up of many projects (or programs)."

Establishing a good governance structure enables the organization to prioritize, validate and manage this ongoing series of projects, and thus fulfill the mission of the data warehouse.

Governance, to put it in its simplest terms, is the art of steering societies and organizations. The need for governance exists anytime a group of people comes together to accomplish an end. It is the process through which groups make decisions that direct their collective efforts. If an organization grows too large or too complex to efficiently make all necessary decisions, it creates an entity—typically comprised of representatives of various units within the organization—to facilitate the process.

Governance is a decision-making process, not a product. Good governance is more than just getting the job done. Because corporate values and culture play an important role in determining both organizational purpose and style of operation, governance is concerned with achieving the desired results and achieving them in the right way. Good governance stems from—and influences—the core values of the organization.

What's the price of ineffective or nonexistent governance? A data warehouse that is not properly guided by its stakeholders runs the risk of organizational crisis and failure. This erodes confidence and ultimately damages an organization's ability to achieve its objectives. Data warehouses require significant investments of money, resources and time, and are, in many instances, highly political. Without proper and effective governance, a data warehouse can end up rudderless, adrift with an unclear mandate and a dwindling pool of support.

Governance and the data warehouse
Data warehouse governance is concerned with how the data warehouse is steered, who has a voice, and how accountability works. The data warehouse governance steering committee, comprised of business and IT professionals, is the forum that oversees the governance process.

Its primary role is to set and ensure compliance with the policies that govern data warehouse use throughout the entire enterprise. This committee must also have the authority to make decisions that cut across all units or functional areas of an organization. The role of such a committee varies by organization, but tasks could include setting strategic directions, determining priorities, committing resources, allocating funding, monitoring and evaluating initiatives, communicating status and influencing key people.

Data warehouse governance puts principles, policies and procedures in place to monitor and measure results, ensuring that the most beneficial work, from an enterprise perspective, is being done. Data warehouse project requests should always be viewed as opportunities to improve the business, and governance is the process that ensures these improvements occur.

Successful data warehouse governance committees meet on a regular basis to provide overall strategic direction for the data warehouse, approve project prioritization, and to discuss and resolve cross-functional issues. These formal meetings are usually held monthly, but can be called at any time to address time-sensitive issues.

"The committee's roles and responsibilities are to establish the priorities of projects, ensure that the projects are adhering to the corporate standards that we use to build the EDW and, if exceptions are made, that there is a pathway to mitigate or remove the exception," says Imhoff. "The committee sets priorities, gets the funding and ensures adherence to corporate standards."

Good governance is built around a set of guiding principles and policies that have been formulated and agreed upon by the data warehouse governance committee, with input from all the stakeholders. Guiding principles are more business-oriented and are the stated direction for the EDW. They are used as the 'litmus test' for planned changes and growth. Policies are more IT-oriented and are an adjunct to the guiding principles and provide further guidance on how the data warehouse environment will be used. They contain explicit descriptions of the rules that must be adhered to by both business and IT.

An example of a guiding principle is: "The quality and integrity of the data in the warehouse must support accurate decision making."

The policies that support that principle may be:

• The integrity of data in the warehouse begins with the source systems; data should be audited at the source
• Changes are made to the source system and not the warehouse
• If an update to the warehouse occurs before an audit/source system change then the warehouse must be able to take changes/final version
• All data that posts to source systems will post to the warehouse

This principle/policies relationship is especially important, because when data is shared across the enterprise it no longer has a single owner. Since no one department or business unit has responsibility for the data, sound data warehouse governance practices need to be put in place and monitored to ensure compliance with corporate data standards and agreed-upon levels of data quality and integrity.

The committee also helps all stakeholders better understand the diverse functional areas within the business and how they interact. This helps to bridge the gap between the functional areas within the business as well as IT. It also facilitates more buy-in and user adoption and helps create sustainable processes for governing the data warehouse environment.

Forming the governance committee
Who should be members of the EDW governance committee? The committee should be comprised of both business and IT professionals. However, the chairperson should come from the business side because strong business knowledge and business ownership is what drives EDW success. IT is a fully participating partner with an equal voice, and ensures that the appropriate technical infrastructure is in place to support business requirements. Members should be well-respected and influential within the organization. Such individuals should be good communicators, thought-leaders and motivators, be results-oriented, and most importantly, be able to separate their functional role from their governance role.

All members should understand their roles within the committee. An orientation session that clarifies roles and responsibilities is a good way to ensure members understand their responsibilities and are committed to leveraging governance to drive enterprise-wide data warehouse success. Members do not have to be the top executives of their departments, either. For example, at one data warehouse site, some executives insisted the organization would be better served with a handpicked group of their direct reports, who were one step closer to the organization's day-to-day operations. The bottom line is that the committee and its activities are sanctioned by the executive level within the organization.

The key is that members have the will and passion for governance and see it is an important part of their overall job accountabilities. Enthusiasm, in fact, can make or break the success of the governance effort. While it is vital that all members have a strong interest in the goals and objectives of the organization, it is also important that they be enthusiastic about governance and leadership.

Communication is the foundation of good governance and is essential between committee members as well as management and all stakeholders. Organizations should focus on creating a working environment where the committee, IT and all stakeholders know their roles and understand where to go with new ideas or stubborn problems.

It is the job of the committee to ensure that stakeholders are kept up to date on the data warehouse organization's activities and objectives. This type of communication process can consist of user forums, regular newsletters, e-mail updates or updates on the organization's Web site. Having good communication throughout an organization creates a greater sense of purpose and a feeling of inclusion and ownership, and it can facilitate a more creative approach to data warehouse initiatives. When all parts of an organization feel that their ideas and contributions are taken seriously, conflicts can be dealt with quickly and directly, all people involved remain fully engaged in their work and their potential contribution is maximized.

Governance best practices
While all data warehouse governance committees have similar responsibilities, there will be great diversity in how they function, because the organizations they govern can be extremely different. However, there is a common set of best practices all governance committees should adhere to.

The governance committee sets data warehouse priorities.
This allows the priorities to be determined with both a business and IT perspective and ensures that the projects that drive the most value are the first ones undertaken.

Prioritization decisions are made based on agreed-upon criteria and not on the flip of a coin or on a first-in/first-out basis. It also helps keep the data warehouse in alignment with key business objectives.

Communicate priorities and progress regularly and clearly.
Once the governance committee has established priorities, it must communicate these priorities clearly to all stakeholders. For example, one company's governance committee agrees on priorities monthly and posts them on an internal Web site, along with the names of persons responsible for each project. To keep the data warehouse focused on value and thinking resourcefully, the company deployed a monthly scorecard to track innovative ideas, flawless execution, and operational excellence.

Develop and support sound enterprise-wide data standards.
These standards help overcome obstacles such as siloed data, conflicting data definitions, poor data quality, access to and ownership of data, a lack of metadata.

Identify relative strategic value—even if you're comparing apples to oranges.
When faced with a stack of potential projects, it's important to compare them on the basis of relative business value as well as cost and risk. Evaluation and ranking tools may help with this process. Treat each data warehouse project as an investment. This requires going beyond simply tracking whether the projects are on time and on budget; scope and value of the project must also be tracked.

Formalize the project request process.
The annals of data warehousing history are filled with horror stories of business managers lobbing requests for IT projects over the wall and then being unhappy when the results don't meet their needs. Requiring formal written project requests is a staple of resourceful governance. It provides the governance committee with the same critical information on every project request, and facilitates the decision-making process. It is the first step toward ensuring that the data warehouse organization is focusing its resources on the highest-priority projects.

Monitor, monitor, monitor.
Once data warehouse projects are chosen, funded and launched, stakeholders need a way to stay on top of their progress. Monthly "traffic reports" provide quick summaries and highlight potential problems, enabling early intervention to prevent snags from escalating into major snafus. In addition, some companies deploy "dashboard" documents for each project, enabling managers to evaluate progress against milestones, resource usage, user involvement and issues. These traffic reports are then reviewed at governance committee meetings and measured against agreed-upon metrics so the appropriate actions can be taken.

Mandate speed with short deadlines.
Time is money, so a company that completes IT projects faster is doing a better job of managing its resources and is likely to see returns much faster. One of the most effective ways to build speed into the culture—and ensure that benefits are still relevant when they are realized—is to mandate delivery of value on short deadlines. One of the jobs of the governance committee is to set hard-and-fast deadlines as part of the project approval process.

Adjust budgets to reflect benefits.
One company has tied its data warehouse investments directly to promised savings or additional revenue. Its policy is that if a data warehouse project claims it will save $1 million, then $1 million is taken from the project owner's budget once the project is approved—period. This is a significant commitment by the functional owner to understand the benefits and then deliver. Managers at the company have quickly been able to understand that there's no such thing as a program funded by "free money." This kind of policy is the impetus that gets managers to start thinking about quantifiable metrics.

Close the loop with "postmortem" audits.
Checking to see whether data warehouse projects deliver on promised value may seem like an obvious best practice, but post-implementation audits are relatively rare. One company has established what it calls a "say/do ratio," which reflects project teams' histories of delivering value on time and on budget. If a team spends more than planned, blows the deadline or doesn't deliver promised quality or financial benefits, it gets a zero rating on that project. If a team has too many zeros within its say/do ratio, the governance committee takes that fact into consideration during project prioritization and tends to prioritize those projects lower. By favoring teams with high say/do ratios, the governance committee believes the investment is more likely to pay off.

Avoid draconian measures.
Of course, the committee shouldn't go overboard with its policies and procedures. A governance structure that is too constraining will backfire, alienating the business and causing people to look for ways to circumvent the process. It's important to strike the right balance. Keep the project request template as simple as possible, communicate often and ensure that everyone involved understands that the governance process is intended to facilitate resourcefulness and is critical to the success of the data warehouse.

Establish service-level agreements.
One of the main reasons some data warehouse implementations have been perceived by corporate executives as failures is not because of inadequate performance or poor data quality, but because user expectations far exceeded what was actually delivered. Service-level agreements help establish, document, monitor and manage user expectations. As with any other service, data warehouse success or failure is not measured by what is actually delivered, but rather by whether the deliverables exceeded, met or fell short of expectations. In other words, the criteria for judging data warehouse success is whether the end user is satisfied with the results.

With great power comes great responsibility. Successful data warehouse governance is guided by business and IT executives, working together, with clear lines of accountability and authority to provide strategic direction, validate priorities, commit resources, allocate funding and provide strategic direction.

Good governance ensures that value is achieved from the data warehouse investment, and that data warehouse initiatives align with corporate strategy. That is, ultimately, how the success of a data warehouse is judged. T

The universal characteristics of good governance

Participation: All functional areas are involved in governance. Equity: All functional areas and IT have an equal voice in decision making. Transparency: Information flows freely between and among all stakeholders. Responsiveness: The data warehouse governance committee and stakeholders have a shared sense of urgency. Consensus orientation: Decisions are mediated to reach a consensus on what provides the greatest benefit to the organization as a whole. Effectiveness and efficiency: Processes produce results that meet the needs of the functional areas while making the best use of data warehouse resources. Accountability: There is shared accountability for the success of the data warehouse. Authority: The governance committee is sanctioned by the executive level of the organization. Strategic vision: Leaders have a broad and long-term perspective on organizational goals and good governance. There is also an understanding of the historical, cultural and social complexities of the organization.

Betty Kight joined Teradata in 2000 as a senior data warehousing consultant. Prior to joining Teradata, she implemented governance at Union Pacific Technologies as the director of enterprise data warehousing.