By the people, for the people

Increased public awareness of data management capabilities is spurring governments to action.

by Keith Ferrell

Among the oldest artifacts of civilization are official records of crop yields, census figures and weather patterns.

Collecting and managing data, analyzing and interpreting its contents, communicating and distributing information—these have been at the heart of government for as long as there have been governing bodies.

Government, in many ways, is information, and always has been.

Today's governments—national, regional and local—are increasingly turning to data warehouses to transform inefficient, slow and outmoded information systems into supple, powerful engines. The objectives include increased efficiency, reduced costs, dramatically enhanced revenue collection, detection and elimination of fraud, and more effective and open communication with citizens.

It is no accident that the transformation in how governments handle information is taking place alongside a revolution in citizens' access to information and their ability to—and interest in—communicating with their government. The revolution works both ways: By consolidating more accurate information for operational purposes, governments are, in turn, able to provide better services to citizens.

Collecting on compliance
Even as politicians and pundits debate various approaches to streamlining tax codes, data warehouses are dramatically increasing the efficiency and capability of revenue departments.

In addition to the obvious—and obviously valuable—benefits of speed, accuracy and enhanced taxpayer communication, data integration enables governments to identify non-compliant taxpayers. The payoff, both literal and figurative, has the potential to be enormous.

Because data warehousing enables more sophisticated analysis and review of tax records by using a single source of information instead of a multitude of databases, government agencies can easily identify individuals and businesses that have not filed or have underpaid their tax obligations.

One state, for example, consolidated more than twenty separate tax-related data systems into a single enterprise data warehouse and used it to identify under-reported or unpaid taxes.

More than $40 million in previously unavailable revenue was collected during the system's first three years of operation, with anticipated ongoing capture of $10 million annually.

Beyond revenue
Just as important as revenue—and possibly more so in the long-term—is the insight that such a data warehouse provides. By aggregating non-compliant taxpayer records and examining demographic and historical patterns, revenue departments are better able to understand the contexts and causes of non-compliance and take steps to minimize it.

There are clear advantages for compliant taxpayers as well, and not just the increased ease and speed of tax-form processing or the ability to access information online. One Pacific Rim nation has used data warehousing as an opportunity to overhaul its relationship with taxpayers. In addition to improving service levels, the nation's revenue office analyzed its data for recurrent error patterns and used that analysis to root out and repair over-complicated or frequently misunderstood tax instructions.

Many governments find that a data warehouse deployed to treat a specific problem, such as tax fraud, ends up having long-reaching effects in other areas as well, not the least of which is improved data accuracy and the resulting increase in consumer confidence.

According to Thom Rubel, vice president for government strategies at Stamford, Conn.-based consulting firm META Group, "Data warehousing has the potential to curb abuse of information. When there are multiple depositories, it becomes more difficult to secure information—errors corrected in one database might remain in others. Properly warehoused, information errors can be detected by conflicts in information."

One state used its data warehouse's ability to cross-correlate and cross-index the records from different departments as a means of cracking down on parents who failed to pay mandatory child support. By comparing names of delinquent parents with names in drivers' license records, the state was able to find the parents and hold them to their responsibilities.

That kind of information correlation is a significant and ongoing benefit to both governments and the citizens they serve. "Governments are held to the highest standards," Rubel notes. "People shouldn't have to address a problem by interfacing with several agencies, some of which have their information wrong. They shouldn't have to run all over the government to find out why their benefits check hasn't come."

The ability to use IT tools to communicate with their government as easily and efficiently as they communicate at work, in the classroom and commercially not only produces a more informed citizenry but also enhances government's relationship with its people. Perhaps CRM should be given an additional definition for the government sector: Citizen Relationship Management.

The logic of logistics
Even the largest enterprise inventory and supply chains are dwarfed by the volume of goods and materials—from classroom supplies to aircraft carriers—processed by state and national governments. And, of course, as constant as the flow of materials is the flow of change-orders, new needs, shifting priorities and fluctuating budgets.

With the help of enterprise data warehousing, agencies and departments are approaching the efficiencies that the commercial sector has long expected from the technology. Inventory control has become far more precise, and the long-desired goal of accurate, immediate analysis has become a reality.

One major department of the United States government spent two years developing and deploying its EDW, centralizing dozens of legacy systems and silos. With its EDW in place, the organization now has a clear view of its inventory, logistics requirements and the infrastructure that supports it—to the tune of 16TB of data and 70 million updates per month. From this vast repository of information, the organization generates 20,000 weekly analytical summaries, each providing finer-grained insights than was previously possible.

The organization expects the EDW to have fully paid for itself within six years of its deployment and to generate ongoing annual savings for the foreseeable future. Some of those savings go beyond the governmental group itself. By integrating its EDW with key suppliers, vendors and contractors, the government was able to work more easily with private sector entities, generating savings on their side of the transaction as well.

Heading in the right direction
Tax compliance, social services and logistics are just three of literally hundreds of thousands of government programs and initiatives, virtually all of which have the potential to be transformed by data warehousing's capabilities and applications.

Already those transformations are being seen in medical and scientific research, academic record-keeping and educational operations, environmental resource management and more.

These efficiencies are enabling government agencies and departments to provide better programs and services to citizens, while at the same time improving information access.

Jon Mitchell, an analyst with Dayton, Ohio-based IT consulting firm The Greentree Group, believes government-driven data warehousing proves its worth in data analysis alone—and does so almost before it's asked.

"Traditional unstructured data analysis usually involves (putting) 80% of the effort into data gathering, with 10% in analysis and 10% in reporting," he explains. "Our goal is to spend 80% on analysis and 20% on reporting—with the data warehouse doing the brute force of data gathering."

As near real-time EDWs become more common, the speed with which governments will be able to identify, react and adapt to trends will be almost breathtaking. In fact, in some areas, Mitchell believes that comparisons to other technologies become ludicrous.

"The deficiency in (traditional) IT solutions is the lack of (scalability)," he notes. "Some text miners say they process a gigabyte an hour as a baseline. With the (data warehouse) products we employ, I can load and process a gigabyte of data a minute. That bridges the gap to the functionality that large government ambitions require."

Growing by leaps and bounds
Nowhere—not even in the largest private enterprises—will you find information demands as large as a government's, and the demands continue to grow daily.

The exponential increase in the volume of information available and the variety of demands placed upon that information will further impel the transformation of government IT systems, according to META Group's Rubel.

"As information piles up," he says, "the redundancy of entering information and of storing it also piles up costs. As budgets tighten, this information pileup increases the imperative to keep costs down."

Among the costs eliminated are the tens of billions of dollars consumed by jammed stovepipe systems, undetected errors and the rising price of maintaining legacy systems whose developers and programmers have long since gone out of business. Such systems are often kept running through costly custom programs and inertia.

The best antidotes to legacy systems are the quantifiable results that emanate from improved logistics, enhanced rev-enue capture and better service to citizens. Each successful deployment leads to future applications. As those applications are planned, something else is going on—an educational process familiar to many in the private sector.

As Ken Collier, senior consultant with The Cutter Consortium, an information management advisory firm in Arlington, Mass., points out, preparation for data warehouse deployment becomes an opportunity for government to review its processes, its purposes and itself.

Collier recommends that everyone involved be engaged in the preparation—end users as well as IT professionals. "To make sure information can be taken advantage of—can be acted on—it helps to have a road map of where you want to go," he says. "The different users of the data warehouse should have some common idea of what they will do with the data when they get it."

Back to basics
What they will do with that data is as varied as the myriad services government provides and the equally large number of purposes it serves. Indeed, just as private sector data warehouse users discover, the variety of applications and abilities empowered by the data warehouse in turn empowers its users to ask new questions, seek new solutions and drive positive change.

The point is that the shift to data warehousing frees government to focus more fully on its reason for being, not on maintaining antiquated systems, trying to correlate disparate information and overcoming communications barriers inherent in older systems.

In short, the government is at last able to work with its information rather than in spite of it. But good government is, above all, in service to the people it governs. It is here that the data warehouse, in both current operation and potential applications, will be seen as transformative as well as efficient, effective and valuable.

Put it all together and you have a technology that facilitates, as state and national administrations worldwide have learned, a government that is, at last, as good as its information. T

Keith Ferrell, former editor of OMNI magazine, has written, spoken and consulted on governance and Web-enabled corporations.