Featured Partners The latest business intelligence, analytics and integration solutions from Teradata partners GoldenGate Real-time data warehousing: It's all about the data! KXEN Lifting predictive analytics' productivity. Protegrity Database-level encryption ensures security. Siebel Enabling the insight-driven enterprise.

Print

Send to colleague

E-mail us

Back to the future with KXEN and Teradata

by Dan Tynan

Predictability is the Holy Grail of business. Reducing uncertainty can lead to better decisions, lower costs and higher revenues. And while technology firms have yet to build a working crystal ball, the next best thing can be a robust data warehouse and predictive analytics software. Companies have found both in the team of KXEN and Teradata.

San Francisco-based KXEN (Knowledge eXtraction ENgines) offers a suite of sophisticated applications that allows companies to quickly analyze huge quantities of data and build mathematical models of customer behavior. Its software is both descriptive (telling you how customers have performed in the past) and predictive (telling you what they'll do in the future).

In an increasingly fast-paced business environment, some of the biggest companies in the world rely on predictive analysis to gain a competitive advantage.

	A leading European wireless company uses KXEN's predictive analytics to identify subscribers who may be thinking of switching carriers so it can reduce churn.
	One of the UK's largest banks employs the technology to optimize the types of offers it provides to customers via its call centers and Web site.
	A huge U.S. retailer uses the software to target mass mailings and other marketing campaigns, increasing the likelihood that households receiving a catalog or an e-mail solicitation will end up buying products.

In fact, predictive analysis is not new to the business world. But until recently, most companies built models using complex data-mining tools, requiring deep expertise in analytic modeling, massive computing resources and a lot of time. What KXEN and Teradata bring to the table are speed, simplicity and cost savings, says Doug Bryan, KXEN's technical director.

Take the case of a retailer that wants a bigger bang from its marketing buck. The process starts by sending out a test mailing—say 10,000 copies of a consumer catalog—and recording which customers make purchases from that catalog. The retailer plugs the results of the test into the KXEN Analytics Framework, which can analyze up to 5,000 variables—such as the customer's location, purchase history, demographic information and so on—to determine how each variable correlates to potential sales. The KXEN software assigns a score to each variable and produces a model that describes the households most likely to buy.

This is where tight integration with the Teradata data warehouse really pays off. For example, a single customer may have information stored in dozens of different data tables spread throughout the warehouse. Before analysts can manipulate data in the KXEN Analytics Framework, they first create an analytic data set (ADS) that will merge, reconcile, cleanse, transform and derive the data from multiple tables into the single logical table required for analysis. This is effectively done using Teradata ADS Generator, which facilitates the creation of the ADS.

The process occurs in two phases, data exploration and preprocessing. During the exploration phase, Teradata ADS Generator searches the data for patterns and anomalies, learning more about the data with each iteration. In the preprocessing stage, the Teradata ADS Generator creates the ADS by aggregating the data and transforming it into a large, flat table that can contain thousands of columns and millions of rows. The functions required in data exploration and preprocessing are highly iterative, extremely data-intensive and potentially CPU-intensive as well, which means they can quickly bog down even the most powerful machines. By using the Teradata ADS Generator, KXEN's Analytic Framework minimizes data movement, avoiding bottlenecks that would be caused by moving terabytes of data across a network.

Once the ADS is created, KXEN uses a sample of the data set and expedites model development by significantly "fast tracking" the model exploration process. KXEN can identify the most influential variables in the ADS and then rapidly build models for prototyping or exploration. The model can be exported in SQL or User Defined Functions (UDF), allowing it to be run inside the Teradata data warehouse.

KXEN's models may provide sufficient results to be used in production, or they may need additional refinement. In the latter case, KXEN can export the model into other complex data mining tools such as SAS, SPSS or Teradata Warehouse Miner, where it can be finetuned.

Once a model has been tested, refined and put into production, the retailer can use it to identify other households within the data warehouse and pull out, for example, the top 100,000 households or only those scoring in the top 10%.

When each piece of direct mail can cost $2 to $3 and the target market is 110 million U.S. households, the cost savings can be tremendous. But that's only one of the benefits of using KXEN's predictive analytics.

Unlike with many business analysis tools, you don't need a Ph.D. in statistics to handle KXEN's Java-based client interface (though familiarity with statistical and quantitative analysis is advised). Analysts can drill down into a model to see how each variable scored and find the ones that matter most with just a few clicks of the mouse.

"We've automated many of the most difficult parts of predictive analytics," says Bryan. "People who aren't expert in algorithms or normalizing data are able to use our software. Many of our users are business analysts working in a marketing department."

Better yet, KXEN's Analytic Framework can produce prototypes and potentially production models in a matter of days, instead of weeks or months. That means companies can apply predictive analytics to a wider range of business problems, using data that's as close to real time as possible.

"Predictive models used to take two to three months to develop, which meant companies could only build a few of them each year," says Bryan. "But people make purchases every day. With technology like KXEN and Teradata, you can update your data warehouse and your models daily, so with every mailing you're using your best insights about customer behavior."

By using KXEN and a Teradata data warehouse to build predictive models for its marketing campaigns, a large retailer was able to cut its mainframe processing costs by 50%, while slashing the time needed to build new models by 90%. Now, instead of building a handful of predictive models each year, the firm uses them to drive all of its direct mail marketing projects.

One of KXEN's clients, a wireless telecommunication company, slashed the time it needed for building models by 70%, simply by eliminating nearly all of the time it used to spend cleaning and reconciling the data. By identifying customers receptive to loan or insurance offers, its banking customers realized a 15% sales boost on inbound calls.

Arlene Zaima, advanced analytics program manager for Teradata in San Diego, says that once Teradata builds the analytic data set, KXEN's Analytic Framework can accelerate the development process by quickly prototyping analytic models. Companies can use the models as they are or refine them with another data mining tool to maximize model accuracy.

"KXEN can significantly reduce any company's model development cycle," she says. "It's a great way to kick start a data mining project." T

Dan Tynan writes about technology from his home in North Carolina. He is the author of Computer Privacy Annoyances (O'Reilly Media, 2005).

© Teradata Magazine-March 2006

back to top