Synonymous with accurate, speedy and secure online transactions around the globe, PayPal Inc. is an online payment-processing powerhouse. Part of that success is due to its industry-leading use of data warehousing—expertise that was hard won over the past four years.

"The Teradata system is the single source of integrated data across all of PayPal globally," says Clay Stanley, senior director of Information Management and Delivery.

The business units at PayPal are demanding users of the data warehouse and created a number of challenges for the company's Information Management and Delivery (IMD) team. Although it wasn't always easy, the team proved itself capable of meeting those demands. The result is an enterprise data warehouse (EDW) environment that's agile and reactive to the needs of the company's business units.

Acquired by eBay Inc. in 2002, PayPal set its sights on becoming a global payments company with world-class systems. But the company discovered early on that its business intelligence (BI) infrastructure was not up to the challenge. Although the legacy data warehousing system was adequate in the early years, by 2003 it was evident the infrastructure was not enabling all the opportunities for growth. The legacy system "was really primitive," says Clay Stanley, senior director of IMD. "It was list pulls, rudimentary analytics, and it was done with a handful of SQL [structured query language] engineers that would process requests on a first-in, first-out basis. That was data warehousing at PayPal—it was an oxymoron."

This realization triggered the decision to look for an alternative. "The change was driven by the technology side of the business, which usually is a recipe for failure," says Stanley. "But it worked because the technology executives at the time recognized that a successful payments company has to be world class in the data warehousing and business intelligence space." The transition undertaken by the technology organization was a proactive move to get the right infrastructure in place to support the anticipated growth of the business, he says.

When it came time to choose the new data warehousing solution, the selection of Teradata "was easy for us," Stanley comments. A veteran of eBay, he had strong ties to the parent company's technology staff, which had successfully moved to a Teradata system. He realized that it made sense to seek a standardized solution that could handle the integration of eBay and PayPal data and do it well.

Lay the groundwork
Coming from the operational side of the business, Stanley plunged into the project with zeal and made some very savvy choices that proved key to the transition's success.

One of those decisions was to model the data going into the EDW based on analysis of the hundreds of reports being generated by the business. Stanley's team analyzed the reports for relevancy to the business as well as to anticipate what the business would need in the future.

In addition, the team conducted detailed discussions with each business unit to capture its specific requirements at a tactical level. Another key strategy was to keep a low profile until the conversion was complete. IT's approach helped prevent requests to simply grab the data from the legacy system and load it into the EDW.

"We weren't evangelizing it yet because we didn't want to forklift the PayPal production schema into the Teradata system." As a result, Stanley's team was able to proceed slowly and model the data, anticipating what the business would need. "That allowed us to be more nimble and responsive to the business and provide a richer solution," he says. "You can get a lot of value out of modeling the data as opposed to making a forklifted schema work in a high-transaction volume, large-scale enterprise data warehouse."

As the conversion came to fruition over a 12-month period, Stanley and his team scheduled quarterly rollouts of EDW releases for various business units. The team addressed transactions and customers first, then accounting and finance, marketing and customer support. The result was a true EDW that includes data from every significant system across PayPal. That proved to be a winning approach, which enabled the team to hone its skills over many months.

When first implemented in January 2004, the data warehouse was a four-node system. An additional four nodes were added in December 2004. As of March 2008, the system has 32 nodes and operates out of the company's new data center in Phoenix. The expanded system allows PayPal to be available to customers around the clock.

To enhance the operation of the EDW, PayPal incorporated a number of BI tools. These include MicroStrategy, which is used as the reporting and dashboarding tool; Informatica; Teradata Warehouse Miner; Oracle Essbase; Unica; and SAS, which is used for enterprise risk management.

Reap the benefits
Has moving to a Teradata environment made a difference? Absolutely, Stanley says, but it wasn't without a little bit of pain in some cases. Before the Teradata solution was implemented, he noted "a growing discord in accounting and finance because their tools weren't good." The department was performing formal management reporting based on a spreadsheet program, and there was frustration because the tools weren't scaling.

Seeking a remedy, the group turned to Stanley's team. "Accounting and finance reached out to us first. We warmly embraced them and proceeded to endure a really tough three-month stretch," he recalls. These staff members "had a very specific set of use cases that they wanted to tackle in a very aggressive timeline. I had no idea the challenges that we'd be forced to face."

Among the obstacles that emerged, the finance organization wanted help augmenting the general ledger reconciliation process for key financial data including revenue, which the forecasting organization needed to analyze at a detailed transaction level via the data warehouse. To ensure the basis of such analysis was sound, finance insisted on transaction accuracy within a small number of basis points. Typically, data warehouses do not require that operational level of accuracy, but Stanley's team overcame that hurdle and in the process developed a methodology that could be replicated for other departments. "Looking back, it was a stroke of good fortune because we built a framework of quality control into our batch process that we have reused over and over again for each business unit," he explains. "By implementing to the higher data quality required by the finance group, we developed a core competence that we now apply to every business unit's data. We have an accurate data warehouse because of that early pain."

With its core data quality and integration challenges solved, PayPal now benefits significantly from its data warehouse infrastructure. The power of the Teradata system is making a huge difference. For instance, one critical application took more than 24 hours to complete on the legacy system, which caused data problems and integrity issues. When the application was ported to the Teradata system, it ran in less than two-and-a-half hours, even without optimization.

PayPal has since leveraged the core EDW into other high-value business opportunities. The risk organization found that it could run some of its scoring models within the EDW because of the breadth and depth of the data available. In addition, the product organizations found they could use payment-flow analysis to monitor the profitability of new product introductions. They can now keep track of key metrics such as the revenue and cost of new features on their Web site. The EDW is the only place where PayPal can calculate detailed revenue.

For a company that lives by its data, the implementation of an EDW from Teradata was beneficial to the PayPal business. "The Teradata system is the single source of integrated data across all of PayPal globally," says Stanley. The EDW enables the organization to model its data to support ad hoc business analysis and ensures that it is integrated and highly accurate. The EDW also helps satisfy the enterprise's insatiable demand for analytics. The business has recently increased its requirements for truly exploratory analytics. With the old data warehouse infrastructure, this could not have been possible. But with the EDW foundation in place, PayPal IMD can now scale up to meet customer and business demand. Teradata puts the power in PayPal's powerhouse performance. T

Meeting the challenge When Clay Stanley became PayPal Inc.'s senior director of Information Management and Delivery (IMD), the company was in the midst of its conversion to a Teradata solution. Because of the scope of the project and the challenge of learning a new system, the conversion effort was behind schedule. Two paths could have been taken to get back on track. One involved hiring an outside consulting firm to take over. The other was to work with the PayPal team to put the conversion project on the right timetable. After evaluating the situation and his staff, Stanley opted to work with his group. This proved to be a wise choice. Stanley added four staff members to his department and enlisted the aid of Teradata's industry consulting services. Teradata's industry consulting approach made the difference between a pure technology implementation and a business-driven one. The strategy resulted in greater business buy-in and ultimately led the IMD organization to restructure itself by bringing business analysts into the group to augment the technical staff. This is reflected in IMD's current composition, in which dedicated vertical teams act as liaisons to the business units to meet their specific needs. "Teradata Professional Services helped us to become Teradata engineers," says Stanley. "They were terrific at partnering with us to get the PayPal team up to speed. There was a lot of one-to-one between Teradata and the PayPal team." Stanley devised a new plan that the department implemented from June 2004 through March 2005. The initiative called for quarterly releases geared to different business groups. "We finished on time and on budget. That's been the genesis of our success," says Stanley. The challenges faced at the beginning of the project have paid off handsomely in terms of experience for his staff. Notes Stanley, "Had we gone with the outside consulting firm, those folks would have done the work, turned it around and left. We would have been in some serious trouble." The key experts on his team today were part of that initial conversion. Their knowledge and experience benefits PayPal daily. "While the technology has been terrific, it's the people who make the difference," he adds. —S.S.S.

Behind the solution: PayPal Inc. Database: Teradata Database V2R6.2 Server: 32-node Teradata 5400 Server Users: 1,000 (45 concurrent) DBAs: Two Data model: Logical—Developed internally Physical—Hybrid: Some star and snowflake schemas and some third normal form Operating system: UNIX MP-RAS Storage: 47TB Teradata Utilities: Teradata Tools and Utilities 8.2: FastExport, FastLoad, MultiLoad, Teradata Dynamic Workload Manager, Teradata Manager, TPump; Teradata Utility Pack: ODBC Driver, JDBC Driver, OLE DB, Administrator, BETQ; Priority Scheduler, Teradata SQL Assistant Tools/applications: Teradata Warehouse Miner and products from Informatica, MicroStrategy, Oracle, SAS and Unica

Shirley S. Savage writes on technology subjects from her base in Maine.

Photography by Saul Bromberger

Teradata Magazine-September 2008