Teradata Blog Feed

Yes, we are all different!

David Placke — 11/6/2009

I’ve been pretty quiet lately on Twitter and haven’t written a blog for a while. While there are many others who have formed the habit of fitting these activities in with their (I’m sure) overly busy schedules, my excuse/reason is that we have just been through the very pointy end of winning a major new customer.

The good news is that we have come through with the right result (more to come on that later when we have something to announce). I’ve been through this process a couple of times, and it is “all consuming” (even dreams end up about the sales process), hence my silence.

Unfortunately the process didn’t conclude in time to allow us to get our new customer to Teradata Partners. I really wanted to get there to provide that sense of confidence in their decision that only another customer can provide, and there is no better place than Partners to talk to customers. I also wanted to personally thank the many of my colleagues and account teams that helped us win the deal.

One of the many write-ups that came out of Partners came from Dr. Mark Whitehorn mulling over continuing speculation of an SAP acquisition of Teradata. He articulates four factors that any potential acquirer would need to make sure they understood in evaluating Teradata:

Teradata has only just become independent
The customers are unusual
The company culture is different
The people are different

The key summary point for me is this:

Teradata is genuinely different from most of the other BI companies -- it has a very unusual set of customers, a very unusual working/research relationship with them, a very different management ethos and very different staff. It is these differences that make it what it is and also make it successful.

The reason this resonates with me is because it is the key reason why we won our new account.
I’ve been at (or around) Teradata now since 1994. It really wasn’t until I went to my first Partners conference in 1996 that “I got it”. Since then, being part of the Teradata culture really has meant being able to tap into those points that Dr. Whitehorn calls out. In the sales campaign we had conference calls with the Teradata executive and many conference calls with accounts and account teams in Asia, Europe and America. Everyone is keen to contribute whatever they can to help grow Teradata.

And that’s what makes writing the “Thank You” email so enjoyable.

And already this week we are putting back in to help an account team in Europe.

Sean Kain

Am I a Dinosaur already?

David Placke — 11/3/2009

Last month I attended the 2009 CIO Executive summit in Sydney. There were a number of interesting and challenging presentations including the closing keynote by Andre Mendes, Global CIO of Special Olympics International.

Andre talked about a wide variety of topics and subjects from Genomics to Immortality, some of it quite challenging and all of it interesting and thought provoking. He also used a personal example of how he went from a young, determined and ambitious 24 year old to a 37 year old ‘dinosaur’ who suddenly started looking at the ideas from the 24 year olds in his team with a ‘that won’t work’ attitude.

I am now of that certain age and find myself being a bit grumpy about Cloud computing, something that naturally got a mention in Andres presentation. I don’t want to be a dinosaur - I genuinely believe the cloud has the potential to be disruptive and game changing and Teradata is already making moves into our cloud offer with Amazon EC2 but I have a couple of issues at the moment with some of the positioning around the cloud.

First security. I am getting increasingly concerned at the soundbite stating that security of data is the number one issue to consider around Data Warehousing in the cloud. Second, I don’t think this is the ‘number one reason for discounting the cloud for Data Warehousing or Analytics. If it’s yours then you probably aren’t really considering a proper Data Warehouse or real Business Analytics.

That security soundbite makes it sound as though the technology is not capable of the required security. I’m happy to stand up and say that Teradata can be used in a cloud environment and the data in the Teradata database can be secured from unauthorised access. Indeed our customers regularly use their internal clouds and secure data between business units and departments. Certainly we are more than up to the task of protecting any data and can be further encrypted using the capabilities that our partnership with Protegrity enables.

This is not the barrier that prevents organisations from moving into the cloud. The security worry is not a technical one, it’s a legal one. The small print of many cloud providers has to enable them to give up your data if they are mandated to by an authority that could be in a country that you were unaware your data had entered. I think this legal issue is the ‘security concern’. It’s important but not the number one issue.

On the second point I recall the numerous ‘lessons learned on outsourcing’ presentations I have attended in the last 5 years which all indicated that commodity business process is ripe for outsource but that IP and competitive differentiators should be guarded carefully. A Data Warehouse used effectively for actionable and operational business process carries both IP and requires strong service level agreements, neither of which put it in the ‘ripe for outsourcing’ category, nor therefore first in line for movement to the cloud. IP and service level guarantees are, in my opinion more important considerations for Business Analytcs in the cloud.

When we talk about the 5 stages of Data Warehousing, we talk about a chasm between stages 1 and 2 and then between 3 and 4. This is a cultural and all too often a technical issue that organisations get into. Whilst organisations can consider some basic reporting as a commodity that could be put into the cloud, they need to be careful that they do not inadvertently trap themselves and make it impossible to move through to value adding Business Analytics and operational intelligence.

In his presentation, Andre Mendes talked about his use of the cloud. In the case of his organisation, there is likely a need for some operational intelligence but the IP conversation is probably less relevant in a not for profit than a commercial business. I think the commercial sector will be watching leading organisations such as his for the lessons learned and best practices in this field.

What is your experience in the cloud – what do you think are the main issues to be addressed?

Alec Gardner

4 conferences, 4 continents, 3 weeks

Paul Barrett — 11/4/2009

My trusty little model t PC gave up the ghost this weekend with a blue screen of death so I am writing this on my MAC that just doesn’t seem to do these things. But I love spending my weekends trying to reconstruct my PC - don’t you?

It has been an exciting few weeks where I got to be surrounded by fascinating, interesting people with incredible insight into the changing online world around us. It reminds me of the days when Skiers and Snowboarders first clashed, but then adopted each other positives to transform the extreme sports world. Online and offline marketers are a few years behind, but when they come together there is going to be an incredible spark of innovation and insight that will transform marketing.

Between the ETIS Conference, eMetrics, CTAM and, of course, Teradata PARTNERS. It has been really cool to see so many ideas presented, audiences engaged and concepts come to fruition. My only regret is that I couldn't be at Webtrends Engage in London. I am sorry I missed Chris Jenkins from Vodafone UK, Zeljka Stojanovic from Total Jobs, Christian Howes from Webtrends and Eduoard Servan-Schreiber speak. Fortunately I got to meet all of them separately outside of the the Engage Conference.

To some extent, I guess I got the virtual Engage experience by speaking at eMetrics with Christian Howes on the demand for integrated online and offline marketing analytics. Christian does a much better job at the describing the analytics rock-october fests in his blog post than I could ever imagine. Leaving only the question “do all Englishmen find pumpkins so fascinating?”. Let’s just say I hope not. By the way, watch this blog for a guest posting by Christian in the coming days.

eMetrics was great having a chance to finally meet Jim Sterne and Eric Peterson. Jim and I have a white paper coming out in the near future about bridging the online and offline gap in your enterprise. Hopefully we can post about it soon. My favorite part of the eMetrics conference was the closing panel discussion with practitioners from a wide ranging set of disciplines. One of the insights coming from that panel was that the challenge to integrate online and offline analysis is more of a people and process challenge than a technology issue. John McKean kicked off that discussion - he is the moderator of The River, a site completely focused on online and offline integration. This is an invitation only site, so Drop John a note and join the conversation.

Teradata PARTNERS was incredible as usual. Teradata customers put on this event and they are always the highlight of the show. It is too bad anybody missed Daniela Calaes from VIVO’s presentation. VIVO is a leading wireless provider in Brazil that has used Teradata to successfully enable multiple strategic business initiatives. VIVO has really used its Teradata investment to drive ROI across the organization.

While at Partners, I had the chance to present with some of the most interesting people working on transforming the marketing space into an integrated, data driven, automated environment. Doug Dorrat from Microstrategy and I did a workshop on new reporting tools for the customer experience. It seems like there is a lot of demand for new tools that bridge this gap between different marketing groups and Doug has some great thought leadership in this area.

On social media, I got the chance to work with a great panel including Sheryl Kingstone from Yankee Group, Christine Baumler from Freddie Mac, John McKean and of course the great James Semenak. Sheryl has been doing some ground breaking work in the convergence of CRM and Social Media and has staked out a clear vision as to how this convergence will transform Social Media into a pervasive marketing technology.

Lastly, our session on real time marketing “Customer Time is now Real Time” was really fascinating because of the insight that Laura Squier (spelled it right this time, I think) from KXEN and Diane Strahan from Nuestar. Diane’s insight into the habits of Generation Y and their mobile phones really framed the challenge that carriers and marketers face in the coming years as customers expect more and more mobility and real time service.

Finally, if you get a chance, check out some of the Podcasts from Partners done by the BeyeNetwork. Rebecca Bucnis, Malcolm Duckett, Christian Howes, and I got a chance to speak with Claudia Imhoff - which was really a treat. Hopefully you might enjoy some of these as well.

CTAM was cool - by that I mean snowy. It was in my hometown of Denver, unfortunately right in the middle of an early season upslope snow storm that dumped 2 feet of snow at my house. I didn’t get to as much of the conference as I would have liked since I was spending my time shoveling October snow and building snowmen. But I really enjoyed the Walter Kaitz dinner. TJ Holmes was the anchor and he would have been great if he knew he was at CTAM instead of NCTA.

Did you get to any of these conferences? If you did, let me know about your experience, your highlights etc. What did you find fascinating?

Paul Barrett

Thanks for the memories

Sandra Worley — 11/2/2009

It’s been more than a week since PARTNERS concluded, and I’m still coming down from my Teradata buzz. At numerous presentations and informal chats, I learned about the exciting, innovative ways Teradata users are leveraging data. That’s certainly a theme for anyone familiar with Teradata Magazine. But even when I think I’ve heard it all, I learned in one session about a retailer that has developed a formula to predict sales of items down to the hour of the day. As a result, analysts will be able to track items that are selling at an atypical rate or when something is about to become out of stock at one of its stores. They will then be able to alert the store manager before the day’s purchase orders are made. This was just one of many innovations I learned about. As I sift through my copious notes, I find myriad nuggets of these types of ingenious initiatives.

At the same time, I met so many dedicated, brilliant business intelligence (BI) professionals. It was a veritable who’s who of data warehousing’s best and brightest—consultants, thought leaders and industry analysts, as well as Teradata’s customers, partners and associates—too many to name. Many of these luminaries have contributed or plan to contribute to Teradata Magazine, which is exciting. But even more rewarding was the positive feedback I received regarding the magazine. It’s high praise indeed coming from this group.

Alas, however, this moment in time is over, and all we can do is wait until next year for PARTNERS 2010, which is Oct. 24-28 in San Diego. Or do we?

For those of us who want relive PARTNERS—or those who couldn’t make the event—more options than ever are available to tap into this experience. Event photos can be seen at Flickr.

Several complete sessions can be viewed at Ustream. And short video reports and more are available on YouTube. Bringing together all of these social networking threads—along with blog posts from the conference—is a new interactive online hub, PARTNERS Connect.

So while PARTNERS 2009 may be over, opportunities remain to gain insight from the event, and gear up for 2010.

Mike Westholder
Managing Editor
Teradata Magazine

Supply Chain Facts and Statistics – The opportunities that lie beneath.

David Placke — 11/6/2009

Are you ready to self fund your business improvements?

In my experience I have found that there are simple steps an organisation can take to self fund their ROI on data warehousing and subsequent supply chain improvements. Let’s start with some recently reported facts.

At a recent Supply Chain conference, Smart Conference 2009, a survey was discussed where senior executives had been asked for their top priority supply chain initiatives.

70% highlighted Supply Chain visibility as a key improvement objective
55% rate cost containment / reduction as highly important
60% identify risk management and risk awareness as key initiatives
58% were actively working to adopt demand driven operations eg: produce only to demand rather then try and forecast what people might buy

In fact each of them agreed that their overall goal was to improve customer service as a foundation for customer retention whilst exploring internal and external processes to identify opportunities for cost reduction but equally ensuring that risk is properly managed.

As you might expect, these initiatives don’t materialise unless you have a clear insight into your business. Some call it a single point of truth. Others have likened it to hovering above your business in a helicopter so you can spot every opportunity below you.

It’s not rocket science to recognise that this is where data warehousing plays a central role. Essentially data warehousing brings all your business data into a single repository from where analytics turns everyday data into business insight. It’s the equivalent to that helicopter or better still, it’s a fleet of helicopters because your business insight can be made available to your operations people so that effective decisions can be made where it counts – at the customer facing level. Put another way, everyone becomes a qualified pilot.

So assuming you have identified the supply chain benefits that can be realised, how can your data warehousing initiative be immediately self funding? There are a number of possibilities and it is worthwhile taking a look inside your own business to explore the potential. Here are several existing real world examples that might provide some stimulus.

Oh and before I forget, all of the following can be achieved long before you recognise the benefits flowing from your supply chain improvement!

UNDER THE DESK REPORTING: Often I find situations in organisations where every business unit supplements existing standard reports with ‘under the desk’ spreadsheet analysis and reports - this can be a significant cost just in administration alone. Data warehousing eliminates the additional administrative burden and you can realize an immediate flow of savings to fund the data warehousing project.

REMOVING DATA SILO’S: Unable to obtain a single answer on the performance of your supply chain? If every business function has a separate data store no wonder each set of data is different from the next! Transfer all this information into a single data warehouse and not only get one answer but also save opex by annualising support costs.

INTEGRATING ACQUISITIONS TO DRIVE SYNERGY BENEFITS: Where an organisation has a strong acquisition growth strategy, by transferring acquisition data alongside group data the company can easily compare and identify synergy opportunities.

MONTH END CONSOLIDATION: Many organisations exist as a necessary combination of disparate organisations collectively operating as a group of companies. Whilst this can have significant operational benefits the challenge of month end and year end financial reporting can mean a lot of hard work for the finance team. If all the business data is transferred into a central group data warehouse, then month end accounting can be completed in a matter or 3 or 4 days.

In so many situations gaining a single view across the entire business is the starting point for recognising Supply Chain improvements and managing associated risk. Moreover, such a move sets the foundation for empowering operational managers with business insight to ensure decision making can be affected through every level.

Obviously any such move has to be supported by strong ROI, and that should always be the case. But as you can see there are several other areas where immediate ROI can be wrought, from administration productivity, and removal of data silos, to integration of acquisitions and more efficient month end consolidation.

There are too many examples of easily achieved and immediate ROI to list here – but please contact me if you would to discuss further. In the meantime, I would encourage everyone to follow the experiences outlined above; you will probably be amazed at what opportunities lie under the desk – so to speak.

Do you have any examples you can share?

Download Data Mart Consolidation White Paper

Robert Heaton

*Image Source: BBC News

I Have a Query

David Placke — 10/29/2009

Most of my conversations during the Washington conference last week were about Teradata’s cloud computing offerings and the significance of the new platform, the first all solid state drive (SSD) data warehouse appliance. We thought that people elsewhere might like to get the same additional explanations, so my colleagues found a quiet corner where they could film one of my conversations and produced a brief video from this material.

If you’re interested you can watch me explain why solid state disks make data warehousing so much faster (it all goes down to the von Neumann problem, as I have written before) and elaborate the difference between private clouds and public clouds. I intend to write a longer post about the latter topic soon, but for the time being you might find this video helpful.

Martin Willcox

Recap of Teradata PARTNERS Conference - This year, there’s a lot to love

Rick Loconto — 10/30/2009

The Teradata team was busy with announcements last week at the annual PARTNERS User Group Conference – from the world’s first solid state data warehouse appliance to new Cloud Computing and vertical industry offers. The headlines below give you other highlights of media coverage of the PARTNERS Conference story. Check out the stories below, and go to www.teradata.com for more on the new announcements.

Teradata Kicks Off Annual Partners Conference with New High-Speed Appliance and Cloud Computing Offerings
Teradata: We still rule over Oracle, IBM in analytics:At conference, says it will offer its data warehousing technology for private corporate cloud
Teradata Labs Chief Downplays Database Machines From Oracle, HP, Netezza:Teradata R&D boss Scott Gnau dismisses competitors
Teradata announcements target public, private clouds
Teradata unveils SSD Blurr appliance
Data warehousing specialist promises cloud flexibility (with limits), and (oh by the way) solid-state hard drive speed
Teradata Bolsters CRM Solution

Since PARTNERS 2009 took place in Washington, D.C., we had government-focused sessions and an event at the National Press Club for government-focused media. Here are a few of the stories:

National Press Club Luncheon: The Role of Technology in Achieving Government Transparency from Teradata on Vimeo.

We ended the week with a great article in Barron’s Magazine. I guess this headline refers to pterodactyl, the flying dinosaur. Actually, the only thing Teradata and that dinosaur have in common is Greek roots for the names. But that’s OK – I loved the Barron’s article. Just read the first couple of paragraphs and you’ll see why:

Sounds Like a Dinosaur, but It’s No Small-Brainer

“Thousands of technology geeks representing some of the biggest companies in the world gathered at a convention center in Washington last week to swap stories, compare notes and make wish lists about their experiences with data warehousing and analytic software applications. Oh, and they had something else in common: they are all clients or partners of Teradata , the undisputed leader in the field of business intelligence. Known as the Partners Conference, the event is the world's biggest confab for data-warehousing and data-mining specialists. Some of Teradata's biggest competitors, including Oracle (ticker: ORCL) and International Business Machines (IBM), ponied up big bucks for the chance to impress the attendees last week -- at the 24th annual meeting of what started as an informal user group of about 25 people.

“In the 30 years since it was incorporated, the result of a collaboration between researchers at CalTech and the technology group at Citibank, Teradata (TDC) has been helping companies gain a competitive edge by analyzing the raw data from their business operations, identifying inefficiencies and providing solutions using super-fast and super-sophisticated software applications.”

If you want to read more about all the announcements we made last week, check out our Conference Newsroom – a lot of good info here on customer wins and partnerships as well.

I’ll share more highlights in another posting, but as you can tell, we had a great event. Look forward to 2010 in San Diego!

Darryl

The World’s Weirdest Group Hug: U2, Big Pharma, Broadband Cable Providers, Youtube & Me!

Colleen Quinn — 10/27/2009

I live in Pasadena, a sunny Southern California city known for hosting the annual New Year’s Day Rose Parade, and a college football Bowl game which follows. I live dangerously close to the Rose Bowl, a massive outdoor stadium which seats 100,000 fans, and hosts the annual game, the local UCLA Bruins, and various other events throughout the year.

In my short decade in Pasadena, residents have become increasingly, shall we say, irritated with loud rock concerts, soccer hooligans and general mayhem resulting from drunken crowds staggering around the Bowl, so it’s not surprising that a recent crack-down in evening events and a strictly enforced noise ordinance have meant that little much fun has happened at the Rose Bowl since the Stones and Pink Floyd raised hell in 1994.

So, imagine how THRILLED I was when I learned back in April that U2—yes, Bono, Edge, Clayton and Mullen—would be gracing a stage just a stone’s throw from my house. I bankrupted myself on tickets, and started the countdown.

All went according to plan until the day of the event, approximately 6 hours prior to show-time. I was listening to the strains of the sound-check from my backyard (trying to explain how “U2” was different from “Me Too” to my three-year-old) when it happened. I became suddenly, unfailingly, painfully ill. All at once. Like being crushed by a truck.

It didn’t matter. I’d waited twenty-odd years, and I was not going to miss this show. So, I staggered to the event. Literally, staggered. I sat on the ground, head between knees, waiting for a queue of 20,000 ticket-holders to snake their way to the entrance. That took over an hour. I shivered as I waited in a concession line for a bottle of water and pretzel, listening as the Black Eyed Peas took the stage. I felt so awful by the time I crawled to my seats, that I clutched the aforementioned pretzel so hard it was reduced back to dough.

I managed to hang in there until U2 took the rocket-ship-shaped stage looking like black specks in the distance—not before announcing what I knew already. This event was being streamed live to 16 nations representing every continent on Youtube.com. We, the mass of humanity in the audience, were instructed to sing-along, look happy, dance “loudly” and show our utter devotion to the band, all in the name of solidarity and world peace. I squeezed my pretzel harder, and remembered my 10Mb/s connection at home. I had my first thought of joining my estimated several-hundred-thousand peers—watching through the assistance of three content-delivery-networks to reduce traffic interruptions—from the comfort of my home.

Without question, the band is a collection of rock icons. They sounded incredible. They looked cool. But, I’m getting ill-er (if that’s a word) by the moment. And, the man next to me is dipping his French-fries in Sriracha “Rooster” Sauce, making matters worse for me all the time. I had to leave… and now.

So, with tears in my eyes, I staggered back home. It’s a lonely walk when your favorite band is playing in your back yard, and you’ve ransomed your child’s college fund for tickets. I made it back to the house, opened the windows and heard “One” wafting clearly through the night sky.

Then, I flipped open the laptop. Audio was delayed a few seconds from what I could hear happening out my window, but I could see—I mean actually see!—Bono leaping around like a man half his age. Then, I launched Twitter, allowing its real-time feed to complement my broadband viewing and backyard lullaby. As wistful as the comments made me feel, I felt like my U2 experience had not only been saved from certain disaster, but was some how richer. I was hearing, seeing, and chatting with friends at the venue from the comfort of my own bed. I wanted to embrace Charter Cable, technology innovators of all kind, and big Pharmaceutical all in the world’s biggest group hug.

Then, I shut off the lights. I think I dozed off to the sounds of “With or Without You” floating through my window. Another case of technology saving the day. Now, it’s time for a visit to the doctor!

Colleen Quinn

Good-bye, Washington!

David Placke — 10/27/2009

The Teradata Partners Conference 2009 is over, and my impression is that I have met a lot of optimistic, and very determined, people. This is no time for enthusiasm, neither in the U.S. nor back home. But the people at this conference generally think they are seeing light at the end of tunnel and are preparing to be ready when more opportunities arise again. It was very inspiring, and shouldn’t I say thank you to our hosts?

The theme of this conference, in my view, was the discrepancy between the abundance and availability of data in many enterprises and government authorities. Knowledge you don’t know you have is wasted. Not putting the small bits together inevitably means you will be missing the big picture. In Washington we have seen enhancements for all the steps that are necessary for data analysis.

The first step is always data integration. And it’s never fully completed. Geospatial data is yet another data type that can be loaded into the data warehouse for analytical purposes. And no doubt there will be more to follow in the future. The second step is to decide what to do with these data. Thanks to the cloud versions introduced this week, business users and developers will have it far easier to make that decision. And they can collaborate via the cloud. The third step is to have the data at your hands when you need them. This is where the new platform, Extreme Performance Appliance 4555 comes in. Its SSD technology speeds it up tremendously, enabling companies, for example, to exploit opportunities even during the briefest interaction with the customer.

Data warehouse is a field that continues to develop fast. There will be Teradata User Group meetings across Europe over the next months and, next spring, our Teradata Universe Conference in Berlin, where we can continue discussing the latest trends. The next Teradata Partners Conference will take place in San Diego – a bit further to the West and very close to Rancho Bernardo with its developers’ centre. See you around, and goodbye from your man in Washington.

Mario Bonardo

Using data to predict the future

Sandra Worley — 10/22/2009

Data is everywhere. This is not news to me, but after attending this, my first PARTNERS conference, I now better understand its relevance. Data is more than simply a customer’s name or number, address or even buying habits. Organizations that use data analytics to their advantage can move past reporting what happened and toward proactively predicting what will or might happen.

During his session “Rethinking BI to Drive Better Decisions,” Gartner’s Garret Herschel explained that most decisions are driven by events—a product’s life cycle, the budget process, etc. The challenge lies in the unplanned events. Instead of dealing with symptoms, he said, organizations should identify the causes of their challenges so they can prevent them in the future.

After listening to Herschel’s presentation, I really began thinking about Teradata customers. What do they use the company’s solutions to enable them to do? Here are some examples based on sessions I attended:

France’s Bouygues Telecom combined disparate databases into a large Teradata enterprise data warehouse (EDW) that is one of the largest in France. The telco did so to reduce costs, improve service levels, share operational data, improve access, become more agile and benefit from operational BI.

Telefonica Chile is part of Telefonica, the largest Spanish telco, which boasts 260 million customers through Europe and Latin America. Telefonica Chile is using Teradata Relationship Manager to reduce customer churn.

The United States Department of Defense’s Defense Logistics Agency and the U.S. Transportation Command deployed a Teradata EDW to modernize aging legacy systems and help the agencies better plan workflows—again through predictive analytics.

STIF, the government organization responsible for public transportation in Paris, implemented its EDW to help reduce the cost of marketing campaigns and centralize information on transportation use and organization routes. Using Oracle Data Integrator as the data hub, a Teradata EDW and IBM Cognos reporting tools, the organization is working to better analyze traffic and identify trends.

After this conference, I may never look at Facebook, my hotel room key or even my doctor’s office in the same way. All three have my data—different parts of it, but it’s data about me nonetheless. I wonder if the next time I stay at this hotel, the room service will be ready with my oatmeal and fresh fruit, predicting that I will place that breakfast order for delivery at 7 a.m. I’ll let you know.

Let us know if you’re doing something cool with your data that we can highlight in Teradata Magazine. We’re always looking for great stories about the ways customers use Teradata technology to improve their organizations.

Evelyn Hoover
Editor-in-Chief
Teradata Magazine

News from Washington

David Placke — 10/21/2009

So, these are the latest news from Washington! (I always wanted to announce this one day.) The most important news story in town, at least in my view, is this: Teradata now supports cloud computing concepts. With three new tools just introduced at the Partners Conference, you can build cloud environments on Teradata, both private and public ones. This provides business users with easy-to-set-up, short-term virtual data marts to do on-the-spot analyses, possibly in response to a certain business situation. Such a capacity will make enterprises more agile.

These are the tools: Teradata Agile Analytics Cloud allows users to create virtual data marts in a Teradata private cloud environment, where they can do analytics in a “sandbox-style”. This flexibility is enabled, among other things, by Teradata Active System Management, which optimizes priorities between experimental analytics and production workloads. The other two cloud tools make Teradata Express, a free developers’ version of Teradata Database, available in VMware and Amazon EC2 environments. The first one enables internal private clouds for developers and IT operation managers, while the latter is meant for the public sphere to support collaborative work from customers around the world.

The idea behind these tools is that we want to give our customers as many choices as possible to leverage the power of Teradata. Obviously, we think an integrated data warehouse is the best solution for many reason, some of our customers might opt for a different kind of architecture. And we strive to deliver for all needs – the same reason why we introduced our platform family.

Geospatial data
The other big news is that a new Teradata solution, Teradata Geospatial, integrates geospatial data into the Teradata Warehouse, making it possible to analyze them there along with other data. Most data (about 80 percent of corporate data) have a location reference – but until now it has been difficult to extract the whole business value from this. The reason is that geospatial solutions have typically been implemented in departmental silos or data marts. And the analysis of such segregated data inevitably creates untimely, imprecise, and inconsistent business intelligence. The Teradata geospatial solution thus reduces complexity and decision latency, because it eliminates specialized geospatial data marts, unnecessary data movement, and system integration.

This will give new analytic power to business analytics. For example, courier services will find it easier to locate and re-route packages to another address if the customer requests so – thus increasing customer satisfaction. Or have you ever got annoyed because of poor phone receptions in certain areas? Well, other people will certainly have had the same problem. Many have changed their provider because of this – but have they told them why? With geospatial data, mobile phone providers can figure out by themselves and do something against further customer attrition. And there are many more examples like this.

Mario Bonardo

Stein tells of doom and hope for the future

Sandra Worley — 10/21/2009

Economist, author and actor Ben Stein, likely most famous for his role in “Ferris Bueller’s Day Off” (remember “Bueller? Bueller? Bueller?”), sees doom and hope simultaneously. Known for his dry delivery, Stein took the stage at PARTNERS today to deliver a keynote address to a packed room.

Stein touched on the decline of education in the United States. A teacher for several years, Stein told the audience he gave up the profession because students didn’t do their homework. As evidence to support his point, he cited a study that says that one-quarter of U.S. high school students are capable of obtaining C’s in college. And then he told of his experience with a reality-type TV show starring models, two of whom he spoke to who didn’t know when World War II was or what language was spoken in London.

This is in contrast with Don Tapscott, who spoke Monday, telling the audience that this generation of students is the smartest despite their use of video games and online learning aids. Putting the two on stage together might be an interesting event.

An economist at heart, Stein says that at the end of the day we have no idea what will help turn things around: “You can watch CNBC all day long, but we don’t know what really works to stimulate the economy.”

Small-business lending is down, another indicator Stein perceives as negative. Couple that with the uncertainty about President Obama’s healthcare plan and the possible nuclear armament of Iran and other “mortal enemies,” and Stein isn’t seeing a bright future.

Where, then, lies hope for this country? Stein sees it in what he calls “real stars.” He’s not talking about the TV and movie stars who make their living in Hollywood. Instead he’s referring to the men and women who daily don body armor and risk their lives for little pay in Iraq, Afghanistan, etc.

During his frequent trips to Washington, D.C. (the site of this year’s PARTNERS conference), Stein, a resident of Malibu, Calif., frequently stops at Walter Reed Army Medical Center, where he visits with hospitalized soldiers and their families, which he refers to as the “backbone of the United States of America.”

Of the soldiers, along with police officers, firefighters and teachers, Stein says, “It is in their hearts, in their courage and their sacrifice that salvation lies.”

Evelyn Hoover
Editor-in-Chief
Teradata Magazine

Transparency and accountability in government

Sandra Worley — 10/20/2009

Last fall’s U.S. election cast a spotlight on technology, making it a focal point for politicians as well as the electorate. Data is the enormous story, and its applicability to our everyday lives is examined in Stephen Baker’s book, “The Numerati.” Baker, a senior writer for BusinessWeek, wrote the book after researching a mathematics article for the magazine. He posits that through understanding data, people will better understand themselves.

Some car insurers, for example, are giving drivers the option of installing a “black box” in their cars to monitor driving habits. In exchange for doing so, the drivers receive a discount. Those who want to remain anonymous are going to pay more down the road, according to Baker.

Baker was part of a technology in government panel held at the National Press Club today. Joining him on the panel, which was moderated by Teradata Vice President of Government Affairs Tim Day, were Teradata CTO Stephen Brobst and Stephen Horne, vice president, Master Data Management and Integration Services, Dow Jones Enterprise Media Group.

Myriad areas of government could benefit from increased transparency and accountability, according to the panel members. Brobst used healthcare as his example, saying that using the past to predict the future and then making decisions based on the analysis of that data could be used to make healthcare more proactive.

Horne focused on financial reporting issues. The government is what he described as “data rich and information poor.” Scores of reports are generated each year in accordance with government regulations, but they simply sit on shelves. No attempts are made to take that data and use it to solve problems. HR 1242 has been introduced in an attempt to change this, creating a database that could then be analyzed to provide meaningful, actionable information, Horne says.

While individual pieces of data may not mean anything, larger amounts of data from multiple points, once filtered to get to the truth, can lead to major insights, according to Horne.

Evelyn Hoover
Editor-in-Chief
Teradata Magazine

Welcome to the Teradata PARTNERS Conference 2009!

David Placke — 10/19/2009

There we are, on the river Potomac, a few miles downstream from the United States’ political centre. Tomorrow, we will venture right into the heart of the capital – one of its cardiac veins, at least – the National Press Club, for a continuation of the testimony I have mentioned in my last post. In the meantime, the data warehousing community has been assembling here in the Gaylord National Resort and Convention Center where most of our sessions will take place. For the moment, I can say little more than that the global community looks a great deal bigger than its EMEA subset. I have promised to shake many hands in this city, and I wasn’t idle at last night’s welcome party.

This morning, our CEO Mike Koehler officially welcomed the attendees and put much emphasis on speed, or rather on timely availability of information. It’s one matter, for example, to know your customer in detail, and another matter to be able to utilize this knowledge while the customer is visiting your website. During this short period, you might want to find out what would be the right offers (or ads) for this customer at this very moment, before the customer leaves your site. To do this, you will have to analyse both the click-through data and the rest of your customer data. This requires real-time capacities that used to be rather expensive. The technical reason for this is that, at some level, hard disk drives are a bottleneck in any data warehouse architecture, as Martin Willcox has elaborated in one of his posts on this blog. Teradata has found a faster way of breaking through this “sonic wall” by utilizing solid state drive (SSD) technology, for our new Extreme Performance Appliance 4555. Unlike hard drives with their electro-mechanical tasks, flash drives don’t produce delays when writing and reading data, making intelligent responses to browsing behaviour much easier to realize than before. It’s a neat innovation that makes the solution to many business problems much more affordable than before.

The rest of today’s agenda is pretty packed with more sessions, media briefings and, tonight, the Partner Impact Awards ceremony. I’ll bring you up to date with some these events tomorrow morning. Which is sometime tomorrow afternoon for you, depending on where you are in the EMEA region.

Mario Bonardo

Blogging from Istanbul

Paul Barrett — 10/19/2009

I just got back from Istanbul where I had the pleasure of participating in the the ETIS Community gathering entitled “Data is King – the Key Asset for Future Telco Business”. This was a really interesting two day conference with Presentation from Deutsche TeleKom, Croatian Telecom, TurkCell, TeliaSonera, Belgacom, CYTA, OTE – and that was just in the BI working group, only one of several sub-groups that met this week. It was one of those conferences where you wish you could be in three rooms at once because there was so much great information shared.

One of the things I liked best was the amount of sharing between the operating companies in different countries. Although everyone was at different levels of maturity there were great ideas to be learned from each other. Even the last slide of the last presentation by Karlis Patjanko from Lattelecom had a key learning about leveraging what users like. His example was that if users like excel, let them use excel in analyzing their business – but you can still invest in data management while giving them tools they like. I am a little prejudice about this because I have been working with Microsoft to create solutions that utilize both Microsoft tools and Teradata in the Contact Center space. I think the future of analysis includes more and better tools at the personal, team and enterprise level that leverage the power of Teradata in ways.

Another great presentation was by Achim Klabunde the Director General for European Commission Information Society and Media. His insight into the Commission’s approach and rationale for it privacy rules was very enlightening. Having just spent 10 days with leading Telecommunications comapanies all over Europe and having heard the entire spectrum of strategies about using customer data, this presentation enable me to understand how these companies can operate so differently under the same set of rules. Achim also did a great job of explaining some of the myths about the EU privacy rules explaining the realty was often different than the expectation.

Iiker Kuruoz, the CIO of Turkcell, welcomed the community to Istanbul and demonstrated the power of location based analysis and the data that could be used in developing marketing strategy and improving the customer experience. Before I left for this european trip I realized that I was going to about 8 countries where I knowing some spanish, french and portuguese would be of no value, so I downloaded a translation app for my iPhone. If that app could have looked at network data and automatically switched to the language I needed that day it would have been so powerful. So I think we are at the early stages of the integration of network information and powering our communications devices.

Overall, this was a great conference and I am looking forward to participating again.

Paul Barrett

Orange passion and innovation

Sandra Worley — 10/19/2009

This is my first PARTNERS conference. I have long known about the passion Teradata employees feel about their company and its products. But I am blown away by customers’ love of all things orange.

At a reception last night, I was chatting with a senior application developer for a company that uses Teradata extensively. Her face positively lit up when she explained how she developed a Web-based dashboard for sales and management use.

Speaking of passion and orange, James Vollmer, chair of the PARTNERS Steering Committee, made quite an entrance at the general session when he drove on stage in an orange sports car adorned with the PARTNERS logo. Vollmer, who professes to be a farm boy, explained how his company, Enterprise Rent-a-Car, reused its existing tools in innovative ways to increase the organization’s agility.

Enterprise’s Automated Rental System was developed to improve customer satisfaction by defining inefficiencies through the analysis of, among other data, how long customers’ cars were in the shop for service. This information is important to insurance companies, repair shops and, of course, Enterprise.

Furthering the innovation theme, Computerworld announced the winners of its Enterprise Intelligence Awards program, which honors best practices in the use of information technology solutions built on Teradata platforms. AT&T received top honors in Customer Intelligence and Management. RBC Financial Group topped the Excellence in BI and Analytics category. The Government and Non-Profit award went to the United States Postal Service. ARC was honored for Industry Innovation. And Australian Pharmaceutical Industries took first in the Integrated View of the Business group.

All of these customers are driving innovation at their companies. Teradata President and CEO Mike Koehler echoed the message of innovation, saying that cutting costs is easy to do, but innovating and growing a business is difficult. As a company, Teradata continues to lead. Koehler’s examples: Teradata 13, Teradata Virtual Storage, the new Solid State Drive Appliance—aptly dubbed “BLURR”—and new Amazon cloud solutions.

How has Teradata helped your company innovate? Which of the new products is your company deploying? We’d love to hear from you.

Evelyn Hoover
Editor-in-Chief
Teradata Magazine

Networking while lending a hand - PARTNERS style

Sandra Worley — 10/18/2009

One of the great things about a conference like PARTNERS is the networking opportunity it provides. Attendees connect throughout the conference in myriad ways—including in and after sessions, during planned networking events and during meals. Other fantastic opportunities, like the Teradata Cares program and the annual golf tournament allow attendees to give back while they make contacts.

Yesterday’s second annual Bike Brigade at PARTNERS 2009 started the conference off on the right note, er, pedal. Approximately 18 Teradata employees, customers and business partners built a total of 35 bikes as part of the Teradata Cares program. The bikes were donated to the Boys and Girls Club of Greater Washington for the club’s Be Great program.

Teradata Cares is also looking for school supply donations as part of the Math Matters campaign, which last year provided supplies to more than 300 children in the Las Vegas community. This year’s Web based program benefits Kits for Kidz, which provides school supplies for underprivileged children in the Washginton, D.C. area. To donate and help Teradata Cares reach its goal of providing supplies for 500 children.

Another way that PARTNERS attendees are making a difference is on the golf course. The annual Driving for Higher Education Golf Tournament was held at National Golf Club at Tantallon and benefited the Hispanic Scholarship Fund. Golfers took to the links at Tantallon, one of the premier golf resorts in the greater Washington, D.C. area.
If you have participated in any of these great programs, let us know why you did it and what you thought of the experience. We’d love to hear from you.

PHOTO CREDIT: Sean Slack, a Teradata employee, makes few final adjustments to a bike for its new owner from the Boys & Girls Clubs of Greater Washington. They earned their bikes through the club’s Be Great program. The bikes were donated by Teradata Corporation, a DC-area employer.

Join the real movers and shakers in Washington!

Darryl McDonald — 10/20/2009

Here’s a news flash! The epicenter of advanced information technology is moving this weekend from Silicon Valley to the US capital! Intelligence is about to break loose in Washington! Active Enterprise Intelligence, that is – as Teradata’s customers and our solutions and partners take center stage. The Teradata fan base from all over the planet is converging on DC!

On Sunday it officially begins – our 24th annual Teradata PARTNERS User Group Conference & Expo, through Thursday, October 22, 2009, at the Gaylord National Resort and Conference Center in Washington, DC. Most of us know that the conference is organized by Teradata customers, and is the world's largest annual data warehousing and enterprise analytics educational conference and exposition.

The conference theme is “Accelerating Insight” – and every educational session plus the networking opportunities at the event will help attendees better manage a decision-making process that is increasing in velocity and complexity. We’ll have participants from hundreds of organizations around the globe in dozens of industries, showcasing new solutions and best practices from Teradata and a record-breaking number of partners.

The conference has more than 200 sessions – that cover all aspects of active enterprise data warehousing and business intelligence. They’ll be led by Teradata customers, industry analysts, consultants, academics from the Teradata University Network, as well as Teradata’s internal experts.

Mike Koehler, our president and CEO, will speak during the Monday general session as will James Vollmer, our conference host from Enterprise Holdings, followed by Don Tapscott, professor at the University of Toronto, consultant and author of Wikinomics: How Mass Collaboration Changes Everything.

I like the sound of that because it echoes this fact: COLLABORATION WITH TERADATA CHANGES EVERYTHING! Okay, I haven’t written a book with this title yet – but it could happen. Teradata’s active enterprise intelligence has positively transformed hundreds of organizations, thousands of groups and millions of users – and will keep doing it for many years!

Whether you are a Teradata customer or a Teradata prospect, I hope you are on your way soon to the Teradata PARTNERS Conference. That’s where you’ll learn more about some of the most agile enterprises on the planet, all leveraging Teradata. This experience could just make you a fan, spark your business vision, and have you saying “Teradata changes everything!”

Look forwarding to seeing you at the conference!

Darryl

Our Man in Washington

David Placke — 10/13/2009

This is just a quick note to say that I will be at the Teradata Partners Conference in Washington D.C. next week and, unless I meet you there, I will keep you up to date with some of the latest conference news, gossip and wonders on this blog. Please bear in mind that it’s the world’s largest data warehousing conference that I am talking about, so I can’t promise to give you anything that resembles a complete picture. Instead, I shall try and shake a few of the many hands that there are to shake in that city and generally be your ‘Man in Washington’. As long as you don’t expect me to leave the conference centre for this.

From a European perspective, Washington D.C. may seem to be worlds away, beyond the great pond. American colleagues from the West Coast, though, correctly point out that they aren’t any nearer to the East Coast (in terms of flight time) than, for instance, London. Still, I am especially curious about the American flair and share of this global conference: the technology trends, the potential new partner firms plus the general vibes in the halls when the attention turns to the economic recovery in the US and elsewhere.

I think it has been a smart idea to hold this conference in the U.S. capital considering that the credit crunch has created an exceptionally high awareness of the cost that can result from ill-informed decisions. Powerful legislative watchdogs like the House Committee on Financial Services have become interested in the use of data warehousing both for the general oversight of the financial sector and the U.S. government’s own bail-out scheme TARP (Troubled Asset Relief Program). Teradata has recently given expert testimony to the House of Representatives on this matter. And I guess holding the Partners Conference more or less next-door makes it easier to intensify the dialogue with political decision-makers than it would have been in the deserts of Nevada. But then again, maybe this is my European view, exaggerating the importance of space and places in the country of ultimate mobility. We’ll see.

Anyway, the Teradata Partners will begin on Sunday afternoon, and I intend to post my first impressions on the following morning – which will probably be during the afternoon in European time. Then I will inform you what has happened over here while you were sleeping. So you’ll hear from me soon!

Mario Bonardo

The importance of context around your data

David Placke — 11/6/2009

I’m a football addict. I play, I coach and I avidly support Newcastle United in England and Motherwell in Scotland so some would say I’m a tragic, or a masochist. That’s football where you kick a (round) ball with your foot by the way. But I’m not one of those addicts that demands or expects that everyone else know everything about the beautiful game, or I hope I’m not anyway. I did however have to laugh at this one …

A while back a colleague of mine was showing the Google Insights for Search capability to another colleague, explaining to him how you can look at stats around searches, how often terms are searched for and compare search terms against each other. A whole array of data can be garnered about searches including the country or city in which the searchers live.

Note my choice of the term – DATA, not information. Information has context, which is necessary for the full picture. My colleague used a few examples then said “there’s a technology called AJAX [pronounced eh-Jax] that is getting a lot of press at the moment” (I guess that ages this story!) so she typed it in and hit search.

“Wow, there’s a lot of interest in the Netherlands!” she exclaimed, “It’s twice as high as anywhere else”.

So, the Netherlands would be an ideal place to set up a software development business, wouldn’t you think? Even beats India into second place.

Those of you who know football, get the point already. Those who don’t, that’s ok – I’m not the kind of addict that demands or expects that everyone else know everything about the beautiful game! Ajax [pronounced I-axe] is one of the most famous football clubs in the world, based in Amsterdam. At the time they had also just sacked their coach, Danny Blind so the hits were high!. (Danny is actually back as director of football, but only an addict would care about that!)

There are, of course, a number of other “Ajax’s” around but I’m guessing that the people Googling in the Netherlands were more likely searching for football related matters than the mythical character from the Trojan War or the Chicago based record label.

Again note my choice of the term – guessing. I can’t be totally sure, can I?

The Web is just one of the many channels for customer interaction. Teradata’s Integrated Web Intelligence solution provides a channel to collect this data, but it’s vital to combine data collected from here with data from other channels to give it full context and a complete view of your customer in the way that Teradata customer JD Williams does. That way, when you see something interesting, challenging or insightful, you can be confident of its context, take action on it, and be sure what you are doing is based on good information.

Have a go on Google Trends, and tell me if you find any interesting or amusing results. For example it appears that Sydney is searched more and is therefore more popular, than Melbourne.

Alec Gardner

The Pentagram of Business Travel

Paul Barrett — 10/8/2009

10,000 meters over the north sea going from Heathrow to Koln. I am not actually going to Dusseldorf, but my hotel messed up my wake-up call. Instead of calling at 05:00 they call at 06:50 so I missed my earlier flight. Even though I raced to LHR and to the airline. Of course the airline charge a fee to change – but it was not my fault, it was the hotel’s I try to explain, but that of course is not their problem.

I guess I will sort it out when I get back to the hotel.

But this made me think about the five major players in business travel – Transportation, Hotels, Communications, Financial services and Government and the amount of data that is generated during a typical business trip. Data on bookings is created from websites, phone calls and physical check-in through your bank or credit card, travel agent and airline and possibly your phone company.

Think about this from a Net Promoter Score standpoint – would I recommend my company experiences to someone else?

My phone is a lifeline to my commitments elsewhere, information about where I am going and of course to my family. I am used to full, no cost data, but internationally this is expensive. I constantly look for WiFI locations to get my e-mail and web access. I bought a package for emergency data – today I need to use it. The prices go from <$1 per megabyte to >$5 mb if I go over my plan. I will spend the day checking my usage… No, I would not recommend it – even though the voice service is much better than my home voice service.

How was your trip? The airline might ask. We’ll not so good, I got to my meeting late, it cost more and it was stressful. It was not their fault – but they took advantage of it by charging me fees to change my ticket that far exceeded any costs they may incur. We know they are simply trying to be like banks and charge as much as possible whenever they have you in a difficult situation. So even if the flight is good, I am still not going to be thrilled. Yes, I would recommend them but I am a bit on the fence

How was your stay the hotel might ask? What data will the hotel analyze to see that I am very unhappy with my stay – because of one small mistake on their part that made me 3 hours late and $200 poorer? A perfectly good stay ruined by a simple mistake. How do you find the data that identifies this problem. Will anyone report it? I am going back to this hotel – will they know? I stay at this chain all the time – will I continue to do so? Whether I would recommend this company or not will be decided in a few days.

How has my banking been? Pretty good, pretty good. Except when they turned off my card for the weekend because their computers didn't like that I paid them twice on the same day. That’s right, they shut off my card because I paid them twice. So I didn't have my card for a weekend while I was 5,000 miles from home. But they did explain it, they have worked with me to make important charges went through so right now, I would recommend them. But we will see how I feel when I get back.

Surprisingly, Although I have had to deal with the governments only at customs and on public transportation, they have been great. Few lines, fast service at the borders and no problems. Yes, I would recommend my government interactions. Maybe that says something about the economy these days that governments are doing better customer service while the companies are not. I also says something about setting expectations.

I am probably basing my expectations on my experiences in the US which probably follow the ACSI index to some extent. I also probably have higher expectations some countries and lower in others.

Paul Barrett

When Worlds Collide

Sandra Worley — 10/12/2009

Throughout the year, we each revolve separately in our own little solar systems. But once a year, all the stars in this galaxy called the Teradata PARTNERS User Group align to become one brilliant and powerful display.

Our lives are inexorably entwined through our data warehousing connections and for a few days we’ll draw together at the PARTNERS Conference & Expo to share our collective wisdom and goodwill. We’ll be both pupil and teacher; competitor and friend. This is my destiny. There’s no place I’d rather be.

Teradata Magazine has a lot going on. From casual tweets to formal interviews for articles, we’ll be doing it all. But of course, our primary focus will be on you--to find out what readers want to see most in the publication. We hope to have the opportunity to talk with you there but even if you're not making the trip in person, you can be right in the middle of the action through PARTNERS Connect where you can check out news, tweets, pics and brand new for 2009, you can even catch some of the sessions as live, streaming broadcasts.

I've seen lots of firsts and bests at the Conference over the years. Honestly, I can't say there was a "most memorable" moment. Although there was that time when a customer's story made the interviewer cry... and the sight of my boss's boss face as we pulled into the station after the Hulk ride...

Obviously, the Conference is what YOU make of it. Do you have a favorite memory--one that you think nobody could top?

Sandra Worley
Executive Editor, Teradata Magazine

How Much Is This Flight in Carbon?

David Placke — 10/6/2009

Is there a “carbon dividend” for the weak economy – a reduction in CO2 emissions due to less economic activity? I would really like to know. If you have followed the recent G20 summit, you may have noticed that in spite of the rampant recession the issue of carbon dioxide emissions was still on the agenda, along with international stimulus packages and banking rules. For good reasons. The perils of global warming are still there and, in the future, we will have to pay a great deal more attention to the balance between prosperity and sustainability that we strike. And I, for one, want to be able to do this in an informed way. I want the facts at hand, just as anyone of our business intelligence customers wants his employees to make informed decisions. Maybe we will have this sooner than many people expect.

For there is an interesting discrepancy here: We are used to hearing (from political commentators, after events like the G20) that not enough progress has been made on the climate issue. But what seems to elude these commentators completely is the progress made outside the political sphere, in the private sector. More and more companies, start-ups as well as long-established ones, are discovering that ecologic solutions can be real business. Taking a hands-on approach, they are coming up with more and more workable ideas to measure and reduce their carbon emissions.

Sabre, an American company that offers travel management and services, is an good example for this. Its computer reservation system enables clients such as travel agents to check the availability of flights, railway ticket and hotel beds and, of course, book them online. With all of the necessary data being available in its Teradata Warehouse, Sabre has developed a sophisticated carbon calculator that estimates the total CO2 emission caused by these activities.

Sabre can now extend its service to travel agents by providing additional information that may be relevant to a growing number of eco-sensitive customers, including corporations with a green travelling policy. They will prefer airlines with cleaner aircraft, the most direct flight routes and, if available, generally less energy-intensive means of travel. Such a service will also help these corporations to generate the exact carbon footprint they want to have. And that’s not the end of it: Sabre could make use of its extensive database to find best green practices for a broad variety of clients. As for private travellers: they will be able to make their personal trade-offs between price, speed and climate-friendliness according to their personal preferences. And take satisfaction in their virtual “carbon dividend” if they choose to.

Here in Europe we are also currently working with a number of manufacturing companies to track at a detailed level the carbon footprint of each individual unit produced (materials, transportation, etc) so that can be audited and even exposed to the consumer, proving that Green initiatives are now seen as both good business and social sense by leading companies.

Niall O’Doherty

Speeding Past Milestones

Darryl McDonald — 10/1/2009

The technology industry is constantly changing - delivering a rapid stream of new innovations and products.

It’s been two years since Teradata became an independent company and in that time, we’ve successfully moved through a lot of product cycles, technology goals and company milestones. I hasten to add we have many more goals set for the coming years.

Like what? Here are a few that will be front and center for us.

We’ll continue to bring to customers more powerful analytic database and platform capabilities and drive greater functionality to our product line. We’re also focusing on improving our predictive analysis and real-time intelligence to help customers make the decisions they need move their businesses forward — and make them quickly.

That's why Teradata has cultivated working partnerships with leading innovators in information and services to deliver the robust, cutting-edge technology you need to build world-class, analytic solutions. Our list of partners reads like a who’s who of industry leaders in technology because over the years we’ve taken great care to form alliances with the best. These relationships give us the power to develop robust offerings that strengthen our enterprise data warehouse solutions.

It’s been a fast two years – just one more reminder of how quickly the tech industry continues to move.

A Pole Position in the Mobile Services Market Race?

David Placke — 9/29/2009

It’s easy to get tired of the hype-of-the-year, in that never-ending technological revolution. If you should ever do, just think of what the world talked about fifteen years ago. “Digitalisation of the media” was one of the catch phrases (and many people actually related this mainly to the success of satellite television). Then it was being said that the Internet “will change our habits” and, no doubt, this has turned out to be completely correct in the meantime. You never realize until you meet someone who hasn’t moved along – sometimes it’s your own mother. This is becoming less likely, though, if social network providers are to be believed that their fastest growing member groups is women aged around 55.

This emboldens me to make a daring statement: the evolution of mobile phone services, and its integration with the Internet, will change our habits. In fact – another daring statement – it’s already happening. The only thing I wonder about is why it’s happening so slowly. On the one hand, we can see a lot of things on the supply side going on. New hardware and transmission capabilities enable new gadgets and services, which result in new business models. There is a richness in applications, and their potential has only just begun to get realized. On the other hand, consumers don’t seem to be in a rush to adopt these new services.

So, the future is wide open. It is unclear, though, who is going to “own” these emerging markets. Network operators, content providers, mobile phone producers – all of them have a stake in this race and strive to take the lead. It will be interesting to see who will win. I believe that the really crucial factor will be the ability to popularize those new mobile services in a true mass market. And my private guess is that this puts network operators in the pole position.

Why? Because, for the time being, they know their customers and their habits best. And they have a decade or so of experience in making customized offers on an individual basis. Regarding new mobile services, they will analyze data to answer questions like this: do customers shy away from mobile surfing but make expensive calls to hotlines instead? Would they be better off if they used the web portal? In this case, they might prefer a bespoken applet to a whole package of bandwidth and other services they’d never use. The result? Changing consumer habits.

Now, don’t get me wrong. The race in the mobile services market is not yet decided and network operators aren’t the only ones who have customer insight either. All the players need to try and penetrate this market deeply in the coming years. And I believe that customer analysis – utilizing large data volumes – will prove to make the crucial difference in this.

Paul O'Carroll

The Netflix Prize and Freeing Data Analytics

Paul Barrett — 9/25/2009

This week, Netflix presented the first Netflix Prize, awarding the Belkour Pragmatic Chaos team the $1 Million grand prize in a ceremony in New York. The most exciting news is that Netflix announced a second round of the Netflix Prize using demographics and other data instead of movie ratings.

There are a lot of great articles about the contest, the winners and the impact on Netflix. This post is not about rehashing the contest. I think it was a masterstroke by Netflix to open up its data sets and harness the power of the net to drive innovation.

There are more contests coming, which is another great thing. And I like the format of the new Netflix contest better – a 6 month interval then an 18 month interval. This starts to approach reasonable pay-back time frames for companies looking to make an investment, compared to the 3 year process that revolved around the first contest.

I was testing the visualization program Tableau last year and I used a data set from Sean Lahman’s Baseball Archive. What was cool was that using salary data I was able to show that the hated yankees of new york have spent more money on salaries since their last world series win (which was sometime last century – that’s right the last century when radio was popular and people watched silent movies) than my beloved Red Sox did in 78 years of frustration. That’s right the yankees have blown more than a billion dollars and have won zip. Nada. Nothing. Meanwhile my children lived the blessed life of only experiencing this rapturous time of Red Sox domination and Yankee futility.

But my point is that bringing that data set allowed me to test the Tableau software, find interesting ways of investigating visualization in a matter of minutes. We are moving to a time when data analytics need to be freed from proprietary data.

Teradata has been winning deals through our benchmark center for years. Proving our scalability and speed with complex problems our customers face. Often, their queries will not even run on their existing infrastructure. For IT this is really important, but for the business user, speed and performance are not the only thing.

I think there maybe a challenge for our customers to provide some open, anonymous datasets so that we can help them not just with performance but also with improving the quality of the analytics they get. Obviously there are privacy and security constraints that need to be taken into account. But I would love to get input from readers about how we could facilitate benchmarking the quality of analytics – not just the performance aspect.

Our team has been working specifically with the integration of online and offline data. If there are customers out there that would be willing to provide data sets for analytics including web visitor data, online advertising, search marketing, social media or other areas we would be excited to hear from you and work with you on developing new insights from this data. With our Integrated Web Intelligence analytical assets and partners such as KXEN, Optimine, Webtrends and Microstrategy we may be able to provide new analytics that transform your multi-channel marketing.

You can reach me through this blog, or at paul.barrett@teradata.com

Shameless self promotion: Check out Webtrends Blog for a guest blog post.

Paul Barrett

Permission Marketing

Jamie Flynn — 10/14/2009

I had an interesting experience last week with permission marketing. I had been to see a house that was for sale around the corner from home and, as a part of viewing the house, I had to leave my mobile number and name with the agent at the door. In the past when I have asked about this process it has been described as security.

So when I received a text message on my mobile about another house for sale in the neighbourhood I was a bit surprised, but let it go - by the third message, I felt the need to do something.

When I questioned this tactic the agent described the messages as a service that they provide, but he was a bit stumped when I asked at what point in our relationship did I give permission to be spammed? Apparently I was the first person to make a complaint, and I have been lead to believe that I have now been taken off their list.

I have always held the view that no one has the right to spam. In Australia if you hold an existing relationship with a customer then you have the right to communicate with the customer unless they opt out.

But what constitutes a relationship? By viewing a house I don’t believe I have a relationship with a real estate agent. If I take that thought further, if I hold an electronic banking profile, do I have a relationship with the bank? The banks answer should be a resounding yes, but it is interesting that as an organisation they have pushed me away from any meaningful contact or relationship and as soon as it is time for them to market a new product or service they conveniently have a relationship again. This is not just a banking issue, my telecommunication provider only ever wants to talk to me when a contract is near due, my utilities providers send someone to my house who tries to bully me into showing him my last bill, and the list goes on.

Legally, before organisations hit ‘send’ on a campaign they must ask for permission to communicate with consumers. This can be in small print, but it does have to be there. Further you are required to have a process that takes care of consumers that want to opt out. These are the basics of direct marketing, but they are not the basis of a relationship.

A relationship requires ongoing care and compassion. So when you next decide you have a relationship with someone, perhaps you should consider if they would think the same. Otherwise you are in danger of thinking you have a relationship when the other person is not aware that it exists, which is both sad and creepy.

If you get it wrong - you are just another spammer and we have laws about that.

How do you ensure that you have a relationship with your customers?

Daniel Tehan
www.linkedin.com/in/danieltehan

When No News is Not Good News

David Placke — 9/22/2009

There isn’t a telecom carrier out there who isn’t trying to understand their customers’ experiences in order to increase satisfaction and protect and grow revenue. But in my 35 years in this industry, I’ve watched carriers wait until the customer tells them there is a problem before doing anything to improve service.

Customer experience analytics has traditionally relied on billing and call center data, which naturally places the carrier in a reactive mode – crediting the customer’s account and apologizing for the inconvenience while striving to improve on mean time to repair a problem.

But there are so many ways customers use their handsets today that carriers have to look at the bigger picture and use all available information to solve problems and answer questions like:

Is the reason the customer can’t get five bars of signal strength due to the network or the handset?
Were delays in accessing the Internet a network problem or a problem with a third-party Web site?
Is the reason they can’t access the Internet due to a provisioning error or location problem?

We’ve all talked about BSS/OSS integration, but I don’t think people have really taken this seriously. Most carriers think accessing network data is too difficult (for technical as well as political reasons). They think it’s too costly because of the volume of network data. And in their experience, the value of this type of analysis wasn’t significant enough. That’s short-sighted thinking. The truth is, no carrier can afford to not integrate data and analyze it on a daily, hourly or even minute-by-minute basis.

I’ve watched as interest in integrating network with business and customer data has grown substantially in the past 2-3 years. In fact, it has become essential in ensuring customer satisfaction by providing the highest quality service and products. Carriers that take time to integrate data gain a single view of all of their customer’s touch points on the network and get new insights to the total customer experience.

Yes, you can perform these tasks by using multiple data marts and associating data, but adding this layer of complexity across multiple data sources and various degrees of freshness is certain to provide misleading information or mistimed responses.

Wouldn’t you rather have a single data warehouse that lets you anticipate and correct problems before customers are even aware of them? And wouldn’t your customers rather talk to service representatives who are not only informed but also empowered to take action, regardless of where in the communications chain they happen to be?

That’s the power of data integration.

Robert Segat

Who should Jean-Claude Trichet call?

David Placke — 9/21/2009

How is private consumption developing in the US? Alan Greenspan, the former Fed’s chairman, once said that if he wanted to find out, he would call Wal-Mart rather than wait for the official statistics. At Teradata, we took this as a compliment. We knew that Wal-Mart, thanks to its enterprise data warehouse, had a very accurate and timely view of its sales. Apparently, this reliable data included enough information to give Greenspan a solid idea of what the statisticians were going to say about the whole nation a little while later.

Over the past months, many decision-makers would have been glad if they had known a little faster what the real state of their economies was. Just take a look at the news: central bankers worry about deflation and monitor prices as tightly as they can. At the same time, some of the sages say that the economy is already recovering (beginning to grow from a very low level). Others say that what is improving is not so much the present situation but expectations about the future – a very changeable indicator. Now let’s say there suddenly was a new round of lay-offs in manufacturing. If you were a central banker, you would want to know the impact on consumer confidence. How would you find out fast? Who would Jean-Claude Trichet call? (The same question goes even more for Gordon Brown or Angela Merkel, considering that expansive government spending has a much faster effect on the economy than the lowering of interest rates.)

In a way, Alan Greenspan had it easy. Wal-Mart is the world’s largest retailer with a substantial market share in the US. (In fact, Wal-Mart’s turnover is only exceeded by a single company in the world, Royal Dutch Shell.) The Euro zone is more diverse than the US, but there are retailers with an impressive market reach like Metro, Carrefour, and Tesco that have similar capabilities of sales analysis and reporting. Their numbers would be as representative of certain national economies as Wal-Mart is in the US. I am aware that such data are strategic assets because enterprises flourish by turning these data into a competitive advantage. They wouldn’t share their insight with anybody. Still I find the idea fascinating that the success of data warehousing in the enterprise world could also result in a clearer, quicker view of a country’s general economic activity and thus support macroeconomic decision-making.

Of course, the economy is made up of more factors than consumers’ spending propensity. Luckily, there are data warehouses in numerous sectors from financial services, manufacturing, transportation – you name it. And as you do, why not name the most telling indicator from your data warehouse, too? Of all the data available, which piece of information could serve as a valid pointer of the general economic development? Expensive ready meals? LCD TV orders? Smart cars or any other durable goods? Or – no joke, sadly – soaring numbers of prescriptions for antidepressants? What would you point Jean-Claude Trichet to if he asked you?

Jonathan Tebay

The Napoleonic Wars - Timely and Near Enough was Good Enough

David Placke — 11/6/2009

In 1815, they didn’t have access to all the information we have today, but sometimes they had the right information when they needed it. Take the Duke of Wellington for instance, knowing that Napoleon had 14,000 seasoned cavalry, what the French infantry tactics were, and capitalising on the bad weather to contain the French, he had the right information at the right time to confidently act and win the Battle of Waterloo. Arguably the most important gap in his intelligence for the day was whether his Prussian allies would arrive on time… Other helpful information could have been amassed about the very high-stakes decision he was about to make but this would have taken time. Wellington made his decision on aspects of the coming battle based on key indicators that were clear, not exact. Knowing the exact number of enemy cavalry or bayonets, sabres and muskets, the names of French Colonels and insignia of their units would have perhaps proved helpful. These were not critical when deciding to act on the threat of Napoleon's army at Waterloo. He used the information he had to make the right decision at the right time to lead his troops to victory.

Today, data quality is something most organisations still grapple with, with varying degrees of success and consensus. Add to this the issue of having "all the detail", rather than just the information that is needed for decision-making or core business process, and intelligence-based decision-making becomes a dream rather than a reality.

Data quality issues can result in mistakes being made when the wrong information is provided to the right person at the right time. Often this leads to a case of organisations being "once bitten, twice shy." The practical realities of what data is relevant to whom, when they need it and what this is used for often do not gel with ideals of data quality perfection and having all enterprise data considered in decision making. There is no point having the cleanest, parsed and transformed data and "nice to know" information on a customer who is at risk of buying from another company once they have already left. Being confident in the quality of the information and being able to assess the relevant risks is what is most important when acting on intelligence.

The pursuit of Data Quality perfection and trying to get all the data available rather than what is needed now can slow (and sometimes cripple) an organisation's ability to respond to the near-to-real time demands of their operating environment. Knowing what is important and understanding the risks and opportunities and how quickly you need to act is the key. Predicting, knowing what will, or will most likely happen in your business before an event is the insight that gives some of Teradata's most successful customers the competitive edge to act. The right information, on enough of the right indicators results in right-time action.

When is information too much and when is it not enough, and how do you know with confidence which you have?

David Bremstaller

High Seas, Low Brows, and Missed Marketing Opportunities

Colleen Quinn — 9/18/2009

What’s insulted, claustrophobic, over-fed, and flying home from Puerto Vallarta? Me, after disembarking from her first-ever cruise holiday! Let me be clear. I “disembarked” after arriving at the first port—day three of a seven day cruise. I’d read someplace in the cruise company’s small print that if I wasn’t having a “fabulous” time, they would fly me home from the first land destination (at their cost) and refund the unused portion of my cruise fare. Within 72-hours of setting foot on the in-aptly named ship, “The Splendor,” I was ready to take the vacation guarantee policy for a test spin.

Let me tell you a bit about myself to offer up some context to this story. I am a snob. There, I said it. I have no grounds for snobbery, raised the poor kid of two working-class parents. But, I am. I read for fun. I don’t own a TV (clearly grounds for a subsequent post). For years prior to the arrival of my son, I had a theater ticket habit that eclipsed the cost of most heroin ones. I run or ride my bike daily, outside, in the mountains, typically wearing brightly colored Lycra on a spendy bike that weighs less than most pumpkins. I don’t own a fanny pack. Cruising and me, well, it was a bad idea from the start.

I considered it an interesting social experiment—one where I thought I’d bask by the pool sipping umbrella drinks, while reading the New Yorker. I went in with a great attitude, amazing company, and the desire to have fun. I won’t list out the numerous offenses to my delicate sensibilities (um, another buffet, anyone?). But, I will say this:

When I want to watch a movie, I want to do so on my time—not in some artificially calendared three-hour rotation. You guys ever hear of something called VOD?
I pick-up my mobile phone and, get this, I actually expect that it will work
Email is a right, not a privilege
Cyber-cafes? Really? I’m not in college any more…

I was not on land, but I was land-locked. Technology locked. Or rather, technology blocked. Here I was being exposed to some of the most shocking displays of mass humanity (really, a conga-line at dinner?), and I had no outlet to whine about it.

So, now I’m home. I’m plugged-in, back to work, Tweeted-out, and ostracized by a pro-cruise contingent I didn’t know existed among friends and family. And, I receive an email in my inbox. From …. Another Cruise Line. Offering me some “fabulous” deal to embark on an “amazing” Mexico adventure. There are so many problems here, I just don’t know where to start!

What an egregious waste of marketing dollars, and more importantly, a way to underscore that this company doesn’t know or care about my interests. It’s not hard to know your consumer—especially not me. While I own a digital copy of every Pixar movie known to man, I don’t much frequent other Disney properties (Point 1). Um, I just had a God-awful cruise experience, and took to EVERY possible social media outlet to share that fact (Point 2). The email message received had The Princesses as its theme, and I have a son (Point 3).

My experience casts the brightest spotlight on one of the biggest gaps for companies in every industry. How can they intelligently bridge the internet and the enterprise? The answer for every company who conducts any business or brand awareness on the internet (and, that would be every business these days) is to have a strategy to monitor online behavior, take the pulse of their brands and service offerings across the social media landscape, and truly understand the interests and demographics of their targeted customers. In short, integrate web intelligence into their enterprises.

When done right, this new view of detailed, integrated customer information allows companies to deliver true customer relationship management—sending me the right messages about services, products and offers that resonate with my needs. The benefit to the company is many-fold: decreased marketing expenses, improved marketing response, dramatically increased engagement with the customer. The benefit to the customer? Well, that’s clear.

Needless to say, I deleted the email. Worse, I denoted it as SPAM. Worst of all, I lost just a little bit of faith that this company “gets” me. I checked-out a little. That’s the biggest risk brands face—losing touch with the consumers who drive their businesses. Smart companies are making in-roads to solving this dilemma. The others, well, they’ll make in-roads too at some point, but it may be too little to late.

But, what do I know? I’m still trying to straighten my nose after a trying voyage. I should just relax, get some work done, and enjoy the fact that I shan’t be encountering my Nth “I’m with Stupid” t-shirt any time soon. One more thing—how can I get some information on a trek to Kilimanjaro? Now, that’s a holiday right up my alley.

Colleen Quinn

You Call This Mobile CRM?

Rebecca Bucnis — 9/17/2009

Not too long ago, I saw an interesting article on mobile coupons titled Coupons You Don’t Clip, Sent to Your Cellphone.

The idea is that you don’t have to clip coupons from the weekend newspaper any more. You can just have them sent to your cell phone, where they’ll sit until you are ready to redeem them.

Good idea. Convenient and cost effective, and there is a very real environmental savings (save those trees). And I really WANT to love it ... but I really just can't.

I've been watching mobile marketing for a long time ... I mean way before it was a thought beyond text messaging. But really, is this what we call improvement? A clip-less coupon sent to my cell phone?

As long as it's mass and impersonal, I say NO WAY. Yes, I get that it’s an effective way to reach a new generation of consumers who will ‘clip’ and use mobile coupons. And from a retailer’s perspective, this is good news. But from MY perspective, clipping coupons from the paper is no different than saving mass marketed promotions to my mobile. I am stuck because it could be so different. And to be honest, I hope for much more.

The ideal, really, is getting rid of the clutter and sending something that makes me want to use my cell as the means to view what's mine, in much the same way I view my personal texts. Imagine if I could receive a message from my local grocer suggesting I order a ready-made meal to pick up on my way home. I get to choose the main dish and the sides from list, and then when I go to pick it up at the store, I get a personalized offer for that meal, as displayed on my cell phone coupon. That's worth the effort to save. (And I’m thinking meatloaf with corn on the cob and mashed potatoes and gravy sounds pretty good right now.)

So, until it really IS new – and not just a 'cool medium' – I won’t be impressed. In the words of the Mobile director of JCPenney, I'm still waiting for mobile CRM. To date, it's only been a mobile mess ...

How do you feel about receiving clip-less coupons on your phone? Or about mobile marketing in general? Let me know. And hey, hold off on using my mobile space until you make it really personal.

Rebecca Bucnis

Wanted, Retail Executives with an appetite to make a difference!

David Placke — 11/6/2009

Tired of hearing, ‘why we can’t, instead of how we can’?

Hands up if you live within a retail brand, working hard day–to-day to get internal stakeholder interest, support, funding and, most importantly, a guernsey on the IT Projects Prioritisation list?

Here are the Top 5 topics our Retail customers & prospects ask about most frequently. Are you finding one or more of these differentiation strategies desired but challenging or even worse, not possible in your world? If so, your organisation may be in desperate need of extending its capability to leverage its most important asset, its data.

On-line Shopping: Integrating your on-line channel performance and call centre customer interaction with your off-line data with Integrated Web Intelligence; enhancing a single view of your customers and presenting an aligned brand offering.
Customer Segmentation: Development of behavioural customer segments, gaining insights and driving optimised local assortments, promotional effectiveness and merchandising presentation to deliver exceptional customer experience.
Market Basket Analysis: Identifying slow moving, poor contributing products to free up inventory investment $’s and valuable shelf space; while also taking into account seasonality, promotions and the most valuable customer purchase behaviour.
On-Shelf Availability: Ensuring your best contributing products are available 24/7; driving up customer satisfaction, reducing lost sales and minimising out of stocks.
Pricing Optimisation: Implement ‘right pricing’ and markdown optimisation; have a positive impact of inventory forecasting and return on investment.

It’s hard to change things in retail, where margins are tight and competition is fierce. So as a ‘change agent’, you have to jump through hoops, source information and do the research required to build a business case that will cut the mustard and get attention to drive momentum.

However, if you have a vision, the courage and the energy to act, you can help your organisation take on new challenges by engaging in discussions that help you understand what is possible and how to facilitate change for good.

There are flexible, responsive and ethical vendors out there willing to engage to help you test and prove what is possible, without impacting your day to day operations, without risk. Ask about their customers and, where possible, ask to speak to those customers. If the vendor has a ‘successful track record’, proven retail processes and solutions, their customers will confirm that …

Take the opportunity to kick-start dialogue about your challenge; it’s just a conversation away…

Geoff Dyer
geoff.dyer@teradata.com

That Solvency Be Kept

David Placke — 9/15/2009

Remember the good old times of 2006? The banking system was going to be fine forever. Large-scale credit losses, a scourge that had plagued booming economies since the early days of capitalism, would become history like the Black Death epidemics, thanks to the world’s most ambitious risk management programme called Basel II. Or so it seemed. Three years later, you might ask: considering the enormous investments in time and money that banks had to make in order to comply with Basel II – was it worth it, given that it has not averted the worst financial crisis since the Great Depression?

Well, first of all, the United States, the largest market for subprime mortgages and the epicentre of the credit crunch, never ratified Basel II. Elsewhere, the programmes accelerated the breaking crisis when money markets dried up, but they had been implemented too late to have any beneficial effects in the run-up. The most important point, though, is this: The riskiest dealings, that are the ones most responsible for the credit crunch, were made by entities that were not subject to conventional capital requirements. All of this argues for, not against stricter rules like those set out in Basel II. The credit crunch has highlighted that the insurance industry, too, must be subject to stricter rules - in fact, one of the largest bailouts involved an insurance company, which had complemented its (sound, as far as I know) core business with the extra risky business of backing up credit defaults.

The EU has already agreed on a set of tighter regulations for this sector under the banner of Solvency II. Much like Basel II, Solvency II defines capital requirements, risk management and supervision procedures as well as standards for financial reporting. And just like Basel II, its implementation will require huge efforts. Now, as we are wondering about Basel II – is Solvency II going to be a success? It’s interesting to see that many of those in the US who are aware of it believe that Solvency II regulations should be adopted, as a recent Teradata study has revealed. One of their reasons is that U.S. providers operating in Europe might be required to hold more capital in reserve for identical products even if there is no perceived risk to solvency. So we might see a dynamic that works towards stricter, internationally harmonized regulations in the near future.

Solvency II and its corresponding US standards take fundamentally different approaches. In the US, capital requirements are set for one product group at a time. Solvency II, on the other hand, follows a top-down approach to monitor risks across the entire enterprise. The latter certainly is the more ambitious approach in data management terms: Insurance companies need to create an accurate, detailed enterprise view on their assets and liabilities as well as their risks in order to meet Solvency II requirements. And this poses a challenge for an industry that holds few (in the true sense) enterprise data warehouses so far, and predominantly relies on siloed environments. There seems to be a lot of work ahead before the adoption deadline in 2012. If there is one piece of advice that banks might have for insurers it is this: don’t underestimate the scale of this project. Better start early than late.

Simon Doherty

Connect-the-dots

David Placke — 9/14/2009

Fires

David Placke — 9/11/2009

Lessons Learned From a (Real) Fire Zone: Keep Your Friends Close, and Twitter Closer

Colleen Quinn — 9/11/2009

So, after raging more than 10 days, consuming over 160,000 acres, and destroying nearly 80 homes and dozens of other structures,the so-called Station Fire is 71% contained. Sad numbers, especially with the recent revelation that the fire was caused by arson. Scary numbers for me, my neighbors and friends who watched the local hills rage with fire day and night from our doorsteps.

The smoke in the skies blacked out the local mountains, which are, usually, clearly visible from the desk where I write this. Ash chunks the size of large flower petals rained from the sky. Breathing became impossible—my three-year-old son and I developed a persistent hack. Our eyes were bloodshot. My gym was crowded. Evacuation notices and the subsequent all-clear notices became a daily occurrence. Fire fighters, engines and brush rigs from all over the state crawled through the city streets, seeming to outnumber civilian traffic. It was surreal.

For those of you who missed the news—like, you’ve been in a coma since August--- LA County has been fighting the largest forest fire in its history. I live in Altadena, which is a sweet little bedroom community at the base of the San Gabriel Mountains. Residents of my city, along with neighboring cities of La Canada Flintridge, La Crescenta, and Pasadena, lived in abject terror for many days, watching and waiting for this blaze to be brought under control. But, control was elusive.

We craved any information about the status of the fire… that, too, was elusive from traditional news sources. While CNN.com covered the fire, and NYTimes.com picked-it-up several days later when the situation was clearly dire, neither of my most frequently visited and highly trusted news-sources offered the detailed minute-to-minute updates I needed and wanted when flames were leaping from hills just across from my house.

Enter my new reliance on Twitter. I’ve been a fan and consistent user for the last 18 months (no Twitter-Quitter here). I’ve often blogged about the importance of Twitter in the social media landscape, and am a true believer. But, up until this fire, the most critical piece of news ever delivered to me via Twitter was that Lance Armstrong’s baby momma had delivered a son whose paternity I question (I’m a conspiracy theorist). I didn’t rely on it as a singular source to keep me connected to the world in a timely and serious way.

My respect shifted overnight. The LA Times set-up a Twitter handle, and what they couldn’t cover in print or online, they delivered through frequent, detailed Tweets. Dozens per day. I located other local residents who alerted me to ever-changing evacuation orders, air conditions, fire lines, and containment numbers. And, in the aftermath, I collected dozens of pictures of scarred trails showcasing the heart-breaking damage to the mountains where I run and ride daily. (As well as one important post informing about the $10k fine for folks violating the forest-closure rules. Few rides are worth that, certainly!)

I’ve worked in traditional news, broadcast and digital media technology for the sum total of my career. I’m as aware as anyone of the perils facing the traditional news business, and the rapid shift to online media as the dominant source for news. But, even I’d failed to make the mental shift to truly understand how a very broad, disconnected, social network would become critical to filling in the gap.

As I re-play the recent fire events, I need to send-out thanks to many people who made the events more tenable: the fire fighters who risked everything to protect my ‘hood; the local merchants who opened their doors to fire fighters for lunches and cold drinks; my friends with the sweet beach-side pad who offered us an escape from the ash and grime; and, now, my heightened appreciation for my new Twitter friends. I may have to Tweet about it!

Colleen Quinn

The Map is not the Territory

Paul Barrett — 9/10/2009

“The word is not the thing, the map is not the territory” is a key principle of General Semantics and non-Aristotolian logic. But is the data the thing? Is the social network the person? We know that the data of a social network is not the person – but it is valuable. Today, I am going to share some thoughts on my social networks and ask for yours.

Let’s start with work – this is a work blog. My linkedin profile tells me a little bit about my work life. I am directly connected to 513 people and they are connected to 88,000. It shows my contacts by geography and industry. New York, Washington and San Francisco are the highest ranked as are Telecommunications, Marketing/Advertising and Information technology.

What is interesting is that my network does reflect a lot of my work history. Ten years at MCI, 3 years living in the New York areas and 5 years living in the DC area. But it also reflects where my friends have gone to work such as the Bay Area.

A cool tool for social network visualization is available on Facebook, it is called Nexus.

The chart above shows a lot of the links I have in Facebook. The dots are all friends and different nodes. As a marketer I could use this information to develop a lot of insight if I knew my behavior because I could theoretically predict the behavior of the people in the graph.

When I look at Twitter it is kind of interesting because the people / groups I am following are kind of a reflection of my tastes, but what about the people that are following me? What does that say about them.

I am following 85 twits: Being followed by 54:

I have to say that I love twitter, not so much for getting my messages out, but to get quickly and succinctly information from people and organizations I find interesting. My twitter list probably most accurately reflects me – to some extent – because it is so active and real-time. I wish I had more time for it because as you weed through there is something there.

Okay, maybe the social network is not the person and we proved that social networking does not follow Aristotolian logic. I know that is a big surprise.

But think about this data – I didn’t know this much about me even a year ago. It is kind of cool to find out about yourself… umm maybe.

Of course all this data comes at a cost – if you google me and go to page five you can find out that Picabo Street beat me by 9 seconds in a ski race at Vail in 2006. This stuff never goes away.

Paul Barrett

Beyond the Buzz: The Quiet Thunder of Active Data Warehousing

Darryl McDonald — 9/10/2009

That cicada buzzing sound you hear in the high-tech media around data appliances may lead some to believe that this is where all the IT excitement is. But it’s a distraction from the real excitement. Sure, I know we’ve contributed to that buzz to some degree – with our recent platform family momentum news.

But the real, rules-changing, money-making excitement in the global Teradata customer community continues to be in active enterprise intelligence™ (AEI), especially for frontline business users. It works from a platform we’ve been calling Active Data Warehousing (ADW) for years. It continues to be Something Worth Talking About. Some prefer the term “pervasive BI” or “operational BI.”

Analyst James Kobielus gets it. In one of his reports1 he wrote: “For the most time-critical decisions, enterprises require ‘really urgent’ analytics.” This is AEI, facilitated by fresh data, in-database analytics, and event triggers that get dramatic business results in real time. Another group that gets it is Judy Davis, Colin White, and Claudia Imhoff, who have a new report: Operational Business Intelligence: The State of the Art.

To cut to the chase, leading edge companies are using AEI to:

Personalize customer web pages in real time with the most recent and relevant detailed information a million times a day (Travelocity)
Manage data from retail point of sale systems at every check stand to track and manage prices, inventory, and customer service in real time (Haggen)
Capture and integrate web interaction data with information across all other selling channels in real time, to ensure personalized individual customer engagements anywhere, anytime they touch the business (JD Williams)
Detect business fraud and crime in real time as it is taking place (Paypal)
Automate claims processing in real time with rules, data mining, and embedded BI – (Highmark)

Database boxes are cool and we get a big share of those, but beyond the buzz is the sound of smart money, investing in active data warehousing – and getting a quiet thunder of applause from IT and analytics players focused on delivering dramatic business value.

1 “Really Urgent Analytics: The Sweet Spot for Real-Time Data Warehousing” Forrester Research, August 2008

The evolving nature of IT partnerships

David Placke — 11/6/2009

I have recently re-joined the corporate world, after taking a few years to pursue a sea change, which is a whole other story. But in rejoining the IT world, I notice how far the partnering models have come.

Partnerships in our industry have traditionally been opportunistic. Company A has a gap in their product offering so they go to company B to fill it. In this clearcut model there are distinct boundaries around who owns what and the selling agreements are well defined which made it easy for the partnership to be formed and ended.

Today companies are forming ecosystems, communities of vendors, customers, developers and industry bodies. This partnering model is active, rather than the traditional passive approach, with the stakeholders working together to develop new intellectual property and complete offerings. With customers and industry bodies now actively involved, the traditional boundaries of vendor and customer are blurring and bringing an industry focus to the deliverables. This approach, while potentially having bigger benefits to those involved, isn't easy and takes significant investment and commitment.

So why has the new model developed and what are the benefits of an ecosystem approach?

I think there are 3 key benefits.

Reducing the complexity of offerings. The whole solution stacks, containing best-of-breed components, can be pre-integrated and pre-tested, greatly reducing implementation times.
It is a more complete solution that meets specific industry or customer requirements.
This creates innovation or co-innovation. Through the nature of the ecosystem new developments are created that answer specific industry or customer problems giving our customers distinct competitive advantages.

How does an ecosystem start?

Initially one or two vendors who have a shared set of technologies or services begin the partnerships, then other vendors, developers, customers and industry bodies begin to join. The system becomes self managing and involvement from customers and developers will grow. Co-innovation means that the participants will want clear definitions around ownership of IP and the more flexible the IP model is, the more attractive an ecosystem will be to the participants.

In my new role at Teradata I will be working closely with SAP. They are a company that have invested heavily in the ecosystem model, and we are now investing in that model together. I am looking forward to experiencing this model firsthand - I hope it lives up to my expectations!

What experiences have you had with this new ecosystem model?

Tracy Gumm
Regional Program Manager

Who Wants Yesterday’s Papers?

David Placke — 9/9/2009

Times are hard, especially for the traditional media. In Europe and the United States, numerous printed publications are running deficits or have been closed down. It’s not so much the economic downturn that is killing them off. It’s the fact that their business model has been undermined by the rise of online media and readers’ reluctance to pay for content they can get for free somewhere else. In a recession, you’d expect weak players to wither and well-capitalized corporations to survive and, in the end, increase their share of a consolidated market. Instead, Rupert Murdoch – neither a lightweight, nor faint-hearted – has called on his competitors to stop providing content for free, in the best interest of the entire industry. If that’s not a wake-up call, what is?

The news industry’s future seems to be seriously threatened and it would be pompous for me to suggest that I have a worked-out, new and viable business plan on offer. (After all, this blog isn’t called The Quest for the Holy Grail. Although this never-ending story actually would be better told in a blog than a printed book. Odysseus, too, definitely would have a web log these days. Sorry to Penguin Classics.) Back to the news business – my point is simply this: I believe that people are indeed less prone to pay for media content, but they are still prepared to pay for some of it. When and what for? It’s the media’s job to find this out. How? By doing customer analysis.

… like show business
Let’s look at the entertainment industry for inspiration. The biz has been facing similar difficulties: Internet radio, “sharing” of movies, TV series appearing on Youtube in violation of copyrights etc. At the same time, such new channels of content distribution open up new opportunities for the industry to promote and sell its products in a more efficient way, for example via Apple’s iTunes, or introduce new payment models like pay-per-view. For the industry, providing content for free can make a lot of sense if they get the usage data in return. The more detailed data from different channels they get, the better they can value the individual product or artist and understand what could be a reason for consumers to buy rather than use. Free and paid content, therefore, will continue to co-exist online.

Roll over, Rupert!
Admittedly, news and music aren’t quite the same. There is no copyright on news. And unlike music, you don’t want to hear the same news over and over again. On the other hand, online news sites allow for a very accurate customer profiling, which creates opportunities that have not yet been fully appreciated. First of all, more and more refined behavioural targeting combined with the demise of the competing printed media will sooner or later lead to higher advertisement prices. Large news corporations with a lot of media outlets will have a natural advantage here, as they can draw on a larger data treasure. Second, the revealed interests of online users includes not only cars, trips and cameras, but also their information needs – the demand side of the news market. As more and more readers are moving online, it’s becoming easier to predict what sort of content they would spend extra money on, and the media can tailor their offerings to that: is a scoop interesting enough for a sufficient number of people to buy the book to get the full details? Then it should be for sale when you are breaking the story. With their enhanced customer insight, media corporations can thus realize an extra profit at a lower risk than in the age before the Internet.

To be clear, I am not saying that commercial interests should interfere with the actual news reporting. Without editorial integrity, journalism would lose its raison d’etre altogether. I am saying that publishers need to find new ways of getting paid for the good work they do. And I think that more customer insight, higher ad prices and smarter content utilization might help to ease some of the pressure that the media are feeling. The future of journalism might not look as bleak as many people are thinking at the moment.

Charles-Yves Baudet

Considering the Data Diet

Paul Barrett — 9/2/2009

I had the unique pleasure of meeting with Pete Fader and Eric Bradlow from the Wharton Interactive Media Initiative last week in Philadelphia. And while it wasn’t the point of the meeting I was really interested in their work about companies going on a data diet. While this is somewhat antithetical to the historical position of data warehousing – where every bit of data should be kept for as long as possible, I think they are doing important work in this space.

Not because I think companies need less data, but because they are going to be getting more and more data. With the emerging Web Squared environment where world wide web meshes with devices, sensors, geospatial and temporal data and the devices interact with this mesh to transform not just the consumer experience but the entire value chain. The volume of data that can be analyzed is not going to be nearly as important as the critical data to the problem you are trying to solve.

As Melyssa Plunket-Gomez from Crimson Hexagon showed me with Social Media data, a lot of what needs to be analyzed is what you exclude. When looking at blog traffic for sentiment about a company you have exclude stuff like job postings because they are irrelevant to the analysis even though they have all the right keywords. I think this is a perfect example of watching what you analyze – a critical part of any data diet.

And while I think the data diet statement can be too simple, I think the concept has an important role to play in how companies utilize tools like Teradata’s multi-temperature data warehousing solutions. Which data is hot and needs high performance disks and chips and which data is not needed all the time and can be moved to lower cost storage? A data diet approach can be useful in determining how to invest in data.

One of the challenges I see for data geeks in the coming years is how to decide what data you need to answer the questions that need to be answered tomorrow? I don’t don’t know what that question is and I don’t know what data I need. There is a big cost to not having the information you need when you need it. A data diet risks summarizing the value out of the data and eliminating data you need, but you don’t know it yet. Sometimes the most expensive data is the data you threw away.

Paul Barrett

---------------
A couple follow-ups to earlier posts:
Consumers Taking Control > The district manager of the book store that didn’t get my money invited me to come to the store and meet and have a coffee. I am looking forward to it – I will probably make him spring for an expensive mocha instead of my usual value priced Iced Double Espresso – under $2 almost anywhere.

Millenials have never lived in a Batch World > I am getting ready for my presentation “Customer Time is Now Real-Time” at the Teradata Partners conference, coming up in October. This presentation is about how the confluence of the real-time web and a young generation of consumers are changing the rules for businesses. If you have comments or thoughts on Millenials or the real-time web and how it is impacting your business, please drop me a note at paul.barrett@teradata.com.

Are you managing Opt In?

Jamie Flynn — 10/14/2009

If the metric of disinterested customers is the number of people that opt out of your marketing, does that also mean that the metric of interest is the number of customers opting in to your marketing?

For the most part I believe we all have an ever shortening attention span, I first noticed this when one hour TV programs moved to 30 minutes, if you remove the ads then you have 23 minutes of content. Was this done because the powers that be thought we could sit on the couch for 60 minutes? It seems as if content is becoming smaller and more compact all the time. As an example I will have to figure out how to condense this message into 140 characters to tweet it later on.

My concern is not so much with the clever minds that condense the information, given we have been asked to do this since the early days of education, but with the comprehension. We condense information so that someone else can condense the condensed version and so on with the concern that in the end we lose the beauty of the detail. That been said, I am not sure that I would like to be sent every documented word ever created about a new product or service but, at the same time, I do want more than a tag line.

As consumers of information we need to be able find the depth of detail that satisfies our needs, this actually means that we should have access to lowest level if required.

The idea of delivering information in layers is not new but it does take some effort to partition. Web based technologies with links have made this process of layering information significantly easier. Consider Wikipedia and the number of supporting pages that are included in embedded links within a single knowledge domain. Click analysis is the way that organisations can start to track the level of interest (opt in) and the types of information they need to provide to satisfy different types of users. Different techniques used for opting in will also describe the level of interest, clicking on a link tells me you are interested in the knowledge domain, signing up for a newsletter tells me you are interested in the brand.

In presenting information in layers you manage the attention span challenge by providing the detail pertinent to an individual's needs.

Interestingly very few organisations are leveraging this opt in information, are you?

Daniel Tehan

www.linkedin.com/in/danieltehan

It Makes a Difference:Teradata Cares

David Placke — 9/1/2009

At this year’s EMEA Universe event in Istanbul, we have launched our Community Relations Programme, called Teradata Cares, for the first time in Europe. The idea was that we give something back to the city that hosted our conference. To be honest, I was wondering if charity and conferences would really fit together at all. As we all know, conferences have tightly packed schedules – so would the attendees really take the time to participate?

Admittedly, we didn’t make it too hard. We put up a booth with Turkish children’s books, written by Çetin Öner, a well-known Turkish author, and asked our guests to write a greeting card and wrap a book – a personal gift to a child in Istanbul. I was glad to see that it caught on. And you could see that many didn’t take the task lightly, trying to find the right words for a personal message to an anonymous child.

In the end, we handed out about 850 books to kids from schools in the area, and we also sent a magician to one of those schools, making it a special day for everyone. The children liked it, and it is always good to see that you can make a difference.

Raising intelligence for tomorrow
The idea behind the book present was, of course, to encourage youngsters to read more. And nothing is more motivating than a good book. Teradata Cares is intended to support the communities we live and work in, improve education, and help the environment. In Istanbul, apart from three wheelchairs we gave to an organisation, we focussed on education. After all, Teradata is about raising intelligence. And there will be more Teradata Cares initiatives in following months in several cities around the world we will be happy to share with you. As for Istanbul, I have experienced that even little gestures can have real impact.

So let’s keep it up!

Mario Bonardo

Consumers Taking Control – Companies Running to Keep Up

Paul Barrett — 8/27/2009

I spent $50 dollars at a leading book store this week and I didn't intend to do it. I wanted to spend it at their competitor across the street, but because that chain doesn’t bring all its data together I chose their competitor.

Well, not really.

I bought a book on a business trip, a really really terrible book. I think it is the worst book I read since Cold Mountain. Probably worse. (I hope Chuck doesn’t review this blog)

So after suffering through 30 or so pages I was inspired to look online to see if I could return it. I searched on Google and got and answer from Yahoo! Answers. Yes I could and off I went to do just that.

This chain doesn’t have a store too close to my house so I made a special trip, showed up at the counter and was told no I could not because I didn’t have my receipt. What? Google and Yahoo! Are wrong.

Nothing could be done by the staff at this store.

But wait I said “I can show you the answer on my iphone, I can show you the book and where I bought it at your rewards site.” They would have none of this new fangled technology – they had their instructions printed on the back of the receipt in light grey 4 pt font. A document that of course I did not have.

So I left the store – satisfaction destroyed – over to their competitor across the street. Not only did I buy my new book, I signed up for their rewards program – oh and they have a store close to my house.

But the story doesn’t end.

I call customer service, relate my tale of woe and am given $10 towards my next purchase.

This is the new reality for businesses, consumers are in control. If there is bad information on Yahoo! You better fix it. If you have a rewards program, you r staff better be able to access it. And if you want to keep customers, you better train your staff to solve problems, not hide behind the back of the receipt.

The data is out there, find it, analyze it and act on it.

Also, break down the silos – this is how customers buy today – Offline, online, real-time.

Have you changed your buying behavior lately because of new technology? Do you get product information from new sources like friends on Facebook? How is this affecting your business?

Paul Barrett

Connect to a Better Bottom Line

David Placke — 9/14/2009

In the Q3 2009 issue of Teradata Magazine our cover story, Connect the dots, highlights how innovative organizations are linking their back office analytical data repositories directly to their front lines for operational decision support. Teradata CTO Stephen Brobst addresses, in Brave new world, the evolutionary shift of traditional data warehousing to an Active Enterprise Intelligence™ environment and explains how enterprise application integration and a service-oriented architecture close the loop between transactional and decisioning services in Unlock the full potential.

In Right (on) time, Mike Ferguson, Intelligent Business Strategies Limited and Dan Graham, Teradata, discuss right-time business optimization is highlighted as the key that enables employees at all levels to work together to reach an organization’s objectives. And, as Lisa Loftis from Strategic Viewpoints explains in Dialing in customers, when it comes to reach and relevance, it’s hard to beat the immediate marketing approach of the mobile channel.

Gartner shares its research on the 9 fatal flaws that can thwart a company’s business intelligence (BI) efforts. Empowering the bank’s employees with information is critical to the success as Standard Chartered First Bank Korea articulates in Money in the bank, where Teradata Relationship Manager enables the development of more effective and individualized marketing campaigns for its prime customers.

When Western Digital sought to improve its quality-control efforts, a Teradata Data Warehouse Appliance empowered the company providing a Key to quality. Also on the subject of appliances, a team of experts from Teradata details points to consider when conducting a data warehouse appliance benchmark in You be the judge.

In A proven path to success tried-and-true project planning advice provided by experienced experts ensures a successful data warehouse implementation. On a larger scale, Extreme breakthrough explains how a large volume data warehouse can cost-effectively analyze high-volume data types that were previously disregarded because of their magnitude. Additionally, The power within showcases how organizations are saving both money and time by performing in-database analytics through various SAS and Teradata Advantage Programs.

Perfect match outlines the exceptional value of partnering business process management with BI technologies. Learn how understanding and controlling hierarchies and cross-reference relationships among data in Maintaining relationships are made easier with the framework provided by Teradata Master Data Management. And finally, discover how to travel back to Moments in time with Teradata Viewpoint Rewind.

Visit http://www.teradata.com/tdmo/ to access these articles and more.

The Million Euro question

David Placke — 8/25/2009

There is something that we call the “Million Euro” question. This is a question that, once it’s answered, will save your company a million Euros. The point is that it’s a question that has never been asked before. Because either the answer was thought to be impossible or too difficult to get. Or because you would only get to the point of asking the big question in a thought process that involves several small questions that, again, were originally thought to be impossible to answer.

Now you may be cynical about this concept – how can one question save a million Euros? But in my career in Teradata, I have personally witnessed a number of these questions and heard of many others. In fact, it often takes little more than a simple SQL query to answer them.

This usually happens in companies that have integrated their data into a central data warehouse and made it available to business users to ask the questions that have always been bugging them. Questions like these: how come we pay royalties on in-flight movies when the aircraft AV system was not working? Why are we covering repairs under warranty when we can prove that the item is past the warranty period? Why are we recalling all products when we know exactly which products in the field have the defective part?

What about this tricky idea for network operators whose network is overloaded at peak times because people use it to send “codes” home (1 ring = home in 10 minutes, 2 rings = home in 20 minutes): If we provide 5 types of free text messages (saying “home in 10 minutes”, “delayed in office” etc.), will this free up enough capacity to avoid capital investment and increase our revenue?

My thought for the day
When you have hired new colleagues, take them aside after a couple of weeks and ask them what is bugging them. The trick is to ensure that they have been in the company long enough so that they have some insight but not so long that they are already frustrated. Then figure out why these bright people do not have access to the data or information necessary to answer their questions themselves – and fix it. As the famous saying goes, “In the beginner's mind there are many possibilities, but in the expert's mind there are few.”

Now spend 10 minutes after reading this thinking back to when you were new in your company and had all those great ideas on how things needed to change. What was your million Euro question?

Niall O’Doherty

Opt Out

Jamie Flynn — 10/14/2009

It recently came to my attention that I had not received any marketing materials from an organisation that I have been a loyalty member of for more years than I care to remember. When reviewing why I had received nothing, I became aware that I opted out of all correspondence over 7 years ago, my address information was wrong and almost all the information that they had captured about me all those years ago had changed.

This got me thinking about the concept of ‘Opt Out’. I am a big believer of giving the consumer control over how and when an organisation interacts with them. But how, as an organisation, can I manage the changing nature of a relationship?

Anyone in marketing will understand that the relationship we have with our customers is paramount and needs to be carefully managed at all times. Campaign management vendors have to provide a variety of capabilities to ensure that a consumer is not over contacted, and that all contacts are filtered by overriding opt out rules.

But as a relationship changes over time, consumers needs and desires change, and with that change does the opt out status from 7 years ago still stand?

I have the view that all customer preferences should be reviewed periodically. As organisations get more sophisticated with opt out rules, we should give the customer the opportunity to refine their choices. When I opted out 7 years ago there was one choice, either you receive everything or you receive nothing. Now that organisations are providing consumers with a broader range of opt out choices, it is time to get consumers to update their preferences.

As a part of your continuing data quality program, it is time to add opt out options to contact details and preferences as a data set that must be maintained. The challenge is how to communicate the value of updating their preferences to a consumer.

How do you manage opt out rules with a consumer over a life time?

Daniel Tehan
www.linkedin.com/in/danieltehan

It’s Just a Question of Time…Data Mobility & Depeche Mode

David Placke — 8/24/2009

I’m having a hard-time writing this morning… I’m still moved by the sight of more than 10,000 40-somethings having a mass sing-a-long to Depeche Mode, who performed in Anaheim last night. I’m a woman, and we keep some things close to our hearts (read: don’t ask my age). But, I will say this: I attended my first Depeche Mode concert more than 20 years ago.

Things have changed for DM and their fans in the last two decades. The new music is fine, but dull. Dave Gahan has crow’s feet. The men in the audience have traded-in leather jackets for pot-bellies.

And, swaying lighters were all but absent—enter the PDA. People holding PDAs high above their heads to light the arena during a tender ballad; using them to furiously Facebook, Tweet, and Flickr moment-by-moment updates. I mean, I knew before I entered the arena what the band would be wearing, because my washed-up 80’s friends had shared it all with me from the Hollywood Bowl the night before. In real-time. Instead of actually watching the show.

All of that data is collected somewhere. Social networking sites, like Facebook, are aggregating volumes of information about who we are, what we do and when we do it. Within walled-garden networks, that data is held inaccessible to the outside web, much to the annoyance of open-web dependants like Google. But who owns that data? Facebook would argue that it does. Google would argue that you do. And, if you do own your data, is it not your right to take it with you?

The concept is called data mobility—think of it as an analog to mobile number portability. You create a wealth of relationships, connections, and profile information in one social media service, and you then choose to join another. Data mobility allows that information to travel with you, or rather, be accessible centrally so you can tap into it wherever, however, whenever you choose.

The concept raises difficult questions for companies who aggregate user information. How do you balance the need to manage user information to drive your business with the need to be a good steward for data policies which promote openness? If-- in addition to driving improved ROIs for Advertisers and Publishers-- an articulated benefit of targeted ad and content delivery is improved user experience, shouldn’t aggregators of relevant user and behavioral data consider sharing that information for the common good? Is there a common good?

Clearly, there should be a middle-road, where companies can both use the right technologies to understand and predict consumer interests internally in balance with a mandate to share make relevant information available externally.

I’m just not sure I want photographic evidence of me-plus-twenty-years still sobbing like a silly teenager over a few washed-up rockers in leather pants. To quote a poet wiser than me: “It’s better to burn-out than just fade away.” Buh-bye lighters. Hello PDAs. Those are words to live by.

Colleen Quinn

It's Time to Take Your Data’s Temperature

Darryl McDonald — 8/25/2009

Don’t look now but there’s a large wave of data at your company’s doorstep. It comes in the form of customer transactions, inventory, shipments, bills, purchase orders, compliance…the list goes on and it’s growing fast.

What to do with it all? All data is not created "EQUAL"!

You can’t keep it all live because that gets expensive. You can’t throw it out because competition is using more data to compete with you, not to mention the many government regulations require you not to. And you can’t stash it in the garage (tape silo) like your summer camping gear. Besides, there’s real business value in all that detailed data.

The answer is virtual storage. Virtual means that the data placement, organization and management are all handled automatically and the right data are always available for reporting and analysis. That translates into timely answers to business questions, while cutting down on wasted time and money.

Teradata Virtual Storage™ will automatically tag the data you use most often as “hot” (and put it in a faster area of the storage system) and the data you use less often as “cold” (in a slower part of storage) - all without administrative intervention. I might point out that our virtual storage, with its temperature gauge, is unique to Teradata.

Staying ahead of the data wave means taking its temperature and treating it correctly. Have you taken your data’s temperature lately? Send me a note. I’d like to hear about it.

The Geography of Telecommunications Marketing

David Placke — 8/20/2009

Years ago, when I was helping start up a local phone company and later starting up a local cable company, I became aware of how “local” these giant businesses can be. Local can drive everything from pricing to network to advertising. I spent several years working on state level pricing in telecom and the difference between two neighboring states could be enormous.

So I was really excited about the geospatial aspects of Teradata 13. For years, CMOs and marketing analyst have been constrained from really understanding the local impact of decisions. With too much data and underserved by analysis and visualization solutions, the CMO has had to make do with limited information in understanding local market trends and the influence of local factors on performance.

Mash-ups with google earth, just can’t deliver the insight needed. George Salvador from Time Warner Cable just did an interview with LBx Journal where he talked about six ways geographic data can be used better in the cable space. By the way – LBX Journal is a cool new magazine on the use of location data for businesses. (Disclaimer, I have worked and think highly of its publisher Natasha Leger)

One geospatial challenge facing Communications, CMOs is that much of their offline advertising is based on local spend – radio, TV, newspaper. Much of the result of that advertising may end up in an online search or online purchase. And other results could be in local store visitors and purchases.

How does the CMO tie all this together to understand the true impact of their advertising spend. With Teradata Integrated Web Intelligence we can bring these disparate data types together into a geographic data set. Local AD spend, Search and Online Advertising geographical information, store traffic and web site results segmented by geography.

The resulting analysis starts to tie the trail of an advertising dollar, euro or peso into an acquisition, upsell or cross-sell. Loyalty is also influenced by these factors – as well as local competitive influences. Here are some questions I think can be answered with this approach – can you send me more?

How does local ad spend correlate to the local search frequency / clicks?
Does local search activity tie to local store traffic, sales and product selecting?
How do online ads on local sites tie to local store traffic?
How do competitive factors influence local sales performance?
Does the quality of the carriers’s network impact search activity geographically?

Let me know if you have other ideas how to use this data?

Paul Barrett

The Devil (and diamonds) are in the detail – Make sure you use it

June Nguyen — 8/26/2009

Did you hear the one about the house that jumped from $109k to $400m in just 2 years? Not in today’s property market surely? It wasn’t that long ago though!

I was recently providing an update on the SAP Teradata partnership and the Teradata Product Family to Deloitte. I discussed the value that SAP customers can get from the ability to store and process detail level data in a Teradata Data Warehouse or a SAP Business Warehouse on the Teradata Database.

When looking for event based reporting, actionable intelligence or accurate analytic capability, the devil (and diamonds) are in the detail. Summarising data will usually mean that any anomalies are lost and averaged out. This is a massive lost opportunity. Randy Mott, formerly the CIO at both Wal-Mart and Dell pointed to this in a speech made some years ago (referenced also in Ron Swifts blog – “What makes a company great in a time of change”) “When a company aggregates or summarizes their decision-making data … the company makes decisions based on aggregates or averages. ... Average decisions; create average companies, which in turn do not create competitive advantage.”

In the presentation to Deloitte I was reminded of, and told, a story from 2006 where a house in Valparaiso, Indiana was valued at $400,000,000. The real valuation had been $121,000 but an accidental keystroke resulted in the inflated entry into the county systems. The data only remained in the system for a day but it was long enough for it to be fed into the budget planning systems for the county authorities. The resulting budget shortfall was $900,000 and local taxing units were asked to repay $3.1 million they had received in advances. A very simple anomaly that wasn’t around for long yet had a very significant impact. I love the quote from the owner of the house – “Oh, we did a little bit of work on it, but nothing extravagant”.

I’m always looking to hear of innovative ways that clients have used data and also of stories that further emphasise the diamonds in the detail, or the issues that organisations have had with data quality or inconsistencies.

Alec Gardner
Area Pre-sales Manager

What’s this button do?

David Placke — 8/18/2009

Having left the wine business and returned to Europe, I ended up first as a supply chain manager for a leading consumer packaged goods company and then as a management consultant for one of the big global System Integrator practices. Common themes that followed me from the US were information analysis, business intelligence and the struggle to manage data.

In my role of supply chain manager I remember the frustration of not being able to get my hands on data, let alone being able to make any sense of it. Simple facts like inventory levels, back orders, invoice data, etc. were almost impossible to get, let alone analyse. In the end, I persuaded the local IT guy to allow me a direct OBDC connection to the ERP system so I could download the required tables and create my own Excel-based analysis. However, if the executives of this Fortune 100 company had known that read-and-write access had been granted to the Core ERP system they would have had heart failure. The fact that I am just "dangerously" IT savvy - I know what is possible and what I want from technology, but the details around how sometimes elude me – means that I am the last person you want directly accessing your corporate systems.

The Golden Rule

I suppose the golden rule is that motivated people will do whatever it takes to try and be successful. If their employer does not provide them with the tools or information that they need then they will figure out “other” ways to succeed. This is the root of most skunk works in modern business, un-official projects that add cost and complexity to the organisation and sometimes result in spectacular mistakes. I firmly believe that if people are provided with access to the data they need (and even to the data they don’t believe that they need) and some tools to make sense of them, then they will be more productive and engaged. This can only be good news for any company. How can this be achieved? I believe that my colleague Martin Willcox has outlined this very well in Clouds, virtualisation, and the data warehouse - Part 2: Canute and the Wellington Boot

Niall O’Doherty

And The Verdict Is…Targeted Mobile Delivery!

Patrick Hempton — 8/17/2009

There’s nothing like the smell of civic responsibility. I didn’t realize that civic responsibility had a “smell” until I was stuck in a stuffy and, indeed, smelly room with several hundred of my new prospective juror friends at the Los Angeles County Criminal Courts building this week. Think OJ’s trial of the century—same place.

Imagine my surprise, nee, sheer joy when I popped-open my laptop to discover (cue choir of angels here). … WiFi access! Was it really possible to have a pleasant day at Jury Duty? Of course not—everyone knows that’s a stretch! At least the day could be mildly productive—I sent a few work emails, Facebooked the presumed guilt of my prospective defendants in an effort to eliminate my possible selection, and searched for the best restaurant for lunch nearby.

An unusual observation struck me as I scanned the Juror Room—more than half of the folks in the jury room were engrossed in some activity with their cell phones or PDAs. There was the man madly texting and laughing across the way, a woman looking bored watching a movie on her iPhone, and countless others, buds plugged firmly in ears, happily browsing the internet. God love the mobile web. But, my observation shouldn’t have been surprising: nearly 60% of online consumers in the US own an internet-enabled phone.

Many industry experts tout targeted mobile advertising as the “next big advertising trend.” While mobile advertising has been slower to penetrate in the US, mobile ad-spend in Europe will approach $3 Billion by 2012. Still, even in the US, mobile advertising budgets are increasing 25% to 30% year-over-year.

To date, much of the mobile advertising strategy in both the US and abroad has been scatter-shot. Few mobile providers have truly leveraged the wealth of data available across their enterprise. Mobile providers have unparalleled access to massive detail about their consumers’ profiles and behaviors. From call detail records to WAP data to mobile portals, wireless companies have the ability to create rich and time-sensitive profiles to target a user’s immediate location, need and interest-- and do so in combination with a rich history of detailed prior customer behavior.

What’s the benefit of capturing, analyzing and delivering against such rich audience intelligence? More than just happy consumers—mobile providers can command premium rates for advertising inventory, increasing revenues and ROI.

Also, as mobile providers evaluate hybrid content distribution models, with free content a loss leader to drive paid-content and increased data usage, rich audience intelligence enables the comprehensive ability to understand engagement of consumers with content. This allows for improved targeting of content based on that data.

So, while I ultimately did find the ideal lunch spot--Kendall’s Brasserie and Bar, with its juror discount and decent wine list—a successful mobile advertising strategy would have done so for me: analyze my location, time-of-day, past interests and current search pattern to deliver a targeted offer for a restaurant happy to cater to an olfactory-overloaded juror.

Colleen Quinn

BI Technology as an enabler for Post-Crisis Economy

Patrick Hempton — 8/17/2009

Having just returned from two excellent forums, the SAP Australia User Group (SAUG) Summit in Sydney and the SAP World Tour 2009 in Melbourne, I was inspired to hear SAP BusinessObjects’ strategy to assist customers reap high rewards in the global economic recovery.

I was particularly elated to hear SAP promote Teradata Enterprise Data Warehouse (EDW) as a viable part of the SAP BI technology stack. Teradata, as the underlying EDW Database Management System, provides the foundation for strategic and operational decision making. It provides deep insight to management to enable them to gain competitive advantage within the bounds of extremely tight budgets.

I heard from SAP customers who are exploiting their existing SAP BW licenses and “using what they have” to consolidate all operational and strategic reporting and analysis on the one EDW platform. Teradata enables consolidation, from small to very large-scale, of SAP and non-SAP data, and makes real time analytics a reality.

I heard amazing stories of significant cost savings and increased revenue achieved by giving field staff access to live information on mobile devices driven by sales occurring right now. Real-time events are triggered, and rules applied, to achieve dynamic product or service replenishment and/or supplementation of staff, based on real-time demand. The exponential increase in sales resulting from the ability to more effectively service the demand, and reduced costs from eliminating unnecessary application of resources at the wrong time, in the wrong place, was astounding.

SAP customers are able to save millions of dollars across the organisation by leveraging the SAP BI technology stack with Teradata as the underlying “Active EDW” technology.

SAP’s Value Engineering and Value Lifecycle Management Process provides customers with tools and methodologies needed to identify, deliver and measure the value of IT projects. There are currently many SAP customers globally who are planning their BI Roadmap to include SAP and Teradata as technology enablers to achieve real value in the current economic recovery.

What hurdles or roadblocks you are experiencing in achieving similar astounding results in your organisation?

Nina Corney
SAP Domain Lead
Teradata Australia

http://www.linkedin.com/pub/nina-corney/2/44a/802

Millennials have never lived in a Batch World

Patrick Hempton — 8/17/2009

Paul Barsch at the smart data collective has a podcast on the Zero Latency Future. It reminded me of something one of our customers said about younger consumers. They have never lived in a batch world. One example was a woman who told me about playing college basketball. After a loss the coach wouldn't let them talk – so they spent the whole bus ride home texting to the person next to them.

The Millennials – or recent grads / coming grads have lived in a world of instant messaging, text, tweeting and constant facebook updates. You are getting a pizza now? I am parking the car. (You can search on Stupid Facebook Status if you are brave.)

But this has big ramifications for companies – especially in the customer service space. These younger customers don’t understand batch processing with records being updated overnight. They expect your systems to talk to each other in real-time. As customer expectation rise, basic performance needs to improve just to keep satisfaction steady.

A wave of customers with high expectations means that companies have to start thinking about improving customer service now. Part of it is integrating the real-time web into the contact center. Sheryl Kingstone from Yankee Group just completed a paper on CRM and Social Media and how it is starting to affect the customer experience and the enterprise.

I think we are just beginning to see the impact of real-time web on businesses. There have been three impact points so far – brand monitoring, Geo-spatial mash-ups and now bringing real-time web into customer interactions. But it will have far reaching affects. Information has to move faster around the enterprise - both the data within the enterprise and the eco-system of data that surrounds that enterprise on the web. Bridging the gap between the internet and the enterprise will be a constant source of new innovation.

I am curious if anybody out there is experimenting or implementing real-time web applications within the enterprise and what they are learning. Please let me know.
-------------
On a different topic, Teradata web analytics partner Webtrends released a new version of their analytics tools this week – to great acclaim. While there is a lot of innovation here, the coolest part is Story View that gives analysis in written format. Check out their page.

Paul Barrett

Uncorking Innovation!

Patrick Hempton — 8/14/2009

I worked in a winery in California once. Straight out of college, this was a great start to any career. As a Mechanical Engineering graduate from Ireland, I began in the distributor industrial engineering group focused on manly things like warehouse layouts, conveyor system optimisation, delivery route planning – basically anything to increase the efficiency of getting cases of wine to the retail shelf. At an average age of the department around 50 years, I was quickly identified as “the kid who could use computers.” In this case, this meant spreadsheets to do analysis. And so began my journey in business intelligence and analytics. We worked on some great and some weird projects. One even included a sales and marketing information system that captured all invoice, shipment and distribution data, a precursor to the BudNet system that Anheuser-Busch runs on Teradata. Another was evaluating a data warehouse solution to host integrated data from internal and external sources (IRI, Nielsen) – that sounds familiar! I even ended up on a small team with 3 other guys from the winery; a bottling supervisor, a tank farm engineer and my old boss in distributor engineering, and a consultant from Rath & Strong (the best consultant I ever worked with and a real mentor – thanks Jim) tasked with re-engineering the whole production process (glass plant, label production, crush, blending, bottling, etc.)!!! Now that was a challenge and I loved every minute; the dynamic of this diverse team, the scale of the problem, the access to senior management. I learnt what is possible when people leave aside their old thinking and really work together. I was involved in innovation and I was hooked.

I often wonder having witnessed so much innovation and change in the wine industry (an industry that goes back centuries) why is there often so little innovation or risk taking in most businesses today? Perhaps shareholders and the markets stifle innovation too much? Perhaps senior management lack vision (I think that is the polite word) to embrace change? Perhaps they lack the people or just can’t notice them?

Niall O’Doherty

La Trahison des Données

June Nguyen — 8/25/2009

There is a potential fallacy that we hardly ever address: Isn’t there a risk that business intelligence systems – seemingly – become just too good? That they describe the market, the customer etc. so accurately that business users will consider this information as reliable as first-hand experience and forget that they can only see what they are being shown? Maybe these users should receive frequent warnings that say something like “Ceci n’est pas un pipe.” These are the famous words that the Belgian surrealist artist René Magritte wrote under his painting La Trahison des Images, which shows – a pipe.

The comment is intended to remind viewers that they are looking at a piece of art, not the actual objet. They can’t smoke this pipe, no matter how well it’s depicted. They can’t even tell what the actual pipe really looks like or whether it has existed in the first place. This is an essential idea to keep in mind when dealing with information systems. It’s tempting to believe that you have full and valid information at your hand, especially in highly endemic systems. But is it really true? During the recent credit crunch, some people in the financial markets found out that it wasn’t. They thought to be Masters of the Universe when they were, in fact, the Boys in the Bubble. Business users, therefore, must rate the informative value of their various indicators. At the same time, they absolutely need to be able to rely on the quality of the underlying data.

Ensuring that data quality is at a high level is one of the most important functions of any data management unit. It is an interminable process that sometimes gets underrated. The backbone of broader quality initiatives is a sound master data management. Master data, as opposed to transactional data, describe items like product and customer names, addresses etc. These data will be stored and reused in the long run but very often change over time. E.g. people change their names or move their residence to new cities. Here they may make a new contract with their previous telecommunications provider at a different point-of-sale than before and may be registered as a new customer, not as the one already known. Thus the quality of the master data deteriorates, unless there are organizational processes put in place that make sure that the customer data are permanently reconciled. Without such procedures, any analysis of the provider’s customer wins and losses would be flawed from the start. So let’s not forget: Ceci n’est pas un client.

Obviously, the EDW approach of storing all data in one place, organized by a logical data model, facilitates quality-ensuring procedures of master data and other data. Still it’s not a trivial matter. The logical data model must be flexible enough to capture unexpected categories like e.g. official (as opposed to biological) birthdays in some countries. In retail, product data must capture temporary changes in package sizes, which are typically used during special offers, to make any meaningful analysis of the sales results possible. And just to return to the financial sector, detection of money laundering would be a great deal harder if banks couldn’t figure out whether a single person own several accounts within (and beyond) their organization. An effective master data management means that the data betrays the fraud, not the bank. And that’s my idea of la trahison de données.

Eric Joulié

Active Data Warehousing for SAP BW – what does it mean?

Patrick Hempton — 8/17/2009

Have you heard end-users ask the perennial questions -

Why should I use SAP BW reports with last-night’s data when I can get real-time information from SAP?
Why should I use SAP BW when the information in it is too summarised?

These are legitimate questions!
It becomes difficult to justify the large costs of data warehousing when, typically, the information contained in data warehouses is not current, nor is it available to the depth of detail required by the vast majority of users. Operational users, in particular, are not very interested in old information and they are very frustrated when the numbers in SAP BW do not match those in SAP. They legitimately need real time, integrated data most of the time.

The ability to run SAP BW on Teradata now opens a whole new world of capabilities. Teradata’s strengths of super fast loading and super fast performance, together with super fast searching, with SAP BusinessObjects Explorer and other BI tools, provide true capabilities of active enterprise data warehousing and pervasive BI with minimal technical heartache and complex performance tuning.

The inherent complementary strengths of SAP and Teradata enable vast amounts of very granular data to be loaded into the SAP BW Teradata single EDW in real-time.

The purpose and function of the Enterprise Data Warehouse now becomes a truly operational system and its place in the applications architecture is moved down into the operational world rather than where it is typically shown above, in the reporting and analytical layer.

Real-time operational and event data is loaded into the EDW, fast analytics are performed on integrated data from multiple systems, and the result or decision is returned back to relevant business users in lightening fast, real-time.

Imagine the benefits to call-centre staff that needs to be aware of all customer interactions, operational users and managers who can make instant decisions, and customers who are not repeatedly asked the same questions by different parts of the organisation.

How do you think active data analytics may assist your business?

Nina Corney
Teradata SAP Domain Lead
http://www.linkedin.com/pub/nina-corney/2/44a/802

^rhq3uijsfx

To Find data Diamonds in the Rough Cast a Wide Net and Test, Test, Test

Patrick Hempton — 8/17/2009

This is a post about strange, but useful data. When I was a kid, there was a book called Strange but TRUE Baseball Stories. It had interesting stories like Pete Gray who was a one armed pitcher in the majors that grew up about 15 miles from my home town of Scranton, PA – no I don’t work for Michael Scott., the world’s best boss.

Anyway, what I want to talk about today is…

Carburetors. Back in the day when I was Director of CRM for a MCI, I spent a lot of time trying to get more data on people’s carburetor so that we could sell more long distance. I know that sounds a little strange, what do carburetors have to do with telephone service? Well, it turned out quite a bit.

Back in the nineties, it was easy to get car registration information – and due to the quirks of the data, they would collect what kind of carburetor was in the car. Because almost everybody who is an active consumer has a car, it was a very complete list. Also, because cars are registered to people at their home, it was easy to match registrations to actual people.

But what was interesting about carburetor was how predictive this data was about a customer behavior – especially likelihood to buy MCI long distance. There were basically three carburetor types: a Four barrel, two barrel or no carburetor – which meant fuel injected.

If a customer had a fuel injected car, then they had a new or relatively new foreign car. If they had a four barrel carburetor, they had an old American car – like the 1968 Pontiac my Dad bought and I drove as a kid.

Once we matched the data up to our recent sales and did regression analysis, it was easy to see how predictive a fuel injected car was to our sales. Customers with fuel injected cars were more open to trying new products than customers with older cars. American car owners were more loyal to the competition, so that made them less likely to buy. This piece of data was probably responsible for tens of millions of dollars of revenue over a few years.

But the real insight came through creativity, data mining and testing. Do you have a story about a hidden gem of data that strangely is tied to your company’s performance? Let me know.

Paul Barrett

Innovation + Affordability = Enterprise Intelligence

Darryl McDonald — 8/24/2009

Who needs intelligence? It’d be hard to find a company that says it doesn’t. But when it comes to the daily grind of IT, most firms are focused on processing lots of data fast.

Because staying a step ahead of growing data volumes is no small feat. But can you gain insights and intelligence about that data at the same time? You can.

At Teradata, our engineers have worked hard to move our new platform into a realm that brings affordable intelligence in the data warehouse to a broader segment of the marketplace. Those customers who previously thought Teradata was only for large customers should check out Teradata 13.

We boosted overall performance by 30 percent. If we consider the combination of software and hardware platform performance, Teradata performance is seven times faster over that same time period.

The way I look at the Gartner Magic Quadrant, Teradata has been the leader for 8 years running by outperforming all of the competition in the marketplace, including Oracle and Netezza. With Teradata 13, we are extending our lead by adding another 30 percent of overall performance, and new important capabilities. Teradata has more than doubled its software performance based on improvements in the last three years. If we consider the combination of software and hardware platform performance, Teradata 13 performance is seven times faster over that same time period.

Teradata 13 is the best price-performer in the marketplace and is the foundation that powers the industry’s only purpose-built data warehouse platform family, which includes the Teradata Active Enterprise Data Warehouse platform, the Data Warehouse Appliance, Extreme Data Appliance, Data Mart Appliance and the Data Mart Edition.

Teradata 13 provides customers with an affordable foundation for a wide range of business analytics spanning departmental and enterprise intelligence for all sizes of companies. And we’ve raised the bar by integrating geospatial processing, and Teradata 13 is now optimized for online analytical processing. Customers can now load location data into the data warehouse and that means a company can quickly perform analysis about the location of customers to correlate buying trends in different areas.

All of this raw performance translates to agility and speed. But it’s the underlying tools that really bring intelligence to the customer. Because, at the end of the day, if you can’t discover insights in your customer data, the brute horsepower just doesn’t matter.

2gcekfjnp6

The $65 Movie…. My Case for a Universal “Pay-Once, View Anywhere” Content License

Patrick Hempton — 8/17/2009

I took my gorgeous three-year-old son, Nathan, to see Ice Age 3: Dawn of the Dinosaurs last week. After four solid weeks of begging, pleading and pointing at every bus and billboard bearing the movie poster, I conceded. We even caught a 5:30PM show. On a Friday. This means I begged-off work by 4:45. Shhh!

Not including the popcorn, hotdog and four attempts at the motorcycle video game, the movie cost me $12 in tickets. (Since I fibbed about Nathan’s age—“tell ‘em you’re two!”—I only paid for a single ticket). But, trust me when I say this will not be the extent of my Ice Age 3 viewing spend.

Case in point—Ice Age 2:

Movie ticket -- $12
DVD -- $14.99
iTunes Download (you need SOMETHING to watch at the restaurant) -- $14.99
Hotel VOD (Teradata duty calls—and the kid tags along) -- $12.99 (give or take a few bucks)
The replacement DVD (because Nate is three-years-old, and frankly, DVDs aren’t that durable; though luckily it’s now on sale!) -- $9.99

Grand Total: $64.96

But, wouldn’t life be grand if I could pay for Ice Age 2, or any other title, one time and own that intellectual property (IP) right across every post-theatrical platform-- mine to watch as many times as I choose, any where for single price? Think of it as a universal license.

Well, life just might be grand after all. While it might seem counter-intuitive from a recurring revenue perspective, large studios and content owners are exploring how to deliver these capabilities to increasingly entitled (read: spoiled) consumers who, like me, are beleaguered by multiple investments for the same intellectual property right.

Why would a studio choose to seemingly risk revenues with a single payment universal license? There are three great reasons, at least:

The ability to “know thy customer”—web-based channels create significant opportunities to measure behavior. Imagine being able to understand a user’s behavior across titles, franchises, even competitive content creators?
Because they must—in knowing their customers, studios are also learning that customers expect this degree of access in the digital realm.
Piracy Protection—a universal- license is a legal license. Giving a customer access across channels decreases the motivation for illegal activity to access that content in those incremental channels.

So, if it makes business sense and customers are demanding it, what’s the hold-up? There are several technical hurdles content owners need to cross to make a universal license a reality. One of the more onerous is intelligent, centralized management of Intellectual Property Rights.

Studios and large content owners are already challenged managing intellectual property rights today across multiple platforms and geographies. Layer onto that the increased complexities associated with managing a right for a given consumer across those mediums and locations. It’s enough to make your head-spin.

Much of Teradata’s foundational work in Media & Entertainment has been to design solutions to address these core challenges:

Master Data Management - we understand that one of the first challenges to overcome is to ensure clear data governance, ownership of the data and accuracy of the data. Think: Ice Age 2 vs. Ice Age Two
M&E-specific Logical Data Model – imagine a logical data model that reflects the realities of the Media & Entertainment business... today. Our LDM offers a blue-print to build-out a tailored data model, with rich support for intellectual property, rights, payments, multiple and derivative formats, and digital distribution.
Centralization of Title and Rights Data – how many data warehouses does your company have? And, how many of those repositories contain only a slice of the information about a title’s rights, descriptive metadata, licensing windows? You’re not alone in the need to consolidate.

If this sounds complex, it is. Good thing there are partners like us who can help you navigate the path through this and other challenges facing M&E companies today.

No one wants to see this road paved quickly more than me—I mean, at $65 a movie, I can’t afford to wait too much longer!

Colleen Quinn

Did you ever want to be a spy when you were a kid?

Patrick Hempton — 8/17/2009

I remember when I was little I wanted to be a spy. So Mum and Dad got a book that taught us about codes, disguises and drop off points. My brothers and I then spent time writing fancy codes and hiding them from one another. Then came James Bond, and I wanted to be a spy for very different reasons. The other day I was looking at the CIA web site (as you can see the passion has not left me) where I came across a document talking about ‘Structured analytic techniques for improving intelligence analysis’. Quite a mouth full I think you will agree.

There are a number of techniques discussed that are relevant to us all that work in the arena of problem solving. We are all familiar with brainstorming, and with risk analysis, but how many of us analyse assumptions in depth. How many of us then understand the impact of poor assumptions or poor sets of assumptions? How many of us consider the source of data and analyse the strengths and weaknesses of its quality and our understanding of its content?

Many of us have experienced situations were incorrect data or poor assumptions have impacted our lives, yet we continue to accept truth in data. Data is as good as the analysis and those that perform it. It should be questioned and, when challenged, re-evaluated.
There are some good techniques in the white paper for those of you that aspire to be a spy or are continuously looking for a better understanding of the intelligence you create in your respective organisations.

I would be interested to hear from anyone who uses some of these techniques and how it has helped.

Daniel Tehan
www.linkedin.com/in/danieltehan

P.S. I no longer want to be a spy, so no job offers please.

It’s data, Jim, but not as we know it - Part 2: A picture worth a thousand hours

Patrick Hempton — 8/14/2009

Whatever we call them, the non-traditional types of data that I discussed in my last post can greatly extend the utility of “traditional”, record-based data warehouses.

Consider the following trivial example. Suppose a convenience grocery retailer loads sales data to its data warehouse in near-real time. Examination of the data as it streams into the warehouse and is integrated with the other data captured there reveals that sales for the Nowheresville store are well down - against both forecast sales and prior-year sales.

By itself, this information can tell an analyst at the retailer that there is an issue, but not what the root-cause of the issue is: maybe a new competitor has opened for business nearby; possibly the store is in need of refurbishment and has seen a long, gradual decline in sales; perhaps there is an availability problem with a key line (milk or bread) and shoppers are abandoning their shopping baskets at the point that they discover this fact; maybe the store has the wrong assortment for the demographic that it serves, etc., etc., etc. A well-designed and comprehensive “traditional” data warehouse should enable our hero – after further analysis – to establish the root cause for the decline in sales in these scenarios. But there are other scenarios in which a data warehouse that captures only the record-based data captured from the transactional systems that run the business on a day-to-day basis may be less useful.

Suppose that the problem is a temporary issue with car parking. There are two convenience stores close to where we live and as we are a moderately disorganized family we frequent both of them on a disappointingly routine basis. Neither is quite close enough to walk to and so for us, buying milk, bread or any of the other daily essentials that we have inexplicably managed to overlook again on our big weekly shop involves getting in the car – and parking it on arrival. If we can’t park at shop A (a Teradata customer, incidentally) then we generally head to shop B (also a Teradata customer). I see lots of other potential customers exhibiting very similar behaviour; either that, or they just like driving around.

So now suppose the analyst at our convenience retailer also has a feed of still images – one every hour, say - from the CCTV cameras at the store available to him alongside the record-based data recording the facts of the sale of a carton of milk, a tube of toothpaste and so on. Examination of these images may reveal that the car park at the Nowheresville branch is currently being re-surfaced by contractors and that there is – literally – nowhere to park. All by itself, this simple fact probably explains the dip in the store’s sales; and if the decline in sales was sharp and coincided with the start of the work, then we can probably safely conclude our analysis without further investigation and without going to the (considerable) trouble of reviewing the store assortment, availability levels, etc., etc.

Of course, this is a trivial example; our intrepid analyst could have achieved a similar result by calling the store’s duty manager to ask him or her if there were any exceptional circumstances currently affecting the store - and without going to the time and expense of loading the image data to the data warehouse. The more interesting examples of the exploitation of these non-traditional data types typically involve “fact extraction” from large volumes of multimedia data (digitized patient medical records, the free text fields that garage mechanics use to report the issues that they encounter when servicing cars, etc., etc.) to create “square” meta-data that can be easily integrated with traditional, “square” record-based data in complex queries that run against the resulting, large, integrated data-sets. In very many cases we will still need to retain the original multimedia data alongside the extracted fact data to support investigations that require access to the “raw” data and because enhanced fact extraction techniques may become available to us as we learn more about exploiting the multimedia data. But in most cases, these data will be relatively “cool” (infrequently accessed) compared with the extracted facts. And they are not - repeat not - unstructured.

The web is the database?
On our recent tour of the EMEA region, Teradata’s CTO Stephen Brobst provided a fascinating example of the use of these fact-extraction techniques when he discussed how leading organizations are starting to use data harvested from the web to better understand how we consumers feel about the products and services that we are consuming – and whether we like them well enough to recommend them to our friends or not.

As Stephen explains in his presentation, in the US it is already the case that the majority of adults in all age ranges up to age 55 now conduct their primary research on which products and services to buy on-line, principally through reviews and recommendations from specialist and social networking web-sites and from trawling the blogosphere. In many cases this information is either already in the public domain, or users can be induced to share it with organizations that appropriately value the relationship that is created or reinforced with the consumer as a result.

From a marketing perspective, this is 24-carat gold information: unvarnished and evolving feedback in the form of narrative text posted by users on web-sites and describing their experiences; harvested for free, or at a cost that is greatly reduced compared with the cost of traditional market research; and increasingly free of the selection and sampling effects that can blight traditional research. The extracted data – cold, hard facts and statistics – are analyzed by leading organizations alongside sales data and all of the other traditional data that originates from inside the organization. Apple Corporation, for example, knows precisely which iPhone features users value the most and has a pretty good idea about the functionality that consumers would like to see enhanced - and how much they would be willing to pay for those improvements. It may be no coincidence that Apple shifted over a million of the new iPhone 3GS models - in the teeth of a fierce recession, remember - within only three days of the product’s launch.

(Incidentally, if you’re reading or analysing this, Mr. Jobs, I really like the spotlight search functionality introduced in release 3.0 of the iPhone software, but I would like a free upgrade to the new 3GS model with the built-in compass even better. And I’ll happily tell you what I think about my MacBook if you could see your way to upgrading my keyboard to the oh-so-cool-glow-in-the-dark version.)

Alas, I sense that I stand about as much chance of banishing “unstructured data” from the vocabulary of technical marketing as I do of relegating “paradigm” to the relative obscurity that it enjoyed until only recently, or of snagging a free iPhone upgrade. But it beats working for a living and it’s good for my blood pressure. And it has kept me away from the Apple Store - and so out of trouble with Mrs W – for the last several hours.

Martin Willcox

Cultivating New Marketing Data

Patrick Hempton — 8/17/2009

Most likely your business has not been around as long as baseball. But you probably have some standard marketing Key Performance Indicators (KPI) that everybody trusts or at least intuitively understands and can quickly decide if things are going well or if things need to change. Are you thinking about new KPI’s? Can they change your business dramatically?

Baseball is. In fact baseball has been through a KPI renaissance in the past few years. Quantitative analysis from the likes of Bill James and Sabermetrics, books detailing new player valuations such as Money Ball and now a new fielding tracking system that will provide detailed data on every play.

A new camera and software system in its final testing phases will record the exact speed and location of the ball and every player on the field, allowing the most digitized of sports to be overrun anew by hundreds of innovative statistics that will rate players more accurately, almost certainly affect their compensation and perhaps alter how the game itself is played. (New York Times July 10, 2009)

If baseball can continually improve its statistical analysis and find new, innovative ways to create competitive advantage from data then your business can too.

And marketing is at the forefront of this opportunity. The rise of web 2.0 has given consumers more of a voice in the marketplace – in fact it is probably an equal voice in the marketplace. So let’s look at three new data sources for marketers that can help transform your business.

Social Media. More then ever it is important to watch and listen what is being said about you, your competitors and your supplies – and it has never been easier. With solutions like CrimsonHexagon, Radian 6 and many others, developing a social media monitoring program provides new insight into how you products, brands and company.
Voice and Text Analytics. Transforming customer interactions from call centers, chat sessions, e-mails provides new qualitative info for companies. One telecom executive I talked to told me how voice analysis had saved her company from implementing a multi-million billing transformation. The call center codes said billing was a problem and some people wanted to change the format of the bill – a major process. But it turned out that people were complaining that the didn’t understand how they would be billed – in advance in stead of in arears. Simply changing consumer communications reduced the problem.
Web Advertising, Search and E-mail detailed data. (Okay, this probably brings us to five data sources. ) Most companies look at their e-mail, search and ad data at a summary level, but there is incredible insight to be gained in the details. I worked with one company that had 98% deliverability of their e-mail and they were very happy. But looking at the details, we found one ISP that was rejecting half of their messages and creating a large number of false, automative unsubscribes. Fixing this problem was easy and increased their universe and sales significantly in that region.

Search and Ad data at the campaign level can provide very good insight – but moving beyond the campaign opens up opportunities to see multiple impressions on a segment or customer, the ability to look at geographic performance that can be tied to brick and mortar locations. Bringing this data together in your data warehouse and increasing the granularity of this data can greatly increases its value.

These data sources may not be new at your company – what are you looking at today that you didn’t last year? Where do you see new data coming from in the next 12 months?

Next time, a data diamond in the rough….

Paul Barrett

Online Retailing – Take2, some food for thought and a big BUT!

Patrick Hempton — 8/17/2009

For today’s consumers ‘Online’ is part of life, like breathing … Is your brand breathing? Is it present virtually as it is physically? Are you providing a socially engaging experience?

As consumers we interact across time zones and physical boundaries, leveraging the web, mobile technology and social networks like Facebook, MySpace and Twitter (… remember email?) in our purchasing decisions.

Any news travels at the speed of thought and this reality is critically relevant to retailers as consumers relay their, “did you know” and “you won't believe”, experiences in a flash. Bad news (experiences) travel even faster!

Talk about effective marketing! If you have an online presence and are committed to making online engagement relevant and rewarding, then you are on the bus ….and if the re-emergence of ONLINE is not (back) on the agenda for you, it should be!

Here are some fundamental reasons why ‘Take2’ for an online retail strategy?

The explosion in an internet savvy population - retail consumers who have matured to a 360 degree application and appreciation of the web in all its forms as a part of daily life.
The expensive lessons (for retailers) of the dotcom era have been learned.
The dramatic and continual improvements in technology behind websites.
Payment security and general trust issues are no longer the deterrent they were.
Consumers are responding to global players who are ‘eating the lunch’ of local retailers through their online presence.
The majority of consumers are using the online channel for researching ‘before they buy’ end up making their purchase in-store.
It is a changing world but get used to it, because if managed with the consumer experience top of mind, it all good news!

Here comes the ‘food for thought’ piece.

Extending your retail brand to include a meaningful and relevant online option will not only deliver incremental returns in terms of sales or market share but more importantly ‘consumer mind share’.

It provides your consumers with a real opportunity to engage with your brand, your products, your marketing and your organisation on every level, if you encourage the interaction, learn from it and reward it.

And here is the big BUT.... learning from it?

Online investment considerations classically focus on marketing, product sourcing, logistics and infrastructure challenges.

Little or, at least, nowhere near enough attention and proactive planning goes into understanding online and critically offline (in store) consumer behaviour as a combined strategy and, therefore, the combined impact it has on the retail brand.

The integration and interpretation of hard and soft data across both physical stores and the online channel is vital if your product and service offering and experience is to evolve with the expectations of the consumer.

Fully understanding and appreciating the web browsing behaviour of new and existing visitors to your site is essential, both before they make any purchases, when they’ve made purchases. More importantly it is essential to understand if and when they return a product, why they return it and what they said about it and the organisation.

If understood and leveraged this insight will be more beneficial than additional marketing campaigns or discounts.

An online strategy, meaningfully pursued, implies lots of additional and different data to integrate, consume and understand in order to leverage – this is where the pain and the pleasure of an online strategy intersects.

What are your plans for Web Intelligence? How do you intend to integrate your Online and Offline Business Intelligence Strategies?

Don’t shy away from the challenge, engage in discussion. It could lead to a Simple, Powerful and Affordable solution.

Geoff Dyer
Retail Industry Specialist, Teradata
http://www.linkedin.com/in/geoffdyer

Who Are the Australians Shaping Our Future?

Patrick Hempton — 8/17/2009

An interesting question was posed by @nickdallariva on Twitter recently; “Can anyone think of an Australian that should be speaking at TED next year?” This received a “Good point. Can’t think of many” from one of Nick’s followers, which got me pondering “just who do we think about, and why?”.

Firstly, I think it’s important to know what TED is. TED sttarted out as a conference for Technology, Entertainment and Design focussing on ‘Ideas Worth Spreading’. One of the things that I find compelling about TED is that the speakers (in the main) are people you have never heard of, and they are truly thinking of BIG IDEAS to shape our future. Scientists, Philosophers, Engineers, Environmentalists, Designers, Artists, Politicians… – a very diverse group. The other compelling thing is that all the talks are online to watch.

That is the key part of the TED vision to me: ‘Spreading’, and there is a lot of discussion today especially about the spreading of media power. Interestingly, a TED speaker provided a fundamental insight to me. Clay Shirky spoke about how social media can make history and the key takeaway for me was about the nature of communication where most mainstream media is “one speaker” to “many listeners”, but with no ability to create a conversation. So most of our media has been about “what is the single largest group that can be spoken to” in order to achieve whatever the goal is, typically audience share and advertising/subscription revenue. This has taken us to the point where politicians, sports stars and entertainers dominate the “names” we think of when we think of prominent Australians. So what can we change so that we think of “names” in other fields?

One other response to Nick said “our tech and science community are under promoted, but think TED could change that”. I think it isn’t just TED, but it is the whole social media phenomenon. Social media sites and capabilities such as YouTube, Twitter, Facebook, LinkedIn (and many others) are about creating conversations in a group of any size where there is a common interest. I have found it fascinating to watch my Twitter usage, followers and people I follow change based on the people, ideas and conversations I’m introduced to, and the people, thoughts and ideas that they in turn introduce me to. Some of these people are also thinking of BIG IDEAS, and haven’t found the forum yet (and never would find a forum but for social media).

To create a forum for those people with the ideas the onus is on all of us to participate and pass on what you find interesting, innovative and challenging. One of the most amazing things Twitter introduced me to is an innovation festival run by AMP, not the first company I would have thought of when it comes to innovation, but clearly one of the leaders despite my bad assumptions. Check out www.amplify.amp.com.au to see what a large corporate can turn on to foster their own big ideas.

So why is this in a Teradata blog? Well, I think at the heart, it is what we have been trying to do for years – put the power of information in the hands of the people who can do the most with it. Working locally with our customers, we have helped turn Campaign Management into Conversation Management, and being relevant and timely for individuals, not segments. We have helped customers get a better understanding of their business in order to create better customer (and citizen) experiences. It is why I enjoy doing what I do so much, and why I am here still some 13 years later (though another Twitter introduction – James Gardner makes me wonder if that is a good idea .

So I guess the answer to my question is that we are all shaping our future, obviously, but some are trying harder than others. In case you’re curious, my nominations were Dr James Bradfield Moody , one of his projects that I think would qualify is solar cell polymer and Prof Graeme Clark, inventor of the Cochlear implant. Who are your nominations (and I’ll pass them on to Nick)?

Sean Kain

www.linkedin.com/in/seankain

Keeping up with the Joneses

Darryl McDonald — 7/30/2009

The idea of benchmarking isn’t new. We constantly compare our homes, our cars, and our jobs to those around us. In business, the idea is embedded in our daily “to do” list as we strive to stay competitive and highlight how we’re different. For some reason, we continually feel the need to compare our performance against others in the markets and customers we serve.

And often, it’s for the good of the company. It’s through competitive benchmarking that we’re able to discover our strengths, identify the areas where we can improve, and ultimately define best practices for our company.

If you focus only on the cost, you are at risk of missing the value. When we talk in terms of pricing and value, we consider a number of factors:

What you get. A single-purpose solution may be cheaper, but if you add up the total cost of all you need to get the same result – are you still saving money?
Lifespan. Technology purchases are investments. What is the lifespan of your purchase and does it fit your organization’s long-term plans? Can you co-exist with multiple generations/releases?
No trade-offs. To receive equal performance or capabilities, do you have to make concessions on timeframe, availability or another dimension? What is the total cost of those tradeoffs?
Support. What services are available to you as part of your purchase?
Reputation. How is the solution rated – by independent experts on qualities that matter to you?
Results. You can’t manage what you don’t measure. To make that point, when it comes to ROI, do you know if your “R” is ultimately larger than your “I”?

Let’s face it, the most valuable asset you have to help you keep up with—and surpass—the Joneses is your company’s data. That’s exactly what makes it priceless. What do you get from your data?

40 YEARS on: From MOON to CLOUDS: What’s HYPE, CREATION and REINVENTION?

Patrick Hempton — 8/17/2009

Major events always make cause to reflect. "Where was I when…?" "What has changed, improved etc?" Well, when Neil Armstrong took his first step onto the moon 40 years ago I was sitting in a class, learning to be an IT professional (ekk assembler (CDC COMPASS), FORTRAN, and even more revolting COBOL). Our instructors had brought in a TV (black and white) so we could watch it live. Being in Canberra we were very close to the space communication dishes supporting NASA so there was more than a passing interest. These were classes run by our employer (the Commonwealth Government) as computing was not yet taught as an undergraduate course. So the 100 or so of us watched in awe as man set foot on the moon. Much more tangible than watching Sputnik flying overhead at night with a dog or monkey in it – still amazed that the animals were alive.

So for the next question – "what’s changed?" Well, being in the computer industry "still" the answer is "a long way" including qualifications, coding/development techniques, hardware and software, but more fundamentally, where and how computers are deployed. Moore's Law holding true and delivering the integrated circuit delivering its exponential growth continues to fuel our capacity to "deliver more with less".

But during the same period many of the vendors have also led with hype and buzz words. How the IT industry likes to reinvent itself! So while there is absolutely no doubt that we are delivering new capability on an ongoing basis, I still view the "new trends" with a great deal of scepticism until the "vapour ware" becomes tangible and proves to be more than just an old idea re-invented. Indeed, SaaS vendors are creating new markets for ERP, CRM, web analytics, supply chain, and other application areas. Gartner and other analysts comment on these new ideas through "Hype Cycles" and rightly so.

So we now have the notion of "clouds" in computing and Gartner comment that "Cloud computing is the most hyped subject in IT today." According to Wikipedia "Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet." http://en.wikipedia.org/wiki/Cloud_computing

But is it really something new? "Users need not have knowledge of, expertise in, or control over the technology infrastructure in the "cloud" that supports them." Seems like a Bureau Service or Outsourcing or an ASP model of delivery, all "reinventions" – but what seems to be new is the combination of technology enablers behind the delivery model. The model can be delivered in a "Private cloud" which is an internal "charge back" framework or the "Public cloud" which is an external "bureau service" with shared infrastructure.

But "the proof of the pudding is in the eating" and at Teradata we have been delivering dynamically scalable solutions with virtual components for years – and many of our customers deliver these "services" to their vendors and customers through the internet – they just don't charge for it. eBay has combined the agility and ease of use of data marts with the advantages of a centralised Teradata data warehouse and has delivered "Analytics as a Service” through the Private cloud.

Call me a cynic but Cloud computing is still a lot of hype, with some creation, and lots of reinvention. So, where were you when Neil Armstrong landed on the Moon – and how much do you think Cloud computing is "Hype, Creation and Reinvention?"

Christine Page-Hanify

It’s data, Jim, but not as we know it - Part 1: What the echo of the Big Bang tells us about the nature of information

Patrick Hempton — 8/14/2009

Possibly I am just turning into a grumpy old man in my middle-age, but there are two words that when used together annoy me beyond almost all reason – yes, even more than the “p-word” that has featured in two of my previous posts: “unstructured” and “data”.

Despite what some vendors – and some commentators, who really should know better – would have you believe, there is nothing remotely formless or “unstructured” about “new” types of data, like image files, audio files, text-based documents, XML documents and so on. Of course for the most part these data hardly qualify as “new”, either, but don’t indulge my pedantry by getting me started down that road.

Data is merely information that has been encoded in some way and the only truly “unstructured data” is “noise”; random signals, representative of nothing much more than a system in equilibrium with its environment. A picture, a song, the complete works of Shakespeare – these are all forms of information and they are emphatically not “unstructured”.

To see the truth of this, take, for example, a GIF file (make sure that it is one that you don’t much care about, or a copy of one that you do) and open it with a text editor. Now mess with and/or delete some of the bytes at random, save the adulterated file and then try and open it with your normal picture editing or viewing software.

In fact a GIF file is highly structured and includes meta-data in the header that, for example, includes a colour table; the height and width of the pixels represented by the bitmap that follows; whether the image is animated or still; etc., etc. All this meta-data is then followed by an array of bytes that define the actual bitmap bits and an end-of-file marker. Monkey with this file structure and you risk reducing the value of the data that it contains to peanuts; monkey with the actual data payload and you likewise either corrupt the file so that it can’t be read or so that it represents a different or a degraded image. Repeat this experiment with just about any multimedia file type and you will get the same result – either a corrupt file that cannot be read correctly or one that is no longer an accurate representation of the original object. These data are not only structured; the nature of that structure is critical to their correct interpretation.

And of course it’s not just the “wrapper” that has structure; the structure of the data itself is critical. Most people would interpret the statement “Dave didn’t marry Sue because she was rich” as meaning that Dave and Sue were married, but that Dave’s motivation for their union was not financial. Conversely, the statement that “Dave didn’t marry Sue, because she was rich” would probably be interpreted as meaning that Dave and Sue did not marry and that is was the difference in their circumstances that got in the way. A single structural element - one comma - makes a big difference to our interpretation of the “same” data. Suppose that during their courtship Dave tells Sue “I love you”; the structure of this sentence is identical to the structure of the sentence “I want you” (subject-verb-object, I think, but if I am mistaken and there are any linguists out there reading this, please feel free to correct me), but the two statements may or may not be synonymous (although I hear that Dave is a good guy, so perhaps we should give him the benefit of the doubt).

In fact, even apparently random noise can convey meaning. Tune a radio telescope to the microwave range of the electromagnetic spectrum and you will hear a faint hum, directionally uniform to 1 part in 500. This is quite literally a distant reverberation of the “Big Bang” in which the Universe was created and which confirms that the Universe was indeed once hot-and-dense, as the Big Bang theory demands that it must have been. That’s important information, as historically there have been other theories of the origin of the Universe that don’t assume an explosive beginning.

From measurements of the cosmic microwave background radiation, as it is called, physicists and astronomers are able either to infer or to calculate directly many other essential truths about the Universe, including the speed at which our galaxy is moving (600 kilometres-per-second towards the constellation of Leo, in case this answer is one day all that stands between you and the “who wants to be a millionaire?” prize money). It turns out that there is an awful lot of important information encoded in that apparently random noise.

Back on Earth, less exotic, “new” types of data are increasingly interesting to the commercial and government organizations that most of us serve. We should probably call these “multimedia data”, “non-record based data” or “non-relational” data. Actually, I’m not crazy about “non-relational” either; whilst this data is typically not relational in the accepted sense – the ordering of the bytes that define the bitmap in a GIF file is important, for example - this data can, after all, be accommodated in tables in a relational database using BLOB and CLOB objects. So long as we regard these objects themselves as atomic, it seems to me these data are as relational as any other attribute of an entity. Things clearly get more complex if we want to examine or “query” the objects themselves (“select all of the pictures in which the sky is red”), but let’s not go there for now.

My recent travelling companion and the main attraction on the “CTO Road Show” that we took on tour across the EMEA region in June – Teradata CTO Stephen Brobst – refers to “non-traditional data types” versus “record-based” or “square” data. These are definitions that I can live with. And I’m sure that engineering PhD Stephen will sleep easier for knowing that the flunky from marketing considers his use of technical vocabulary to be correct and not in the least aggravating!

Martin Willcox

But Why?

Patrick Hempton — 8/17/2009

My three year old has been challenging me with the “But Why?” Question for the last few days. As a parent I want to encourage learning, so I have been doing my best to answer the tirade of questions. What has been most interesting is that I have to explain any given statement between 4 and 10 “but whys”, which has forced me to think without assumptions.

In the working world we have many problems that require solutions, if you take the “but why” approach to understanding the problem, you will annoy your co-workers, but interestingly you will reach one of two places: 1. Assumptions that are fundamentally floored or 2. An understanding of the real problem. Either way, you are far closer to understanding the problem and consequently the solution.
Let’s see how it works:

We need to get into social networking. But Why?
To get closer to our customers and future customers. But Why?
So that they understand how our products and services can be used. But Why?
So that there lives are enhanced and we stay in business. But Why?
Because providing value to our customer’s is the corner stone of business. But Why?
Because it follows X.

OK, so I need more patience. Interestingly, why we need to get involved with social networking has been clarified. Getting closer and providing information are a part of the journey but not the destination, so now I have the opportunity to refocus on the destination.
If you have an alternative to “Because I said so” or “Because it follows X” I would be ever grateful to hear from you.

Daniel Tehan
www.linkedin.com/in/danieltehan

Winding_Road_Paris_Scrnshot

David Placke — 7/20/2009

My Winding Road to Paris…living vicariously through Twitter

Patrick Hempton — 8/17/2009

I am an addict-- a Tour de France addict. I have been since I started racing road bicycles more than 10-years ago and fell in love with the sound of 200 wheels speeding in unison. I think no sport is more powerful, chivalrous and sophisticated. (And, to accomplish that while wearing neon lycra is no small feat.)

Funny thing is, in the 12+ years I’ve spent obsessing over Tour results, I’ve seen, all-in, maybe 20 hours of imagery of men riding bikes—and 90% of that time was spent on the Champ Elysee or at a Stage of the REAL event. Do the math and you’ll conclude that I’ve spent next to no time watching the coverage on television. It’s a shame.

This year, I am “watching“ the Tour on Twitter. Lance Armstrong rides with one hand, and Tweets with the other. Not just words—

pictures and video, too. I was actually angry one evening when I saw late-night posts of Lance goofing off with Robin Williams. I mean, Lance has a bike race to win! Focus.

Point is, I’m not alone. It’s no surprise that how we consume news, sports and professional content has dramatically changed. My ability to stream, download and control how and how much content I watch has exploded. And, my ability to supplement that activity with web-site publications, blogs and real-time commentary and feedback from the content creators or participants heralds the start of a massive shift in how users will expect to consume media-- and, how media companies will need to target those consumers.

Imagine the power for a brand-holder who can create a comprehensive picture of my brand-interest across every channel? For example: Colleen is a bike junkie. She Tweets the Tour, spends time on the Tour De France web-site, selects articles on the subject from the NY Times, spends MUCH TOO MUCH of her income online buying bike-related merchandise.

Voila. Here’s what I’ve allowed you to do—rather, here is what my comprehensive data stream (which can be captured and analyzed) has allowed you to do. You now know me. And, if you’re a brand-person, that is gold. You can narrowly target my interests. You can target content, advertising and offers when I’m on your site. You can make your brand meaningful to me by sending me the right kind of messaging.

If you’re a publisher, you serve me an ad I’m likely to click-on—and you command a premium from your advertiser to do so.

If you’re an advertiser, you create a view of your consumers’ behavior across every channel.

Teradata brings this value home to our customers all day long. While mining Web 2.0 and understanding how social media impacts business may be new for some, it’s already common business practice for Teradata customers. The Teradata Voice of the Customer Analytics Solution allows customers to mine web-blogs, Twitter posts and other external Web 2.0 sources (as well as internal customer communications) for trends, reach and impact. Built on top of Teradata’s proven data warehousing platform, and in combination with our suite of analytics and business intelligence tools, our solution allows brand-holders to gain deep insight into how their consumers are engaging, and how they can convert that interest to revenue.

It’s powerful insight… maybe even more powerful than the Tweet on my Tweet-deck from an arrogant Texan riding his bike through France. Livestrong!

Colleen Quinn

Managing Master Data across SAP BW and non-SAP EDW Environments

Patrick Hempton — 8/17/2009

One of the most difficult challenges in operational and data warehousing environments is the management of master data (or reference data) across multiple systems.

One of the key benefits of Enterprise Data Warehousing (EDW) is the ability to “conform” data across multiple source systems to provide one view of disparate data.

Two key factors in achieving "conformed" master data in the EDW are:

Identifying and defining Business Owners and Custodians to make decisions on master data across all systems.
Logical and physical database environments, tools and techniques which assist in the management and cleansing of master data in the EDW environment.

Consistent master data and, therefore, the ability to perform cross-functional analysis, becomes very difficult to achieve when multiple, independent, data marts are built, where master data is sourced from different systems, and where there has been no thought or effort expended to cleanse and conform the master data across the multiple data marts.

Enabling all SAP and non-SAP data to be loaded into ONE logical and physical EDW forces the master data management issue to be imperative. Decisions must be made on master data conflicts before the data is physically loaded into the one environment, whereas, if they were physically different environments the issue is most times overlooked.

SAP and non-SAP data can now reside in a cohesive, single Teradata EDW which provides, for the first time, the ability to truly manage master data across all source systems. Imagine the possibilities this brings to business users – a truly integrated view of the entire organisation's data.

Share you thoughts on MDM-related issues and challenges your organisation.

Nina Corney
Teradata SAP Domain Lead
http://www.linkedin.com/pub/nina-corney/2/44a/802

End of the Beginning : Evolution of Interactive Marketing

Patrick Hempton — 8/17/2009

It seems appropriate that the best place to start this blog about marketing and data is at the The End of the Begining. As Winston Churchill said, “this is not the end, it is not even the beginning of the end but perhaps it is the end of the beginning”. Another reference might be the Restaurant at the end of the Universe – either place seems like the right place to start.

There have been a lot of starts and stops in data driven marketing over the past decade - so why is this time different?

The biggest change for marketers is the freeing of data driven marketing from the customer-centric perspective that has been its core for 20 years. Historically, data from marketing was primarily at the customer level – CRM, Contact management, Campaign management, householding, usage analysis etc all were driven by customer activities. CRM Technologies such as Teradata Relationship Manager have delivered tremendous results for companies, but investment in large data systems required a customer-centric business approach. Brand centric, advertising centric and product centric companies didn’t genetrate the data to justify the marketing and IT investment.

More & More Marketing Data is Available and Relevant

Search is probably the leading indictor in this direction. Search is not a customer centric marketing activity – yet it has a directly measureable ROI, data analytics can drive huge business improvements. Optimine Software (a teradata partner) has helped companies double their Search ROI and reduce search costs by as much as 15%. For large search advertisers that will be millions of dollars a year.

Search creates data at a pace even faster than customer marketing. A million placements might create as much as a terabyte of data. But bids on keywords can change within minutes and optimizing placement value means continuously analyzing results – not just clicks, but throughout a visit. , Adding click-stream data to search results creates an information rich data stream that can reduce costs, improve conversions and better understand advertising effectiveness.

But here at the end of the beginning nothing in marketing works in a silo. Forrester recently released researched that showed that Display Advertising drove Search activity. About as many customers did a search because of a display ad as those that clicked on an ad. That means that display and search results are connected – which is intuitive, but not the way most companies look at their marketing spend.

Back in the old days, we had very crude tools to look at advertising effectiveness. But we knew what worked. When I work in telecom, we had a product called 1-800-Collect. We could drive more revenue just by putting up more ads – and there was a direct relationship to advertising and that months revenue.

Now we can connect the data from across channels to see previously undectable relationships. For example using temporal data analysis, we can match web site traffic to traditional tv advertising. Does traffic rise when ads hit – yes. One extreme example was a customer’s site that almost crashed immediately after a superbowl ad.

But an ad isn’t an event anymore. It is a piece of long tail content that is owned by the market. When an ad or piece of content goes viral on Facebook, twitter or other social media sites, the impact of the ad continues to live. Social media sites have pushed the concept of customer control to a new level. Every customer experience is ripe for social media analysis.

Suddenly data is about Connectedness of people, connectedness of content and connectedness of marketing. Data is what brings them all together. The days of the database marketing being relegated to direct customer marketing are over. One new Teradata offering – Integrated Web Intelligence starts connecting the enterprise with the internet. One insight on this was developed by Teradata’s Advanced Analytcis team in europe and covered in Teradata Magazine.

Next time – new marketing data and how it fits together. In the meantime – what else is driving change in your marketing spend and approach? Are you looking at linking online and offline data together?

Paul Barrett

ghw3dz5yjf

What Should Keep You Up At Night?

Darryl McDonald — 7/30/2009

In tough times and not so tough and in good times and bad, corporate executives spend sleepless nights for all sorts of reasons. Maybe it’s opening in a new market, maybe it’s declining sales or maybe it’s staffing issues. Problems like these are the ones executives know about, and presumably are already working to correct.

But what about the problems senior management doesn’t know about? The ones that creep up, like an eroding customer base, or hidden costs canceling out the profit margin? Or any of a dozen other ominous problems that could jeopardize the company’s survival?

What you don’t know will hurt you. It’s not only a matter of asking the right questions. Rather, it’s a matter of looking at the complete pool of data and zeroing in on the right answer.

Companies using an enterprise data warehouse to deliver an integrated view of their business activities already have at their fingertips all the data across applications and divisions. They can spot a threatening ailment and take action. Consider a big North American retailer that was able to detect and stop a crime ring from bilking the stores through fraudulent merchandise returns. By using near real-time verification and validation of customer receipts, this company reduced returns fraud by $2.3 million in the first year and realized 100% ROI within five months. This retailer and other customers have that enterprise data advantage of being able to spot hidden problems before they become bigger problems.

When information is in one place and fully integrated, senior executives have the tools at their disposal to identify and address company-threatening hidden problems. That translates into better leadership and fewer things to keep an executive up at night when there’s more important things to do, like get some rest so you can strategize on how to leapfrog competitors in the next market upturn.

2thfmq37wy

Consolidate SAP BW instances and non-SAP Data marts onto one Teradata Enterprise Data Warehouse

Patrick Hempton — 8/17/2009

Since the announcement in April 2009 of SAP and Teradata Extended partnership there has been a great deal of interest from customers to understand the value and benefits this will bring to their business and technical operations.

SAP and Teradata have commenced discussions with their joint customers to outline how SAP BW will run on Teradata Database Management System from release of SAP Netweaver 7.2 in early 2010.

SAP BW will now run on Teradata!

For the first time in data warehousing history, the combined strengths of SAP BusinessObjects’ diverse range of Data Integration and Business Intelligence tools, SAP BW’s rich business content, together with extreme performance, scalability and active data warehousing from Teradata provide a true Enterprise Data Warehouse solution.

One logical and physical Enterprise Data Warehouse for Active, Real-time, Operational and Strategic Reporting and Analytical needs.

The journey can commence right now with non-SAP data being sourced into Teradata Enterprise Data Warehouse, and SAP BW cubes migrated and/or commenced to be built on Teradata early in 2010.

Are you ready to reduce your data warehousing TCO and provide a quantum leap in business benefits to your end-users?

Are you experiencing any issues with trying to analyse SAP and non-SAP data?

Nina Corney

Teradata SAP Domain Lead
http://www.linkedin.com/pub/nina-corney/2/44a/802

Clouds, virtualisation, and the data warehouse - Part 3: 1995 and all that

Patrick Hempton — 8/14/2009

Probably everyone except my Mum has heard enough from me on this subject by now in parts 1 and 2 of this series, but she says that reading this really helps fill in the time she is spending in the dentist’s waiting room between root canal treatments at the moment, so here goes nothing…

Where the Cloud makes most sense right now is of course in the use of products like VMWare, Xen, or Microsoft Hyper-V to balance workloads across many poorly utilised servers. These tools allow server consolidation and enable huge savings in hardware, power, and cooling costs. This technology is a subject dear to my heart, as it also enables my (non-standard and in violation of corporate policy) MacBook to pass for a (standard and compliant) Windows-based PC when connected to the Teradata network (it’s a good job that no one is reading this, or I might come to regret saying that out loud). Given that the majority of blade servers run at 20-30% utilisation, it is no wonder that organisations can’t deploy these technologies fast enough for the localized, transactional workloads where they make most sense. But for database vendors to claim they are “virtual” or “in the cloud” because of the good work of these tools is much more about opportunistic marketing than it is about genuine innovation in information management.

Teradata has been riding the virtualization train ever since uni-processor hardware nodes were replaced by virtualized parallel processing elements (these are the “AMPs” in Teradata terminology and can be thought of virtualized, co-operating database kernels). You would expect me to say that, of course, but it happens to be true; keen students of technology history (possibly not including my Mum) will recall that this re-architecting of the Teradata RDBMS was launched on an unsuspecting world with version 2 of the product, way back in 1995 and long before the current fad for “grid computing”.

Re-architecting Teradata in this way was an inspired decision (I can’t take any credit, I was disappointing my Mum by spending too much time partying and too little time studying at University) that brilliantly anticipated modern multi-threaded and multi-core hardware and regularly enables our systems to run at 80-90% CPU utilisation. Indeed, we have several clients who run their machines at 98-99% CPU utilisation all day long. What many would call an under configured machine, they call “getting their money’s worth”. Prospective new customers regularly question whether we are “low-balling” completive bids because we often recommend systems with so few processors, compared with the competition. But “traditional” decision-support workloads are I/O bound; and the newer, “active” workloads typically consume only very small amounts of CPU resource in a Teradata environment because our unique filesystem means that the distribution index also permits direct access to the data. There are no prizes for throwing processors at data warehousing workloads, only for them to sit idle because the I/O sub-system cannot serve up data fast enough to keep them busy. Not unless you need to heat your data centre. And most of you tell us that you have the opposite problem.

This virtualisation is also the basis of our patented workload migration approach, which permits a Teradata system to survive the failure of a hardware node and that continues to give Teradata an enduring advantage over competitor platforms where single system availability is concerned. And it enables “co-existence”, which enables Teradata customers to extend the useful lives of their Teradata systems by adding additional processing power in the form of new-generation nodes without forcing them to “floor-sweep” the entire system. I could go on. And on. And on. But I sense that even my Mum is losing the will to live now and so I’ll stop.

Teradata Virtual Services (TVS) – enabled by the Teradata 13 release of the database - is our next foray into virtualization and puts us at the forefront of advances in storage management. It means that storage in a Teradata system can now be pooled (read, virtualized) at the “clique”, rather than the node level. In plain English, this means that Teradata can now support multiple drive sizes in a single system, enabling organizations to exploit the new, higher-density, cheaper-per-terabyte disk drives to store "cool" data alongside “hot” data, whilst preserving lower-density disk drive performance levels for the “hot” data. (Teradata uses “temperature” as a metaphor for how frequently data are accessed, so the “hot” data are both frequently accessed and are typically subject to more aggressive service levels than are the “cool” data.) This technology promises to enable organizations to react to regulatory and competitive pressures by allowing them to store more valuable data for longer, or to achieve significant reductions in TCO, at the expense of reduced scan performance for the “cool” data.

Fans of certain other database management systems may not exactly be blown away by the ability to mix different disk sizes in a single system. However, TVS is transparent and enables the Teradata RDBMS to continue to spread data evenly over the available storage for best-possible parallel performance, all without manual intervention and data placement. And TVS has another trick up its sleeve, as it automatically and transparently detects data “temperature” and moves “hot” data to the outer cylinders of the disk, where I/O rates are highest. Those same fans of competing products should note that their DBAs have neither the time nor the tools that would be required to constantly re-balance data between the faster and slower cylinders, so that this temperature-based data optimisation simply isn’t possible with those other products. Automating the DBA task is what makes TVS a disruptive innovation.

This automated data migration is a clever trick and in the near future it will enable a Teradata system to support different types of storage in the same system – solid state storage for the “hottest” or most frequently accessed data, fibre-channel drives like those used today for “hot” and “warm” data and slow-as-mud 1TB-and-up SATA drives for data that still has value to the organization, but that will be accessed only infrequently, by fewer users and with relaxed query performance service levels (solid state storage is not subject to the same performance limitations as are electromechanical storage devices and are truly “intelligent storage” devices). Data placement and migration – across fast and small, large and slow, silicon and disk - will continue to be automatic and transparent in this scenario, as well.

Go on, admit it; now you’re impressed.

Martin Willcox

fdh9vc5i2t

Beyond Campaigns

Patrick Hempton — 8/17/2009

It appears as if campaigns as we know them today are been challenged on all fronts, there are less customers willing to take up offers and the marketing budget has been slashed. So it leaves me wondering how marketing organisations are going to make good on the commitments made in plans.

There is an emerging belief that social media tools are the answer, but a new medium of communicating is not a silver bullet. With any new communication medium, organisations need to understand the capabilities and align those capabilities with the marketing vision.

The challenge that is becoming more evident is what works in traditional mediums is not working in the social media mediums. “Sell more stuff” is not a strategy that resonates in a medium that relies on recommendations between people that are not controlled by the organization trying to sell. Organisations that will succeed in the new medium are going to have to find a way to help customers achieve their goals rather than focus on achieving the organisations goals, such a change in focus will require innovations in the marketing department not seen since event based marketing.

So the challenge for marketing is juggling what they know (traditional marketing) with what they are yet to learn (social media marketing), some big risks need to be taken and some big failures are going to happen, but for those that find the right mix a differentiation will be realised.

How are you going to proceed into the new medium?

Daniel Tehan
www.linkedin.com/in/danieltehan

Freescale Semiconductor Wins Best Practices Award with Help from Teradata

Darryl McDonald — 7/30/2009

At Teradata we look at business data in an original way. While other business intelligence players look at data marts as the way to deploy information, we take a more centralized and integrated view. That’s what makes us different. It’s also what makes our customers different. Different as in distinguished, as in recognized leaders in their industries.

As in Freescale Semiconductor, which just won Data Warehousing Institute’s prestigious Best Practices Award in the Enterprise Business Intelligence category. When Freescale came to us, they said they wanted an integrated information environment across the enterprise built on well-defined, timely data, with easy-to-use front-end tools to make workers more effective in their jobs. We said, no problem.

A first for the semiconductor industry, Freescale’s joint business and IT program now combines factory, engineering and business data in one data warehouse, enabling cross-domain analytics. But the program has had business benefits beyond the IT department. Freescale launched successful reengineering initiatives in its supply chain, manufacturing, customer service, quality and finance divisions and also shored up underlying weaknesses in its business intelligence infrastructure.

Results soon followed. Freescale has:

Increased the percent of salable chips produced in each manufacturing run, contributing millions of dollars to the gross margin
Reduced response to customer quality incidents from an average of 25 days to 14 days
Cut revenue reporting to 10 minutes, down from a full day

The Freescale Enterprise Business Intelligence program is now the information and analytics backbone of virtually all the company’s business applications. Some 92 percent of employees use it weekly and the demand for it continues to grow.

We congratulate Freescale for its award, which recognizes its innovative business intelligence and data warehousing program.

Clouds, virtualisation, and the data warehouse - Part 2: Canute and the Wellington Boot

Patrick Hempton — 8/14/2009

My last post on this subject could be perceived as aligning me with the conservatives in the Great Cloud As The Data Warehouse debate and might have led you to conclude that I am, Canute-like, engaged in a desperate and futile struggle to hold back unstoppable forces of nature. (King Canute, for those of you unfamiliar with English and Danish history, sat his throne upon the beach and commanded the rising tide to retreat. Or as I learnt at school: When the waters came up to King Canute / He wished they’d invented the Wellington Boot!)

Not a bit of it. I happily accept that cloud and virtualization technologies are already changing the industry in fundamental ways and that the consequences will be far-reaching. I just don’t accept that the cloud will change everything.

Those of you who are familiar with the Gartner hype cycle framework will recognise a familiar game being played out right now where cloud and virtualization technologies are concerned. These technologies are currently screaming up the "peak of inflated expectations" at a dizzying rate - with vendors rushing to put "cloud", "virtual", "service" and "grid" in front of the names of their established offerings – and nothing is surer than the fact that they will come crashing down into the "trough of disillusionment" sooner-rather-than-later. When the dust settles, these technologies will then move up the "plateau of productivity" (please don't blame me for the names, it’s Gartner's 'ology) and become an established part of the technology landscape. Typically, not all of the inflated promise of new technologies is realized. Witness XML; invented to address the data interchange problem, some of its more wild-eyed advocates were prophesising a few years ago that XML data stores would replace relational databases. Whilst XML and related technologies are now "mature" and are clearly valuable, no one is claiming that any more. The cloud is coming, but beware the more zealous prophets.

So let's leave the Inmon / Kobielus spat - the subject of part one of this post several weeks ago - behind for a moment and look instead at trends and consequences that are already in the line of sight and about which there can be little debate concerning technical feasibility.

Cloud database-as-a-service (DaaS) offerings will make it even easier and even cheaper for departmental managers to build their own low-volume data marts. The proliferation of information that this implies will likely have a negative impact for information management in many organizations, but it will also create opportunities for organizations to experiment with new technologies and new data sources. And create new opportunities for database vendors to position themselves as DaaS providers, of course!

Conversely, some organizations are already starting to position their existing data warehouses as "private clouds" to internal, departmental customers. The deal that IT offers the business goes something like this: you can continue to build data marts – in fact, we'll even provide the infrastructure that you need for free - but you do so on the "private cloud" (in reality, an unused corner of the data warehouse) and where some or all of the data that you need already exists on the data warehouse, err, we mean the private cloud, you re-use the existing data instead of re-loading it.

This is the ultimate "win-win" deal. Ambitious departmental managers get to build new analytical systems on their terms and their schedule, not those of the IT department - and get a leg-up in doing so because some or even all of the data that they need is already available. IT gets to control the proliferation of redundant data that would otherwise result and to minimise the organization's investment in technology infrastructure. Failed projects "fail fast" – and free-up technology resources that go back into the pool to be exploited by the next smart business user with a smart idea, instead of depreciating unused in a corner of the data centre. And IT has visibility of those projects that do succeed, so that these become much easier to bring into full, supported production operation in due course than would otherwise have been the case.

As smart as this approach undoubtedly is, it is ultimately more about pre-emptive data mart consolidation than it is about a new technology paradigm (gah! I said the "p-word" out loud!); whilst it is true that sophisticated mixed-workload management is critical to this endeavour, the rest is more about good management and flexible governance than anything else.

Some analytical applications and BI tools may move to the (public) cloud, following the Software-as-a-Service (SaaS) model. I can't see a future in which complex, federated queries run against large data-sets spread across distributed databases connected only by the Internet, for the reasons that I outlined last time. But cloud-based tools and applications that exploit "push-down optimisation" strategies, so that the I/O intensive heavy lifting takes place on a parallel database and just the result-set is shipped back to the user via the cloud are certainly technically feasible. But how much additional value will cloud-based analytical applications bring to organizations if they anyway have to go the trouble-and-expense of building their own analytical databases?

Many analytical applications and BI tools are likely to be extended by the public cloud, however. BI mash-ups - or "bash-ups" as they have been christened by the wags - are already being deployed and this trend will surely continue and accelerate. A "mash-up" consists of one data source overlaid on another, (generally web-) data source. If you have used an iPhone App that overlays geo-spatial data – for example, the location of a recommended restaurant – on a Google map, then you are familiar with the concept (and if you're pushing 40 like me, now you know what it is called).

Mash-ups and bash-ups are a great example of federation (purists might argue that point, incidentally, but I left my ivory tower without my PhD and for our purposes I think we can call it this) about which I had lots to say last time around. Where data access is localized, data volumes are small and interfaces are simple and well-defined, this sort of federation can work well. There is a world of difference, however, between passing the latitude and longitude of a small, discrete set of locations to a web-based mapping application and in trying to calculate the average distance between the home address of 10M consumers and their nearest store, so that we can then join the sub-set of the consumers for whom this distance is more than one standard deviation from the mean to the transaction table to establish if their buying patterns are different from the rest of the population.

All of which is why Jim Kobielus and Bill Inmon are both right: Jim because federation is a valuable technology which has its place; and Bill because that place does not involve replacing the complex, I/O intensive processing that takes place in the modern data warehouse, but rather in supplementing it.

Believe it or not, we're not done with this topic yet. In part 3, we’ll go back to the future - and you'll get introduced to my Mum.

Martin Willcox

Das Auto

Patrick Hempton — 8/14/2009

In one of the presentations that my travelling companion and the main attraction on this tour - Teradata's CTO Stephen Brobst - delivers, he makes the point that he has been driving for 30 years and has yet to have an accident.

Don't get the impression, incidentally, that Stephen is a boastful man. He tells this story to make an important point about how active data warehousing can be used to accurately assess and price motor insurance risk - and even to incentivise younger drivers to avoid situations in which they are disproportionately likely to have an accident.

I've been driving for half that time and, ahem, I have had one or two (dozen) shunts - all of them someone else's fault, naturally, as I'm a great driver. But then as Stephen also explains in the same presentation, 95% of men believe that their driving skills are above average. Which is clearly mathematically pretty unlikely - and only possible at all if we make some rather sweeping assertions about the driving skills of the other 51% of the population. That has the potential to get us chaps into a lot of trouble, so we should park this particular discussion straightaway.

These days I'm more careful. I'd like to tell you that age has mellowed me, but the truth is that I am just unbelievably precious about the blood-red paintwork on the BMW that is my pride-and-joy and desperately want to avoid it becoming a dented rust-bucket, like most of the rest of my cars before it. The chances, however, are better than even that the BMW will end up dented like all the rest; if you study the data, as Stephen has, then you know that red cars are involved in more accidents than average.

German cars are very popular in the UK. One comedian even jokes that you can tell a true Englishman; he's the guy that drives to the supermarket in a German car to buy a case of German lager.

For the Germans, though, cars are no laughing matter. With as many as one in seven German jobs depending directly or indirectly on the automotive industry, many equate the condition of the German economy with the health of their carmakers – presently questionable, as recent news and stock prices indicate. Some employees have been working short time since January and the "scrappage" („Abwrackprämie“) scheme that the Federal Government has been running seems to be encouraging Germans to buy cheaper, smaller foreign cars, rather than the larger, premium models that the major German manufacturers are best known for – and have historically derived their biggest profits from. At the other end of the market and just outside Frankfurt - location for the German stop of our tour of the region - former General Motors subsidiary, Opel, is at last about to be sold to a new investor and has thus narrowly escaped bankruptcy, for the moment at least.

What next? It seems inevitable that globally the car industry will have to cut costs and capacity. Increased efficiency in the supply chain is one obvious way for any manufacturer to cut costs, whilst simultaneously improving agility. In recent decades, German carmakers have expanded their production networks across Europe and beyond, making their supply chains more and more complicated and harder to manage; that makes improved supply chain management (SCM) an increasingly attractive option.

Active SCM
Any supply chain involves networked decisions at many levels. How many cars are we going to sell next month? And which types? What is the current inventory? How many cars do we need to build to meet demand? On what schedule and which materials are required? Where do those materials need to be delivered? Who can provide those materials and components, to the required quality, at the best price and just-in-time? And should I hold the lorry taking parts to the assembly line to wait for components that are late arriving from a supplier (and risk halting the entire line), or send it only partly-loaded (and risk not being able to finish the high-value models whose completion is waiting on those parts)? Etc., etc., etc.

Teradata and the European Business School (EBS), a university near Frankfurt, are running a joint project that explores the precise information needs of German automotive supply chains. Whilst the research is still on-going, we can see already that there is huge potential for the application of active data warehousing to the industry. In one sense, this comes as no surprise; Ford Motor Company, one of Teradata's long-time customers in the US, has been using its Teradata enterprise data warehouse to support just such decision-making. Once you have a clear, complete and consistent view of stock and demand at the various stages of the supply chain, you can drive process efficiencies; and fast analyses allow for better decisions at short notice, so that we can make the smartest possible decision about that partly-loaded truck.

Chips under the hood
There are three other areas in which Teradata automotive customers are seeing impressive results, both in terms of hard cash saved and in customer loyalty earned.

The first of these is Early Warning Systems. For the manufacturers, warranty claims are expensive and, historically at least, failure rates have also been hard to predict. That uncertainty has made it difficult for manufacturers to budget for the impact of these claims and difficult for them to understand when parts are failing more often than they should and there is an underlying problem that needs to be addressed.

With enough detailed data in a Teradata database and enough clever maths supplied by our Advanced Analytics group, it turns out that expected failure rates can be accurately predicted – and then compared with actual failure rates. In this way, components that are either defective through design or through poor manufacture can be identified, proactively replaced where they have already been installed, and replaced with parts built to alternative designs or sourced from alternate suppliers where they have not.

This matters because customers generally prefer a polite call and a scheduled appointment at the garage to being stranded by the side of the motorway through an entirely avoidable malfunction. And I should know, as the memory of the pain-filled 24 hours that followed the death of our family car en route to our summer holiday last year is with me still (tip: if you must break down in France, avoid doing so over the course of a holiday weekend). I doubt very much that I will ever buy another car from the manufacturer – not BMW, I should say straightaway - that torpedoed our summer holiday because they apparently can't find a reliable method of attaching a radiator hose to a radiator.

If you can include garage reports – verbatim text, the notes made by the mechanics - in the database and mine this information alongside the record-based history data, then you can gain an insight into how vehicles are actually maintained in the field, rather than how the manual says they should be. That, too, is a goldmine of information that has not typically been available to vehicle designers seeking to ensure that new vehicles can go even longer between services.

Lastly, back to those reckless young – and young-at-heart – drivers. If you measure your virility by how quickly your car can pull away from the lights, it helps to have a few extra horsepower under the hood. And now that engines, like just about everything else, are typically controlled by microprocessors, all sorts of people are able to equip themselves with the relatively inexpensive kit needed to persuade the on-board engine management system to deliver the goods.

This "chipping" might look like good, clean fun for safe young drivers like Stephen was, but it places stresses on the rest of the car's systems that are greater than they were designed to handle, causing them to fail more often - and the manufacturer to incur increased, fraudulent warranty costs. Manufacturer know that this fraud and others like it cost them millions of Euros, but only those companies that have brought together the data generated by the microprocessors in the cars themselves and the product, warranty and service data in an active data warehouse are able to identity this fraud and decline these claims.

A few million Euros should bring a smile to the face of even the most jaded of German industrialists right now. Vorsprung durch Teradata, anyone?

Martin Willcox

Intelligent organisations are already reacting, are you?

Patrick Hempton — 8/17/2009

We find ourselves in a new economy; the global financial crisis has impacted organizations and customers alike. Marketing departments that successfully turned out thousands of customer leads for the channels to execute up are now being asked to reduce costs and get more focused. Customers no longer have discretionary capital, are nervous about spending and have become far more prudent with all financial outgoings.

This leaves the database marketing world with the dilemma of doing more with less, while still delivering to targets. Traditional entrepreneurial approach using intuitive understanding of the business needs to be supported by data analysis more than ever.
It is time to work smarter, extract more value out of the knowledge that an organisation has about the customers before delivering the offer. Suppositions need to analytically critiqued, campaign execution needs to be tested and revalidation needs to be integrated into the ongoing operations. Analytic models need to be recalibrated with weightings on recent data, Campaign ROI models need to be revisited to see if they still stand up in the current environment and the marketing manager needs to determine where investment should be made and cut.

It's a new world with new behaviour, intelligent organisations are already reacting, are you?

Daniel Tehan
www.linkedin.com/in/danieltehan

Leveraging your Business Intelligence to Drive Revenues and Profits

Rick Loconto — 7/17/2009

Recently there have been a deluge of articles and speeches about how to leverage BI and your information assets to maintain/increase revenues and also growth the company now or in the neat future. As the economic downturn has led us all to be much more conservative, there seems to be a quadrant of leadership organizations that are investing in consolidations and re-architecting of their analytical environments. This group of companies, many of which I have visited with, heard on webinars, or witnessed through various other communications channels are defining methods of cost savings while also increasing opportunity.

There seems to be three major themes that are emerging in this effort to reduce cost and increase revenue opportunities:

1) Focus on the elimination of redundancy of data files, work actions, processes, approvals, and also direct cost expenditures that provide no value to the company. Throughout the years many firms have been building separated data bases, data warehouses, and analytical files which cost large sums of financial resources (in comparison to the value achieved) from redundant or replicated infrastructure or work actions. Examine your uses (or non-uses) of data and databases. You will find gigantic opportunities to utilize known valuable data and also to merge them into an integrated data warehouse that fortunately is much more manageable and much lower cost than in distributed environments.

2) Seek ways to interact and engage customers beyond the normal actions or channels and coordinate these efforts. Synchronization of current and useable information, discovery of unknown facts and issues, and alignment with customer management processes will provide a surge in revenues and resells which will ignite the profitability opportunities. There is no revenue without customer(s) and no profit without creating a wedge between expenses and revenues.

Profitable companies know that analytics provide much more than simplified or even complex graphical reporting. Reporting is looking backwards and not utilizing data to provide management with decision-making capabilities. Find ways to ignite management's thinking by including risk and returns in the reporting or analytic forecasting. This method of using Business Intelligence (BI) provide alternatives for making the right decisions and then re-evaluating the actions taken (and deciding to set new or updated rules-based equations into your BI or decision processes).

3) Achieve higher levels of management support for making smart decisions inside your company and new methods of utilizing customer or supply data to drive down costs and drive up revenues. Plan to automate customer marketing in a high percentage of offers and known repetitive processes. Discontinue management reports that are not reviewed or not even requested anymore. Maybe a quarter review of the most expensive and the least used reports and models should be accomplished with the business users. Using BI should be a Critical Success Factor in all processes. Align the reporting, forecasting and operational BI communications into the decision actions of management. Gain support of CXO's by directly aiming your deliverables towards THEIR KPI's or Metrics. Understand THEIR goals and success criteria and utilize analytics and the Data Warehouse to drive information to support them.

There are hundreds of ways to provide information to people to make decisions, take actions, evaluate results, redefine objectives or goals, and to improve their business processes. Plotting out a short and longer term roadmap or deliverables plan for inclusion of advanced BI techniques, dashboards with actionable data, and alternatives to make decisions upon…will drive success for you and the entire company. This is an optimum time and a real opportunity for your firm to 'raise it's intelligence'. Don't wait for some economic resurgence to overwhelm you with requests. Perform the right actions now, not when a deluge of people will need your support and you will struggle to provide it. Being ahead of the game is half the battle. The other half is using your already satisfied customers to communicate your value to the organization.

2gdyfr9cb8

Privacy and Dirty Laundry

Patrick Hempton — 8/14/2009

Monday finds me in Zurich and Bern with Teradata's CTO, Stephen Brobst, on the penultimate leg of our European tour.

The previous leg of the tour involved our spending ten continuous days on the road – eleven, if you count the unscheduled overnight stop that I was forced to make in Frankfurt on my way back home from Vienna on Friday. So you can imagine my dismay, upon my eventual return with an over-stuffed carry-on bag full of dirty laundry, at discovering that our family washing machine had died in my absence.

The Swiss banks, too, are often thought to have their fair share of grubby secrets and have been making headlines in the international press lately, as accusations fly that their famous bank secrecy laws have led them to take their responsibilities in respect of the detection of tax evasion, money laundering, and other related ills rather too lightly.

Probably the Swiss are a rather soft target for other European governments seeking to distract attention that would otherwise be directed at embarrassing failures in bank regulation that lie rather closer to home. In fact, the question of how banks and other financial institutions should reconcile legitimate economic privacy with the international fight against serous crime is a genuinely difficult one, with some parallels to the issues surrounding the capture and retention of telecommunications data for law enforcement purposes that I explored recently in a previous post.

The morals and ethics of all of this are difficult and political, so let's look instead at the data, which are less ambiguous.

In 1885, George Rae wrote in "The Country Banker" –

"In the case of people who bank with you, you have in your ledgers a record which will enable you, in many cases, to check and rectify the estimate of their means and position which is current out of doors. A man’s bank account will not necessarily disclose what he is worth: but its entries, debtor and creditor, will serve as tracks to indicate with some degree of clearness the line of progress along which he is moving towards either failure or success. Your customers are unconscious diarists of a portion of their lives. Every account in your books is a record, more or less graphic, of the financial history and progress of the customer, contributed by himself."

There are many interesting observations in these few sentences, but for me at least, the most interesting is that even in a time when a British country banker could have been expected to know his important customers personally and to be intimately acquainted with their position in local society, Rae is clear that the real story is in the detailed data – each entry, debit and credit - in the ledger.

These days, of course, the major banks have millions - even tens, or hundred of millions - of customers and banking has become an altogether more impersonal affair for most of us. A situation that suits the fraudsters and the money launders just fine.

Money laundering in particular involves a complex web of transactions, purposely designed to obscure the link between the dirty cash and the cleansed accounts. The first issue that we face is resolving the links between these accounts and parties. Customers with multiple dormant, or semi-dormant accounts; account ownership claims (e.g.: sole accounts that are changed to joint accounts and then back to sole accounts in the second party's name); accounts with large volumes of transactions, especially from-and-to high-risk geographies and businesses (saunas, for example, are often even less legitimate than they are salubrious) – all should attract our attention. It follows, of course, that we need visibility across different business units, products and accounts to construct this integrated picture; the use of business unit-, product- or account-specific data-sets will drastically limit our ability to traverse the complex and evolving web that the fraudsters spin.

Uncharacteristic, high-value cash transactions and large credits from high-risk countries are obviously suspect; so much so that practiced fraudsters rarely resort to such unsophisticated means, preferring instead to launder money using large numbers of smaller transactions. These transactions typically look perfectly ordinary at first-sight – which is the intention, of course. If the detailed data that Rae advises us to study is available, however, we can establish, for example, that a customer is systematically depositing amounts of cash that fall just below the legally prescribed reporting line into multiple accounts.

Capturing – and retaining – the detailed data is critical to this endeavour, as we need to be able to "roll-up" these multiple transactions in different ways as we uncover new relationships between different parties and multiple accounts that we later discover are in fact controlled by a single party. Prematurely summarising the data destroys our ability to "replay history" in this way, as we discover these new linkages.

Retaining the detailed data also enables us to identify suspicious patterns of behaviour on the basis of models created from known fraudulent transactions. This is important because it can support the scoring of transactions as they are loaded to the database, so that we move from reactive, to proactive, identification of fraudulent activity.

All of this represents current best practice. Let’s think for a moment about what next-practice might look like.

"Social network analysis" is a current focus for many B2C organizations and also for many government agencies: the companies that serve us want to know what we tell our friends and relations about their products and services; and the governments whose duty it is to protect us want to know who the bad guys are talking to, in order that they can further their understanding of criminal and terrorist networks.

Less well publicised is the use of similar technology in a context that is internal to the corporation. But many frauds involve internal actors, who may either have been recruited by external criminal elements to ignore or facilitate illegal activity, or because the fraud has been conceived and executed internally by employees, who know how the internal systems operate and are uniquely well-placed to subvert them.

Electronic communication records – and email in particular – are excellent chronicles of who said what and to whom. Analysis of email traffic can reveal the existence of relationships and internal cliques that are designed to execute or conceal fraud. Linguistic analysis of email content can also unearth covert meaning in apparently innocent exchanges.

Some organizations are now starting to use these techniques, but relatively few are combining the new analyses of electronic communications data with existing, best-practice analyses of record-based, transaction histories.

This could change, as multi-temperature data warehousing techniques enable relatively infrequently accessed data to be brought into the data warehouse at reduced unit cost, alongside existing, frequently accessed or "hot" data. Linguistic analysis can give us design; network analysis can tell us who was involved; and existing, best-practice styles of analysis will tell us what was stolen or laundered, when and on whose behalf. Bring all three together in one repository and you have motive, actors and method: game, set and match.

Much has changed in the one hundred and twenty-four years since Rae wrote his seminal work, but this much, at least, has stayed the same: we are all of us "unconscious diarists" to our activities - nefarious or otherwise - and examination of detailed, integrated data will continue to bear witness to our progress, in whichever direction we have chosen.

Examination of my recent financial affairs will show that I purchased a new washing machine on Saturday. Freshly laundered, the road show continues!

Martin Willcox

Mind the gap!

Patrick Hempton — 8/14/2009

The issue that was the subject of my last post – the yawning gap between the 5 million-fold increase in processing power over the course of the last 30 years and the 5-fold increase in disk access performance - presents a challenge for the entire industry, not just for Teradata. And it is a challenge to which the various database and platform vendors have responded very differently.

Teradata's response has been characterized by a focus on making the database software smarter so that we can achieve more and better results with Intel standard high-volume server technology and standard enterprise class storage from EMC and LSI. In recent releases we have, for example: continued to improve our cost-based optimization so that the database chooses the smartest query plan available as often as possible, reducing the system resources that each query consumes; introduced new index types (for example, Join Indexes and Aggregate Join Indexes) that allow more queries to access data directly without those queries having to be re-written so that they point directly at the resulting materialized structures; supplemented "vertical partitioning" of data across the multiple parallel processing units with advanced "horizontal" or "multi-level range partitioning" that supports partition elimination and the avoidance of full-table scans in many cases; introduced “synchronized scanning” so that queries can "piggyback" on one another and share data cached in memory, etc., etc., etc. This innovation is continued in Teradata 13; in one of the presentations that Stephen Brobst has been delivering on our tour of the region, he discusses his personal "top 10" major performance improvements delivered with the latest release of the database – and he had 100 major new features to choose from when selecting his personal favourites.

Specifically, Teradata has also introduced the concept of "multi-temperature warehousing" to the industry. Our new storage virtualization product – Teradata Virtual Storage (TVS) – further expands our capability in this space and will be the subject of another post in due course. Try and contain your excitement!

Oxymoron, meet storage

Other vendors have taken a different approach that is often characterized or described as "intelligent storage". The theory goes something like this: since it is the components between the processor and the data that get in the way, let's try and reduce the amount of data flowing around the computing system by discarding data as it flows off the disk drives. Typically these designs support simple "projection" operations (discarding columns that don't qualify for the query); simple restriction operations (discarding rows that don’t qualify for the query); and, in some cases, simple join operations.

The first thing to notice about this approach is that "intelligent storage" is actually a complete misnomer; the storage is a conventional disk drive and is as dumb as it ever was. The intelligence is adjacent to the disk drive, not embedded within it. For the hard-pressed disk drive, life is as hard as it ever was; it is only the rest of the computing system that is spared the burden of carrying the "wasted" I/O. Other approaches to restriction - like Teradata's range partitioning, for example – prevent some or all of the data that does not qualify for the query ever being lifted from the disk in the first place. This is a much more efficient approach, since it is after all the disks that are the rate-limiting factor.

The second thing to notice about this approach is that the extent to which the filtering will improve query performance is heavily dependant on the nature of the physical data model and the nature of the queries. If we have a heavily denormalized data model, for example, and the majority of the queries select only a small fraction of the columns from the wide, multi-column tables that result, then filtering the data as is streams off the disk will indeed reduce the I/O burden on the rest of the system. If we have a normalized physical data model and more complex queries, the reduction of this burden will be much more marginal.

The third issue with this approach is that it implies a multi-level architecture; if we perform only the simple projection and restriction processing adjacent to the disk, then it follows that we must do everything else, somewhere else. This matters because in a parallel system the "parallel speedup factor" is a function of the "degree of parallelism" (how many parallel processors are working on the problem at the same time) and is constrained by any processing that is performed serially, or with reduced parallelism. In those designs in which the massively parallel processing is confined to only part of the query, it is axiomatic that the parallel speedup factor will be reduced.

Fourthly, these designs make it much harder to share cached data. Remember that the most expensive operation is the lifting of the data off the disks. Once I've paid the cost of doing this, I want to reuse the data in memory wherever I can, which is exactly the way that "synchronized scanning" works in a Teradata system. But if I only carry a subset of a table into memory, then I can't reuse this data to support other queries unless they require only the same or an even smaller subset of the data.

Lastly, note that these designs are typically based either on proprietary hardware technology or use special-purpose filtering software. In the former case, the proprietary hardware typically limits concurrency and may make impossible the use of indexing techniques that permit direct access to data; in the latter case, the intelligence is no closer to the data than it is in a Teradata system, because data still has to get off the disks and across the bus to memory before it can be accessed by the software filtering process running on a processor. In fact, total path length is increased in these designs, because the filtered data than has to be sent to the second processing tier before it can be joined, aggregated and sorted, possibly serially, certainly with reduced parallelism.

Is intelligent storage really such a smart idea?

The message of some of the vendors of "intelligent storage" platforms is that customers need to choose fast parallel scanning or index-directed access. This completely misses the point. If we are to support "traditional" data warehousing workloads alongside "active" workloads from the same database, then we need to be able to support fast parallel scanning and index-directed access, simultaneously. If we turn every simple tactical query into a table-scan, it doesn't much matter how fast we can scan data – the system will still be overwhelmed at the point that we have to support hundreds, thousands or even tens of thousands of concurrent queries supporting pervasive BI deployments. Conversely, since we can't possibly anticipate every single access path that will be required to support evolving, complex and ad-hoc query requirements, we must be able to support high-performance parallel scan, join, sort and aggregation operations; even if we could predict all the likely access paths, there simply wouldn’t be enough time in the batch schedule to build and maintain all of the indexes that would otherwise be required. And that assumes there still is a batch schedule, of course; but with Gartner predicting that 90% of data warehouses will be populated on an intra-day basis by the end of 2010 and also predicting that 50% will be populated multiple times-per-hour by then, you should probably assume that your batch schedule is going the same way as Harry Lime.

So, is intelligent storage really such a smart idea? My answer: not if we are trying to build an application-neutral, multi-user database. Which is surely what we are trying to achieve if we are seeking to build – and activate - an enterprise data warehouse. Make mine a Von Neumann Iced Tea!

Martin Willcox

A Disaster Doesn't Have to Be a Disaster

Darryl McDonald — 10/5/2009

Mention "natural disaster" to most people and they'll think of floods, earthquakes or hurricanes. Mention "human-induced disaster" to IT people and they'll have equally frightening thoughts. But whether natural or human--storm or hacker--disasters are easier to get through if you’re prepared.

That's why we developed the Teradata Disaster Recovery Solution. It delivers the functionality you need without requiring your chief technology officers to bear the expense--in time and money--of building a system on their own.

The Teradata Disaster Recovery Solution is a dedicated and secured facility. It comes with equipment, software licenses, database administrators trained in disaster recovery and much more. It's a complete package, including the cost of cooling, powering and running servers.

The first step we take with clients is a simulated failure orchestrated by our team of disaster-recovery engineers. This is always an eye-opening experience, as customers see the extent of their vulnerability and realize it's far better to discover it in a test than in the aftermath of a real event.

In other words, we like to think of disaster recovery the same way a company might think of security. Smart companies don't fix security holes after a breach. And they approach disaster recovery the same way: they don't wait till after a disaster to start taking it seriously.

And when you do consider it seriously, the next step is to consider the Teradata Disaster Recovery Solution. It's an effective shield against data-warehouse downtime and, in the unfortunate event of an unforeseen failure, it includes the industry's best tools and services to recover from those challenging times.

Have you ever faced a crisis in your data warehouse? Hacker break-in? Busted pipe? Leave a comment and tell us about it.

Von Neumann and the dots

Patrick Hempton — 8/14/2009

Vienna is beautiful, historic – and somewhere I have been only once before. So I regret that I can’t start this post with an amusing anecdote. But stay with me.

As a fan of film noir whose formative years flashed by in the 80s, when I think of Vienna I think of two things: the Ultravox song of the same name and the film "The Third Man".

For those of you unfamiliar with the film (I assume that everyone over 30 knows the song and that anyone under 30 is too busy tweeting to read this), it is concerned with an apparently deceased villain (Harry Lime) who exploits the chaos of post-war Vienna to steal penicillin - then a priceless wonder drug - in order to adulterate it and sell the weakened version on the black market. The scheme profits Lime, but leaves behind him a trail of dead and crippled children. Well, it is a film noir.

In an effort to recruit the hero of the piece – Holly Martins, a writer of pulp fiction – to this diabolical enterprise, Lime takes him for a ride on the giant "Riesenrad" ferris wheel, likens the people of Vienna below them to dots and asks Martins if he would really feel pity "if one of those dots stopped moving, forever".

Martins declines to join his old friend in his criminal venture, ultimately kills him - and still doesn’t get the girl. Now that's what I call noir!

In another life, Martins could have been a Teradata engineer or a Teradata customer. It's because we care about the dots that we build a platform that enables our-equally-caring customers to build analytical databases stuffed so full of detailed data that they can recognise the dots for who they really are – individuals - and treat them differently, according to their wants and needs. As my travelling companion on this tour, Teradata's CTO, Stephen Brobst, likes to say: "if you’re treating me and my brother the same, you're doing a bad job."

Of course, Teradata customers don’t have to care too much about how their systems are screwed together – as the "original appliance vendor" (Gartner’s description of Teradata, not mine) – we do that for them, so that they can get on instead with the more interesting job of building and exploiting analytical databases. But our R&D engineers have to care. And one of their concerns is the "von Neumann" bottleneck.

The "von Neumann bottleneck" might sound like an exotic cocktail, but it actually relates to one of the oldest problems in computer science.

All modern computer hardware follows the same basic design pattern, named after John von Neumann - mathematician, physicist, chemist, Los Alamos atom bomb maker, computer scientist and some more besides - who described it in a paper written in 1945 (genius and polymath though he undoubtedly was, he was actually describing work largely undertaken by J Presper Eckert and John Mauchly and so doesn't really deserve all of the credit). In a von Neumann machine, instructions - and the data on which they operate - are stored in persistent storage and both follow the same path to the processor (the component where the computation is actually performed): off the storage; across an interconnect ("bus"); into memory; back out across the bus; and, finally, into the processor.

The genius of the Von Neumann architecture is its flexibility; the "stored instructions" are software - and we can change the functionality of the machine just by replacing or changing the software. Once upon a time, computers were hard-wired and computer scientists had to set switches and insert patch leads to make them do different things between "runs". That programming model didn’t scale at all well; the "stored-program" model pioneered by von Neumann and friends gave birth to the whole modern ICT industry in which we are all employed today. Probably we should create a cocktail in their honour, after all.

There is a catch, however. The physical separation of the processor and memory limits the rate at which instructions and data can be passed between the two, limiting the rate at which the processor can do useful work, so that where a von Neumann processor is asked to perform relatively simple calculations on large volumes of data, the processors are often sat idle, waiting for the data which they need to continue working.

"Relatively simple calculations performed on large volumes of data" is, of course, a pretty fair description of many "traditional" decision-support workloads, which is why Teradata systems spread those large volumes of data across multiple disks attached to multiple computers ("nodes") in a massively parallel processing (MPP) architecture. If you engineer an MPP system well enough to remove the other bottlenecks, then the "I/O bandwidth" (the rate at which you can get data off the storage) available to each computing node is a function of the number of disk spindles attached to it; in this way you can make sure that there are sufficient disk spindles available to "saturate" the computing node so that it can get the data it needs just as fast as the bus connecting the storage, memory and processor - and the physical separation between them - will allow.

So far, so good. But as Stephen points out in one of the presentations that he has been delivering on our European tour, processor speeds have increased by a factor of five million over the course of the last 30 years, whilst disk access speeds have increased by a factor of… five. Making those magnetized platters spin faster whilst ensuring that we can still read the data encoded on them accurately turns out to be much harder than the semiconductor physics that has driven the improvements in processing speed. Either that, or the mechanical engineers just aren't pulling their weight (which is an easy thing for a former physicist to say).

In fact it's worse than that, because the same advances that have supported the dramatic improvement in processing power have enabled disk storage densities to double roughly every 18-24 months. When I joined Teradata five years ago, our systems at "design centre" configuration were built with 73GB drives; today it is almost impossible to source these newer drives and we have to use 146GB or even 300GB drives instead. That's not all bad – these newer drives offer a better price-per-gigabyte than did the 73GB drives – but since we need more-and-more spindles to saturate each new generation of more powerful processor and since the disks hanging off these spindles are getting bigger-and-bigger, each computer node has more-and-more data to process. We are approaching the point where the next generation of data warehouse platforms will have worse scan performance than the current generation, unless CFOs are willing to pay for high-density disk drives and then fill them only part-way. Which seems about as likely as Harry Lime passing the interview with St. Peter.

Where is our Martins, sent to save us from the villain that disk access times are becoming? Tune in next time…

Martin Willcox

The morning after the decade before

Patrick Hempton — 8/14/2009

The enduring tourist images of London are probably Big Ben, red “Routemaster” double-decker buses and Beefeaters guarding the Crown Jewels at the Tower of London. In fact the Routemaster buses have largely gone now, replaced, to the intense annoyance of many Londoners, by the mobile roadblocks that are more properly known as “bendy buses”. And for a time at the end of last year, it looked as though London might lose another cherished asset: the reputation of the capital’s banks and financial institutions for expertise and probity, in the face of the global credit crunch and the collapse in asset valuations.

In the aftermath of the G20 summit – also held in London - and the first, fragile signs of recovery across the world, it’s easy to forget now just how close we came to a complete meltdown of the global financial system. But when the objective histories are written a few years from now, they will almost certainly show that we came far too close for comfort.

For UK plc, this is a big deal. “I would rather see Finance less proud and Industry more content”, wrote the then Chancellor of the Exchequer, Winston Churchill, in 1925. In the intervening 84 years the UK has become even more dependant on the financial and service sectors for its prosperity, to an extent unknown in the rest of Europe. There is much more at stake in the UK than the size of bankers’ bonuses, although no one seems to have told the British tabloids that.

For the banks, the worst is probably now over. By-and-large they have found new sources of capital to replace the moribund wholesale credit markets, either overseas or closer to home in the form of loans and guarantees backed by the UK taxpayer. What they are engaged in now is the long, slow process of repairing their balance sheets in the face of those drastically reduced asset prices.

In the UK that is a particular challenge, because UK households are, on average, more heavily indebted than those almost anywhere else. We bought too many over-priced houses, too many expensive cars and too much bling when the good times were rolling - and we didn’t pay cash. And since employment traditionally lags economic recovery, even if the optimists are right and the long, slow rehabilitation of the economy is now underway, more of us yet are at risk of losing our jobs and defaulting on those debts.

The banks have traditionally assessed our credit-worthiness by using scorecards, derived from both their own data and from data purchased from credit-risk agencies. But the data that originates from the credit-risk agencies is collected monthly from all of the banks, building societies and other financial institutions and must be processed and aggregated before it is usable. By the time the summarised data is available to the credit risk officer in a bank, it is typically several months old and can only tell you that I was a good risk before I lost my job, my wife and my house; not whether I am still a good risk (unlikely, under the circumstances).

It turns out that many of the bigger banks can do a better job themselves - and much more quickly. By assembling a cross-channel view of their customers in an enterprise data warehouse and then “activating” it - by loading data more frequently and using event triggers to identify abnormal behaviours - the banks can actually learn a great deal about the state of our current financial health, without having to wait for the credit risk agencies to note two or three missed monthly payments on a car loan and catch-up with our reduced circumstances.

For example: taking lots of cash advances on a credit card; putting lots of low-value purchases that would ordinarily have been debited to a current account on a credit card; trading-down from a “high-end” to a “low-end” supermarket for grocery shopping; requesting bank statements more frequently and more irregularly; logging-on to Internet banking services daily when monthly was the norm; all these are potential signs that a customer is in financial trouble. And in fact there are scores of early warning indicators like these. Taken in isolation, these events may signify nothing; where several signals occur or re-occur within a short space of time, however, it becomes statistically much more likely that a customer is in trouble.

And establishing early on that a customer is in trouble is key, as it both gives the bank an opportunity to assist the customer in resolving those difficulties – and to minimise its exposure to any eventual default.

Note that you can’t do this with summarized data, or with channel-specific data-marts, or with an operational data store that captures the current state of a customer’s accounts but not the history of their transaction movements. Integration of data across channels and products, so that, for example, we can see that a customer is using a credit card to make loan payments, is critical; so is the ability to identify significant events – abnormal behaviour – from the noise of day-to-day activity. To know if you’re acting weird, I need first to know what your normal looks like; to know what your normal looks like, I need your history. In detail.

If the British government’s forecasts are correct, the UK economy will bounce back next year, growing at a healthy 3%. Britons of all political stripes and none are united in fervently hoping that this assessment is correct, whilst worrying that it is optimistic. But whether the government’s forecast is right-or-wrong, credit will be much harder to come by in the UK for the foreseeable future than it has been in the recent past. And the banks that repair their balance sheets the quickest are likely to be those that are smartest – and fastest – in ensuring that they don’t throw scarce good money after bad debtors.

Martin Willcox

Q2 2009 Teradata Magazine

Rick Loconto — 6/24/2009

Organizations must adjust their strategies to fluctuating economic conditions to compete effectively. Short term wins and greater risk-taking might have paid off for companies sprinting through boom times. However, crossing the finish line in the current environment requires greater efficiency and prudent use of resources.

The Q2 2009 issue of Teradata Magazine focuses on the improved productivity and effectiveness that data warehousing and business intelligence (BI) can uncover and enable.

Senior director of enterprise BI at Cisco Systems, Kevin Bolden, explores ways to leverage data in lean times. Randy Lea, vice president of Global Product and Services Marketing at Teradata, explains the value of data mart consolidation in reducing costs and making more intelligent decisions.

Innovative organizations such as Poste Italiane, SNS Bank and ARC provide prime examples of how to better serve, communicate with and retain customers by tapping into the capabilities of an enterprise data warehouse. At street level, growing numbers of IT professionals are facing a tough job market by honing their skills and broadening their knowledge through certification.

As companies rethink the traditional workplace—often as a means to reduce office expenses—the number of remote workers is rising. This brings new opportunities and challenges for IT and management.

Revitalizing an existing data warehouse is just one way businesses are making the most of what they have. For those just starting a data warehouse implementation, careful planning is required.

In Tech2Tech, Larissa Moss, president of Method Focus Inc., presents her Business Intelligence Roadmap. Learn about the new Teradata Virtual Storage option which enhances system management by enabling optimal system balance in the data warehouse. In addition, we examine how query banding can provide more detail in workload management and workflow analysis.

In the long run, organizations need to rethink their business operations to become leaner and improve agility if they are to succeed. Please visit http://www.teradata.com/tdmo/ to access these articles and more.

A Tale of Two Tourists III

Patrick Hempton — 8/14/2009

Continued...
This will create new business models for someone, but not necessarily for the mobile operators. I don’t need to know whom you are to count you - and your cell phone broadcasts your position to anyone that cares to listen, not just to your operator. However, as this digital eavesdropping makes people queasy even though it generally doesn’t break any laws, many commentators speculate that for these schemes to gain public confidence and realize their full potential the anonymized, aggregated data will need to be enriched with some personal data that customers are incentivized to provide. As the owners of the relationship with the consumer, the operators should be well-placed to broker this exchange and to market information about their customers to third parties wanting to deliver new services to them. Assuming privacy concerns can be overcome by creating additional value for consumers, this vision could become reality very soon. And the evidence – from a whole gamut of services ranging from Google Mail to Evernote - is that consumers will happily sacrifice some privacy in return for high-quality, “free” services.

For example, I’m sure that the Louvre knows approximately how long the queue outside its entrance is - so why not tell me, whilst I’m on the train? Had we two bohemians known that there was a chance to get to see the Mona Lisa without enduring a long wait, we might well have disembarked from the train and taken the opportunity to gaze into those enigmatic eyes (you know, from the right angle, she looks a little like Juliette Binoche). Incidentally, if you think this is far-fetched (the bit about the Louvre, not the alleged resemblance of the Mona Lisa and Juliette Binoche), then you don’t know how a leading amusement park operator uses Teradata to manage queue length and so increase customer satisfaction. There is much more than fun-and-games at work in the best-managed parks these days.

So much for our Paris experience – even when I am not looking for them, all I see are new data warehousing applications. Perhaps I should get out less often! Except, of course, that Juliette is out there somewhere…

Martin Willcox

A Tale of Two Tourists II

Patrick Hempton — 8/14/2009

Continued...
So when we came to the French capital yesterday we decided that, in the little time that we had to spare, we were ready to experience all that Paris has to offer. We took the métro and felt pretty bohemian just for getting closer to the cité, cultural epicentre of the Continent. Well, almost the cultural epicentre of the Continent. Seasoned travellers will know, of course, that the real cultural epicentre is my adopted home town of Sheffield in South Yorkshire and that haute cuisine and vin rouge from Bordeaux are all very well, but that you can’t beat a chip butty and a pint of Tetley’s.

Anyway, I digress. Waiting at the station, we heard a phone ring and a man standing next to us started talking affectionately into his mobile. He made a sort of a show of this and it soon became apparent that the woman he was talking to was only metres away from us on the crowded platform. And that she looked a little bit like Juliette Binoche.

I digress encore. The two lovers kept on talking to one another on their phones and exchanging amorous glances across the crowded platform. All of which reminded me of a conversation topic of this tour; the mobile operators are worried that their profits are shrinking as a result of the competition and the legislation that is driving down the cost to customers of making calls on their cell phones. As voice and data services become increasingly commoditized, what else do the operators have to sell and how can they keep customers loyal? The answer may lie in the data that all those mobile phones produce and what this data tells us about the behaviour of their owners.

For example, you could very easily use signal data to count how many mobiles – increasingly, a pretty accurate proxy for the number of people, now that even my Dad has one - there were on our crowded platform in Paris. Then you could warn people at the stations further up the line that the approaching train has standing-room only. If these people could get this information from the platform display - or even better, on their smartphones whilst en route to the station – then they could change their plans accordingly, divert to a pavement café for un jus d’orange et un croissant and travel a little later. Incidentally, if this sounds far-fetched to you, then you haven’t read the latest Technology Quarterly in the June 6th-12th issue of The Economist magazine. This cell phone monitoring technique is real and is already being used, for example, to measure footfall around shopping centres.

On a meta-level, city officials could use these same data to optimize the schedules of their interconnected transport networks. Back in Madrid, where we were on Monday, the Spanish traffic authority already analyzes road safety and utilization using Teradata. With fresh mobile phone data available and in a closed environment like the metro, the city officials could monitor passenger flows in real-time and then take appropriate action as necessary, to, for example, hold an early-running, lightly-loaded train at the station so that the surge of commuters that the system had just detected arriving could be efficiently accommodated. This wouldn’t work in London, where all trains are always late and over-crowded, but Paris is altogether more civilised. Almost up to Sheffield’s high standards, in fact.

Martin Willcox

A Tale of Two Tourists

Patrick Hempton — 8/14/2009

One of the disorienting things about the road show and the tight schedule that Teradata’s CTO Stephen Brobst and I are operating on is how little we get in touch with the countries that we find ourselves passing through. I guess any frequent business travellers out there will recognise the phenomenon; your friends and relations all marvel at the interesting job that takes you all around the world and you somehow don’t have the heart to tell them that one airport departure lounge looks a lot like another; or of the slight melancholy that spending long periods away from your wife and kids leaves you with; or that the snatched glimpses of the sights and sounds of the city away from your hotel room are few-and-far between.

If all of that sounds a little self-pitying then I should say straightaway that I am one of the luckiest guys alive and that I wouldn’t change my job for any other, with the possible exception of fast-jet pilot or personal assistant to Juliette Binoche (Mandy, if you’re reading this I’m joking about re-training to be a fast-jet pilot and anyway they wouldn’t take me at my age and with my eyesight). The reason for this is the sheer range of interesting projects and fascinating people I get exposed to. The air-conditioned meeting rooms and hotels that we meet in could mostly be anywhere, but the people are alternately inspiring and challenging and have endlessly fascinating stories to tell. Stephen and I have been touring Europe and North Africa continuously for more than a week now, meeting all sorts of smart people from all sorts of organizations - ranging from a geophysicist at the Russian Academy of Sciences to senior bankers - gaining and exchanging insight. It’s genuinely invigorating. Still, there is only so much data warehousing and business intelligence that even the most hardened enthusiasts can take…

Martin Willcox

Cross-Country Musings on Cable and the Future of Advanced Advertising

Patrick Hempton — 8/17/2009

Aside from the to-ing and fro-ing associated with a rapid business trip to NY (especially when you hail from Los Angeles), I’m excited to meet with several senior executives at one of the major Cable MSOs this week to talk about Teradata’s offer in the addressable advertising space—our Audience Intelligence Solution.

Cable has a long-standing successful business for local advertising, approximating $7bn in the US per year. Recent developments with Canoe Ventures—a consortium of the five largest MSOs-- and independent activity at those top tier MSOs indicate that Cable is the new innovator in advanced advertising. They are uniquely positioned to change the dynamics of advertising through the enablement of addressable advertising through the television.

Cable reaches more than 100 million subscribers in the US. Currently, cable MSOs have been able to command top-dollars for their ability to serve locally relevant advertising to their local footprint. Advances in addressable advertising, along with business innovation driven by Canoe Ventures, is leading to the ability for MSOs to reclaim all advertising time with the promise of making every avail addressable—initially at the market level, with the long-term promise of delivery at the household level.

The challenge for cable companies—and satellite TV providers, IPTV providers, any organization with rightful access to data detailing customer engagement with content at the set-top-box—is how to aggregate and understand what audiences are doing, watching, engaging with, caring about when it comes to content and advertising. Only then is the technology meaningful, because only then can the right message be delivered to the right audience at the right time. Teradata Audience Intelligence puts this control in the hands of the data owner, enabling the MSO to carefully leverage data acquired through set-top-boxes, subscriber records and online activity to create and present detailed profiles of viewer behavior back to programmers planning their ad-buys. Coupled with new advances in ad-insertion technologies, MSOs will be able to deliver all avails to Network programmers as addressable, enabling dramatically improved revenues and increased value to programmers.

This is the story I’m thrilled to tell in t-minus 36-hours… I just hope the peanuts are worth it!

Colleen Quinn

Grand frère?

Patrick Hempton — 8/14/2009

And so to Brussels, home to the best moules-frites served anywhere on the planet and a daunting array of bewitchingly strong – and occasionally bewilderingly flavoured - beers.

Sadly, last time I was here I didn’t get the opportunity to sample either. I was presenting to the European Commission, specifically to the committee charged with the evaluation of the “Data Retention Directive”. And it was a “smash-and-grab” trip, meaning an early start to catch the first flight out in the morning and home on the last flight back in the evening, which was then delayed by several hours due to a violent electrical storm. C’est la vie!

If turnout and results in the recent elections to the European Parliament are anything to go by, we Europeans are somewhere between blasé and downright hostile where our European institutions are concerned. Some blame this on their perceived remoteness and lack of democratic accountability; others on the inherent tensions between national legislatures and the supranational one; whilst others believe that European elections are often no more than referenda on the popularity of national governments, claiming that low turnout is symptomatic of no more and no less than voters’ disenchantment with their own governments.

All plausible explanations, I’m sure. There is another one, however, which is that the best work that these institutions perform is detailed and unglamorous and not easily reduced to a 30 second sound bite. On this line of reasoning, the EU is not so much unloved, as invisible – until the point when an unpopular national government needs a convenient scapegoat for a difficult decision.

I’m not sure that I’m qualified to express an opinion on such weighty questions - or that Teradata corporate marketing would want me to! But I will make an observation: the work that the Commission and the Parliament do well, they do astoundingly well.

Take, for example, the single European currency. Regardless of whether you think this is a good thing or a bad thing, the simultaneous implementation of the new currency across 12 member states in 1999 without a single serious major issue was a triumph of planning and organization. And despite the difficulties that follow from a “one size fits all” interest rate in a time of recession a decade later, none of the countries that have adopted the Euro show any signs of wanting seriously to leave - and plenty more would like to join.

Which leads us rather neatly back to the Data Retention Directive. If you are not familiar with the Directive, it seeks to harmonize the collection and storage of electronic communications data to support criminal and terrorist investigations by national law enforcement agencies (LEAs). Balancing the right of individuals to privacy with the responsibility of governments to protect citizens is politically explosive and so it is interesting to note that no less an authority than Prof. Francesca E. Bignami from the Duke School of Law sees the Data Retention Directive as an exemplary piece of legislation, that should serve as a yardstick for the use of this data to improve criminal prosecution whilst protecting fundamental rights.

Others disagree, of course, regarding this as part of a wider slide towards a surveillance society. But what is important to remember is that member states had by-and-large already given themselves the power to know whom we called and when (but not what was said, that’s “lawful interception”, related, but different in kind to data retention). The intention of the EU was to harmonize the often-informal arrangements between national governments and national operators; and the Directive in any case has to be written into local law in each member state through the normal legislative procedures.

Retention of traditional, circuit-switched telephony data (call detail records or “CDRs”) appears to be relatively uncontroversial in most member states, I think because consumers have largely embraced the idea of differential pricing and understand and accept the requirement this creates on Communication Service Providers (CSPs) to store usage data. If I want my operator to bill calls made after 6PM to a local number at a different rate to those made before 6PM to an international number, then they have to know which calls I made and when. We could debate the extent to which those outside the industry understand how this same data is also used to segment customers according to their profitability, create churn models and so on. But for most ordinary citizens, this doesn’t seem to pass the “so what?” test.

Retention of IP communications data is much more controversial, partly, I think, because this data is by nature more intrusive and partly because the nature of most IP-data pricing models means that the retention of this data does not create any direct value for the citizen. (There should, of course, be an indirect benefit, as we might hope and expect that the LEAs will catch more criminals, leading to safer streets. But we humans are fickle and generally prefer our gratification to be instant.)

For example, I call the same Sheffield number at least once every evening when I am travelling overseas. It wouldn’t take a genius at my mobile phone operator to figure out that I’m calling home – and since my operator is currently migrating its CDR data warehouse to Teradata, it is clearly full of some very smart people. But the analyst at my service provider doesn’t know whether I am whispering sweet nothings, negotiating the terms of a messy divorce, or just catching-up with my wife and our three kids. At a very basic level, my privacy is preserved.

The situation where IP data is concerned is very different. To learn anything useful for law enforcement purposes, we need to capture source and destination IP addresses, which in almost all cases gives us the domain (e.g.: www.bbc.co.uk) that individual surfers have visited (although not the details of all of the pages that they have actually viewed and the data that they have exchanged, which takes us back into lawful intercept territory). If I know this, I have a pretty good idea who you bank with, whether you are a pornography user or a Nazi sympathiser, and all sorts of other things besides that you might not otherwise have chosen to reveal to a complete stranger. Many people find this troubling, even if it is only the LEAs that are able to access the data and if exploitation by the operators for commercial gain is prohibited. And the issue is compounded by the fact that there is at present no direct value that is created for the citizen by the retention of this data, as most of us aren’t on any sort of differential billing scheme for IP service usage that might incentivise us to share this level of information.

There are other issues: the inherent anonymity of some new, IP-based communication mechanisms and ambiguity over the question of exactly who should capture certain types of IP-based communications data (for example, should it be the responsibility of a social networking site to capture data or is it the responsibility of the CSP that provides access to the social networking site?). Legislating against the backdrop of a constantly evolving Internet without granting the LEAs carte blanche to eavesdrop at will is no easy task, hence the good Professor’s admiration.

Timely access - the ability to directly access large volumes of data stored in a high-performance database without having to first recover data from offline media – is critical. Many in the EU will tell you that this wasn’t given enough focus in the drafting of the Directive (it is the data retention directive, after all, not the data retention and access directive). And many representatives from LEAs at the recent conference bemoaned the fact that criminals are still escaping prosecution – often for child pornography offences – because it takes some operators months to recover IP traffic data that is only available on tape backups and which cannot be restored without taking network infrastructure out of production. For a parent of three young children, one of whom has severe special needs and is almost uniquely vulnerable, that is troubling.

Also important is the concept of “multi-temperature” data management, which enables rapid access to the most frequently accessed “hot” data (typically the newer data) but that still permits this data to be combined and integrated with older, “cooler” data to support more complex investigations, without incurring prohibitive expense. All of the Teradata customers that have built Data Retention solutions on Teradata – and there are several, which is why we were one of only two technology vendors invited by the Commission to speak at the recent conference - are exploiting our “mark 1” multi-temperature data management capability. Teradata Virtual Storage (TVS) – our “mark 2” implementation - will enable Teradata customers to store both the “hot” and the “cold” data in a single database, whilst enabling the “cooler” data to be stored on even-cheaper-per-gigabyte high-density storage devices, at the cost of some degradation in access performance for this “cold” data.

This is important, because complying with the Directive is not cheap. Whilst it is true that the unit price of computer storage has reduced significantly in recent years and that vendors such as Teradata have introduced new product lines (e.g.: the Teradata Extreme Data Appliance) that enable organizations to store detailed data more cost-effectively, it is still the case that organizations need to make a significant investment in infrastructure to comply with the Directive. In addition, large information systems attract significant additional costs in respect of design, development, operations and maintenance. And even if you are OK with the state knowing where you have been on the Internet, you probably don’t want a criminal impersonating a police officer to access your data, so access to these systems needs to be controlled on an on-going basis, which means more management processes for the operator, meaning more cost. This is a sensitive issue, because the EU left it to the governments of the member states to decide whether – and how - to reimburse the costs of the operators, so that for operators in some geographies compliance is cost-neutral, whilst for others it emphatically is not.

In fact, the importance of multi-temperature approaches to Data Retention are likely to increase still further in coming years, given that volumes of IP-traffic are increasing fast year-on-year and that there are, at least as yet, relatively few identified opportunities for CSPs to re-use this data to business advantage (and ambiguity over the legal position of their doing so in many cases).

It is also the case that LEAs are increasingly required to co-operate across national borders, as criminal activity is every bit as subject to the forces of globalization as are other activities. In the absence of an EU-wide database of electronic communication records – a formidable undertaking and anyway almost certainly about 25 steps-to-far for the more Euro-sceptic nations like my own, which has recently abandoned plans even for a national database - this means that LEAs will increasingly need to re-assemble large and complex data sets obtained from several CSPs on an ad-hoc basis, in order to resolve links between suspects that involve intermediaries, multiple communication methods and multiple operators.

For example, if A calls B using CSP 1, B sends an SMS to C using CSP 2 and C sends an e-mail to D using CSP 3, then A and D are related – but three separate data-sets must be stitched together to establish this fact. The EU has developed data interchange standards in this respect – another example of the good work done in those unglamorous committees - but they are not binding. That’s a pity.

In theory, the Directive should now have been implemented in all member states and for all electronic communications; in practice, progress is patchy, especially outside conventional telecommunications.

The questions of how long data must be retained for and the extent to which IP traffic data in particular can be re-used for commercial advantage are key to improving progress. Some telephony CSPs, for example, have chosen to address the requirements of the Directive by deploying dedicated systems for this purpose, whilst others have extended and enhanced their existing commercial analytical databases. Since the latter is, in general, a more cost-effective approach and one that has the potential to create additional value for the CSP, the willingness of CSPs to make the substantial investments in the infrastructure, people and process that will be required to improve access service levels to LEAs and to extend effective data retention beyond traditional telecommunications data may depend upon the resolution of these questions. This in turn is likely to require a much wider debate, both with and amongst EU citizens, than has taken place thus far and that addresses legitimate privacy concerns. When does State oversight infringe, rather than guarantee the liberties that are so precious to we Europeans? What limits should be placed on the companies that serve us, but which are ultimately accountable to institutional shareholders? What types of institutions and checks-and-balances should be placed between the state and the data, to ensure that those who govern us are protected from the temptation of attempting to extend their tenure indefinitely by misusing the data to suppress democratic dissent?

So much already accomplished - and so much still to do. Bring on the frites and the strong beer!

Martin Willcox

The Demand for Change

Patrick Hempton — 8/17/2009

Everything flows, nothing stands still. - Herakleitos of Ephesus - ca. 535–475 BC

Since at least the time of the ancient Greek philosophers, constant movement has been thought of as fundamental to understanding the cosmos. Today we might more easily recognise the paraphrasing “The only constant is change”, but whatever the field of endeavour, the understanding of change is core to the understanding of the field. In some fields such as biology, an ability to be more ‘fit’ leads to survival as Darwin famously theorised. In the same field, Dr. Phillip V. Tobias pushed the point by stating “…extinction is the rule, evolution the exception”.

These observations hold true for information systems too. There are many external forces that exert pressure to change on these systems (e.g. the economy; competition; regulation; technology advances) and perhaps just as many internal forces (e.g. business strategies; business performance; merger and acquisition activity). In order to survive, any information system needs to be able to evolve to the change in demand placed on it.

Has the change in economic conditions changed demands on your Information/Data Warehouse/Business Intelligence systems? More or less? How are you prepared to manage this? I would like to hear your comments.

Sean Kain
www.linkedin.com/in/seankain

Create spectacular experiences: Make your business a theme park!

Darryl McDonald — 7/30/2009

Daring, visionary retailers that lead the pack in customer management today focus on creating spectacular customer experiences. One of these is Cabela’s, an outdoor lover’s paradise that transforms store interiors into virtual theme parks for customers. That’s a huge differentiator – like stepping into another world.

The term “experiential marketing” was coined by Bernd Schmitt, who wrote in his book of the same name: “The degree to which a company is able to deliver a desirable customer experience – and to use information technology, brands, and integrated communications and entertainment to do so – will largely determine its success in the global marketplace of the new millennium.”

He was right, and I focus on Cabela’s because they are doing exactly this. They are getting lots of attention from retail news media because they are so successful. Well, okay I admit they fascinate me also because their managers are informed by a Teradata intelligence engine running SAS. You can and should read all about Cabela’s here.

Cabela’s is also great at personalizing customer experiences. Years ago a book called The Experience Economy contained a smart chart which I have adapted here. It shows the value of a customer experience rising in value as it becomes more personal and differentiated. The book refers to customers not as buyers of goods and services, but as “buyers of experiences.” Good point. Cabela’s has put all of this into practice.

Hey! Someone forgot to tell Cabela’s there is an economic downturn. Their customers forget it too, the minute they walk in. That’s the kind of retail experience you get when you bring together database-driven intelligence with unlimited imagination.

This Is Your Mousetrap Speaking

Patrick Hempton — 8/14/2009

A mousetrap is a mousetrap, right? Well, no. Those in the new Wembley Stadium are equipped with mobile phone chips. After the snap, these traps send a message to inform maintenance, who can otherwise leave them unchecked. I like this example because it demonstrates how even traditional household items will soon be woven into an intelligent, inter-connected grid of machines that communicate with one another and with back-end information systems.

There has been a revolution in sensor technology in recent years. In the very near future, tiny, rugged sensors capable of transmitting a wealth of environmental information to one another and back-to-base will be cheap and reliable enough to be widely deployed. And there is no shortage of potential applications, from mousetraps to industrial machinery and beyond.

For example, Spain, where we stayed yesterday, is known for its abundance of sun and smiles - and also for its chronic shortage of water. Extensive networks of sensors could check farmland humidity and allow for efficient management of agricultural irrigation. Another frequent problem is the threat of forest fires. A network of distributed sensors (detecting heat, smoke etc.) in high-risk areas could help fire-fighters to quickly identify and locate new fires in an instant – and to respond fast. If this information can be collected and integrated in near real-time, fire-fighters would also understand how the fires were spreading and at what speed. And no, it doesn’t stop there. Each and every fireman could carry additional sensors on his uniform, transmitting information like heat or location to – well, to where?

The past is another country?
Inevitably, these data – a mixture of established and newer types, for example geo-spatial data - will be collected and processed in a database. And since these sensors are capable of generating a lot of data and we will need to be able to integrate and analyse these data in a hurry, that database had better scale, had better perform and had better support simultaneous, high-concurrency load and query operations.

It’s tempting to see this as an entirely new paradigm, but I am engaged in a (failing!) one-man campaign to ban the “p-word” from technology marketing, and, anyway, as Mae West might have said: “information management isn’t one damn thing after another, it’s one damn thing over-and-over.” Specifically, I’ll bet you a good steak dinner that five trends that have been repeated over-and-over in the last 20 years will be very much in evidence.

Initially organizations will deploy these sensors to realize improvements in operational efficiency. That makes sense. When retailers first started introducing electronic point of sale systems (EPoS) all those decades ago, by-and-large it wasn’t because they wanted to build data warehouses to understand consumer purchase behaviour; it was because they wanted to shorten queue lengths whilst simultaneously reducing labour costs by improving the efficiency of their checkout staff. Italian utility Enel has reportedly deployed 30M smart electricity meters to its customers since 2001, at a cost of EUR 2.1B. Since these meters don’t need to be read by a small army of staff, but rather call home like the Wembley mousetrap, Enel saves EUR 500M per annum, so over five years they get their money back. That’s good business. But the evidence from every industry over the last 20 years is that the data that are captured from IT systems introduced to automate business processes are rapidly exploited for other purposes. If you’re not convinced by the Retail scenario, then consider telecommunications; even a decade ago it was not commonplace for service providers to store “billed call detail records” for much longer than was necessary to bill the customer, but today it is routine for providers to build data warehouses that store this information for significant periods of time to support analysis of customer behaviour, trends in profitability, customer propensity to churn, etc., etc. The same trend will certainly play out with the new sensor data.

In fact, in the geographies where regulations permit, many communications service providers are now extending these databases to capture even lower-level data so that they can, for example, understand the impact of network failures on their best customers and so enhance their churn models still further. That’s trend number two – the relentless drive for more detailed data. Plenty of the organizations that deploy networks of these new smart sensors will initially attempt to deal with the flood of new data by discarding the raw data in favour of summary data, only to discover that the devil – and the value - really is in the detail.

To understand the third trend – the use of time-variant data for predictive modelling - we need to go back to London and the smart mousetrap. If we know where each trap is located and whether it is temporary home to one of our furry friends or not, then we know whether we need to empty it. If we also have historical data that enables us to forecast whether that same trap, in that same location, at this time of year and given the current levels of activity in the stadium, can be expected to catch another rodent within, say, the next 12 hours, then we can decide whether we need to pay overtime to empty it right now, or whether we can afford to wait until tomorrow to do so. At this point, we’re no longer reacting to events, we’re controlling them.

And if we can compare this historical data with other, integrated data (trend number four) - like, say the locations of the fast-food concessions around the stadium – then we can reveal correlations that will enable us to discover, for example, that in addition to slowly poisoning their customers, SalmonellaBurger plc. aren’t managing their trash properly. And increase their rent to reflect our increased costs. Whilst making a mental note to eat somewhere else next time.

Plenty of other “ordinary” industrial machinery and household items will soon get the smart mousetrap treatment. Which brings us to the fifth trend: information consolidation.

According to the best data available – IDC’s information growth forecast – the volume of data created, captured, or replicated exceeded the amount of available storage for the first time ever in 2007. IDC also forecasts that the gap will continue to widen.

Think about that for a moment. We are on the verge of a technological revolution – the so-called era of machine-to-machine (M2M) interaction - that will generate a tidal wave of new data, but we are already incapable of capturing all of the existing data that we are generating. Clearly something has to give.

That something is likely to be the length of the “digital shadow” cast by each data element. In the same survey, IDC estimates that a 1MB e-mail generates 50MB of stored data. Some of that redundancy is for good engineering reasons; use of RAID storage so that data survives hardware failure, backup copies of data on offline media, etc., etc. Some of that redundancy is simply inefficient and unnecessary.

This is also true of enterprise data. It has always been the case that a network of multiple, overlapping, redundant data marts costs more to deploy, maintain and administer than does a centralized enterprise data warehouse - especially when you count the costs of redundant data acquisition. But faced with an exponential increase in the demand for storage, this choice is likely to become even more acute in future.

M2M technologies will be highly disruptive; by definition, not everything about their impact can be confidently predicted. But don’t bet on those petabytes of sensor data being stored two, three, four, five, even six times over in disparate operational data stores, multiple data warehouses and sundry data marts, each of which then stores the data redundantly in indexes, materialized views, RAID arrays, near-line and off-line backup systems, etc., etc, etc. Not unless you want to buy me dinner.

Martin Willcox

Scale like an Egyptian

Patrick Hempton — 8/14/2009

Cairo is the very definition of a bustling metropolis, literally bursting with youthful energy. “Would you want to drive in this traffic?” enquired the chatty driver on my last visit. As he asked me this question he was weaving at 50 miles-per-hour through traffic dense enough to walk on and with only inches to spare. Disconcertingly, he appeared to be looking at me and waiting for my response, rather than concentrating on the vehicles all around us (it took me several days to figure out that acute peripheral vision is a basic qualification for all Cairo taxi drivers). “Not in my car,” I told him, “but maybe in yours!”

Cairo is also a pretty decent metaphor for lots of data warehouses built on technology platforms designed to handle transactional, rather than analytical workloads; rapid and unpredictable growth is accompanied by some fairly serious growing pains. The brief, adrenaline-inducing 50 mile-per-hour dashes are accompanied by lots more waiting, waiting, waiting in traffic jams. In six years of travelling on business, I had only missed one flight prior to my last visit to Cairo; on the morning of my departure last time I was here, I missed two and only narrowly made the third.

Still, I love Cairo. The people are warm and friendly and I defy anyone to look out over the teeming city towards Giza and the pyramids shimmering in the heat haze without experiencing a quickening of the pulse.

As it turns out, the ancient Egyptians developed their own version of the “shared nothing” processing that characterises a Teradata system when building the pyramids all those thousands of years ago. I know this, incidentally, because Teradata guru Todd Walters is also something of an amateur Egyptologist in his spare time.

When people marvel at the – quite literally, towering - achievement that the pyramids represent, they tend to focus on the engineering ingenuity that the ancient Egyptians displayed. Equally impressive when you consider it, however, is the organizational brilliance that must have accompanied their construction. As the old military maxim has it: “amateurs study tactics; professionals study logistics.”

The workforce that built the pyramids is estimated to have numbered tens of thousands at its peak and to have fluctuated considerably at different stages of construction. All of these people needed to be fed-and-watered and only rudimentary technology and poor communications were available. And of course it had to be possible to scale-up and to scale-down these arrangements, as circumstances dictated.

The best available evidence appears to indicate that the ancient Egyptians solved this issue by taking the “sole proprietor” model and replicating it. So neighbourhoods for the workers were constructed with all of the amenities – a baker, a butcher, and so on – that they needed. When the workforce was scaled-up, a new neighbourhood was added adjacent to the existing ones. Because the amenities weren’t shared across neighbourhoods, this model scaled without issue. The baker and the butcher are metaphors for the resources like memory and storage that other computing architectures share, resulting in the resource contention that limits performance and scalability. Because in a Teradata system these resources are not shared, a Teradata system “scales out” linearly.

Linking the chain
A database, of course – even one built on an incredibly sophisticated parallel computing infrastructure – is just a link in an information systems chain. And any chain is only as strong as the weakest link. I think I only understood this properly almost exactly ten years ago.

About this time in 1999 I had just proposed to my wife. I had also recently started a new job with a grocery retailer. Their Teradata data warehouse was un-loved and un-used. My new boss made it clear to me that there was an opportunity for rapid advancement, but only if I could put things right. I was getting married, but still burdened by student debts. This, I knew, was my big break. If I could just get this right….

One of our first new projects - and my opportunity to demonstrate that there was a new Sheriff in town - was the “wastage and reduced-to-clear” project. All retailers discount short shelf-life products (“reduced-to-clear”) – and throw away the stuff that gets damaged or goes out of date (“wastage”). This one just didn’t know how often this was happening, or how much it was costing.

We got the data onto the warehouse with some effort - and then my problems really began. We were using one of the leading BI tools to provide the business with access to this new data. The business had been very clear that they weren’t much interested in just seeing ranked lists of wastage and reduced-to-clear figures, as they thought that in many cases these would just be a proxy for sales volumes. So the centrepiece of the new reporting suite was to be a report that showed the financial loss arising from discounted sales and product wastage as a percentage of the total sales of that product. But when we tried to run the canned-report that we had developed on top of the BI tool to this effect, it caused our development PC to churn for 45 minutes, only to present us with the Windows98 blue screen of death, rather than the piercing business insight that my chances of promotion depended upon.

The problem, it turned out, was the BI tool. With two computing environments available to it to join the multi-billion row data-sets together, create the derived measure and to sort the joined data-set – a massively parallel processor running a parallel database and a humble PC – it was choosing the PC, because it wasn’t smart enough to write SQL that did much more than simply fetch data from the database. When a mere human wrote-and-submitted the necessary SQL statement so that the database did the heavy lifting, it dealt with the request in a little over 2 minutes without breaking a sweat. But my sponsors were BI tool users, not database programmers and couldn’t be expected to write SQL – that’s what they had bought the tool for. It seemed unlikely that the guests at our wedding breakfast would be toasting us with Veuve Clicquot; at this rate they might not be toasting us at all…

Of course you can build a pre-calculated aggregate table to address this issue – which in the short-term is exactly what we did. But this misses the point. In an ad-hoc query environment you can’t anticipate every requirement and there is a price to pay for every redundant data structure that you create, in terms of extra development and maintenance effort, storage, ETL processing time and so on. Which is why in the medium-term, shortly after Mandy and I were married, we replaced it with the only BI tool then available that was capable of sophisticated “in-database processing”.

“In-database processing” refers to the practice of taking the processing to the data, rather than the other way around, by, for example, creating a temporary derived table on the database to join large data-sets together instead of bringing each data-set back over the network to a middle-tier to perform the same processing there on a commodity platform never designed for the purpose. In principle, it’s a very simple idea. In practice, it requires sophisticated SQL generation capabilities.

Incidentally, note also that whist even the dumbest database optimizer on the market should be able to do a better job of this than a middle-tier application server or PC workstation, this approach does make additional demands of the database server, which is going to be doing more work, more often in this scenario and which had better be able to swallow whatever complex generated SQL the tools choose to throw at it without getting indigestion. You’d expect me to say this, but one database management system in particular makes a better fist of this than the others. Caveat emptor!

This capability is now mainstream where BI tools are concerned, insofar as this approach has been accepted by just about all of the vendors of these tools and implemented in them to a greater-or-lesser degree, often with the help and advice of Teradata’s R&D department. In the last half-a-dozen years or so, it has been adopted by many of the ETL tool vendors, also with our advice and encouragement, so that many of these products now enable some or all of the data transformation processing to take place on the data warehouse. The final frontier for this approach is the data-mining tool space.

To this end, Teradata and SAS announced a strategic partnership several years ago to optimize the integration of the SAS ETL, BI and data-mining tools and applications with Teradata. This partnership has some way still to go, but it has already achieved remarkable things; for example it has enabled Warner Home Video to reduce processing time in the calculation of one of their critical business models from 36 hours, down to 75 minutes. Reduced cycle time means that more models can be built and that they can be deployed more quickly. That can mean the difference, for example, between establishing that a customer is about to churn in time to schedule an out-bound call to persuade them not to - and calling them a day too late, after they have already defected.

The benefits of this approach for model development and scoring are manifold: less data movement; leading to reduced data redundancy and improved data management; meaning reduced investment in computing platforms; and in model data and detailed data being available side-by-side on the same platform to support further analysis and increased model refinement. Our joint customers and prospects in Egypt are so keen to get an update on our progress with this initiative that we have changed the agenda of our Cairo meeting to include a speaker from SAS.

My travelling companion on this tour, Teradata’s CTO Stephen Brobst, has been promoting the virtues of in-database processing for years now and I imagine that it must be very satisfying for him to see this idea become mainstream and be adopted so widely. In the end, it seems that you just can’t keep a good idea down. Much like the people of Cairo, who keep smiling and weaving through their city’s traffic jams.

Martin Willcox

Drill-down for Oil – in the Data Warehouse

Patrick Hempton — 8/14/2009

I’m fortunate that there have been many memorable moments in my life; getting married, the birth of my three kids, the part-time undertakers dropping my Grandfather’s coffin (with him in it) as they lifted it through the window of the last house that he and my Grandmother shared together in rural France (as a fan of slapstick humour, I’m confident that he would have seen the funny side); Sheffield Wednesday beating Leeds United 6-2 at Hillsborough; and so on.

One moment that is right up there with these others for me, though, is looking up at the Kremlin. You see, as a balding (very nearly) 40 year old I grew up watching grim-faced BBC journalists report against the backdrop of Red Square on the latest escalation in tension between East and West on the small black-and-white TV in our family home. And so it was genuinely emotional for me to travel to Russia years later, with the Cold War consigned to history, to be so warmly welcomed by Muscovites and to stand in Red Square and in the shadow of the Kremlin.

Probably it also helps that this epiphany took place only last week. My memory is already not what it was, but it’s not that bad yet. Although if anyone reading this knows where I’ve put my pen down, do please drop me a line.

The first thing you notice about Russia as you fly east is that it is vast; travelling from Manchester to Moscow involves crossing 3 time zones, with another 11 (I think) between Moscow and the eastern rim of the Russian Federation. And so are the Russian oil fields. Experts estimate that Russian oil makes up 6 percent of the world’s reserves (or even 16 percent of untapped oil resources). As one of the so-called BRIC nations, a lot of Russia’s recent economic growth has been fuelled by its energy exports, and oil will continue to be big business in the world’s largest country. Nevertheless, not even the presently abundant and easily accessible Russian deposits will last forever. The industry needs to discover new reservoirs and to exploit them as efficiently as possible.

You won’t be surprised to hear a guy from Teradata claim that this is, at least in part, an information management problem (are there any worthwhile problems that aren’t?). Seismic exploration generates vast volumes of “sub-surface data” that are typically processed over-and-over. Critically, this data must also be carefully managed. Collecting the data is often fantastically difficult and is always fantastically expensive. And a potential deposit that it is not economically viable to exploit today may well look a lot more attractive in several years time, as increased scarcity and increasing demand send fuel prices back upwards. No one wants to send a second exploration team to the same site a few years after the first because the original, raw data-set has been lost. So it might surprise you to learn that many operators are somewhat haphazard in their approach to the management of this information.

And you certainly won’t be surprised to learn that it takes a lot of computing to turn seismic and other data into geological models. The critical question is though: how reliable are these models and can they be further enhanced on the basis of recent learning from other sites? As the exploitation of the fields begins, do these models prove to be consistent with incoming information? This is often hard to determine at present. The oil industry is largely stuck in the age of data silos, in which different sets of geospatial data, maintained on a variety different systems and offline media, tell different stories and cannot easily be compared with one another.

Best spot for a drill
The first step towards building more dependable models, therefore, is data integration. Back in the UK, Teradata and the University of Manchester have recently developed a data model that incorporates geospatial data, production data and more. Storing sub-surface seismic data in this way alongside other data sources provides a single, searchable view of the data that enables geologists, geophysicists, reservoir engineers, drilling engineers, well managers etc., etc., to all work from, search through and analyse the same validated and consistent data; and which liberates them from working instead from multiple, limited and untrustworthy data-sets that divide, rather than unite, these functions. And the 3D visualisations are very pretty (just please don’t ask me to tell you what they mean).

The benefit? The more accurate and detailed these models are, the better you can, e.g.: locate the best spot for a new drill hole, which can help to reduce the number of wells that must be sunk when the field starts to deplete. It also helps the producer to quickly identify new reserves and estimate remaining reserves more precisely – which is always a sensitive issue. The Teradata data model includes both upstream and downstream data, allowing the entire demand chain to benefit from insights driven from detailed data. These insights can drive significant value beyond exploration: fuels trading, supply chain optimisation, and improved returns on the forecourt and retail activities.

Making geospatial data actionable
From a data warehousing perspective, the integration of geospatial data with existing data is a hot topic right now and is the reason that we have introduced extensive, integrated capabilities for the storage and exploitation of geospatial data in the latest release of the Teradata RDBMS (“Teradata 13”). There are many angles: retailers want know how far customers are prepared to drive for a major household purchase like a flat screen TV; insurance companies want to find out about the claim history on properties within a high-risk flood or fire zone, etc., etc.

What unites these disparate applications are the requirements to integrate the geospatial data with existing data and to be able to manipulate the combined data quickly and efficiently. The pretty visualisations are important, but so are being able to use millions, even billions, of geospatial data points in calculations to derive new measures. The potential applications for this new capability are vast, in Russia and beyond.

And so to Cairo. Has anyone seen my boarding pass?

Martin Willcox

The lesson of the Palace of Culture and Science

Patrick Hempton — 8/14/2009

If you have ever doubted that architecture (of the bricks-and-mortar variety) can make a political, as well as an aesthetic statement, then you should come to Warsaw and stand in the long shadow of the Palace of Culture and Science. Built between 1952 and 1955 - ostensibly as a gift from the people of the Soviet Union to the people of Poland - many Poles bitterly resented what they saw instead as a potent symbol of Soviet domination that for good measure destroyed the balance of their only recently restored city.

And it’s easy to see why. At 237m tall, it is still today the eighth tallest building in the EU. 3,500 men laboured to complete it – and 16 were killed in the process. Constructed to an imposing Soviet template, the building continues to dominate the Warsaw skyline, despite the fact that the city has subsequently acquired other skyscrapers of comparable height. Many of the locals refer to it dismissively and an old joke holds that the best views of Warsaw are available from it, because it is the only place in the city from where the Palace itself cannot be seen.

Personally, I rather like it, but then I have never had to live with oppression. Unless you count my wife confiscating my credit card when we first got married.

Happily, times have moved on for all of us. The Poles proved themselves to be remarkably resilient in the face of both Nazism and Communism and can legitimately claim to have played a major part in the ending of the Cold War. And that resilience obviously runs deep, as Warsaw is Europe’s boom capital at the moment. Whilst the entire continent – in good company, it should be said - has gone into recession, Poland is still managing to create some growth. The Polish economy has slowed, but it hasn’t yet contracted like pretty much all the rest. The country’s domestic market seems to be more – that word again! - resilient to the international shocks that have so affected most of the rest of the world. As our recent Q1 results demonstrate, Teradata has been better able to weather the storm than many of our peers and so it is fitting that we start our tour d’ horizon here in Warsaw.

Software architectures, too, are often political, with different groups and factions from within the same organization often taking a very different view of the best approach to addressing the same business requirements.

One source of tension in businesses and IT departments in recent years, for example, has been the evolution of the role of the traditional data warehouse. Many organizations have now accepted the proposition that a well-designed data warehouse, deployed on appropriate technology, can support near real-time “operational reporting” alongside traditional reporting, OLAP and data-mining workloads and that the cost savings and business value that arise from adopting this “active data warehouse” model are real. In other organizations, die-hards refuse even to contemplate this “dual use” of the data warehouse. The die-hards bought in to the original data warehouse vision of a “non-volatile” database fifteen years ago or more and refuse to acknowledge that best practice (and the best technology) have moved on; instead they ring their data warehouses with redundant operational data stores. Which is a pity, because these architectures are more expensive to develop and maintain and because the operational data stores typically lack the detailed historical data that are required to identify the significant events in our businesses from the “noise” of day-to-day activity.

So we are especially pleased to be able to report that our new developer exchange site has been live for a month-or-so now and is already attracting a healthy following. Let me take this opportunity to articulate in a little more detail what we are trying to achieve with Developer Exchange and why I hope it can help with some of these political challenges.

Highly specific challenges
Let’s say a web-programmer in a customer or a partner organization wants to connect a web portal to a Teradata Warehouse in support of a pervasive BI project to push information to hundreds, thousands, even tens of thousands of “front-line” end-users. Our hero may well be from the O-O application development side of the IT shop, with little relational database experience and no useful Teradata experience. It goes without saying that he hasn’t done this before and will need to invest a great deal of time to get to know how to hook the web applications, with which he is familiar, up to Teradata, with which he is not. He may well have to find solutions for problems that are so specific that nobody in his organization – including the Teradata DBA across the hall, who may not know the Java or .NET technologies that our hero is working with - or even in his usual online forums can help him. He will need detailed advice from people who work with the same tools and technology – but where to turn?

Developer Exchange is intended to provide the sort of resources that our erstwhile hero will need. The site offers instructional blog posts and programme code samples. Any registered user can comment on these posts or ask for help in the forums (keen-eyed readers will note that these aren’t available quite yet, but they are coming very soon). In this way, our hero may then get direct access to the Teradata engineer who actually developed the function, tool or interface in question. So think of Developer Exchange as your hotline to our development engineers and professional services consultants.

Collaborative support
It gets better, however. Our idea in unleashing Developer Exchange on the Teradata World is also to network you and your fellow practitioners in end-user organizations, as a means of improving your productivity and the quality of the code that you develop and maintain every day. Third-party partners, academics: all are welcome. We promise faithfully that this will be a genuine forum for the exchange of information, not a sales and marketing tool. That means that discussion, dissent and constructive criticism are welcome; and “light touch” moderation intended only to make sure that everyone plays nicely with one another, not to stifle debate. And absolutely no marketing materials, no selling, no advertising - and definitely no spamming of registered users with marketing email.

Our ultimate goal is no less than the creation of a digital community of technical experts solving problems and sharing ideas and code, so that everyone in the Teradata ecosystem is enabled to build more effective Teradata solutions, more easily. This kind of sharing – “reuse”, in the jargon - of architectural patterns, design, and ultimately code is an incredibly powerful idea. If you doubt this, then note that the joke about the Palace of Culture and Science actually originates from 19th century Paris; substitute “Eiffel Tower” for “Palace of Culture and Science” and you have the original version, coined by Guy de Maupassant, possibly in the restaurant at the base of the tower where he used to take lunch in order to avoid looking at it.

So why not get involved? By sharing hard-won, cutting-edge learning and by demonstrating what can be achieved we should be able to convert the die-hards. Hopefully, the solutions that we build together will be as durable and majestic as the Palace of Culture and Science, and rather less divisive.

Martin Willcox

What Makes a Company Great in a Time of Change?

Rick Loconto — 6/18/2009

As many executives are turning to their staffs to provide insight on how to reduce budgets for the coming year(s) or for recommendations on how to maintain or increase revenues: there is a mandate for making better decisions through better information. Really knkowing your business and also being able analyze various scenarios is very important in this business era.

Most companies are still struggling to bring together their decision-making data to ensure stability in the short-term and position their firms for the longer-term.

Recently there was a brief, but excellent, article by Michael Schmidt titled 3 Secrets of Successful Companies

So what might be the immediate actions and success criteria for corporate executives? There are numerous answers to this challenging question.

First, and foremost, is strategy and leadership. These can be assisted by the inclusion of external information resources, industry studies, consultant’s surveys, with deep internal analysis of customer and services data. This provides a basis for effective planning and setting actions in place for the entire organization. Leaders know use this data to change KPI's and also analyze opportunities.

Second is financial, asset, and risk management. In today’s world of dynamic change it becomes more important (“than ever”) to understand and analyze the financial markets, changing interest rates, requirements for capital investment to achieve goals, and management obligations for accurate reporting to governmental, industry, or stakeholders. Most companies who lead their industries utilize financial systems that allow for closing of their books within days of the end of a period; and in some cases, the ability to close the books nightly to ‘know’ exactly where they stand (internally). But you may say that financial services companies have been doing this for years. Possibly. But recent public disclosures illuminate that many FS organizations that did not have a centralized, comprehensive, INTEGRATED, and accessible (BI) analysis. In fact, many banks have used centralized data warehouses for analysis of customers and did not also use them for RISK Management and asset assessments. These were accomplished by models driven by transactions and also may have NOT included external data to compare and contrast internal data within the same DW/database. In fact, the predictive aspects were possibly inaccurate for the assets they were holding or selling. More on this later.

Third, is marketplace analysis to keep the company in a position of ‘knowing’ about external businesses (e.g. competition). These analyses were left to the market research staff or an outside firm and is accomplished on an ‘as needed basis’. Unfortunately, this should be accomplished in an ongoing manner and achieved during the processes where a firm is analyzing its own customers and sometimes re-segmenting them for marketing or sales purposes. Leading firms have mixed their analysis work on complex data warehouses that provide them more confidence in their knowledge and decisions about their business.

Fourth, is the ability to drive and support ‘above-average’ management decisions and actions and finally, analysis of the processes, decisions, reporting, and even the KPI’s. Leading companies who are utilizing integrated data (in their ENTERPRISE-WIDE Data Warehouses) find their ability to learn, change, and evaluate brings them stronger management capabilities. Learn from the leaders…

Randy Mott, formerly the CIO at both Wal-Mart Stores and Dell made a speech some years ago where he focused on the management problem of not using available data. He basically stated: “When a company aggregates or summarizes their decision-making data, and does not make it available (in detail) for use in analysis and decisions, the company makes decisions based on aggregates or averages. These types of using DW/BI are not leveraging the data or information. Average decisions; create average companies, which in turn do not create competitive advantage.”

Mott’s statement has been proven out in many firms that have not established a strong position. This advice is excellent for all executives to think about.
Leaders use their DW/BI systems with detailed data for long periods of time and across multiple disciplines (i.e. departments or organizations).

Long-term successful firms are leading because they learn, act, decide, and evaluate differently. Additionally, these firms make information available to people who make decisions that affect your customers, suppliers, and stakeholders.

Fifth, but not last, is to focus on the information to knowledge problem in a firm. Many systems handle transactions, some provide statistical or predictive analytics and some provide fast reporting. But few are truly integrated and utilize the cross-business data that is a foundation for success. Executives who allocate resources need to understand the value of this integration, accessibility, scalability, and decrease response time to get to and view critical information which drives the business processes and customer profits.

Smarter executives are acting now. What are the actions they focus on now?

Define and execute a strategy to integrate as much transactional, financial, resource, asset, channel, interaction, sales, marketing, performance, and KPI information. Some leaders call this database or data mart consolidation. This can fund much of what follows in information and decision excellence.
Focus on the processes that could be improved within your business. Invest in multiple projects of “Business Improvement Opportunities” (BIO). Seek consultation on industry-leading best practices and successes in BIO’s.
Develop a new awareness of ‘management reporting’ and evolve into action or analytical decisions at all levels of the company. Make accessible the data necessary to truly support decisions. Not just estimates or summaries.
Educate and motivate management and non-management staff to utilize data to make better decisions. Eliminate the human need for replicating decisions that are either common, repetitive, or very basic. Allow systems and BI analysis to help determine the decision and provide it to staff or the customer.
Eliminate wasteful processes, channel actions, and reporting infra-structures that do not provide any added value to the actual job performance. Many firms produce hundreds or thousands of reports (daily/weekly) that are NOT utilized and rarely help make a critical decision or even a basic decision. Ask all managers ‘what actions to they take, as a result of receiving this report, and how it adds VALUE to the business?’ Transform to online and intranet reporting, analysis, prediction, and operationalizing analytical activities. This, in itself, will save hundreds of thousands or millions of dollars each month/year.
Take this opportunity, to realign business and the processes, to position your firm for the next growth stage. Become efficient, effective, but also innovative. Use the above actions to create a closer and more responsible team. Share data, share customers, share products, share channels, share results. Share positive change to create the competitive advantages.
Communicate your successes, share your experiences, learn and teach those that follow you --- not just what and why it is achieved --- but how to foster positive changes (that will be required far beyond this year).

Throughout the many workshops and executive forums that I facilitate each year, there is an opportunity to interact with many levels of management, in many industries, in many cultures, in many countries. The problems are mostly the same throughout the world. And, there seems to be one critical success factor (CSF) that all executives and their smart managers know. The single most important CSF is: “Contribution to the Business”. What actions are you taking now that will make a big difference in six months or six years ?

On the Road with Stephen Brobst

Patrick Hempton — 8/14/2009

So our little tour d’ horizon is about to begin. We have cancelled this year’s Teradata Technology Factory because travelling restrictions have made it difficult for too many of our usual guests to attend. To make up for this, we have organized the “CTO Road Show”. Stephen Brobst and I will be travelling to 10 cities across Europe and North Africa in not many more days. We are about to begin the tour with our first stop in Warsaw on Wednesday.

The reason for this tour is partly consciousness-raising; progress never stops, not even in these difficult times. If anything, difficult circumstances act as a catalyst for new developments that may shape the next generation of an entire industry. We want to make sure that our customers don’t miss out on new data warehousing trends and to discuss their likely impact on our customer’s businesses. And, of course, we will answer practitioners’ questions about our platform family and database software.

Which trends do we have in mind? The rise and rise of pervasive BI – meaning the low latency acquisition of data to support the optimisation of operational business processes and the resulting opening up of the data warehouse to hundreds, thousands, even tens of thousands of new users; the impending deluge of new data from distributed sensor networks, streaming directly into data warehouses and changing the way businesses do everything and anything, from pipeline monitoring to production management; Web 2.0 and the requirement for the integration of non-traditional data types alongside traditional, record-based content in the warehouse; the forces that are pushing more and more processing into the database; etc., etc., etc. I could go on, but honestly, you should come to one of the events yourself and hear it from Stephen. If he isn’t the smartest guy in the industry, well then I haven’t met the smartest guy in the industry.

So, I will be on the road with Stephen. Walking a mile in his shoes will be interesting. He likes to joke that “sleep is for the weak” and there will be plenty of opportunities to test that particular aphorism, not least on the 18^th June when I am scheduled to arrive into Moscow at 01:20 and will find myself on stage only a few hours later! Conferences make me curious, but actually travelling to our customers’ countries is fascinating. They say hard times are everywhere, although our Q1 results demonstrate that the world is still turning. But how exactly does the current recession affect business – and, more specifically – the business of using information? Over the next fortnight, I will share my thoughts and impressions with you right here – on the road. After Warsaw, we will stop in Moscow, Cairo, Madrid, Brussels, Paris, London, Vienna, Zurich and Frankfurt. If you would like to register for any of these events, please click on the desired city to contact the Teradata marketing team.

And by the way, if you have any specific technical questions and need a detailed answer fast, try the new Teradata Developer Exchange site. It’s definitely worth a look.

If you’d like to “stay tuned”, there is an RSS feed you can subscribe to.

Martin Willcox

Proof Demand Chain Visibility Pays

Darryl McDonald — 7/30/2009

Which Fortune 500 companies have the best managed supply chains? The AMR Research 2009 list of 25 Supply Chain Winners tells the tale. These winners are on what AMR Research calls a "flight to quality," and they see a demand-driven supply chain as the competitive pathway.

We were gratified to note that from the top of the list to the bottom and many points in between, these leaders are Teradata customers, 15 in all. Writes AMR Research, "Their ability to sense and respond to shifting levels of demand is especially critical during a downturn," and they benefit from better visibility into their operations to make smarter decisions.

AMR Research used three financial metrics to rank Fortune 500 list retailers and manufacturers: return on assets, inventory turns, and revenue growth. That last one, impact to bottom line, is especially important as it is what allows these companies to continue to make strategic investments in IT and other parts of their business. Those investments will help them emerge faster and stronger when their demand begins to signal the economic turnaround.