by Keith Ferrell

Ours is an age of information, communication and interaction, yet the tools we most often employ to drive these processes fail to accommodate non-verbal cues such as facial expression and body language. So far.

Working in concert with engineers and researchers at the University of Southern California’s Integrated Media Systems Center (IMSC), Teradata funds and participates in the “E-Motions” Smart Machine project, an initiative aimed at developing ways to incorporate non-verbal communication cues into the customer interaction mix.

The good news is that “non-verbal communication cues” is a high-end description of the array of frowns, smiles, squints, furrowed brows and other informative facial expressions that comprise as much as 50% of a typical customer interaction.

Humans are well aware of these expressions and how to interpret them; they’re present in virtually every face-to-face conversation. We recognize, process and react or respond to non-verbal cues almost immediately and essentially without being aware of doing so.

The challenge is that more and more of our interactions, particularly customer- and information-based transactions, take place at machines, which traditionally have not been equipped to recognize expressions, much less respond to them during the transaction. That’s all about to change.

Tell the ATM how you really feel
Teradata and IMSC’s engineering team, with crucial input from a cross-disciplinary panel including clinical psychologists and experts on various aspects of facial recognition, are seeking to create a “Smart Machine” that not only recognizes emotional cues, but also responds to them in real-time.

The first step has been the creation of a continuously expanding Teradata-housed database consisting of countless facial “captures” representing the wide range of emotional expressions. Approaching those captures from various analytical perspectives, such as head geometry and musculature, full facial expression, changes in specific facial regions such as the mouth or the eyes, and so on, the E-Motions project has created a foundation against which individual emotional expressions can be compared.

As that comparison is made, the system formulates a response. For instance, when a frown is captured by an ATM’s camera, the system tries to transform an initially negative customer experience into a positive one through various mechanical adjustments, text messages or other “problem solvers.”

Dave Schrader, Teradata’s director of e-business marketing and an integral member of the E-Motions project, points out that the CRM implications of a Smart Machine and the underlying technology are enormous.

Consider that ATM transaction during which the customer grows increasingly frustrated. Perhaps the sun is too bright on the ATM screen, or the customer forgot his glasses and can’t read the on-screen font. The frown deepens, the eyes narrow.

“A Smart Machine,” Schrader says, “will capture the customer’s facial expressions by camera, extract the key features and compare them to its database of known facial reactions, allowing it to interpret the customer’s emotions and present the appropriate response.”

That response might take the form of an increased font size, an adjustment of screen illumination to minimize glare, and so on, with the system further calibrating its responses as the customer’s face displays its own reactions to the machine’s efforts.

Additionally—and critically—the overall system will remember the individual customer’s face, storing features and expressions for use during the next, even faster and more convenient transaction. The more frequently the individual customer uses the system, the better the system “knows” that customer and provides individualized service.

Purely theoretical exercise? Science fiction? Or, worse, fantasy? Hardly. Already under-going rigorous laboratory and early field tests, with its “emotional database” continuing to grow, the E-Motions Smart Machine might be ready to “meet” the general public in as little as three years.

The CRM payoff could be huge. Teradata’s parent company, NCR, currently handles 20 billion automated transactions a year, by far the largest of any company, and the number continues to grow. For customers, anything that facilitates and smoothes self-service transactions is likely to increase both the frequency of such transactions and also the range and type of interactions they’re willing to undertake with a machine.

For Teradata clients, the benefits are equally appealing. Initially, the payoff comes in the form of a reduced staff size; the more transactions that can be converted to self-service at a high level of customer satisfaction, the fewer costly human customer service employees required.

But longer-term yields might be even greater. Because the E-Motions project retains individual customer information, that information can be used to enhance the transaction experience with appropriate marketing information and offers.

“A customer’s positive or negative emotional reaction to a marketing offer included in a self-service transaction tells us a lot about the effectiveness of the offer itself, as well as the customer’s level of interest in a particular product,” Schrader says. That information, in turn, provides the marketer with tools to further focus and fine-tune the marketing effort. “The transaction becomes more of an actual dialogue,” he notes.

In short, customer interaction is poised on the brink of becoming customer-tailored interaction.

Coming to a kiosk near you
Teradata and the IMSC team remain attentive to privacy issues, fully aware that assuaging those concerns is critical to effective marketing of any information-collection initiative, but perhaps even more critical when dealing with something so ultimately private as our emotions.

“The key point to understand,” Schrader says, “is that the goal is not to invade privacy but to provide better service to customers.” Field testing will help develop appropriate strategies for communicating that message to the public, as well as gathering a less quantifiable but no less vital sense of how the public feels about the technology.

While Teradata’s primary current focus is directed at collecting and analyzing expressions common to financial transactions, the company perceives large opportunities for the system in other self-service venues, including self-training and education as well as free-standing marketing and information kiosks. The effectiveness of the technology might make “Smart Machines” ubiquitous, with initial deployment on NCR ATMs and kiosks, and future migration to other vendors’ front-ends and, ultimately, camera-equipped PCs.

Research also continues toward addressing global questions; the facial geometry of emotion differs in both slight and significant ways in different cultures. As these and other lingering E-Motions questions and challenges are, well, faced and overcome, the likelihood increases that we’ll be seeing more smiles and fewer frowns at kiosks everywhere. T

Keith Ferrell, former editor of OMNI, has written, spoken and consulted extensively on supply chain and customer resource management, Web-enabled corporations and other enterprise aspects of the information revolution.

PHOTO BY BRUCE AYRES/TONY STONE/GETTYONE IMAGES