User relevance is the key to finding meaningful information in scorecards and dashboards.
by Cindi Howson
Dashboards are a hot topic in the business intelligence (BI) industry because they engage important user constituents who are sometimes blasé
about BI: managers and executives. Dashboards offer all of the dazzle to turn mundane metrics into visually engaging, informative displays
that help improve business performance. That's the vision. In reality, achieving that result requires more than fancy software.
The concept of dashboards is not new. Early Executive Information Systems (EISs) of the late 1980s tried to deliver similar capabilities.
EISs were often custom-coded, inflexible applications based on quarterly data. New dashboards are more often user-built, flexible and
sometimes updated in real time. Alerting agents will provide on-screen messages within the dashboard when a metric performs below expectations.
Dashboards also increasingly leverage advanced visualization capabilities that facilitate greater insights and convey more information in less space.
Executives and managers are not the only dashboard users, although they often are the sponsors of initial efforts to deploy dashboard
applications. Front-line workers can also benefit from dashboards that are more operational. The differences lie in the level of detail, the
frequency at which information is updated and the types of metrics that are displayed. An executive may have financial, customer, people and
operational metrics aggregated by month, whereas call center agents may see only metrics related to their individual sales, average call
center volume and promotional product lists, all updated hourly.
Making metrics and dashboards relevant
For metrics and dashboards to be successful, they must be pertinent. If the dashboard does not display the information that most affects the
business or an individual user, it won't be used. While this sounds obvious, it is a characteristic that is hard to achieve. Far too often,
potential users say most of the BI tools and applications delivered are irrelevant to them.
In the process of building dashboards, IT often asks users what they want. This poses a problem in that users rarely know what they want until
they see it. Further, they may not even know what is possible. In this regard, if the business users and IT can build dashboards more
collaboratively, the end design will be more germane. Agile development techniques are helpful in achieving greater impact. Using another
approach, some companies will push as much of the dashboard development directly to the business units to improve relevance.
As an example, call center agents for a major online retailer were frustrated with inadequate information access. Agent turnover was high, and
on exit interviews, agents complained that they were compensated based on metrics beyond their control. Agents were paid commissions on several
performance measures, but these measures were available only via a piece of paper posted on the wall the next day—too late, too aggregated,
too inaccessible to be actionable.
As the company began designing its first BI application, it studied what motivated call center agents and what information could help them do
their jobs better. The BI team worked side by side with the agents to the extent that the BI experts could even handle an incoming call.
In the initial prototype, the dashboard showed agents their daily performance. Call center managers thought this would be a big win for the
agents. In debating the dynamics of the call center, senior executives noted a degree of healthy competition among them.
Executives wanted to tap into this competitive nature to drive better performance and thought that showing agents what percentile they were
performing in would create a kind of horse race among the agents. Finally, increasing the dashboard update to refresh every 15 minutes would
allow agents to take action that same day. The company saw an immediate lift in sales the same week the call center dashboard was implemented.
Relevance, then, includes not only what information to display and in what context, but also how timely the data must be. For call center
operators, it needs to be in near real time. For others, daily and even weekly updates may be adequate. In the example above, BI experts
didn't have a "build it and they will come" mentality, nor did they build what was asked for; instead, they studied the activities of these
potential users and delivered a product that would benefit the individuals. This development process can be used to increase a dashboard's
importance.
Personalizing the dashboard, which goes beyond security and involves tailoring the software interface, including menus and capabilities, has
a role in relevance. Another important aspect of personalization involves ensuring each individual sees only what's pertinent. This doesn't
mean IT should build a personal dashboard for each individual. Instead, the dashboard software should allow customized displays based on roles
and individual user logins.
Relevance also demands agility. Business requirements change constantly, so if the dashboard cannot be modified to accommodate a changing
business environment, it will see only modest use. For example, Continental Airlines, a company profiled in my book "Successful Business
Intelligence: Secrets to Making BI a Killer App," added a warning indicator to its flight dashboard that shows when any plane has been on the
runway for more than three hours. This indicator was added last winter, after passengers of other airlines were left parked on the runway for
up to 10 hours. Because the business maintains its own dashboard, the indicator could be added quickly, without going through a lengthy IT
prioritization, justification and documentation process.
From insight to action
As a technology, dashboards and the metrics presented by them provide businesspeople a way of monitoring the state of the business at a
glance, but the ultimate value lies in transforming insight gleaned from a dashboard to action. For example, when a vehicle's gauge warns that
you are low on fuel, the value in this information is when you refill the tank. When do you act? Do you wait as long as possible, until a
warning light starts flashing? Or until you are stranded on the roadside?
It would seem that taking action when an indicator is not performing adequately would be a natural response, but often it's not. Part of the
challenge is technology-related, but a larger part is corporate culture and politics. A warning or a negative indicator in a dashboard begins
a complex series of additional investigations and decision making.
One of the biggest technical challenges with many dashboards is when they stop with the warning only. If dashboard users cannot investigate
the problem further, dashboards experience lackluster success. Managers have to call experts to perform further analysis or run queries in
isolation from the dashboard discovery. A pretty display deployed in isolation from the rest of the BI environment is an ineffective approach.
Culturally, though, an equally big challenge results from over-analysis (and inaction). This may be "safer" than action, particularly if in
hindsight the action produced undesirable outcomes. If a sales dashboard indicates a product is not performing well, taking action to increase
promotions (spending more money) or recommending the product be discontinued can trigger a series of political conflicts. Spending more marketing
dollars on a product that can't sell is a "bad" course of action that may be criticized in hindsight. The campaign manager risks being fired.
In some companies, it's safer to maintain the status quo and investigate other causes for poor product sales than to take action. The company
culture must support actions determined by fact-based decisions.
The degree to which company culture supports fact-based decision making and subsequent action can be difficult to assess and transform. A
starting point is to understand how incentives prompt action once information is received. In the case of the aforementioned call center,
agents earn a commission according to sales, so the dashboard is a meaningful tool to help them improve their commissions. Financial
compensation, however, is only one form of incentive. Others include:
|
A desire to win or to outperform colleagues
|
|
A desire to do a good job
|
|
Wanting to remove or avoid a frustration, whether a co-worker conflict or complaining customers
|
For airline pilots and flight operation managers, an alert for excessive time on the runway must be assessed with myriad conditions and
sometimes conflicting incentives. The best course of action is influenced by the number of planes in the queue ahead, the cause of the delay,
the amount of time the crew has already been flying, subsequent connections waiting for that particular plane, etc. Maintaining passenger
satisfaction is a clear incentive, but passengers often have conflicting demands, with some travelers willing to wait longer to ensure they
ultimately get to their destination versus those passengers who want to disembark sooner. Financial incentives for on-time performance will
also influence the pilot's actions. The crew's desire to return home is yet another incentive that comes into play. A dashboard alert, then,
is one very small piece in whether a positive action is taken. Incentives need to be aligned in a way that allows for positive action based on
new information.
BI experts can design dashboards to facilitate insight. They also can ensure those dashboards are deployed in a way that allows for
investigation. Acting on that insight is what ultimately leads to improved business performance. Action comes down to the individual dashboard
user. Company culture has to support fact-based decision making, and executive leadership must consider how incentives, politics and
organization structures enable or discourage acting on new insights.
More than eye candy
Dashboards have gotten bolder and more engaging in recent years. Using compelling visualizations, dashboards empower decision makers of all
levels in an organization by supplying them with the most important information at a glance. To ensure dashboards are more than just eye
candy, the information presented must be relevant to the users, whether they are managers or front-line workers. Finding this relevance means
changing the requirements definition process from "What do you need?" to a careful study of what drives the business and motivates the
individual. Transforming this information into action is the biggest hurdle that demands companies go beyond technology to address barriers in
culture, politics and incentives. T
Cindi Howson is the founder of BIScorecard, a Web site for BI product reviews, and the author of the recently released book "Successful
Business Intelligence: Secrets to Making BI a Killer App."
Teradata Magazine-June 2008
| Defining dashboards |
|
Business intelligence (BI) dashboards are similar to car dashboards—they provide multiple indicators or reports in
a highly visual way. A dashboard may comprise one or all of the following visualizations:
| > |
A map that color-codes sales performance
|
| > |
A trend line that tracks stockouts
|
| > |
A cross tab of top-selling products
|
| > |
A key performance indicator (KPI) with an arrow to show whether sales are occurring according to plan
|
Dashboard software has recently become more sophisticated and powerful:
| > |
Geographic maps are now live so hovering a cursor over a particular region displays additional
information, allowing users to drill down for details.
|
| > |
Trend lines that used to take up half a screen now may start as spark lines, condensing 10 years of
data into a space the size of a word.
|
| > |
Interactive sliders allow dashboard users to zoom into particular time periods.
|
| > |
Bullet graphs increasingly replace traditional gauge dials to more effectively display actual and
target information.
|
Several specialty vendors provide only dashboard software, and many BI vendors offer dashboard capabilities within their
suites. The MicroStrategy dashboard in the figure uses spark lines to show the trend for open cases and bullet graphs for
closed cases versus target, packing more information in a smaller display.
Ideally, users want to assemble their own dashboards with the information relevant to their jobs. Not all tools
allow this, though, and IT may need to build dashboards in advance. A key characteristic of dashboards is that
they present information from multiple data sources. Exactly how they do this and what constraints exist in the
accessibility and number of data sources vary widely by product.
—C.H.
|
|
| Defining scorecards |
|
Although the terms "dashboards" and "scorecards" are often used interchangeably, they are indeed different. A
major distinction between them is that a scorecard compares a given metric with a forecast or target, whereas a
dashboard presents multiple numbers in various ways. Some dashboards may include metrics and targets with visual
traffic lighting to show the performance of that metric, but not all dashboard tools support this capability.
Strategic scorecards contain metrics from four key areas—people, customers, financial and operations—that drive
business success and include strategy maps that show how the metrics relate to one another. Such scorecard products
are often certified by the Balanced Scorecard Collaborative. Vendors that offer performance management software
often have strategic scorecard capabilities, and numerous specialty scorecard products are available. The SAP
scorecard in the figure shows with a red traffic light that the KPI of increasing revenue is currently below
target but that it is trending upward.
Scorecard software also often provides the ability to assign ownership for each metric and to track actions.
Although many powerful scorecard products are available, the biggest challenge in deploying scorecards is in
getting the business to agree on common objectives, drivers, targets and accountability. Configuring the software
requires significantly less effort.
—C.H.
|
|
|