Register | Login

Executive Center

Technology Portfolio Management Series

"Mitigating Risk in Technology Investments through Information Technology Portfolio Management"
An Interview With Mark Jeffery

Clinical Assistant Professor of Technology
Center for Research on Technology and Innovation
Kellogg School of Management at Northwestern University

Synopsis: We all know that IT projects can, and sometimes do, fail. But looking beyond that fact of business life, what are the greatest risks to companies today - both internal and external? Internally, every company faces potential risks in the areas of technology itself, technology projects, and corporate organization. Where are enterprises most vulnerable in these areas? How can enterprises mitigate risk proactively, both via technology and organizational attributes? And when projects do fail, what needs to be in place for the best possible reaction?

This interview is third in a series of interviews focusing on Information Technology Portfolio Management (ITPM) that appear periodically in Teradata.com's Executive Center. To prepare for this interview, Mark talked with CIOs across multiple industries about their experiences with risk and how they mitigate it. He also took valuable ideas from his Kellogg Executive program. Below, Mark not only discusses how ITPM can help businesses mitigate risks, but offers key risk mitigation strategies that can be applied today by companies of any size in any industry.

Mark Jeffery is a clinical assistant professor at the Kellogg School of Management. His research expertise is in technology portfolio management, real options applied to technology projects, and quantifying the business value of information technology initiatives. He has 30 publications in scientific and technology journals, and has developed over 12 case studies that are used in the Kellogg MBA course he teaches on Technology Portfolio and Program Management.

Mark is also the academic director of Kellogg's new executive Technology Management Program: "Driving Strategic Results through IT Portfolio Management,". This three-day, highly interactive program is designed for IT executive management teams (CIO/CTO level and their direct reports), and for business unit executive sponsors of the IT team (including CFOs and SBU heads). The program offers an intense immersion in IT executive management and decision making best practices. To learn more, go to www.kellogg.northwestern.edu/DrivingResults

Question: In general, just what does "risk" mean in today's business environment for IT projects?

A: At the start of any IT project, those responsible for the project - from the business unit sponsors, CIOs, to the project managers - usually start by defining the value of the project. That value could be business benefits such as increased revenue or customer loyalty or more efficient processes, cost reductions, or strategic value such as better understanding of future direction or of customer needs or of competitive opportunities. Risk, in a nutshell, is that if something goes wrong with an IT project, the enterprise may not fully realize the projected value that drove the need for the IT project in the first place.

Once we recognize that risk for an IT project is really the potential for missing out on the full value of the project, we can then start to see where risk comes from. And there are two major components of risk - organizational and technological.

Organizational risk is that the organization, or enterprise, itself will not properly manage the change necessary to maximize the IT project's value. This could mean anything from trying to short-cut processes in order to save time on the project, to senior management not properly positioning and championing the project, to departmental leaders and workers not fully believing in the project.

Technological risk is that the technological tools purchased or developed to bring the project to life will fail in some fashion. This could mean anything from the project taking much longer than planned, the technology not functioning as promised, to costing more than anticipated to maintain or upgrade, to becoming obsolete in our always quickly changing technological world.

Now, what's both fascinating and ironic is that if you ask business or marketing managers which type of risk is greatest, in most cases they will say technological risks. Perhaps that's because it's easier to pinpoint technological risks; organizational risks are more amorphous, more difficult to define. Or perhaps it's because we don't like to admit that sometimes we create our own risks internally by not taking a close look at the organizational risk factors of an IT project from its inception.

However, if you ask technology managers which type of risk poses the greatest threat to the success of IT projects, you'll get a very different view. An excellent piece of research literature that discusses IT project risk is A Framework for Identifying Software Project Risks, by Mark Keil et al.,* for which the authors assembled panels of experienced software project managers from Finland, Hong Kong, and the U.S. and asked them to identify risk factors and then rank them in order of importance. The results were strikingly the same, no matter which part of the world the managers were from. And the top three IT risks which these software project managers overwhelming identified as most important were:

  1. Lack of top management commitment to the project
  2. Failure to gain user commitment
  3. Misunderstanding the requirements

What's more, technological risk was only mentioned once - as "introduction of new technology" - and that was item 9 of 11.

So, from the experience of those who work most closely with IT projects, the greatest risks are organizational issues. It turns out that mitigating technological risks is not the hard part; mitigating organizational risks is the biggest challenge to the success of IT projects.

*Communications of the ACM, November 1998/Vol. 41, No. 11


Question: But how do CIOs tend to view risk today?

A: I recently had the opportunity to ask CIOs from fifteen leading, world-class enterprises that very question. While I can't identify the CIOs or enterprises for confidentiality reasons, I can tell you that risk is a critical issue that CIOs consider whenever they make any technological or management decision. This was true for all the CIOs in my interviews, no matter which industry the CIO represented.

Essentially, the CIO view of risk breaks down into three major concerns:

  1. Organizational: Does the project have the right level of sponsorship, and is the business committed to the business process change necessary to make the project a success?
  2. Project Management: For a large firm and enterprise-wide project - can you manage a complex program across multiple divisions and with external partners?
  3. Technology risk: The focus today is on building platforms, not solutions, and minimizing the amount of new technology introduced in each new project.

Question: Are there specific risk concerns for EDW projects?

A: Another seminal piece of research literature addresses that very question. This recent research, conducted by Dr. Barbara Wixom and Dr. Hugh Watson*, was a detailed study of factors that effect EDW success. Their research found that EDW projects need a high level of management support, senior executive buy-in, and significant levels of user participation-starting at the inception of the project, through its development, and into the final stages of implementation. This research supports the Keil report I just cited, and brings empirical evidence to the fact that these factors are key to mitigating risk in EDW projects and ensuring, as much as can be ensured, their long-lasting success.

*An Empirical Investigation of the Factors Affecting Data Warehousing Success, by Barbara H. Wixom, McIntire School of Commerce, University of Virginia and Hugh J. Watson, Department of MIS, Terry College of Business, University of Georgia. Published in MIS Quarterly, March 2001.


Question: All right then - what are your specific suggestions for how CIOs, CEOs and project managers can achieve optimal risk mitigation in their enterprises' IT projects?

A: First of all, we have to get rid of the perception that we start IT projects and risk happens and there's nothing anyone can really do to mitigate risk. That's simply not true. Risk mitigation is a very real and valuable exercise - and a key part of the Information Technology Portfolio Management (ITPM) process.

Firstly, risk mitigation goes right to the core of ITPM; that is, aligning projects with strategy. If the strategy is in place, and management from the top down believes in that strategy, and IT projects are created only in support of that strategy, then right away, the enterprise has created risk mitigation. Secondly, communication is key to risk mitigation. Upper management must have a dialogue with the IT management team and with business managers from all areas of the company to assess concerns about potential risks. In fact my colleague Joe Norton, of the Norton Solutions Group, who teaches the risk portion of our Kellogg Executive Program (Driving Strategic Results through IT Portfolio Management), created a checklist of factors for identifying potential risk. [Please see checklist at end of interview.] It's simple, really, but discussing new projects and going through this list of risk factors is a great starting point for bringing the discussion of risk into the initial project planning stages. That brings me to another key point. Risk must be discussed at the inception of any IT project; it's not something to start thinking about down the road.

Once potential risks are identified, the next step is to assess those risks by the probability of the risk event actually occurring and by the severity of consequences. That puts all the risks into perspective. We have several risk assessment techniques that Joe Norton and I discuss in the Information Technology Portfolio Management sessions [please see summary of these techniques at the end of the interview], but a simple technique that I favor and would like to point out for this interview is one that I call Quadrant Risk Mapping. Literally, I recommend that executives take the list of risks they've gathered through their initial dialogue using the key risk factor checklist and identify which quadrant each risk belongs to:

Obviously, those potential risks in the upper right quadrant - with a high probability of occurring and a high impact on the value of the project-must receive the greatest attention for developing a risk management strategy. What that strategy should be will vary with the risk. If the risk is that not everyone within the organization is aligned with the goals of the project, then perhaps the risk mitigation strategy is to educate from the top down about the value of the project across the enterprise. If the risk is that the project will not get done in a timely fashion, then perhaps the risk mitigation strategy is to fund additional expert resources that can help speed along the project.


Question: The truth is, though, that no matter how many risk mitigation strategies are created upfront in an IT project, sometimes things do go wrong. What then?

A: That's very true in business as in life. We can't always predict, for example, the surprise snowstorm. But we can have shovels and salt handy just in case. Likewise, in business we can't always predict changes in the world economy, or in a company whose software we're buying, or in legal requirements, and so forth. It's important to realize that events can and will happen that are out of our control. But that's why basic risk mitigation strategies that are important to every IT project must be in place - for example, having a contingency fund, having resources cross trained in tasks on the critical paths of the project, having a back up system in place. It then becomes a case of if we do have to rely upon these contingencies, then we can; if we don't have to, then great!


Question: Any final thoughts about IT risk mitigation?

A: People tend to be very optimistic at the start of any IT project - and that's good, because that positive energy is needed to give the project momentum. But risk events do happen. Things do go wrong. So, I can't emphasize enough the incredible importance of being proactive upfront about identifying risks and creating risk mitigation strategies. Literally just spending one percent of the total project's effort up front on thinking about risks can avert so many problems later in the life of the project. Look at the risk factors, and then assess the best case of the outcome of the IT project as well as the worst case. Engage all levels of management across the enterprise in this process at the inception of the project. Then, at an executive level, decide based on this information what is acceptable risk. This process will empower the enterprise to make a much more informed decision about the project and how it should be managed to return the greatest value.

Key Factors For Identifying The Presence of Risk:
The project sponsor and/or the project manager does not recognize that every project is a risk exercise.
This project is very different from the last one.
There is a feeling of uneasiness.
When the project is in its earliest phase, project risk and uncertainty are highest.
The project scope, objectives and deliverables are not clearly defined or understood.
A large number of alternatives are perceived as possible.
Some or all technical data is lacking.
The technical process and design are not mature.
Standards for performance are unrealistic or are absent.
Costs, schedules and performance are not expressed in ranges.
The future timing of activities and events are vague.
Design lacks production engineering input.
Prototype of a key element is missing.
There is a higher than usual R&D component.
Some or all environmental permits are lacking.
Other similar projects have been delayed or cancelled.
Wide variations in bids are received.
Some key subsystems and/or materials are sole source.
No appropriate contingency plans have been developed.
The project team relies entirely on the contingency allowance.
Someone starts "hedging his or her bets"!

Risk Analysis Techniques:
Brainstorming:
Simple & Effective risk ID technique.
Sensitivity Analysis:
Simplest form of Risk Analysis - change one variable at a time.
Probability Analysis:
Overcomes limitations of Sensitivity Analysis.
Specify a probability distribution for each variable.
Delphi Method:
Derive a consensus using a panel of experts.
Useful in arriving at probability assessments relating to future events.
Scenario based.
Monte Carlo:
Powerful yet simple method of incorporating probabilistic data.
Random change of range variables.
Automate 1,000 simulation runs & plot frequency of outcomes.
Decision Tree:
Provides graphical means to bring information together.
Forces consideration of the probability of each outcome.
Utility Theory:
Takes into account "attitudes" about risk.
Endeavors to formalize management's attitude towards risk.

Viewed as rather theoretical.

Decision Theory:
Points to best possible course whether or not the forecasts are accurate.

Read other articles in this series >


Company Newsroom Site Help Site Map Privacy/Legal Contact Us