WEAKNESSES IN EACH TECHNIQUE
The strengths of EVM and RM have been well described elsewhere, as their proponents seek to encourage wider
uptake and use. Each technique however has at least one key weakness which presents a significant danger to
those relying on the output to support strategic or tactical decision-making.
For EVM, one of the main perceived weaknesses (at least by non-experts) is its reliance on a key assumption,
that future performance can be predicted based on past performance. Calculated performance measures (CPI,
SPI, CV, SV etc) are used to predict forwards and estimate cost at completion or overall duration. Unfortunately
there is no guarantee that the basic EVM assumption will be true, and it is likely that the future will deviate from
that predicted by simply extrapolating from past performance. Einstein defined “insanity” as “doing the same
thing over and over again and expecting different results.” Indeed the management element of EVM is designed
to ensure that the future predicted by calculated performance measures does not materialise, since it encourages
the taking of corrective action in response to the analysis results. In addition to being affected by the actions
deliberately taken by management, the remaining elements of the project are also subject to risk, both positive
opportunity and negative threat, introducing variation and ambiguity into future performance.
The strength of EVM lies in its rigorous examination of what has already occurred on the project, using
quantitative metrics to evaluate project past performance. It goes on however to predict future performance by
extrapolating from the past. But it is not possible to drive a car by only looking in the rear-view mirror. A
forward view is also required, and this is what RM offers.
While project planning looks at the next steps which lie immediately ahead, RM has a horizon further into the
future. It acts as a forward-looking radar, scanning the uncertain and unclear future to identify potential dangers
to be avoided, as well as seeking possible additional benefits to be captured. However this undoubted strength of
© 2004 David Hillson/Risk Doctor Limited
Page -2-Originally published as a part of PMI 2004 Global Congress Proceedings – Anaheim, California, USA
being resolutely and exclusively future-focused is also one of the key weaknesses in RM. Anything which
occurred in the past is of little or no interest to the risk process, since there is no uncertainty associated with past
events. RM starts with today’s status quo and looks ahead. How the project reached its current position is not
relevant to the risk process, unless one is seeking to learn lessons to assist RM on future projects. As a result
RM as commonly implemented often lacks a meaningful context within which to interpret identified risks, since
it has no means of capturing past performance and feeding this into the decision-making process.
If EVM is weakened by assuming that future performance can be predicted from past performance, and if RM is
weakened by looking only forwards with no real awareness of the past, a useful synergy might be obtained if a
combined EVM-RM approach were able to address these weaknesses. Combining a rear-view mirror with a
forward-looking radar would use the strengths of complementary approaches to compensate for the weaknesses
inherent in using each alone. Consequently it is possible to produce significant benefits by using RM to provide
the forward view required by EVM, and by using EVM to provide the context required for RM.
SYNERGIES FROM A COMBINED APPROACH
Given the common aims of EVM and RM to examine and expose drivers of project performance in order to
focus management attention on achievement of objectives, and given their differing perspectives towards the
past and the future, a number of areas of possible synergy exist between the two techniques. These are outlined
in the following sections. The steps required to implement these synergies are summarised in Exhibit 1.
Creating the baseline spend plan
The foundation for EVM is the baseline plan of expected spend over time, creating the profile of “Budgeted
Cost of Work Scheduled” (BCWS) or “Planned Value” (PV) against which project performance is measured.
This baseline is derived from a costed and resourced project plan, including fixed and variable costs arising
from financial and human resources. The BCWS profile is typically presented as a cumulative curve, or S-curve.
Good practice EVM recommends that allowance should be made in the baseline BCWS to account for
uncertainty and risk, and the integrated baseline review (IBR) process should verify the robustness and
suitability of this allowance. However in reality many baselines are created without proper consideration of risk,
and simply include a “contingency element” for unplanned work. Indeed contingency is often hidden in the
baseline to avoid its removal by management prior to project start. And this contingency is usually unrelated to
defined risks, but reflects an intuitive assessment of what might be required “just in case”.
The baseline BCWS exists as the benchmark against which project performance will be measured. However one
of the first things a project manager learns is that reality will never precisely match the project plan. As soon as
work starts, there are variations in productivity, resource and information availability, delivery dates, material
costs, scope etc. This is why a rigorous change control process is vital to successful project management.
Although not all changes can be foreseen before the project starts, it is possible to assess the degree of
uncertainty in a project plan, in both time and cost dimensions. This is the domain of RM. One of the first
contributions that RM can make to EVM is to make explicit the consideration of uncertainty and risk when
constructing the baseline BCWS.
By undertaking a full risk assessment of the project plan before the project starts, addressing uncertainties in
both time and cost, it is possible to evaluate the degree of risk in the baseline project plan. Quantitative risk
analysis techniques are particularly useful for this, especially the use of Monte Carlo simulation on integrated
models which include both time and cost uncertainty. These risk models take account of variability in planned
values, also called “estimating uncertainty” (for example by replacing planned single-point estimates of duration
or cost with three-point estimates or other distribution types), and they should also model the effect of discrete
risks to reflect their assessed probability of occurrence and the subsequent impact on project time and/or cost
(using stochastic branching constructs, both probabilistic and conditional). Both threats and opportunities should
be addressed in the risk model, representing the possibility of exceeding or failing to meet the project plan. The
risk model should also take account of planned responses to risks, developed during the risk process. These must
also be reflected in the expected spend profile for the project.
The results of the risk analysis allow the best case project outcome to be determined, representing the cheapest
and quickest way to reach project completion. Similarly a worst case profile can be produced, with highest cost
and longest duration. All other possible outcomes are also calculated, allowing the “expected outcome” within
this range to be identified. These can be shown as a set of three related S-curves, as in Exhibit 2, which take
account of both estimating uncertainty (variability in planned events) and discrete risks (both positive
opportunities and negative threats). The ellipse at the end of the curves represents all possible calculated project
outcomes (90% confidence limit), with the top-right value showing worst-case (highest cost, longest schedule),
© 2004 David Hillson/Risk Doctor Limited
Page -3-Originally published as a part of PMI 2004 Global Congress Proceedings – Anaheim, California, USA
the bottom-left giving best-case (cheapest and quickest), and the centre of gravity of the ellipse being at the
expected outcome of project cost and duration. This ellipse is known by risk practitioners as the “eyeball plot”
(or the “football plot” in US). It may be thought to correspond with the “Box of uncertainty” described by some
EVM practitioners referring to the area bounded by extrapolation from actual cost (ACWP) and earned value
(BCWP). However the risk ellipse is derived from calculations based on defined risks, rather than merely
extrapolating from past performance, so it is likely to be a more accurate representation of the range of possible
future project outcomes.
The existence of this set of possible project outcomes raises the question of where the baseline spend profile for
EVM should be set. The recommendation from a combined approach to EVM and RM is to use the expected
value cumulative profile from a quantitative time-cost risk analysis as the baseline for BCWS. In other words,
the central S-curve in Exhibit 2 would be used as the baseline instead of the original S-curve. This ensures that
the EVM baseline fully reflects the risk associated with the project plan (including an appropriate amount for
contingency which is automatically incorporated in the risk analysis results), rather than measuring performance
against the raw “all-goes-to-plan” plan.
Clearly the risk analysis must be conducted using the same units as those required for EVM, i.e. measuring cost
in monetary value (£, $, € etc), or as resource cost (man hours, days, weeks, months etc). It is also necessary to
use an integrated risk analysis model which can simultaneously vary time and cost, including the “cost of time”
(noting that some popular risk analysis tools do not support integrated time-cost risk modelling). Finally the
issue of dependency and correlation in the risk model must be carefully considered to ensure that results are
realistic and feasible.
Predicting future outcomes
Both EVM and RM attempt to predict the future outcome of the project, based on information currently known
about the project. For EVM this is achieved using calculated performance indices, with a range of formulae in
use for calculating Estimate At Completion (EAC). Most of these formulae start with the Actual Cost of Work
Performed to date (ACWP, or Actual Cost AC), and add the remaining budget adjusted to take account of
performance to date (usually using the Cost Performance Index CPI, or using a combined Performance
Efficiency Factor based on both CPI and SPI). These calculations of the Estimate To Complete (ETC) are used
to extrapolate the ACWP plot for the remainder of the project to estimate where the project might finally end
(EAC). However calculating EAC in this way does not take explicit account of the effect of future risks on
project outcome. One simple way to do this is by adding an amount into the EAC calculation to account for riskweighted
contingency or management reserve.
RM predicts a range of possible futures by analysing the combined effect of known risks and unknown
uncertainty on the remainder of the project. When an integrated time-cost risk model is used, the result is a set
of S-curves similar to Exhibit 2, but covering the uncompleted portion of the project. In the same way that the
initial spend baseline should be determined using both risk and earned value data, the remaining element of the
project should also be estimated using both sets of information.
It is also possible to use risk analysis results to show the effect of specific risks (threats or opportunities) on
project performance as measured by earned value. Since the risk analysis includes both estimating uncertainty
and discrete risks, the model can be used to perform “what-if” scenario analysis showing the effect of
addressing particular risks. For example, if a key threat is modelled using a probabilistic branch, a “what-if”
analysis can set the probability of the threat occurring to zero, simulating the result if that risk is removed.
Similarly the effect of capturing key opportunities can also be shown. The result is a series of cumulative
probability distribution curves (S-curves), sometimes called an “onion-ring diagram”, showing the cumulative
effect of addressing key risks. This allows identification of the most significant risks which need to be addressed
as a priority. If the same technique is applied to the planned spend profile (BCWS or PV), the risk analysis will
reveal which risks have the greatest influence over earned value and project performance, allowing management
attention to be focused appropriately.
Similarly, the risk model will show the effect of outstanding risks and planned responses on the remainder of the
project, reflected in the expected result from the quantitative risk analysis, and this information must also be
taken into account when determining the expected spend profile (BCWS or PV) from time-now to project
completion. The standard approach to EVM allows for draw-down of contingency (or management reserves)
into the baseline if significant changes arise as a result of scope change or risk occurrence. The results from the
quantitative risk analysis indicate how much contingency should be reallocated into the baseline to cover the
expected level of risk in the remaining portion of the project.
© 2004 David Hillson/Risk Doctor Limited
Page -4-Originally published as a part of PMI 2004 Global Congress Proceedings – Anaheim, California, USA
Evaluating risk process effectiveness
A risk can be defined as “any uncertainty that, if it occurs, would have a positive or negative effect on
achievement of one or more project objectives”. RM aims to address this uncertainty proactively in order to
ensure that project objectives are achieved, including completing on time and within budget. As a result, if RM
is fully effective, actual project performance should closely match the plan.
Since EVM performance indices (CPI, SPI) measure deviation from plan, they can be used to indicate whether
the risk process is being effective in addressing uncertainty and controlling its effects on project performance.
•
If CPI and/or SPI are below 1.0 indicating that project performance is falling short of the plan, then oneof the most likely underlying causes is that the risk process is failing to keep the project on course. An
ineffective risk process would fail to avoid adverse risks (threats) proactively, and when threats
materialise into problems the project incurs delay and/or additional cost. Either the risk process is not
identifying the threats, or it is not preventing them from occurring. In this situation, management
attention should be directed to the risk process, to review its effectiveness and consider whether
additional resources are required, or whether different techniques should be used.
•
Conversely, if CPI and/or SPI are above 1.0 indicating that project performance is ahead of plan, therisk process should be focused on exploiting the opportunities created by this situation. Best-practice
RM addresses both threats and opportunities, seeking to minimise threats and maximise opportunities.
When EVM indicates that opportunities exist, the risk process should explore options to capture them
and create additional benefits for the project.
•
It should also be noted that if CPI and/or SPI far exceed 1.0, this may indicate other problems in theproject and may not simply be due to the existence of opportunities. Typically, if actual performance is
much greater than expected or planned, this could indicate poor planning or incorrect scoping when
setting up the initial baseline plan. If this highly anomalous behaviour continues, a baseline re-planning
effort should be considered, which of course will involve the need for further risk management.
•
Similarly if CPI and/or SPI are well below 1.0, this may not simply be due to the impact of unmanagedthreats, but may indicate problems with the baseline plan or scope.
Exhibit 3 illustrates the relationship between the values of EVM indices (CPI and/or SPI) and RM process
effectiveness.
The key to using EVM indices as indicators of RM effectiveness is to determine appropriate thresholds where
action is required to refocus the risk process. Clearly some variation of EVM indices is to be expected as the
project unfolds, and it would not be wise to modify the risk process in response to every small change in CPI
and/or SPI. However if a trend develops and crosses the thresholds of “common variance”, action should be
considered. Exhibit 4 illustrates this, with the thresholds of “common variance” for CPI and/or SPI set at
≥ 0.9and
≤ 1.25. A further “warning threshold” is set at 0.75, suggesting that an adverse trend is developing andpreparatory steps should be taken.
The thresholds of 0.75, 0.9 and 1.25 used in Exhibit 4 are illustrative only, and organisations may be able to
determine more appropriate threshold values by reviewing historical trend data for CPI and SPI, and identifying
the limits of “common variance” for their projects.
Plotting the trend of CPI and SPI over time against such thresholds also gives useful information on the type of
risk exposure faced by the project at any given point. For example Exhibit 4 indicates that the project schedule
is under pressure (SPI trend is consistently below 1.0), suggesting that the risk process should focus on
addressing sources of time risk. The exhibit also suggests that cost savings are possible which might create
opportunities that can be exploited, and the risk process might be able to maximise these. These recommended
action types are illustrated in Exhibit 5, corresponding to the following four situations :
1. Both CPI and SPI high (top-right quadrant), creating opportunities to be captured
2. Both CPI and SPI low (bottom-left quadrant), requiring aggressive action to address threats
3. High SPI but low CPI (top-left quadrant), requiring focused attention to cost risk, with the possibility
of spending additional time to address this
4. High CPI but low SPI (bottom-right quadrant), where attention should be paid to addressing schedule
risk, and cost trade-offs can be considered
© 2004 David Hillson/Risk Doctor Limited
Page -5-Originally published as a part of PMI 2004 Global Congress Proceedings – Anaheim, California, USA
Exhibit 5 also suggests that if either CPI or SPI (or both) remain abnormally high or low, the baseline plan
should be re-examined to determine whether the initial scope was correct or whether underlying planning
assumptions were unfounded.
It is important to note that these action types should be viewed only as first options, since other considerations
may lead to different actions. For example in projects with high schedule-constraints (e.g. product launch, event
management etc), the trade-off between time and cost may be prioritised differently than in cost-constrained
projects.
CONCLUSION
Both Earned Value Management (EVM) and Risk Management (RM) seek to improve decision-making by
providing a rational framework based on project performance. EVM examines past performance against clearlydefined
quantitative metrics, and uses these to predict the future outcome for the project. RM looks ahead to
identify and assess uncertainties with the potential to affect project performance either positively or negatively,
and develops responses to address each risk proactively.
Both techniques share a focus on project performance, and have the same purpose of developing effective
actions to correct unwelcome trends in order to maximise the likelihood of achieving project objectives. One
(EVM) does this by looking back at past performance as an indicator of likely future performance. The other
(RM) looks ahead at possible influences on future project outcomes.
These two approaches are not in conflict or mutually exclusive. Indeed their commonalities imply a powerful
synergy, which is available through combining the complementary strengths of each technique and using
insights from one to inform the application of the other (as summarised in Exhibit 1). The practical suggestions
outlined in this paper indicate that if they are used together, EVM and RM provide a potent framework for
managing change on a project, based on a realistic assessment of both past performance and future uncertainty.
1. Creating the baseline spend plan (BCWS/PV)
a. Develop costed WBS to describe scope of work, without hidden contingency
b. Produce fully costed and resourced project schedule
c. Assess estimating uncertainty associated with initial time/cost estimates
d. Perform risk identification, risk assessment and response development
e. Quantify time and cost risk exposure for each risk, taking account of the effect of agreed responses
f. Create integrated time/cost risk model from project schedule, reflecting both estimating uncertainty (via 3-point
estimates) and discrete risks (via stochastic branches)
g. Perform Monte Carlo simulation on integrated risk model to generate “eyeball plot”
h. Select risk-based profile as baseline spend profile (BCWS/PV); it is most common to use the “expected values”,
although some other confidence level may be selected (say 80%)
2. Predicting future outcomes (EAC)
a. Record project progress and actual cost spent to date (ACWP), and calculate earned value (BCWP)
b. Review initial time/cost estimates for activities not completed, to identify changes, including revised estimating
uncertainty
c. Update risk identification, assessment and quantification, to identify new risks and reassess existing risks
d. Update integrated time/cost risk model with revised values for estimating uncertainty and discrete risks, taking
account of progress to date and agreed risk responses
e. Repeat Monte Carlo simulation for remaining portion of project to generate updated “eyeball plot”
f. Select risk-based calculation as estimate of final project duration and cost (EAC), using either “expected values”,
or some other confidence level (say 80%)
g. Use risk-based profile as updated expected spend from time-now to project completion
3. Evaluating risk management process effectiveness
a. Determine threshold values for CPI and SPI to trigger corrective action in risk process (or use default values of
0.75, 0.90 and 1.25)
b. Calculate earned value performance indices (CPI and SPI), plot trends and compare with thresholds
c. Consider modifications to risk process if CPI and/or SPI cross thresholds, enhancing the process to tackle
opportunities more effectively if CPI and/or SPI are high, or refocusing the process on threat reduction if they are
low
d. Take appropriate action either to exploit opportunities (high CPI/SPI), address threats (low CPI/SPI), spend
contingency to recover time (high CPI/low SPI), or spend time to reduce cost drivers (high SPI/low CPI)
e. Consider need to review initial baseline, project plan or scope if CPI and/or SPI persistently have unusually high
or low values
Exhibit 1 : Summary of steps to integrate EVM and RM
© 2004 David Hillson/Risk Doctor Limited
Page -6-Originally published as a part of PMI 2004 Global Congress Proceedings – Anaheim, California, USA
Exhibit 2 : Risk-based cumulative spend profile
Exhibit 3 : Relationship between EVM indices and RM process effectiveness
© 2004 David Hillson/Risk Doctor Limited
Page -7-Originally published as a part of PMI 2004 Global Congress Proceedings – Anaheim, California, USA



