Be careful what you govern for!

Governance is an interesting and subtle process which is not helped by confusing governance with management or organisational maturity. A recent discussion in PM World Journal on the subject of governance and management highlighted an interesting issue that we have touched on in the past.

The Romans were undoubtedly good builders (see: The Roman Approach to Contract Risk Management). They also had effective governance and management processes, when a contractor was engaged to build something, they had a clear vision of what they wanted to accomplish; assigned responsibilities and accountability effectively; and failure had clearly understood, significant consequences.

Roman bridge builders were called pontiff. One of the quality control processes used to ensure the effective construction of bridges and other similar structures was to ensure the pontiffs were the first to cross their newly completed construction with their chariots to demonstrate that their product was safe.

An ancient Roman bridge

This governance focus on safety and sanctions created very strong bridges some of which survive in use to the present day but this governance policy also stymied innovation. Roman architecture and engineering practice did not change significantly in the last 400 years of the empire!

No sensible pontiff would risk his life to test an innovative approach to bridge design or construction when the governance systems he operated under focus on avoiding failure. Or in more general terms; the management response to a governance regime focused on ‘no failure’ backed up by the application of sanctions is to implement rigid processes. The problem is rigid process prevents improvement.

To realise the significance of this consider the technology in use in the 17th century compared to the modern day – the vast majority of the innovations that have resulted in today’s improved living standards are the result of learning from failure (see: How to Suffer Successfully).

But the solution is not that simple, we know that well designed and implemented, processes are definitely advantageous. There is a significant body of research that shows implementing methodologies and processes using CMMI, OPM3, PRINCE2, P3M3 and other similar frameworks has a major impact on improving organisational performance and outcomes.

However, organisational maturity is a similar ‘two edged sword’ to rigid governance and management requirements. We know organisational maturity defined as the use of standardised processes and procedures creates significant benefits in terms of reduced error and increased effectiveness compared to laissez-faire / ad hoc systems with little or no standardisation. But these improvements can evolve to become an innovation-sapping straightjacket.

Too much standardisation creates processes paralysis and a focus on doing the process rather than achieving an outcome. In organisations that that have become fixated on ‘process’, it is common to see more and more process introduced to over come the problem of process paralysis which in turn consume more valuable time and resources until Cohn’s Law is proved: The more time you spend in reporting on what you are doing, the less time you have to do anything. Stability is achieved when you spend all your time doing nothing but reporting on the nothing you are doing.

Avoiding this type of paralysis before a review is forced by a major crisis is a subtle, but critical, governance challenge. The governing body sets the moral and ethical ‘tone’ for the organisation, determines strategy and decides what is important. Executive Management’s role is to implement the governing body’s intentions, which includes determining the organisation’s approach to process and methodology, and middle and lower level management’s role is to implement these directives (for more on this see: Governance Systems & Management Systems). The governance challenge is working out a way to implement efficient systems that also encourage an appropriate degree of innovation and experimentation. The ultimate level in CMMI and OPM3 is ‘continuous improvement’. But improvement means change and change requires research, experimentation and risk taking. As Albert Einstein once said, “If we knew what it was we were doing, it would not be called research, would it?”

To stay with the Roman theme of this post: Finis origine pendet (quoting 1st century AD Roman poet and astronomer Marcus Manilius: The end depends upon the beginning). The challenge of effective governance is to encourage flexibility and innovation where this is appropriate (ie, to encourage the taking of appropriate risks to change and improve the organisation) whilst ensuring due process is followed when this is important. The challenge is knowing when each is appropriate and then disseminating this understanding throughout the organisation.

Organisations that follow the Roman approach to governance and avoid taking any form risk are doomed to fade into oblivion sooner or later.

_______________

Note: According to the usual interpretation, the term pontifex literally means “bridge-builder” (pons + facere). The position of bridge-builder was an important one in Rome, where the major bridges were over the Tiber, the sacred river (and a deity). Only prestigious authorities with sacral functions could be allowed to ‘disturb’ it with mechanical additions.

However, the term was always understood in its symbolic sense as well: the pontifices were the ones who smoothed the ‘bridge’ between gods and men. In ancient Rome, the Pontifex Maximus (Latin, literally: greatest pontiff) was the high priest of the College of Pontiffs (Collegium Pontificum), the most important religious role in the republic. The word pontifex later became a term used for bishops in the early Catholic Church and the Bishop of Rome, the Pope, the highest of bridge-builders sumus pontiff.

What’s the Probability??

The solution to this question is simple but complex….

There is a 1 in 10 chance the ‘Go Live’ date will be delayed by Project 1
There is a 1 in 10 chance the ‘Go Live’ date will be delayed by Project 2
There is a 2 in 10 chance the ‘Go Live’ date will be delayed by Project 3

What is the probability of going live on March 1st?

To understand this problem let’s look at the role of dice:

If role the dice and get a 1 the project is delayed, any other number it is on time or early.
If you role 1 dice, the probability is 1 in 6 it will land on 1 = 0.1666 or 16.66% therefore there is a 100 – 16.66 = 83.34% probability of success.

Similarly, if you roll 2 dice, there are 36 possible combinations, and the possibilities of losing are: 1:1, 1:2, 1:3, 1:4, 1:5, 1:6, 6:1, 5:1, 4:1, 3:1, 2:1. (11 possibilities)

The way this is calculated (in preference to using the graphic) is to take the number of ways a single die will NOT show a 1 when rolled (five) and multiply this by the number of ways the second die will NOT show a 1 when rolled. (Also five.) 5 x 5 = 25. Subtract this from the total number of ways two dice can appear (36) and we have our answer…eleven.
(source: http://www.edcollins.com/backgammon/diceprob.htm)

Therefore the probability of rolling a 1 and being late are 11/36 = 0.3055 or 30.55%, therefore the probability of success is 100 – 30.55 = 69.45% probability of being on time.

If we roll 3 dice we can extend the calculation above as follows:
The number of possible outcomes are 6 x 6 x 6 = 216
The number of ways not to show a 1 are 5 x 5 x 5 = 125

Meaning there are 216 combinations and there are 125 ways of NOT rolling a 1
leaving 216 – 125 = 91 possibilities of rolling a 1
(or you can do it the hard way: 1:1:1, 1:1:2, 1:1:3, etc.)

91/216 = 0.4213 or 42.13% probability of failure therefore there is a
100 – 42.13 = 57.87% probability of success.

So going back to the original problem:

Project 1 has a 1 in 10 chance of causing a delay
Project 2 has a 1 in 10 chance of causing a delay
Project 3 has a 1 in 5 chance of causing a delay

There are 10 x 10 x 5 = 500 possible outcomes and within this 9 x 9 x 4 = 324 ways of not being late. 500 – 324 leaves 176 ways of being late. 176/500 = 0.352 or a 35.2% probability of not making the ‘Go Live’ date.
Or a 100 – 35.2 = 64.8% probability of being on time.

The quicker way to calculate this is simply to multiply the probabilities together:

0.9 x 0.9 x 0.8 = 64.8%

These calculations have been added to our White Paper on Probability.

A Technical question for the risk experts??

Three schedule activities of 10 days duration each need to be complete before their outputs can be integrated.

Activity 1 & 2 both have a 90% probability of achieving the estimated duration of 10 days.

Activity 3 has an 80% probability of achieving the 10 days.

Scenario 1:

The three activities are in parallel with no cross dependencies, what is the probability of the integration activity starting on schedule?

Possible solution #1

There is a 10% probability of the start being delayed by Activity 1 overrunning.
There is a 10% probability of the start being delayed by Activity 2 overrunning.
There is a 20% probability of the start being delayed by Activity 3 overrunning.

Therefore in aggregate there is a 40% probability of the start being delayed meaning there is a 60% probability of the integration activity starting on time.

Possible solution #2

The three activities are in parallel and the start of the integration is dependent on all 3 activities achieving their target duration. The probability of a ‘fair coin toss’ landing on heads 3 times in a row is 0.5 x 0.5 x 0.5 = 0.125  (an independent series)

Therefore the probability of the three activities achieving ‘on time’ completion as opposed to ‘late’ completion should be 0.9 x 0.9 x 0.8 = 0.648 or a 64.8% probability of the integration activity starting on time.

Which of these probabilities are correct?

Scenario #2

The more usual project scheduling situation where activities 1, 2 and 3 are joined ‘Finish-to-Start’ in series (an interdependent series). Is there any way of determining the probability of activity 4 starting on time from the information provided or are range estimates needed to deal with the probability of the activities finishing early as well as late?

There is a correct answer and an explanation – see the next post
(its too long for a comment)

Value is created by embracing risk effectively

The latest briefing from the real ‘Risk Doctor’, Dr David Hillson #75: RESOLVING COBB’S PARADOX? starts with the proposition: When Martin Cobb was CIO for the Secretariat of the Treasury Board of Canada in 1995, he asked a question which has become known as Cobb’s Paradox: “We know why projects fail; we know how to prevent their failure – so why do they still fail?” Speaking at a recent UK conference, the UK Government’s adviser on efficiency Sir Peter Gershon laid down a challenge to the project management profession: “Projects and programmes should be delivered within cost, on time, delivering the anticipated benefits.” Taking up the Gershon Challenge, the UK Association for Project Management (APM) has defined its 2020 Vision as “A world in which all projects succeed.” The briefing then goes on to highlight basic flaw in these ambitions – the uncertainty associated with various types of risk. (Download the briefing from: http://www.risk-doctor.com/briefings)

Whilst agreeing with the concepts in David’s briefing, I don’t feel he has gone far enough! Fundamentally, the only way to achieve the APM objective of a “world in which all projects succeed” is to stop doing projects! We either stop doing projects – no projects – no risks – no failures. Or approximate ‘no risk’ by creating massive time and cost contingencies and taking every other precaution to remove any vestige of uncertainty; the inevitable consequence being to make projects massively time consuming and unnecessarily expensive resulting in massive reductions in the value created by the few projects that can be afforded.

The genesis of Cobb’s Paradox was a workshop focused on avoidable failures caused by the repetition of known errors – essentially management incompetence! No one argues this type of failure should be tolerated although bad management practices mainly at the middle and senior management levels in organisations and poor governance oversight from the organisation’s mean this type of failing is still all too common. (for more on the causes of failure see: Project or Management Failures )

However, assuming good project management practice, good middle and senior management support and good governance oversight, in an organisation focused on maximising the creation of value some level of project failure should be expected, in fact some failure is desirable!

In a well-crafted portfolio with well managed projects, the amount of contingency included within each project should only be sufficient to off-set risks that can be reasonably expected to occur including variability in estimates and known-unknowns that will probably occur. This keeps the cost and duration of the individual projects as low as possible, but, using the Gartner definitions of ‘failure’ guarantees some projects will fail by finishing late or over budget.

Whilst managing unknown-unknowns and low probability risks should remain as part of the normal project risk management processes, contingent allowances for this type of risk should be excluded from the individual projects. Consequently, when this type of risk eventuates, the project will fail. However, the effect of the ‘law of averages’ means the amount of additional contingency needed at the portfolio level to protect the organisation from these ‘expected failures’ is much lower than the aggregate ‘padding’ that would be needed to be added to each individual project to achieve the same probability of success/failure. (For more on this see: Averaging the Power of Portfolios)

Even after all of this there is still a probability of overall failure. If there is a 95% certainty the portfolio will be successful (which is ridiculously high), there is still a 5% probability of failure. Maximum value is likely to be achieved around the 80% probability of success meaning an inevitable 20% probability of failure.

Furthermore, a focus on maximising value also means if you have better project managers or better processes you set tighter objectives to optimise the overall portfolio outcome by accepting the same sensible level of risk. Both sporting and management coaches understand the value of ‘stretch assignments’ – people don’t know how good they are until they are stretched! The only problem with failure in these circumstances is failing to learn and failing to use the learning to improve next time. (For more on this see: How to Suffer Successfully)

The management challenge is firstly to eliminate unnecessary failures by improving the overall management and governance of projects within an organisation. Then rather than setting a totally unachievable and unrealistic objective that is guaranteed to fail, accept that risk is real and use pragmatic risk management that maximises value. As David points out in his briefing: “Projects should exist in a risk-balanced portfolio. The concept of risk efficiency should be built into the way a portfolio of projects is built, with a balance between risk and reward. This will normally include some high-risk/high-reward projects, and it would not be surprising if some of these fail to deliver the expected value.”

Creating the maximum possible value is helped by skilled managers, effective processes and all of the other facets of ‘good project management’ but not if these capabilities are wasted in a forlorn attempt to ‘remove all risk’ and avoid all failure. The skill of managing projects within an organisation’s overall portfolio is accepting sensible risks in proportion to the expected gains and being careful not to ‘bet the farm’ on any one outcome. Then by actively managing the accepted risks the probability of success and value creation are both maximised.

So in summary, failure is not necessary bad, provided you are failing for the ‘right reason’ – and I would suggest getting the balance right is the real art of effective project risk management in portfolios!

Stakeholders and Risk

Probably the biggest single challenge in stakeholder communication is dealing with risk – I have touched on this subject a few times recently because it is so important at all levels of communication.

Projects are by definition uncertain – you are trying to predict a future outcome and as the failure of economic forecasts and doomsday prophets routinely demonstrate (and bookmakers have always known), making predictions is easy; getting the prediction correct is very difficult.

Most future outcomes will become a definite fact; only one horse wins a race, the activity will only take one precise duration to complete. What is uncertain is what we know about the ‘winner’ or the duration in advance of the event. The future once it happens will be a precise set of historical facts, until that point there is always a degree of uncertainty, and this is where the communication challenge starts to get interesting……

The major anomaly is the way people deal with uncertainty. As Douglas Hubbard points out in his book the Failure of Risk Management: “He saw no fundamental irony in his position: Because he believed he did not have enough data to estimate a range, he had to estimate a point”. If someone asks you what a meal costs in your favourite restaurant, do you answer precisely $83.56 or do you say something like “usually between $70 and $100 depending on what you select”? An alternative answer would be ‘around $85’ but this is less useful than the range answer because your friend still needs to understand how much cash to take for the meal and this requires an appreciation of the range of uncertainties.

In social conversations most people are happy to provide useful information with range estimates and uncertainty included to make the conversation helpful to the person needing to plan their actions. In business the tendency is to expect the precisely wrong single value. Your estimate of $83.56 has a 1 in 3000 chance of actually occurring (assuming a uniform distribution of outcomes in a $30 range). The problem of precisely wrong data is discussed in ‘Is what you heard what I meant?’.

The next problem is in understanding how much you can reasonably expect to know about the future.

• Some future outcomes such as the roll of a ‘true dice’ have a defined range (1 to 6) but previous rolls have absolutely no influence on subsequent rolls, any number can occur on any roll.
• Some future outcomes can be understood better if you invest in appropriate research, the uncertainty cannot be removed, but the ‘range’ can be refined.

This ‘know-ability’ interacts with the type of uncertainty. Some future events (risks) simply will or won’t happen (eg, when you drop your china coffee mug onto the floor it will either break or not break – if it’s broken you bin the rubbish, if it’s not broken you wash the mug and in both situations you clean up the mess). Other uncertainties have a range of potential outcomes and the range may be capable of being influenced if you take appropriate measures.

The interaction of these two factors is demonstrated in the chart below, although it is important to recognise there are not absolute values most uncertainties tend towards one option or the other but apart from artificial events such as the roll of a dice, most natural uncertainties occur within the overall continuum.

Putting the two together, to communicate risk effectively to stakeholders (typically clients or senior managers) your first challenge is to allow uncertainty into the discussion – this may require a significant effort if your manager wants the illusion of certainty so he/she can pretend the future is completely controllable and defined. This type of self-delusion is dangerous and it’s you who will be blamed when the illusion unravels so its worth making the effort to open up the discussion around uncertainty.

Then the second challenge is to recognise the type of uncertainty you are dealing with based on the matrix above and focus your efforts to reduce uncertainty on the factors where you can learn more and can have a beneficial effect on future outcomes. The options for managing the four quadrants above are quite different:

• Aleatoric Incidents have to be avoided (ie, don’t drop the mug!)
• Epistemic Incidents need allowances in your planning – you cannot control the weather but you can make appropriate allowances – determining what’s appropriate needs research.
• Aleatoric Variables are best avoided but the cost of avoidance needs to be balanced against the cost of the event, the range of outcomes and your ability to vary the severity. You can avoid a car accident by not driving; most people accept the risk and buy insurance.
• Epistemic Variables are usually the best options for understanding and improvement. Tools such as Monte Carlo analysis can help focus your efforts on the items within the overall project where you can get the best returns on your investments in improvement.

Based on this framework your communication with management can be used to help focus your efforts to reduce uncertainty within the project appropriately. You do not need to waste time studying the breakability of mugs when dropped; you need to focus on avoiding the accident in the first place. Conversely, understanding the interaction of variability and criticality on schedule activities to proactively managing those with the highest risk is likely to be valuable.

Now all you have to do is convince your senior stakeholders that this is a good idea; always assuming you have any after the 21st December!*

____________________

*The current ‘doomsday’ prophecy is based on the Mayan Calendar ending on 21st December 2012 but there may be other reasons for this:

Averaging the Power of Portfolios

The interaction between dependent or connected risk and independent risk is interesting and will significantly change the overall probability of success or failure of an endeavour or organisation.

As discussed in my last post on ‘The Flaw of Averages’  using a single average value for an uncertainty is a recipe for disaster. But there is a difference between averaging, connecting and combining uncertainties (or risk).

Adding risk

Where risk events are connected, the ability to model and appreciate the effect of the risk events interacting with each other is difficult. In ‘The Flaw of Averages’ Sam Shaw uses the simile of wobbling a step ladder to determine the uncertainty of how safe the ladder is to climb. You can test the stability of one ladder by giving it a good ‘wobble’. However, if you are trying to determine the stability of a plank between two stepladders doubling the information from wobbling just one is not a lot of help. Far more sophisticated modelling is needed and even then you cannot be certain the full set of potential interactions is correctly combined in the model. The more complex the interactions between uncertainties, the less accurate the predictive model.

However, when the risks or uncertainties are independent, combining the risks through the creation of a portfolio of uncertainties reduces the overall uncertainty quite dramatically.

The effect of portfolios

Consider a totally unbiased dice, any one throw can end up anywhere and every value between 1 & 6 has an equal probability of being achieved. The more throws, the more even the results for each possibility and consequently there is no possibility of determining the outcome!

The distribution after 10, 100 and 1000 throws.

As the number of throws increase, the early distortions apparent after 10 throws smooth out and after 1000 throws the probabilities are almost equal.

However, combine two dice and total the score results in a very different outcome. Whilst it is possible to throw any value between 2 & 12, the probability of achieving a number nearer the middle of the range is much higher than the probability of achieving a 2 or a 12. The potential range of outcomes starts to approximate a ‘normal distribution curve’ (or a bell curve). The reason for this is there is only one combination of numbers that will produce a 2 or a 12; there are significantly more combinations that can make 7.

The more dice you add to the ‘throw’, the closer the curve becomes to a ‘normal distribution’ (or bell curve), which is normally what you expect/get, which is the origin of the name!

The consequence of this phenomenon is to demonstrate that the creation of a portfolio of projects will have the effect of generating a normal distribution curve for the outcome of the overall portfolio, which makes the process of portfolio management a more certain undertaking than the management of the individual projects within the portfolio. The overall uncertainty is less than the individual uncertainties……

Each project carries its level of uncertainty and has a probability of succeeding off-set by a probability of failing (see Stakeholder Risk Tolerance) but as more projects are added the probability of the overall portfolio performing more or less as expected increases, provided each of the uncertainties are independent! This effect is known as the Central Limit Theorem.

One important effect of the Central Limit Theorem is the size if the contingency needed to achieve a desired level of safety for a portfolio of projects is much smaller than the sum of the contingencies needed to achieve the same level of ‘safety’ in each of the individual projects. Risk management is a project centric process; contingency management is better managed at the portfolio level. Not only is the overall uncertainty reduced, but the portfolio manager can offset losses in one project against gains in another.

Whist this theorem is statistically valuable, the nature of most organisations constrains the potential benefit. From a statistical perspective diversity is the key; this is why most conservative investment portfolios are diversified. However, project portfolios tend to be concentrated in the area of expertise of the organisation which removes some of the randomness needed for the Central Limit Theorem to have its full effect.

It is also important to remember that whilst creating a portfolio will reduce uncertainty, no portfolio can remove all uncertainty.

In addition to the residual risk of failure inherent in every project, there is always the possibility of a ‘black swan’ lurking in the future. Originally conceptualized by philosopher Karl Popper and refined by N. N. Taleb, a ‘black swan’ is a risk event that has never occurred before, if it did occur would have and extreme impact and is easy to explain after the event, but is culturally impossible to predict in advance (ie, the event could be foreseen if someone is asked to think about it but it is nearly impossible to think the thought for a compelling reason). For more on black swans see our blog post  and White Paper.

The Law of Averages

The Central Limit Theorem is closely aligned to The Law of Averages. The Law of Averages states that if you repeatedly take the average of the same type of uncertain number the average of the samples will converge to a single result, the true average of the uncertain number. However, as the ‘flaw of averages’ has demonstrated, this does not mean you can replace every uncertainty with an average value and some uncertain numbers never converge.

Summary

Both the Law of Averages and Central Limit Theorem are useful concepts; they are the statistical equivalent of the adage “don’t put all your eggs in one basket”. When you create a portfolio of projects, the average probability of any one project succeeding or failing remains the same as if the project was excluded from the portfolio, but the risk of portfolio suffering an overall failure becomes less as the number of projects included in the portfolio increases.

However, unlike physical laws such as gravity, these laws are not immutable – drop an apple within the earths gravitational pull and it will fall; create a portfolio and there is always a low probability that the results will not conform to normal expectations!

Certainly the probability of a portfolio of projects ‘failing’ is lower then the average probability of each project failing but a reduced level of risk still leaves a residual level of risk.

The flaw of averages

The flaw of averages defined in a book of the same name by Sam L. Savage, states in effect, any plan based on average assumptions is wrong on average! http://www.flawofaverages.com/

However, every duration estimate, cost estimate, risk impact and other estimate our project plans are based on an ‘average’ or ‘expected value’ derived from past experience. And as naturalist Stephen Jay Gould commented, our culture encodes a strong bias either to neglect or ignore variation. We tend to focus instead on measures of central tendency, and as a result we make some terrible mistakes, often with considerable practical import.

The flaw of averages ensures plans based on a single average value that describes an uncertainty will be behind schedule and over budget! A typical example from the book looks at a stocking problem – the business is planning to import short shelf life exotic fruits with a high profit margin, the marketing team have analysed the market and developed a profile of likely sales. The boss looks at the distribution and demands a single figure. All the marketing team can do is take the ‘average’ expected sales and decide 500 cases per month are the most likely level of sales. Based on a profit of $100 per case the boss predicts a net profit of $50,000 per month. However, this is a very optimistic estimate, if less than 500 cases are sold, the fruits will spoil with losses of $50 per case, if more than 500 cases are required the cost of airfreighting extra cases is $150 per case resulting in a loss of $50 or the sales have to be foregone with a risk of losing the customer.

The highest possible monthly profit is $50,000 – if more or less are sold the profit reduces. On average each month more or less than 500 cases will be sold, resulting in returns lower than the estimated $50,000. The only time the predicted profit will be realised in the occasional month when exactly 500 cases are sold.

Even if the company decides not to airfreight additional cases on average the monthly profit will be less than $50,000. Without airfreight, for roughly half the time demand will exceed 500 cases but with no additional stock, profit is capped at $50,000. For the other months, sales will be less than 500 and there will be spoilage costs. Meaning on average, the monthly profit will be less than predicted!

The average is correct, the way the manger is using the average is the ‘flaw’. The same problem shown in the cartoon above, ‘on average’ the pond is only 1 meter (3ft) deep! But averages are rarely what is needed for prudent management.

To properly analyse the projected profits more in-depth analysis is needed, using techniques such as Monte Carlo analysis with the variability of sales being represented by the input probability distribution, the costs and income expected modelled in the tool and the resulting profits predicted in the output probability distribution.

The challenge is getting valid data to model. Projects are by definition ‘unique endeavours’ which means there is no pool of directly valid data; this problem is discussed in our paper The Meaning of Risk in an Uncertain World . When managing project uncertainties our basic data is uncertain!

Recognising this simple fact is a major step towards better project management. To quote George Box (Stamford University) ‘All models are wrong, some models are useful’. No model should be taken as correct, this includes schedules, cost plans, profit predictions, risk simulations and every other predictive model we use! They are never complete representations of exactly what will occur, but a successful model will tell you things you did not tell it to tell you (Jerry P. Brashear).

Building a successful model such as a useful schedule (useful schedules are useful because they are used) should go through the five stages defined by Donald Knuth:
1. Decide what you want the model to do
2. Decide how to build the model
3. Build the model
4. Debug the model
5. Trash stages 1 through 4 now you know what you really want.

And to get a large model to work, you must start with a small model that works, not a large model that does not work. If you want to understand flight what is more useful, a large highly detailed model of a Boeing Jumbo jet built out of Lego blocks that cannot fly or a simple paper aeroplane that does?

The complex Lego model may be visually impressive but is likely to be less useful in understanding a dynamic process such as flight.

The same is likely to be true for most dynamic project models. Edward Tufte says ‘Clear and precise seeing becomes as one with clear and precise thinking’, and John W. Tukey adds ‘It is far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise.’ It is dumb to be too smart!

These concepts are consistent with the PMBOK® Guide idea of ‘progressive elaboration’ and are embedded in the scheduling technique called ‘Schedule Density’ where the initial schedule is developed at ‘Low Density’ and additional detail added as needed (see more on Schedule Density).

The message from this blog is building a useful model is a skilled art, regardless of the subject being modelled (time, cost, risk). A good start is to keep the model simple, if you don’t understand how the model works how will you will be able to judge what it shows you? The model is never the truth; at best it is a useful! And its usefulness will be severely reduced if you rely on averages such as single point estimates without at least using some probability analysis. Melding the need for precision with probabilistic assessments are discussed in our paper Why Critical Path Scheduling (CPM) is Wildly Optimistic.

Whilst this post has focused on one dimension of uncertainty (time and schedule), the principles can be applied to any area of uncertainty.

Real Risk Management

Are risk management and gambling are two side of the same coin? Both involve investing in an attempt to tip the outcome of future events in your favour so you are better off. The similarities between the two processes were highlighted in a Melbourne Comedy Festival show presented by English eccentric and holder of 4 world records, Tim Fitzhigham see: http://www.fitzhigham.com/

Apart from being a very enjoyable hour, we learned a lot about gambling in the 18th century around the time considerable intellectual effort was being put into understanding risk by mathematicians such as Gauss, Leibnitz and Newton. Most of the recorded bets involved the considerable redistribution of wealth, often involved one Lord’s ‘man’ doing something strenuous, dangerous or both against either the clock or another Lord’s ‘man’ and generally any horses involved in the bets did not survive. However, the amounts at stake would certainly focus ones attention on anything that may tip the odds in your favour……

Whilst the show was great fun, and the comedy festival wraps up this week for another year, I’m left wondering is there is any real difference between a bet on which raindrop will reach the bottom of the window first and responding to a bank’s suggestion to fix (or un-fix) the interest rate on your home mortgage which in Tim’s view is a bet against the bank on the difference between current interest rates and those that will be being charged in 5 or 10 years time??? I guess as he pointed out, we are all gamblers, only some of us know it! Certainly Tim’s effort to recreate 10 of the most bizarre recorded bets in history makes entertaining listening and involved some big stakes and some serious risk management…… or was that a just a bet????

Either way, the historical fascination with gabling has influenced modern language, bets were recorded in ‘Gentleman’s Club Betting Books’ – the origin of the term Bookie and Book Maker, and the original meaning of the term ‘stakeholder’ refers to the independent, trusted person who held the ‘stakes’ during the course of the bet.

Managing risk

One of the most overlooked processes for effectively managing the day-to-day uncertainty that is the reality for every single project, everywhere, all of the time, is an effective performance surveillance process. This involves more than simply reporting progress on a weekly or monthly basis.

An effective surveillance system includes regular in-depth reviews by an independent team focused on supporting and helping the project team identify and resolve emerging problems. Our latest White Paper, Proactive Project Surveillance defines this valuable concept that is central to providing effective assurance to the organisation’s key stakeholders in management, the executive and the governance bodies that the project’s likely outcomes are optimised to the needs of the organisation.

Stakeholder Risk Tolerance

Managing the inherent risk associated with undertaking any project, anywhere, in any industry is a critical organisational capability. Within the organisations overall Project Delivery Capability (PDC) the maturity of its risk management approaches is central to the organisation’s ability to generate value (see more on PDC Maturity).

Only very immature or deluded organisations seek or expect to run ‘risk free’ projects. To quote Suzanne Finnamore: “Delusion detests focus and romance provides the veil.” Any sensible analysis of any business activity will indicate levels of risk; effective organisations understand and manage those risks better then ineffective organisation.

The skills that a mature organisation brings to the art of ‘risk management’ is to focus effort on managing risks that can be managed, providing adequate contingencies for those risks that cannot be controlled and deciding how much residual risk is sensible. The balance that has to be struck is between the cost and time needed to reduce the risk exposure further (the pay-back diminishes rapidly), the impact of the risk if it occurs and the profit to be made or value created as a result of the total expenditure on a project.

The sums are superficially simple; adding another $100,000 to the cost of a project to reduce its risk exposure by $10,000 reduces the value of the project by $90,000. In competitive bids, increase your bid price too much and the value drops to $Zero because the organisation fails to win the work! However, the situation is more complex; the nature of the risk may require the expenditure regardless of the potential saving (particularly in areas of safety and quality) and whilst expenditures are reasonably quantifiable, the actual cost of a risk event and the probability of it occurring are variable and cannot be precisely defined for a unique project. Our paper ‘The Meaning of Risk in an Uncertain World’ discusses these issues in more depth.

To develop a mature approach to risk management, each layer of management has a role to play:

• The organisation’s governing body (typically a Board of Directors) is responsible for developing an appropriate risk taking policy and defining the organisations ‘risk appetite’.
• The Executive are responsible for creating the culture and framework that approached the management of risk within the parameters set by the Board in a capable and effective way.
• Senior management are responsible for implementing the risk management system.

The mark of a mature organisation is the recognition at all levels of management that having implemented these systems, the organisation still has to expect failure! Every single project has an associated risk and properly managed, these risks are at an acceptable level for the organisation. But if there is a probability for success, there has to be a corresponding probability of failure!

Assuming the organisation is very conservative and requires budgets to be set with appropriate contingencies to offer a 90% certainty of being achieved, and this setting is applied to all projects consistently, the direct consequence is an expectation that 1 in 10 projects will overrun cost. Certainly 9 out of 10 projects will equal or underrun cost but there is always the remaining 10%. Mature organisations expect the profits and un-spent contingencies on the ‘9 underruns’ to more then offset the ‘1 overrun’. However, these ‘expected failures’ tend to be totally ignored by immature executives who want to pretend there is ‘no risk’ and then blame the PM for the failure.

There are two aspects of dealing with the ‘expected failures’ implicit in any realistic risk assessment. The first is setting the boundaries of accepted risk at an appropriate level of the organisation. Aggressive ‘risk seeking’ organisations will set a lower threshold for acceptability and experience more failures that conservative organisations. But the conservative organisations will achieve far less.

Source: Full Monte Risk Analysis

Looking at the cost aspect of risk for the project above, the most likely cost for this project is $17,500 but this is optimistic with a less then 50% chance of being achieved. The range of sensible options are to set the budget at:

• The Mean (50% probability of being achieved) is $17,770.
• Add one standard deviation to the Mean increases the probability of achieving the project to 84%, but the budget is now $18,520.
• Add two standard deviations to the Mean and the probability of achieving the budget increases to 97% but the budget is now up to $19,270.

From this point, the pay-back diminishes rapidly, to move from 97% to 99.99% (six sigma), an additional $3,000 would be required in contingencies making a total contingency of $4,770 to effectively guaranteed there will be no cost overruns. Because of this very high cost for a very limited change in the probability of achieving the objective most projects focus on either the 80% or the 90% probabilities.

However, even within these relatively sensible ranges, making an appropriate allowance for risk has consequences. Assuming all projects have a similar cost distribution and the organisations total budget for all projects is $10 million the consequences are:

• To achieve a 50%/50% probability of projects achieving budget, approximately 1.6% of the budget will need to be allocated to contingencies: $160,000
• To achieve an 84% probability of projects meeting the allocated budget, approximately 5.8% of the budget will need to be allocated to contingencies: $580,000
• To achieve a 97% probability of projects meeting the allocated budget, approximately 10.1% of the budget will need to be allocated to contingencies: $1,010,000

Whilst the mathematics used above are highly simplified, the consequences of risk decisions are demonstrated sufficiently for the purpose of this post (for more on probability see: WP1037 – Probability). To be 97% sure there will be no cost overruns, more than 10% of the available budget to undertake projects will be tied up in contingencies that may or may not be needed, the consequence is less than 90% of the possible project work will be undertaken by the organisation in a year. The projects ‘not done’ are opportunities foregone to be ‘safe’.

In a competitive bidding market, adding 10% to your estimate to be 90% sure there will be no cost overruns is likely to have a more dramatic effect and price the organisation out of the market resulting in no work. In either situation a careful balance has to be struck between accepted risk and work accomplished, this is a governance decision that needs input from the executive and a decision by the Board.

The governance challenge is getting the balance ‘right’:

• The higher the safety margin the more likely most projects will underrun and the greater the probability some of the contingent reserves will not be used and therefore opportunities to use the funds elsewhere are foregone.
• However, reducing the reserves increases the probability that more projects will overrun (ie, ‘fail’) and this increases the probability that in aggregate the whole project budget will be exceeded.

The challenge for the rest of management is making sure the data being used is as reliable as possible.

The second key feature of mature organisations is the existence of efficient scanning systems to see problems emerging backed up with effective support systems to proactively help the project team achieve the best outcome. The key words here are ‘proactive’ and ‘help’. The future is not set in concrete and timely interventions to help overcome emerging problems can pay dividends. This requires a culture of openness and supportiveness within the organisation so that the root cause of the emerging issue can be quickly defined and appropriate support provided, promptly and effectively. This approach is the antithesis of the approach adopted by immature organisations where the ‘blame game’ is played out and the project team ‘blamed’ for every project failure.

In summary, the organisation’s directors and executive managers need to determine the appropriate risk tolerance levels for their organisation and then set up systems that have the capability of keeping most projects within these accepted boundaries. Understanding and managing risk is a key element of PDC. But having done all of this, mature risk organisations know there are still ‘Black Swans’  lurking in the environment and remain vigilant and responsive to unexpected and unforeseen events.