Tag Archives: Risk Management

The flaw of averages

The flaw of averages defined in a book of the same name by Sam L. Savage, states in effect, any plan based on average assumptions is wrong on average! http://www.flawofaverages.com/

However, every duration estimate, cost estimate, risk impact and other estimate our project plans are based on an ‘average’ or ‘expected value’ derived from past experience. And as naturalist Stephen Jay Gould commented, our culture encodes a strong bias either to neglect or ignore variation. We tend to focus instead on measures of central tendency, and as a result we make some terrible mistakes, often with considerable practical import.

The flaw of averages ensures plans based on a single average value that describes an uncertainty will be behind schedule and over budget! A typical example from the book looks at a stocking problem – the business is planning to import short shelf life exotic fruits with a high profit margin, the marketing team have analysed the market and developed a profile of likely sales. The boss looks at the distribution and demands a single figure. All the marketing team can do is take the ‘average’ expected sales and decide 500 cases per month are the most likely level of sales. Based on a profit of $100 per case the boss predicts a net profit of $50,000 per month. However, this is a very optimistic estimate, if less than 500 cases are sold, the fruits will spoil with losses of $50 per case, if more than 500 cases are required the cost of airfreighting extra cases is $150 per case resulting in a loss of $50 or the sales have to be foregone with a risk of losing the customer.

The highest possible monthly profit is $50,000 – if more or less are sold the profit reduces. On average each month more or less than 500 cases will be sold, resulting in returns lower than the estimated $50,000. The only time the predicted profit will be realised in the occasional month when exactly 500 cases are sold.

Even if the company decides not to airfreight additional cases on average the monthly profit will be less than $50,000. Without airfreight, for roughly half the time demand will exceed 500 cases but with no additional stock, profit is capped at $50,000. For the other months, sales will be less than 500 and there will be spoilage costs. Meaning on average, the monthly profit will be less than predicted!

The average is correct, the way the manger is using the average is the ‘flaw’. The same problem shown in the cartoon above, ‘on average’ the pond is only 1 meter (3ft) deep! But averages are rarely what is needed for prudent management.

To properly analyse the projected profits more in-depth analysis is needed, using techniques such as Monte Carlo analysis with the variability of sales being represented by the input probability distribution, the costs and income expected modelled in the tool and the resulting profits predicted in the output probability distribution.

The challenge is getting valid data to model. Projects are by definition ‘unique endeavours’ which means there is no pool of directly valid data; this problem is discussed in our paper The Meaning of Risk in an Uncertain World . When managing project uncertainties our basic data is uncertain!

Recognising this simple fact is a major step towards better project management. To quote George Box (Stamford University) ‘All models are wrong, some models are useful’. No model should be taken as correct, this includes schedules, cost plans, profit predictions, risk simulations and every other predictive model we use! They are never complete representations of exactly what will occur, but a successful model will tell you things you did not tell it to tell you (Jerry P. Brashear).

Building a successful model such as a useful schedule (useful schedules are useful because they are used) should go through the five stages defined by Donald Knuth:
1. Decide what you want the model to do
2. Decide how to build the model
3. Build the model
4. Debug the model
5. Trash stages 1 through 4 now you know what you really want.

And to get a large model to work, you must start with a small model that works, not a large model that does not work. If you want to understand flight what is more useful, a large highly detailed model of a Boeing Jumbo jet built out of Lego blocks that cannot fly or a simple paper aeroplane that does?

The complex Lego model may be visually impressive but is likely to be less useful in understanding a dynamic process such as flight.

The same is likely to be true for most dynamic project models. Edward Tufte says ‘Clear and precise seeing becomes as one with clear and precise thinking’, and John W. Tukey adds ‘It is far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise.’ It is dumb to be too smart!

These concepts are consistent with the PMBOK® Guide idea of ‘progressive elaboration’ and are embedded in the scheduling technique called ‘Schedule Density’ where the initial schedule is developed at ‘Low Density’ and additional detail added as needed (see more on Schedule Density).

The message from this blog is building a useful model is a skilled art, regardless of the subject being modelled (time, cost, risk). A good start is to keep the model simple, if you don’t understand how the model works how will you will be able to judge what it shows you? The model is never the truth; at best it is a useful! And its usefulness will be severely reduced if you rely on averages such as single point estimates without at least using some probability analysis. Melding the need for precision with probabilistic assessments are discussed in our paper Why Critical Path Scheduling (CPM) is Wildly Optimistic.

Whilst this post has focused on one dimension of uncertainty (time and schedule), the principles can be applied to any area of uncertainty.

New CIOB Contract for Complex Projects

The Chartered Institute Of Building (CIOB) has launched a new contract for construction and engineering projects. The CIOB Contract for Complex Projects has been written for the 21st Century. It is designed to permit the CIOB’s Guide to Good Practice in the Management of Time in Complex Projects to be put into practice.

The contract can be used for collaborative design with a building information model (BIM) and anticipates and encourages competence in the use of computerised transmission of data. It requires collaborative working in the management of risks and transparency of data used in such management.

The contract has been drafted to be used in any country and legal jurisdiction around the world to provide a means of managing the causes and consequences of delay (the single most common cause of uncontrolled loss and cost escalation in complex building and engineering projects) where the design is produced by the employer, the contractor with or without a building information model.

The key principles embedded in the contract design include:

  • It is written in plain English, suitable for both building and engineering projects and may be adopted for other types of work. It can be used for turnkey, design and build, for construction only, or for part contractor’s design, both in the UK and internationally.
  • It permits a variety of contract documents including BIM (building information model) and requires electronic communications either via a file transfer protocol or a common data environment for collaborative working.
  • The contract contains new roles for the Project Time Manager, Design Coordination Manager and Auditor, as well as the Contract Administrator and the design team.
  • It requires complete transparency in planned and as-built information in compliance with the CIOB’s Guide to Good Practice in the Management of Time in Complex Projects. It is currently the only standard form of contract available which requires a resourced critical path network, a planning method statement and progress records to a specified, quality assured standard, with significant redress for a failure to comply with the contract requirements.
  • The contractor’s schedule (or programme as it is called in other contracts) is to be a dynamic critical path network in varying densities, described and justified in a planning method statement. It is to be designed in different densities compatible with the information available, reviewed and revised in the light of better information as it becomes available, updated with progress and productivity achieved and resources used and impacted contemporaneously to calculate the effect of intervening events on time and cost (see more on Schedule Density).
  • The contractor’s schedule is not only the time control tool but also the cost control tool against which interim valuations are made and the predicted cost of the works is calculated contemporaneously permitting out-turn cost and total time prediction on a daily basis though the updated working schedule.
  • The contract contains detailed requirements for the identification and use of time and cost contingencies, defines float and concurrency and sets down rules for their use. It provides the power for the contractor to keep the benefit of any time it saves by improved progress as its own contingency, which cannot be taken away.
  • It contains a procedure for contemporaneous expert resolution of issues arising during construction. In the absence of reference to experts specified issues concerning submittal rejections and conditional approvals are deemed to be agreed, helping to avoid doubt about responsibilities and escalation of disputes. The experts used during the course of the works can be called as a witness by either party in any subsequent adjudication and/or arbitration proceedings and, in order to help to give transparency to the way dispute resolvers deal with the contract and help to make sure it functions in the way it is supposed to, the adjudicator’s decision and/or arbitrator’s award is to be a public document, unless the parties agree otherwise.

The Chartered Institute of Building would like to receive your comments and criticism on the Review Edition of the CIOB Contract by Monday, 30 July 2012. All comments will be acknowledged and taken into consideration in future review and revision of the form and its constituent standard form documents. To review the contract see: http://www.ciob.org.uk/CPC

Real Risk Management

Are risk management and gambling are two side of the same coin? Both involve investing in an attempt to tip the outcome of future events in your favour so you are better off. The similarities between the two processes were highlighted in a Melbourne Comedy Festival show presented by English eccentric and holder of 4 world records, Tim Fitzhigham see: http://www.fitzhigham.com/

Apart from being a very enjoyable hour, we learned a lot about gambling in the 18th century around the time considerable intellectual effort was being put into understanding risk by mathematicians such as Gauss, Leibnitz and Newton. Most of the recorded bets involved the considerable redistribution of wealth, often involved one Lord’s ‘man’ doing something strenuous, dangerous or both against either the clock or another Lord’s ‘man’ and generally any horses involved in the bets did not survive. However, the amounts at stake would certainly focus ones attention on anything that may tip the odds in your favour……

Whilst the show was great fun, and the comedy festival wraps up this week for another year, I’m left wondering is there is any real difference between a bet on which raindrop will reach the bottom of the window first and responding to a bank’s suggestion to fix (or un-fix) the interest rate on your home mortgage which in Tim’s view is a bet against the bank on the difference between current interest rates and those that will be being charged in 5 or 10 years time??? I guess as he pointed out, we are all gamblers, only some of us know it! Certainly Tim’s effort to recreate 10 of the most bizarre recorded bets in history makes entertaining listening and involved some big stakes and some serious risk management…… or was that a just a bet????

Either way, the historical fascination with gabling has influenced modern language, bets were recorded in ‘Gentleman’s Club Betting Books’ – the origin of the term Bookie and Book Maker, and the original meaning of the term ‘stakeholder’ refers to the independent, trusted person who held the ‘stakes’ during the course of the bet.

Managing risk

One of the most overlooked processes for effectively managing the day-to-day uncertainty that is the reality for every single project, everywhere, all of the time, is an effective performance surveillance process. This involves more than simply reporting progress on a weekly or monthly basis.

An effective surveillance system includes regular in-depth reviews by an independent team focused on supporting and helping the project team identify and resolve emerging problems. Our latest White Paper, Proactive Project Surveillance defines this valuable concept that is central to providing effective assurance to the organisation’s key stakeholders in management, the executive and the governance bodies that the project’s likely outcomes are optimised to the needs of the organisation.

Stakeholder Risk Tolerance

Managing the inherent risk associated with undertaking any project, anywhere, in any industry is a critical organisational capability. Within the organisations overall Project Delivery Capability (PDC) the maturity of its risk management approaches is central to the organisation’s ability to generate value (see more on PDC Maturity).

Only very immature or deluded organisations seek or expect to run ‘risk free’ projects. To quote Suzanne Finnamore: “Delusion detests focus and romance provides the veil.” Any sensible analysis of any business activity will indicate levels of risk; effective organisations understand and manage those risks better then ineffective organisation.

The skills that a mature organisation brings to the art of ‘risk management’ is to focus effort on managing risks that can be managed, providing adequate contingencies for those risks that cannot be controlled and deciding how much residual risk is sensible. The balance that has to be struck is between the cost and time needed to reduce the risk exposure further (the pay-back diminishes rapidly), the impact of the risk if it occurs and the profit to be made or value created as a result of the total expenditure on a project.

The sums are superficially simple; adding another $100,000 to the cost of a project to reduce its risk exposure by $10,000 reduces the value of the project by $90,000. In competitive bids, increase your bid price too much and the value drops to $Zero because the organisation fails to win the work! However, the situation is more complex; the nature of the risk may require the expenditure regardless of the potential saving (particularly in areas of safety and quality) and whilst expenditures are reasonably quantifiable, the actual cost of a risk event and the probability of it occurring are variable and cannot be precisely defined for a unique project. Our paper The Meaning of Risk in an Uncertain World discusses these issues in more depth.

To develop a mature approach to risk management, each layer of management has a role to play:

  • The organisation’s governing body (typically a Board of Directors) is responsible for developing an appropriate risk taking policy and defining the organisations ‘risk appetite’.
  • The Executive are responsible for creating the culture and framework that approached the management of risk within the parameters set by the Board in a capable and effective way.
  • Senior management are responsible for implementing the risk management system.

The mark of a mature organisation is the recognition at all levels of management that having implemented these systems, the organisation still has to expect failure! Every single project has an associated risk and properly managed, these risks are at an acceptable level for the organisation. But if there is a probability for success, there has to be a corresponding probability of failure!

Assuming the organisation is very conservative and requires budgets to be set with appropriate contingencies to offer a 90% certainty of being achieved, and this setting is applied to all projects consistently, the direct consequence is an expectation that 1 in 10 projects will overrun cost. Certainly 9 out of 10 projects will equal or underrun cost but there is always the remaining 10%. Mature organisations expect the profits and un-spent contingencies on the ‘9 underruns’ to more then offset the ‘1 overrun’. However, these ‘expected failures’ tend to be totally ignored by immature executives who want to pretend there is ‘no risk’ and then blame the PM for the failure.

There are two aspects of dealing with the ‘expected failures’ implicit in any realistic risk assessment. The first is setting the boundaries of accepted risk at an appropriate level of the organisation. Aggressive ‘risk seeking’ organisations will set a lower threshold for acceptability and experience more failures that conservative organisations. But the conservative organisations will achieve far less.

Source: Full Monte Risk Analysis

Looking at the cost aspect of risk for the project above, the most likely cost for this project is $17,500 but this is optimistic with a less then 50% chance of being achieved. The range of sensible options are to set the budget at:

  • The Mean (50% probability of being achieved) is $17,770.
  • Add one standard deviation to the Mean increases the probability of achieving the project to 84%, but the budget is now $18,520.
  • Add two standard deviations to the Mean and the probability of achieving the budget increases to 97% but the budget is now up to $19,270.

From this point, the pay-back diminishes rapidly, to move from 97% to 99.99% (six sigma), an additional $3,000 would be required in contingencies making a total contingency of $4,770 to effectively guaranteed there will be no cost overruns. Because of this very high cost for a very limited change in the probability of achieving the objective most projects focus on either the 80% or the 90% probabilities.

However, even within these relatively sensible ranges, making an appropriate allowance for risk has consequences. Assuming all projects have a similar cost distribution and the organisations total budget for all projects is $10 million the consequences are:

  • To achieve a 50%/50% probability of projects achieving budget, approximately 1.6% of the budget will need to be allocated to contingencies: $160,000
  • To achieve an 84% probability of projects meeting the allocated budget, approximately 5.8% of the budget will need to be allocated to contingencies: $580,000
  • To achieve a 97% probability of projects meeting the allocated budget, approximately 10.1% of the budget will need to be allocated to contingencies: $1,010,000

Whilst the mathematics used above are highly simplified, the consequences of risk decisions are demonstrated sufficiently for the purpose of this post (for more on probability see: WP1037 – Probability). To be 97% sure there will be no cost overruns, more than 10% of the available budget to undertake projects will be tied up in contingencies that may or may not be needed, the consequence is less than 90% of the possible project work will be undertaken by the organisation in a year. The projects ‘not done’ are opportunities foregone to be ‘safe’.

In a competitive bidding market, adding 10% to your estimate to be 90% sure there will be no cost overruns is likely to have a more dramatic effect and price the organisation out of the market resulting in no work. In either situation a careful balance has to be struck between accepted risk and work accomplished, this is a governance decision that needs input from the executive and a decision by the Board.

The governance challenge is getting the balance ‘right’:

  • The higher the safety margin the more likely most projects will underrun and the greater the probability some of the contingent reserves will not be used and therefore opportunities to use the funds elsewhere are foregone.
  • However, reducing the reserves increases the probability that more projects will overrun (ie, ‘fail’) and this increases the probability that in aggregate the whole project budget will be exceeded.

The challenge for the rest of management is making sure the data being used is as reliable as possible.

The second key feature of mature organisations is the existence of efficient scanning systems to see problems emerging backed up with effective support systems to proactively help the project team achieve the best outcome. The key words here are ‘proactive’ and ‘help’. The future is not set in concrete and timely interventions to help overcome emerging problems can pay dividends. This requires a culture of openness and supportiveness within the organisation so that the root cause of the emerging issue can be quickly defined and appropriate support provided, promptly and effectively. This approach is the antithesis of the approach adopted by immature organisations where the ‘blame game’ is played out and the project team ‘blamed’ for every project failure.

In summary, the organisation’s directors and executive managers need to determine the appropriate risk tolerance levels for their organisation and then set up systems that have the capability of keeping most projects within these accepted boundaries. Understanding and managing risk is a key element of PDC. But having done all of this, mature risk organisations know there are still Black Swans’  lurking in the environment and remain vigilant and responsive to unexpected and unforeseen events.

Lessons Not Learned

Melbourne’s Swanston Street is undergoing a major upgrade to create a primarily tram and pedestrian precinct. This includes new tram stops, but the new Swanston St. stops are dangerous.

The new tram stop outside of Melbourne Central is probably one of the most dangerous pieces of public architecture produced in the last several years. The design ignores basic building standards established for over 100 years and incorporates a small ‘trip’ line of around 4cm in height in the middle of what is otherwise a flat walking area.

The almost invisible ‘trip line’ before the yellow paint line was added.

Steps and kerbs should be a minimum of 10cm in height (preferably 15cm or 6 inches) so walkers can clearly see the change in level. The shallow trip line incorporated into this design is too low to notice but big enough to catch anyone walking normally. I have no idea how many people will need to fall and then sue the Council for negligent design before this dangerous ‘feature’ is corrected but you can guarantee there will be many accidents and near misses on a daily basis.

Another view of the tripping hazard.

What is tragic is the apparent inability of the designers of this tram stop to learn from similar stops created in other locations in the network or from published design principles. This type of ‘tripping hazard’ was a major consideration in the Bourke St. Mall design a couple of years ago and an elegant solution was developed.

Even without this experience, there is plenty of information available that clearly shows it is dangerous to put a small ‘trip line’ at right angles to the direction of travel of most pedestrians. Good design suggests the ‘trip’ is either eliminated by a small change in level or protected by a hand rail.

This ‘feature’ has apparently been deliberately included in the design to keep the pedestrian footpath and bike lane differentiated by having pedestrians ‘step down’ into another zone. A great idea but the same separation effect could easily have been achieved by using a couple of well placed bollards or even a painted line or change in surface texture – the focus on one aspect of safety without looking at easily learned lessons on another has created a hazard that will cause serious injury to many people if it is not quickly corrected.

Unfortunately a few cents of design effort to review and ‘learn’ appropriate lessons will require $thousands to fix now the stops have been built. The danger has obviously been recognised with a pretty yellow line now painted along the length of the trip line (which it totally useless if you cannot see the ground for people). My guess is nothing further will happen until the council’s insurers force the issue after receiving a barrage of insurance claims. Getting designers, bureaucrats and politicians to admit they have screwed up the design is next to impossible. But until this happens ‘enjoy your trip’ will have a completely different meaning in Swanston St.

Photographs copied from http://treadly.net/2011/12/01/swanston-st-the-upgrade/

Project and Organisational Governance

One of the themes running through several of my recent posts is the importance of effective Governance. Both organisational governance and its sub-set project governance.

Good governance is a synonym for ‘good business’, structuring the organisation to deliver high levels of achievement on an ethical and sustainable basis. This requires the optimum strategy and the right approach to risk taking supported by sufficient processes to be reasonably confident the organisations limited resources are being used to achieve the best short, medium and long term outcomes.

Project governance focuses on the portfolios of programs and projects used by the organisation to deliver many of the strategic objectives. This process focuses first on doing the right projects and programs constrained by the organisations capacity to undertake the work – Portfolio Management; secondly, creating the environment to do the selected projects and programs right- developing and maintaining an effective capability; and lastly systems to validate the usefulness and efficiency of the ongoing work which feeds back into the selection and capability aspects of governance.

 

Within this framework, portfolio management is the key. Strategic Portfolio Management focuses on developing the best mix of programs and projects to deliver the organisations future within its capacity to deliver. This means taking the right risk and having sufficiently robust system in place to identify as early as possible the ‘wrong projects’, so they can be either be reframed or closed down and the resources re-deployed to other work.

It is impossible to develop an innovative future for an organisation without taking risks and not every risk will pay off. Remember Apple developed the ‘Apple Lisa’ as its first GUI computer which flopped in the market, before going on to develop the Apple Macintosh which re-framed the way we interact with machines.

Apple Lisa circa. 1983

Obviously no organisation wants to have too many failures but good governance requires ‘good risk taking’. Apple had no guarantees the i-Pod and its i-Tunes shop would succeed when it started on the journey of innovation that has lead to the i-Phone, i-Pad and Apple becoming one of the largest companies in the world based on capitalisation. As Richard Branson says – ‘you don’t bet the company on a new innovation’ but if you don’t innovate consistently, obsolescence will be the inevitable result.

The balance of project governance focuses around creating the environment that generates the capability to deliver projects and programs effectively, effective sponsorship, effective staff development, effective and flexible processes and procedures, simple but accurate reporting and good early warning systems to identify issues, problems and projects no longer creating value (a pharmaceutical industry saying is that if a project is going to fail it is best to fail early and cheap!).

Good questions outrank easy answers! Every hour and dollar spent on governance processes is not being spent on developing the organisation. The challenge of good governance is to have just enough reporting processes embedded in an effective culture of openness and accountability to provide an appropriate level of assurance the organisation’s resources are being used effectively; whilst at the same time allowing innovation and development. Restrictive and burdensome governance processes are simply bad governance – they restrict the organisation’s ability to achieve excellence.

To help organisations understand these key governance processes we have updated our two White Papers on the subject:
Corporate Governance: http://www.mosaicprojects.com.au/WhitePapers/WP1033_Governance.pdf
Project Governance: http://www.mosaicprojects.com.au/WhitePapers/WP1073_Project_Governance.pdf

For more discussion around the subject of governance see the previous posts on this blog.

Project Governance

Corporate governance is defined as aligning as nearly as possible the interests of individuals, the organisation and society. Good governance is good business!

Project governance is a sub-set of corporate governance, focused on systems that ensure the right projects and programs are selected by the organisation, and the selected ‘few’ are accomplished as efficiently as possible. Projects that no longer contribute value to an organisation should be terminated in a way that conserves the maximum value and the resources reallocated through the portfolio management process to more valuable endeavours.

Project Governance Structure

The framework for effective project governance is laid out above, and is an executive management responsibility. Sponsors and the Portfolio Selection/Management processes provide the key link between the executive and the working project and programs (for more see our Governance White Paper).

The focus of this post is to look at the pre-selection activities that inform the portfolio selection processes. One of the key conclusions to be drawn from the Ombudsman’s Report discussed in my earlier post Cobb’s Paradox is alive and well  was that many of the projects that contributed to the $1 billion in failures were set up to fail – the projects had absolutely no chance of delivering within the announced parameters: the inputs to the portfolio selection process were grossly flawed (or were non-existent).

This appears to be a wide spread issue. Most project management standards such as ISO21500 and the PMBOK® Guide start with an approved project and a business case or similar that defines what has to be accomplished; this is the end of the portfolio selection process outlined above and is assumed to set realistic and achievable objectives.

What is missing, are the steps leading up to this point; the life of a ‘project’ starts with an idea, need, opportunity, requirement or threat (the ‘concept’). The organisation assesses and studies the ‘concept’ hypothesises options and solutions and frames a proposal that becomes the foundation of a future project. These key investigative elements of a project generally sit under the portfolio umbrella developing information to allow a proper decision to be made. In mining this can represent exploration, feasibility studies, ‘bankability’ studies and concept designs which between them can cost $millions, leading to project funding. Importantly, this ‘Front End Loading’ (FEL) is seen as the key to a successful mine in most major mining corporations.

Similar problems exist in major infrastructure projects, defining a solution to prison overcrowding can involve building a new major prison, building several smaller prisons, extending current prisons, changing the way criminal justice system works to reduce the need for prison places, or a combination of the foregoing options (substitute University/hospital/school, into the previous sentence to see just one dimension of the challenge). However, unlike mining, most government and many corporate organisations see effective ‘front end loading’ as unnecessary.

Other organisations use the process to formulate definitive solutions to problems they have no real understanding of (typical in ICT) and then pretend the defined solution has no associated risk (because it is defined) despite the fact the full dimensions of the problem the project is supposed to solve are still unknown, and are frequently changing over time.

The challenge, requiring informed judgement and effective governance is recognising which development processes suits what type of ‘concept’:

  • Sometimes, the ‘investigation’ requires a significant amount of work (eg, a bankability or feasibility study); this work may be treated as a project in its own right, and is time, cost and resource constrained with a defined deliverable (the report).
  • If the work is expected to flow forward and will only be stopped in exceptional circumstances, project phases work best, with some form of ‘gateway’ or transition review.
  • In other circumstances, studies are undertaken as part of the portfolio by corporate or PMO professionals with no dedicated budgets, assessing multiple proposals as an ongoing process, but once a concept gets the go ahead a project is created and a budget and resources allocated.
  • Other concepts (particularly problems) cannot be defined and an ‘agile’ approach is needed where elements of a partial solution are developed and put into use developing new learning that will then allow the next module to be developed in a progressive sequence. However, whilst this may be the most suitable and cost effective way of developing an effective solution, budgeting in a traditional ‘iron triangle’ concept of fixed cost, time and scope is impossible.

The challenge is recognising which type of project is being proposed (based on Project Typology), and then deciding which type of process will develop the best input to the portfolio selection process and what level of uncertainty (risk) is associated with the proposal once developed. Certainty is not important, what matters is appreciating the extent of the risks and the likely benefits, so an informed investment decision can be made. Most ‘game changing’ initiatives involve high risk, high reward projects that create a totally new future!

OGC Gateway™

The OGC ‘Gateway Reviews’ is a flexible process that addresses this part of major projects from the client’s perspective:
Gateway 1 = Business Justification, options identified and appraised, affordability, achievability and value for money established.
Gateway 2 = Procurement strategy, will the proposed strategy achieve the project objectives?
Gateway 3 = Investment decision, based on realistic project cost information (eg, tenders or bids) can the business case be confirmed from both the cost and the benefit perspective?
Gateway 4 = Readiness for service. The completion of the project work and a reassessment/confirmation of the expected benefits as the deliverable is put into ‘service’.
Gateway 5 = Benefits evaluation. Did we get what was expected now the project’s outputs are being used?

Summary

Most of the risks and rewards associated with a project or program are determined long before the project manager is appointed; if these decisions are wrong (or non-existent) project and program management cannot resolve the problem.

The role of effective project management is to deliver a realistic and achievable outcome efficiently; if the parameters for the project are unrealistic in the first place, the best project management can do is stop the situation deteriorating further! As far as I know, none of the various BoKs and methodologies, including the PMBOK® Guide has a ‘miracle’ process that will magically transform an impossible set of objectives into achievable set of objectives. Wishful thinking is not an effective substitute for effective project governance!

Cobb’s Paradox is alive and well

In 1995, Martin Cobb worked for the Secretariat of the Treasury Board of Canada. He attended The Standish Group’s CHAOS University, where the year’s 10 most complex information technology (IT) projects are analysed. The high level of failure led Cobb to state his now famous paradox: “We know why projects fail; we know how to prevent their failure—so why do they still fail?”

In 2011, another report into the management of IT projects asks the same question! This time the report was prepared by the Victorian Government Ombudsman, in consultation with the Victorian Auditor-General, it documents another series of failures largely created by executive management decisions. The report entitled Own Motion Investigation into ICT – Enabled Projects, examines 10 major Victorian Government ICT projects that experienced difficulties such as budget and timeframe blowouts or failure to meet requirements.

Portfolio Management
Problems identified by the Ombudsman in the area of Portfolio management and governance include a lack of effective leadership, accountability and governance. He was particularly concerned about poor project governance, the lack of accountability of project stakeholders and a lack of leadership — a reluctance to take tough decisions.

These failures contributed to poor decision making, and an inability or reluctance to make difficult, but necessary decisions. Leaders lead and determine governance practices; the resources needed to implement these facets of effective Portfolio management are readily available including:

Project Definition
It is impossible to deliver a project successfully if the decision to proceed is based on inaccurate assessments in the business case. The Ombudsman commented on the inadequacy of business cases, the failure to fully define requirements for new systems, a general reluctance to change business processes to better fit with off the shelf products (to reduce cost and risk) and a ‘tick the box’ approach to risk management (ie, avoiding any real assessment of risks and opportunities).

Linked to this lack of definition major project funding decisions were announced publicly before the business case was fully developed (representing either wishful thinking or a wild guess?), and high risk decisions being made to only partially fund some projects.

The solution to these issues is a robust and independent PMO that has the skills and knowledge needed to validate business cased before they go forward to management for decisions. Many years ago, KPMG released a series of reports that highlighted the fact that organisations that failed to invest in effective PMOs were simply burning money! The Ombudsman’s report shows that ‘burning public money’ is still a popular pass time.

For more on PMOs and to download the KPMG reports see: http://www.mosaicprojects.com.au/Resources_Papers.html#Proj_Off

Risk Management
Many of the factors identified above and in my view the primary cause of most bad decisions is the abject failure of senior management to insist on a rigorous risk management process. Risk management is not about ‘ticking boxes’, it is about having the ethical courage to objectively explore the risks and then take appropriate actions to either mitigate the risk or provide adequate contingencies within the project budget. This failure was manifest by an inconsistent approach to contingency funding. There are many examples of high risk decisions being made without any contingency provisions:

  • The Myki ticketing system was let to an organisation that had never delivered a ticketing system before. No contingencies were made for this high risk decision and the project is years late, $millions over budget and will only deliver a small part of the original scope.
     
  • Agencies preferred to be on the leading edge rather than leveraging what had been done by others elsewhere. This may be justified but not without proper risk assessment, mitigation and contingency.

Government agencies are not alone in failing to effectively manage risk in ICT procurements. The same problem has been identified in major infrastructure projects, in a series of reports by Blake Dawson; see: Scope for improvement

There are always difficulties in transferring project risks to vendors, and dealing with large vendors who may be more experienced in contract negotiation than their agency counterparts. Whilst modern forms of contract provide opportunities to adopt innovative procurement processes that could significantly reduce project risks for vendors and customers these were not used.

As our paper, The Meaning of Risk in an Uncertain World  and the Blake Dawson reports clearly demonstrate, not only is it impossible to transfer all of the project risk to a vendor, it is totally counterproductive to try! Organisations that try to transfer ‘all of the risk’ end up with a much poorer outcome than those organisations that actively manager the risks in conjunction with their vendors.

Large ICT projects are inherently complex and necessarily involve some significant risks. But these can be mitigated to some degree by taking heed of the Ombudsman’s observations, lessons learnt in other projects and the implementation of robust and independent systems.

The PMI Practice Standard for Risk Management provides  good starting point.

Recommendations
The Ombudsman’s recommendations on how to address these issues can be applied to ICT and other projects undertaken by other state, local and Commonwealth government agencies, and in the private sector: Download the report.

In my opinion, the primary cause of these failings, referenced but not highlighted by the Ombudsman, is cultural. Executives and senior managers overtly preferring the status quo and the current power structures they have succeeded within over leading the implementation of change that will deliver improved outcomes for their organisations but make people more accountable and redistribute organisational power. This was the focus of my last posting; Culture eats strategy for breakfast 2!

As Martin Cobb observed in 1995, “We know why projects fail, we know how to prevent their failure — so why do they still fail?” Unfortunately this is still a valid question more that 15 years later and, without leadership from the very top, I expect the effect of this report will be little different to the dozens of similar reports generated over the years and we will still be asking the same question in 2020.

The answer is culture and leadership – to change the culture within senior management ranks, the owners of organisations need to take actions similar to the Australian Federal Government and mandate effective processes and then measure performance in their implementation and use. The implementation of the Gershon Report that is being forced through the federal government departments is a Cabinet level initiative. It is still too soon to judge wether the initiative will be successful, effective culture change takes years to embed in major organisations, but at least the push has started at the right level. My feeling is that if the pressure is maintained for another 3 or 4 years (the original report was released in 2008) there may be some real benefits. To avoid similar reports to this one in the future, the leaders of other organisations need to take similar robust, strategic action tailored to the needs of their organisation.

Project professionals can help by effectively communicating to your top-level executives the real benefits of effective project governance. For many ICT and other technical/engineering professionals this represents is a whole new set of skills to learn, my book Advising Upwards may help!

The Roman Approach to Contract Risk Management

An interesting paper in the September edition of the Project Management Journal co-authored by a colleague, Derek Walker contrasts the delivery of public projects in the Roman era with modern project management. The primary conclusion is that nothing much has changed; the Romans outsourced most of their major works to contractors, with both public accountability and a legal framework as key governance constraints.

What’s significantly different, is the consequences of failure! If a project went badly wrong in Roman times, the responsible public official would suffer a major career limiting event that could affect the prospects of his descendants for generations to come. Whilst the retribution applied to the contractor could be even more serious including death as well as retribution for generations to come.

Applying the Roman approach could give a whole new meaning to the ‘pain share’ bit of an Alliance contract…… as well as removing by execution many of the worst performing contractors. Rome was not built in a day but their empire did last for close to 1000 years.

This history contrasts with several recent studies that clearly demonstrate the ineffectiveness of penalty clauses as a mechanism for managing client risk two relevant papers are:
CIOB; Managing the Risk of Delayed Completion in the 21st Century
Blake Dawson; Scope of Improvement

It would seem the sanctions offered under today’s laws are insufficient to make penalties really effective (and in modern contracts they only work one way rather than the two-way effect of the Roman sanctions). Therefore the only option is proactive management.

The unacceptable alternative is to hope the problem goes away…… but burying your head in the sand leaves a very tempting target for someone’s boot.

If you are interested in the history of project management, my papers offer a good starting point, starting with The Origins of Modern Project Management at: http://www.mosaicprojects.com.au/Resources_Papers_050.html

For more in-depth coverage see: http://www.lessons-from-history.com and of course: Frontinus – A Project Manager from the Roman Empire Era by Walker & Dart (Project Management Journal Vol.42, No.5, 4-16)