Defining project success – moving beyond benefits realisation!

What you measure is what you are likely to get – so do the so-called measures of project success used by The Standish Group[1] and other really help?  Certainly, the CHAOS definition has been updated from the ‘traditional’ assessment of on-time, on-budget, and on-target (scope) to the ‘modern’ definition of on-time, on-budget, and a satisfactory result[2]; but does this really change anything?

Challenged projects failed to achieve one or more of the measures, failed projects were cancelled before completion or the deliverables were not used.  The problem is do these measures really matter?  The Panama canal expansion was planned to finish in 2014, it was actually finished in 2016; its costs were estimated at US$5.25 billion, it actually cost will be in the range of $6 to $8 billion (depending on the outcome of disputes).  But, the expanded canal is operating close to capacity and has had to restrict bookings since January 2017 to minimise delays and the latest estimates project that fiscal transfers from the Canal to the central government are expected to increase 60% to a total USD 1.6 billion in the current fiscal year.

Given the canal is 100 years old and the new works can be expected to have a similar lifespan what is the real measures of success? In the last 11 years of Panamanian administration, canal revenues grew at a compound rate of five percent annual of the fiscal year 2006 to 2016. Given the core mission of the Panama Canal is to generate income and support the growth of the Panamanian economy is this really a ‘challenged project’?

We have been suggesting for many years real success is a much more complex issue that requires far more sophisticated measures and management than simply focusing on time, cost and scope:

• Our 2007 paper Avoiding the successful failure looked at the managing relationship with stakeholders as a key underpinning of success;
• Our 2012 presentation to the Benefits Realisation summit asked the question Who really benefits?; and
• Who values value? looked at the vexed question of what is really valuable.

All of these papers lead towards the same conclusion, project success is founded in the creation of deliverables that facilitate the realisation of benefits. But real success needs something more; the benefits have to be seen as valuable by a large proportion of the key stakeholders – success is very much in the ‘eye-of-the-stakeholders’ and if they declare the project a success it is, if they don’t see the outcomes as valuable it is not!

The simple measures used by The Standish Group are only relevant if they advantage or impact the value perceived by the project’s stakeholders.  A number of projects in Queensland leading towards the 2018 Commonwealth Games undoubtedly have time as a key component in providing recognisable value to stakeholders. In many other projects time may be almost irrelevant. Cost may affect profitability (and therefore value in the ‘eyes’ of some stakeholders) but is probably far less important than delivering an output that delights the end users.  Quality and scope should be similarly balanced against the value perceived by stakeholders.

The problem with the proposition that success is based on outcomes of a project being perceived as successful by its stakeholders are many:

• Different stakeholders will have different views of what is important and ‘valuable’ – these differences may be irreconcilable.
• Stakeholder’s perceptions change over time – the Sydney Opera House went for a ‘white elephant’ that suffered massive time and cost overruns to a UNESCO World Heritage landmark in record time.
• It is impossible to know how people will react to the eventual project outcome in advance –success, or failure, emerges after the project has delivered and everyone involved has ‘gone home’.

I don’t have an easy answer to this conundrum – but I do believe two major shifts in project governance and the overall ‘management of project management’ are needed:

1. The concept of project success is built over time; it starts during the earliest stages of a project when the concepts are being formulated – no one benefits from delivering the wrong project on time and on budget.
2. Everyone involved in the management of the project including sponsors and portfolio managers through to the project manager need to have in-depth discussions with their stakeholders about what success looks like and what is really important to the client and end users of the deliverable. This discussion needs to be framed by the constraints of cost and time (to the extent they matter) but not limited to predetermined artificial values, to create a prioritised list of success criteria that directly relate to the needs of the stakeholders (which may include time and/or cost, but equally may not); see: Defining Project Success using Project Success Criteria.

Finally, the ‘what’s really valuable’ discussion needs revisiting on a regular basis to keep the work of the project aligned with the evolving needs and perceptions of the stakeholders.  You can call this ‘agility’ if you like (or simply effective stakeholder engagement) but by now we all should recognise that producing ‘failed’ projects helps no one and driving to achieve arbitrary and/or unnecessary time and cost targets is a good way to destroy real value.

Making these shifts presents some real challenges:

• The challenge the project controls community needs to start looking at is how do we start measuring success? Most organisations can’t even measure benefits!
• The challenge for people involved in the overall management of projects is primarily answering the question which stakeholders are important in this conversation and how do we engage them?
• The portfolio management challenge is focused on developing ways to quantify and assess these intangible metrics to select the most valuable projects.
• The governance challenge is putting rigour around the whole framework to encourage innovation, satisfy stakeholders and maintain overall accountability.

My feeling is that project success is a complex, emergent, characteristic of a project that manifests after the work of the project has been completed.

Your thoughts are welcome.

_____________________________

[1] See: http://www.standishgroup.com/outline

[2] Presumably this change is to accommodate agile project where the scope is defined through the course of the work.