Ask for information you can use

Folowing on from my earlier post ‘A though on PMOs and Project Controls’ , PMOs and projects regularly collect and report on data but you have to ask how often is the data converted into information the organisation can use?

I have just wasted 15 minutes trying to respond to a survey commissioned by our local city council. The structure of the survey was so bad there is no way the data collected would allow any useful information to be gathered. Two of the questions (from memory) were:

• ‘Overall how do you rate the Port Phillip City Council’s performance’ with a range of 5 options from bad to excellent. Sound like an insightful question but what does the answer tell anyone?? The Council delivers a wide range of totally separate services from refuse collection to maternal and child welfare. Any overall answer is meaningless; unless the Councillors are spending rate payer’s money on a ‘feel good’ ego trip and hope they will be rated ‘good’. It does not matter if the answer is bad, good or indifferent unless the data collected identifies what is good or bad so management action can be taken to lock in the good and improve the bad.
• “How do you rate the street cleaning service overall?”  More specific but still useless data. The council cleans shopping strips on a daily basis, other roads on an occasional basis and numerous lanes and footpaths very rarely. My answer was shopping strips great, major roads OK, side roads like the one I live in fairly poor and the laneways are a total disaster….. But how would I rate the service overall??? At this point I terminated the interview!

  Overall rating should be complied from specific measurements weighted appropriately. In this case a weighted assessment of the relative importance of clean shopping strips -v- clean local streets -v- clean laneways to the local citizens. How do you assess the weightings? Ask a sample group of people which they feel is more important on a comparative basis before running the survey. Then collect specific data and build some useful information that can help direct improvements in the overall service.

When designing this sort of data gathering, you also need to filter out influences like staff culture (the Port Phillip staff are really great and helpful) from systemic issues such as the contract conditions the street cleaning contractor operates under.

Then by compiling all of the various rating for the specific services, an overall rating for the council could be compiled and more importantly the high performing areas noted and lessons from these areas transferred to the less well performing areas.

There’s a lot to learn from this example of bad surveying. Designing surveys and collecting data is not a trivial exercise. There is nothing simpler than bogging a PMO down collecting masses of data that can never be converted into useful information. Do this and the PMO will be seen as a useless bureaucracy and sooner or later it will be reorganised out of existence.

Focus the data collection on a limited range of key factors that can provide useful management information and the PMO will be seen as a real value add to the business.

So how do you rate your PMO overall?     Only kidding…..

On a closing note –
The number of really bad surveys seem to be increasing exponentially – I think around 80% of the various project management surveys I look at, mostly from post graduate students, seem to be either designed to support a pre-determined answer or so badly designed the data could be interpreted to suit any answer the researcher chooses. There is a real discipline and skill in developing an effective survey; unfortunately it’s a skill that seems to be in very short supply.

A thought on PMOs and Project Controls

The quality guru W. Edwards Deming said ‘In God we trust, all other bring data’. However, recent developments in the Victorian Government health system offer a salient reminder to any PMO manager on the value of data.

Victorian hospitals are rewarded for good performance and fined for poor performance. One measure being the length of waiting lists another is the time patients wait in casualty/emergency area before admittance. Rumours have been circulating for several months that some administrators were manipulating data to avoid fines and win bonuses – both adjusted against the hospital’s funding for the year (ie, no one personally benefitted from the manipulation). An audit report has confirmed many of the rumours and has found behaviours that in many cases are worse for the patients and worse of the hospital than if nothing had occurred.

One example is transferring patients from emergency care to the operating theatre waiting area to avoid a fine for failing to admit the patient within the prescribed maximum time. The consequences of this action include:

• The patient being removed from an area focused on emergency care to an area with little monitoring capability – a reduction in care to the patient.
• The reduction in throughput in the operating suites due to overcrowding and skilled theatre staff having to spend time on patient care rather than operations.
• A net reduction in the overall delivery of service by the hospital; but improved statistical reports to Government.

This farce was in the interests of everyone except the people the Government and hospital are supposed to serve, the public needing hospital care. Some of the vested interests include:

• The government’s desire to look good by reducing waiting time.
• The hospitals desire to achieve the maximum budget income for the year.
• The administrators’ desire , both in the hospital and the government, to avoid ‘rocking the boat’.
• The built in bias in government spending to fully spend each year’s budget allocation.

Apart from the obvious issue around the intelligence of a system that removes funding from an institution that probably needs more funding to meet the demands on its services – the only practical way to treat more patients is to provide more beds and staff which cost money… there are a number of important lessons for all PMO managers to consider when setting up ‘project dash boards’ and the like:

• What you measure will change behaviours. Focus on things that matter like value and benefits not easy to measure statistics like time.
• Make sure the data you use is validated.
• Identifying a problem is not enough! You should work with the project team and management to make sure an effective solution is crafted. It’s not the PMOs job to solve the problem but it can be a powerful facilitator of solutions by measuring the right statistics and asking the right questions.

This is a more challenging role but also one that can really contribute to the overall performance of you organisation.

For more on this topic see:

• Reports on the problem from the Age newspaper
• Blog – Value is in the eye of the Stakeholder
• Mosaic’s Published Papers focused on PMOs