On the 12th February 2003, the Australian navy came within 20 seconds of losing HMAS Dechaineux, one of its new Collins Class submarines.
The cause of the problem was a burst flexible pipe (hose) in the auxiliary seawater cooling system. The hose burst when the Dechaineux was close to its maximum operating depth flooding the engine space with some 12 tones of water in 7 or 8 seconds.
As with the automatic shutdown of the CERN Large Hadron Collider (LHC) discussed in ‘If it can go wrong……’ [view the post] trained people and automatic responses to the emergency saved the submarine: the external valves in the hull were closed, the submarine was brought to full speed and maximum rate of ascent and all ballast was ‘blown’. Most of the responses that saved the submarine were implemented in the first 15 seconds following the disaster.
What makes this a really interesting study is the cause of the hose failure has never been identified. The short term solution was to limit the submarines maximum operating depth; the long term solution was to re-engineer the connection to eliminate the flexible hoses.
The flexible hoses were always identified as a critical safety item. Of every batch of hoses delivered, 10% were tested to destruction – none failed at less that 4 times the maximum operating pressure and every hose fitted was tested to more than the maximum operating pressure. The broken hose has been microscopically examined and no flaws identified.
In exactly the same way, the LHC management cannot identify the cause of the substation tripping out (causing the LHC shutdown in November), the Navy does not know what caused the flexible hose on the Dechaineux to fail.
What both incidents clearly demonstrate is that it is impossible to predict every source of risk and/or potential catastrophic failure. Other approaches are needed.
- The first is to design good emergency procedures that can avert disasters even if the precise cause of the failure is unknown or unpredicted. This should be a major consideration in any technical design. Both failures were in a predictable ‘class’ – the LHC could predicts power outages, submarines will occasionally suffer major leaks or flooding.
- The second is regardless of the statistical data collected, past performance cannot guarantee future outcomes. Any statistical simulation, including Monte Carlo has a range error. The result may be 90%, 95% or even 99% reliable but there is still a possibility of a result falling outside of the predicted range. The key message from ‘The illusion of control: dancing with chance’ [view the post], is the need to accept that there are things that you simply can’t control, and one of those things is the future.
It is only through recognising and embracing uncertainty can systems be developed to deal with the risk you did not foresee.
For more on the Dechaineux incident see: http://www.newsawards.com.au/files/pdf/05/sir-keith-02_1.pdf?download=1&filename=sir-keith-02_1.pdf