Basing safety on evidence

At the PACDEFF conference for CRM developers and trainers in Auckland, evidence-based training was highlighted in a wide range of research reports. While a significant number of the papers presented were focused on safety in an airline environment, there was a lot of valuable information for other sectors. This certainly provided food for thought around how to secure more reliable data on safety in general aviation and especially aerial work operations.

The critical message was that training related to actual events and observations – including threat and error management – was very valuable in improving an organisation’s bottom line as well as safety outcomes. It appears the trick is being able to capture, analyse and make sense of incident and performance data so as to focuson training initiatives that can make a real difference.

In a GA environment, this may be easier said than done.

There is a cultural chasm between doing a safety-related activity to get a tick in the box or to have the paperwork looking good for the next audit, and actually building a system that honours and delivers continuous improvement.

The misleadingly ‘simple’ task of establishing robust internal reporting systems for companies is a starting point for safety management systems (SMS), equipment maintenance, worksite inspections and of course, aviation safety and training.

Small to medium size companies seem to struggle with both the ‘why’ and the ‘how’ of this critical safety initiative – to know what is really happening in your company so you can get ahead of the game. While some of the issue is resource driven, a lot of the resistance is about culture and a dislike of paperwork. And yet, when companies persevere with internal reporting of incidents and actually respond to what they are finding, business owners describe that reporting system as possibly the most valuable business management tool they have.

AAAA has integrated such a system into its AIMS accreditation program and made it a key dashboard indicator of the health of a company’s SMS. If no-one is reporting anything or if

In a GA environment, this may be easier said than done.

management is not tracking the issues identified and fixing them, then there is probably a lot of other things not going well in the company.

In the recent annual AAAA survey of members, it was clear that there remain significant impediments to reporting incidents and accidents to the ATSB. Eighty per cent of respondents indicated a fear of misuse of data by CASA against the company as a clear disincentive to report. AAAA is aware of instances of CASA using allegedly ‘de-identified’ ATSB data to take administrative action and it doesn’t take many such actions to undermine the reporting system. Until the ATSB gets serious about de-identifying accident reports by removing VH identifiers and CASA gets seriousabout honouring a ‘just culture’, there is little hope that confidence in the system will grow.

Without good data, any safety manager will struggle.

The AAAA survey also found that 88 per cent of operators indicated that their most recent CASA audit had not led to any identifiable improvement in safety. If evidence is to have any value in driving change within CASA, then clearly there should be a reappraisal of what audits are meant to be achieving – other than ticking the box. Audits should be a key learning experience as well as a compliance tool, but this is difficult when the auditor has little to no experience in the sector and the desired outcome by the regulator is a ‘gotcha’ moment.

The Forsyth review found plenty of evidence that backs ongoing industry calls for the reform of CASA. The report made 37 recommendations which are still awaiting action, despite industry clearly supporting most recommendations.

All the evidence points to the September 14 start date for Part 61 on licensing being a debacle, with few – if any – companies transitioned to the essential Part 141 training organisations for aerial application. Evidence would suggest another deferment would be prudent, especially if combined with a review of Part 61 along the lines identified in the Forsyth report.

While evidence-based training has long been the AAAA approach to improving safety, evidence-based regulation for Australian aviation appears to be a step too far.