Sometimes policy-makers look to research not to provide evidence on the most effective program or intervention, but to confirm a decision already taken and to improve processes in future. This was the case for the Ontario Ministry of Labour (MOL) with respect to the Institute for Work & Health (IWH)’s research on its High Risk Firm Initiative.
According to Sophie Dennis, the Ministry of Labour’s assistant deputy minister of operations, the Institute’s research on the initiative was very helpful in highlighting the limitations of using workers’ compensation claims data as the sole basis for determining where to target occupational health and safety (OHS) enforcement or consultation interventions. The Ministry had already begun to move towards using a wider array of information to inform its targeting strategy,
says Dennis. The IWH study confirmed the importance of doing so.
The IWH research also informed the Ministry’s process for introducing new programs in future. The research findings underscored for us the importance of thinking carefully in the design phase of proposed OHS interventions about what outcome measures should be used to shape strategy,
adds Dennis. We have changed some of our internal processes as a result of this work.
Initiative targets employers with poor OHS performance
In 2004, the MOL launched the High Risk Firm Initiative (HRFI), aimed at reducing work injuries by targeting employers with poor OHS performance. The HRFI ran from 2004 to 2008.
In each year of the HRFI, the OHS performance of all firms registered with the Ontario Workplace Safety and Insurance Board (WSIB) was ranked using data on workers’ compensation claims. The two per cent of firms with the worst rankings were targeted to receive intensive inspection by MOL officers (four inspections in a year).
The next eight per cent of firms were referred to their sector-specific health and safety association (HSA) for consultation services. These firms could either be approached by the HSA with offers to review their OHS practices or could be referred back to the MOL for inspection (at most two inspections in a year).
HSAs invite IWH to help assess program
Early in 2006, the HSA responsible at that time for the service sector, the Ontario Service Safety Alliance (OSSA), and the HSA responsible for the manufacturing sector, the Industrial Accident Prevention Association (IAPA), approached the Institute for Work & Health (IWH) with a proposal regarding the implementation and evaluation of the HRFI for the firms assigned to them—the “eight per cent” group. OSSA and IAPA had determined that they would not be able to approach all the firms that had been assigned to them for OHS consultation services.
These two HSAs, with the support of the MOL and the WSIB, worked with a team of IWH researchers led by IWH Senior Scientist Dr. Sheilah Hogg-Johnson to randomly divide the firms on their lists into three groups:
- firms to be approached by the HSA with an offer to provide OHS consultation services;
- firms to be referred back to the MOL for an inspection schedule of one or two inspections over the year; and
- firms assigned to receive no additional services (business as usual).
IWH was then asked to evaluate the different outcomes for the three groups.
Initiative not associated with better rates
The key finding of the research was that work injury claims measures did not significantly differ among the firms in the three groups in the two years after the intervention. In both the service and manufacturing sectors, no group differences were observed in overall claim rates, lost-time claim rates, no-lost-time claim rates or for the rate of disability days.
The absence of a program effect could have been due to a number of factors. Evidence suggests that some of the firms selected were not at high risk: 60 per cent of inspected firms had no orders, and the types of orders written were typically aimed at how the firm organized OHS rather than specific hazards. The actual programs did not reach all the firms on the lists, and one of the HSAs described its approach as a “light touch.” It is also possible that two years’ follow-up was not sufficiently long for changes in work injury statistics to occur.
According to Dennis, the IWH study confirmed the limitations of using workers’ compensation claims data as the only basis for targeting interventions, something the Ministry had already begun to address. It also highlighted how important it is to define what measures will be used to assess the effectiveness of a program in the early design phases of the program.
Dennis also notes the value in having a trusted, independent source for research. The Institute is highly respected for the scientific rigour of its work,
she says. So we trust the independence of its findings about health and safety programs and interventions, and take them to heart when looking at our own work.