The construction sector has long been identified as one with a relatively high risk of occupational injuries and fatalities. According to a 2010 report by Ontario’s Expert Advisory Panel on Occupational Health and Safety, the construction sector in Ontario experienced more fatalities than any other sector, most of which were caused by falls from heights.
Indeed, the Expert Advisory Panel was created by the province’s Minister of Labour following a workplace tragedy that brought the dangers of working at heights into sharp focus. On December 24, 2009, four construction workers died after falling 30 meters from a suspended work platform that had collapsed. In 2010, at the close of its review of the province’s occupational health and safety system, the Panel made 46 recommendations and placed priority emphasis on the development of mandatory fall protection training for working at heights. In response to the findings, the Ministry of Labour (as it was then known) began consulting with key stakeholders, particularly in the construction industry, about the development of a new fall protection training standard.
The resulting Program Standard for Working at Heights (WAH) Training came into effect in Ontario on April 1, 2015. It applied to workers needing to use a fall protection method on a construction project, typically those working three meters or more above ground in settings unprotected by other means such as guardrails. Affected workers had until October 1, 2017, to pass the new training. A separate standard (the training provider standard) set out certification requirements for organizations intending to deliver the training.
The WAH training standard specified learning objectives, a minimum class length, a mandatory classroom component, and a maximum class size. It also stipulated that trainees pass a knowledge test and a demonstration of skills. The old regulations did require training but did not include these specifications. As a result, before the new standard, working-at-heights training had varied markedly across training providers. It often had been quite brief and not always hands-on.
In light of a commitment by the Ministry’s Chief Prevention Officer to review the new WAH training program standard every five years, IWH saw an opportunity to provide evaluative evidence that would both be useful to the Ministry’s review and make a contribution to the research literature. The evaluation was led by IWH Scientist Dr. Lynda Robson. Data collection began in the spring of 2017.
The IWH evaluation
The IWH evaluation study asked two key questions:
- To what extent has the working-at-heights training reached its target audience?
- What impact has the introduction of the working-at-heights training requirements had on fall prevention on construction projects?
The initial evaluation used multiple sources of data, including: 1) administrative records on training activity from the Ministry of Labour; 2) interviews with Ministry of Labour inspectors; 3) results of a knowledge test given before and after training; 4) surveys of training providers, employers and workers (learners) who took the training provided by the Infrastructure Health & Safety Association (IHSA); and 5) lost-time claims data (data on work injuries that require time away from work) from the Workplace Safety and Insurance Board (WSIB). The learner survey was conducted at three intervals: at one week after the training, at four weeks after, and at seven weeks post-training.
The findings, contained in a 2019 report, indicated that the program standard for WAH training was implemented as intended. Most of the surveyed training providers reported that the new training was longer, gave exposure to a wider variety of equipment and offered more practical hands-on training than before the new WAH training standard was in place. Among construction employers with six or more employees, 92 per cent were found to be compliant with the requirement to ensure that affected workers were trained by the deadline of October 1, 2017. By the transition deadline, about 420,000 Ontario-based learners had successfully completed the training.
The IWH study also found evidence that the training improved knowledge, use of safe practices, and safety outcomes. The average score on the knowledge test rose from 6.8 out of 10 in pre-training tests to 9.5 post-training. The learner survey showed improved compliance with WAH safety practices four weeks after the training compared with pre-training practices. It also identified learners’ interest in more practical instruction on the use of ladders. A comparison of lost-time claims data for 2017 with that for 2012-14 found a statistically significant impact of the WAH training intervention on falls in the construction sector targeted by the intervention.
Beginning in 2019, Robson and her team conducted a follow-up study. This included a survey of learners (from the initial study) two years after completion of the training, which incorporated another knowledge test, and an updated examination of workers’ compensation claims data. The results on the knowledge test had slipped to 7.5 out of 10, but remained higher than before the training. The knowledge items that had eroded the most tended to relate to questions about regulation and policy. On questions related to on-the-job knowledge, such as what types of ladders to use or what “bottoming out” means, the scores indicated that knowledge improvement was retained. Based on the survey, there was no slippage or erosion of the improvements in safety practice.
Robson’s team also compared claims data from 2017 to 2019 with that from the pre-intervention period of 2012-2014 in Ontario, as well as with claims data from other Canadian provinces. Other provinces, which had seen no change in regulations for fall protection training over the same period, were used as a comparison group. Whereas Ontario saw a 19 per cent decline in the incidence rate of fall injuries targeted by the training, that incidence rate in the other nine provinces fell by only six per cent, suggesting that the Ontario program was effective in reducing the targeted falls. A similar decline in fatalities arising from falls was seen in Ontario too.
Impact of the evaluation
On May 17, 2023, the now renamed Ministry of Labour, Immigration, Training and Skills Development (MLITSD) released an updated WAH Training Program Standard, to take effect on April 1, 2024. Key changes include:
- additional learning outcomes on ladders, skylights and damaged equipment;
- additional required personal protective equipment;
- elimination of duplicative language and learning outcomes; and
- enhancement of language to foster more inclusive engagement.
According to officials at MLITSD, the IWH evaluation of the WAH training standard was valuable in several ways. One was to confirm the effectiveness of the training standard. The IWH evaluation provided an important validation of the Working at Heights Training Standard,
says Annie Siddiqi, former Manager of the Training and Awareness Unit in the Prevention Division. It indicated that the training was effective in enhancing knowledge, improving safety practices and reducing injuries. The evaluation, along with our stakeholder consultations, showed that while some improvements to the standard could be made, it was fundamentally sound.
Another contribution of the study was to help make the case to stakeholders of the value of the training. The new program required a substantial investment on the part of employers as well as training providers,
says Jules Arntz-Gray, former Director of the Training and Awareness Branch in the Prevention Division and currently the Director of the Occupational Health and Safety Branch of the Ministry’s Fair, Safe and Healthy Workplaces Division. A key question was, was it worth it? The IWH evaluation showed that the new program yielded valuable results. In particular, the reduction in fall injuries targeted by the training was sizeable.
The Ministry also considered the IWH report on the initial evaluation, in particular the suggestions from provider and learner surveys about improvements to the WAH curriculum, in the development of the modifications to the standard announced in May 2023. For example, the report noted that the most frequent suggestion from learners concerning technical matters was about wanting more practical instruction on the use of ladders. The updated standard strengthens the expected learning outcomes in this area.
What’s more, the IWH study helped reinforce the importance of program evaluation at the Ministry. The IWH evaluation of the Working at Heights Training Standard is a good example of how program evaluation can help ensure that programs are designed and implemented in a way that delivers results,
says Monica Bienefeld, Acting Director of the Analytics and Evidence Branch in the Prevention Division. Arntz-Gray echoes this sentiment. The Ministry is now giving more attention to measurement and evaluation,
he says. The IWH study was influential in this culture shift. We point to it often.
The Ministry wasn’t alone within Ontario’s prevention system in recognizing the value of the IWH study. IHSA, which has played a key role in the delivery of the training and was a partner in the evaluation research, has also found the evaluation helpful in confirming the importance of having a detailed training standard in the regulations under Ontario’s Occupational Health and Safety Act. The IWH research, through respected, scientific methods, established improvements in knowledge, practices and outcomes,
says Enzo Garritano, CEO of IHSA. It validated the importance of the training standard. The approach used by IWH could be a model for the evaluation of other training programs.