Review finds low quality in workplace economic evaluations

Institute scientists are conducting a systematic review to assess the quality and quantity of evidence on economic evaluations of interventions for occupational health and safety (OHS). A pilot review completed in 2005 found only a handful of studies, and the quality was generally quite low.

Very few occupational health and safety studies have also undertaken an economic analysis, says Emile Tompa, a Scientist and economist at the Institute for Work & Health. After systematically screening a subset of studies in this field, researchers found only 23 relevant studies. Just 11 of the 23 were full economic evaluations that considered both the costs and consequences of the intervention. The rest were partial evaluations that only considered the savings of a given intervention, and not the costs. The interventions included ergonomics, participatory ergonomics and return-to-work interventions in settings as diverse as offices, warehouses, hospitals and manufacturing companies.

A lot of the studies were not undertaken by economists, says Claire de Oliveira, a PhD candidate in economics who is involved in the review. This may explain why the studies lacked the depth of a full economic evaluation. In addition, in most studies researchers did not consider multiple viewpoints or question the figures provided to do the analysis. Also, few of the studies lasted long enough to see if the programs could be sustained. There was also not enough information on the context and on the way things were measured. This made it difficult for researchers to evaluate the quality of the studies, or for a reader to be able to assess the intervention’s applicability in other settings.

As with other systematic reviews, researchers met with stakeholders early in the planning stages to help refine the review questions, search strategy and analysis plan. Stakeholders included individuals from the Workplace Safety & Insurance Board, Workers Health & Safety Centre, Ministry of Labour, Ontario Service Safety Alliance, the University of Waterloo and Dofasco. We want to ensure the product is something that they will use, says de Oliveira. Once the review is complete, likely by the fall, stakeholders will meet again to discuss the findings. In addition to preparing a report summarizing their findings, the researchers will produce a document outlining best practices in the use of economic evaluation methods in OHS intervention studies.

Source: At Work, Issue 45, Summer 2006: Institute for Work & Health, Toronto