By Trish Ennis CSP, ARM, CRIS
Executive Director, Colorado Safety Association
Managing risk on a large project or energy campus is a critical activity for protecting business assets. Requiring suppliers and contractors to go through a prequalification process is one element of reducing risk. A comprehensive prequalification process will enable an owner to benchmark the financial strength, risk management procedures, and safety performance of a potential contractor against acceptable standards. There are numerous services available to provide these prequalification services to owners, such as ISNetworld, Avetta, PEC Safety, BROWZ, Ariba and many other prequalification systems.
Metrics used to measure safety performance typically include an evaluation of written policies and procedures, training activities, risk management metrics such as the experience modification rate (EMR), and OSHA recordkeeping data. The Total Recordable Incident Rate (TRIR) is the OSHA recordkeeping data used the most widely in the prequalification process. TRIR measures the rate of OSHA recordable incidents per 200,000 worker-hours. Because this rate is an accepted formula by OSHA and the Bureau of Labor Statistics, it is used to compare incident rates across industries and business sectors. In the construction and energy industries the TRIR is used by businesses to measure the past safety performance as well as to predict future expected safety performance of contractors.
What are we actually learning about a contractor, or our own company performance for that matter, by looking at the TRIR? Is this data that can be reliably used to predict the future safety performance of an organization, business unit, team, or project manager?
Recent research by the Construction Safety Research Alliance (CSRA) calls the statistical validity of the TRIR into question. The research report titled “The Statistical Invalidity of TRIR as a Measure of Safety Performance” (Hallowell et al. 2020), describes the results of research that includes 17 years of data and 3.2 trillion worker-hours. Key takeaways from the data revealed that there is no discernable relationship between TRIR and fatalities, the occurrence of recordable incidents is almost entirely random, TRIR is not precise, and in almost every circumstance it is statistically invalid to use TRIR to compare companies, businesses, teams or projects. The researchers found that TRIER is only predictive of possible future performance when over 100 months of data is evaluated. To read the whole report, follow this link: https://www.colorado.edu/lab/csra/sites/default/files/attached-files/the_statistical_invalidity_of_trir_reduced_size.pdf
The results of this research point to the need for new approaches to safety measurement. In order to be used reliably, any measurement must be standardized and consistently applied. In the meantime, evaluating potential contractors on a range of activities, leading indicators, and quality control measures will help to offset over reliance on data that may not be as predictive as we might think.