Impact of Variability in Job Coding on Reliability in Exposure Estimates Obtained via a Job-Exposure Matrix

Abstract

Abstract Objectives The use of a job-exposure matrix (JEM) to assess exposure to potential health hazards in occupational epidemiological studies requires coding each participant’s job history to a standard occupation and/or industry classification system recognized by the JEM. The objectives of this study were to assess the impact of inter-coder variability in job coding on reliability in exposure estimates derived from linking the job codes to the Canadian job-exposure matrix (CANJEM) and to identify influent parameters. Method Two trained coders independently coded 1000 jobs sampled from a population-based case–control study to the ISCO-1968 occupation classification at the five-digit resolution level, of which 859 could be linked to CANJEM using both assigned codes. Each of the two sets of codes was separately linked to CANJEM and thereby generated, for each of the 258 occupational agents available in CANJEM, two exposure estimates: exposure status (yes/no) and intensity of exposure (low, medium, and high) for exposed jobs only. Then, inter-rater reliability (IRR) was computed (i) after stratifying agents in 4 classes depending, for each, on the proportion of occupation codes in CANJEM defined as ‘exposed’ and (ii) for two additional scenarios restricted to jobs coded differently: the first one using experts’ codes, the other one using codes randomly selected. IRR was computed using Cohen’s kappa, PABAK and Gwet’s AC1 index for exposure status, and weighted kappa and Gwet’s AC2 for exposure intensity. Results Across all agents and based on all jobs, median (Q1, Q3; Nagents) values were 0.68 (0.59, 0.75; 220) for kappa, 0.99 (0.95, 1.00; 258) for PABAK, and 0.99 (0.97, 1.00; 258) for AC1. For the additional scenarios, median kappa was 0.28 (0.00, 0.45; 209) and −0.01 (−0.02, 00; 233) restricted to jobs coded differently using experts’ and random codes, respectively. A similar decreasing pattern was observed for PABAK and AC1 albeit with higher absolute values. Median kappa remained stable across exposure prevalence classes but was more variable for low prevalent agents. PABAK and AC1 decreased with increasing prevalence. Considering exposure intensity and all exposed jobs, median values were 0.79 (0.68, 0.91; 96) for weighted kappa, and 0.95 (0.89, 0.99; 102) for AC2. For the additional scenarios, median kappa was, respectively, 0.28 (−0.04, 0.42) and −0.05 (−0.18, 0.09) restricted to jobs coded differently using experts’ and random codes, with a similar though attenuated pattern for AC2. Conclusion Despite reassuring overall reliability results, our study clearly demonstrated the loss of information associated with jobs coded differently. Especially, in cases of low exposure prevalence, efforts should be made to reliably code potentially exposed jobs.