Inter-Rater Reliability of Western University’s On-Road Assessment

Published: April 1st, 2016

Category: Peer Reviewed Publications

Classen, S., Krasniuk, S., Knott, M., Alvarez, L., Monahan, M., Morrow, S., & Danter, T. (2016). InterRater Reliability of Western University’s On-Road Assessment. The Canadian Journal of Occupational Therapy, 83(5) 317-325. DOI: 10.1177/0008417416663228.

Studies involving rater judgement require that there is significant agreement between all raters. In this study, researchers assessing on-road driving performance sought to establish empirical evidence for the interrater reliability of the Western University’s On-Road Assessment. The interrater reliability of on-road outcomes between a certified driving school instructor (DI) and occupational therapist and certified driving rehabilitation specialist (CDRS) were quantified. Results from the data analysis indicated excellent agreement, values of Kappa above 80%, between the DI and CDRS on the Global Rating Scale (GRS) on two levels (pass/fail) and GRS with four levels (pass, pass with recommendations, fail remediable, or fail), indicating interrater reliability. Therefore, this study shows that two raters of different backgrounds can achieve excellent agreement while assessing via UWO on-road assessment. This study encourages occupational therapists who are working with DIs to test interrater reliability before conducting on-road assessments and making fitness-to-drive decisions.