What’s Up with CCO’s “Trick” Questions?
The intent of CCO written exams is to reliably determine whether an individual has the required knowledge to practice within the profession competently and safely. But how is that accomplished? It takes a team of load handling subject matter experts and a team of psychometric specialists.
What is Psychometrics?
Psychometrics—coined from the Greek words for “mental” and “measurement”—refers to the field in psychology devoted to testing, measurement, assessment, and related activities. Essentially, psychometrics is the science behind professional testing. It provides guidance on exactly how to best assess knowledge through testing, including how test questions should—and shouldn’t—be asked, what constitutes a good test question, and how to best gauge knowledge on multiple choice tests through answer options.
CCO uses psychometric standards in conjunction with hundreds of load handling subject matter experts to create CCO exams. Subject matter experts determine what should be on the exam, first by completing a job task analysis that outlines the knowledge, skills, and abilities that an individual in a specific role would need. The team then begins to build specific questions and answer options for those questions.
Psychometrically sound questions require the test-taker to read and evaluate all options before answering. Nevertheless, a competent candidate should be able to mentally formulate an answer after reading the question without seeing the options, and then find the answer they formulated on the list.
“Trick” Questions
The goal of the exam is to assess whether the candidate has the knowledge to successfully perform the assigned task. Someone without the required knowledge may think the question is intended to trick them because they believe there may be more than one correct or even no correct answer, forcing them to guess. This is not the case. CCO exams primarily contain multiple-choice questions and have only one correct answer. However, the correct answer cannot be obvious because the question would then not be a good assessment of knowledge.
For instance, if a math assessment test question was:
What is 2+2?
Blue
Red
4
White
Most test takers would obviously choose option C, whether they knew how to do basic arithmetic or not. This would make the question not valid for assessing arithmetic knowledge. However, if answer options were more plausible, this question could be psychometrically sound. This same process is applied to CCO exams.
So, while there is not a single question on CCO exams intended to trick anyone, questions are intended for the candidate to read and understand the question, apply their knowledge, and identify the single correct answer.
