Article: Predicting Student Success on the National Board for Certification in Occupational Therapy® Examination
Authors: Amy L. Kurowski-Burt, SueAnn Woods, Shay M. Daily, Christa L. Lilly, Brian Scaife, Diana Davis
Research Design: Causal-comparative research
Objective: Determine the likelihood of passing the NBCOT® Exam based on characteristics of academic and professional behaviors of student cohorts.
Participants: 315 graduates of a three-year, entry-level MOT Program, eight cohorts from graduation years 2009-2017.
Measures: Researchers looked at the relationship between passing/failing the NBCOT® Exam on the first attempt, demographics, performance on admission criteria, academic coursework, overall academic performance, and Occupational Therapy Knowledge Test scores.
Specifics on Measures:
Demographics: gender, race, in-state student status, and geographic location (metropolitan, micropolitan, small town, and rural).
Admission criteria: ACT scores, SAT scores, high school GPA.
Academic Coursework criteria: one Neuro course, two Physical Disabilities courses, three Level I Fieldwork courses and two Level II Fieldwork courses.
Academic performance: GPA for prerequisite courses, semesterly GPA in MOT Program, GPA while in MOT program below 3.0, final college GPA.
Occupational Therapy Knowledge Test scores: three scores on the NBCOT-created Occupational Therapy Knowledge Exam (OTKE). Test was taken by students at three separate points: (1) at the beginning of occupational therapy program (fall semester, first year) (2) before completing first level II Fieldwork (spring semester, second year) and (3) before completing the final Level II Fieldwork (spring semester, third year).
Impact of demographics on predicting NBCOT® pass rate: slight correlation with gender, but the other variables did not contribute to predicting pass rates.
Results specific to students: There were predictable patterns with student pass rates on the NBCOT® Exam and student performance on outcome measures. Students that passed the NBCOT® Exam performed better in all outcome measures when compared to students that failed the exam. However, there were some interesting patterns that emerged. The overall college GPAs of students that passed the certification exam compared to those that did not pass were not markedly higher. The caveat of the GPA was the average GPA of students that passed was 3.65/4.0 and those that failed 3.54/4.0, both high GPAs. A small difference in performance was also apparent with all three Level I Fieldwork courses when NBCOT® Exam pass and fail results were compared. The greatest differences in pass and fail results was seen with the performance on the first Level II Fieldwork and performance on all three attempts of the OTKE.
Takeaway for students: Collectively, performing well in all academic coursework is important with passing the NBCOT® Exam. Of equal importance is performance on Fieldwork and the ability to show knowledge of occupational therapy with objective tests such as the OTKE.
Results specific to educators: Based on the results of the comparative analysis, the researchers formulated a Success Score Calculator to predict the probability of passing the NBCOT® Exam using the measures that were compared. The Success Score Calculator was based on the following specific measures: ACT score, GPA, first Level II Fieldwork score, and performance on all three of the OTKE exams. Using those measures, researchers were able to establish a cutoff score that predicted with 71.7% accuracy success on the NBCOT® Exam. The Success Score Calculator also has a maximized sensitivity and specificity of eighty-five percent. Case studies were presented and in each of the three case studies, the Success Score Calculator correctly predicted the outcome on the NBCOT® Exam.
Takeaway for educators: Using an objective measure to determine the knowledge base of students is an important component for preparing students for the NBCOT® Exam. A large component of that preparedness is ensuring a strong knowledge base with academic coursework, monitoring performance on a semesterly basis, adequate preparation for fieldwork, and access to tools such as the OTKE to assess overall knowledge of occupational therapy content. The Success Score Calculator is also a good tool that educators can use to establish the probability of a student passing the NBCOT® Exam to determine a plan of student support if a student has a high probability of failing the exam.
Limitations of study: There is no information on the number of students that were not included in the study. To be included in this study, students needed to have taken the NBCOT® Exam. However, there is no information on the number of students that did not take the exam. This number can include students that failed out of the program or those that chose not to take the exam for whatever reason, which would impact results. This was a convenience sample. The sample is homogenous in that they are all from the same program. The results could be due to factors related to programmatic preparedness and not factors inherent to the students. The OTKE is taken three times and is large outcome measure with the Success Score Calculator. The students take the exam three times and testing validity may be a limitation.
The full article can be accessed here on AJOT’s webpage: https://ajot.aota.org/article.aspx?articleid=2765592 AOTA membership is required for full access. Full access to the article would allow for a deeper review of the statistical analysis, specific performance results in each area compared by the researchers, Success Score Calculator makeup and point breakdown, and the case study examples with score analysis of probability of NBCOT® Exam success using the Success Score Calculator.
Citation: Kurowski-Burt, A. L., Woods, S., Daily, S. M., Lilly, C. L., Scaife, B., & Davis, D. (2020). Brief Report—Predicting student success on the National Board for Certification in Occupational Therapy Examination®. American Journal of Occupational Therapy, 74,(4) 1-10. https://doi.org/10.5014/ajot.2020.037622
OT Research Corner 