Researching Examinee Performances on NBME® Subject Examinations Administered Remotely During COVID-19
Updated May 16, 2023
This article was originally published on September 16, 2021 and was updated on August 12, 2022.
Given the need for social distancing that arose at the start of the COVID-19 pandemic, NBME psychometricians have continued to analyze data from medical school test administrations to investigate the potential effect of the change to a remote proctoring setting on examinee performance since April 2020. Principal psychometrician, Carol Morrison, PhD, compared results from examinees who took NBME® Clinical Science Subject Examinations with remote proctoring versus traditional in-person administrations during the following time periods:
- April 7 to May 21, 2020
- July 29 to Oct. 31, 2020
- November 1, 2020 to Jan. 31, 2021
- Feb. 1 to April 30, 2021
- May 1 to July 31, 2021
- August 1 to December 31, 2021
- January 1 to March 31, 2022
- April 1 to June 30, 2022
- July 1 to September 30, 2022
- October 1 to December 31, 2022
Results from NBME Subject Examinations (Medicine, Surgery, Obstetrics and Gynecology, Pediatrics, Psychiatry, Clinical Neurology, Family Medicine and Emergency Medicine) were analyzed with a focus on examinees from LCME-accredited schools testing for the first time for end-of-course or end-of-clerkship with standard timing.
The findings: No consistent Subject Examination score differences were associated with proctoring setting when comparing examinees who tested under remote proctoring versus in-person proctoring during the same time period, controlling for group ability based on pre-clinical performance. Groups of examinees who tested remotely had slightly higher scores for some exams and slightly lower scores for others. Additionally, effect sizes for proctoring setting were generally very small, indicating no meaningful difference in performance between the remote proctoring group and the in-person proctoring groups of these low-stakes exams. Examinees from the specified time period were also compared to examinees testing during the same time period in the previous year, and no meaningful differences in performance were found for any of the time periods.