Update on Web-Based Examinations and Pilot Projects
Date Posted: July 10, 2009
Customized Assessment Services
To meet the need for high-quality standardized assessment materials the NBME has developed a web-based, Customized Assessment Services (CAS) program that permits faculty members to build examinations from secure NBME item banks.
With an annual subscription to the service, schools are able to build and administer any number of examinations. Faculty members log into a secure website to create examination specifications to match their curriculum.
Automated test assembly processes at the NBME then build a draft exam to those specifications and also provide “substitute” items. Faculty members review a draft exam and can replace individual items to improve the fit to course content.
Faculty members also define subscores, as well as assign items to customized score-reporting categories. After an exam has been finalized, it is made available to the school for administration.
Schools are responsible for providing a secure, proctored environment and for assuring that local technical environments meet NBME system specifications. NBME systems “lock down” the computers, preventing examinees from accessing any other programs that might compromise test security. Through an online interface, proctors launch the examination session, monitor the progress of examinees, end the session, confirm the examinee group to be scored, and file electronic reports of test administration.
Preliminary total test percent correct scores are available immediately after the close of the test session. Final score reports, individual examinee performance profiles, and an item analysis in three formats are provided:
In 2008, 21 medical schools administered more than 80 web-based tests to 8,390 students. Faculty at each school designed examinations to meet various evaluation needs. The pool of items numbers about 8,000. Exam lengths varied from 50 to 200 items. The examinations were delivered in a variety of settings, ranging from computer labs with fixed workstations to multiple rooms with students using personal laptops in both wired and wireless environments.