This is one school’s experience with NBME assessments. Results or outcomes may vary and depend on factors beyond NBME’s control, such as student participation, individual study habits, faculty support and engagement, and/or variations in curriculum, among other things.
When Kansas College of Osteopathic Medicine (KansasCOM), part of Kansas Health Science University, welcomed its inaugural class in 2022, the institution faced a challenge shared by many new medical schools: there was no historical student performance data, established assessment framework, or legacy curriculum metrics to guide decision-making. Faculty, learning specialists, and institutional leaders needed reliable ways to evaluate student performance, assess curricular effectiveness, and understand learner readiness for key milestones in the curriculum. Assessment and benchmarking resources in the foundational biomedical sciences have continued to expand. As part of its assessment strategy, KansasCOM intentionally uses multiple external benchmarking resources, including osteopathic and NBME measures, as complementary sources of evidence to support student assessment, curricular evaluation, and continuous quality improvement.
To address these challenges, KansasCOM adopted NBME Customized Assessment Services (CAS) in 2024. The school sought a testing solution that would align with its biomedical science curriculum while also providing access to national benchmarking data. As a new osteopathic medical school, KansasCOM needed a way to better understand how students were progressing and whether they were on track for success on COMLEX Level 1. CAS enabled faculty to build assessments that aligned with their curriculum while leveraging national performance data to gauge student readiness. The ability to compare performance against established benchmarks provided valuable insight into both individual learner progress and curriculum effectiveness. Faculty found the platform's validity, reliability, security, and robust reporting capabilities particularly valuable as they developed and refined their assessment program and prepared students for COMLEX Level 1 success.
As KansasCOM's assessment program matured, its collaboration with NBME expanded. They recently participated in a pilot of INSIGHTS℠ for institutions, which brings NBME assessment results together in one place, helping medical schools more easily review learner performance and prepare for advising conversations. Working alongside NBME, KansasCOM provided valuable feedback that helped shape the user experience. For learning specialists and academic support teams, easier access to a learner's NBME assessment history can help them recognize opportunities to offer support earlier and bring assessment evidence into learner conversations. By facilitating greater access to learner performance data, INSIGHTS℠ for institutions supports KansasCOM's commitment to student success, COMLEX Level 1 preparedness, and continuous curricular improvement.
NBME CAS has become an integral part of our preclinical assessment. The detailed, discipline-specific reporting has been instrumental in evaluating our performance against national data. I look forward to the INSIGHTS℠ for institution as well as the student performance feedback. This will enable students to have increased understanding of their areas of opportunity to enhance their success.
Jennifer Dennis, PhD, MS, Associate Dean for Pre-clinical Education
At KHSU–KansasCOM, NBME Customized Assessment Services (CAS) testing has served as an independent validation of educational quality. National benchmarking has provided objective evidence to inform curriculum review, support accreditation, and guide continuous quality improvement initiatives that have strengthened both student success and program effectiveness. This collaboration continues as we build on these insights to further enhance our curriculum, improve student outcomes, and ensure our graduates are prepared to meet the highest standards of medical education and patient care.
Rosemary Niedens, MA, Associate Dean of Assessment and Outcomes
NBME CAS exams have become a staple in our curriculum. They provide an excellent way to administer summative exams with historical data and performance. As a member of the assessment team, the data that CAS exams provide helps us make informed decisions about students and the allocation of resources. Getting to work with the NBME development team on the new INSIGHTS℠ for Institution was an absolute joy. We are excited for this development and its eventual roll out.
Scott Mudloff, MSEd, Education Director for Testing Services and Data Management, Assistant Professor
NBME Portfolio Parts Used
KansasCOM's assessment strategy incorporates Customized Assessment Services (CAS) exams in their preclinical curriculum.
Supporting Evidence
Keltner, C., Haedinger, L., Carney, P.A. et al. Preclinical Assessment Performance as a Predictor of USMLE Step 1 Scores or Passing Status. Med.Sci.Educ. 31, 1453–1462 (2021). https://doi.org/10.1007/s40670-021-01334-7
Puri N, McCarthy M and Miller B (2022) Validity and Reliability of Pre-matriculation and Institutional Assessments in Predicting USMLE STEP 1 Success: Lessons From a Traditional 2 x 2 Curricular Model. Front. Med. 8:798876. doi: 10.3389/fmed.2021.798876
More to Discover
Join our educator community, explore assessment research fellowship opportunities, and see how your peers are innovating assessment best practices.
