INNOVATIONS IN ASSESSMENT

Research into new innovations in assessment offers solutions to the most pressing challenges facing the medical education community, with a focus on enhancing the formative assessment of knowledge, skills, behaviors and attitudes for learning.

two researchers standing at a desk

About Innovations in Assessment

As the health care landscape continues to evolve, new and innovative assessment methods are needed to measure the competencies that are essential to patient care.

Our research team supports the development of assessments that not only provide formative feedback to health care professionals on their educational journey, but also go beyond testing knowledge using multiple-choice questions by measuring skills and behaviors in formats that mirror real-life patient scenarios.

Additionally, our psychometricians and data scientists conduct research and engage the medical education community to find solutions to challenges related to test development, exam scoring and more.

flame made up of small circles

Reassess the Future

The Reassess the Future campaign brings together the medical education community to explore innovative and inclusive assessment tools to help future generations of physicians develop not only the knowledge, but the skills and behaviors needed to treat patients.

Learn More

NBME in Action

Discover how we’re working to drive innovations in assessment by viewing our current initiatives below.

Automated Item Generation

After running an initial proof-of-concept pilot in early 2020, NBME trained 38 subject matter experts from our test development committees on creating Automated Item Generation (AIG) models, which led to the generation of questions that were pretested on four NBME® Clinical Science Subject Exams and the Health & Wellness Coach Certifying Exam. Item analyses indicate the AIG questions are performing at least as well as traditionally written items.

The success of this pilot inspired the creation of a new AIG tool specially tailored to NBME test items called IMAGE (Item Modeling and Automated Generation Engine), which is currently being used to generate questions for additional NBME exams.

Natural Language Processing (NLP)

To help advance the area of clinical text scoring, NBME sponsored a Kaggle competition on the topic of Natural Language Processing (NLP). Data scientists from all over the world were invited to develop an automated method to map clinical concepts from an exam rubric (e.g., “diminished appetite”) to various ways in which these concepts are expressed in clinical patient notes written by medical students (e.g., “eating less,” “clothes fit looser”).

Nearly 1,500 teams enrolled and the May 2022 results remain open source so others can benefit and continue developing shared value for the community.

Video-Based Communication Assessment

The Video-Based Communication Assessment (VCA) from NBME is a unique formative assessment tool designed to improve physicians’ patient-centered communication skills. Using an app that’s accessible on both desktop and mobile devices, physicians record their spoken responses to a series of vignettes that simulate real-life telemedicine appointments.

A pool of online raters then listens to the recorded responses and evaluates their effectiveness. Physicians receive feedback, including ratings and examples of highly-rated responses, helping them to further develop their communication skills.