RESEARCH LIBRARY

Diagnosis: Volume 10, Issue 1, Pages 54-60

This op-ed discusses the advantages of leveraging natural language processing (NLP) in the assessment of clinical reasoning. It also provides an overview of INCITE, the Intelligent Clinical Text Evaluator, a scalable NLP-based computer-assisted scoring system that was developed to measure clinical reasoning ability as assessed in the written documentation portion of the now-discontinued USMLE Step 2 Clinical Skills examination. 

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - p 2880–2886

This paper presents a corpus of 43,985 clinical patient notes (PNs) written by 35,156 examinees during the high-stakes USMLE® Step 2 Clinical Skills examination.

Academic Medicine: March 2019 - Volume 94 - Issue 3 - p 314-316

The United States Medical Licensing Examination Step 2 Clinical Skills (CS) exam uses physician raters to evaluate patient notes written by examinees. In this Invited Commentary, the authors describe the ways in which the Step 2 CS exam could benefit from adopting a computer-assisted scoring approach that combines physician raters’ judgments with computer-generated scores based on natural language processing (NLP).

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health, 19(1)

This review is a descriptive summary of the development of National EM M4 examinations, Version 1 (V1) and Version 2 (V2), and the NBME EM Advanced Clinical Examination (ACE) and their relevant usage and performance data. In particular, it describes how examination content was edited to affect desired changes in examination performance data and offers a model for educators seeking to develop their own examinations.