NBME’s second Creative Community explored ways in which medical schools collect, analyze, and utilize workplace-based assessment data in support of advancing workplace-based assessment and data analysis in medical education.
Workplace-based assessments are designed to evaluate the competencies of medical students in a real-world clinical setting. By observing students’ behaviors, documenting their performance, and providing multisource feedback, these assessments are often used for learning and play a critical role in preparing future physicians for effective patient care.
This project involved aggregating quantitative and qualitative data from select medical schools’ workplace-based competency assessments with a focus on communication, clinical reasoning, and teamwork skills. By combining school data with NBME’s assessment, we hoped to provide deeper insights into learners' competencies, support formative feedback, and inform curricular decisions.
Our data experts created visualizations and explore Natural Language Processing (NLP) applications to support school's undergraduate medical education-level data services for workplace-based assessments. With NBME's robust data infrastructure, secure management practices, and psychometric expertise, we explored the role that we can play in supporting medical schools’ assessment of students in the workplace.
Findings and Next Steps
Through our Creative Community efforts, our findings from this project highlighted many challenges in the collection, storage and use of workplace-based assessment (WBA) data across 5 institutions. Competing purposes and use cases, limited variability, data complexity, and construct coverage issues all lead to challenges in developing assessment data that supports programmatic assessment and student learning. Better understanding of WBA data can help practitioners reflect on their own assessment data collection and use in a variety of contexts and have bettered NBME’s understanding of schools’ data needs. This will shape the ways in which we think about data and reporting as we develop new assessment products.
Learn more about these efforts through a PRESENTATION sharing key learnings and practical ways to improve work-placed base assessments: Connecting programmatic learning objectives with practice: Insights from an analysis of Workforce-Based Assessments.
Manas Das, MD, MS - UMass Chan Medical School
Komal Kochhar, MD, MHA - Indiana University School of Medicine
Christopher J. Mooney, PhD, MPH, MA - University of Rochester School of Medicine and Dentistry
Nitin Puri, MD, PhD - Marshall University Joan C. Edwards School of Medicine
Cayla Teal, PhD, MA - University of Kansas School of Medicine
Stay Up to Date
Join our mailing list to get the latest info and offers from NBME.
