NBME OPEN ENROLLMENT WORKSHOPS 

Workshop Format

• Synchronous virtual (Zoom) meeting
• 2 hours and 30 minutes session
• 1 hour 30 minutes prework for each participant to complete prior to the online meeting.

Workshop Description

Item-Writing Workshops give you the tools to develop exam questions that more effectively test your examinees’ knowledge. They are designed and administered by a team of internationally recognized experts in the fields of test development, psychometrics, and medical education research.

Whether you’re new to the process of developing exam items or have years of experience, we’ll teach you skills and techniques that you can apply immediately on your own assessments. 

By the end of the workshop, you will be able to:

• Understand practical methods that promote versatility in multiple-choice question (MCQ) writing and use
• Distinguish MCQs requiring simple recall from those requiring critical thinking and application of knowledge
• Identify and correct common MCQ flaws, including irrelevant difficulty and clues that benefit test-wise examinees
• Write, review, and revise MCQs that focus on application of knowledge and cover topic areas in both clinical and foundational sciences

Workshop Format

• Synchronous virtual (Zoom) meeting
• Two-hour session
• No pre-work required for participants

Workshop Description

In this interactive workshop, you will learn the fundamental measurement concepts of reliability and validity. We will cover the importance of identifying the intended purpose and use of an assessment, how an assessment’s intended purpose and use relate to the validity of its test scores, and why score reliability is critical in supporting the claims and inference we make about examinees. You will also learn approaches to assess reliability and collect validity evidence throughout the assessment design and evaluation process.

By the end of the workshop, you will be able to:

• Articulate the importance of identifying the intended purpose and use of an assessment
• Define the measurement concepts of reliability and validity
• Explain how reliability and validity are related and why they are important to the quality of an assessment
• Describe different approaches to assess score reliability
• Identify and gather relevant evidence to support score use

Workshop Format

• Synchronous virtual (Zoom) meeting
• Two 2-hour sessions
• No pre-work required for participants

Workshop Description

In NBME’s Item-Writing Workshops, participants learn how to write, review and revise multiple-choice questions (MCQs) to assess clinical and foundational sciences. But how do we know that the MCQs we have written are behaving as intended? Are they easy or hard? Are they differentiating between examinees who perform well on the overall test from those who perform poorly? How do we know whether it is okay to include an MCQ on an exam form?  These are questions that an item analysis can help answer. In this workshop, you will learn how to calculate and interpret item statistics for MCQs and how to use them in the assessment development process. 

Specifically, this goal of this workshop is to help you understand:

• The importance of the intended purpose and use on an assessment
• What it means for a test (score) to be reliable and valid
• How to calculate and interpret common item statistics, such as item difficulty and discrimination
• How to interpret common reliability indices
• How item analysis results can be used in the assessment development process