News & Events
Latin America Grants Program Funds Development Opportunities
In support of NBME’s mission, the Latin America Grants program was created in 2017 to give institutions outside of the U.S. and Canada the opportunity to improve their assessment tools through funding and counsel from NBME. Each grant provides up to $50,000 over a two-year period to foster the development of a regional health professions school evaluation consortium.
As a requirement for receiving funding for the second year, the three consortia selected for the 2018-2020 cycle submitted reports of their work, including progress, unexpected outcomes, and next steps, after one year. M. Brownell Anderson, Vice President of Medical Education Global Initiatives at NBME, is pleased to share that all consortia are making great progress in their projects and none have requested extensions or changes to their initial goals.
Here are some highlights regarding each of the funded projects.
Teacher Development for Programmatic Evaluation of Students in Health Degree Courses in Brazil submitted by a consortium of six Brazilian medical schools
This work focuses on creating a critical mass of trained teachers at the participating schools and degree courses to develop projects that will result in the implementation of programmatic student assessment, institutional responsibility, and centralized coordination. There is an emphasis on the formative assessment of clinical abilities, sustained by actual teacher development initiatives.
By the end of 2019, principal investigators, Drs. Valdes Bollela and Luiz Troncon, will have conducted 11 highly interactive workshops and trained 396 faculty from four different states in Brazil. These workshops introduce participants to a range of assessment tools and techniques as well as general assessment principles. The workshops provide participants with actual tools they can employ when returning to their “day jobs.” Drs. Bollela and Troncon have organized the workshops so that multiple faculty at a single school participate, which reinforces the likelihood that changes will be made to school-wide assessment programs.
MOOC Design for Training Teachers in Educational Assessment in Health Sciences submitted by a consortium of three medical schools in Mexico
The goal of this project is to design, create, pilot, and implement a Spanish MOOC (Massive Open Online Course) for educational assessment in health sciences on the Coursera platform. The project prioritizes the assessment of clinical proficiency and, in the process, training 30 teaching physicians from three Mexican universities.
During the first year, the consortium held meetings and workshops to define the contents of the MOOC and begin development. NBME staff member Miguel Paniagua, MD, presented two days of workshops on clinical assessment using technology platforms. Fifteen professors from the consortium were trained and have developed the MOOC content. With the support of the Coursera production team, Promotional and “Welcome to the Course” videos were recorded with the project leaders, as well as an interview with Dr. Paniagua, to be included in the MOOC course as complementary material. In January 2020, the course will be available to the public.
Assessment of Abilities and Skills by Direct Observation in Medical Internship submitted by two medical schools in Brazil
The primary goal of the project is to develop a continuous recording methodology for abilities and skills acquired over the course of the internship. There are four specific goals: 1) To create a roster of abilities and skills to be recorded in a common manner in the medical courses involved in the project. 2) To train internship teachers and supervisors to use the same criteria in certifying the acquisition of abilities and skills, with the ability to provide qualitative feedback to the student in each evaluation. 3) To create a secure recording system that can be shared by all institutions and accessed remotely. 4) To evaluate the results obtained.
During the first year, the consortium had fourteen working meetings with the professors and students to discuss the best strategies to create a matrix of skills and assess students’ skills during internship by direct observation. Additionally, two open courses were held for all professors who are involved with the internship to introduce the faculty to the topics considered most important, disseminate ideas, and discuss work proposals with the faculty. The consortium members were able to produce instructional material for the training and certification of reliability activities of students.
M. Brownell Anderson is excited to see what 2020 will hold for the grant recipients. She said, “The work they have accomplished with $25,000 this past year has been exceptional. They each have used the funds as ‘seed’ money to develop workshops that include dozens of faculty members or for project development. The tools they have created can be used for many schools beyond the consortium grantee schools.”
The 2020-2022 grant cycle has been announced and information can be found here.
2019 NBME Staff Publications and Presentations
How can innovative tools and item types be used to improve the authenticity of a medical licensure exam? How have electronic health records been used in the medical education of students during clerkships? NBME staff recently explored these and other questions through research papers and presentations.
NBME staff conduct, publish, and present research on a wide variety of assessment and medical education topics. So far this year, NBME staff have authored 9 publications and delivered 34 presentations at 10 different venues.
Some of the research topics of this year have focused on:
- Score interpretation
- Natural language processing
- Test speededness
- Rapid item generation
- Innovative tools and item types
To see all of the research topics that have been published and presented by our researchers, you can view the NBME Staff Publications and Presentations from January - June of 2019 on the Office of Research page.
Discussing Crucial Issues Facing Today’s Medical Regulators
The International Association of Medical Regulatory Authorities (IAMRA) is hosting an invitation-only symposium in Chicago September 9-10, 2019. The event will explore balancing assurance and improvement of continued competency. NBME is proud to co-sponsor the forum to help bring together leading experts in medical professional regulation from around the world to discuss key issues facing today’s medical regulators.
Medical regulation exists to protect, promote, and maintain the health and safety of the public and IAMRA supports the world’s medical regulatory authorities by ensuring proper standards for medical professionals are met. Through scientific, educational, and collaborative activities, IAMRA strives to encourage best practices among the world’s medical regulatory authorities and to respond to both their current and future needs.
The event is designed to meet the continuing professional development needs of those who have experience or knowledge in developing, implementing, and/or evaluating continued competency interventions, as well as those who are just beginning their career in the field. It is anticipated that attendees will gain critical knowledge to help them fulfill their roles as public protectors.
NBME serves the health of the public through state-of-the-art assessment of health professionals, and is dedicated to research and development in evaluation and measurement. The organization recognizes the role assessment plays in supporting the inferences made by regulatory authorities.
In addition to NBME, the symposium is also sponsored by the Accreditation Council for Graduate Medical Education (ACGME), the American Board of Medical Specialties (ABMS), and the Education Commission for Foreign Medical Graduates (ECFMG). NBME looks forward to an informative and collaborative event.
New Health Systems Science (HSS) Examination Now Available to All Medical Schools
Faculty can now use the Health Systems Science (HSS) Examination, a high-quality, standardized assessment developed in collaboration with the American Medical Association, as a new tool to support the implementation of HSS curricula and educational initiatives. Together with clinical and basic science instruction, HSS prepares medical students for effective clinical practice during residency and throughout their careers.
Some benefits of using the HSS Exam:
- You will be able to provide students with feedback to evaluate areas of strength and weakness as well as to assess readiness (in HSS content) for residency;
- Faculty can use this as a resource to supplement and evaluate their educational programs; and,
- Schools can measure curriculum effectiveness as well as monitor the impact of curricular interventions.
The fee for this exam is the same as the clinical science subject exams; however, as a special introduction, there will be no fee charged for schools that order and administer the exam before November 15, 2019.
For more information about the new HSS Examination, including availability, content, purpose, length, cost, score reporting and research components, please review the HSS At-A-Glance and Frequently-Asked-Questions.
If you have any questions about ordering the examination, previewing content, or obtaining score reports, please contact the NBME Subject Examination Services team at email@example.com.
NBME Activities at the Association for Medical Education in Europe (AMEE) 2019 Annual Conference
August 24-28, 2019
Will you be attending the 2019 AMEE Annual Meeting in Vienna, Austria? Join us at our booth (#N29) to see how you and your colleagues can benefit from exciting enhancements to our products and services including International Foundations of Medicine (IFOM) exams, subject exams, Customized Assessment Services (CAS), and self-assessments, as well as our consultation and faculty development workshops.
New Ambulatory Care Subject Examination Is Available
A new subject examination for Ambulatory Care began administration on August 1 and is now available for medical school faculty. The Ambulatory Care subject examination replaces the former Adult Ambulatory Medicine subject examination.
Staff from NBME collaborated with members of the Clerkship Directors in Internal Medicine (CDIM) Task Force to create the new Ambulatory Care examination. The CDIM Task Force selected all of the test items to ensure the content was relevant and guided by the ambulatory portion of the CDIM curriculum.
“The Ambulatory Care subject examination was refreshed with extensive input from the CDIM members with the goal to create something relevant to clerkship students on ambulatory rotations,” said Michael Elnicki, MD, Director of International Medical Education Programs and Combined Ambulatory Medicine Clerkship. “We use the exam at the University of Pittsburgh for our ambulatory clerkship and are pleased with it.”
“NBME has been collaborating with external stakeholders more than it ever has in the past,” said Marie Maranki, Product Manager for NBME’s subject examinations. “We’re listening to feedback and making efforts to improve our products to better meet their needs.”
Some key features of the new Ambulatory Care examination include:
- New score categories are reported from the refreshed content.
- Equated percent scores are reported to align with score reporting for the other clinical science subject examinations.
- A score equivalency table has been provided on the NBME Services Portal (NSP) to help medical school faculty translate scaled scores from the former Adult Ambulatory Medicine examination to equated percent correct scores on the new Ambulatory Care examination.
NBME staff are conducting informational webinars for medical school faculty to learn more and ask questions about the new Ambulatory Care examination on September 18 (register here) and October 2 (register here).
In addition to the webinars, for more information regarding the Ambulatory Care subject examination, please contact the NBME subject examination team at firstname.lastname@example.org.
New Comprehensive Subject Examination Score Reports Align with USMLE
The comprehensive subject examinations, including the Comprehensive Basic Science Examination (CBSE) and Comprehensive Clinical Science Examination (CCSE), began using new score scales and score feedback reports that align with the redesigned format for the United States Medical Licensing Examination® (USMLE®) score report on August 5.
“The new comprehensive subject examination score scales and score reports provide medical school students with the information that will help students gauge their readiness for USMLE and facilitate correct score inferences,” said Carol Morrison, a Principle Psychometrician at NBME.
An example of the new score report can be seen here.
During the initial work on the redesign of the USMLE score reports, which the new comprehensive subject examination score reports mirror, NBME staff addressed feedback from examinees and medical schools and aimed to provide more useful information in a way that was user-friendly. In response to that feedback, some of the key features of the new comprehensive subject examination reports show:
- An examinee’s performance compared with recent first-time takers from US and Canadian medical schools
- Performance in individual content areas relative to overall examination performance
- Information regarding the allocation of items across content area
The score feedback reports for the remainder of the subject examinations are now available with an updated and modernized format. These score reports have only minor formatting changes but no content changes at this time.
NBME staff are conducting informational webinars for medical school faculty to learn more and ask questions about the new score scales and score reports in August and September.
In addition to the webinars, for more information regarding the new comprehensive subject examination score report or any other subject examination score feedback, please contact the NBME subject examination team at email@example.com.
Act Local, Think Global
NBME Convenes the Global Medical Education Horizon Group
Over a dozen thought-leaders in medical education gathered at the NBME’s offices this summer to explore how best to support a global culture of assessment. The Global Medical Education Assessment Horizons group aims to share the state of assessment and explore new assessments.
“Staff organized a meeting of leading experts in medical education and assessment drawn from throughout North America and Europe to review significant trends and potential developments in education, training, and practice,” said Peter Katsufrakis, President and CEO. “Several participants were past recipients of the Hubbard Award and Stemmler grants. The meeting was convened to facilitate information exchange and brainstorming between these invited guests and NBME staff in order to identify potential future opportunities to expand the horizons of health care professional assessment.”
Brownie Anderson, Vice President, Medical Education Global Initiatives, said that the meeting began with presentations from the invited guests in order to provide background and the current “state of the art” on topics such as:
- Assessment and simulation
- Knowledge assessment
- Assessment of professionalism
- Validation of licensing exams
- Systems of assessment/big data
- Workplace based assessment
- Assessment in the context of practice
The group members then explored the greatest areas of need and began to discussion possible actions for each.
“The Global Horizons meeting was exciting—for the ideas discussed and the information shared—and because of the engagement of the guests who brought us some new perspectives and who were so willing to spend their time helping NBME think about the future of assessment,” said Anderson. “The discussions with the guests and the NBME staff will help inform our ideas for research, innovation, and strategic directions.”
Score reports meet the needs of examinees and faculty when the correct inferences can be drawn. NBME receives feedback about scoring and uses that feedback as input to its continuous improvement efforts. Award winning NBME psychometrician Carol Morrison recently published a paper with additional NBME staff members Linette Ross, Gregory Baker, and Marie Maranki in Medical Science Educator about this very topic.
“Implementing a New Score Scale for the Clinical Science Subject Examinations: Validity and Practical Considerations" first received praise when it was presented at the American Educational Research Association (AERA) annual meeting in 2017 and was awarded the Established Investigator Award for Division I (Education in the Professions). Carol was presented with the award in April of 2018, with revisions to the paper and publication taking place in June of 2019.
NBME implemented a new score scale in 2015 for the clinical science subject examinations, to better meet the needs of medical school faculty and students. Response thus far has been positive, with users finding the new scale much more intuitive. The hope is that this paper will demonstrate a practical application of this score type, which is seldom discussed in the health professions literature, and will allow others to explore its utility for their examination programs.
You can read the full paper here.
The Conversation Continues: Exploring Possible Changes to USMLE Score Reporting
Preliminary recommendations on USMLE score reporting from the convening organizations of the Invitational Conference on USMLE Scoring (InCUS) are now available online. From now until July 26, 2019 individuals and organizations interested in this United States Medical Licensing Examination® (USMLE®) topic can review the recommendations, hear podcasts about InCUS, sign up for a webinar, and submit comments about the recommendations.
South Africa Pilots Assessment for Physician Licensure
In April 2019, the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners® (NBME®) signed an agreement with representatives from the Health Professions Council of South Africa (HPCSA) that was nearly a year in the making. Specifically, the HPCSA asked FSMB and NBME to help them deliver a high-quality written exam that can be used as a standardized assessment for those international graduates who wish to practice medicine in South Africa.
Like the US, some physicians who desire to practice in South Africa received their education and training in other countries. To practice in South Africa, these physicians must pass a written exam and then a clinical skills assessment.
Discussions with HPCSA covered the physician regulatory system in South Africa, the concept of an examination for physicians seeking to join its registry, and an overview of services potentially available through the FSMB and/or NBME.
To assist in advancing healthcare worldwide, FSMB and NBME presented examples of assessment tools to the HPCSA and other attendees at the August 2017 Association of Medical Councils of Africa (AMCOA) conference in Cape Town, South Africa. Staff from the two organizations then travelled to Ghana in July 2018 to attend AMCOA and meet with African nations about high-quality assessments.
HPCSA selected an assessment currently administered as part of the Post-Licensure Assessment System (PLAS) in the United States, the Special Purpose Examination® (SPEX®). Using SPEX, FSMB and NBME will create a pilot for administration this fall at a Prometric Center in South Africa to a group of specially selected candidates. SPEX is a secure computer-based examination and is an objective, standardized assessment of current knowledge requisite for the general, undifferentiated practice of medicine.
In the future, there is the potential for further discussions with other African nations interested in exploring the feasibility of independent assessment as part of a formal licensing examination.
Medical Educators Meet to Connect, Grow, and Learn
Over 250 medical educators attended the 2nd annual Invitational Conference for Educators on May 14-16 in Indianapolis, Indiana, hosted by the National Board of Medical Examiners® (NBME®). Medical school faculty across the United States are integral to the work of NBME, with over 400 educators yearly writing test items, collaborating and reviewing the test items, and sharing their knowledge about medical school curricula. The invitational conference is an opportunity to support newer medical educators in learning best practices in the realm of written and performance assessments.
Paul Wallach, MD, Vice-Chair of the NBME Executive Board and Chair of the Strategic Educator Enhancement Fund (SEEF), kicked off the conference by inspiring attendees to keep connecting, growing, and learning. Dr. Wallach is also Executive Associate Dean for Educational Affairs and Institutional Improvement at Indiana University School of Medicine. Dr. Wallach has been a volunteer on NBME committees for over 25 beginning as an item-writer and content expert in internal medicine and through this process he learned ways to better assess his own students. In his remarks, Dr. Wallach expressed gratitude for the national faculty of content experts that contribute test items and curriculum expertise to NBME’s examinations.
NBME President and CEO, Peter Katsufrakis, MD, also shared his thanks, and invoked the NBME’s mission, which encompasses the spectrum of health professionals along the continuum of education, training and practice, and includes research in evaluation as well as development of assessment instruments. Dr. Katsufrakis gave an overview of the broad array of services and activities the NBME provides to medical schools and students.
Many faculty members across the country have expressed the desire to learn better ways to create assessments, write items, and evaluate student performance. “Medical students are motivated for mastery,” said Graham McMahon President and CEO of the Accreditation Council for Continuing Medical Education, and keynote speaker at this year’s conference. “That is what makes them the ideal learner.”
He continued, “The skills needed to be a functional and effective physician today require an enormous range of competencies. Our job is to try to give feedback to our learners, so they know where they are and where they have room to grow, and to support them and build their competencies in every dimension that is necessary for their development.”
Miguel Paniagua, MD, Medical Advisor at NBME and organizer of the workshops about written-assessment said that he hopes the impact of the conference will grow year after year, as more faculty members are able to attend.
“This conference allows us to meet new faculty and make new allies in the work of assessment. In the future, this conference will continue to teach state-of-the art assessment techniques. Every year, if we can create a new group of experts, we’ve done something special.”
Nearly 300 medical school faculty to attend 2nd Annual NBME Invitational Conference for Educators
Medical school faculty are critical to the education of future physicians. In addition to teaching, faculty must assess their students’ knowledge to know if students are meeting objectives set for courses and clerkships and if changes are needed to the educational program. The intensity of graduate school and medical school that most medical school faculty experienced leaves little opportunity to learn how to create good assessments. A goal of the National Board of Medical Examiners® (NBME®) is to equip medical educators with all the tools and best practices for writing better school-based assessments for their learners.
For the second year in a row, nearly 300 US medical school educators will participate in NBME’s Invitational Conference for Educators being held this year in Indianapolis on May 15-16. The event will enable faculty to network with each other and learn how to write better assessments through two assessment tracks - Written Assessment Track and Performance Assessment. Both tracks will be led by expert NBME staff to furnish participants with the knowledge and current techniques for building new skills.
Last year’s conference key-note speaker, Alison Whelan, Chief Medical Officer at the Association of American Medical Colleges (AAMC), said “When assessment is done right, it is helping students to see their progress and become independent physicians.”
She also noted “Why do we assess in medical education? It’s for our learners. It’s for our programs- are we meeting our goals? It is for others. We assess so our students can get licensed. And most importantly, we do it for the public. We have to show that the people we are training to care for them have the competence and the abilities to do what they want to do. “
At the conclusion of last year’s event, many participants expressed the desire to volunteer with NBME in the future and felt “motivated and excited to share with my colleagues and to put what was learned into action.” This year’s conference aims to make the same impact by highlighting the value of assessment and deepening the knowledge and skills of the community.
NBME is committed to providing resources and benefits to the medical education community in recognition of the valuable contributions of the faculty who give their time and energy to our products and services. More than 400 individuals serve on a United States Medical Licensing Examination® (USMLE®) committee, panel, or task force each year. While NBME has had item-writing workshops and educator guides available for several years, leaders and staff wanted to do more. In 2017 NBME’s Executive Board endowed the Strategic Educator Enhancement Fund to sustain annual conferences and other activities to help NBME engage with, and support, the medical education community. We are grateful for the extraordinary commitment and dedication of faculty members in advancing NBME’s mission.
To see more of the resources that NBME has for medical educators, click here.
New Fellowship Announced for Medical Education Research
The National Board of Medical Examiners® (NBME®) is proud to announce a unique research opportunity for medical school faculty who wish to develop their skills in medical education assessment research.
The Strategic Educator Enhancement Fund (SEEF) Medical Education Research Fellowship Selection Committee, sponsored by NBME, is now accepting letters of intent for a specially designed cohort of eight participants. The cohort will bring together like-minded participants with mentors and subject matter experts to develop a research protocol and research program for one or both of the following topics:
- assessment of clinical reasoning/critical thinking and/or
- diversity/bias and disparities in assessment.
The new SEEF Medical Education Research Fellowship holds many benefits for the medical education community including the creation of new assessments that can be used in the continuum of physician learning. The goal of the Fellowship is to provide an opportunity to US medical school faculty who:
- Are interested in developing skills in medical education assessment research
- Have a commitment to working with a team of interested colleagues
- Are interested in assessment-focused scholarship about innovative approaches that will enhance the assessment of individuals across the continuum of medical education and in active independent practices
To learn more about the Fellowship, view our online program information here.
Psychometric Staff Win Data Competition
Two National Board of Medical Examiners® (NBME®) psychometricians have received recognition for their work in data visualization. Richard Feinberg, PhD, and Dan Jurich, PhD, recently won the Educational Measurement Issues and Practices (EM:IP) cover graphic/data visualization competition for the fourth consecutive year with their entry Visualizing Hierarchical Score Inferences.
Psychometricians help ensure validity throughout the process of designing, building, scoring and reporting performance on assessments by being mindful of the inferences that can be made by users of the score information. Their winning graphic illustrates different inferences that may be desired by examinees and how reporting performance within these different contexts can change the interpretations made from the score report. This work reinforces the NBME’s commitment to understanding what’s meaningful to examinees, and using that understanding to create useful score reports that facilitate evidence-based decision making.
EM:IP is one of two prominent journals run by the National Council on Measurement in Education and a leading forum for measurement and psychometric research.
NBME Executive Board Election
The NBME is pleased to announce newly elected and re-elected officers and at-large members of its Executive Board. The election occurred at the annual meeting of the NBME membership on March 29, 2019, in Philadelphia.
The following individuals were elected/re-elected to Executive Board:
- Alfred F. Tallia, MD, MPH: Elected to a first two-year term as chair
- Paul M. Wallach, MD: Elected to a second two-year term as vice chair
- Latha Chandran, MD, MPH: Elected to a first two-year term as treasurer
- Donna D. Elliott, MD, EDD: Elected to a second two-year term as an at-large member
- Marie C. Foley, PHD, RN, CNL: Elected to a first two-year term as an at-large member
- Reena Karani, MD, MHPE: Elected to a second two-year term as an at-large member
- Pat Mastors: Elected to a second two-year term as an at-large member
- Karen M. Sanders, MD: Elected to a first two-year term as an at-large member
- David Milling, MD: Elected to a first two-year term as an at-large member
- Kamili Wilson: Elected to a first two-year term as an at-large member
Suzanne T. Anderson serves as immediate past chair, and NBME president and CEO, Peter J. Katsufrakis, MD, MBA, is an ex officio member of the Executive Board.
Profiles of NBME’s Executive Board members are available here.
2018 NBME Annual Report Available
The 2018 NBME Annual Report: Engagement and Action and 2018 Directory: NBME & USMLE Committees & Volunteers are now available for viewing. Click here to see the report’s interactive format that quickly transports readers to deeper content or directly to online assessment information.
This year’s report spotlights progress achieved through NBME’s engagements and assessment improvements. You’ll also read about updates to programs and services, recognition and grants, and assessment-related research.
Lively infographics offer fast facts about a variety of topics. For more information regarding the topics within, please contact our Director of Communications at firstname.lastname@example.org.
Engaging with Faculty at Item-Writing Workshops
On March 4, 2019, NBME’s Director of Test Development Services (TDS) conducted a workshop for 20 participants at the American College of Obstetricians and Gynecologists (ACOG). For the past 10 years ACOG has reached out to NBME for a spring workshop to train subject matter experts writing items for ACOG’s PROLOG (self-directed learning) and CREOG (residency in-training) assessments.
NBME has developed test items and examinations for licensure and other assessment purposes since 1915. NBME staff members are internationally recognized experts in the field of test development, psychometrics, and medical education research. For over 20 years, members of NBME’s professional test development staff have served as a resource to medical school faculty wishing to improve their item-writing techniques. Additionally, faculty from other health professions (e.g. veterinary, physical therapy, podiatry) reach out to NBME to conduct workshops to improve their assessments.
As part of its goal of collaborating with organizations to develop better assessment tools for learning, measurement, education, and patient outcomes, TDS conducts many off- and on-site domestic workshops throughout the year. In 2018, TDS facilitated 18 offsite workshops, in addition to many onsite workshops for the United States Medical Licensing Examination® (USMLE®) and its client exam programs. For 2019, 16 have already occurred or are planned through mid-year.
The item-writing workshops are designed to help faculty members improve the quality of multiple-choice questions written for their examinations. Through informative presentations and discussions, faculty members are able to explore structural issues relating to constructing quality multiple-choice items for testing of basic science and/or clinical science content. An overview of item formats concentrating on the traditional one-best-answer item formats that require examinees to select the single best response is presented. One-best-answer questions are the most widely used multiple-choice-item format. Several examples of well-constructed one-best-answer items are presented. Each consists of a focused item-stem (e.g., a clinical vignette) and a lead-in question, followed by a series of options with one correct answer for each item. Several examples of technical item flaws are presented and discussed during the session.
Each participant is provided with a printed copy of the manual “Constructing Written Test Questions for the Basic and Clinical Sciences,” which is published by NBME. The manual includes the material that is presented during the session and additional templates that can be used to assist item writers develop items. Copies of this manual can be obtained at www.nbme.org.
The workshop day consists of a short educational session followed by a rewriting exercise. NBME provides workshop participants with a handout of poorly structured items (constructed by our own in-house medical experts); participants rewrite items in small groups, and then present the rewritten items to the full group. The group discusses and provides both content and structural suggestions. The facilitator will also display the NBME version of the rewritten item.
Feedback from the workshops has been very positive. Participants have expressed that they feel that these workshops improve their ability to create their own assessments, recognize test question flaws, and rehabilitate poor questions. Participants are surveyed at the end of the session to assess their knowledge gained and obtain feedback on how sessions can be improved.
You can learn more about how you can have NBME conduct a workshop at your institution here.
NBME Welcomes New Vice President of Test Development
The National Board of Medical Examiners® (NBME®) is pleased to announce that Ken Ridgley joined the organization as Vice President, Test Development in mid-February 2019.
“Ken’s industry knowledge of innovative approaches, processes, and systems is the right complement for the momentum building in NBME’s continuous improvement journey,” said Craig Mills, Senior Vice President. “His desire to support our mission of serving the public health through state-of-the-art assessment makes his selection all the more appropriate.”
Through Ken’s work with a diverse assessment client base, he has gained experience in alternative methods of item development across multiple industries, which will benefit all NBME examinations. His expertise with industry-leading technology solutions for test-item databases, authoring, and exam construction all align with NBME’s objectives to increase value in its current examinations, explore alternative assessments, and discuss new approaches and competencies to serve the healthcare consumers.
Mr. Ridgley was previously Vice President, Test Development for Prometric where he worked for ten years leading their Global Test Center Support function, Operations Command Center and most recently, Test Development. Prior to this, he spent 10 years in progressive leadership roles managing large scale, global teams in process improvement, change management, strategic planning, and organizational development.
Coming Soon: New Comprehensive Basic Science Self-Assessment Forms 20, 21, and 22
Three new practice exam forms (Forms 20, 21, and 22) of the Comprehensive Basic Science Self-Assessment (CBSSA) will be available on MyNBME on March 25, 2019. Two additional new forms (Forms 23 and 24) will be released in late spring 2019. The performance feedback provided will remain the same. These updates are part of NBME’s annual operations to provide new content.
CBSSA Forms 13, 15, 16, 17, and 19 will be retired when the new forms are released. CBSSA Form 18 will continue to be available for purchase. Participants who purchased CBSSA Forms 13, 15, 16, 17, or 19 prior to March 25 will still have 90 days from date of purchase to launch and complete their self-assessment. For example, if you purchased Form 19 on March 24, the self-assessment will be available to you until June 21, 2019.
NBME Educator Guides Now Available on Test Blueprinting and OSCE Topic Areas
Health profession educators can now learn about assessment creation and delivery through additional NBME Educator Guides now available at NBME.org.
Previously, these lessons were available for purchase, but now are available for free as "companion" lessons to complement the NBME "Constructing Written Test Questions for the Basic and Clinical Sciences" (sometimes referred to as the Gold Book).
The seven companion lessons cover topics not included in the NBME Gold Book and include content on test blueprinting as well as several clinical skills topic areas.
The NBME believes that medical school faculty should have access to high-quality professional development resources that address quality practices in assessment. To accurately assess student understanding, faculty training resources are very important, especially as it has become common practice for medical students to learn required content from many sources, including outside the classroom.
Commitment to quality assessment is fundamental to us, and we are excited to offer these additional lessons for free. In addition to the Gold Book, these lessons will provide additional education and guidance in multiple areas of assessment. We are pleased to offer these lessons for use in medical education and assessment.
Lessons are available here.
New Website for NBME Self-Assessment Services Benefits Examinees
A new customer website, MyNBME, is now available for examinees who are taking NBME Self-Assessment Services (NSAS) to gauge their readiness for the United States Medical Licensing Examination® (USMLE®), NBME Clinical Science Subject Exams, or the IFOM Clinical Science Examination.
MyNBME was designed based on feedback obtained from students and residents during focus groups and pilot trials. To gather the information, customer journey maps were completed, and experience surveys were deployed to benchmark improvement efforts. Both students and residents told us they wanted to quickly and easily:
- purchase products across different product lines in a single transaction
- view and download score reports
- complete other activities and communications, not only with NBME but also with collaborators
Examinees are encouraged to share their feedback so NBME can keep getting better. If you have any questions regarding NSAS, please contact us here.
CDC grant awarded to NBME Collaborator
The International Consortium for Health & Wellness Coaching (ICHWC), a nonprofit controlled affiliate of the National Board of Medical Examiners (NBME), recently received a grant from the Centers for Disease Control and Prevention (CDC) for a coaching program designed to help prevent diabetes. The organization will partner with the American College of Preventive Medicine (ACPM) to develop the following:
- group coaching competencies
- a training program
- a certificate examination
This work will support dissemination of the National Diabetes Prevention Program lifestyle change program. NBME staff views the collaboration on these high-quality certification examinations as having great impact on the organization’s mission. Learn more here.
Philadelphia READS and NBME reads with it
NBME has always looked for ways to be involved with the local community. In 2008, a staff member suggested the company participate in the Philadelphia READS' "Power Partners" program, which matches socially conscious organizations with children from Philadelphia classrooms for one-on-one literacy coaching sessions that run from October through May of each school year. During weekly visits to NBME, these children receive both academic support and exposure to positive adult role models.
Since that time, NBME volunteers have worked with a number of different schools. Volunteers become reading coaches for the program and are given the opportunity to spend one-on-one time with a child. In addition to reading books, the coaches get to know the child they are partnered with, sharing information about the world, their own lives, and common topics of interest. This one-on-one interaction helps the students to gain confidence as their reading level increases.
"When you have a breakthrough with your reading partner, it is really rewarding," says Jesse Delaney, a senior case developer for NBME. "Sometimes that breakthrough is as simple as finding a topic the student enjoys and tailoring the reading list to that interest. Other times, it is seeing a student flawlessly and effortlessly read a word aloud that he or she struggled with earlier in the year."
Pat Callahan, program support specialist for Philadelphia READS, says that the program allows the children to interact with positive role models and experience authentic volunteerism in the workplace. The students value their time with their reading partners, with teachers noting that attendance seems to be highest on the days when they are scheduled to participate in the program. NBME staff members are equally excited on those days and take great joy in making a difference in a young person's life.
Philadelphia READS is a child literacy nonprofit program with a mission of providing access to books and community programs to foster a love of reading and increase literacy in Philadelphia. NBME is proud to be a part of a program that generates and encourages the love of reading in Philadelphia.
NBME Brings Health Profession Organizations Together for Collaborative Approaches in Continuing Certification
NBME hosted an invitational conference on Collaborative Approaches in Continuing Certification in December for key collaborators.
"The meeting afforded attendees opportunities for connecting with colleagues, deepening the conversation about continuing certification, and affirming the importance of initial and continuing certification in their professions and for the public they serve," said Lisa Rawding, Senior Vice President of Assessment Programs.
Pat Mastors, founder of Patients' View Partners and NBME Executive Board member, opened the event with a rousing presentation to the group. She challenged attendees to consider how to help patients more readily understand the competence of the medical providers that they are choosing. Using examples from around the globe, Pat highlighted the opportunity for healthcare professionals to take ownership of communicating the value of certification.
The main program included presentations and small group activities designed to explore goals, challenges, and opportunities in continuing certification. Representatives from allied health, physician, and veterinary communities worked together to explore key themes raised during the focus group discussions. For many it was the first opportunity to explore these questions with colleagues from different specialties. "We heard resoundingly positive feedback from participants about how valuable it was to hear the experiences of colleagues across varied professions," said John Phebus, Assessment Officer.
NBME greatly appreciated the opportunity to provide a forum for this important discussion in support of the health professions we serve. To learn more about NBME's services, please contact Amber Williams at email@example.com.
NBME Collaborates on a New Evidence-Based Interprofessional Professionalism Assessment Tool
The Interprofessional Professionalism Collaborative (IPC) is pleased to announce the publication of its landmark article – The Intersection of Professionalism and Interprofessional Care: Development and Initial Testing of the Interprofessional Professionalism Assessment (IPA). The final published article is now on Taylor and Francis Online and will be assigned to a future issue of the Journal of Interprofessional Care. The IPA instrument was created by the IPC, a national group representing 11 entry-level health professions and the NBME.
"NBME is honored to have collaborated on the Interprofessional Professionalism Assessment and its accompanying tool kit. This work advances assessment of interprofessional professionalism and collaboration among health professionals," said Dr. Peter Katsufrakis, President and CEO of the National Board of Medical Examiners. "We serve the public through state-of-the-art assessment of health professionals, and we remain committed to research and development in evaluation and measurement."
Interprofessional Professionalism is the consistent demonstration of core values evidenced by professionals working together, aspiring to and wisely applying principles of altruism and caring, excellence, ethics, respect, communication, and accountability to achieve optimal health and wellness in individuals and communities.
"The ability of health care practitioners to demonstrate interprofessional professionalism is essential to the delivery of quality care to patients, families, communities, and society," said Dr. Jody Frost, Education Consultant and IPC Facilitator. "As profiled in the Journal of Interprofessional Care, the new psychometrically sound assessment instrument developed by the IPC can be applied in both academic and practice settings to cultivate environments that promote interprofessional professionalism and collaborative care models."
The IPA measures observable behaviors of health care professionals-in-training that demonstrate professionalism and collaboration, which are essential to person-centered care. The instrument was created and evaluated over several years through a comprehensive, multi-phasic process, which included the development of a construct and observable behaviors; instrument design, expert review, and cognitive interviews; and psychometric testing.
"As a pharmacist, who worked on multidisciplinary health care teams, I believe the construct of interprofessional professionalism is integral to successful collaboration among health care professionals in providing optimal patient care," said Jennifer Wise, NBME representative on the IPC. "I am proud to be a member of the IPC and believe the IPA and the accompanying Tool Kit will be great educational resources."
The IPA contains 26 items or observable behaviors representing six domains of professionalism (altruism and caring, excellence, ethics, respect, communication, accountability), and was tested by 233 preceptors who rated health profession learners in the final year of their practical training.
In addition to the new journal article, the IPC has created an online tool kit to facilitate interprofessional professionalism, which includes the IPA Instrument, case scenario videos, an on-demand webinar, and related materials. To access these resources, visit http://www.interprofessionalprofessionalism.org.
RENEW Task Force Explores Effects of Stress on Examinees at Recent Session
The Re-Examining Exams: NBME Effort on Wellness (RENEW) Task Force held a successful meeting at the end of October. The task force made progress in fulfilling its charge. To learn more, click here..
M. Brownell Anderson honored with AAMC GEA award in her name
The Group on Educational Affairs of the Association of American Medical Colleges announced a new Award for Emerging Educators. It is named in honor of NBME's M. Brownell (Brownie) Anderson, M.Ed., for her outstanding commitment to the Group on Educational Affairs (GEA).
The purpose of the GEA is to advance medical education and medical educators through faculty development, curriculum development, educational research, and assessment in undergraduate, graduate, and continuing medical education. Brownie began her work with them in 1983 and continued that work for more than 25 years. In that time, in collaboration with many medical educators in all of the schools, she increased the attention to and support for medical education at the AAMC.
The AAMC has a number of affinity groups for faculty and staff of AAMC member medical schools, teaching hospitals; the Group on Educational Affair is one of those groups. NBME President and CEO, Peter Katsufrakis, was the national chair of another of AAMC's affinity groups: the Group on Student Affairs (GSA).
Brownie began working at the National Board of Medical Examiners (NBME) in 2011 and the GEA wants to show recognition for her long-standing work with and commitment to the community of medical educators. They also want to honor her uncanny ability to identify, encourage, and welcome individuals early in their careers to the community of medical education.
The GEA states: In the spirit of M. Brownell Anderson, this award recognizes early career medical educators who have demonstrated enthusiasm, passion, commitment and creativity in the field of medical education. This award identifies and supports emerging educators in their career trajectory in medical education and serves as an opportunity to recognize and validate their potential and talent in the field. The GEA owes a debt of gratitude to M. Brownell Anderson for recognizing and cultivating new leaders for many years, establishing a tradition that lives on through this award.
In Brownie's own words, she expressed "I am overwhelmed, humbled, and a bit terrified by having this award named for me and above all, incredibly grateful. All I did was ask people who inspired me to help me get the work done. I am so fortunate to have been given the opportunities I was given and to have been able to work with and learn from so many incredible people."
Official nomination information and criteria will be made available in early 2019.
NBME Way in Action: Improving Primum
When you have a continuous improvement mindset, a question that occurs often is, "How could this be better?" That's exactly the question Nicole Fortuna, Senior Test Development Analyst, asked herself regarding preparation materials for the Computer-based Case Simulation (CCS) portion of the USMLE Step 3 examination. The result of her question was a new video, which you can view here.
Based on end-of-survey comments, staff found that not all examinees understand how to use the Primum® software on which the Step 3 examination runs. They reported dissatisfaction with the practice materials on the website; for example, while downloadable to PC users, the practice cases were not downloadable to Mac users. Staff began to understand that some examinees were experiencing a disconnect with the software because of a lack of quality educational materials.
After applying continuous improvement principles, Nicole arrived at an idea: to create a video that would support a better understanding of the Primum software.
"When I mapped out possible solutions, the best solution wasn't to get rid of any materials," she said. "It was to provide a more seamless piece of technology that anyone can use anywhere in order to prepare to better navigate the software on exam day."
She hopes that by creating better ways to prepare students, and by sharing this through our social media channels and website, we'll be better able to serve our examinees and to ensure that we are their first stop for information.
NBME CEO participates in AMA initiative to transform residency training
In late October, the American Medical Association (AMA) announced a new "Reimagining Residency" initiative and its committee members who will work to transform residency training and better address the workforce needs of the current and future health care system. The five-year, $15-million grant program will support innovations that provide meaningful and safe transitions from undergraduate medical education (UME) to graduate medical education (GME). The program will establish new curricular content and experiences to enhance readiness for practice and promote well-being along the continuum of medical training.
NBME is in full support of this initiative with Dr. Peter Katsufrakis, President and CEO, stating:
“The AMA’s leadership over the past five years with its Advancing Change in Medical Education initiative demonstrated the power of convening a collaborative of educators committed to enhancing education. This experience naturally leads to considering how best to help transform residency training. The NBME’s critical role in ensuring the qualifications of health care providers aligns fully with the AMA’s support to enhance residency training. I am happy to help guide the development and implementation of this initiative.”
The NBME is an independent, mission-driven organization that provides high-quality assessments for the health professions. We serve the health of the public through state of the art assessment of health professionals, and we are committed to research and development in evaluation and measurement.
NBME Psychometrician Joins ATP Board of Directors
The NBME is pleased to announce that Kimberly Swygert, PhD, Director of Research and Development in Professional Services, has been elected to the 2019-2020 Association of Test Publishers (ATP) Board of Directors.
The NBME is a long-standing member of ATP. According to its website, ATP "is a non-profit organization representing providers of tests and assessment tools and/or services related to assessment, selection, screening, certification, licensing, educational, or clinical uses. ATP members are pledged to promote and advance the integrity of assessment services and products and their value to society; and dedicated to the highest level of professionalism and business ethics within the test publishing community."
"I am delighted and honored to be elected to serve in this role and I'm greatly looking forward to representing the Certification/Licensure field in upcoming discussions of topics, such as new industry standards, recertification, test security, best practices and other important areas of exploration and research." she said.
Dr. Swygert is the co-editor and co-author of the most recent edition (2017) of best practices in item writing for medical educators, Constructing Written Test Questions for the Basic and Clinical Sciences. In addition to her work at NBME, Dr. Swygert has taught graduate courses in biostatistics and psychometrics for Drexel University and the Uniformed Services University of the Health Sciences, where she is currently on an external advisory board.
Andrew F. Farella to Join NBME as Sr. Vice President of Information Technology
Posted: October 18, 2018
Following a nationwide search, the National Board of Medical Examiners® (NBME®) is pleased to announce Andrew F. Farella will become Sr. Vice Present of Information Technology in November 2018.
"Andy's experience and perspective were just what we were looking for, an ideal balance that will enable him to fit in well at the NBME while also bringing new ideas and a fresh point of view to our organization," said Peter J. Katsufrakis, MD, President and CEO of NBME.
Mr. Farella is currently Associate CIO, AVP Business Applications of Children's Hospital of Philadelphia where he spent the last nine years leading teams focused on implementation and support of the patient access, CHOP's web presence (SharePoint, intranet, internet site), custom application/mobile development, financial systems, and enterprise business applications. His experience also includes almost a decade serving as Director, IT Applications North America for Elan Pharmaceuticals.
His scope of strategic leadership will include Application Services, Infrastructure Services, IT Project Management, IT Security, Disaster Recovery and Compliance, IT Continuous Improvement and Contract Management.
More Good News for Medical Students about NBME Self-Assessments
If you are preparing for the NBME Clinical Science Subject Examinations or the IFOM Clinical Science Examination, learning from your mistakes and reviewing those questions you guessed at during practice tests can be just as important as reviewing content. Together, these strategies offer you a great opportunity to fully understand the broad array of concepts required for your test.Read the full article
NBME Psychometrician Recognized
Congratulations to Kimberly Swygert, PhD, Director of Research and Development, for being selected as a Fellow of the College of Physicians of Philadelphia. The Admissions Committee and the Board of Trustees recently announced their decisions, and Dr. Swygert will be inducted via the College Night ceremony at the Mütter Museum on Friday, November 16. Fellows of the College of Physicians of Philadelphia are selected for their scientific contributions to work that advances the understanding of medicine. Kimberly is thrilled to join this distinguished group that already includes two physicians at the NBME – our president, Dr. Peter Katsufrakis; and our Medical Advisor, Dr. Miguel Paniagua.
Dr. Swygert’s work on performance assessments, examinee timing and pacing, examinee repeater behavior, and score reporting has been presented at conferences and published in journals such as Academic Medicine, Advances in Health Sciences Education, the Journal of General Internal Medicine, and the Journal of Educational Measurement. Her chapter on performance assessments in the content of credentialing examinations appeared in 2015 in the updated Handbook of Test Development. She is the co-editor and co-author of the most recent edition (2017) of best practices in item writing for medical educators, Constructing Written Test Questions for the Basic and Clinical Sciences. In addition to her work at NBME, Dr. Swygert has taught graduate courses in biostatistics and psychometrics for Drexel University and the Uniformed Services University of the Health Sciences, where she is currently on an external advisory board.
New Question Formats Available
The Medicine Clinical Science Subject Examination now contains several short-answer questions to closely reflect day-to-day physician decision making. The new items allow examinees to provide short, free-text answers to assessment questions.
Medical schools use NBME subject examinations in the basic and clinical sciences to evaluate the educational achievement of individuals in specific subject areas. The examination results may provide insight into the educational progress of physicians or other health professionals.
The NBME is committed to developing innovative item formats that more closely reflect clinical practice. As part of this commitment, the exam now features up to five short answer questions in a given administration. When making a clinical decision, physicians use their knowledge and training to make decisions based on the clinical information available. Short-answer items are designed to more closely reflect this decision making process.
The Medicine Clinical Science Subject Examinations are administered at both medical schools and Prometric centers. Instructions for practicing with these new item formats are provided in the exam tutorial.
Tutorial instructions and a sample item are available here.
For your reference and convenience, we have provided frequently asked questions.
If you have any questions, contact us at firstname.lastname@example.org
New NBME Presence at the AMEE Conference
NBME staff members are excitedly preparing for the Association for Medical Education in Europe (AMEE) conference later this month. Here’s how we’re showing up in a new way:
- New booth. In addition to making our booth more inviting, we’ve created a question card that features questions from the International Foundations of Medicine (IFOM) self-assessment. Can You Answer A Question correctly?
- New opt-in card. Now that the European Union’s General Data Protection Regulation (GDPR) is in effect, we created an opt-in card so you can receive updates from us.
- New brochure. In addition to information about our international products (IFOM and other customized international examinations), our new brochure includes information about other helpful offerings, including Subject Exams, Customized Assessment Services, NBME Self-Assessment Services, and Item Writing Workshops.
Will you be there? If so, we can’t wait to meet you.
Correct Answers Available for NBME Self-Assessments Beginning July 2
No matter how much you prepare, sometimes the correct answer may not be apparent to you. And later you may not remember the question or the answer you chose.
If you are preparing for a United States Medical Licensing Examination® (USMLE®) Step, learning from your mistakes, or going over those questions you guessed at during practice tests can be just as important as reviewing content. Together, it offers you a great opportunity to fully understand the broad array of concepts required.
The Stemmler Medical Education Research Fund issues its 2018-2019 call for letters
In support of innovation in medical education assessment, the Edward J. Stemmler, MD Medical Education Research Fund from the NBME has issued its 2018-2019 call for Letters of Intent.
The purpose of the Stemmler Fund is to provide support for research and development in innovative evaluation methodologies or techniques, with the potential to advance assessment in medical education or practice.
Applicants should consider how their proposed research provides a departure from previous thought and practice, and advances the field of assessment in medical education. The application deadline is July 15, 2018 Applicants may visit here for more detailed information.
Seminal Research by NBME Staff Announced
NBME staff publications and presentations contribute to the body of scholarship on assessment, explore emerging test constructs, and demonstrate the continuous reliability and validity of our existing examinations.
A comprehensive list of the work created by staff can now be found under Research on NBME.org. From the 1900s through the 2010s, staff members have made great contributions to the seminal research in assessment. The work is presented by decade, with an introduction that delineates the themes that emerged from that time period. Both the publications and presentations made at national and international conferences and meetings are included.
For those contributing to the body of knowledge about assessment, this list provides a useful starting point before doing a deep dive into the literature.
Measurement Experts at NBME Receive National Recognition
Three National Board of Medical Examiners® (NBME®) researchers have received recognition for their influential studies in psychometrics, the science of measurement. Drs. Brian Clauser, Richard Feinberg, and Carol Morrison have been honored with awards for career and research contributions.
Read the full article
New Score Scale for the Emergency Medicine Advanced Clinical Examination
Register Below to Learn More
Beginning with exam administrations on June 1, 2018, the National Board of Medical Examiners® (NBME®) will report scores for the Emergency Medicine Advanced Clinical Examination as equated percent correct scores instead of scaled scores. The new equated percent correct score reflects an examinee's mastery of the content domain and can be used to compare and track school and examinee performances over time and across test administrations.
This new scale will also help to enhance the quality of score reporting and interpretation for the Emergency Medicine exam.
Informational webinars for medical school faculty who want to learn more about the changes are being made available by NBME Psychometricians Linette Ross and Seohong Pak.
Register here to join a webinar. (Sessions will close when they reach capacity.)
You can also send questions to us at email@example.com.
2019 John P. Hubbard Award Call for Nominations
The NBME invites nominations for the 2019 John P. Hubbard Award. This award recognizes individuals who have made a significant and sustained contribution to the assessment of professional competency and educational program development at any level along the continuum of medical education and delivery of healthcare.
The Hubbard Award recognizes significant and sustained contributions to the pursuit of excellence in the field of evaluation in medicine. Recognition of contributions to medicine in curricular development, instructional methodology, or medical training methods is not the primary purpose of the award.
Read the full article
Health and Wellness Coach Certifying Exam Helps Demonstrate Excellence
This June, hundreds of potential health and wellness coaches will take the Health and Wellness Coach Certifying Examination (HWCCE) to demonstrate their competency in helping people achieve their personal goals for health and wellbeing.
National Board of Medical Examiners® (NBME®) leadership and executive leaders from the International Consortium for Health & Wellness Coaches (ICHWC) collaborated on the first national certification for individual health and wellness coaches.
Read the full article
NBME Collaborates with Hospital Sírio-Libanês
After a recent meeting in Brazil to discuss its ongoing collaboration activities with NBME, Hospital Sírio-Libanês (HSL) hosted a symposium with attendees from the student body and faculty of a number of medical schools, government officials, and other healthcare luminaries. The symposium, presented in Portuguese with a simultaneous English translator, provided insight into the NBME/HSL collaboration, the Qualifications in Medicine (QM) exams, and progress toward fostering a culture of assessment in Brazil.
“In the US we have a culture of assessment,” said M. Brownell Anderson, Vice President for International Programs . “We use examinations to make decisions about people’s progress in educational settings, their qualifications to work in a particular field, drive a car, or offer a service.”
Brazil is seeking to build a culture in which healthcare professions can demonstrate the appropriate standard of care.
Read the full article
Employees donate art supplies to Children's Hospital through Lucy Belle's Rainbow
From March 12 through April 16, employees at the National Board of Medical Examiners® (NBME®) collected art supplies for "swag bags" donated to children undergoing treatment at a local children's hospital.
The NBME proudly supports employee involvement in the community through a dedicated "Volunteering at NBME" group, which fundraises for and builds relationships with several local organizations. The NBME also provides employees with paid volunteer hours that employees are encouraged to use to support causes about which they are passionate.
The NBME worked with Lucy Belle's Rainbow for the collection of crayons, sketchbooks, watercolor kits, glue sticks, markers, colored pencils, and other supplies. Lucy Belle's Rainbow is an initiative to bring the power of art and art therapy to children undergoing cancer treatment, in an effort to help children and their families cope with stress. The organization was inspired by the memory of Lucy Belle Perkins "whose love of art and all things beautiful will always exist."
To learn more about Lucy's Belle's Rainbow, including where you can donate or how to become involved, click here.
Education grants awarded to organizations in Brazil and Mexico
The National Board of Medical Examiners® (NBME®) Latin America Grants Program awarded grants to three collaborations of Latin American medical schools for their proposals to enhance the quality of evaluation in health professions education in their respective regions. The Latin America Grants Committee selected the recipients in February after receiving more than a dozen proposals.
Over its 103-year history, the NBME has pursued and maintained the highest standards of excellence in its examination programs. As part of its Centennial celebration in 2015, the NBME supported three consortia of Latin American medical schools in their work to develop evaluation capacity in their countries. Building on the success of that program, the Latin America Grants Program presented the three recent awards.
Each proposal selected includes ideas for sustainable faculty development activities, creation of a Massive Open Online Course (MOOC), and development of a methodology for longitudinal assessment of abilities and skills acquired over the course of a medical internship. The proposals are designed to have a broad, sustainable impact by building local capacity to evaluate key learning outcomes in the education of those pursuing health professions. They are also expected to improve program quality at participating schools.
Read the full article
RENEW Task Force Seeks to Support Examinee Wellness
The National Board of Medical Examiners® (NBME®) is committed, along with like-minded organizations, to identifying potential sources of burnout for physicians in training and contributing to solutions. In February, the NBME convened the inaugural Re-examining Exams: NBME Effort on Wellness (RENEW) task force. The organization’s goal in this initiative is to address the challenge of physician wellness and to acknowledge the stress caused by working in the health professions that begins during the educational and training processes.
The RENEW Task Force was created to:
- Understand the role high-stakes exams, such as the United States Medical Licensing Examination® (USMLE®), as well as secondary uses of those exams, play in affecting medical student and physician wellness.
- Develop and execute a research agenda that will lead to actionable interventions to promote wellness and mitigate burnout.
- Support actionable interventions to promote wellness amongst learners and trainees in the context of assessment.
Outreach to Students Continues
April 10, 2018 — In an effort to broaden its outreach, colleagues from the United States Medical Licensing Examination® (USMLE®) program hosted the inaugural meeting of the Medical Student and Resident Advisory Panel to the USMLE Program. The meeting, facilitated by staff from the National Board of Medical Examiners and the Federation of State Medical Boards (FSMB), was held at the FSMB’s headquarters in Euless, TX. The meeting was designed to:
- Collect feedback from medical students and residents on issues and topics specific to USMLE
- Test assumptions about the USMLE with an examinee audience
- Gain insight into the perspective of examinees on USMLE policy issues
Dr. Paul Cunningham awarded 2018 Jacobi Medallion
Executive Board member, Dr. Paul Cunningham, received the 2018 Jacobi Medallion for his contributions to the medical profession and for his achievements in academic leadership.
In addition to serving on the NBME Executive Board, Dr. Cunningham is Dean Emeritus of the Brody School of Medicine at East Carolina University. He served as Dean at Brody School of Medicine from 2008-2016.
Dr. Cunningham was awarded the 2018 Jacobi Medallion on March 15 from the Icahn School of Medicine at Mount Sinai in New York, which is also where Dr. Cunningham completed his residency in surgery.
Read the full article
NBME Announces Stemmler Grant Awards
The National Board of Medical Examiners® (NBME®) announced that the 2017-2018 Stemmler Grant award winners are from University of Michigan, University of Calgary and Uniformed Services University of the Health Sciences.
This year, three projects were chosen to receive grant funds, totaling $393,549 in funds designed to provide support for research or development of innovative assessment approaches that will enhance the evaluation of those preparing to, or continuing to, practice medicine.
Read the full article
2017 NBME Annual Report Available
Updated Educator Guide Online
Faculty can learn how to write high-quality, multiple-choice questions related to the foundational and clinical sciences by viewing NBME's updated Educator Guide.
New Histology Basic Science Subject Exam Launches July 2018
Dr. Louis Pangaro selected by NBME as Recipient of the 2018 John P. Hubbard Award
The National Board of Medical Examiners is pleased to announce that Louis N. Pangaro, MD was selected as the recipient of the 2018 John P. Hubbard Award.
Read the full article
Dr. Theodorus Jan (Olle) ten Cate selected by NBME as Recipient of the 2017 John P. Hubbard Award
The National Board of Medical Examiners is pleased to announce that Theodorus Jan (Olle) ten Cate, PhD was selected as the recipient of the 2017 John P. Hubbard Award.
Read the full article
Dr. Peter J. Katsufrakis Appointed President of National Board of Medical Examiners (NBME).
Following a national search, the Executive Board of the National Board of Medical Examiners has selected Peter J. Katsufrakis, MD, MBA to lead the NBME as its next president.
Read the full article
NBME Activities at the 2016 AAMC Annual Meeting
The NBME will host a variety of sessions in conjunction with the 2016 AAMC Annual Meeting, November 11-15, in Seattle, Washington. Activities include interactive exhibits, focus groups, demonstrations, and workshops designed to elicit ideas and feedback on USMLE and NBME assessment services. Session details are listed below. Invitations for closed sessions will be sent in early October.
Read more information
Eric S. Holmboe Selected as Recipient of 2016 John P. Hubbard Award
The National Board of Medical Examiners is pleased to announce that Eric Holmboe, MD was selected as the recipient of the 2016 John P. Hubbard Award.
Read the full article
Step 2 CS FAQs
New FAQs regarding the Step 2 Clinical Skills (CS) exam have been posted to the USMLE website.
NBME and the VA: Joining Forces to Help Veterans and Their Families
With a generous grant from the Veterans Administration, the NBME is pleased to do its part to help ensure good healthcare for military personnel and their families. Read about it here: http://khn.org/news/its-on-the-test-new-questions-require-doctors-to-learn-about-military-medicine/
NBME and the 2015 AAMC Annual and Medical Education Meetings
The NBME will host a variety of sessions in conjunction with the 2015 AAMC Annual Meeting (November 6-10) and the AAMC Medical Education Meeting (November 10-12) in Baltimore, MD. Activities include interactive exhibits, focus groups, demonstrations, and workshops designed to elicit ideas and feedback on USMLE and NBME assessment services.
Read more information
New Customized Test Battery Platform
Based on feedback from faculty, the NBME has developed the Customized Test Battery platform, which enables administration of a series of exams using a combination of fixed-form, standardized subject tests and examinations created through the Customized Assessment Services (CAS) program. The Customized Test Battery platform can expand your assessment capabilities. For example:
- If your curriculum emphasizes fundamental science in the clerkship, you can administer a clinical science subject test along with a custom module of items covering associated basic science concepts.
- You can administer a basic science subject test and supplement the exam with items from other basic science content and/or with advanced clinical vignettes to match your assessment needs.
The "Battery Builder" is housed on the Build Exam tab of the NBME Services Portal (NSP). The tool is easy to use and includes the functionality to:
- create a test battery from a list of all NBME subject tests and any customized exams that are also available in the online ordering system
- set the test delivery sequence for the selected exams
- place one or more "off the clock" breaks between exams
- lock, edit, copy and delete test batteries
Test batteries are intended for administration to a single group of examinees and delivered in one exam session. The NBME web-based testing proctoring system has been modified to reflect test battery administrations. The examinations are scored separately, and reports are posted to the appropriate subject exam or CAS folders in NSP.
Suzanne Anderson Elected Chair of the NBME
The National Board of Medical Examiners (NBME) is pleased to announce that Suzanne T. Anderson was elected to a two-year term as Chair of the National Board of Medical Examiners.
Read the Press Release
Larry Gruppen Selected as Recipient of 2015 John P. Hubbard Award
The NBME is pleased to announce that Larry D. Gruppen, PhD was selected as the recipient of the 2015 John P. Hubbard Award.
Read the full article
Phased Elimination of Paper-based Subject Examination Program Services
Updated: May 8, 2015
The NBME transitioned all basic science and clinical science subject examinations to a web-based format in 2012. In October 2014, all schools were advised to transition to web-based testing for the upcoming 2015-2016 academic year.
Plan to Eliminate Paper-based versions of the Clinical Science Subject Examinations
The paper-based versions of the Clinical Neurology, Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry, and Surgery subject examinations are at the end of their useful lives as measures of current knowledge and practice. During the upcoming 2015-2016 academic year, NBME staff plan to actively manage the reduction and ultimate depletion of existing inventories of these outdated paper exams. It is projected that the current stock of these examinations will begin to run out after the end of the 2015 calendar year.
Beginning in January 2016, the paper-based versions of these examinations originally developed in 2010 will be retired. As this will further limit the inventory of paper examinations in rotation, there will be increased likelihood that schools may receive a recently administered form. As long as stock remains in these Clinical Science disciplines, we will continue to provide schools with these examinations in paper-based format through June 2016.
Reminder about Eliminating Paper-based Subject Examination Program Services
As a reminder, the first phase of the discontinuation of paper goes into effect August 1, 2015, when the following examinations will no longer be available for administration via paper-based format:
- Introduction to Clinical Diagnosis (added)
- Family Medicine
- Comprehensive Clinical Science
- Behavioral Sciences
- Gross Anatomy
- Histology and Cell Biology
Also effective July 23, 2015, expedited score reporting (a.k.a. "Rush Score Reports") for paper-based examinations will no longer be offered.
If you have any questions, please contact the NBME Subject Examinations team at firstname.lastname@example.org.
Characteristics of Clinical Clerkships: NBME Survey Results
The National Board of Medical Examiners (NBME) is pleased to announce that detailed results of the Characteristics of Clinical Clerkships Survey are now available. In 2012, the NBME emailed Executive Chief Proctors at over 200 medical schools and satellite campuses asking them to forward an invitation to complete an on-line questionnaire to the clerkship directors in:
- Clinical Neurology
- Family Medicine
- Obstetrics and Gynecology
The purpose of the survey was to gather descriptive information about clerkship characteristics and use of computer case simulations. Data gathered for each clerkship included type and length of clerkship rotations, curriculum type for the 2012-2013 academic year, purpose of the exam, and for the 2011-2012 academic year, procedures used to determine minimum passing scores, minimum passing score and the score required to achieve Honors. We hope you will find these results informative.
Assessment of Professional Behaviors Materials Available in DREAM
The NBME is pleased to announce that its Assessment of Professional Behaviors (APB) program is now available through the Directory and Repository of Educational Assessment Measures (DREAM), a peer-reviewed collection of tools within MedEdPORTAL. The APB program materials are available under the Creative Commons Attribution-Noncommercial-Share Alike license, and are thus free for use in the educational setting.
The APB program was developed to support continuous learning around communication and interpersonal skills, professionalism, and practice-based learning and improvement. It is centered on a multisource feedback instrument, designed to be administered using an electronic evaluation system. The materials available for download include assessment instruments, implementation guidance, sample feedback reports, and training modules for raters and for feedback facilitators. A third-party, peer -reviewed critical analysis is included in the download package.
You can access the APB material here. After more than 10 years of development, the NBME is pleased to provide this tool as a service to the educational community.
Data Commons Launches Physician Data Services
Washington, D.C., July 28, 2014 — Data Commons, LLC, a new company that delivers data about physicians, launched its first services today. Data Commons provides data about physician enrollment, licensure and certification-on-demand and directly from multiple source organizations — all through a single hub.
Notice about the Heartbleed Bug
The NBME has been working to assess the impact on our customers in the wake of the recent disclosure of CVE-2014-0160, known colloquially as the "Heartbleed" bug. The Heartbleed bug is a serious vulnerability in the popular OpenSSL encryption software library used to secure the Internet.
Robyn Tamblyn Selected as 2014 Hubbard Award Recipient
The NBME is pleased to announce that Robyn Tamblyn, BScN, MSc, PhD was selected as the recipient of the 2014 John P. Hubbard Award.
Read the full article
Succeed on your career path
Preferred standard for career documentation by the AAMC and NBME
The Pivio® system is a new software tool that will connect data needed by medical students, residents, and physicians across their academic and professional careers, in order to assist users with lifelong learning, career planning, and individual growth.
The Pivio® service is a "secure lock box" that will enable users to control the storage and transfer of data created throughout their academic and professional careers.
NBME Plans Its Centennial
On May 5, 2015, the NBME will complete 100 years of service to the public and the professions.
NBME Executive Board Election
The National Board of Medical Examiners (NBME) is pleased to announce newly elected as well as re-elected members of the National Board of Medical Examiners Board of Directors ("NBME Executive Board"). The election occured at the Annual Meeting of the NBME Membership on April 5, 2013 in Philadelphia.
2014 John P. Hubbard Award: Call for Nominations
The due date for letters of nomination for the 2014 John P. Hubbard Award is September 12, 2013. Information about the Hubbard
Award is found at the link below.
Stemmler Fund Call for Letters of Intent
The due date for the 2013-2014 Call for Letters of Intent is July 1, 2013. An overview of the Letter of Intent application process is provided in the link below.
NBME Partners to Launch Data Commons, LLC
The National Board of Medical Examiners (NBME) and several other leading national physician education and assessment organizations have announced their partnership to launch the Data Commons, LLC. The Data Commons is an information-sharing company that is developing a system to improve the accessibility and use of data about physicians and other healthcare professionals. The core of this new system is a secure data hub that will allow multiple organizations to make available selected information from their individual databases. The five other organizations joining in the partnership are the American Board of Family Medicine, the American Board of Pediatrics, the Association of American Medical Colleges®, the Educational Commission for Foreign Medical Graduates, and the Federation of State Medical Boards. The Data Commons, LLC, is a joint partnership that will administer the new system.
Kevin Eva Selected as 2013 Hubbard Award Recipient
The NBME is pleased to announce that Kevin Eva, PhD was selected as the recipient of the 2013 John P. Hubbard Award.
Read the full article
2012 Fall/Winter NBME Examiner Now Available!
The 2012 Fall/Winter Examiner includes articles on NBME Renewal, the new Global Evaluation Management System (GEMS), and NBME's new online item-writing tutorial.
Read the 2012 Fall/Winter Examiner
International Foundations of Medicine (IFOM) Clinical Science Exam Administered Nationwide in Lebanon
The National Board of Medical Examiners® (NBME®), in conjunction with the Faculties of Medicine at the American University of Beirut, Beirut Arab University, Lebanese University, Holy Spirit University of Kaslik, Université Saint-Joseph de Beyrouth, and University of Balamand, is pleased to announce the first nationwide administration of the International Foundations of Medicine (IFOM) Clinical Science Examination (CSE). Approximately 300 students from these six Lebanese medical faculties participated in the examination, administered on July 14, 2011 at the Lebanese University.
Read the Press Release
2012 Hubbard Award Call for Nominations
The NBME invites nominations for the 2012 John P. Hubbard Award. This award recognizes individuals who have made a significant and sustained contribution to the assessment of professional competency and educational program development at any level along the continuum of medical education and delivery of healthcare.
Learn More About the 2012 Call For Nominations
Lewis R. First Elected Chair of the NBME
We are pleased to announce that Lewis R. First, MD, MS was elected to a two-year term as Chair of the National Board of Medical Examiners at the Annual Meeting of the NBME Membership on April 1, 2011.
Learn More About Dr. First
Judy Shea selected by NBME as recipient of 2011 John P. Hubbard Award
The NBME is pleased to announce that Judy A. Shea, PhD was selected as the recipient of the 2011 John P. Hubbard Award. The award is given to individuals who have made outstanding contributions to the pursuit of excellence in the field of evaluation in medicine. Susan M. Spaulding, a member of the 2011 Hubbard Award Committee, announced Dr. Shea's selection at the Annual Meeting of the NBME Membership on March 31, 2011.
Learn More About Dr. Shea
March 2011 Comprehensive Review of USMLE Update
Since our last update in June 2010, staff has completed successful pilot work in several areas. Test material committees have developed multiple choice questions that assess an examinee's ability to appropriately interpret information presented both in the form of a research abstract and a pharmaceutical advertisement. Additional study is planned, and the items that ultimately appear in the examination may differ from the current design being tested.
Read More About the Comprehensive Review March Update
2010 Fall/Winter NBME Examiner Now Available!
The 2010 Fall/Winter Examiner includes articles on the fall 2010 IAMRA conference and new item formats that are under development.
Read the 2010 Fall/Winter Examiner
NBME @ Events
2019 American Board of Specialties (ABMS) Conference
2019 Europe Association of Test Publishers (ATP) Conference
Melia Castilla Hotel and Convention Center
2019 International Association of Communication in Healthcare (ICCH) Conference
Sheraton San Diego Hotel & Marina
San Diego, CA
2019 AAMC Learn Serve Lead (LSL) Annual Meeting
Phoenix Convention Center
2019 Diagnostic Error in Medicine (DEM) Annual International Conference
Hyatt Regency Washington
2019 Press Ganey National Client & Executive Leadership Conferences
Orlando World Center Marriott Resort & Convention Center
2019 Institute for Credentialing Excellence (ICE) Annual Conference
Hilton San Diego Bayfront
San Diego, CA
*As of 2018