Doctor looking at the viewer

DATA SHARING

The NBME Data Sharing and Collaboration Program enables external researchers to request de-identified data from NBME.

two

Collaborate with NBME on Your Next Research Project

NBME collects valuable data and insights through the administration of our assessments and the work of our research team. The NBME Data Sharing and Collaboration Program allows us to share these findings with external researchers pursuing topics of their own interest that also have the potential to benefit the medical education and measurement communities. All data shared by NBME is de-identified to protect the personal information of examinees.

Each approved project is assigned to an NBME staff member, who will partner directly with you and your organization to support the initiative from its early stages to publication.

researcher at desk

Application

Applicants can complete an initial online data request proposal by clicking the submit button below. You can save and come back to your online submission at any time. Additionally, you can invite others to collaborate on your submission. Please read all instructions below and at the top of the online submission form.

Applicants may contact ORS@nbme.org with any online submission issues.

Submit Data Request Proposal

Decision

Researchers will receive a decision regarding their initial proposal within 5 business days. If the initial proposal is accepted, applicants will be invited to submit a detailed data request which includes a formal study design and a current bio sketch or CV for all involved researchers. The invitation to submit a detailed data request does not guarantee final acceptance. A final decision requires an evaluation of the below information and will be communicated within 10 business days after receiving the required materials.

A detailed study design should include the below information:

  • The purpose of the study
  • The specific research questions or research hypotheses
  • The variables to be included (e.g., predictor and outcome variables)
  • The proposed analyses
  • The importance, value, or implications of the research, and plans for disseminating results
  • A study timeline (including all major milestones)
  • All members of the research team, along with their titles, and affiliations
    • A statement describing their roles in the proposed project

For those study designs requiring data from organizations other than NBME (e.g., AAMC; medical schools), the researcher must include a statement that such organizations have agreed to make data available.

Data Sharing Requests can be submitted by any researcher with a genuine interest in conducting original research in the field of measurement or assessment in health professions education. Preference is given to those with an established record of research, study designs which align to NBME’s mission, and who show promise of being able to complete a publishable research project. 

To further aid applicants, some common reasons NBME will decline to share data for a proposed research project:

  • Unapproved secondary use of USMLE data, particularly regarding residency selection. NBME and USMLE want to disassociate scores from residency selection practices to highlight the exam's primary purpose of licensure.
  • Unclear purpose of study, research question, or intended use of NBME data
  • A study design which does not account for nested data structure where necessary
  • A study, as currently proposed, which can be completed without USMLE data
  • Reliance on data we do not release, such as Step 2 CS outcome data, 3-digit score data for Step 1, and/or subscores or item level data
  • Potential for examinees to be identified based on data or combination of data to be used

Review Criteria and Process

Data Sharing Requests are evaluated by NBME research and programmatic staff and are judged on the following:

  • The clarity of the research question
  • The importance or value of the research
  • The capability of data to answer the research question
  • The quality of data analyses 
  • The qualifications of researchers
  • The researchers’ capability to maintain security of the data

Approved data sharing requests will be required to comply with NBME data security requirements. Data sharing projects will not be approved if there is risk that the investigator or others could determine the identity of individuals, programs, or institutions. 

Complex proposals or those involving multiple organizations may require additional correspondence between the investigator and NBME to verify study details, to discuss certain conditions (e.g., possible fees, de-identification process), or to confirm that any third parties can supply the necessary data.  Although NBME will facilitate such discussions, it is the investigator’s responsibility to anticipate such needs to allow time for such discussions.