NATIONAL BOARD OF MEDICAL EXAMINERS® (NBME®)
Serving the Public Through Evaluation of Healthcare Professionals Worldwide
OVERVIEW OF NBME SERVICES AND ASSESSMENTS FOR MEDICAL SCHOOLS AND STUDENTS Agata P. Butler, PhD Vice President, NBME
NBME Subject Exam Program Serve the assessment needs of medical schools by providing highquality assessment tools as well as information and assistance to medical educators
NBME Subject Tests: Basic Science Exams
Gross Anatomy (+/-Embryology) Histology & Cell Biology Biochemistry Physiology (+/- Neurophysiology) Neuroscience (+/- Neuropath, Neuropharm) Behavioral Sciences Microbiology (+/- Immunology) Pharmacology Pathology Introduction to Clinical Diagnosis
NBME Subject Tests: Clinical Science Exams Core Clerkship Exams Ambulatory Medicine Family Medicine (modular) Internal Medicine Clinical Neurology Obstetrics & Gynecology Pediatrics Psychiatry Surgery
NBME Subject Tests: Clinical Science Exams Advanced Clinical Exams
Internal Medicine (2009) Surgery (2010) Pediatrics (2011) Emergency Medicine (2013)
NBME Subject Tests: Comprehensive Exams
Comprehensive Basic Science Examination Comprehensive Clinical Science Examination
NBME Subject Exam Program
Used by many medical schools as end-ofcourse and end-of-clerkship exams National
norms and descriptive information of students at the same stage of training
Multiple
parallel forms of exam available Test scores are equated and can be directly compared over time
NBME Subject Exam Program High-quality,
USMLE-like items that have been carefully reviewed and pretested Items written by a "national faculty" (USMLE Test Committees) Gives students practice for USMLE
Performance Profiles Report performance profiles to schools for individual examinees taking the comprehensive subject exams, clinical science exams, and web-based basic science exams.
Student Performance Profile
Look-up Table Provide a look-up table for schools and students to use as a benchmark on passing USMLE (comparable score)
A table provides approximate performance equivalents for USMLE Step 1, making it possible to translate the CBSE score to the scale used for USMLE Step 1.
Web-Based Subject Exams
Transitioned subject tests from a paper format to web-based platform Enhance security Streamline score reporting Provide new opportunities for growth
NBME provides test administration software that
provides a mechanism for the school to use laptop and/or desktop computers
“locks” the browser, preventing examinees from copying screens or communicating exam content
enables the Proctor to open exam access, monitor examinee progress and control other aspects of the test administration
New Directions in 2012
NBME subject examinations are now available at Prometric Test Centers
Initially limited to the comprehensive and clinical science subject tests
Future Directions
Include pretest items in subject tests Include multimedia in items
Pilot modular tests
Collaborate with schools and faculty in developing new and innovative item types
Customized Assessment Services
Faculty members can access the NBME item bank to design an assessment to fit their needs
Customized Assessment Services: Uses
In-course tests
End-of-course tests
Individual discipline-based courses Individual organ-system blocks
Integrated, tests
Mid-term exams
End-of-semester/block exams
Basic science progress tests
NBME Self-Assessment Services Self-assessment services provided directly to students Comprehensive Basic Science (Step 1) Comprehensive Clinical Science (Step 2 CK) Comprehensive Clinical Medicine (Step 3 MCQ)
NBME Self-Assessment Services Designed to reflect the format and content coverage of USMLE Provided as a service to students and can be taken via the web at any time and from any location that the examinee chooses
CBSSA: Sample Performance Profile
A table provides approximate performance equivalents for USMLE Step 1, making it possible to translate the CBSSA score to the scale used for USMLE Step 1.
Recent Research Publication on NSAS
12,224 U.S. and Canadian medical students Completed an English version of the CBSSA exam between 12/06 and 12/08 Took the exam under standard-paced conditions First Step 1 attempt was on or before 11/17/09.
The most recent CBSSA prior to their first Step 1 attempt and first Step 1 score was used CBSSA performance is strongly related to Step 1 performance (67 – 69% of variance in Step 1 scores explained by CBSSA)
Research Publications on NSAS
Using the NBME Self-Assessments to Predict Performance on USMLE Step 1 and Step 2: Impact of Test Administration Conditions. Academic Medicine. Vol. 70. No. 10. October/Supplement 2004. Relationship Between Performance on the NBME Comprehensive Basic Sciences Self-Assessment and USMLE Step 1 for US and Canadian Medical School Students. Academic Medicine. 85(10):S98-S101, October 2010.
Content Feedback
Longitudinal Performance Profile Performance profile graphically displays the relative strengths and weaknesses in general content areas as horizontal bars Enhance feedback to display the performance bar for each content area vertically with each column (bar) of the report identified to show progress over multiple testing sessions
Longitudinal Performance Profile
New: Clinical Science Mastery Series
New self-assessments for the clinical science subject examinations Two, 50-item forms for each of the following: Clinical Neurology Obstetrics/Gynecology Psychiatry
Medicine Pediatrics Surgery
Content Feedback for Psychiatry Self-Assessment
A table provides approximate performance equivalents for the Medicine Subject Test, making it possible to translate the selfassessment score to the scale used for the Medicine Subject Test.
Future Directions: Enhance Feedback for Students
Enhance feedback regarding timing and pacing for the self-assessments Enhance score reports with more specific feedback about the items Provide opportunity for students to review the questions and answers after the test for customized assessments
To learn more about the National Board of Medical Examiners, visit us at www.nbme.org
Performance on Most Recent CBSSA, First Step 1, and Pass/Fail Outcomes CBSSA
Group
N
Study Sample
12,224
All US/Canadian Examinees
57,329
Step 1
Mean
SD
Mean
SD
% Passing
456
109
225
21
96.5%
221
23
93.1%
CBSSA and Step 1 Performance by Number of CBSSA Takes CBSSA Attempts 1
N 6489
Percent of Study Sample 53%
2
3546
3
CBSSA Score Mean SD
Step 1 Score
438
110
Mean 224
SD 21
29%
466
103
225
21
1402
12%
484
105
227
21
4
602
5%
503
108
230
21
5
113
1%
525
115
232
23
6
65
<1%
550
94
236
19
Model Summary Model R
R Square
Adjusted R Square
Std. Error of the Estimate
1
.819(a)
.671
.671
12.140
2
.828(b)
.685
.685
11.875
3
.831 (c)
.691
.691
11.762
a. Predictors: (Constant), LAST_CBSSA_BF_STEP1 b. Predictors: (Constant), LAST_CBSSA_BF_STEP1, NUM_DAYS _BET_STEP1 c. Predictors: (Constant), LAST_CBSSA_BF_STEP1, NUM_DAYS__STEP1, NUM_CBSSA_TAKES_BF_STEP1
Stepwise Multiple Regression
Dependent variable: First Step 1 score Independent variables: Most
recent CBSSA before first Step 1 attempt Number of days between most recent CBSSA and first Step 1 attempt Mean
= 14 days Median = 8 days Mode = 7 days Number
of CBSSA takes before first Step 1
