When studying for Step 1, students regularly have questions about which NBME Practice Tests to use in their preparation. Students want to know which practice tests are more difficult and which are more predictive. After emailing the NBME and surveying students, we can share the following information.

The NBME reports that all of the forms were built to the same specifications, so there are no major differences in forms. The forms are statistically equated, so any slight differences in difficulty are adjusted so that the scores are directly comparable across forms.

In addition, the clinical vignettes are not longer across forms, though the exams may seem longer because there are no nonvignettes. More recent forms have all clinical or experimental vignettes, unlike the older forms, which had nonvignettes, as well

My interpretation of this NBME information, along with student feedback, is that the older forms, especially 3 and 4, have fewer questions using the clinical or experimental vignette format. This may make the question stems seem more straightforward, and therefore “easier” in terms of completing them, especially within the standard time limit of 60 minutes.

The older forms (6, 7) have more questions with experimental and clinical vignettes which students find tougher to answer in terms of conceptualization, application, and completing within time limits. Not all of the questions are in vignette format however, as students will see on the 2011 Step 1 examination. These forms, according to student surveys, also seem to be the exams that are most representative of Step 1 content – particularly Form 7.

Forms 11 and 12 are the most recent NBME self-assessment forms and contain questions almost exclusively using the vignette format. Students often report that these forms give insight into how to pace themselves for the actual Step 1 exam and that they have difficulty completing the practice exam in the time allotted.

Some students also report that Forms 11 and 12 are content skewed toward certain subjects, such as anatomy. However, the NBME states (see above paragraphs) that the forms are built to the same specifications.

Also, recent polls at USMLE Forums show that Form 12 is moving up in the ranks as a good predictor of Step 1 exam scores along with Form 7 (Please see limitations of poll in next paragraph). We may see this form increase in popularity as students become more familiar with it and more students use it in their preparation.

The limitation that I see from the USMLE Forum poll is that the timing of self-assessment is not factored into the predictability of the form. This is not a formal study; however, the closer a form is used to the actual exam and therefore, the greater the preparation of the student, the more effect on validity of predictive results. A practice test taken at the outset of studying for Step 1 cannot be expected to have the same predictive validity as a practice test taken two days prior to a student’s actual test day. When calculating potential scores, the NBME does not ask the student how many days of preparation remain before their exam to enter into a formula to predict a Step 1 score following 4-6 weeks of preparation. The score is a prediction of how well you would do if you took the exam that day with your current level of preparation.

In summary, here is how I would suggest that students use their NBME practice tests:

Form 3 or 4 – Take these at the beginning of your preparation to establish a baseline and help determine how much study time you need. It can also help you decide if you need a live board review course that is more of a foundational course.

Form 5 – Use this form about half way through preparation to monitor progress (or do not use at all).

Form 6,7,11,12 – Spread these out starting 15 days before actual exam so you can strategize how to change or alter your study plan if need be. Take another practice exam or two one week prior to test day to make sure you do not need to push your exam to a later date.

Never just take one practice test to establish readiness for your USMLE exam. Also factor in how you have been performing on your last 15-20 blocks of timed sessions of your electronic question bank.

Be sure that your practice exam score places you well beyond 1 Standard Deviation of the passing score to establish that you are ready to sit for the USMLE.