Tuesday, July 11, 2006

2006 USMLE BULLETIN : Scoring and Score Reporting

Examination Results and Scoring

When you take Step 1, Step 2 CK, or Step 3, the computer records your responses. After your test ends, your responses are transmitted to the NBME for scoring. The number of test items you answer correctly is converted to two equivalent scores, one on a three-digit score scale and one on a two-digit score scale. Both scales are used for score-reporting purposes.

On the three-digit scale, most Step 1, 2 CK, or 3 scores fall between 140 and 260. The mean score for first-time examinees from accredited medical school programs in the United States is in the range of 200 to 220, and the standard deviation is approximately 20. Your score report will include the mean and standard deviation for recent administrations of the examination. The two-digit score is derived from the three-digit score. It is used in score reporting because some medical licensing authorities have requirements that include language describing a "passing score of 75." The two-digit score is derived in such a way that a score of 75 always corresponds to the minimum passing score.

For Step 2 CS, examinees are assessed on their data-gathering and communication skills (including spoken English) by the standardized patient, and on their ability to complete an appropriate patient note by physician raters. Performance on Step 2 CS is reported as pass or fail, with no numerical score.

USMLE score reports and transcripts show your scores (for Step 1, Step 2 CK, and Step 3) and an indication of whether you passed or failed (for all examinations). The same information is sent to medical licensing authorities upon your authorization for their use in granting the initial license to practice medicine.

Except as otherwise specified below, to receive a score on Step 1, Step 2 CK, and Step 3, you must begin every block of the test. If you do not begin every block, no final results are reported, and the "incomplete examination" attempt appears on your USMLE transcript. If you register for but do not begin an examination, no record of the test will appear on your transcript.

If your Step 1, Step 2 CK, or Step 3 examination is incomplete, you may request that a score be calculated and reported, with all missed test items or cases scored as incorrect. This score is likely to be lower than the score you would have achieved had you completed all sections of the examination. If you receive notification that your examination resulted in an incomplete attempt, contact the NBME in writing no later than 45 days after the date the notification is mailed to you if you would like further information on having the score calculated and reported. If you decide to request calculation and reporting of your score, the score will appear on your USMLE transcript as though it were complete; it will remain the permanent score for the examination administration.

For Step 2 CS, if you do not begin every case, your performance may be assessed on those cases completed. If this assessment would result in a passing outcome no matter how poorly you may have performed on the missed case(s), then a "pass" will be reported. If this assessment would result in a failing outcome no matter how good your performance on the missed case(s), then a "fail" will be reported. Otherwise, the attempt may be recorded as an "incomplete examination."

If it is determined that you took a Step for which you were not eligible, results for that test may not be reported or, if previously reported, may be revoked (see Change in Eligibility Status and Irregular Behavior).

Some examination materials are included in the USMLE to enhance the examination system and to investigate the measurement properties of the examinations. Such materials are not scored.

Scoring for Multiple-Choice Items

Multiple-choice items are provided in blocks of approximately 45 to 60 minutes. Blocks of items are constructed to meet specific content specifications. As a result, the combination of blocks of items creates a form of the examination that is comparable in content to all other forms. The percentage of correctly answered items required to pass varies from form to form. However, examinees typically must answer 60 to 70 percent of items correctly to achieve a passing score.

Scoring for Primum CCS

The CCS scoring process compares your patient management strategy with policies obtained from experts. Actions resembling a range of optimal strategies will produce a higher score. You must balance thoroughness, efficiency, avoidance of risk, and timeliness in responding to the clinical situation. Dangerous and unnecessary actions will detract from your score.

Scoring for Step 2 CS

Passing performances on all three subcomponents (ICE, CIS, SEP) in a single administration are required to obtain the overall designation of passing on Step 2 CS.

Minimum Passing Scores

The USMLE program provides a recommended pass or fail outcome on all Step examinations. Recommended performance standards for the USMLE are based on a specified level of proficiency. As a result, no predetermined percentage of examinees will pass or fail the examination. The recommended minimum passing level is reviewed periodically and may be adjusted at any time. Notice of such review and any adjustments will be posted at the USMLE website.

Note: Visit the USMLE website for up-to-date information.

A statistical procedure ensures that the performance required to pass each test form is equivalent to that needed to pass other forms; this process also places scores from different forms on a common scale.

For Step 3, your performance on the case simulations will affect your Step 3 score and could affect whether you pass or fail. The proportional contribution of the score on the case simulations is no greater than the amount of time you are allowed for the case simulations.

Official Examinee Score Reports

For Steps 1, 2 CK, and 3, the official examinee score report you receive after you take the examination includes a pass/fail designation, numerical scores, and graphical performance profiles summarizing areas of strength and weakness to aid in self-assessment. These profiles are developed solely for your benefit and will not be reported or verified to any third party.

Performance on Step 2 CS is reported as pass or fail. Examinees who fail receive performance profiles, which reflect the relative strengths and weaknesses of the examinee's performance across the subcomponents of Step 2 CS.

Note: To avoid misinterpretation and protect your privacy, scores or pass/fail outcomes are not provided by telephone, e-mail, or fax to anyone. You should retain your official score report for your records.

Results for computer-based examinations are typically available in time to mail your report within three to four weeks after your test date. Results for Step 2 CS are typically available within eight weeks of your test date. However, delays are possible for various reasons. In selecting your test date and inquiring about results, you should allow at least eight weeks after your test date to receive your score report.

Score Rechecks

For Steps 1, 2 CK, and 3, standard quality assurance procedures ensure that the scores reported for you accurately reflect the responses recorded by the computer.

For Step 2 CS, score rechecks first involve retrieval of the ratings you received from the standardized patients and from the physician note raters. These values are then re-summed and re-converted into final scores in order to confirm that the reported pass/fail outcome was accurate. There is no re-rating of your encounters or of your patient notes; videos of encounters are not reviewed. Videos are used for general quality control and for training purposes and are retained only for a limited period of time.

Patient notes are carefully reviewed, in some instances by multiple physicians, before scores are released. As part of the quality control procedures for initial scoring, examinees who fail Step 2 CS solely on the basis of the Integrated Clinical Encounter subcomponent and who are performing at a level that is near the minimum passing point, have their patient notes rated by multiple physician note raters. Therefore, patient notes are not reviewed again when a recheck is requested.

For all Steps and Step Components, a change in your score or in your pass/fail outcome based on a recheck is an extremely remote possibility. However, a recheck will be done if you submit a written request and service fee to the entity that registered you for the examination. Your request must be received by your registration entity no later than 90 days after your result was released.

Score Reporting

After you take a Step or Step Component, you should allow at least eight weeks after your test date to receive your score report. If you do not receive your original score report or you receive a damaged score report, a request for a duplicate score report will be honored up to 90 days after your score report release date. You must make your request to the entity that registered you for the test. If more than 90 days have passed since your score report release date, scores will be reported to you only in the form of a USMLE transcript after you submit a signed request and pay the required fee.

The NBME reports the results of the USMLE to LCME- and AOA-accredited medical school programs for their students, who are enrolled at the time of application, and graduates.

If you are a student or graduate of an LCME- or AOA-accredited medical school program and you do not want your Step 1, Step 2 CK, or Step 2 CS results reported to your medical school, you must send a signed request to the NBME. Your request must be postmarked or faxed at least two weeks before your scheduled test date. If you make this request, the score information provided to the medical school will list your name and the notation "record withheld at the request of the examinee." If you want your scores reported to your medical school subsequently, you must submit a signed request and pay the required fee (see below).

If you are a graduate of an LCME- or AOA-accredited medical school program, you must indicate on your Step 3 application your preference for reporting your Step 3 score to the school from which you graduated.

Official USMLE Transcripts and Providing Scores to Third Parties

If you want to send your USMLE scores to a third party, you must submit a request and pay a fee. Your scores will be provided in the form of a USMLE transcript.

Examination data (including performance information and recorded patient encounters) from USMLE examinations may be used by the USMLE program or made available to third parties for research. In such instances, the data will be confidential, and individual examinees will not be identified in any publication. If you do not wish your score to be made available for research purposes, you must advise the USMLE Secretariat in writing.

Except as described in this bulletin, USMLE results will not be reported to you or third parties without your request and payment of the transcript fee.

Your USMLE transcript includes the following:

  • your name and other personal identification information including your date of birth;
  • your complete results history of all Steps and Step Components that you took;
  • your history of any examinations for which no results were reported;
  • indication of whether you have previously taken the former NBME Parts I, II, or III, Federation Licensing Examination (FLEX), or ECFMG Clinical Skills Assessment (CSA);
  • annotation(s) if you were provided with test accommodations;
  • annotation(s) and information documenting classification of any scores as indeterminate;
  • annotation(s) and information documenting any irregular behavior; and
  • notation(s) of any actions taken against you by medical licensing authorities or other credentialing entities that have been reported to the FSMB Board Action Databank.
Note: Graphical performance profiles, which are included on your original Step 1, Step 2 CK, Step 2 CS (failing examinees only), and Step 3 score reports, are not included in your USMLE transcript.

To obtain your USMLE transcript or have it sent to a third party, you must contact the ECFMG, FSMB, or NBME. Which entity you contact depends on which examinations you have taken and where you want your transcript sent. Contact the FSMB if you want your transcript sent to a medical licensing authority at any time. If you have not registered for or taken Step 3 and want your transcript sent to anyone other than a medical licensing authority, the request should be sent to the last entity that registered you.

Electronic Residency Application Service (ERAS®)

If you use ERAS, you may request electronic transmittal of your USMLE transcript to residency programs that participate in ERAS. Information on ERAS is available for students and graduates of LCME- and AOA-accredited medical school programs from the medical schools, and for students and graduates of medical schools outside the United States and Canada from the ECFMG. The Association of American Medical Colleges (AAMC) and ECFMG websites also include information on ERAS and/or links to the ERAS website.

Visit these websites for information on ERAS:
http://www.aamc.org
http://www.ecfmg.org

Step(s)/Component(s) Taken
Recipient of Transcript
One or more USMLE Steps Medical licensing authority
FSMB
All three USMLE Steps; or
Step 1 and Step 2 CK and CS (if required), only when registered for or after taking Step 3
Any recipient
FSMB
Step 1 and/or Step 2 CK and/or CS only, registered by ECFMG Any recipient other than a medical licensing authority
ECFMG
Step 1 and/or Step 2 CK and/or CS only, registered by NBME Any recipient other than a medical licensing authority
NBME



http://www.usmle.org/bulletin/2006/scoring.htm

No comments:

Post a Comment