Low Correlation between Subjective and Objective Measures of Knowledge on Surgery Clerkships

Timothy M. Farrell, Geoffrey P. Kohn, Stacey M. Owen, Michael O. Meyers, Robyn A. Stewart, Anthony A. Meyer

Research output: Contribution to journalArticleResearchpeer-review

22 Citations (Scopus)

Abstract

Background: Medical student knowledge is assessed during surgical clerkships subjectively and objectively. Subjective evaluation depends on faculty assessment during clinical and didactic interactions. Objective measurement derives from standardized tools, such as the National Board of Medical Examiners Surgery Subject test (shelf). Few efforts have been made to characterize the correlation between subjective and objective measures of medical knowledge. Study Design: All 308 third-year medical students who completed the 8-week surgery clerkship at the University of North Carolina at Chapel Hill between July 2005 and June 2007 received subjective assessment of knowledge on 3 clinical rotations (one 4-week core and two 2-week elective rotations) and a longitudinal small-group tutorial. Faculty evaluators assigned percentile scores to rate students' knowledge base relative to their peers. In addition, students took the shelf test the last day of clerkship, and percentile scores were assigned based on National Board of Medical Examiners-supplied normative data from first-time test-takers within the same academic quarter. Subjective versus objective knowledge scores were plotted overall, and Pearson product-moment correlation coefficients were generated for core, elective, and tutorial assessments. Results: There were only weak linear relationships noted between subjective faculty-assigned knowledge scores and objective shelf scores. Pearson correlations were 0.24 for core rotations (4 weeks exposure), 0.14 for elective rotations (2 weeks exposure), and 0.22 for tutorials (1-hour exposure/week during 8 weeks), with p values <0.0001. Conclusions: Faculty assessment of knowledge is only weakly correlated with shelf performance. Faculty evaluations after 4-week rotations or longitudinal small-group interactions are better correlated with shelf scores than after 2-week electives.

Original languageEnglish
Pages (from-to)680-683
Number of pages4
JournalJournal of the American College of Surgeons
Volume210
Issue number5
DOIs
Publication statusPublished - May 2010
Externally publishedYes

Cite this