Rating format research revisited: format effectiveness and acceptability depend on rater characteristics

Rating format research has largely been ignored since Landy and Farr's (1980) call for a moratorium over a decade ago. Their conclusion that ratings were not affected by changes in scale format was based on research that treated all raters alike. However, individuals differ in the way in which they perceive and integrate information. This article investigates the proposition that differences in rating accuracy associated with different rating formats are contingent on rater characteristics. The study tested the rating accuracy and affective reactions toward performance appraisal of field-dependent (FD) and field-independent (FI) raters on four different performance measures. As hypothesized, FIs were more accurate raters than FDs only when scale formats were holistic, and only FDs' ratings were significantly affected by the level of structure in the scale format. FI raters were also more confident in their ratings and less frustrated and confused with the rating task than were FDs.