Learning a similarity-based distance measure for image database organization from human partitionings of an image set

In this paper we employ human judgments of image similarity to improve the organization of an image database. We first derive a statistic, κ_B which measures the agreement between two partitionings of an image set. κ_B is used to assess agreement both amongst and between human and machine partitionings. This provides a rigorous means of choosing between competing image database organization systems, and of assessing the performance of such systems with respect to human judgments. Human partitionings of an image set are used to define a similarity value based on the frequency with which images are judged to be similar. When this measure is used to partition an image set using a clustering technique, the resultant partitioning agrees better with human partitionings than any of the feature-space-based techniques investigated. Finally, we investigate the use of multilayer perceptrons and a distance learning network to learn a mapping from feature space to this perceptual similarity space. The distance learning network is shown to learn a mapping which results in partitionings in excellent agreement with those produced by human subjects.