Background: Criteria for mild cognitive impairment require objective evidence of a memory deficit but do not require objective evidence of memory decline. Application of these criteria may therefore result in the misclassification of older patients with memory decline as normal because their neuropsychological test performance at a single point in time may be within normal limits. This study aimed to identify and characterize older people with memory decline. Method: Word list delayed recall (WLDR) test performance was assessed on five occasions during a 2-year period in a cohort of healthy older individuals. Older people with declining (n = 35) and nondeclining (n = 66) WLDR scores were identified. Both subgroups were then compared on apoE genotype, Clinical Dementia Rating, and neuropsychological test performance at the fifth assessment. Results: Thirty-four percent of the group with declining memory recorded a Clinical Dementia Rating of 0.5, compared with 5% of the nondeclining memory group. No between-group differences were observed in cognitive domains other than memory, self-reported cognitive failures, or the proportion of each group carrying the apoE epsilon 4 allele. Conclusions: A large proportion of healthy older individuals show memory decline, which may represent the early stages of a potentially more severe cognitive impairment. Further investigation is necessary to determine the relationship between apoE genotype, self-reported cognitive impairment, and memory decline in older people.