Background Advanced paternal age (APA) is associated with an increased risk of neurodevelopmental disorders such as autism and schizophrenia, as well as with dyslexia and reduced intelligence. The aim of this study was to examine the relationship between paternal age and performance on neurocognitive measures during infancy and childhood. Methods and Findings A sample of singleton children (n = 33, 437) was drawn from the US Collaborative Perinatal Project. The outcome measures were assessed at 8 mo, 4 y, and 7 y (Bayley scales, Stanford Binet Intelligence Scale, Graham-Ernhart Block Sort Test, Wechsler Intelligence Scale for Children, Wide Range Achievement Test). The main analyses examined the relationship between neurocognitive measures and paternal or maternal age when adjusted for potential confounding factors. Advanced paternal age showed significant associations with poorer scores on all of the neurocognitive measures apart from the Bayley Motor score. The findings were broadly consistent in direction and effect size at all three ages. In contrast, advanced maternal age was generally associated with better scores on these same measures. Conclusions The offspring of older fathers show subtle impairments on tests of neurocognitive ability during infancy and childhood. In light of secular trends related to delayed fatherhood, the clinical implications and the mechanisms underlying these findings warrant closer scrutiny.