The current rate of cerebral palsy (CP) per live births in Australia is between 0.14% and 0.2%, worldwide the rate has been static for 60 years at 0.2%. Typically a CP diagnosis is delayed until around age 2 years; this delay decreases the likelihood of a long-term positive patient outcome. Current early detection is by visual examination of newborns 10 to 20 weeks post gestation. A screening program based on filming babies and processing the video via artificial intelligence (AI) will allow increased early detection and intervention. This paper outlines the practical development, and initial results from, a recurrent deep neural net solution for the classification of newborn videos, specifically targeting CP, using the largest fidgety movements dataset in Australia.