On 23 June 2016, the British electorate voted to leave the European Union (EU). We analyse vote and turnout shares across 380 local authority areas in the United Kingdom. We find that exposure to the EU in terms of immigration and trade provides relatively little explanatory power for the referendum vote. Instead, we find that fundamental characteristics of the voting population were key drivers of the Vote Leave share, in particular their education profiles, their historical dependence on manufacturing employment as well as low income and high unemployment. At the much finer level of wards within cities, we find that areas with deprivation in terms of education, income and employment were more likely to vote Leave. Our results indicate that a higher turnout of younger voters, who were more likely to vote Remain, would not have overturned the referendum result. We also compare our UK results to voting patterns for the far-right leader Marine Le Pen in the 2017 French presidential election. We find similar factors driving the French vote. An out-of-sample prediction of the French vote using UK estimates performs reasonably well.