Know your population and know your model: using model-based regression and poststratification to generalize findings beyond the observed sample

Psychology research often focuses on interactions, and this has deep implications for inference from nonrepresentative samples. For the goal of estimating average treatment effects, we propose to fit a modelallowing treatment to interact with background variables and then average over the distribution of thesevariables in the population. This can be seen as an extension of multilevel regression and poststratification(MRP), a method used in political science and other areas of survey research, where researcherswish to generalize from a sparse and possibly nonrepresentative sample to the general population. In thisarticle, we discuss areas where this method can be used in the psychological sciences. We use our methodto estimate the norming distribution for the Big Five Personality Scale using open source data. We arguethat large open data sources like this and other collaborative data sources can potentially be combinedwith MRP to help resolve current challenges of generalizability and replication in psychology