Restoring reproducibility of Jupyter notebooks

Jupyter notebooks-documents that contain live code, equations,visualizations, and narrative text-now are among the most popular means to compute, present, discuss and disseminate scientificfindings. In principle, Jupyter notebooks should easily allow to reproduce and extend scientific computations and their findings; butin practice, this is not the case. The individual code cells in Jupyternotebooks can be executed in any order, with identifier usages preceding their definitions and results preceding their computations.In a sample of 936 published notebooks that would be executablein principle, we found that 73% of them would not be reproduciblewith straightforward approaches, requiring humans to infer (andoften guess) the order in which the authors created the cells.In this paper, we present an approach to (1) automatically satisfydependencies between code cells to reconstruct possible executionorders of the cells; and (2) instrument code cells to mitigate theimpact of non-reproducible statements (i.e., random functions) inJupyter notebooks. Our Osiris prototype takes a notebook as inputand outputs the possible execution schemes that reproduce theexact notebook results. In our sample, Osiris was able to reconstructsuch schemes for 82.23% of all executable notebooks, which hasmore than three times better than the state-of-the-art; the resultingreordered code is valid program code and thus available for furthertesting and analysis.