On computational reproducibility -- surely requirements.txt is enough? Right?

I got an interesting question by email today.

How is using Docker containers any better than strict descriptiuons of all packages with requirements.txt (Python) or Renv (R)?

There are some benefits to using containers:

Your workflow has a combination of multiple languages. Maybe you have bash scripts for running genome alignments, followed by stats in R and ML in Python. A container will keep track of all of it.
External binaries like Samtools and BWA can be included in the container.
Requirements.txt doesn't enforce a Python version. Some packages only work with certain versions of Python or R and so you'll have a bad time if you're trying to reproduce an old study and you don't know what version of Python or R was used.
Installing an older Python version may not work. Code from 10 years ago is very difficult to get working properly unless it was designed with longevity in mind [1].
Packages are in a non-official location that doesn't support requirements.txt. Sometimes, R or python packages are uploaded to and distributed from GitHub and they don't have version IDs that can be managed by requirements.txt.
Link rot is a significant problem in terms of long term reproducibility. It means that some datasets or code assets might not be available long term. These files can be included in the container.
R has over 300 system dependencies [2], and these may not behave consistently across different operating systems (think Windows, Mac and various flavours of Unix/Linux). These system dependencies also change over time and could cause the same code to behave differently. I presume the situation is the same for Python too.
It is faster to download and run a container than it is to install a stack of dependencies (most of the time). Containerising makes it much easier for the reproducer as compared to the developer, but sometimes the reproducer is the developer, just a bit older :)
If you have many projects happening, it can be a headache to maintain several different Python or R versions, swapping between them can be difficult even when using conda. By containerising, you keep the host computer tidy. The host computer doesn't even need to have an installation of R or Python.
The container is auditable - in that if the software is misconfigured the person reproducing can trace the problem. This is good in situations of large teams working on a common project.
Even when the requirements are stated, most scripts have errors [3]. In a way, containerising forces the developer to think carefully about how the code should go about accessing the data, so that it will work for anybody [4].
Using containers encourages you to reproduce your own work before you ship it.

Altuna Akalin also wrote about the need for controlling the environment [5].

You could also argue that the compute environment could be considered part of the research data and primary materials and should be retained for 5 years from the date of publication [6].

Containerising your work into a docker image won't solve all your problems. There are still many limitations. But it is a quantum leap for reproducibility. My recommendation is to start small. Take one small data analysis task and put it in a container. Then expand that a bit for your next project. Then after a while you'll be ready to run your entire study from a container. Good luck!

Search This Blog

Genome Spot

On computational reproducibility -- surely requirements.txt is enough? Right?

References

On computational reproducibility -- surely requirements.txt​ is enough? Right?

References

On computational reproducibility -- surely requirements.txt is enough? Right?