When working on analytical projects, it is important to use a requirements.txt file to specify the dependencies required for the project. This file lists all the external libraries and packages that the project relies on, along with their version numbers, ensuring that everyone working on the project is using the same set of dependencies. This helps avoid issues with version conflicts and makes the project more reproducible.
Creating requirements.txt is easy, just execute:
pip freeze > requirements.txt
A requirements.txt file is important for CI/CD, as it helps to automate the environment building in the pipeline. To install these requirements, simply run:
pip install -r requirements.txt
There is one problem with requirements.txt file, though. It only contains first level dependencies and next level of dependencies are not necessarily locked in. The solution is to create file typically called requirements. in, using pip-compile.
This article assumes that you use Python for your analytical project. There are similar tools exist for other environments.
This article reflects my personal views and opinions only, which may be different from the companies and employers that I am associated with.