15  Containers

Similar to dependencies, it is important to record and manage your development environments. This involves documenting the specific configurations, tools, and versions used during development to ensure that everything runs consistently across different setups.

By recording your environments, you create a reproducible framework that significantly reduces the infamous “it works on my machine” syndrome, meaning that the code works on one machine but not on others.

Environment management includes specifying your operating system, programming language (and their versions), system libraries, and any other software or tools required. This practice not only helps in maintaining consistency across development, testing, and production stages but also streamlines onboarding for new team members, as they can quickly set up their environment to match the recorded specifications.

15.1 Containerisation

Containers remove the need to manually install or troubleshoot anything if something is not working on another user’s machine. They function as a self-contained unit, where everything is bundled in a single package, simulating a complete operating environment, and are bundled in a single file (image).

The definition files, like Dockerfiles or Singularity definition files, are essentially instruction manuals to build a container image.

Consider a container image as a detailed script for a movie, outlining every scene and dialogue. When the decision is made to shoot the movie (“executing” the container image), a temporary set is constructed. This set serves as the container, where all filming and acting occurs, confined to this controlled environment. After the shooting wraps up (or the task is completed), the set is dismantled, and all props are stored away, but the script (the container image) remains unaltered. This allows to recreate scenes as and if needed, with each shoot starting from the same original script, ensuring consistency across takes while leaving the script intact for future use.

Containers can be useful for various purposes:

  • For example, if installing specific software is complicated or incompatible with your OS, you might check if an image is available and run the software from the container.
  • Likewise, if you want to make sure that the people you are working with use the exact same environment, you could provide them with an image of your container.
  • If you are facing issues due to different system architectures you can distribute a definition file to build an image that tailors to different machines. While it might not replicate your environment exactly, it often provides a sufficiently close alternative.

Popular containerization solutions:

Both Docker and Singularity are robust options for containerization, with Docker being more widely used in general software development, and Singularity addressing the specific needs of HPC environments.

Docker’s official samples and examples

Further reading

15.2 Advantages and disadvantages of using containers

Containers have gained widespread popularity due to their significant benefits in solving various challenges:

  • They enable a smooth transition of workflows across different operating systems and configurations.
  • They address the common issue of software behaving differently on different machines by providing a consistent runtime environment.
  • For software with nested dependencies, containers can be a vital tool for ensuring long-term reproducibility.
  • Containers bundle the software in an isolated file, and that simplifies the management, installation, and removal of the software (compared to a traditional uninstallation process).

But it is important to consider the downsides:

  • The convenience of containers might lead to overlooking underlying software installation issues and not adhering to good software development practices.
  • There’s a risk of creating a new form of dependency - software that “only works in a specific container”, which could limit flexibility and interoperability.
  • Container images can grow in size substantially, especially if not carefully managed.
  • Modifying existing containers can sometimes be challenging, requiring a good understanding of the container’s configuration.
Caution

It’s crucial to source your container images from reputable and official channels. There have been instances where images were found to be malicious, so it is very important to apply the same caution as when installing software packages from untrusted sources.