Add CircleCI pipeline for automated Docker image builds and publishing

[DickPostma]

Summary

This PR introduces an initial CircleCI pipeline for automated Docker image validation, building, and publishing.

The goal is to move long-running image builds from developer workstations to a centralized CI/CD environment. Some DataSHIELD profile images can take several hours to build due to the large number of R packages they contain. Running these builds in CI improves reproducibility, traceability, and developer productivity.

Approach

The pipeline is intentionally designed to be generic and self-discovering.

Pull Requests

For pull requests, the pipeline will:

• Detect modified image directories automatically
• Verify that required files exist (Dockerfile and Makefile)
• Validate that name= and tag= are defined in the Makefile
• Check whether the target image tag already exists on Docker Hub
• Fail early if validation errors are found

No Docker images are built during PR validation.

Main Branch

After a PR is merged into main, the pipeline will:

• Detect modified image directories
• Build the corresponding Docker image(s)
• Push the resulting image(s) to Docker Hub

Why this approach?

The repository contains many independent image profiles and new profiles may be added at any time.

Instead of maintaining a hardcoded list of images in CircleCI, the pipeline automatically discovers image profiles based on the repository structure. This means that adding a new profile only requires creating a new directory containing a Dockerfile and Makefile.

Example:

donkey-henk/
├── Dockerfile
├── Makefile
└── README.md

[ymarcon]

FYI the base branch uses GH actions to build-push to Docker hub AND to produce an Attestation to track the build origin:
https://github.com/datashield/docker-rock/attestations
Is CircleCI able to do that?

[DickPostma]

My understanding was that these profile images were primarily built locally using the provided Makefiles. Stuart mentioned he typically builds them on his workstation, and at the time I didn't have access to the GitHub Actions configuration, so I assumed there was no centralized build process yet.

The motivation behind this PR was not CircleCI specifically, but avoiding long-running local builds. When I started working on the new molgenis-rock-possum profile, I found myself waiting ~7 hours for a local build and thought this would be a good candidate for CI automation.

It sounds like everything is already well covered with the existing GitHub Actions approach though 😉

[ymarcon]

@DickPostma thx for the context. Indeed I originally thought the base branch could be brought into the main one with a per directory approach. The difference is that the base branch is semver version based and tags all variants 6, 6.5, 6.5.1 etc

[ymarcon]

@DickPostma The other important feature is that the publication of the base image is always associated to a tag in this repository, in order to be able to track the image build source code.

[DickPostma]

@ymarcon For profile images, I wonder if a similar approach could be used. Most profiles are preconfigured with static package versions (https://wiki.datashield.org/en/opmanag/analysis-profiles/list), so they are typically built once and then remain unchanged for a long time.

My main motivation was simply to avoid long-running builds on Stuart's workstation 😉. Having those builds run in a central CI environment would be beneficial, even if the resulting image only needs to be built once.

For molgenis-rock-possum, the situation is slightly different. We want to provide a new network with the latest available package versions, so a tag-based release workflow would make a lot of sense there.

I'll discuss this further with @StuartWheater to determine what the most appropriate approach would be.

[ymarcon]

@DickPostma @StuartWheater I would strongly suggest to NOT name an image by molgenis as it may indicate that it is Molgenis specific, which is not the case. If you do anything Molgenis specific, do it in the Molgenis GH organisation. That is what I do for obiba.

[StuartWheater]

Being able to generate Analysis Profile Images using CI/CD environment is progress, but in truth is it didn't cause a problem doing it manually.

As for the names of profiles, the original scheme was aimed at having a name which allows research project, developers and study sites to have an abstract name with which to promote clarity in communications. I was wondering if should start to add LABEL information like author, maintainer, ... into the image.