[Cluster][Discussion] Local cluster setup on MacOs- minikube and kind

[jankalinic]

This PR adds systemtests/cluster-setup/ - a small set of ready-to-run scripts that provision a local Kubernetes cluster for console-operator systemtests, so nobody has to sit down and work out cluster networking, OLM installation, or Strimzi/Kafka/Console wiring from scratch just to test something locally.

Whats added:

• kind - the recommended option. Maps ports 80/443 straight to the host at cluster-creation time and binds ingress-nginx via hostPort, so everything is reachable portlessly with no sudo and no background process to babysit. This matters especially on macOS, where Docker Desktop/Podman Desktop/Colima run the container engine inside a hidden VM, so a plain minikube setup (systemtests/scripts/setup-minikube.sh, which assumes the node IP is directly reachable — true on Linux CI, not true on macOS) doesn't work out of the box.
• minikube - an alternative for anyone who wants minikube specifically. Includes a script to switch to portless access via minikube tunnel (one sudo prompt, everything else automated) for parity with kind's UX.
• common - cluster-agnostic scripts (setup-catalogsource.sh, deploy-example-console.sh) that install OLM + catalogSource and deploy Strimzi, a Kafka cluster, the Console operator, and a Console instance, using the project's existing examples/kafka and examples/console quickstart manifests.

All scripts are idempotent (safe to re-run), include teardown scripts (full or --keep-cluster), and auto-detect podman vs. docker so the same scripts work on both macOS and Linux.

Note: minikube/create-cluster.sh is not a replacement for systemtests/scripts/setup-minikube.sh
They solve different problems. systemtests/scripts/setup-minikube.sh builds the Console images from source and pushes them into the cluster's registry - it's part of the CI build pipeline. minikube/create-cluster.sh does none of that: it only provisions the cluster itself (starts minikube, sets up ingress + SSL passthrough, exposes it to the host) and expects you to point it at an already-built operator image via common/setup-catalogsource.sh. These scripts assume you already know how to build and push your own images (or using a published ones) - building/pushing images is intentionally outside their scope in case you just want to setup a cluster.

PR also includes a small systemtests/config.yaml tweak: default CONSOLE_OLM_CATALOG_SOURCE_NAMESPACE from openshift-marketplace to olm (matches this setup's default on non-OCP clusters), and bumps the referenced console-operator bundle version.

[MikeEdgar]

I've only just browsed through this PR, but it looks very promising so far. I am thinking cluster-setup might even belong in a top-level tools directory if it could be used for other things. E.g., this probably replaces the setup-minikube.sh script recently added, no?

edit: I now see this 😆

Note: minikube/create-cluster.sh is not a replacement for systemtests/scripts/setup-minikube.sh

[jankalinic]

Yea, i wanted to bring you the PoC of both cluster setups first - show the difference between them and talk about the macOS "tunnel", that's (to me) annoying to run, once you deploy console. It's not as smooth as using the kind cluster, but i wanted to show, that it is still a possible solution. Regarding placement +1. Regarding the setup-minikube - sure, i did not want to touch that build part, but i guess we could modify the current MAKE or create a build script to keep BUILD and CLUSTER SETUP as two independent stages.

[jankalinic]

So what do you think @MikeEdgar , should i keep only the kind cluster setup, or both? Also i do not have linux machine right now, so i only hope it's working. If you want i can address the build script and/or change the make as well, not sure if it's right to do in this PR though.

[MikeEdgar]

So what do you think @MikeEdgar , should i keep only the kind cluster setup, or both? Also i do not have linux machine right now, so i only hope it's working. If you want i can address the build script and/or change the make as well, not sure if it's right to do in this PR though.

I am interested in having both. I will try to test them on Linux today and see how they behave and suggest any changes if needed. Changing other scripts to use this can be in another PR.

[MikeEdgar]

On Linux, during creation of the nginx ingress controller, I ran into a security issue that ultimately required me to install the ip_tables kernel module. Maybe we could add a check during startup if the OS is Linux, the runtime is podman in rootless mode, and the ip_tables module is not detected.

Podman rootless flag:

podman info | yq .host.security.rootless

Checking for ip_tables module on Linux:

if [ -z "$(lsmod | grep ^ip_tables)" ] ; then
    echo "Kernel module ip_tables is required for podman on Linux in rootless mode"
    # exit the script...
fi

[jankalinic]

On Linux...

Thank you so much for testing that! I'm adding the condition.

[MikeEdgar]

I tested the minikube setup and I hit an issue (again) with podman rootless. Based on the Minikube driver docs [1] I think the list below would be the most reliable options. Basically "docker" if available, otherwise the platform-specific VM-based runtime. Podman is listed as experimental for both OSes, so probably best to stay away from it for now.

• MacOS
  • Docker - VM + Container (preferred)
  • VFkit - VM (preferred)
• Linux
  • Docker - container-based (preferred)
  • KVM2 - VM-based (preferred)

[1] https://minikube.sigs.k8s.io/docs/drivers/

[jankalinic]

@MikeEdgar i applied the suggestion around engines, tested on my mac, documented the differences and emphasized usage of those in docs. If you want to try it on linux i think it's ready.

After this PR i will dig into local registries and building images + pushing/loading using scripts for easier local test setup.

(I've been playing around with the yaml and olm upgrades and it's a bit complex for the entry level users that want to run these tests without much effort, so i'd like to automate it as well - we have that in the github workflows and you already though about it in the setup-minikube script, so the integration of it would be my next step)

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[MikeEdgar]

If you want to try it on linux i think it's ready.

I hope to try this today or tomorrow.