Skip to content

feat: emit commands.json with Docker build/run commands for delivery capture - #189

Open
itej89 wants to merge 4 commits into
ROCm:developfrom
itej89:feature/emit-commands-json
Open

itej89 wants to merge 4 commits into
ROCm:developfrom
itej89:feature/emit-commands-json

Conversation

@itej89

@itej89 itej89 commented Sep 16, 2026

Copy link
Copy Markdown

Summary

After run_models_from_manifest completes, writes a commands.json file alongside perf.csv containing the exact Docker build and run commands used for each model.

This enables client-perf-hub model_runner to capture these commands in MongoDB for customer delivery packages — standalone, madengine-free instructions that customers can use to reproduce benchmark results.

Changes

src/madengine/core/docker.py (1 line)

  • Store the assembled docker run command on self.docker_run_cmd before executing it.

src/madengine/execution/container_runner.py (75 lines)

  • Capture docker_run_cmd from the Docker instance into run_results after container creation.
  • Pass docker_run_cmd through to successful_runs dicts.
  • Add _emit_commands_json() method that writes commands.json after all models complete.

commands.json Format

[
  {
    "model": "moonshotai/Kimi-K3",
    "docker_build_cmd": "docker build --network=host -t ci-... --pull -f docker/pyt_vllm.ubuntu.amd.Dockerfile ...",
    "docker_run_cmd": "docker run -t -d --network host -u root --group-add video ...",
    "base_docker_tag": "vllm/vllm-openai-rocm:nightly",
    "base_docker_digest": "vllm/vllm-openai-rocm@sha256:abc123...",
    "dockerfile_path": "docker/pyt_vllm.ubuntu.amd.Dockerfile"
  }
]

Design Notes

  • Best-effort: failures to write commands.json are logged but never block the run.
  • No breaking changes: commands.json is a new output file; all existing behavior is unchanged.
  • Build info (docker_build_cmd, base_docker, docker_sha) was already captured in build_info dict — this change just persists it to a file.
  • The docker_run_cmd was previously constructed but discarded — now stored on Docker.docker_run_cmd.
  • base_docker_digest uses the immutable image@sha256:... format so customers can pin to the exact image.

Context

  • Feature request: AMD-ROCm-Internal/client-perf-hub#1172
  • ADR: ADR-004 — Delivery Command Capture
  • SRS: SRS-DL-001 requirements DL-CAP-001 through DL-CAP-007

Test plan

  • Existing tests pass (no behavior change for existing flows)
  • commands.json is written after a successful madengine run --manifest-file
  • commands.json contains correct docker_build_cmd matching build_manifest.json
  • commands.json contains correct docker_run_cmd with GPU flags and mounts
  • base_docker_digest is populated when docker manifest inspect succeeds
  • base_docker_digest is null when digest cannot be resolved (DL-CAP-007)
  • Missing commands.json write does not fail the run

🤖 Generated with Claude Code

After `run_models_from_manifest` completes, writes a `commands.json`
file alongside `perf.csv` containing the exact Docker build and run
commands used for each model. This enables downstream tools
(client-perf-hub model_runner) to capture these commands in MongoDB
for customer delivery packages.

Contents per model entry:
- docker_build_cmd: exact `docker build` command
- docker_run_cmd: exact `docker run` command with GPU flags/mounts/env
- base_docker_tag: human-readable base image tag
- base_docker_digest: immutable digest-pinned reference (image@sha256:...)
- dockerfile_path: relative path to Dockerfile

Best-effort: failures to write commands.json are logged but never
block the run.

Ref: AMD-ROCm-Internal/client-perf-hub#1172
Spec: SRS-DL-001 (DL-CAP-001 through DL-CAP-007)

Co-Authored-By: Claude <noreply@anthropic.com>
Tej Kiran and others added 3 commits September 16, 2026 09:23
Build info (docker_build_cmd, base_docker, docker_sha, dockerfile)
is already in build_manifest.json which is available in both split
and direct paths. commands.json only needs to capture the docker run
command which is constructed at run time on the GPU node.

model_runner reads both files:
- build_manifest.json → build info (from build machine)
- commands.json → docker run command (from run machine)

Co-Authored-By: Claude <noreply@anthropic.com>
The docker run command (GPU flags, mounts, security options) is
independent of the model — it's the same container execution
context. Emit a single object instead of an array.

Output: {"docker_run_cmd": "docker run -t -d --network host ..."}

Co-Authored-By: Claude <noreply@anthropic.com>
….json

No separate file needed. build_manifest.json already has all the build
info — just append docker_run_cmd to it after the run completes.
model_runner reads one file with everything.

Co-Authored-By: Claude <noreply@anthropic.com>
@coketaste

Copy link
Copy Markdown
Collaborator

docs/superpowers/ was removed from develop and main (#193, #194) and must not be reintroduced. This PR head still contains those plan/spec files (inherited from an older develop). Please delete docs/superpowers/ (and keep it gitignored) or rebase onto current develop. A merge without that cleanup can put the folder back in the file list even if git would often drop unchanged copies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants