Conversation
After `run_models_from_manifest` completes, writes a `commands.json` file alongside `perf.csv` containing the exact Docker build and run commands used for each model. This enables downstream tools (client-perf-hub model_runner) to capture these commands in MongoDB for customer delivery packages. Contents per model entry: - docker_build_cmd: exact `docker build` command - docker_run_cmd: exact `docker run` command with GPU flags/mounts/env - base_docker_tag: human-readable base image tag - base_docker_digest: immutable digest-pinned reference (image@sha256:...) - dockerfile_path: relative path to Dockerfile Best-effort: failures to write commands.json are logged but never block the run. Ref: AMD-ROCm-Internal/client-perf-hub#1172 Spec: SRS-DL-001 (DL-CAP-001 through DL-CAP-007) Co-Authored-By: Claude <noreply@anthropic.com>
itej89
requested review from
Cemberk,
Rohan138,
coketaste,
gargrahul and
leconcio
as code owners
September 16, 2026 00:07
Build info (docker_build_cmd, base_docker, docker_sha, dockerfile) is already in build_manifest.json which is available in both split and direct paths. commands.json only needs to capture the docker run command which is constructed at run time on the GPU node. model_runner reads both files: - build_manifest.json → build info (from build machine) - commands.json → docker run command (from run machine) Co-Authored-By: Claude <noreply@anthropic.com>
The docker run command (GPU flags, mounts, security options) is
independent of the model — it's the same container execution
context. Emit a single object instead of an array.
Output: {"docker_run_cmd": "docker run -t -d --network host ..."}
Co-Authored-By: Claude <noreply@anthropic.com>
….json No separate file needed. build_manifest.json already has all the build info — just append docker_run_cmd to it after the run completes. model_runner reads one file with everything. Co-Authored-By: Claude <noreply@anthropic.com>
Collaborator
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
After
run_models_from_manifestcompletes, writes acommands.jsonfile alongsideperf.csvcontaining the exact Docker build and run commands used for each model.This enables
client-perf-hubmodel_runner to capture these commands in MongoDB for customer delivery packages — standalone, madengine-free instructions that customers can use to reproduce benchmark results.Changes
src/madengine/core/docker.py(1 line)docker runcommand onself.docker_run_cmdbefore executing it.src/madengine/execution/container_runner.py(75 lines)docker_run_cmdfrom theDockerinstance intorun_resultsafter container creation.docker_run_cmdthrough tosuccessful_runsdicts._emit_commands_json()method that writescommands.jsonafter all models complete.commands.jsonFormat[ { "model": "moonshotai/Kimi-K3", "docker_build_cmd": "docker build --network=host -t ci-... --pull -f docker/pyt_vllm.ubuntu.amd.Dockerfile ...", "docker_run_cmd": "docker run -t -d --network host -u root --group-add video ...", "base_docker_tag": "vllm/vllm-openai-rocm:nightly", "base_docker_digest": "vllm/vllm-openai-rocm@sha256:abc123...", "dockerfile_path": "docker/pyt_vllm.ubuntu.amd.Dockerfile" } ]Design Notes
commands.jsonare logged but never block the run.commands.jsonis a new output file; all existing behavior is unchanged.docker_build_cmd,base_docker,docker_sha) was already captured inbuild_infodict — this change just persists it to a file.docker_run_cmdwas previously constructed but discarded — now stored onDocker.docker_run_cmd.base_docker_digestuses the immutableimage@sha256:...format so customers can pin to the exact image.Context
Test plan
commands.jsonis written after a successfulmadengine run --manifest-filecommands.jsoncontains correctdocker_build_cmdmatchingbuild_manifest.jsoncommands.jsoncontains correctdocker_run_cmdwith GPU flags and mountsbase_docker_digestis populated whendocker manifest inspectsucceedsbase_docker_digestisnullwhen digest cannot be resolved (DL-CAP-007)commands.jsonwrite does not fail the run🤖 Generated with Claude Code