STO-691: fix serverless update --model-reference/--clear-models on GPU endpoints - #340
Open
jebenexer wants to merge 1 commit into
Conversation
…tModels
REST GET /v1/endpoints/{id} never returns "gpuIds" (the graphql
pool-id string saveEndpoint's write side expects), only "gpuTypeIds"
(an array of gpu type names, e.g. "NVIDIA A40"). UpdateEndpointModels
round-tripped endpoint.GpuIDs straight from that REST read, which was
always empty for a GPU endpoint, so saveEndpoint rejected every
`serverless update --model-reference`/`--clear-models` call on a GPU
endpoint with "gpuId(s) is required for a gpu endpoint".
Resolve gpuTypeIds to a pool id via ResolveServerlessGpuPoolID (the
same helper cmd/serverless/create.go uses for --gpu-id) whenever the
REST read comes back with an empty gpuIds and a non-empty gpuTypeIds.
An unresolvable gpu type now fails loudly instead of passing an empty
gpuIds on to saveEndpoint.
Testing:
- internal/api/endpoints_test.go: gpuTypeIds resolves to the expected
pool id in the saveEndpoint input, and an unresolvable type errors
without ever reaching the write.
- e2e/cli_test.go: TestCLI_ServerlessUpdateModelReferenceOnGPUEndpoint
creates a real GPU endpoint and exercises --model-reference and
--clear-models, the gap called out in the STO-360 test plan.
- AGENTS.md documents the gpuIds vs gpuTypeIds REST/GraphQL asymmetry.
Fixes STO-691.
jebenexer
force-pushed
the
benjaminbrannaka/sto-691-runpodctl-serverless-update-model-reference-clear-models
branch
from
September 10, 2026 02:28
0fb29f0 to
c8e04d4
Compare
Collaborator
Author
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit c8e04d4. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes STO-691.
Problem
serverless update <id> --model-reference <ref>and--clear-modelsbothfailed on every GPU endpoint:
Root cause
UpdateEndpointModelsround-trips the endpoint's current config throughsaveEndpoint, reading the GPU pool id fromendpoint.GpuIDs. But RESTGET /v1/endpoints/{id}never returnsgpuIds— onlygpuTypeIds, an arrayof gpu type names (
"NVIDIA A40"), a different identifier space from thepool ids
saveEndpointwants ("ADA_24"). SoGpuIDswas always empty, andsaveEndpointrejects an emptygpuIdson any endpoint withoutinstanceIds.Fix
When the REST read yields an empty
gpuIdsand a non-emptygpuTypeIds,UpdateEndpointModelsresolves the pool id viaResolveServerlessGpuPoolID—the same helper
cmd/serverless/create.gouses for--gpu-id. An unknown gputype now errors instead of sending an empty
gpuIdson tosaveEndpoint.Also adds the
gpuIdsvsgpuTypeIdsREST/GraphQL asymmetry to AGENTS.md.Testing
internal/api/endpoints_test.go:gpuTypeIdsresolves to the right pool idin the
saveEndpointinput; an unresolvable type fails loudly and neverreaches the write.
e2e/cli_test.go:TestCLI_ServerlessUpdateModelReferenceOnGPUEndpointcreates a real GPU endpoint and exercises
--model-referenceand--clear-models— the gap called out in the STO-360 test plan.go vet ./...,go build ./...,go test ./...,govulncheck ./...go test -tags e2e ./e2e(needs a live API key against dev.runpod.io)Note
Medium Risk
Touches serverless endpoint updates that full-replace config via saveEndpoint; scope is narrow (GPU id resolution on model-reference/clear-models) but mistakes could mis-assign GPU pools or break updates.
Overview
Fixes STO-691:
serverless update --model-referenceand--clear-modelsno longer fail on GPU endpoints withgpuId(s) is required for a gpu endpoint.Those updates round-trip the endpoint through GraphQL
saveEndpointafter a RESTGET. REST only returnsgpuTypeIds(display names), notgpuIds(pool ids likeADA_24), sogpuIdswas always empty on write.UpdateEndpointModelsnow mapsgpuTypeIds→ pool id viaResolveServerlessGpuPoolID(same as create) whengpuIdsis missing; unknown types error instead of sending an emptygpuIds. TheEndpointstruct gainsGpuTypeIDsfor the REST read shape, and AGENTS.md documents the asymmetry.Unit tests cover successful resolution and unresolvable GPU types; a new e2e test creates a GPU endpoint and exercises both update flags.
Reviewed by Cursor Bugbot for commit c8e04d4. Configure here.