Skip to content

STO-691: fix serverless update --model-reference/--clear-models on GPU endpoints - #340

Open
jebenexer wants to merge 1 commit into
mainfrom
benjaminbrannaka/sto-691-runpodctl-serverless-update-model-reference-clear-models
Open

STO-691: fix serverless update --model-reference/--clear-models on GPU endpoints#340
jebenexer wants to merge 1 commit into
mainfrom
benjaminbrannaka/sto-691-runpodctl-serverless-update-model-reference-clear-models

Conversation

@jebenexer

@jebenexer jebenexer commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Fixes STO-691.

Problem

serverless update <id> --model-reference <ref> and --clear-models both
failed on every GPU endpoint:

{"error":"failed to update model references: graphql error: gpuId(s) is required for a gpu endpoint","code":"graphql_error"}

Root cause

UpdateEndpointModels round-trips the endpoint's current config through
saveEndpoint, reading the GPU pool id from endpoint.GpuIDs. But REST
GET /v1/endpoints/{id} never returns gpuIds — only gpuTypeIds, an array
of gpu type names ("NVIDIA A40"), a different identifier space from the
pool ids saveEndpoint wants ("ADA_24"). So GpuIDs was always empty, and
saveEndpoint rejects an empty gpuIds on any endpoint without instanceIds.

Fix

When the REST read yields an empty gpuIds and a non-empty gpuTypeIds,
UpdateEndpointModels resolves the pool id via ResolveServerlessGpuPoolID
the same helper cmd/serverless/create.go uses for --gpu-id. An unknown gpu
type now errors instead of sending an empty gpuIds on to saveEndpoint.

Also adds the gpuIds vs gpuTypeIds REST/GraphQL asymmetry to AGENTS.md.

Testing

  • internal/api/endpoints_test.go: gpuTypeIds resolves to the right pool id
    in the saveEndpoint input; an unresolvable type fails loudly and never
    reaches the write.

  • e2e/cli_test.go: TestCLI_ServerlessUpdateModelReferenceOnGPUEndpoint
    creates a real GPU endpoint and exercises --model-reference and
    --clear-models — the gap called out in the STO-360 test plan.

  • go vet ./..., go build ./..., go test ./..., govulncheck ./...

  • go test -tags e2e ./e2e (needs a live API key against dev.runpod.io)


Note

Medium Risk
Touches serverless endpoint updates that full-replace config via saveEndpoint; scope is narrow (GPU id resolution on model-reference/clear-models) but mistakes could mis-assign GPU pools or break updates.

Overview
Fixes STO-691: serverless update --model-reference and --clear-models no longer fail on GPU endpoints with gpuId(s) is required for a gpu endpoint.

Those updates round-trip the endpoint through GraphQL saveEndpoint after a REST GET. REST only returns gpuTypeIds (display names), not gpuIds (pool ids like ADA_24), so gpuIds was always empty on write. UpdateEndpointModels now maps gpuTypeIds → pool id via ResolveServerlessGpuPoolID (same as create) when gpuIds is missing; unknown types error instead of sending an empty gpuIds. The Endpoint struct gains GpuTypeIDs for the REST read shape, and AGENTS.md documents the asymmetry.

Unit tests cover successful resolution and unresolvable GPU types; a new e2e test creates a GPU endpoint and exercises both update flags.

Reviewed by Cursor Bugbot for commit c8e04d4. Configure here.

…tModels

REST GET /v1/endpoints/{id} never returns "gpuIds" (the graphql
pool-id string saveEndpoint's write side expects), only "gpuTypeIds"
(an array of gpu type names, e.g. "NVIDIA A40"). UpdateEndpointModels
round-tripped endpoint.GpuIDs straight from that REST read, which was
always empty for a GPU endpoint, so saveEndpoint rejected every
`serverless update --model-reference`/`--clear-models` call on a GPU
endpoint with "gpuId(s) is required for a gpu endpoint".

Resolve gpuTypeIds to a pool id via ResolveServerlessGpuPoolID (the
same helper cmd/serverless/create.go uses for --gpu-id) whenever the
REST read comes back with an empty gpuIds and a non-empty gpuTypeIds.
An unresolvable gpu type now fails loudly instead of passing an empty
gpuIds on to saveEndpoint.

Testing:
- internal/api/endpoints_test.go: gpuTypeIds resolves to the expected
  pool id in the saveEndpoint input, and an unresolvable type errors
  without ever reaching the write.
- e2e/cli_test.go: TestCLI_ServerlessUpdateModelReferenceOnGPUEndpoint
  creates a real GPU endpoint and exercises --model-reference and
  --clear-models, the gap called out in the STO-360 test plan.
- AGENTS.md documents the gpuIds vs gpuTypeIds REST/GraphQL asymmetry.

Fixes STO-691.
@jebenexer
jebenexer force-pushed the benjaminbrannaka/sto-691-runpodctl-serverless-update-model-reference-clear-models branch from 0fb29f0 to c8e04d4 Compare September 10, 2026 02:28
@jebenexer

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c8e04d4. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant