Skip to content

feat(script): migrate agent/router/evaluator LLM keys to BYOK by model prefix#5607

Open
TheodoreSpeaks wants to merge 5 commits into
stagingfrom
feat/xai-byok-migration
Open

feat(script): migrate agent/router/evaluator LLM keys to BYOK by model prefix#5607
TheodoreSpeaks wants to merge 5 commits into
stagingfrom
feat/xai-byok-migration

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Summary

  • Extend the block→BYOK migration script with a model-aware mode (--model-map <modelPrefix>=<providerId>, e.g. grok=xai)
  • Lets existing LLM keys entered into agent/router/evaluator blocks migrate to workspace BYOK, so they keep using their own key instead of falling back to the rotating key once the provider joins the hosted pool

Why a new mode

The existing --map <blockType>=<providerId> only works for dedicated single-provider blocks. LLM-family blocks (agent, router, evaluator) share one model-gated apiKey subblock across every provider, so a block-type map would misattribute every provider's key. The new --model-map only takes the key when the block's selected model starts with the given prefix.

Type of Change

  • New feature

Testing

  • Dry run + live run verified against staging; migrated rows well-formed (iv:encrypted:authTag), created_by set, one key per workspace with full coverage of eligible blocks
  • bun run check:api-validation:strict passed
  • Biome formatting applied

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jul 12, 2026 2:57am

Request Review

@cursor

cursor Bot commented Jul 11, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
The script inserts encrypted workspace BYOK rows from user API key material; wrong --model-map or missing --require-key-prefix could attribute keys to the wrong provider, though dry-run, conflict handling, and the new guards reduce that risk.

Overview
Extends migrate-block-api-keys-to-byok.ts so LLM-family blocks (agent, router, router_v2, evaluator, translate, guardrails, pi) can migrate shared apiKey values into workspace BYOK using --model-map <modelPrefix>=<providerId> (e.g. grok=xai), only when the block’s selected model matches the prefix. --map is rejected for those block types to avoid mis-attributing one shared key to a single provider.

Adds optional --require-key-prefix <providerId>=<prefix> to skip stale keys left in the shared field after a model switch, and longest-prefix-first resolution so overlapping --model-map entries are deterministic. Startup logging and workspace discovery now account for model mappings alongside existing block-type maps; --map values are trimmed and lowercased.

Reviewed by Cursor Bugbot for commit 5cee10c. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts
@greptile-apps

greptile-apps Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds model-aware BYOK migration for shared LLM block credentials. The main changes are:

  • New --model-map support for model-prefix to provider mapping.
  • Rejection of unsafe --map usage for LLM-family blocks.
  • Optional --require-key-prefix filtering for stale shared keys.
  • Provider ID normalization and trimmed CLI parsing.
  • Workspace discovery for LLM-family blocks when model mappings are used.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
packages/db/scripts/migrate-block-api-keys-to-byok.ts Adds model-prefix based BYOK migration for LLM-family blocks and tightens CLI parsing around provider mapping.

Reviews (5): Last reviewed commit: "fix(script): resolve overlapping model p..." | Re-trigger Greptile

Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts Outdated
Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts

// LLM-family blocks share one model-gated `apiKey` subblock across every provider, so the
// key is only attributed to a provider when the block's selected `model` matches a prefix.
const LLM_FAMILY_BLOCK_TYPES = ['agent', 'router', 'evaluator'] as const

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TheodoreSpeaks what about translate, I know we deprecated vision? and idk if there are more or not

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — added `translate` and `guardrails` (both use `getModelOptions()` + the shared `getProviderCredentialSubBlocks()` apiKey, same as agent/router/evaluator, so they can hold a grok key). Left `vision` out on purpose: it is deprecated and its curated `VISION_MODEL_OPTIONS` list has no grok models, so it can never hold an xai key. The set is now ['agent','router','evaluator','translate','guardrails'].

@TheodoreSpeaks TheodoreSpeaks changed the title feat(byok): migrate agent/router/evaluator LLM keys to BYOK by model prefix feat(script): migrate agent/router/evaluator LLM keys to BYOK by model prefix Jul 12, 2026
…er ids, reject mixed mapping, add key-prefix guard
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

Addressed the review feedback in 1a3ef5f8:

  • @waleedlatif1 — added translate and guardrails to the LLM-family set (same getModelOptions() + shared getProviderCredentialSubBlocks() apiKey structure). Excluded deprecated vision since its curated model list has no grok models.
  • Greptile (whitespace provider rows) — provider ids are now .trim()-ed in both --map and --model-map parsers.
  • Greptile / Bugbot (mixed mapping duplicates)--map targeting an LLM-family block type is now rejected at startup, so an LLM block can never be processed by both the block-type and model-map paths.
  • Bugbot (stale keys misattributed) — added an opt-in --require-key-prefix <providerId>=<prefix> guard. The live xai run will use --require-key-prefix xai=xai- so a stale non-xai key left in the shared apiKey field after a model switch is skipped rather than migrated. (Note: a stale non-grok key would already have been failing that block's grok runs pre-migration, so this closes the remaining edge where it could shadow the hosted rotating key.)

@greptile review

Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts Outdated
Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

Round 2 addressed in 948721f2:

  • Bugbot (missing pi block) — good catch, added pi to LLM_FAMILY_BLOCK_TYPES. It uses getPiModelOptions() (a filter over getModelOptions()) + the shared getProviderCredentialSubBlocks() apiKey, and xai is in PI_SUPPORTED_PROVIDER_IDS, so Pi blocks can hold a grok key.
  • Greptile (trim prefix value)--require-key-prefix now trims the prefix too, so xai= xai- no longer rejects every valid xai-... key.

@greptile review

Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts
Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

Round 3 addressed in 02f06325:

  • Bugbot (active router type omitted, High) — correct, added router_v2. router.ts defines both RouterBlock (router, legacy/hideFromToolbar) and RouterV2Block (router_v2, active); both are registered and both use the shared getProviderCredentialSubBlocks() apiKey, so both can hold a grok key. Verified the full set against every block that spreads getProviderCredentialSubBlocks() — it's now agent, router, router_v2, evaluator, translate, guardrails, pi (complete).
  • Greptile (canonicalize provider ids) — provider ids are now lowercased in all three parsers (--map, --model-map, --require-key-prefix), so grok=XAI stores xai and matches the runtime BYOK lookup.

@greptile review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 02f0632. Configure here.

Comment thread packages/db/scripts/migrate-block-api-keys-to-byok.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

Latest re-review triage (HEAD is now 5cee10ca):

  • Bugbot — "Prefix map picks shortest match" (Low) — valid, fixed in 5cee10ca. resolveModelProvider now iterates prefixes longest-first, so the most specific mapping wins deterministically regardless of arg order.
  • Bugbot — "Active router type omitted" (High)⚠️ false positive. router_v2 was already added in 02f06325; it's on line 87 of LLM_FAMILY_BLOCK_TYPES (agent, router, router_v2, evaluator, translate, guardrails, pi). The comment re-posted the round-2 text against the commit that already fixed it.
  • Greptile — "Canonicalize Provider Ids" — already fixed; that comment reviewed the older commit 948721, but 02f06325 lowercases provider ids in all three parsers (lines 65, 101, 156).

@greptile review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants