Repository navigation
Commit fa18cab
authored
SK-3225: Split SDK into common / skyvault / flowvault; add FlowVault Node SDK (#348)
* SK-3225: Split SDK into common/skyvault/flowvault; add FlowVault Node SDK
Restructure the single `skyflow-node` package into an npm-workspaces monorepo
mirroring the Java SDK split (common / skyvault / flowvault).
- common (private, bundled into each published package): the contract-agnostic
core — errors, utils, jwt, logging, service-account auth, config/credentials,
generic validations, mutable SDK identity, Base* request/response/options
classes, the BaseSkyflow facade base, and the generated auth client. Builds
and tests in isolation.
- skyvault -> skyflow-node (2.x): the existing contract, public API unchanged
(74 exports, byte-for-byte). Its Skyflow/models extend the common Base*
classes; keeps connections, detect and its data-plane generated client.
- flowvault -> skyflow-flowvault-node (1.x): new unary v2 contract
(insert/get/update/delete/query/detokenize/getTokens/uploadFiles/deleteFiles)
wired to the generated v2 client; reconciled with the Java/Python flowvault
SDKs.
Each layer re-exports common and bundles it (tsup) into a self-contained
tarball (verified via npm pack); the mutable SDK identity makes each product
self-report its own name@version. Independent versioning: plain-semver tags
publish skyflow-node, flowvault/* tags publish skyflow-flowvault-node.
Tests: common 232, skyvault 878 (backward compatibility preserved), flowvault
132 (~99.85% coverage). Per-package api-extractor contract snapshots gate the
public API of both SDKs. PR CI builds and tests both SDKs with one command.
Docs: docs/sdk-split-plan.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: fix PR CI (eslint ignores + gitleaks allowlist for generated/dist)
- eslint.config.mjs: ignore all generated dirs (**/_generated_/**, legacy
leading-space ** / _generated_/**), all workspace test dirs (**/test/**),
and build output across workspaces — the old ignores only matched the
single-package paths.
- Replace three banned magic-string comparisons in hand-written code
(typeof === "symbol"/"function") with equivalent non-literal checks.
- Rule/gitleaks.toml: allowlist the maintainer-owned generated dirs (example
JWTs) and the bundled dist/ output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: fix skyvault jest — don't babel-transform compiled common/lib
CI resolves the skyflow-common symlink to its realpath (outside node_modules),
so babel-jest was transforming the compiled common/lib and injecting core-js
(an undeclared dep) -> suites failed to run. Add `common/lib` to
transformIgnorePatterns so the already-CJS common package is loaded as-is.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: add flowvault branch-coverage tests to clear codecov/project
Covers the defensive edge branches the codecov/project gate flagged
(net-new flowvault code dipped overall coverage 0.44% vs base):
- empty/absent `records`/`response` bodies map to [] across all ops
- per-record errors with no x-request-id header -> requestId null
- getRequestId with an absent raw response
- uploadFiles: non-Error PUT rejection, no-name/trailing-dot content-type,
response record with no matching prepared columns
- VaultClient: default log level, unrecognized cached auth type
flowvault branches 92.51% -> 96.89% (statements/lines 100%); 132 -> 153 tests.
No source/generated/config changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault insert/update — per-record table + separate tokens/data
Aligns the Node flowvault insert/update contract with the Java/Python SDKs:
- InsertRecord/UpdateRecord gain an optional per-record `table`; when any
record sets it, a single insert/update call targets multiple tables (the
generated InsertRecordData/UpdateRecordData already carry tableName).
- Insert/Update response records now expose `tokens` and `data` separately
instead of a single merged `fields`.
Updated unit tests (incl. multi-table coverage) and the api-report snapshot.
flowvault 156 tests pass (100% lines, branches 96.97%); common 232, skyvault
878 unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: add flowvault README + samples (parity with python flowvault)
Expand flowvault/README.md into a full SDK guide (overview, install, auth,
client init, all 9 operations, custom headers, error handling) and add
flowvault/samples/ — one runnable TypeScript example per operation under
vault-api/, plus service-account token-generation samples — mirroring the
skyflow-flowvault-python samples. All samples type-check in strict mode against
the built SDK types.
flowvault-only; skyvault and common untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault timeout/retry config + wire custom request headers
Config (parity with python flowvault, limited to what the Node generated client
can honor):
- VaultConfig gains optional timeout (seconds), maxRetries, and vaultUrl;
the flowvault VaultClient forwards timeout/maxRetries as the generated
request's timeoutInSeconds/maxRetries, and vaultUrl overrides the derived URL.
(connect/read/write timeouts and retry-delay knobs are fixed in generated code.)
Custom headers:
- options.setRequestHeaders(...) is now actually wired — merged into every
operation's request headers. Added the missing options params to delete,
query, getTokens, uploadFiles, deleteFiles so headers work on all 9 ops.
Samples/README updated; added tests (162 pass, 100% stmts/lines). flowvault +
the shared VaultConfig only; skyvault untouched; api-report unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault parity — per-record insert upsert + multi-table get fields
Closes two parity gaps vs the Java/Python flowvault SDKs (generated client
already supports both; hand-written layer only):
- InsertRecord gains optional per-record `upsert`; when any record sets it, each
record sends its own upsert (falling back to the request-level one) and the
request-level upsert is dropped.
- GetRecord (multi-table get) gains optional `columns` / `columnRedactions` /
`uniqueValues`, wired per record.
Added tests (164 pass, 100% lines); api-report regenerated. flowvault-only;
skyvault and common untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault update updateType + align README/samples with python
- UpdateRequest gains an optional request-level `updateType` ("UPDATE" |
"REPLACE"), wired to the generated UpdateRequest.updateType (no generated
change needed). Closes the last parity gap vs Java/Python flowvault.
- README + samples updated to match the python flowvault SDK: per-record
`upsert` on insert, multi-table get per-record columns/redactions/uniqueValues,
and update `updateType`.
166 tests pass (100% lines); api-report regenerated; samples type-check.
flowvault-only; skyvault/common untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: fix flowvault vault URL — use skyvault.* host, not vault.*
FlowVault (v3) is served from {clusterId}.skyvault.skyflowapis.{dev|com|
-preview.com|tech}, a different subdomain than the shared getVaultURL
(vault.*, used by skyvault/v2). The facade was reusing getVaultURL, so every
call 404'd. Added a flowvault-local getFlowVaultURL with the four confirmed
skyvault.* hosts and used it in buildVaultEntry.
flowvault-only; skyvault/common untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: fix flowvault insert/update table placement + per-record error extraction
Two bugs surfaced running the e2e against the live vault:
- Insert/update sent tableName at BOTH the request level and per-record when any
record carried its own table, which the API rejects (400 "table name ... at
one place"). Now the request-level tableName is omitted whenever per-record
tables are used (table lives in exactly one place), matching Java/Python.
- A per-record failure returned with a 4xx HTTP status (e.g. delete/get/update
of a non-existent id -> 404 whose body carries a records/response array) was
thrown as a whole-call error. Now those per-record records are surfaced as a
normal response (via unaryError), and genuine whole-call errors still throw.
Added error-extraction tests for insert/get/update/delete/detokenize/getTokens/
deleteFiles. 174 tests pass (99.4% lines). flowvault-only; skyvault/common untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault get — allow empty ids when uniqueValues are set
validateGetRequest rejected empty ids even for a get-by-unique-value, where the
lookup is by GetOptions.setUniqueValues (not ids). It now accepts a single-table
get with empty ids as long as uniqueValues are provided on the options
("ids OR uniqueValues", matching Java/Python flowvault).
175 tests pass. flowvault-only; skyvault/common untouched; api-report unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault custom headers — typed interceptor + CustomHeaderKey
Replace the arbitrary setRequestHeaders(Record<string,string>) on options
with the same typed mechanism the Python and Java FlowVault SDKs expose: a
RequestInterceptor set via options.setInterceptor(), which receives a
RequestContext and may attach only the supported headers in CustomHeaderKey
(x-skyflow-account-id, x-skyflow-account-name, x-request-id). The context
carries the operation name (INSERT, GET, ...) so headers can vary per call.
- New: CustomHeaderKey enum, RequestContext (+ RequestInterceptor type),
FlowVaultOptions base carrying the interceptor.
- All 9 *Options now extend FlowVaultOptions; controller.requestOptions runs
the interceptor per operation and maps CustomHeaderKey -> wire header.
- README, sample, and tests updated; contract snapshot regenerated.
common and skyvault are untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* SK-3225: flowvault docs — vaultUrl override uses the skyvault.* host
The vaultUrl config overrides the URL the SDK actually calls, and FlowVault is
served from the skyvault.* subdomain (getFlowVaultURL), not vault.*. Fix the
two override examples so a user who copies them points at the right host. The
clusterId derivation comments keep vault.skyflowapis.com to match the Python
SDK (that is where the clusterId is read from).1 parent 0c8a2d4 commit fa18cab
600 files changed
Lines changed: 18045 additions & 2013 deletions
File tree
- .github/workflows
- Rule
- common
- src
- _generated_/rest
- api
- client
- requests
- errors
- resources
- authentication
- client
- requests
- types
- core
- auth
- fetcher
- stream-wrappers
- runtime
- errors
- base
- config
- connection
- credentials
- vault
- error
- codes
- messages
- service-account
- client
- skyflow
- types
- utils
- jwt-utils
- logs
- validations
- test
- demo-credentials
- error
- service-account
- utils
- docs
- flowvault
- _generated_/rest/src
- api
- client
- requests
- errors
- resources
- files
- client
- requests
- query
- client
- requests
- records
- client
- requests
- types
- tokens
- client
- requests
- types
- core
- auth
- fetcher
- stream-wrappers
- runtime
- errors
- api-report
- samples
- service-account
- vault-api
- src
- client
- controller/vault
- enums
- model
- options
- delete-files
- delete
- detokenize
- get-tokens
- get
- insert
- query
- update
- upload-files
- request-context
- request
- delete-files
- delete
- detokenize
- get-tokens
- get
- insert
- query
- update
- upload-files
- response
- delete-files
- delete
- detokenize
- get-tokens
- get
- insert
- query
- update
- upload-files
- skyflow
- types
- validations
- test
- client
- controller
- model
- skyflow
- validations
- scripts
- skyvault
- api-report
- src
- _generated_/rest
- api
- errors
- resources
- audit
- client
- requests
- types
- authentication
- client
- requests
- binLookup
- client
- requests
- files
- client
- requests
- types
- guardrails
- client
- requests
- query
- client
- requests
- records
- client
- requests
- types
- strings
- client
- requests
- types
- tokens
- client
- requests
- types
- core
- auth
- fetcher
- stream-wrappers
- form-data-utils
- runtime
- errors
- management
- config/management
- controller
- policy
- role
- vault
- model
- request
- create-vault
- delete-policy
- response
- create-vault
- delete-policy
- skyflow
- types
- utils/validations
- vault
- client
- controller
- audit
- binlookup
- connections
- detect
- vault
- model
- options
- deidentify-file
- bleep-audio
- deidentify-text
- token-format
- transformations
- detokenize
- fileUpload
- get
- insert
- reidentify-text
- update
- request
- deidentify-file
- deidentify-text
- delete
- detokenize
- file-upload
- get-column
- get-detect-run
- get
- inkove
- insert
- query
- reidentify-text
- tokenize
- update
- response
- deidentify-file
- deidentify-text
- delete
- detokenize
- file-upload
- get
- insert
- invoke
- query
- reidentify-text
- tokenize
- update
- skyflow
- types
- test
- demo-credentials
- mockData
- utils
- vault
- client
- controller
- model
- skyflow
- src
- error/messages
- vault
- config/vault
- skyflow
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
16 | 18 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
33 | 38 | | |
34 | 39 | | |
35 | 40 | | |
36 | | - | |
| 41 | + | |
37 | 42 | | |
38 | 43 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
3 | 9 | | |
4 | 10 | | |
5 | 11 | | |
6 | | - | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
7 | 22 | | |
8 | 23 | | |
9 | 24 | | |
10 | 25 | | |
11 | 26 | | |
12 | 27 | | |
13 | | - | |
14 | 28 | | |
15 | 29 | | |
16 | 30 | | |
| |||
24 | 38 | | |
25 | 39 | | |
26 | 40 | | |
27 | | - | |
28 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
29 | 50 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
35 | 66 | | |
36 | 67 | | |
37 | 68 | | |
38 | | - | |
| 69 | + | |
39 | 70 | | |
40 | 71 | | |
41 | | - | |
42 | 72 | | |
43 | | - | |
44 | 73 | | |
45 | 74 | | |
46 | 75 | | |
47 | 76 | | |
48 | | - | |
49 | | - | |
50 | 77 | | |
51 | 78 | | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | 79 | | |
62 | 80 | | |
63 | 81 | | |
64 | 82 | | |
65 | | - | |
| 83 | + | |
66 | 84 | | |
67 | 85 | | |
68 | | - | |
69 | | - | |
70 | | - | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
71 | 89 | | |
72 | | - | |
73 | | - | |
| 90 | + | |
| 91 | + | |
74 | 92 | | |
75 | 93 | | |
76 | 94 | | |
77 | 95 | | |
| 96 | + | |
78 | 97 | | |
79 | | - | |
| 98 | + | |
80 | 99 | | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
89 | 103 | | |
90 | 104 | | |
91 | 105 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
28 | 30 | | |
29 | 31 | | |
30 | 32 | | |
| |||
34 | 36 | | |
35 | 37 | | |
36 | 38 | | |
37 | | - | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
38 | 44 | | |
39 | | - | |
| 45 | + | |
40 | 46 | | |
41 | 47 | | |
42 | 48 | | |
43 | | - | |
44 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
45 | 53 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
3 | 6 | | |
4 | 7 | | |
5 | 8 | | |
| |||
10 | 13 | | |
11 | 14 | | |
12 | 15 | | |
| 16 | + | |
| 17 | + | |
13 | 18 | | |
14 | 19 | | |
15 | 20 | | |
16 | 21 | | |
17 | 22 | | |
18 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
19 | 26 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
16 | 18 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
3 | 5 | | |
4 | | - | |
| 6 | + | |
| 7 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
58 | 65 | | |
59 | 66 | | |
60 | 67 | | |
| |||
0 commit comments