Skip to content

Commit db30ec9

Browse files
avalletedivyasharma95claude
authored
feat(cli): add inspect db collation-drift with report output (#7460)
## TL;DR Adds `supabase inspect db collation-drift`, a read-only check for btree indexes whose collation version no longer matches the version the OS now provides. It lists the affected indexes and prints the remediation workflow (amcheck → `REINDEX INDEX CONCURRENTLY` → `REFRESH VERSION`) with the real object names. Supersedes #6666 by @divyasharma95, whose commits are kept as-is; this branch adds the port to the current inspect runner and the review fixes. ## Why After a glibc or ICU upgrade, btree indexes on text columns can be mis-ordered with no error: queries may miss rows, sort incorrectly, or let duplicates past a unique constraint. Postgres only warns about a version mismatch; it does not say which indexes are affected or in which order to fix them, and refreshing the recorded version before rebuilding hides the problem. ## What changed - New `inspect db collation-drift` command, sharing target selection, permissions and connection resolution with the other `inspect db` commands. - Detects drift for the database default collation (PG15+) and for named ICU and libc collations; on PG14 and earlier only named collations are checked. - Lists each affected index once, constraint-backing indexes first, and skips partitioned parent indexes in favour of their partitions. - Text output is a structured report: header, warning, table, and three ordered remediation steps. `--output-format json` emits `{ rows, report }`, keeping the `rows` shape of the other inspect commands. - Introduces a small `Report` document model (`apps/cli/src/output/report.types.ts`) and its text renderer, used only by this command for now. Known limitation: expression indexes (for example `lower(name)`) are not detected; the remediation text tells users to rebuild them before refreshing. ## Example output ```text Database : postgres Affected indexes : 1 Keys / unique : 1 (a wrong sort order here can admit duplicate rows) Default collation : 0.1 (recorded) → 2.41 (current) ✗ | public.k_pkey | public.k | e | default | 0.1 | 2.41 | PRIMARY KEY | 8192 bytes 1. Diagnose the listed indexes (optional) 2. Rebuild every listed index 3. Record the new collation version — only after every rebuild has finished ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Divya Sharma <divya.sharma@supabase.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent bb34a18 commit db30ec9

15 files changed

Lines changed: 1023 additions & 31 deletions

‎apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md‎

Lines changed: 35 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,11 @@
22

33
Target: selectable, no consent — see [Command target](../../../../docs/command-target.md)
44

5-
Single shared side-effect document for all 14 active `inspect db` subcommands and
5+
Single shared side-effect document for all 16 active `inspect db` subcommands and
66
their 12 deprecated aliases. Every subcommand has the same surface — it resolves its target, runs one read-only
77
`SELECT`, and renders the result as an ASCII table. They differ only in the SQL run and
8-
the columns rendered (see the per-subcommand `<name>.query.ts`).
8+
the columns rendered (see the per-subcommand `<name>.query.ts`). `collation-drift` is the
9+
exception: it renders a report (see Output).
910

1011
## Files Read
1112

@@ -60,23 +61,28 @@ Each subcommand runs one read-only `SELECT` (the embedded `<name>.sql`). The
6061
7 schema-filtered queries take `$1` = the LIKE-escaped internal-schema list;
6162
`db-stats` additionally takes `$2` = the database name.
6263

63-
| Subcommand | SQL file | InternalSchemas param? |
64-
| -------------------- | ------------------------ | --------------------------- |
65-
| db-stats | db_stats.sql | yes (`$1`) + db name (`$2`) |
66-
| index-stats | index_stats.sql | yes (`$1`) |
67-
| bloat | bloat.sql | yes (`$1`) |
68-
| vacuum-stats | vacuum_stats.sql | yes (`$1`) |
69-
| table-stats | table_stats.sql | yes (`$1`) |
70-
| toast-sizes | toast_sizes.sql | yes (`$1`) |
71-
| xid-age | xid_age.sql | yes (`$1`) |
72-
| replication-slots | replication_slots.sql | no |
73-
| locks | locks.sql | no |
74-
| blocking | blocking.sql | no |
75-
| outliers | outliers.sql | no |
76-
| calls | calls.sql | no |
77-
| long-running-queries | long_running_queries.sql | no |
78-
| role-stats | role_stats.sql | no |
79-
| traffic-profile | traffic_profile.sql | no |
64+
| Subcommand | SQL file | InternalSchemas param? |
65+
| -------------------- | ------------------------------------ | --------------------------- |
66+
| db-stats | db_stats.sql | yes (`$1`) + db name (`$2`) |
67+
| index-stats | index_stats.sql | yes (`$1`) |
68+
| bloat | bloat.sql | yes (`$1`) |
69+
| vacuum-stats | vacuum_stats.sql | yes (`$1`) |
70+
| table-stats | table_stats.sql | yes (`$1`) |
71+
| toast-sizes | toast_sizes.sql | yes (`$1`) |
72+
| xid-age | xid_age.sql | yes (`$1`) |
73+
| replication-slots | replication_slots.sql | no |
74+
| locks | locks.sql | no |
75+
| blocking | blocking.sql | no |
76+
| outliers | outliers.sql | no |
77+
| calls | calls.sql | no |
78+
| long-running-queries | long_running_queries.sql | no |
79+
| role-stats | role_stats.sql | no |
80+
| traffic-profile | traffic_profile.sql | no |
81+
| collation-drift | inline in `collation-drift.query.ts` | no (no params) |
82+
83+
`collation-drift` first runs `SELECT current_setting('server_version_num')::int` on the same
84+
session, then one inline parameterless detection `SELECT` built for that version (PG15+ also
85+
reads `pg_database.datcollversion` and `pg_database_collation_actual_version()`). Both are read-only.
8086

8187
Deprecated aliases run an active subcommand's query: `cache-hit`→db-stats;
8288
`index-usage`/`index-sizes`/`unused-indexes`/`seq-scans`/`table-record-counts`→index-stats;
@@ -97,6 +103,9 @@ flags, run no query and open no connection, and exit 1 with
97103
| `0` | success |
98104
| `1` | resolution, connection, or query failure |
99105

106+
`collation-drift` also exits `1` before connecting when `-o/--output` is passed; it supports
107+
`--output-format` only.
108+
100109
## Telemetry Events Fired
101110

102111
| Event | When | Notable properties |
@@ -116,14 +125,20 @@ whitespace runs collapsed to single spaces (long-running-queries' query is NOT
116125
collapsed). One `Target: …` banner naming the resolved target is written to
117126
**stderr** before the query runs.
118127

128+
`collation-drift` instead prints a report: a key/value header, a callout, a table of
129+
affected indexes, and numbered remediation steps (verify with amcheck, `REINDEX INDEX
130+
CONCURRENTLY`, then `ALTER ... REFRESH ... VERSION`). With no drift it prints only the
131+
healthy callout.
132+
119133
### `--output-format json`
120134

121135
A single object: `{ "rows": [ <raw driver rows, snake_case keys> ], "target": { … } }`,
122136
where `target` describes the resolved target. The stderr banner is still printed.
137+
`collation-drift` emits `{ "rows": [...], "report": { … } }`.
123138

124139
### `--output-format stream-json`
125140

126-
A `result` event carrying the same `{ rows }` payload.
141+
A `result` event carrying the same `{ rows }` payload (`{ rows, report }` for `collation-drift`).
127142

128143
### Deprecated aliases
129144

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import { Command } from "effect/cli";
2+
import { withPermissions } from "../../../../command-internal/command-permissions/command-permissions.annotation.ts";
3+
import { inspectDbCollationDrift } from "./collation-drift.handler.ts";
4+
import {
5+
INSPECT_DB_FLAGS,
6+
INSPECT_DB_PERMISSIONS,
7+
inspectDbCommandHandler,
8+
withInspectDbTargetPolicy,
9+
} from "../inspect-db-command.ts";
10+
import { inspectDbRuntimeLayer } from "../db.layers.ts";
11+
import { cliConfigValuesLayer } from "../../../../config/cli-config-values.layer.ts";
12+
import { withCliConfigFlags } from "../../../../config/cli-config-flags.ts";
13+
14+
export const inspectDbCollationDriftCommand = Command.make(
15+
"collation-drift",
16+
INSPECT_DB_FLAGS,
17+
).pipe(
18+
Command.withDescription(
19+
`Show indexes affected by collation version drift, with the fix workflow.
20+
21+
Postgres stores btree indexes on text columns in sorted order, using rules from
22+
the system collation library (glibc or ICU). When that library is upgraded the
23+
rules can change, and indexes built under the old rules are no longer correctly
24+
ordered. Postgres reports no error: queries may quietly return missing rows,
25+
sort incorrectly, or let duplicates past a unique constraint.
26+
27+
The output lists the affected indexes and the exact statements to verify
28+
(amcheck), rebuild (REINDEX CONCURRENTLY), and record the new version — in the
29+
order they must be run. This command itself is read-only.`,
30+
),
31+
Command.withShortDescription("Show indexes affected by collation version drift"),
32+
withPermissions(INSPECT_DB_PERMISSIONS),
33+
Command.withHandler(inspectDbCommandHandler(inspectDbCollationDrift)),
34+
Command.provide(inspectDbRuntimeLayer("collation-drift")),
35+
Command.provide(cliConfigValuesLayer),
36+
withCliConfigFlags(INSPECT_DB_FLAGS),
37+
withInspectDbTargetPolicy,
38+
);
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
import { makeInspectDbReportHandler } from "../inspect-report.ts";
2+
import { collationDriftSpec } from "./collation-drift.query.ts";
3+
4+
export const inspectDbCollationDrift = makeInspectDbReportHandler(
5+
collationDriftSpec,
6+
"inspect.db.collation-drift",
7+
);
Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
import { describe, expect, it } from "@effect/vitest";
2+
import { Cause, ConfigProvider, Effect, Exit, Layer, Option } from "effect";
3+
import { vi } from "vitest";
4+
5+
vi.mock("node:net", (importOriginal) =>
6+
Promise.all([
7+
import("../../../../../tests/helpers/refuse-invalid-db-hosts.ts"),
8+
importOriginal<typeof import("node:net")>(),
9+
]).then(([helper, net]) => helper.refuseInvalidDbHosts(net)),
10+
);
11+
12+
import { mockOutput } from "../../../../../tests/helpers/mocks.ts";
13+
import {
14+
buildTestRuntime,
15+
jsonResponse,
16+
mockCommandPlatformApi,
17+
mockCommandSettings,
18+
mockTelemetryStateTracked,
19+
} from "../../../../../tests/helpers/command-mocks.ts";
20+
import { configValuesLayer } from "../../../../../tests/helpers/config-values-layer.ts";
21+
import { DbConfigResolver } from "../../../../command-internal/db-config.service.ts";
22+
import type {
23+
DbConnectionInput,
24+
ResolvedDbConnection,
25+
} from "../../../../command-internal/db-config.types.ts";
26+
import {
27+
DbConnection,
28+
type PgConnInput,
29+
} from "../../../../command-internal/db-connection.service.ts";
30+
import { DnsResolverFlag, OutputFlag, YesFlag } from "../../../../command-internal/global-flags.ts";
31+
import { TargetPolicy } from "../../../../command-internal/target/target-policy.ts";
32+
import type { InspectConnectionFlags } from "../inspect-query.ts";
33+
import { inspectDbCollationDrift } from "./collation-drift.handler.ts";
34+
35+
const LOCAL_CONN: PgConnInput = {
36+
host: "127.0.0.1",
37+
port: 54322,
38+
user: "postgres",
39+
password: "postgres",
40+
database: "postgres",
41+
};
42+
43+
const DRIFT_ROW = {
44+
database: "postgres",
45+
name: "public.users_email_key",
46+
table: "public.users",
47+
columns: "email",
48+
collation: "default",
49+
stored_version: "2.36",
50+
current_version: "2.39",
51+
key_type: "UNIQUE",
52+
size: "16 kB",
53+
can_refresh: true,
54+
};
55+
56+
const flags: InspectConnectionFlags = {
57+
dbUrl: Option.none(),
58+
remote: Option.none(),
59+
linked: false,
60+
local: false,
61+
projectRef: Option.none(),
62+
};
63+
64+
function setup(opts: {
65+
serverVersionNum: number;
66+
rows: ReadonlyArray<Record<string, unknown>>;
67+
format?: "text" | "json";
68+
connectFails?: boolean;
69+
}) {
70+
const out = mockOutput({ format: opts.format ?? "text" });
71+
const queries: string[] = [];
72+
const connection = Layer.succeed(DbConnection, {
73+
connect: () =>
74+
opts.connectFails === true
75+
? Effect.die("connect must not be reached")
76+
: Effect.succeed({
77+
exec: () => Effect.void,
78+
execBatch: () => Effect.void,
79+
extensionExists: () => Effect.succeed(false),
80+
queryRaw: () => Effect.succeed({ fields: [], rows: [], commandTag: "" }),
81+
copyToCsv: () => Effect.succeed(new Uint8Array()),
82+
query: (sql: string) => {
83+
queries.push(sql);
84+
return Effect.succeed(
85+
sql.includes("server_version_num")
86+
? [{ server_version_num: opts.serverVersionNum }]
87+
: opts.rows,
88+
);
89+
},
90+
}),
91+
});
92+
const resolver = Layer.succeed(DbConfigResolver, {
93+
resolveConnection: (input: DbConnectionInput) =>
94+
Effect.succeed({
95+
conn: LOCAL_CONN,
96+
isLocal: true,
97+
target: input.target,
98+
} satisfies ResolvedDbConnection),
99+
resolvePoolerFallback: () => Effect.succeed(Option.none()),
100+
});
101+
const runtime = buildTestRuntime({
102+
out,
103+
api: mockCommandPlatformApi({
104+
handler: (request) => Effect.succeed(jsonResponse(request, 200, [])),
105+
}),
106+
cliSettings: mockCommandSettings({
107+
workdir: "/tmp/collation-drift-test",
108+
projectId: Option.none(),
109+
}),
110+
telemetry: mockTelemetryStateTracked().layer,
111+
});
112+
const layer = Layer.mergeAll(
113+
runtime,
114+
resolver,
115+
connection,
116+
Layer.succeed(DnsResolverFlag, "native"),
117+
Layer.succeed(YesFlag, false),
118+
Layer.succeed(TargetPolicy, { group: "selectable", consent: "none" }),
119+
ConfigProvider.layer(ConfigProvider.fromEnvRecord({}, { preserveEmptyStrings: true })),
120+
configValuesLayer(),
121+
);
122+
return { layer, out, queries };
123+
}
124+
125+
describe("inspect db collation-drift", () => {
126+
it.effect("PG15+ reads the default collation version and prints remediation steps", () => {
127+
const { layer, out, queries } = setup({
128+
serverVersionNum: 150000,
129+
rows: [DRIFT_ROW],
130+
});
131+
return Effect.gen(function* () {
132+
yield* inspectDbCollationDrift(flags);
133+
expect(queries[1]).toContain("pg_database_collation_actual_version");
134+
expect(out.stdoutText).toContain("public.users_email_key");
135+
expect(out.stdoutText).toContain("bt_index_check");
136+
expect(out.stdoutText).toContain("REINDEX INDEX CONCURRENTLY public.users_email_key;");
137+
expect(out.stdoutText).toContain('ALTER DATABASE "postgres" REFRESH COLLATION VERSION;');
138+
}).pipe(Effect.provide(layer));
139+
});
140+
141+
it.effect("pre-15 skips the default collation check and prints the healthy callout", () => {
142+
const { layer, out, queries } = setup({
143+
serverVersionNum: 140000,
144+
rows: [],
145+
});
146+
return Effect.gen(function* () {
147+
yield* inspectDbCollationDrift(flags);
148+
expect(queries[1]).not.toContain("datcollversion");
149+
expect(out.stdoutText).toContain("No recorded collation version mismatch");
150+
expect(out.stdoutText.replace(/\s+/g, " ")).toContain("not checked before PostgreSQL 15");
151+
}).pipe(Effect.provide(layer));
152+
});
153+
154+
it.effect("json mode carries raw rows and the report severity", () => {
155+
const { layer, out } = setup({
156+
serverVersionNum: 160000,
157+
rows: [DRIFT_ROW],
158+
format: "json",
159+
});
160+
return Effect.gen(function* () {
161+
yield* inspectDbCollationDrift(flags);
162+
expect(out.messages).toContainEqual(
163+
expect.objectContaining({
164+
type: "success",
165+
message: "inspect db collation-drift",
166+
data: {
167+
rows: [DRIFT_ROW],
168+
report: expect.objectContaining({
169+
command: "collation-drift",
170+
severity: "critical",
171+
}),
172+
},
173+
}),
174+
);
175+
expect(out.stdoutText).toBe("");
176+
}).pipe(Effect.provide(layer));
177+
});
178+
179+
it.effect("rejects -o/--output before connecting", () => {
180+
const { layer } = setup({ serverVersionNum: 170000, rows: [], connectFails: true });
181+
return Effect.gen(function* () {
182+
const exit = yield* Effect.exit(inspectDbCollationDrift(flags));
183+
expect(Exit.isFailure(exit)).toBe(true);
184+
if (Exit.isFailure(exit)) {
185+
expect(Cause.pretty(exit.cause)).toContain(
186+
"the -o/--output flag is not supported by inspect db collation-drift; use --output-format json|stream-json instead.",
187+
);
188+
expect(Cause.pretty(exit.cause)).not.toContain("connect must not be reached");
189+
}
190+
}).pipe(Effect.provide(Layer.mergeAll(layer, Layer.succeed(OutputFlag, Option.some("json")))));
191+
});
192+
});

0 commit comments

Comments
 (0)