Repository navigation
Conversation
…s that have an OpenAI-compatible API....)
…er Health image - a new filter by ended/not ended events
… image - a new filter by ended/not ended events
RDoc-3257 & RDoc-3259 Cloud -> Portal -> Products Tab & Cloud -> Scaling Tab - update images related to changing instance type / disk to support pre & post product costs calculation
RDoc-3261 & RDoc-3262 Add AI Embeddings cloud feature
…ForIssues RDoc-3265 Update "Enable-logs-for-ongoing-issues" (v7.0)
…esults RDoc-3250 Remove LongQueryResults from the docs + other updates
…t code + Update images + Small fixes to flow charts + Fix to configuration + Updated the expiration policy
…tion RDoc-3254 Embeddings Generation via Tasks
RDoc-3406 Fix the FindProjectedPropertyNameForIndex convention
RDoc-3394 & RDoc-3393 Azure Data disks
RDoc-3402 Cloud -> Overview - add a note a single email can be an owner of multiple accounts
RDoc-3381 Cannot change UID:GID of container user
Created MIB-generation monitoring page
…-latest RDoc-3405 Indexes > Storing data in index [C#]
Added AI Agents documentation
…ument RDoc-3403 A clone made from an archived document will not be archived
…dexingForCsharp RDoc-3256 Document extensions > Attachments > Indexing [Fix article - C#]
New documentation section for CDC Sink, a new ongoing task that streams changes from SQL databases (PostgreSQL) into RavenDB documents. Core pages: - overview, how-it-works, schema-design - embedded-tables, linked-tables, column-mapping - patching, delete-strategies, property-retention, attachment-handling - configuration-reference, api-reference, server-configuration - monitoring, failover-and-consistency, troubleshooting PostgreSQL-specific pages: - prerequisites-checklist, wal-configuration, permissions-and-roles - initial-setup, replica-identity, replica-identity-manual-setup - cleanup-and-maintenance, monitoring-postgres, studio-ui (with screenshot TODOs) PostgreSQL examples: - simple-migration, denormalization, event-sourcing, complex-nesting SQL Server stub (future expansion placeholder)
commented
Apr 2, 2026
|
@ayende can we move this to current (7.2) not 7.1? Should go to this directory: https://github.com/ravendb/docs/tree/main/docs |
|
|
||
| CREATE TABLE variant_attributes ( | ||
| attr_id SERIAL PRIMARY KEY, | ||
| product_id INT NOT NULL, -- denormalized root PK (required for deep nesting) |
There was a problem hiding this comment.
go ahead and create a unique index on the required columns, and then user REPLICA INDEX below
| { | ||
| Name = "Products", | ||
| SourceTableName = "products", | ||
| PrimaryKeyColumns = new List<string> { "product_id" }, |
There was a problem hiding this comment.
use [] collection expression whenever possible to simplify the code.
| ", | ||
| OnDelete = new CdcSinkOnDeleteConfig | ||
| { | ||
| Patch = @" |
There was a problem hiding this comment.
Use """ """ for multi line strings (across the board)
| or the user does not have the required permissions. | ||
| See the PostgreSQL [Permissions and Roles](../../../server/ongoing-tasks/cdc-sink/postgres/permissions-and-roles) page. | ||
|
|
||
| * **WAL level not set to `logical`** — CDC Sink requires logical replication enabled |
There was a problem hiding this comment.
this is for postgres, make a note here
| * **Exceeded fallback timeout** — the source was unreachable for longer than | ||
| `CdcSink.MaxFallbackTimeInSec`. The task moves to error state after this timeout. | ||
| Restore connectivity and re-enable the task. | ||
|
|
There was a problem hiding this comment.
an error in the script across all docs
| 2. **Null reference** — `$row` properties and `$old` may be `null` for certain event | ||
| types. Use optional chaining: `$old?.Amount || 0`. | ||
|
|
There was a problem hiding this comment.
we already do null coalescing - so that won't generate an erro
| 2. **Null reference** — `$row` properties and `$old` may be `null` for certain event | ||
| types. Use optional chaining: `$old?.Amount || 0`. | ||
|
|
||
| 3. **`get()` returns null** — a document loaded with `get()` may not exist yet if CDC |
| 3. **`get()` returns null** — a document loaded with `get()` may not exist yet if CDC | ||
| Sink processes tables out of dependency order. Guard with a null check: | ||
| ``` | ||
| const related = get("Collection/123"); |
There was a problem hiding this comment.
| const related = get("Collection/123"); | |
| const related = load("Collection/123"); |
… features New features (from product PR #22570): - CdcSinkConfiguration.Postgres (CdcSinkPostgresSettings) with SlotName/PublicationName for manual slot and publication naming - CdcSinkConfiguration.SkipInitialLoad flag - Document both auto-generated and custom naming options in initial-setup Review comment fixes (PR #2382 review by ayende): - how-it-works: Add callouts explaining CDC initializes before initial load starts; stub document behavior during mid-load inserts; CDC start position note - column-mapping: Note that PK column mapping is optional; add type conversion note; clarify property name vs column name context ($row/$old vs this) - delete-strategies: Document Value-type embedded DELETE behavior; add IgnoreDeletes persistence note; rewrite root doc audit trail to use put(); add IgnoreDeletes=false variant for embedded; mark REPLICA IDENTITY section as PostgreSQL-specific - embedded-tables: Fix navigation example to use .find() not bracket notation; "multiple levels" wording - attachment-handling: Fix content type claim (application/octet-stream); use [] syntax - configuration-reference: Add CdcSinkPostgresSettings table; add SkipInitialLoad; add REPLICA IDENTITY note to embedded tables section - api-reference: Expand UpdateCdcSink warning to explain slot/publication implications of adding/removing tables; add WAL accumulation warning for paused tasks - initial-setup: Add explicit SlotName/PublicationName via CdcSinkPostgresSettings; restructure to show both manual and auto-generated naming approaches - monitoring-postgres: Remove get() mention, refer to Studio stats instead - permissions-and-roles: Add admin security note about minimal privilege setup - cleanup-and-maintenance: Add note explaining why RavenDB cannot auto-delete slots - studio-ui: Update editing warning to cover both auto-named and custom-named slots - example-complex-nesting: Add FK on product_id; use REPLICA IDENTITY USING INDEX; use [] collection expressions throughout - example-event-sourcing: Use C# raw string literals for multi-line patch scripts; use [] collection expressions
Replace the 7.1 .markdown files with MDX files in docs/server/ongoing-tasks/cdc-sink/ targeting the current Docusaurus-based 7.2 docs site. - Remove Documentation/7.1/.../cdc-sink/ (old custom markdown format) - Add docs/server/ongoing-tasks/cdc-sink/ (Docusaurus MDX format)
| SourceTableName = "accounts", | ||
| PrimaryKeyColumns = new List<string> { "account_id" }, | ||
| PrimaryKeyColumns = ["account_id"], | ||
| ColumnsMapping = new Dictionary<string, string> |
There was a problem hiding this comment.
can you use collection expressions here as well to make the code nicer for dictionaries as well?
| **Constraints:** Names must be valid PostgreSQL identifiers — alphanumeric characters | ||
| and underscores only, maximum 63 characters. |
There was a problem hiding this comment.
drop this, enforced by postgres anyway
| slot name. This prevents naming conflicts between multiple CDC Sink tasks on the | ||
| same PostgreSQL instance, but means that renaming a task or adding/removing tables | ||
| produces a new slot — the old one becomes orphaned. |
There was a problem hiding this comment.
This does not happen. If you mutate the CDC task, it will use the same slot name as it did when you created that.
You need to be aware of that if you create a new one later, with the same name / tables / etc - that would cause a problem (both cdc sinks using the same slot)
| `CdcSinkPostgresSettings`), adding or removing tables changes the hash and therefore | ||
| the names. A new slot and publication are created under the new names, and the old |
There was a problem hiding this comment.
no, see the earlier comment on that.
once it was created, it is fixed
There was a problem hiding this comment.
ravendb will add the right configuraiton for a slot if you modify it, but the old name will remain.
it is effectively opaque, so not a big deal.
|
|
||
| If you specified **custom names** via `CdcSinkPostgresSettings.SlotName` and | ||
| `PublicationName`, the slot name stays the same but the publication may need to be | ||
| updated by a database administrator to include the new tables. |
There was a problem hiding this comment.
ravendb will error if this isn't the case
| new tables. If the slot and publication were auto-named (hash-based), this causes | ||
| a new slot/publication to be created under a new hash, and the old ones become |
There was a problem hiding this comment.
not correct, the old name is still used
| a short time. | ||
| See [Monitoring PostgreSQL](../../../server/ongoing-tasks/cdc-sink/postgres/monitoring-postgres). | ||
| {WARNING/} | ||
|
|
There was a problem hiding this comment.
also add warning for SQL Server - if the task is paused for too long, SQL Server will trim the CDC tables, leading to missing updates when you start (you'll have to reset the CDC to start from scratch)
| [Child Before Parent](../../../server/ongoing-tasks/cdc-sink/how-it-works#child-before-parent) below). | ||
|
|
||
| {NOTE: } | ||
| **CDC is initialized before the initial load begins.** |
There was a problem hiding this comment.
This is wrong. We first create the CDC setup, but do NOT listen to it.
We do the full initial loading, then start the CDC streaming process, which will give us events from the before we started the initial load.
Need to call out this sequence of ops:
loaded all orders (took 2 hours)
started to load order lines (1 hours in)
user add a new order + order lines
we finish the order lines (including the new one), but we missed their order (since we already read all the items from there)
that will generate a stub document for the order
After we start the CDC streaming, we'll get the event about the new order (as well as the new order line), and match up everything properly.
Also - merge these two notes into a single one.
| {NOTE/} | ||
|
|
||
| {NOTE: } | ||
| **New rows added mid-load become stub documents.** |
There was a problem hiding this comment.
CDC streaming / event handling is not run concurrently with initial load.
10 participants
CDC Sink Documentation
Adds comprehensive documentation for the CDC (Change Data Capture) Sink ongoing task feature.
Changes
35 new documentation files with ~4600 lines of content.