Skip to content

DOC-6854 Add redis-rb (Ruby) vector search docs: vecsearch page + ruby_home_query_vec [PARKED]#3652

Open
andy-stark-redis wants to merge 1 commit into
mainfrom
DOC-6854-redis-rb-vecsearch-docs
Open

DOC-6854 Add redis-rb (Ruby) vector search docs: vecsearch page + ruby_home_query_vec [PARKED]#3652
andy-stark-redis wants to merge 1 commit into
mainfrom
DOC-6854-redis-rb-vecsearch-docs

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Preemptive docs for redis-rb's Query Engine vector search: a new "Index and query vectors" page for Ruby (content/develop/clients/ruby/vecsearch.md) and a run-verified ruby_home_query_vec example set. Follow-up to the parked Query Engine docs (DOC-6843 / #3641), which deliberately deferred vector search pending an embedding-library decision — now resolved by reusing the semantic-cache Ruby choice (informers running the ONNX all-MiniLM-L6-v2 encoder locally).

⛔ Do not merge yet

The Query Engine landed upstream in redis-rb#1356 (merged) but is not yet in a released gem — the latest is v5.4.1 (July 2025), which predates it. Merging now would document APIs (Redis::Commands::Search::*, create_index, vector_field, KNN search) that gem install redis cannot provide. The page carries a preview bannerText and this PR is labelled parked + do not merge yet.

Park manifest

Ticket: DOC-6854
Parked at: 2026-07-17
Trigger to pick up: redis-rb#1356 shipped in a released redis gem (a version later than v5.4.1 whose lib/redis/commands/modules/search is present)
Labels: parked, do not merge yet

Pinned sources (state observed at park time)

Source State at park Re-fetch
redis-rb#1356 — Query Engine support state: closed, merged: true, head SHA 659d885199d09b997172d753a2f67c39ee23769e, base master, milestone: none, updated 2026-07-14 (squash-merged to master as 71706ec) gh api repos/redis/redis-rb/pulls/1356 --jq '{state, merged, head_sha: .head.sha, base: .base.ref, milestone: .milestone.title, updated_at}'
redis-rb released gem latest released v5.4.1 (July 2025) — predates the Query Engine; no release yet contains #1356 gh api repos/redis/redis-rb/tags --jq '.[0:5] | map(.name)' (and check rubygems.org/gems/redis/versions)
informers gem (embedding lib) verified working at 1.3.0 (+ onnxruntime 0.11.3, tokenizers 0.7.0) on Ruby 4.0.6; ONNX all-MiniLM-L6-v2, 384-dim — already released, NOT a merge blocker (listed for provenance) gem list informers onnxruntime tokenizers

Observed shape the page assumes — confidence: MEDIUM

The example was run-verified on 2026-07-17 against the local redis-rb fork (master, PR #1356 merged, head 71706ec) on Redis 8 with informers 1.3.0: ruby -I <fork>/lib home_query_vec.rb exited 0, all REMOVE-block asserts passed, and the hash and JSON paths returned identical results. Confidence is MEDIUM rather than HIGH only because it was verified against fork master, not a released gem, so the surface could still shift before release. Key assumptions:

  • Schema DSL Search::Schema.build { text_field; tag_field; vector_field 'embedding', 'HNSW', type: 'FLOAT32', dim: 384, distance_metric: 'L2' }; vector_field extracts :as as a field-level option (verified in lib/redis/commands/modules/search/schema.rb), so the JSON path alias vector_field '$.embedding', 'HNSW', as: 'embedding', … works.
  • KNN query via index.search('*=>[KNN 3 @embedding $vec AS vector_distance]', sort_by:, return_fields:, params: { 'vec' => bytes }); sort_by defaults to ascending (nearest-first). Document#[] exposes doc['vector_distance'].
  • Search::Index strips the index key prefix from returned doc IDsdoc.id is 0, not doc:0 (unlike redis-py). The page/example call this out.
  • Vector encoding: hash fields store raw little-endian float32 (Array#pack('e*')); JSON stores the embedding as a plain Array<Float>, but the query parameter is packed bytes either way.
  • Embeddings from informers (model.(text, pooling: 'mean', normalize: true), flat 384-element Array) reproduce the canonical L2 distances to ~7 significant figures — 0.114169895649 / 0.610845208168 / 1.48624789715. Normalisation (unit vectors) is load-bearing for those distances (see commit Constraint: trailers).

Re-check checklist

  • (highest risk) Re-run local_examples/client-specific/ruby/home_query_vec.rb against the released gem on Redis 8.x; confirm all steps + asserts still pass and the API shape above is unchanged. (Note: the Redis TCP connection needs the sandbox OFF — Operation not permitted on localhost otherwise.)
  • Confirm the run-verified distances in both expected-output blocks of vecsearch.md still hold on the released gem; update if they shift.
  • Add a "requires redis-rb vX.Y+" note to vecsearch.md once the release version is known.
  • Remove the preview bannerText from vecsearch.md once released.
  • Confirm examples.json regenerates the Ruby tab from ruby_home_query_vec in a full CI build (not run locally).
  • When reconciling, preserve the commit Constraint: trailers — keep normalize on for every encode, and keep the embedding model/library aligned with the semantic-cache Ruby example — or stored vectors stop being comparable across examples.
  • No command-api-mapping work in this PR (the FT.* family is covered by DOC-6843 / DOC-6843 Add redis-rb (Ruby) Query Engine docs: queryjson [PARKED] #3641).

On pickup, then

When the trigger fires, run /pickup <this PR number>: it diffs each pinned source against the snapshot above, walks this checklist, then takes the PR through the normal /reflect/finalize pipeline to merge. The do not merge yet guard holds until /finalize completes.

🤖 Generated with Claude Code


Note

Low Risk
Documentation and example-only changes with no runtime or production code; main risk is documenting unreleased redis-rb APIs, which the page banner and park manifest already flag.

Overview
Adds a new Ruby client doc page Index and query vectors (vecsearch.md) that walks through indexing and KNN-searching text embeddings with redis-rb’s Query Engine, using the informers gem and all-MiniLM-L6-v2 locally.

The page is wired to a new run-verified example set ruby_home_query_vec (home_query_vec.rb): hash flow with to_bytes / FLOAT32 packing, then a JSON section covering path aliases, array vs binary storage, and packed query params. It also documents redis-rb behaviors such as default query dialect 2, stripped key prefixes on doc IDs, and a preview banner noting Query Engine APIs are not in a released gem yet.

Reviewed by Cursor Bugbot for commit 8aedcc0. Bugbot is set up for automated code reviews on this repo. Configure here.

…y_home_query_vec example

Adds the Ruby "Index and query vectors" page and its backing example set,
mirroring the redis-py/jedis vecsearch pages (hash + JSON). The embedding
library was the open question that got vector search deferred out of
DOC-6843; resolved it by reusing the semantic-cache Ruby choice — informers
running the ONNX all-MiniLM-L6-v2 encoder locally — the same 384-dim model
every other language's page uses, so vectors stay comparable across examples.
Normalising the embeddings is what makes the reported L2 distances line up
with the canonical page; run-verified against the fork (QE branch) and
Redis 8, where informers matched the reference distances to ~7 significant
figures, much closer than the ONNX drift I had expected.

Learned: informers with normalised embeddings reproduces the canonical L2 distances almost exactly; the deferred embedding-lib decision was already answered by the semantic-cache Ruby example
Constraint: keep normalize on for every encode — the L2 distances and cross-example comparability depend on unit vectors
Constraint: embedding model/library must stay aligned with the semantic-cache Ruby example (all-MiniLM-L6-v2 via informers) or stored vectors stop being comparable
Recheck: when redis-rb#1356 ships in a released redis gem (page is a parked preview against the fork)
Ticket: DOC-6854
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@andy-stark-redis andy-stark-redis added do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead. labels Jul 17, 2026
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

DOC-6854

@github-actions

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant