Skip to content

feat(storage): support Jindo for OSS scan planning - #696

Draft
XiaoHongbo-Hope wants to merge 1 commit into
apache:mainfrom
XiaoHongbo-Hope:codex/jindo-storage
Draft

feat(storage): support Jindo for OSS scan planning#696
XiaoHongbo-Hope wants to merge 1 commit into
apache:mainfrom
XiaoHongbo-Hope:codex/jindo-storage

Conversation

@XiaoHongbo-Hope

Copy link
Copy Markdown
Contributor

Purpose

Allow Rust scan planning to use an installed JindoSDK for OSS metadata and index reads. The existing OpenDAL OSS implementation remains the default.

Changes

  • add an optional storage-jindo feature selected by fs.oss.impl=jindo
  • dynamically load the Jindo C ABI without bundling an SDK binary
  • support object stat, ranged reads, suffix reads, and full reads
  • run blocking SDK calls on Tokio blocking workers
  • discover the SDK through an explicit path, environment variables, JINDOSDK_HOME, or an installed pyjindosdk package
  • fail explicitly for writes, deletes, copies, and listings in this initial read-only integration

Validation

  • Jindo configuration and backend selection unit tests
  • Python SDK discovery unit test
  • default OSS and storage-jindo feature builds
  • Clippy with warnings denied
  • read-only end-to-end planning against an OSS-backed REST catalog using automatic pyjindosdk discovery

Notes

No JindoSDK binary is linked or packaged by this project. Selecting Jindo without the feature or a loadable SDK returns a configuration error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant