[python] Support DLF token path authentication - #9089
Conversation
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
One Java-alignment issue needs to be addressed.
| ) | ||
| role_name = options.get(CatalogOptions.DLF_TOKEN_ECS_ROLE_NAME) | ||
| return DLFECSTokenLoader(ecs_metadata_url, role_name) | ||
| if loader == 'local_file': |
There was a problem hiding this comment.
Could we make the credential-source precedence match Java here? When dlf.token-path (or dlf.token-loader=local_file) is configured together with static dlf.access-key-id / dlf.access-key-secret, this factory returns a loader, but AuthProviderFactory also passes DLFToken.from_options(options) to DLFAuthProvider. Because that static token has no expiration, get_token() never calls the loader, so the token file is silently ignored. Java selects the loader/path before static AK/SK. Please choose a single source in the Python factory (for example, pass token=None whenever a loader exists) and add a mixed-configuration regression test.
There was a problem hiding this comment.
Thanks for catching this! You are right. I updated AuthProviderFactory to select a single credential source: when an explicit token loader or the implicit local_file loader from dlf.token-path is present, the provider now gets token=None, so the loader takes precedence over static AK/SK, matching Java. Unknown explicit loader names now fail instead of falling back to static credentials. I also added regression coverage for mixed configurations with both implicit dlf.token-path and explicit dlf.token-loader=local_file.
Add a local-file DLF token loader with retry and credential-safe errors, and wire dlf.token-path into DLF loader selection and refresh. Co-Authored-By: Codex <noreply@openai.com>
Ensure configured token loaders take precedence over static credentials and reject unknown loader identifiers.
d5ec018 to
f9bf147
Compare
|
+1 |
2 participants
Purpose
Add
dlf.token-pathauthentication support to PyPaimon REST catalogs.PyPaimon now loads temporary DLF credentials from a local JSON file, retries transient file-read failures, and reloads expiring credentials. The behavior aligns with the existing Java
local_filetoken loader.Tests
PYTHONPATH=. python -m pytest pypaimon/tests/rest/token_loader_test.py pypaimon/tests/rest/dlf_signer_test.py pypaimon/tests/rest/api_test.py -q