feat(kernel): route Azure Entra auth (azure-sp-m2m + azure-oauth) through the auth bridge

[eric-wang-1990]

What

Teaches the use_kernel=True auth bridge to route both Azure Entra auth flows to the kernel (databricks/databricks-sql-kernel#263):

• Azure service-principal M2M (auth_type="azure-sp-m2m") — forward the SP credentials to the kernel, which owns Azure resolution.
• Azure AD U2M (auth_type="azure-oauth") — route to the kernel's OAuth U2M flow, identically to databricks-oauth.

How

src/databricks/sql/backend/kernel/auth_bridge.py:

• azure-sp-m2m → kernel azure-sp-m2m — forward azure_client_id / azure_client_secret (+ optional azure_tenant_id, azure_workspace_resource_id). The kernel builds the Entra v2.0 token endpoint and the {effective_app_id}/.default scope, and auto-discovers the tenant from the workspace's /aad/auth redirect when azure_tenant_id is omitted (Thrift parity). The Authorization bearer (data token) alone authenticates a workspace-member SP; when azure_workspace_resource_id is set, the kernel also sends the Azure SP management token (X-Databricks-Azure-SP-Management-Token) + X-Databricks-Azure-Workspace-Resource-Id header (matching the JDBC driver), so an SP with an Azure RBAC role but no workspace membership authenticates. PECOBLR-4141.
• azure-oauth → kernel oauth-u2m — routed through the shared U2M branch, exactly like databricks-oauth. The kernel runs the in-house workspace-federated browser flow, which Azure Databricks workspaces support (the workspace federates login to Entra). It forwards the connector's databricks-sql-python app bundle, not Thrift's Azure app (96eecda7 / redirect port 8030): that app is registered for Thrift's direct-Entra flow (login.microsoftonline.com config + {tenant}/user_impersonation scopes), which the kernel does not perform. So on the kernel path azure-oauth and databricks-oauth are the same flow. The U2M-vs-M2M ambiguity guard now also covers azure-oauth, so azure-oauth + oauth_client_secret is rejected as ambiguous rather than silently falling through to M2M. PECOBLR-4120.
• src/databricks/sql/session.py threads the azure_* connection kwargs into kernel_auth_options (the kernel owns Azure resolution, so these raw kwargs are the only source).

The binding is thin — it does not construct endpoints or scopes; the kernel does.

Tests

• TestKernelAzureSpM2M: forwards creds; tenant optional (kernel auto-discovers); azure_workspace_resource_id forwarded; required-creds error; federation client id.
• TestKernelOAuthU2M: azure-oauth maps to oauth-u2m with the databricks-sql-python bundle (identical to databricks-oauth); custom client_id/port/scopes honored; the ambiguity guard (U2M auth_type + oauth_client_secret) is parametrized over both databricks-oauth and azure-oauth.
• session-threading test asserts the azure_* kwargs reach auth_options.

Bridge + session + auth unit tests pass; black clean. (test_kernel_client.py skips without the kernel wheel, as before.)

E2E verification (live Azure workspace)

Ran the full connector path (sql.connect(use_kernel=True, ...)) against a live Azure Databricks workspace, kernel wheel built from databricks/databricks-sql-kernel#263. All three Azure kernel-path flows round-trip SELECT 1:

• Azure U2M (auth_type="azure-oauth") — full interactive browser authorization-code flow (cache cleared to force it). Log confirms the design: authorize URL is the workspace's own OIDC endpoint ({host}/oidc/v1/authorize), client_id=databricks-sql-python, redirect_uri=http://localhost:8020, scope=sql offline_access — i.e. the in-house workspace-federated flow, not Thrift's direct-Entra app (96eecda7 / port 8030 / user_impersonation). Authenticated interactively and returned SELECT 1. ✅
• Azure SP M2M, data token (auth_type="azure-sp-m2m", no azure_workspace_resource_id) — workspace-member SP, Row(one=1, who='<sp-app-id>'). ✅
• Azure SP M2M, management token (azure-sp-m2m + azure_workspace_resource_id) — the kernel mints the ARM management token via the Entra v1.0 resource= grant (aud https://management.core.windows.net/, trailing slash) and sends the X-Databricks-Azure-SP-Management-Token + -Workspace-Resource-Id header pair; round-trip succeeds. ✅

Negative controls confirm the pipeline is real, not a no-op: a mismatched SP secret surfaces Entra AADSTS7000215, and a service principal with no workspace access returns HTTP 403 User not authorized (a real control-plane authz decision, not a bridge error).

Note: this required a kernel wheel with the azure-sp-m2m pyo3 surface — the currently-published databricks-sql-kernel 0.2.0 wheel lacks it (Session.__new__() got an unexpected keyword argument 'azure_client_id'), so merging this depends on a kernel release that includes #263. (Azure U2M works on the old wheel — oauth-u2m predates it.) Not yet covered: the RBAC-only authorization case (a non-member SP authorized purely by an Azure role via the management token), which needs an Azure role assignment still pending.

Related

• Kernel side: databricks/databricks-sql-kernel#263.
• JIRA: PECOBLR-4141 (azure-sp-m2m), PECOBLR-4120 (azure-oauth → kernel OAuth U2M).

This pull request and its description were written by Isaac.

[Copilot]

Pull request overview

This PR extends the use_kernel=True auth bridge to support Azure Entra (Azure AD) OAuth by routing the connector’s Azure auth types (azure-oauth U2M and azure-sp-m2m M2M) onto the kernel’s generic OAuth flows with Azure-specific overrides.

Changes:

• Route auth_type="azure-oauth" to kernel oauth-u2m using the Azure OAuth app bundle and Azure scope mapping.
• Route auth_type="azure-sp-m2m" to kernel oauth-m2m using Entra v2.0 token URL and {effective_app_id}/.default scope; require azure_tenant_id.
• Thread hostname into kernel_auth_kwargs (via KernelDatabricksClient.open_session) and add/adjust unit tests + changelog entry.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/databricks/sql/backend/kernel/auth_bridge.py	Adds Azure auth-type routing and related override plumbing for kernel session kwargs.
src/databricks/sql/backend/kernel/client.py	Passes the server hostname into kernel_auth_kwargs during session open.
tests/unit/test_kernel_auth_bridge.py	Updates U2M tests for azure-oauth routing and adds azure-sp-m2m routing/validation tests.
CHANGELOG.md	Documents Azure Entra OAuth support on the kernel backend.

Suppressed comments (1)

src/databricks/sql/backend/kernel/auth_bridge.py:262

• auth_type='azure-sp-m2m' uses azure_client_id/azure_client_secret, but because this branch returns before the generic ambiguity guards, a caller can also supply oauth_client_id/oauth_client_secret and have them silently ignored. This makes misconfigurations hard to diagnose.

    if auth_type == "azure-sp-m2m":
        azure_client_id = opts.get("azure_client_id")
        azure_client_secret = opts.get("azure_client_secret")
        azure_tenant_id = opts.get("azure_tenant_id")
        if not (azure_client_id and azure_client_secret):

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[peco-engineer-bot]

Already resolved in the current code. The early-return azure-oauth branch the comment targeted was refactored away: azure-oauth now falls through to the shared U2M branch (auth_bridge.py:419), which runs AFTER the ambiguity guards. The guard at auth_bridge.py:294-300 explicitly rejects auth_type in ('databricks-oauth','azure-oauth') together with oauth_client_secret (and the JWT-M2M guard at 316-322 covers oauth_jwt_key_file), so a stray M2M secret is now a loud NotSupportedError rather than silently ignored. Covered by TestKernelAuthAmbiguity::test_u2m_auth_type_plus_client_secret_is_rejected, parametrized over both databricks-oauth and azure-oauth — passing.

[peco-engineer-bot]

This concern no longer applies to the current branch. The azure-sp-m2m branch (src/databricks/sql/backend/kernel/auth_bridge.py:256–283) was refactored: it no longer calls get_effective_azure_login_app_id(hostname or "") and no longer computes oauth_scopes in Python. Azure endpoint/app-id/scope resolution (including the {effective_app_id}/.default scope and Entra v2.0 token endpoint) now happens inside the kernel, and the bridge only forwards the azure-sp-m2m selector plus the Azure SP credentials and optional passthroughs. There is therefore no PROD-app-id fallback and no scope-defaulting path that could go wrong when hostname is omitted, so no code change is needed.

[peco-review-bot]

Verdict: 1 Medium

Routing logic is clean and well-tested overall. One medium concern: the new Azure branches return before the ambiguity guards, so azure-oauth + oauth_client_secret (or + credentials_provider) is silently accepted as U2M whereas the parallel databricks-oauth case raises NotSupportedError. Nit (not filed inline): the step-3 comment # Only databricks-oauth reaches here (azure-oauth rejected up front) in the U2M block is now stale — azure-oauth is routed up front, not rejected.

[peco-review-bot]

Verdict: 1 Low

Looks good — clean, well-documented routing with strong test coverage for both new Azure flows (routing, required tenant/creds, federation client id). One low-severity consistency note: the azure-oauth branch returns before the ambiguity guards, so oauth_client_secret / credentials_provider are silently ignored there, unlike the databricks-oauth U2M path which rejects them.

[peco-review-bot]

Verdict: 1 Medium · 2 Low

Overall solid — the azure-sp-m2m v2.0 .default scope + v2.0 token endpoint correctly mirror the Thrift path's v1.0 resource= form, and the empty-hostname fallback is safe. Main issue is a stale module docstring (F1, medium) that claims the azure-oauth bridge synthesises the client id/port/scope via AzureOAuthEndpointCollection when the code actually just forwards the azure-oauth selector and lets the kernel own resolution — the inline comment already contradicts it. Plus two low doc/consistency nits.

[peco-review-bot]

Verdict: 1 Medium · 1 Low

Looks good overall — the Azure routing logic is thin, well-tested, and correct. Two cleanup items: an unused hostname parameter now threaded through client.py for no effect (medium), and a dead test import (low). Nit: the comment at auth_bridge.py:318 ("azure-oauth rejected up front") is now stale — azure-oauth is routed up front, not rejected.

[peco-review-bot]

Verdict: 1 Low

Looks good — clean, thin forwarding of both Azure Entra flows with thorough unit coverage (routing, ambiguity guards extended to azure-oauth, required-creds, optional passthroughs, session threading). One low-severity consistency note: the azure-sp-m2m early return bypasses the module's fail-loud ambiguity guards, so conflicting cross-namespace signals are silently ignored rather than rejected.

[peco-review-bot]

Verdict: 1 Low

Looks good — clean, thin binding that correctly forwards the Azure selector + creds and lets the kernel own resolution, with thorough unit coverage (bridge + session threading + ambiguity guards parametrized over both U2M types). One low-severity note about the intentional ambiguity-guard exemption on the azure-sp-m2m branch diverging from the module's loud-failure philosophy.

[eric-wang-1990]

azure-oauth for Kernel mode is same with databricks-oauth, which is inHouse OAuth U2M

[peco-engineer-bot]

Agreed — and the doc already states exactly this. CONNECTION_PARAMETERS.md:72 says azure-oauth on the kernel path "routes to the kernel's OAuth U2M flow identically to databricks-oauth (the in-house workspace-federated browser flow, which Azure workspaces support)." That matches your point that kernel-mode azure-oauth is the same in-house OAuth U2M as databricks-oauth, so no change is needed.

[peco-review-bot]

Verdict: 1 Low · 1 Nit

Looks good — a clean, thin binding that forwards the Azure selector + credentials to the kernel, with the azure-sp-m2m branch correctly placed before the ambiguity guards and session.py threading the azure_* kwargs only into the kernel path. Test coverage is solid; just one low (untested silent-ignore asymmetry for conflicting signals on azure-sp-m2m) and one nit (stale docstring count).

Other findings

• ⚪ Nit — Module docstring says "Three auth shapes are supported on the kernel path" but the bullet list now enumerates four top-level categories (PAT, OAuth M2M, OAuth U2M, and Azure Entra) plus JWT private-key M2M. The count was already slightly stale before this PR (JWT M2M), and this PR's new Azure Entra section makes it more so. Consider dropping the fixed count (e.g. "The following auth shapes are supported on the kernel path:") so it doesn't need updating each time a flow is added.

[peco-review-bot]

Verdict: 1 Low

Looks good — clean, well-tested addition routing azure-sp-m2m and azure-oauth through the kernel auth bridge, with matching session threading and thorough unit coverage (branch ordering, ambiguity guards, and the intentional azure-sp-m2m asymmetry are all exercised). One low note on log visibility for the silently-ignored conflicting-credential path.

[peco-review-bot]

✅ No issues identified by the review bot.

[peco-review-bot]

Verdict: 1 Low

Looks good — the Azure Entra routing (azure-sp-m2m up-front handling, azure-oauth folded into the shared U2M branch, both ambiguity guards extended) is correct, and session.py threads the azure_* kwargs only on the kernel path. Tests cover every new branch. Only one minor docstring-count inaccuracy noted inline.

[peco-review-bot]

✅ No issues identified by the review bot.