Visitar URL original
feat(feature-flags): generalize driver-side cache by cathleeny · Pull Request #974 · databricks/databricks-sql-python · GitHub
Skip to content

feat(feature-flags): generalize driver-side cache - #974

Open
cathleeny wants to merge 6 commits into
mainfrom
cathleeny/general-driver-flags
Open

cathleeny wants to merge 6 commits into
mainfrom
cathleeny/general-driver-flags

Conversation

@cathleeny

@cathleeny cathleeny commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Generalize the existing feature-flag cache for driver-owned flags on the Thrift path and before kernel/session initialization, once authenticated transport is available.

  • Share raw flag values by workspace ID, with normalized host as a fallback; keep credentials and HTTP clients caller-owned.
  • Add typed Boolean, int32, int64, double, string, and string-list getters. Integer validation uses standard-library fixed-width types.
  • Keep the blocking initial fetch and background refresh, retain stale values on refresh failures, and avoid queuing duplicate refreshes.

How is this tested?

  • Full non-real-kernel unit suite: 1,032 passed, 5 skipped, 1 deselected; 351 subtests passed.
  • Focused telemetry/feature-flag suite: 58 passed.
  • Black and mypy checks passed.
  • Covers all six types, defaults/range validation, workspace sharing, and caller-owned refresh/authentication.

Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:30 — with GitHub Actions Active
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:30 — with GitHub Actions Active
@cathleeny cathleeny added the kernel-e2e Trigger preview run of the Kernel E2E workflow on this PR label Oct 7, 2026
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:32 — with GitHub Actions Active
@cathleeny
cathleeny marked this pull request as ready for review October 7, 2026 17:38
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 17:39 — with GitHub Actions Active
@cathleeny
cathleeny requested review from a team, jay-xiao446 and vuanhphung and removed request for a team October 7, 2026 17:39

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Solid, well-tested generalization of the feature-flag cache — the typed getters correctly exclude bool-as-int, enforce fixed-width ranges, and reject NaN/inf, and the shared _CacheState + RLock refresh coordination is sound. One low-severity lifecycle note: get_instance is now created eagerly per non-kernel connection while remove_instance has no production caller, so the refresher executor/cache is never cleaned up.

Comment thread src/databricks/sql/session.py Outdated
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 18:06 — with GitHub Actions Active
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 18:06 — with GitHub Actions Active
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 18:06 — with GitHub Actions Active
@github-actions github-actions Bot removed the kernel-e2e Trigger preview run of the Kernel E2E workflow on this PR label Oct 7, 2026

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks solid overall — the typed getters are correct and well-covered (bool/int range via ctypes, double finiteness, strict list typing), and the workspace-keyed sharing with host fallback is sensible and tested. One low-severity lifecycle concern: remove_instance (which shuts down the shared refresh executor and evicts cache state) has no production caller, so the executor and per-workspace cache leak for the process lifetime. Minor nit below.

Nit (no anchor needed): the typed getters' default_value parameters (get_int32/get_int64/get_double/get_string/get_string_list, and _get_int) lack type hints, unlike the annotated return types; adding them would match the repo's type-hint convention (CONTRIBUTING.md).

Comment thread src/databricks/sql/common/feature_flag.py
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny
cathleeny deployed to azure-prod October 7, 2026 22:53 — with GitHub Actions Active

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a clean, well-tested generalization of the feature-flag cache (shared _CacheState per workspace, typed JSON getters, refresh dedup, pre-session reader). One low-severity note: get_bool is a stricter parser than the str(...).lower() == "true" telemetry gate it replaces, so the telemetry flag now depends on the server emitting a bare lowercase JSON boolean. Nit (summary-only): the typed getters annotate return types but leave default_value/integer_type/name params untyped in _get_int, get_int32/64, get_double, etc. — CONTRIBUTING.md asks for type hints; adding default_value: Optional[int] = None style annotations would keep them consistent with get_bool/get_string.

try:
return json.loads(raw) if raw is not None else None
except (TypeError, ValueError):
return None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — get_bool is stricter than the telemetry gate it replaces. The old path did str(flag_value).lower() == "true", which tolerated "True", "TRUE", and a quoted-string value. get_bool now requires the raw flag value to be a bare JSON boolean (json.loads(raw) must yield type(value) is bool). A server value of True/TRUE, or a JSON-quoted "\"true\"", now parses to a non-bool and silently returns the False default — disabling telemetry where it previously was enabled.

The unit mocks store the bare lowercase form (str(enabled).lower()), so tests don't exercise this. Worth confirming the connector-service contract guarantees the enableTelemetryForPythonDriver flag is emitted as a bare lowercase JSON boolean before relying on the stricter parse.

This branch was successfully deployed

1 active deployment
azure-prod — 8a2c905d Deployed Oct 7, 2026 by peco-review-bot[bot] via followup #977
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant