Visitar URL original
fix(bigquery): preserve existing configured dataset by cyberhck · Pull Request #7377 · rudderlabs/rudder-server · GitHub
Skip to content

fix(bigquery): preserve existing configured dataset - #7377

Open
cyberhck wants to merge 1 commit into
rudderlabs:masterfrom
cyberhck:fix/bigquery-existing-namespace
Open

cyberhck wants to merge 1 commit into
rudderlabs:masterfrom
cyberhck:fix/bigquery-existing-namespace

Conversation

@cyberhck

@cyberhck cyberhck commented Sep 11, 2026 •

Copy link
Copy Markdown

Description

Preserve an explicitly configured BigQuery dataset ID when that exact dataset already exists.

The warehouse namespace builder snake-cases custom namespaces before the BigQuery manager is created. bq_events_zm_dev_dts_euwe1, for example, becomes bq_events_zm_dev_dts_euwe_1 due to its letter-to-digit boundary. The BigQuery setup path now checks the original configured dataset after establishing its client. If it exists, subsequent schema management and load operations use it verbatim. If it does not exist, the existing sanitised namespace behaviour remains unchanged.

Added regression coverage for the exact letter/digit case, absent datasets, both datasets existing, and failed/no-op checks.

Linear Ticket

No linear provided, but I have filed a support ticket for a finding, this would solve that (I think)

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

New behavior

  • Exact configured dataset exists: preserve its ID, including letter-to-digit boundaries.
  • Exact configured dataset is absent: retain the existing sanitised name and creation flow.
  • Dataset lookup errors: retain the existing sanitised name rather than failing a previously valid setup path.

@contributor-support

Copy link
Copy Markdown

Thank you @cyberhck for contributing this PR.
Please sign the Contributor License Agreement (CLA) before merging.

@cyberhck cyberhck closed this Sep 11, 2026
@cyberhck
cyberhck deleted the fix/bigquery-existing-namespace branch September 11, 2026 11:10
@cyberhck
cyberhck restored the fix/bigquery-existing-namespace branch September 11, 2026 11:34
@cyberhck cyberhck reopened this Sep 11, 2026
@cyberhck

Copy link
Copy Markdown
Author

I've reviewed, I'm very new to the codebase (first contrib), but it was helped with LLM (I have used it as a flashlight, to debug what went wrong what is the call site, and how to fix), please let me know if I'm missing something or something needs to be added. Happy to change.

Basically any dataset that someone might have convention for: <dataset-name>-region might have numbers in them, and if someone has already created the dataset, and they try to configure this, currently it's normalizing and adding a _ in front. (in our instance bq_events_zm_dev_dts_euwe1 got changed into bq_events_zm_dev_dts_euwe_1

this PR attempts to retain the original data from config and then if the dataset exists, tries to use that instead.

@cyberhck

Copy link
Copy Markdown
Author

hello team, is there anything we can do about this?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant