Visitar URL original
fix(bigtable): count pre-establishment forceClose failures only by nimf · Pull Request #14978 · googleapis/google-cloud-java · GitHub
Skip to content

fix(bigtable): count pre-establishment forceClose failures only - #14978

Open
nimf wants to merge 1 commit into
mainfrom
fallback_fix
Open

nimf wants to merge 1 commit into
mainfrom
fallback_fix

Conversation

@nimf

@nimf nimf commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Fixes two issues in FallbackChannelPool:

  • Pre-establishment forceClose masking: Track per-stream establishment outcome with a single AtomicBoolean (outcomeRecorded) instead of unconditionally setting ignoreError = true in forceClose(). This ensures pre-establishment forceClose() calls (such as SessionImpl.checkOpenTimeout()) count toward fallback while post-establishment forceClose() calls are still ignored once onBeforeSessionStart succeeds.
  • Post-close() rescheduling & callbacks: Add a closed state guard so close() is idempotent and prevents scheduleCheckRates(), checkRates(), updateConfig(), and in-flight stream listeners from rescheduling tasks or firing fallback callbacks after the pool is closed.

@nimf
nimf requested review from a team as code owners November 12, 2026 03:58

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a "closed" state to FallbackChannelPool to safely handle pool closure, prevent post-close callbacks, and avoid rescheduling tasks. It also replaces the "ignoreError" flag with an AtomicBoolean "outcomeRecorded" to prevent double-counting stream outcomes. The reviewer suggests verifying whether a circular lock dependency actually exists before moving the "onFallback" callback outside of the synchronized block in "updateConfig", offering a code alternative if no deadlock risk is present.

Comment on lines +100 to 117
synchronized void updateConfig(FallbackConfiguration newConfig) {
if (closed) {
return;
}
config =
config.toBuilder()
.setEnabled(newConfig.isEnabled())
.setCheckInterval(newConfig.getCheckInterval())
.setErrorRate(newConfig.getErrorRate())
.build();

if (triggerCallback) {
if (!config.isEnabled() && currentPool.compareAndSet(secondary, primary)) {
poolFallbackListener.onFallback(
localConfig.getFallbackName(),
localConfig.getPrimaryName(),
config.getFallbackName(),
config.getPrimaryName(),
ChannelFallbackReason.FALLBACK_DISABLE);
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Before refactoring the code to move the poolFallbackListener.onFallback callback outside of the synchronized block, please verify if a true circular lock dependency exists. Nested lock acquisition is safe if the methods called within the locked block only perform thread-safe operations (such as updating atomic variables or statistics) and do not acquire other locks or trigger callbacks that could reverse the lock acquisition order.

If you find that a circular lock dependency or deadlock risk does exist, then we should proceed with refactoring to perform state updates inside the synchronized block and invoke the callback outside of it.

  void updateConfig(FallbackConfiguration newConfig) {
    boolean triggerCallback = false;
    FallbackConfiguration localConfig;
    synchronized (this) {
      if (closed) {
        return;
      }
      config =
          config.toBuilder()
              .setEnabled(newConfig.isEnabled())
              .setCheckInterval(newConfig.getCheckInterval())
              .setErrorRate(newConfig.getErrorRate())
              .build();
      localConfig = config;

      if (!localConfig.isEnabled()) {
        triggerCallback = currentPool.compareAndSet(secondary, primary);
      }
    }

    if (triggerCallback) {
      poolFallbackListener.onFallback(
          localConfig.getFallbackName(),
          localConfig.getPrimaryName(),
          ChannelFallbackReason.FALLBACK_DISABLE);
    }
  }
References
  1. Before refactoring code to avoid potential deadlocks by moving method calls outside of synchronized blocks, verify if a true circular lock dependency exists. Nested lock acquisition is safe if the methods called within the locked block only perform thread-safe operations and do not acquire other locks or trigger callbacks that could reverse the lock acquisition order.

@nimf
nimf requested a review from igorbernstein2 November 12, 2026 04:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant