Visitar URL original
xds: ring_hash `requestHashHeader` is not validated at config parse time; a `-bin` or invalid header name panics the channel · Issue #13151 · grpc/grpc-java · GitHub
Skip to content

xds: ring_hash requestHashHeader is not validated at config parse time; a -bin or invalid header name panics the channel #13151

Open

Description

@hidingmyname

What version of gRPC-Java are you using?

master at 15d64a4 (1.86.0-SNAPSHOT). The code involved is unchanged on the current master. This requires GRPC_EXPERIMENTAL_RING_HASH_SET_REQUEST_HASH_KEY=true.

What is your environment?

Linux 5.15, OpenJDK 21.0.2, Gradle 8.14.5 (wrapper).

What did you expect to see?

gRFC A76 says:

Upon loading the load balancing config, if the request_hash_header field contains a value that is not a valid header name, then the configuration is rejected. If the request_hash_header refers to a binary header (suffixed with -bin), the configuration is also rejected.

So RingHashLoadBalancerProvider.parseLoadBalancingPolicyConfig should return an error for values like "header-bin" or "!invalid", and the channel should handle that like any other invalid service config.

What did you see instead?

The provider accepts any string. The invalid name only fails later, in RingHashLoadBalancer.acceptResolvedAddresses, where Metadata.Key.of(...) throws IllegalArgumentException inside the synchronization context. The channel goes into panic mode, and every RPC fails with INTERNAL: Panic! This is a bug!.

Location (grpc/grpc-java @ 15d64a4):

    String requestHashHeader = "";
    if (GrpcUtil.getFlag("GRPC_EXPERIMENTAL_RING_HASH_SET_REQUEST_HASH_KEY", false)) {
      requestHashHeader = JsonUtil.getString(rawLoadBalancingPolicyConfig, "requestHashHeader");
    }
    requestHashHeaderKey =
        config.requestHashHeader.isEmpty()
            ? null
            : Metadata.Key.of(config.requestHashHeader, Metadata.ASCII_STRING_MARSHALLER);

Problem

Metadata.Key.of(name, ASCII_STRING_MARSHALLER) rejects names with invalid characters and names ending in -bin. Because the provider does not do this check, a service config with such a requestHashHeader gets past parsing, and the exception is thrown from the LB policy. That exception panics the whole channel, when only the config should have been rejected. This only happens when the experimental GRPC_EXPERIMENTAL_RING_HASH_SET_REQUEST_HASH_KEY flag is enabled. Mixed-case names work today, because Metadata.Key lower-cases them.

Reproduction

RingHashRequestHashHeaderValidationTest.java (xds/src/test/java/io/grpc/xds)
/*
 * Copyright 2026 The gRPC Authors
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package io.grpc.xds;

import static com.google.common.truth.Truth.assertThat;
import static org.junit.Assert.assertThrows;

import com.google.common.collect.ImmutableList;
import io.grpc.CallOptions;
import io.grpc.EquivalentAddressGroup;
import io.grpc.ManagedChannel;
import io.grpc.NameResolver;
import io.grpc.NameResolver.ConfigOrError;
import io.grpc.NameResolverProvider;
import io.grpc.Status.Code;
import io.grpc.StatusOr;
import io.grpc.StatusRuntimeException;
import io.grpc.inprocess.InProcessChannelBuilder;
import io.grpc.inprocess.InProcessSocketAddress;
import io.grpc.internal.JsonParser;
import io.grpc.stub.ClientCalls;
import io.grpc.testing.TestMethodDescriptors;
import io.grpc.xds.RingHashLoadBalancer.RingHashConfig;
import java.io.IOException;
import java.net.SocketAddress;
import java.net.URI;
import java.util.Collection;
import java.util.Collections;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

/**
 * Tests that {@link RingHashLoadBalancerProvider} rejects an invalid requestHashHeader while
 * parsing the config, as required by gRFC A76.
 */
@RunWith(JUnit4.class)
public class RingHashRequestHashHeaderValidationTest {

  private static final String ENV_VAR_NAME =
      "GRPC_EXPERIMENTAL_RING_HASH_SET_REQUEST_HASH_KEY";

  private final RingHashLoadBalancerProvider provider = new RingHashLoadBalancerProvider();

  @Before
  public void setUp() {
    System.setProperty(ENV_VAR_NAME, "true");
  }

  @After
  public void tearDown() {
    System.clearProperty(ENV_VAR_NAME);
  }

  @Test
  public void requestHashHeader_binSuffixRejected() throws IOException {
    String lbConfig = "{\"requestHashHeader\": \"header-bin\"}";
    ConfigOrError configOrError =
        provider.parseLoadBalancingPolicyConfig(parseJsonObject(lbConfig));
    assertThat(configOrError.getError()).isNotNull();
    assertThat(configOrError.getError().getCode()).isEqualTo(Code.UNAVAILABLE);
  }

  @Test
  public void requestHashHeader_invalidCharactersRejected() throws IOException {
    String lbConfig = "{\"requestHashHeader\": \"!invalid\"}";
    ConfigOrError configOrError =
        provider.parseLoadBalancingPolicyConfig(parseJsonObject(lbConfig));
    assertThat(configOrError.getError()).isNotNull();
    assertThat(configOrError.getError().getCode()).isEqualTo(Code.UNAVAILABLE);
  }

  @Test
  public void requestHashHeader_mixedCaseAccepted() throws IOException {
    String lbConfig = "{\"requestHashHeader\": \"X-Foo\"}";
    ConfigOrError configOrError =
        provider.parseLoadBalancingPolicyConfig(parseJsonObject(lbConfig));
    assertThat(configOrError.getError()).isNull();
    assertThat(((RingHashConfig) configOrError.getConfig()).requestHashHeader).isNotEmpty();
  }

  @Test
  public void invalidRequestHashHeaderFromResolver_doesNotPanicChannel() throws Exception {
    final Map<String, ?> serviceConfig = parseJsonObject(
        "{\"loadBalancingConfig\": [{\"ring_hash_experimental\": "
            + "{\"requestHashHeader\": \"header-bin\"}}]}");
    final String target = "fake:///ring-hash-test";
    NameResolverProvider resolverProvider = new NameResolverProvider() {
      @Override
      public NameResolver newNameResolver(URI targetUri, final NameResolver.Args args) {
        return new NameResolver() {
          @Override
          public String getServiceAuthority() {
            return "ring-hash-test";
          }

          @Override
          public void start(Listener2 listener) {
            listener.onResult(
                ResolutionResult.newBuilder()
                    .setAddressesOrError(StatusOr.fromValue(ImmutableList.of(
                        new EquivalentAddressGroup(
                            new InProcessSocketAddress("ring-hash-test")))))
                    .setServiceConfig(args.getServiceConfigParser()
                        .parseServiceConfig(serviceConfig))
                    .build());
          }

          @Override
          public void shutdown() {}
        };
      }

      @Override
      public String getDefaultScheme() {
        return "fake";
      }

      @Override
      protected boolean isAvailable() {
        return true;
      }

      @Override
      protected int priority() {
        return 5;
      }

      @Override
      public Collection<Class<? extends SocketAddress>> getProducedSocketAddressTypes() {
        return Collections.singleton(InProcessSocketAddress.class);
      }
    };

    ManagedChannel channel = InProcessChannelBuilder.forTarget(target)
        .nameResolverFactory(resolverProvider)
        .build();
    try {
      StatusRuntimeException e = assertThrows(StatusRuntimeException.class, () ->
          ClientCalls.blockingUnaryCall(
              channel,
              TestMethodDescriptors.voidMethod(),
              CallOptions.DEFAULT.withDeadlineAfter(5, TimeUnit.SECONDS),
              null));
      assertThat(e.getStatus().getDescription()).doesNotContain("Panic");
    } finally {
      channel.shutdownNow();
    }
  }

  @SuppressWarnings("unchecked")
  private static Map<String, ?> parseJsonObject(String json) throws IOException {
    return (Map<String, ?>) JsonParser.parse(json);
  }
}

Run: ./gradlew :grpc-xds:test --tests io.grpc.xds.RingHashRequestHashHeaderValidationTest -PskipCodegen=true -PskipAndroid=true

RingHashRequestHashHeaderValidationTest > invalidRequestHashHeaderFromResolver_doesNotPanicChannel FAILED
    expected not to contain: Panic
    but was                : Panic! This is a bug!
        at app//io.grpc.xds.RingHashRequestHashHeaderValidationTest.invalidRequestHashHeaderFromResolver_doesNotPanicChannel(RingHashRequestHashHeaderValidationTest.java:163)

RingHashRequestHashHeaderValidationTest > requestHashHeader_invalidCharactersRejected FAILED
    value of          : getError()
    expected not to be: null
        at app//io.grpc.xds.RingHashRequestHashHeaderValidationTest.requestHashHeader_invalidCharactersRejected(RingHashRequestHashHeaderValidationTest.java:87)

RingHashRequestHashHeaderValidationTest > requestHashHeader_binSuffixRejected FAILED
    value of          : getError()
    expected not to be: null
        at app//io.grpc.xds.RingHashRequestHashHeaderValidationTest.requestHashHeader_binSuffixRejected(RingHashRequestHashHeaderValidationTest.java:78)

4 tests completed, 3 failed

Log from the channel test:

Oct 31, 2026 7:36:13 PM io.grpc.internal.ManagedChannelImpl$3 uncaughtException
SEVERE: [Channel<1>: (fake:///ring-hash-test)] Uncaught exception in the SynchronizationContext. Panic!
java.lang.IllegalArgumentException: ASCII header is named header-bin.  Only binary headers may end with -bin
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:466)
	at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:972)
	at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:966)
	at io.grpc.Metadata$Key.of(Metadata.java:708)
	at io.grpc.Metadata$Key.of(Metadata.java:704)
	at io.grpc.xds.RingHashLoadBalancer.acceptResolvedAddresses(RingHashLoadBalancer.java:112)
	at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.acceptResolvedAddresses(AutoConfiguredLoadBalancerFactory.java:116)
	at io.grpc.internal.ManagedChannelImpl$NameResolverListener.onResult2(ManagedChannelImpl.java:1792)
	at io.grpc.internal.RetryingNameResolver$RetryingListener.onResult2(RetryingNameResolver.java:101)

Environment: commit 15d64a4, OpenJDK 21.0.2, Gradle 8.14.5.

Already fixed in grpc/grpc-go

PR: grpc/grpc-go#8255, "Cherry-pick #8159 and #8243 to v1.72.x" (merged 2025-04-16T10:09:31Z, merge commit fd6f5852919). It brings grpc-go's gRFC A76 implementation to the 1.72 branch. grpc-go is another implementation of the same gRFC. Its parseConfig lower-cases requestHashHeader, validates it as a metadata key, and rejects -bin names, returning a config error.

How grpc-go fixed it (relevant hunks, verbatim):

--- a/xds/internal/balancer/ringhash/config.go
+++ b/xds/internal/balancer/ringhash/config.go
@@ -66,5 +69,18 @@ func parseConfig(c json.RawMessage) (*LBConfig, error) {
 	if cfg.MaxRingSize > envconfig.RingHashCap {
 		cfg.MaxRingSize = envconfig.RingHashCap
 	}
+	if !envconfig.RingHashSetRequestHashKey {
+		cfg.RequestHashHeader = ""
+	}
+	if cfg.RequestHashHeader != "" {
+		cfg.RequestHashHeader = strings.ToLower(cfg.RequestHashHeader)
+		// See rules in https://github.com/grpc/proposal/blob/master/A76-ring-hash-improvements.md#explicitly-setting-the-request-hash-key
+		if err := metadata.ValidateKey(cfg.RequestHashHeader); err != nil {
+			return nil, fmt.Errorf("invalid requestHashHeader %q: %v", cfg.RequestHashHeader, err)
+		}
+		if strings.HasSuffix(cfg.RequestHashHeader, "-bin") {
+			return nil, fmt.Errorf("invalid requestHashHeader %q: key must not end with \"-bin\"", cfg.RequestHashHeader)
+		}
+	}
 	return &cfg, nil
 }

Suggested fix

Validate the name in the provider with the same Metadata.Key rules used at pick time, and return an UNAVAILABLE config error if it is invalid. This also stores the name in its normalized lower-case form:

diff --git a/xds/src/main/java/io/grpc/xds/RingHashLoadBalancerProvider.java b/xds/src/main/java/io/grpc/xds/RingHashLoadBalancerProvider.java
--- a/xds/src/main/java/io/grpc/xds/RingHashLoadBalancerProvider.java
+++ b/xds/src/main/java/io/grpc/xds/RingHashLoadBalancerProvider.java
@@ -22,6 +22,7 @@ import io.grpc.Internal;
 import io.grpc.LoadBalancer;
 import io.grpc.LoadBalancer.Helper;
 import io.grpc.LoadBalancerProvider;
+import io.grpc.Metadata;
 import io.grpc.NameResolver.ConfigOrError;
 import io.grpc.Status;
 import io.grpc.internal.GrpcUtil;
@@ -106,6 +107,16 @@ public final class RingHashLoadBalancerProvider extends LoadBalancerProvider {
       return ConfigOrError.fromError(Status.UNAVAILABLE.withDescription(
           "Invalid 'minRingSize'/'maxRingSize'"));
     }
+    if (!requestHashHeader.isEmpty()) {
+      // gRFC A76: reject header names that are invalid or refer to a binary (-bin) header.
+      try {
+        requestHashHeader =
+            Metadata.Key.of(requestHashHeader, Metadata.ASCII_STRING_MARSHALLER).name();
+      } catch (IllegalArgumentException e) {
+        return ConfigOrError.fromError(Status.UNAVAILABLE.withDescription(
+            "Invalid 'requestHashHeader': " + requestHashHeader).withCause(e));
+      }
+    }
     return ConfigOrError.fromConfig(
         new RingHashConfig(minRingSize, maxRingSize, requestHashHeader));
   }

Verified on 15d64a4 (OpenJDK 21.0.2, Gradle 8.14.5): the test above fails without this patch and passes with it; no other test results change (:grpc-xds:test: 4473 tests; FakeControlPlaneXdsIntegrationTest.changeClusterForRoute[enableRfc3986UrisParam=true] fails with and without the patch).

I can submit a PR for this.

I searched issues and PRs for requestHashHeader/request_hash_header and ring hash A76. The only hit is #11881, which added the field without this validation.

Activity

  1. shivaspeaks commented on Nov 2, 2026

    @shivaspeaks
    Member

    I agree with the issue. Send out a PR for review.

  2. added this to the Next milestone on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions