[Bug] DUPLICATE KEY tables accept unsupported BITMAP/HLL/QUANTILE_STATE distribution columns #68749

Search before asking

• I had searched in the issues and found no similar issues.

Version

4.1.4.1

What's Wrong?

A DUPLICATE KEY table can distribute on a non-key BITMAP column. With enable_non_aggregate_table_state_types enabled, HLL and QUANTILE_STATE columns are also accepted as distribution columns, although these types lack complete, correct write-side routing support:

• BITMAP and QUANTILE_STATE have no case in RawValue::zlib_crc32() and reach the default DCHECK(false) followed by return 0. A real tablet-finder test reproduced BITMAP assertion termination in an ASAN subprocess. QUANTILE_STATE routing was inspected in source, not separately executed.
• HLL has a case, but it hashes the C++ object memory returned by ColumnHLL::get_data_at(). In a real tablet-finder test, an original HLL, its copy and an independently deserialized instance have identical serialization but receive different bucket IDs.

Distribution-type validation already rejects ARRAY, MAP, STRUCT, JSONB, VARIANT and floating-point types, but omits these three state types. Relaxing table-model restrictions through the compatibility switch should not also permit unsupported distribution columns.

What You Expected?

CREATE TABLE should reject unsupported BITMAP, HLL and QUANTILE_STATE distribution columns with a clear error, even when the non-aggregate state-type compatibility switch is enabled.

How to Reproduce?

1. With the default configuration, execute:

CREATE DATABASE IF NOT EXISTS issue_state_distribution;
USE issue_state_distribution;

DROP TABLE IF EXISTS bitmap_hash;
CREATE TABLE bitmap_hash (k INT, s BITMAP)
DUPLICATE KEY(k)
DISTRIBUTED BY HASH(s) BUCKETS 1
PROPERTIES ('replication_num' = '1');

SHOW CREATE TABLE bitmap_hash;

Actual: CREATE succeeds and the definition retains HASH(s), where s is BITMAP. This unsupported distribution definition should be rejected.

2. Temporarily enable the compatibility switch on the same test cluster and execute the complete definitions for the other two state types:

ADMIN SET FRONTEND CONFIG
('enable_non_aggregate_table_state_types' = 'true');

DROP TABLE IF EXISTS hll_hash;
CREATE TABLE hll_hash (k INT, s HLL)
DUPLICATE KEY(k)
DISTRIBUTED BY HASH(s) BUCKETS 1
PROPERTIES ('replication_num' = '1');

SHOW CREATE TABLE hll_hash;

DROP TABLE IF EXISTS quantile_hash;
CREATE TABLE quantile_hash (k INT, s QUANTILE_STATE)
DUPLICATE KEY(k)
DISTRIBUTED BY HASH(s) BUCKETS 1
PROPERTIES ('replication_num' = '1');

SHOW CREATE TABLE quantile_hash;

ADMIN SET FRONTEND CONFIG
('enable_non_aggregate_table_state_types' = 'false');

Actual: both HLL and QUANTILE_STATE tables are created with state-type distribution columns. With the switch disabled, these two DUP definitions are rejected by the aggregate-table-only restriction. Once it is enabled, no independent distribution-type check rejects them.

Anything Else?

This admission gap is specific to DUPLICATE KEY tables because they allow non-key columns as distribution columns. AGGREGATE KEY and UNIQUE KEY tables require distribution columns to be key columns, and BITMAP, HLL and QUANTILE_STATE are already prohibited as key columns. Those existing restrictions prevent these types from being used for distribution in AGGREGATE KEY and UNIQUE KEY tables, whereas DUPLICATE KEY tables require an explicit distribution-type check.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct