Repository navigation
[Bug] Value-only format table silently drops partitions whose value starts with '_' or '.' #10239
Description
Activity
Hi @LuciferYang, thanks for the excellent write-up — the root cause analysis around the write/read asymmetry in PartitionPathUtils.validatePartitionValueForPath (value-only layout silently dropping _/. prefixed partitions) is really clear.I noticed the issue is marked with "I'm willing to submit a PR", but there's no associated PR or assignment yet. May I ask whether you've had a chance to start working on the fix? If you're already on it, no worries at all — please disregard this. But if you haven't started yet, I'd be glad to pick it up and submit a PR. Just let me know and I can go ahead.Either way, happy to help get this resolved. Thanks!
@suntectec feel free to submit pr
Search before asking
Paimon version
master
Compute Engine
Spark / Flink (format table)
Minimal reproduce step
In a value-only partition layout (
formatTablePartitionOnlyValueInPath=true) the write validator inPartitionPathUtils.validatePartitionValueForPathrejects only null, empty, "." and "..". A partition value such as_xor.tmpis therefore written to a directory whose name starts with "_" or ".".What doesn't meet your expectations?
The read path treats any value-only directory whose name starts with "_" or "." as a hidden staging or metadata directory (
PartitionPathUtils.isHiddenFile/isHiddenName) and skips it, exempting only the table's configured default partition name. The write and read sides disagree, so the_xpartition and all of its data silently disappear from every scan, with no error at write or read time.Expected: a value that cannot be represented as a visible value-only directory is rejected at write time, so the failure is loud instead of silent. The configured default partition name stays allowed because scans un-hide exactly that name.
Anything else?
No response
Are you willing to submit a PR?