Repository navigation
[fix](fe) Expose Paimon FileIO access failures during catalog creation - #68766
Open
suxiaogang223 wants to merge 1 commit into
Open
suxiaogang223 wants to merge 1 commit into
suxiaogang223 wants to merge 1 commit into
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
run buildall |
2 similar comments
Member
Author
|
run buildall |
Contributor
|
run buildall |
Contributor
TPC-H: Total hot run time: 29079 ms |
Contributor
TPC-DS: Total hot run time: 151741 ms |
Contributor
ClickBench: Total hot run time: 24.43 s |
Contributor
FE UT Coverage ReportIncrement line coverage `` 🎉 |
suxiaogang223
force-pushed
the
codex/paimon-fileio-error-details
branch
from
October 10, 2026 09:50
df19b59 to
bf64853
Compare
Member
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29112 ms |
Contributor
TPC-H: Total hot run time: 28636 ms |
Contributor
TPC-DS: Total hot run time: 151954 ms |
Contributor
ClickBench: Total hot run time: 24.38 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
924060929
approved these changes
Oct 12, 2026
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Paimon FileIO.get reports failed filesystem access probes as UnsupportedSchemeException and attaches the actual failures as suppressed exceptions. Doris previously displayed only the scheme-resolution message, so an OBS timeout looked like a missing FileIO implementation. Include all failed access attempts in the catalog-creation error and use the first access failure as its cause, allowing root-cause consumers such as SHOW CATALOGS to expose the actual error. Preserve the original aggregate as a suppressed exception for complete diagnostics. Errors without failed FileIO probes retain their existing wrapping. Add offline tests exercising the real Hadoop fallback and covering nested errors, multiple attempts, missing implementations, and unrelated errors. ### Release note Paimon catalog initialization now reports the underlying filesystem access failures instead of only a misleading missing-FileIO message. ### Check List (For Author) - Test: Paimon connector module package with 657 unit tests (656 passed, one live-connectivity test skipped); six new tests passed; Checkstyle and connector import gate passed. Used Thrift compiler 0.24.0 through doris.thrift.executable. No live OBS regression run. - Behavior changed: Yes, catalog error messages and their diagnostic cause chains expose FileIO access failures. Storage access and retry behavior are unchanged. - Does this need documentation: No
suxiaogang223
force-pushed
the
codex/paimon-fileio-error-details
branch
from
October 12, 2026 02:19
bf64853 to
e9fae44
Compare
Member
Author
|
run buildall |
Member
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review conclusion: no substantiated new inline finding for PR #68766 at e9fae44. No existing inline P0/P1 thread applies. This review is complete and static only; I did not build, run tests, or modify source.
Critical checkpoints:
- Goal and evidence: the changed catch exposes Paimon FileIO probe failures; the new helper promotes the suppressed access error so FE root-cause consumers, including SHOW CATALOGS, can display it. Six added tests cover a simulated real Hadoop/OBS fallback and synthetic exception shapes. Their source was inspected, not executed in this runner; no live OBS result was verified.
- Scope: one catch-site change, one small helper, and focused tests. The change matches adjacent RuntimeException wrapping while retaining the original aggregate as a suppressed stack.
- Concurrency: ensureCatalog still uses volatile publication and synchronized lazy creation. Failed creation leaves the catalog unset for a later retry. No thread, lock, or lock order is added; failure-only formatting is small within the existing initialization lock.
- Lifecycle: the existing finally block restores the thread context classloader on success and failure; the helper does not alter catalog ownership or close behavior.
- Configuration: no new item or dynamic setting is introduced.
- Compatibility: no SPI, FE/BE protocol, persisted format, thrift, or protobuf definition changes. The intended user-visible error message and cause chain change is limited to the matched FileIO exception case.
- Parallel paths: filesystem, JDBC, HMS, REST, and DLF creation share the catch boundary. FileIO resolution is reached by the applicable filesystem/JDBC/HMS paths; other failures retain ordinary wrapping.
- Conditional logic: only an UnsupportedSchemeException with suppressed failures is promoted. The no-suppressed and unrelated-exception cases retain the original cause; tests cover both.
- Tests and results: the six new offline test cases have relevant assertions for message, root cause, preserved aggregate, and classloader restoration. This invocation performed no build or test; the PR author's passing-test claim was not treated as runtime proof.
- Observability: the top-level message lists attempted failures and resolution context; the full stack retains the aggregate. The FE deferred-error path stores the promoted root cause for SHOW CATALOGS.
- Transactions, persistence, and writes: no transaction, edit-log, data-write, or crash-recovery path changes.
- FE/BE variables: none added or forwarded.
- Performance: work is limited to failed catalog creation and the number of FileIO probes; no successful hot path changes.
- Other risks: inspected Paimon 1.4.2 loader ordering and Doris packaging. Supported object-store paths have a single Hadoop failure, so the first-suppressed choice does not mask a later timeout. No reachable retry or type classifier needs the original aggregate in the primary cause chain.
User focus: no additional focus was provided. All three required review passes returned NO_NEW_VALUABLE_FINDINGS; the final changed-file and candidate sweep found no remaining issue.
Contributor
TPC-H: Total hot run time: 28937 ms |
Contributor
TPC-DS: Total hot run time: 152265 ms |
Contributor
ClickBench: Total hot run time: 24.44 s |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
An OBS access timeout during Paimon catalog initialization can be reported as
Could not find a file io implementation for scheme 'obs'. Paimon'sFileIO.getstores the failed access probes as suppressed exceptions on an
UnsupportedSchemeException, while Doris previously displayed only its message.Expose the underlying access failures in the catalog-creation error and use the
first failed probe as the cause so root-cause consumers, including
SHOW CATALOGS,can report the actual error. Retain the original aggregate as a suppressed
exception, preserving every attempted FileIO failure in the diagnostic stack.
Other catalog errors retain their existing wrapping.
Add six offline tests covering the real Paimon/Hadoop fallback with a simulated
OBS timeout, nested wrappers, multiple failures, errors without messages,
missing filesystem implementations, and unrelated suppressed exceptions.
Release note
Paimon catalog initialization reports underlying filesystem access failures
instead of only a misleading missing-FileIO message.
Check List (For Author)
Test
Offline Paimon connector unit tests, Checkstyle, and plugin packaging passed.
No live OBS regression was run.
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)