Visitar URL original
DiskManager: include the description in file names by Tpt · Pull Request #26110 · apache/datafusion · GitHub
Skip to content

DiskManager: include the description in file names - #26110

Open
Tpt wants to merge 4 commits into
apache:mainfrom
Tpt:tpt/tmp-file-name
Open

Tpt wants to merge 4 commits into
apache:mainfrom
Tpt:tpt/tmp-file-name

Conversation

@Tpt

@Tpt Tpt commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Ease debugging: the file names are now .tmp.{description}.{randomness} instead of .tmp{randomness}

This PR only changes the temporary file names, a test is extended to ensure that escaping works properly.

Ease debugging: the file names are now `.tmp.{description}.{randomness}` instead of `.tmp{randomness}`
@github-actions github-actions Bot added the execution Related to the execution crate label Oct 7, 2026
@codecov-commenter

codecov-commenter commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.43590% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 82.77%. Comparing base (7833125) to head (e4bf444).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/execution/src/disk_manager.rs 97.43% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26110      +/-   ##
==========================================
- Coverage   82.77%   82.77%   -0.01%     
==========================================
  Files        1147     1147              
  Lines      450904   450941      +37     
  Branches   450904   450941      +37     
==========================================
+ Hits       373250   373280      +30     
- Misses      54931    54936       +5     
- Partials    22723    22725       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me -- thank you @Tpt

/// A `.tmp.` is added at the beginning and a last `.` is set at the end.
///
/// For example "This kind.of /input" becomes ".tmp.this_kind_of_input."
fn escape_for_temp_file_name_prefix(text: &str) -> String {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should also potentially cap the length of text too? like to 20 characters?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Definitely. 20 characters seems a bit small to me. Still quite short descriptions like sort_merge_join_buffered_spill will be truncated. What about something larger like 128 to avoid extremely long file names will still keeping the full description in most case?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

128 sounds good to me

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Will implement that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done f3033c9

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 thank you

@alamb
alamb added this pull request to the merge queue Oct 9, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 9, 2026
@Tpt

Tpt commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

@alamb Thank you so much for your time reviewing this MR. Sadly merge train has failed. I have rebased the MR using the GitHub button, may you try merging again 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

execution Related to the execution crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants