Visitar URL original
UI: Prefetch Dag, run and task page data in route loaders by safaehar · Pull Request #76978 · apache/airflow · GitHub
Skip to content

UI: Prefetch Dag, run and task page data in route loaders - #76978

Open
safaehar wants to merge 2 commits into
apache:mainfrom
safaehar:ui-route-loader-prefetch
Open

safaehar wants to merge 2 commits into
apache:mainfrom
safaehar:ui-route-loader-prefetch

Conversation

@safaehar

@safaehar safaehar commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

On an initial load, the Dag, run and task pages send their first requests from their components. Those requests wait for the root route loader (the UI config) and the first render. With the JavaScript cached, the page's data requests start 330–460 ms into the load:

Step Ends at
HTML ~35 ms
Main bundle executes ~110 ms
/ui/config (root route loader) ~150–230 ms
First render (one long task, ~120 ms) ~270–360 ms
Page data requests start ~330–460 ms

Requests that need another response wait longer. The task log needs the task instance's try number, so it starts only after the task instance request, which itself waits for the first render.

React Router runs the loaders of all matched routes in parallel. This adds loaders that start those requests while the app loads:

  • dags/:dagId: the Dag, its details and its latest run.
  • dags/:dagId/runs/:runId and the task instance routes (mapped or not): the Dag, its details and the run. The Dag details are the breadcrumb's query, which every run and task page renders.
  • The task instance Logs tab, and the default tab when the user's default task instance tab is Logs: the task instance, then the log of its current try.

The loaders use the generated prefetchUse… / ensureUse…Data helpers, so each query key is the one the page's own query builds, and the page renders from the cache. They don't wait for their requests and never fail, so they don't delay or break rendering. If a page's query ends up with another key, it just sends its own request, as it does today.

What the loaders leave to the page:

  • URLs with try_number, region_id or region_index, the only parameters that change the task instance or log keys. Historical tries and loop iterations keep their current behavior. Other parameters, such as the Logs tab's log_level and log_source filters, keep the prefetch.
  • Tries that have not started (try_number 0, no state, up_for_retry), for which the Logs tab sends no log request either.
  • The Grid's runs: their page size depends on the panel width, which a loader can't know.

Measurements

I measured an equivalent change on an Airflow 3.3 deployment. It had the same loaders, plus a prefetch of the Grid's first page, which this PR doesn't include. The A/B compared the same UI sources built without and with the loaders, with the JavaScript cached, against a real API server. A and B loads alternated, 10–12 rounds per page. Medians in ms from navigation start, p25–p75 in brackets:

Page Request Request done End of the first request burst
Dag Dag details 520 → 352 [320–404] 762 → 708 [678–815]
Run Dag details (breadcrumb) 588 → 344 [335–407] 1,028 → 958 [908–1,071]
Task Dag details (breadcrumb) 550 → 342 [317–419] 1,152 → 908 [866–1,047]
Task log 1,271 → 1,017 [863–1,324] —
  • The task page gains most, ~240 ms at the end of its first burst: its log no longer waits for the first render and then the task instance.
  • The run page's load ends with the Grid's TI summaries, which the Grid only requests once it renders, so it gains less (~70 ms).
  • There were no duplicate requests for the Dag, its details or the run, so every prefetched key matched the page's.
  • On the task page, the task instance is sometimes fetched twice (1 to 3 loads in 10–12), because its query uses staleTime: 0 and refetches on mount when the loader's request has already finished.

An earlier version of this description showed larger run- and task-page gains (−221 ms and −460 ms). They came from a non-interleaved A/B, where the API server's latency drifted between the A and B halves. The table above replaces them.

Testing

  • routePrefetch.test.ts (13 tests), covering:
    • the cached keys equal the hooks' keys, including the Dag details on run pages;
    • the log uses the task instance's try and loop coordinates;
    • the map index from the route;
    • URLs with try_number or loop coordinates are skipped;
    • URLs with only the Logs tab's filters keep the prefetch;
    • tries that haven't started are skipped;
    • the default-tab setting;
    • a failed task instance request is swallowed.
  • The full UI test suite passes (pnpm test), along with pnpm build, oxfmt, oxlint, eslint and tsc. On the latest commit, one unrelated test, UserSettingsButton "counts the teams it does not list…", failed once in a full run with a socket hang-up; it passes when run alone.

No visual change: the pages render the same content, earlier.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 5.5) following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

On an initial load, these pages sent their first requests from their
components, so the requests waited for the UI config (the root route
loader) and the first render. The task log waited longer: it needs the
task instance's try number, so it started after the task instance request
that itself waited for the first render.

React Router runs the loaders of all matched routes in parallel, so route
loaders now start these requests while the app loads, with the generated
prefetch helpers and the same query keys the pages use. The loaders do not
wait for their requests and never fail, so they do not delay or break
rendering.
…arams

The breadcrumb on every run and task page requests the Dag details, but the Dag loader is on a sibling route, so that request still waited for the first render. The log prefetch was skipped for any query string, although only try_number, region_id and region_index change its keys, so filtered log links lost it.
@bbovenzi

Copy link
Copy Markdown
Contributor

Let's wait until #76091 is merged. I feel like we'll need a refactor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. need workflow approval

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants