Visitar URL original
ci: run CodSpeed benchmarks on macro runners in walltime mode by ellie · Pull Request #3786 · atuinsh/atuin · GitHub
Skip to content

ci: run CodSpeed benchmarks on macro runners in walltime mode - #3786

Merged
ellie merged 1 commit into
mainfrom
ellie/codspeed-macro-benchmarks
Jul 28, 2026
Merged

ellie merged 1 commit into
mainfrom
ellie/codspeed-macro-benchmarks

Conversation

@ellie

@ellie ellie commented Jul 28, 2026 •

Copy link
Copy Markdown
Member

Gives us a reliable machine to run benchmarks on, with no architecture changes happening randomly and producing noise

Checks

  • I am happy for maintainers to push small adjustments to this PR, to speed up the review cycle
  • I have checked that there are no existing pull requests for the same thing

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Switches CodSpeed benchmarks from simulation mode on Depot runners to walltime mode on CodSpeed macro runners.

  • Restores the default benchmark scales.
  • Keeps build and execution modes consistently configured for walltime.

Confidence Score: 5/5

The PR appears safe to merge.

The walltime build and action modes are aligned with the macro runner, and restoring the larger default scale removes a simulation-specific limitation.

Important Files Changed

Filename Overview
.github/workflows/codspeed.yml Updates the runner and CodSpeed mode consistently; no concrete workflow defect identified.

Reviews (1): Last reviewed commit: "ci: run CodSpeed benchmarks on macro run..." | Re-trigger Greptile

@codspeed

codspeed Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

🆕 51 new benchmarks
⏩ 43 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 WallTime deserialize[1] N/A 1.9 µs N/A
🆕 WallTime deserialize[100] N/A 81.1 µs N/A
🆕 WallTime serialize[1] N/A 1.9 µs N/A
🆕 WallTime serialize[100] N/A 109.3 µs N/A
🆕 WallTime reorder_fuzzy_bench[10] N/A 7.9 µs N/A
🆕 WallTime reorder_fuzzy_bench[1000] N/A 710.4 µs N/A
🆕 WallTime reorder_fuzzy_bench[200] N/A 126.1 µs N/A
🆕 WallTime decrypt[1] N/A 18.7 µs N/A
🆕 WallTime decrypt[100] N/A 1.7 ms N/A
🆕 WallTime encrypt[1] N/A 22.4 µs N/A
🆕 WallTime encrypt[100] N/A 2 ms N/A
🆕 WallTime push_batch[1] N/A 425.9 µs N/A
🆕 WallTime push_batch[10] N/A 890.9 µs N/A
🆕 WallTime push_batch[100] N/A 5.3 ms N/A
🆕 WallTime smart_sort[100] N/A 41.3 µs N/A
🆕 WallTime smart_sort[10000] N/A 10.3 ms N/A
🆕 WallTime smart_sort[1600] N/A 1.2 ms N/A
🆕 WallTime smart_sort[200] N/A 98 µs N/A
🆕 WallTime smart_sort[400] N/A 220.6 µs N/A
🆕 WallTime smart_sort[800] N/A 817.6 µs N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing ellie/codspeed-macro-benchmarks (a2c6372) with main (827a43d)

Open in CodSpeed

Footnotes

  1. 43 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@ellie
ellie merged commit 11d0bc3 into main Jul 28, 2026
30 checks passed
@ellie
ellie deleted the ellie/codspeed-macro-benchmarks branch July 28, 2026 22:24
ellie added a commit that referenced this pull request Jul 30, 2026
### Bug Fixes

- *(config)* Preserve comments in `atuin config set` (#3766)
- Fix atuin-search-bench/benches/search.rs (#3788)
- Close OSC133 C when launching Atuin AI to prevent content erasure
(#3794)

### Documentation

- Clarify update check network, provide build command (#3767)
- Remove old i18n docs (#3787)

### Features

- *(config)* Validate new settings in `atuin config set` before writing
them to disk (#3769)
- Remove word with ALT-Backspace ALT-D (#2457)

### Miscellaneous Tasks

- *(ci)* Fix `package` step (#3770)
- Remove Store trait (#3779)
- Daemon search benchmark (divan + CodSpeed) (#3763)
- Run CodSpeed benchmarks on macro runners in walltime mode (#3786)
- Add `OrFilter` as a replacement for `Vec`s as filter lists (#3768)
- Add `atuin-domain` crate for shared domain types (#3790)
- Run CodSpeed walltime benchmarks on depot-macos-26 (#3791)

### Testing

- Adopt rstest across the workspace test suite (#3785)

### Daemon

- Replace nucleo with frizbee, rank by match quality with frecency
tiebreak (#3782)
ellie added a commit that referenced this pull request Aug 3, 2026
### Bug Fixes

- *(client)* Avoid panic decoding an empty encryption key
([#3803](#3803))
- *(client)* Keep sqlx-postgres out of the client crate
([#2884](#2884))
([#3823](#3823))
- *(config)* Preserve comments in `atuin config set`
([#3766](#3766))
- *(search)* Correct fuzzy result ranking
([#3603](#3603))
([#3624](#3624))
- Fix atuin-search-bench/benches/search.rs
([#3788](#3788))
- Close OSC133 C when launching Atuin AI to prevent content erasure
([#3794](#3794))
- Authenticate beta release push as atuin-bot
([#3809](#3809))
- Copy vendored axoasset into docker builder stage
([#3816](#3816))


### Documentation

- Clarify update check network, provide build command
([#3767](#3767))
- Remove old i18n docs
([#3787](#3787))
- Use shell-agnostic install command in quickstart pages
([#3820](#3820))
- Document named stats periods (today/week/month/year)
([#3821](#3821))
- State minimum supported Rust version in install guide
([#3822](#3822))
- Document iTerm2 tmux popup incompatibility
([#3826](#3826))
- Clarify key binding snippets belong in shell startup file
([#3837](#3837))
- Mention RustRover in IDE integration guidance
([#3834](#3834))


### Features

- *(client)* Show intent in inspector
([#3804](#3804))
- *(client)* Enable pty-proxy via config
([#3828](#3828))
- *(config)* Validate new settings in `atuin config set` before writing
them to disk ([#3769](#3769))
- *(lab)* Experimental `atuin lab share`
- *(search)* Remove the "skim" engine
([#3825](#3825))
- Remove word with ALT-Backspace ALT-D
([#2457](#2457))
- Add scheduled beta release workflow
([#3805](#3805))
- Migrate TLS backend from rustls to native-tls
([#3807](#3807))
- Make `update` a real subcommand
([#3813](#3813))
- Capability negotiation protocol (client + server)
([#3797](#3797))


### Miscellaneous Tasks

- *(ci)* Fix `package` step
([#3770](#3770))
- Remove Store trait
([#3779](#3779))
- Daemon search benchmark (divan + CodSpeed)
([#3763](#3763))
- Run CodSpeed benchmarks on macro runners in walltime mode
([#3786](#3786))
- Add `OrFilter` as a replacement for `Vec`s as filter lists
([#3768](#3768))
- Add `atuin-domain` crate for shared domain types
([#3790](#3790))
- Run CodSpeed walltime benchmarks on depot-macos-26
([#3791](#3791))
- Fix deny.toml and get it to run in CI
([#3808](#3808))
- Remove codspeed workflow
([#3812](#3812))
- Richer logging for atuin-server
([#2564](#2564))
- Hide --active sharing
([#3846](#3846))


### Refactor

- *(deps)* Bump thiserror from 2.0.18 to 2.0.19
([#3839](#3839))
- *(deps)* Bump strum_macros from 0.27.2 to 0.28.0
([#3841](#3841))
- *(deps)* Bump tonic-types from 0.14.5 to 0.14.6
([#3842](#3842))
- *(deps)* Bump pulldown-cmark from 0.13.3 to 0.13.4
([#3840](#3840))
- *(deps)* Bump itertools from 0.14.0 to 0.15.0
([#3838](#3838))
- Adopt enum_dispatch for enum trait delegation
([#3806](#3806))


### Testing

- Adopt rstest across the workspace test suite
([#3785](#3785))


### Daemon

- Replace nucleo with frizbee, rank by match quality with frecency
tiebreak ([#3782](#3782))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant