Visitar URL original
fix(vmm): commit virtio-mem block state after the KVM slot update by not4s · Pull Request #6176 · firecracker-microvm/firecracker · GitHub
Skip to content

fix(vmm): commit virtio-mem block state after the KVM slot update - #6176

Merged
not4s merged 2 commits into
mainfrom
fix/virtio-mem-kvm-first
Sep 2, 2026
Merged

not4s merged 2 commits into
mainfrom
fix/virtio-mem-kvm-first

Conversation

@not4s

@not4s not4s commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Changes

  • VirtioMem::update_range now applies a plug/unplug request one KVM slot at a time. For each slot it updates KVM first and only then commits that slot's block state (plugged_blocks, plugged_size), discarding an unplugged slot after it is committed.
  • GuestRegionMmapExt::update_slot commits its per-slot bitmap only after the KVM_SET_USER_MEMORY_REGION update succeeds (and restores the slot's protection if the update fails).
  • Added a unit test that plugs a range spanning two KVM slots with the second slot's memory-region update forced to fail, asserting only the first slot's block stays committed. A test-only fault injector on set_user_memory_region drives the failure.

Reason

update_range committed the block state for the whole request up front and only then updated the KVM slots, and update_slot flipped its per-slot bitmap before issuing the KVM ioctl. On a multi-slot plug/unplug that failed part way through (e.g. a slot update returning an error), the committed block state covered the entire range while KVM reflected only the slots that had actually been updated.

Committing each slot's state only after its KVM update succeeds keeps the block state and the KVM slots consistent on a partial failure: the operation stops with both reflecting exactly the slots that were updated, and a block is never discarded before it is committed as unplugged. The whole-range discard is now done per slot as each commits, which issues one discard per intersecting slot instead of one for the range.

  1. Why not rollback:
  • On a partial failure the operation stops and returns an error, leaving the block state and the KVM slots consistent with each other, rather than rolling back the slots that already succeeded. Rolling back would require re-issuing KVM slot updates, which allocate per-slot metadata and can themselves fail, most likely under the same resource pressure that caused the original failure, with no reliable way to recover if the unwind fails. Committing each slot's state only after its KVM update succeeds keeps the two consistent by construction.
  1. Scope/spec note:
  • The virtio-mem specification does not define behavior for a request that fails mid-execution, so this is a robustness fix. A slot update only fails under resource pressure (for example, hitting the KVM memory-slot limit), not on a normal request.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkbuild --all to verify that the PR passes
    build checks on all supported architectures.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

ghost commented Sep 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.56522% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.08%. Comparing base (a6146c8) to head (39eca9d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/vmm/src/vstate/memory.rs 84.21% 3 Missing ⚠️
src/vmm/src/devices/virtio/mem/device.rs 96.49% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6176      +/-   ##
==========================================
+ Coverage   83.06%   83.08%   +0.01%     
==========================================
  Files         277      277              
  Lines       31049    31106      +57     
==========================================
+ Hits        25792    25845      +53     
- Misses       5257     5261       +4     
Flag Coverage Δ
5.10-m5n.metal 83.36% <94.56%> (+0.01%) ⬆️
5.10-m6a.metal 82.73% <94.56%> (+0.02%) ⬆️
5.10-m6g.metal 80.21% <94.56%> (+0.02%) ⬆️
5.10-m6i.metal 83.36% <94.56%> (+0.01%) ⬆️
5.10-m7a.metal-48xl 82.72% <94.56%> (+0.02%) ⬆️
5.10-m7g.metal 80.21% <94.56%> (+0.02%) ⬆️
5.10-m7i.metal-24xl 83.33% <94.56%> (+0.01%) ⬆️
5.10-m7i.metal-48xl 83.33% <94.56%> (+0.01%) ⬆️
5.10-m8g.metal-24xl 80.21% <94.56%> (+0.03%) ⬆️
5.10-m8g.metal-48xl 80.21% <94.56%> (+0.03%) ⬆️
5.10-m8i.metal-48xl 83.33% <94.56%> (+0.01%) ⬆️
5.10-m8i.metal-96xl 83.33% <94.56%> (+0.01%) ⬆️
5.10-m9g.metal-48xl 80.21% <94.56%> (+0.02%) ⬆️
6.1-m5n.metal 83.38% <94.56%> (+0.01%) ⬆️
6.1-m6a.metal 82.74% <94.56%> (+0.01%) ⬆️
6.1-m6g.metal 80.21% <94.56%> (+0.02%) ⬆️
6.1-m6i.metal 83.38% <94.56%> (+0.01%) ⬆️
6.1-m7a.metal-48xl 82.74% <94.56%> (+0.02%) ⬆️
6.1-m7g.metal 80.21% <94.56%> (+0.02%) ⬆️
6.1-m7i.metal-24xl 83.39% <94.56%> (+0.01%) ⬆️
6.1-m7i.metal-48xl 83.39% <94.56%> (+0.01%) ⬆️
6.1-m8g.metal-24xl 80.20% <94.56%> (+0.01%) ⬆️
6.1-m8g.metal-48xl 80.21% <94.56%> (+0.03%) ⬆️
6.1-m8i.metal-48xl 83.39% <94.56%> (+0.01%) ⬆️
6.1-m8i.metal-96xl 83.39% <94.56%> (+0.01%) ⬆️
6.1-m9g.metal-48xl 80.21% <94.56%> (+0.02%) ⬆️
6.18-m5n.metal 83.38% <94.56%> (+0.02%) ⬆️
6.18-m6a.metal 82.74% <94.56%> (+0.01%) ⬆️
6.18-m6g.metal 80.23% <94.56%> (+0.02%) ⬆️
6.18-m6i.metal 83.37% <94.56%> (+0.01%) ⬆️
6.18-m7a.metal-48xl 82.74% <94.56%> (+0.02%) ⬆️
6.18-m7g.metal 80.23% <94.56%> (+0.02%) ⬆️
6.18-m7i.metal-24xl 83.39% <94.56%> (+0.02%) ⬆️
6.18-m7i.metal-48xl 83.40% <94.56%> (+0.01%) ⬆️
6.18-m8g.metal-24xl 80.23% <94.56%> (+0.02%) ⬆️
6.18-m8g.metal-48xl 80.23% <94.56%> (+0.03%) ⬆️
6.18-m8i.metal-48xl 83.39% <94.56%> (+0.01%) ⬆️
6.18-m8i.metal-96xl 83.39% <94.56%> (+0.01%) ⬆️
6.18-m9g.metal-48xl 80.23% <94.56%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ghost
ghost force-pushed the fix/virtio-mem-kvm-first branch 2 times, most recently from b4f338b to 53e07bb Compare September 1, 2026 16:47
@ghost
ghost changed the base branch from fix/virtio-mem-discard-upstream to main September 1, 2026 16:47
@ghost
ghost force-pushed the fix/virtio-mem-kvm-first branch 2 times, most recently from 5e5e032 to abb8638 Compare September 1, 2026 17:04
@ghost ghost added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Sep 1, 2026
@ghost
ghost marked this pull request as ready for review September 1, 2026 17:16
Comment thread src/vmm/src/vstate/memory.rs Resolved
Comment thread src/vmm/src/vstate/memory.rs Resolved
@ghost
ghost force-pushed the fix/virtio-mem-kvm-first branch 2 times, most recently from feef684 to 83b241f Compare September 2, 2026 11:41
Jay Chung added 2 commits September 2, 2026 11:50
update_range committed the block state (plugged_blocks, plugged_size)
for the whole request up front and only then updated the KVM slots, and
update_slot flipped its per-slot bitmap before issuing the KVM ioctl. On
a multi-slot request that failed part way through, the committed state
covered the whole range while KVM reflected only the slots that had been
updated, leaving the device's bookkeeping inconsistent with KVM.

Update KVM one slot at a time and commit that slot's block state only
after its KVM update succeeds; discard an unplugged slot only once it
is committed. A partial failure now stops with the block state and the
KVM slots reflecting exactly the slots that were updated, and never
discards a block that has not been committed as unplugged.

In update_slot, commit the slot bitmap only once both the protection
change and the KVM slot update have succeeded, and roll back the first
of the two when the second fails, so a failed slot update leaves the
slot exactly as it was. A failed rollback would leave the host mapping
and KVM out of sync with no way back, so it panics.

The whole-range discard is now done per slot as each slot commits. This
issues one discard per intersecting slot instead of one for the range,
which is not on a hot path.

Add a test that plugs a range spanning two KVM slots with the second
slot's memory-region update forced to fail, and checks that only the
first slot's block is left committed. A test-only fault injector on
set_user_memory_region drives the failure.

Signed-off-by: Jay Chung <jaehoc@amazon.com>
Record that virtio-mem now commits a slot's block state only after its
KVM update succeeds, keeping the device's bookkeeping consistent with
KVM when a plug or unplug request fails part way through.

Signed-off-by: Jay Chung <jaehoc@amazon.com>
@ghost
ghost force-pushed the fix/virtio-mem-kvm-first branch from 83b241f to 39eca9d Compare September 2, 2026 11:52
@ghost
ghost added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 9384f39 Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants