Visitar URL original
[visionOS] Shrink input hit-test volume for orbit in connected volumetric scenes by yoelhawa · Pull Request #76302 · WebKit/WebKit · GitHub
Skip to content

[visionOS] Shrink input hit-test volume for orbit in connected volumetric scenes - #76302

Merged
webkit-commit-queue merged 1 commit into
WebKit:mainfrom
yoelhawa:eng/yhawa/cvs-shrink-wrap-input
Oct 10, 2026
Merged

webkit-commit-queue merged 1 commit into
WebKit:mainfrom
yoelhawa:eng/yhawa/cvs-shrink-wrap-input

Conversation

@yoelhawa

@yoelhawa yoelhawa commented Oct 8, 2026 •

Copy link
Copy Markdown

647e75b

[visionOS] Shrink input hit-test volume for orbit in connected volumetric scenes
https://bugs.webkit.org/show_bug.cgi?id=326794
<rdar://189229747>

Reviewed by Mike Wyrzykowski and Etienne Segonzac.

The entity that captures drags in a connected volumetric scene fills the
whole volume, so it consumes taps meant for windows behind the volume,
even where no content is in front of them.

Fit it to the content's bounding sphere instead. Whenever the model process
fits the content to the volume, it sends the sphere through the web process
to the UI process, which places the entity there. Orbiting only rotates the
content within that sphere, so nothing needs to be sent while it moves, and
a refit that leaves the sphere unchanged sends nothing either.

The entity is positioned relative to the RealityView's root, which settles
at the center of the volume's depth after the view renders and moves as the
volume resizes. The content is placed at the same point, so the entity
stays on it.

Drive-by fixes:
- Place the content at the center of the volume's depth, read from the
  scene, instead of estimating the depth from the volume's smaller in-plane
  extent.
- Clip the volume to its bounds, keeping both content and input inside it.

* Source/WebKit/ModelProcess/cocoa/ModelProcessModelPlayerProxy.h:
* Source/WebKit/ModelProcess/cocoa/ModelProcessModelPlayerProxy.mm:
(WebKit::ModelProcessModelPlayerProxy::applyVolumetricPresentationTransform):
Send the fitted content's bounding sphere when it changes.
(WebKit::ModelProcessModelPlayerProxy::exitVolumetricPresentation):
Forget the last sphere sent, so a reopened volume gets one.

* Source/WebKit/WebProcess/Model/ModelProcessModelPlayer.cpp:
(WebKit::ModelProcessModelPlayer::didUpdateVolumetricHitSphere):
* Source/WebKit/WebProcess/Model/ModelProcessModelPlayer.h:
* Source/WebKit/WebProcess/Model/ModelProcessModelPlayer.messages.in:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::updateVolumetricSceneHitSphere):
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/UIProcess/WebPageProxy.messages.in:
* Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::updateVolumetricSceneHitSphere):
* Source/WebKit/UIProcess/Model/PortalPresentationManagerProxy.h:
* Source/WebKit/UIProcess/Model/PortalPresentationManagerProxy.mm:
(WebKit::PortalPresentationManagerProxy::updateVolumetricSceneHitSphere):
Relay the sphere to the volume's scene.

* Source/WebKit/Platform/spi/ios/UIKitSPI.h:
Declare the scene's 3D size and the RealityKit clip.

* Source/WebKit/UIProcess/ios/WKPortalVolumetricSceneController.h:
* Source/WebKit/UIProcess/ios/WKPortalVolumetricSceneController.mm:
(-[WKPortalVolumetricSceneController _applyContentPlacement]):
Center the content in the volume's depth.
(-[WKPortalVolumetricSceneController setHitSphereCenter:radius:]):
(-[WKPortalVolumetricSceneController _applyInputSurfaceHitSphere]):
(-[WKPortalVolumetricSceneController installInputSurfaceWithBegan:changed:ended:]):
(-[WKPortalVolumetricSceneController updateLayoutForVolumeSize]):
(-[WKPortalVolumetricSceneController hostContentWithContext:pid:]):
(-[WKPortalVolumetricSceneController _applyInputSurfaceExtents]): Deleted.
Hand the sphere to the input surface, whether it arrives before or after
the surface is installed.
(-[WKPortalVolumetricSceneController _configureWithScene:]):
Clip the volume to its bounds.

* Source/WebKit/UIProcess/Cocoa/WKPortalVolumetricGestureController.h:
* Source/WebKit/UIProcess/Cocoa/WKPortalVolumetricGestureController.swift:
(WKPortalVolumetricGestureController.setHitSphereCenter(_:radius:)):
(WKPortalVolumetricGestureController.updateProxyExtents(withWidth:height:depth:)): Deleted.
Make the collision shape the sphere, positioned relative to the content root.

Canonical link: https://commits.webkit.org/323303@main

322af67

Misc iOS, visionOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 win
✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug ✅ 🧪 wpe-wk2   🧪 win-tests
✅ 🧪 webkitperl ✅ 🧪 ios-wk2 ✅ 🧪 api-mac ✅ 🧪 api-wpe
✅ 🧪 ios-wk2-wpt   🧪 api-mac-debug
  🧪 api-ios ✅ 🧪 mac-wk2 ✅ 🛠 gtk3-gcc
✅ 🛠 ios-safer-cpp   🧪 mac-AS-debug-wk2 ✅ 🛠 gtk
✅ 🛠 vision ✅ 🧪 gtk-wk2
✅ 🛠 🧪 merge ✅ 🛠 vision-sim   🧪 mac-intel-wk2 ✅ 🧪 api-gtk
✅ 🧪 vision-wk2 ✅ 🛠 mac-safer-cpp ✅ 🛠 playstation
✅ 🛠 tv ✅ 🧪 mac-site-isolation
✅ 🛠 tv-sim
✅ 🛠 watch
✅ 🛠 watch-sim

@yoelhawa yoelhawa self-assigned this Oct 8, 2026
@webkit-early-warning-system

webkit-early-warning-system commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

@yoelhawa yoelhawa added the WebKit Misc. For miscellaneous bugs in the WebKit framework (and not JavaScriptCore or WebCore). label Oct 8, 2026

@etiennesegonzac etiennesegonzac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't think we should introduce a timer for this (and potentially lose interactions right after opening the volume).

We should use the bounding sphere radius (which is constant during orbit), and send this over. Only updating when the contents of the portal change and we re-run auto-fit.
Looks like RealityKit even supports CollisionComponent(shapes [.generateSphere(radius: radius)]).

(Orbit and auto-fit are tied together, you can't orbit if auto is not present in the portal-transform, and CSS transforms apply after all of this, so let's not worry about this for now.)

@yoelhawa
yoelhawa force-pushed the eng/yhawa/cvs-shrink-wrap-input branch from 390b8b6 to 016383a Compare October 9, 2026 00:47
@webkit-early-warning-system

webkit-early-warning-system commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Comment thread Source/WebKit/ModelProcess/cocoa/ModelProcessModelPlayerProxy.mm
Comment thread Source/WebKit/UIProcess/Cocoa/WKPortalVolumetricGestureController.swift Outdated
Comment thread Source/WebKit/UIProcess/Cocoa/WKPortalVolumetricGestureController.swift Outdated
@yoelhawa
yoelhawa force-pushed the eng/yhawa/cvs-shrink-wrap-input branch from 016383a to 322af67 Compare October 9, 2026 23:17
@yoelhawa
yoelhawa requested a review from mwyrzykowski October 9, 2026 23:25
@yoelhawa yoelhawa added the merge-queue Applied to send a pull request to merge-queue label Oct 9, 2026
…tric scenes

https://bugs.webkit.org/show_bug.cgi?id=326794
<rdar://189229747>

Reviewed by Mike Wyrzykowski and Etienne Segonzac.

The entity that captures drags in a connected volumetric scene fills the
whole volume, so it consumes taps meant for windows behind the volume,
even where no content is in front of them.

Fit it to the content's bounding sphere instead. Whenever the model process
fits the content to the volume, it sends the sphere through the web process
to the UI process, which places the entity there. Orbiting only rotates the
content within that sphere, so nothing needs to be sent while it moves, and
a refit that leaves the sphere unchanged sends nothing either.

The entity is positioned relative to the RealityView's root, which settles
at the center of the volume's depth after the view renders and moves as the
volume resizes. The content is placed at the same point, so the entity
stays on it.

Drive-by fixes:
- Place the content at the center of the volume's depth, read from the
  scene, instead of estimating the depth from the volume's smaller in-plane
  extent.
- Clip the volume to its bounds, keeping both content and input inside it.

* Source/WebKit/ModelProcess/cocoa/ModelProcessModelPlayerProxy.h:
* Source/WebKit/ModelProcess/cocoa/ModelProcessModelPlayerProxy.mm:
(WebKit::ModelProcessModelPlayerProxy::applyVolumetricPresentationTransform):
Send the fitted content's bounding sphere when it changes.
(WebKit::ModelProcessModelPlayerProxy::exitVolumetricPresentation):
Forget the last sphere sent, so a reopened volume gets one.

* Source/WebKit/WebProcess/Model/ModelProcessModelPlayer.cpp:
(WebKit::ModelProcessModelPlayer::didUpdateVolumetricHitSphere):
* Source/WebKit/WebProcess/Model/ModelProcessModelPlayer.h:
* Source/WebKit/WebProcess/Model/ModelProcessModelPlayer.messages.in:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::updateVolumetricSceneHitSphere):
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/UIProcess/WebPageProxy.messages.in:
* Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::updateVolumetricSceneHitSphere):
* Source/WebKit/UIProcess/Model/PortalPresentationManagerProxy.h:
* Source/WebKit/UIProcess/Model/PortalPresentationManagerProxy.mm:
(WebKit::PortalPresentationManagerProxy::updateVolumetricSceneHitSphere):
Relay the sphere to the volume's scene.

* Source/WebKit/Platform/spi/ios/UIKitSPI.h:
Declare the scene's 3D size and the RealityKit clip.

* Source/WebKit/UIProcess/ios/WKPortalVolumetricSceneController.h:
* Source/WebKit/UIProcess/ios/WKPortalVolumetricSceneController.mm:
(-[WKPortalVolumetricSceneController _applyContentPlacement]):
Center the content in the volume's depth.
(-[WKPortalVolumetricSceneController setHitSphereCenter:radius:]):
(-[WKPortalVolumetricSceneController _applyInputSurfaceHitSphere]):
(-[WKPortalVolumetricSceneController installInputSurfaceWithBegan:changed:ended:]):
(-[WKPortalVolumetricSceneController updateLayoutForVolumeSize]):
(-[WKPortalVolumetricSceneController hostContentWithContext:pid:]):
(-[WKPortalVolumetricSceneController _applyInputSurfaceExtents]): Deleted.
Hand the sphere to the input surface, whether it arrives before or after
the surface is installed.
(-[WKPortalVolumetricSceneController _configureWithScene:]):
Clip the volume to its bounds.

* Source/WebKit/UIProcess/Cocoa/WKPortalVolumetricGestureController.h:
* Source/WebKit/UIProcess/Cocoa/WKPortalVolumetricGestureController.swift:
(WKPortalVolumetricGestureController.setHitSphereCenter(_:radius:)):
(WKPortalVolumetricGestureController.updateProxyExtents(withWidth:height:depth:)): Deleted.
Make the collision shape the sphere, positioned relative to the content root.

Canonical link: https://commits.webkit.org/323303@main
@webkit-commit-queue
webkit-commit-queue force-pushed the eng/yhawa/cvs-shrink-wrap-input branch from 322af67 to 647e75b Compare October 10, 2026 09:02
@webkit-commit-queue

Copy link
Copy Markdown
Collaborator

Committed 323303@main (647e75b): https://commits.webkit.org/323303@main

Reviewed commits have been landed. Closing PR #76302 and removing active labels.

@webkit-commit-queue
webkit-commit-queue merged commit 647e75b into WebKit:main Oct 10, 2026
@webkit-commit-queue webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

WebKit Misc. For miscellaneous bugs in the WebKit framework (and not JavaScriptCore or WebCore).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants