ViroReact 2.57.0 - Object detection, Expo 56 support, improved depth
This release adds on-device object detection, improves AR depth accuracy on iOS, fixes layered glTF/GLB animation playback, aligns the OpenXR loader to the Android 16 KB page-size requirement, and adds Expo SDK 56 support.
ViroObjectDetector
A new component that runs YOLOE object detection through ONNX Runtime on the device. It can open its own camera (CameraX on Android, AVCaptureSession on iOS) or share the AR session when you set useARSession. Detections are reported through onDetection with a label, confidence, and normalized bounding box.
In AR mode each detection also includes a screenBoundingBox in density-independent points, aligned to the on-screen camera preview, so it maps directly onto an absolutely-positioned overlay without extra math. Detection and the 2D overlay work the same on iOS and Android. The 3D worldPosition raycast is currently iOS only.
Inference runs in the companion package @reactvision/react-viro-onnx, which uses the NNAPI execution provider with FP16 on Android. The detector has no built-in inference path: if the provider is not installed, it produces no detections and calls onError.
npm install @reactvision/react-viro @reactvision/react-viro-onnx
{ "expo": { "plugins": ["@reactvision/react-viro", "@reactvision/react-viro-onnx"] } }
Setup, model bundling, and the coordinate system are documented in ViroObjectDetector.
AR depth on iOS
Depth points are now read directly from the AR depth map instead of being approximated, and the monocular depth model is warmed up before its first use. This removes the inaccurate and late depth values that previously showed up on the first frames and improves occlusion and hit-testing on devices without LiDAR.
ViroARScene gains an onDepthReady event on iOS and Android. It fires once, when AR depth first becomes available, so you can wait for it before enabling features that depend on depth.
Layered GLB animations (VIRO-5741)
glTF/GLB skeletal clips that mixed STEP and LINEAR interpolation, or whose channels used several independent time-grids, were being dropped or flattened and froze instead of playing. This is common in Blender exports with layered animations. The loader now resamples all of a skin's channels onto one common time-grid and merges each joint's channels into a single index-aligned keyframe animation, with a per-frame density cap so skinning cost stays bounded on large clips. These animations now play to the end.
OpenXR loader 16 KB page-size alignment
The bundled OpenXR loader library, libopenxr_loader.so, is now aligned to a 16 KB page size (the loader was updated from 1.1.38 to 1.1.49). This native .so previously used 4 KB alignment, which fails the 16 KB memory-page requirement for Android 15 and later and blocks Google Play and Meta Quest Store submission. Apps that do not use XR are not affected.
Expo SDK 56
The config plugin and prebuilt artifacts build against Expo 56 and React Native's new architecture.
Other fixes
- VR controller input works again after moving the VR event listener to the new architecture.
onDraginStudioSceneNavigatornow fires.- visionOS-only sources are excluded from the iOS CocoaPods build, which fixes compile errors in iOS-only targets.
Experimental
- Initial visionOS renderer: a Metal-based renderer and driver, the React Native bindings, and the renderer bridge. This is still in progress and not ready for production.
Upgrading
npm install @reactvision/[email protected]
# For ViroObjectDetector:
npm install @reactvision/react-viro-onnx
npx expo prebuild
There are no breaking API changes. If you use ViroObjectDetector, bundle your .onnx model into the native project as described in the component docs. Note that expo prebuild --clean removes manually placed assets.