Hire a ReactVision Expert 🛠️Launch Studio 🖥️Sponsor on GitHub ❤️
HomeGuidesAPI ReferenceChangelogDiscussions
Hire a ReactVision Expert 🛠️Launch Studio 🖥️Sponsor on GitHub ❤️Log In
Changelog
Back to All
added

ViroReact 3.1.0: Advanced VPS, a world mesh that persists and headset-native Quest scenes

about 5 hours ago by ReadMe API

Advanced VPS arrives: scan a place with a phone, then find it again and pin content to it, matched on the device or on the GPU server. Scans can carry the device's depth, so blank walls reconstruct. The world mesh now fuses depth across frames into a room that persists, and Meta Quest gets headset-native Editor scenes with the wearer's real hands showing through virtual content. Renderer binaries are rebuilt on virocore 3.1.0, released with ReactVision Spatial 1.4.0. The web build uses @reactvision/viro-web-renderer 1.1.0. Read Check when upgrading before you update.

Advanced VPS

  • <ViroVPS> finds a scanned place and keeps tracking it. startTracking(id) takes a known location, "auto" searches by GPS and "visual" by camera image. Results come through onLocalized and onTrackingState.
  • Two ways to localise, behind one prop. localization="device" (the default) downloads the place's map and matches on the phone. localization="server" sends camera frames to the GPU server, which holds up much better across days and lighting and needs no map download (paid plans, billed as Advanced VPS compute). StudioSceneNavigator takes the same choice as vpsLocalization, with onVpsStateChange and retryVps().
  • <ViroVPSAnchor> puts content in the place's frame, and the scanned mesh gives occlusion and physics. Both follow the map every frame.
  • Scan from your app. uploadScanRecording(target, outputDir, options?) zips a recording natively and uploads it, carrying on in the background on iOS and Android. meshQuality (low, medium, high) picks the mesh's detail, keepRecording keeps the recording for merging scans, and getScan(scanId) follows processing. Each scan reports quality.mesh_confidence.level (good, fair or poor) with reasons for whoever recorded it.
  • Record depth while scanning (iOS with LiDAR, Android with the ARCore Depth API): startRecording(outputDir, { depth: true }), with depthCaptureEnabled set on the navigator when it mounts. stopRecording() now resolves to a ViroRecordingResult, and isRecordingSupported({ depth }) says what the device can do.
  • Help whoever is recording: useViroTrackingReady and useViroCaptureGuidance (hints such as turning too fast or pointing at a blank wall), and onTrackingUpdated on ViroARSceneNavigator.
  • Locations and users: listLocations, createLocation, renameLocation, deleteLocation, listScans, deleteScan and findNearbyLocations. createUserTokenCredentials() lets an app act for its signed-in user, with no secret key on the device.
  • No set-up beyond your keys: VPS calls need nothing beyond the Editor session or the Expo plugin's rvApiKey and rvProjectId, and requests time out instead of hanging.

See Advanced VPS and Scanning, ViroVPS and ViroVPSAnchor.

World mesh

  • The mesh fuses depth frames into a room that persists on the depth-image and monocular paths. worldMeshConfig takes accumulate, voxelSize and maxMemoryMB, sceneNavigator.resetWorldMesh() starts a new room, and getWorldMeshStats() reports source (lidar, depth, monocular, plane) and accumulated.
  • Android: the mesh is in the right place, and confidence weights samples instead of dropping them. iOS without LiDAR: estimated depth is placed where it was seen, so fusing it no longer duplicates surfaces.
  • The world mesh types are exported from the package root, and the ViroWorldMeshConfig reference now matches the type that ships. See ViroARSceneNavigator.

Meta Quest

  • Editor scenes feel native in the headset. The headset view opens at once with a loading panel, the menu opens on the left controller's menu button (or a palm pinch) instead of Y, B closes it, and the panels are a quarter of the size.
  • The room stays put. Content on room surfaces, tap-placed models and the co-location frame keep their place when the wearer recentres, and onRoomMoved(listener) reports a recentre to your app.
  • Input behaves like the platform's. Y goes back as B does, only a trigger or pinch selects, and ViroController.getControllerRayAsync(source) says where a ray points.
  • Hand occlusion in passthrough: handOcclusion on ViroXRSceneNavigator and ViroVRSceneNavigator (off by default; handOcclusionMode "mesh" or "capsules"). questPermissions picks the runtime permissions asked for at launch.
  • Expo plugin: Quest builds declare minSdkVersion 32, the Horizon OS SDK as <metavr:uses-metavr-sdk>, a network security config and Meta's recommended activity settings, and no longer declare NFC.

See HorizonOS Setup Guide.

Web

  • Web AR content stays in the room on iPhone, and hides while tracking is lost. Set hideContentWhenLost: false in arOptions to keep it visible. Skinned GLBs draw. Needs viro-web-renderer 1.1.0. See Web Setup Guide.

visionOS

  • Editor scenes render in the ImmersiveSpace. They were blank before, because their root was a ViroARScene, which renders nothing on visionOS.

iOS Simulator

  • Apps using ViroReact build and run in the iOS Simulator. ViroKit and the bridge ship as ViroKit.xcframework and ViroReact.xcframework, each with a simulator slice (arm64 and x86_64). AR scenes still need a device.

Everywhere

  • Every scene navigator sends one usage ping when it mounts, replacing the device registration ReactVision Spatial used to do. It carries no API key. Turn it off with ViroTelemetry.setEnabled(false) or the Expo plugin's telemetry: false. The iOS privacy manifest now ships with the pod. See Usage Data.
  • Co-location: getColocationDeviceId(), credentials for user tokens, and joins refused for the team's compute billing say so (COMPUTE_SPEND_LIMIT_REACHED, COMPUTE_BILLING_REFUSED).
  • Viro3DObject reads more of glTF: KHR_texture_transform, KHR_node_visibility, KHR_animation_pointer, extra texture coordinate sets and emissive textures, with fixes for opaque, masked and blended materials.
  • A click goes to what is drawn on top, and a click on nothing reaches ViroController's onClick again. Text sits where its vertical alignment puts it.
  • Types: registerAnimations accepts names and nested chains, Viro360Image takes stereoMode (steroMode is a deprecated alias), and ViroMaterial types colorWriteMask as an array.
  • ViroKit targets iOS 15.1, the minimum the podspec already declared.

Check when upgrading

  • Emission now shows on physically based materials. A model that relied on it being dropped may now glow.
  • A style on a scene navigator now applies. Check the layout of any navigator you pass one.
  • cloudAnchorProvider and geospatialAnchorProvider are honoured when provider is not set. cloudAnchorProvider="arcore" now uses ARCore Cloud Anchors and needs a Google Cloud API key. Remove the prop, or pass provider, to keep ReactVision's.
  • The iOS binaries moved to ios/dist/ViroRenderer/ViroKit.xcframework and ios/dist/lib/ViroReact.xcframework. Update any script or Podfile that names ViroKit.framework or libViroReact.a by path, and run expo prebuild (or pod install) again.
  • Text moves to where its vertical alignment puts it. Drop any offset you added to make up for the old position.
  • Quest: the menu button and the palm pinch no longer exit a scene (handle source 13 in a ViroController if you want them to), and the manifest changes above apply after prebuild.
  • Recording audio, corrected: the 3.0.3 notes said session recordings carry an iOS audio track and need RECORD_AUDIO on Android. Both apply to startVideoRecording, not startRecording.

Every change in 3.1.0

This is the whole of the 3.1.0 section of CHANGELOG.md, shortened.

Added

Advanced VPS

  • <ViroVPS localization="server"> sends a camera frame every serverFixIntervalMs (default 1000) to the VPS server and fuses its answers into tracking, then one every serverRefreshIntervalMs (default 10000) once converged. <ViroVPSAnchor>, the world mesh and onLocalized work as in device mode.
  • StudioSceneNavigator takes vpsLocalization ("device" or "server"); a change restarts tracking of the scene on screen.
  • VPS calls accept a user token (ViroVPS, ViroVPSClient, ViroVPSScanUpload): an app with its own sign-in acts for its signed-in user, with no secret key on the device.
  • createUserTokenCredentials({ projectId, fetchToken }) keeps a user token between requests, refreshes it before it expires, shares one fetch between concurrent requests and replaces a refused token once.
  • uploadScanRecording takes meshQuality ("low", "medium" or "high") and onProgress(sentBytes, totalBytes). ViroVPSMeshQuality is exported.
  • The rest of the VPS device routes: listLocations, getLocation, createLocation, renameLocation, deleteLocation, listScans, deleteScan, getVPSServerAddress, getLocationFileDownload(locationId, kind) and getScanFileDownload(scanId, kind).
  • ViroVPSErrorCodes, the codes ViroReact itself gives a ViroVPSError.
  • Capture guidance for recording a place: createViroCaptureGuidance and useViroCaptureGuidance (hints fast, pitch, close and blank, and a count of tracking jumps), and useViroTrackingReady(state, { normalMs, limitedMs }).
  • startRecording(outputDir, { depth: true }) records the device's depth, with depthCaptureEnabled on ViroARSceneNavigator (iOS with LiDAR, Android with the ARCore Depth API).
  • stopRecording() resolves to a ViroRecordingResult, and isRecordingSupported({ depth }) is on the AR scene navigator.
  • onTrackingUpdated on ViroARSceneNavigator, and ViroARTrackingReasonConstants.TRACKING_REASON_INSUFFICIENT_LIGHT (4; Android only).

World mesh

  • sceneNavigator.resetWorldMesh(), and worldMeshConfig takes accumulate, voxelSize (default 0.04 m) and maxMemoryMB (default 64).
  • getWorldMeshStats() reports source ("lidar", "depth", "monocular", "plane" or "unknown") and accumulated.
  • The world mesh types are exported from the package root.

Co-location

  • getColocationDeviceId(); the replication socket sends it as x-rv-device, so a device is billed once across its two sockets.
  • Co-location takes credentials, as <ViroVPS credentials> does, so a device can join with a user token minted with the colocation scope (needs the rebuilt native binaries).
  • A join refused for the team's compute billing says so: COMPUTE_SPEND_LIMIT_REACHED or COMPUTE_BILLING_REFUSED (StudioColocationErrorCode).

Meta Quest

  • Hand occlusion in passthrough: handOcclusion, handOcclusionMode ("mesh" or "capsules") and handOcclusionInflateMm on ViroVRSceneNavigator and ViroXRSceneNavigator.
  • onRoomMoved(listener) reports a recentre, or a headset waking, as a 4x4 move.
  • ViroController.getControllerRayAsync(source) says where a ray last pointed.
  • The Editor scene menu highlights the item a controller or hand points at.

Everywhere

  • Every scene navigator sends one usage ping when it mounts. Turn it off with ViroTelemetry.setEnabled(false) or the Expo plugin's telemetry: false. See Usage Data.
  • The iOS privacy manifest ships with the pod (ViroReact_privacy).
  • iOS Simulator builds (ViroKit.xcframework and ViroReact.xcframework, arm64 and x86_64 simulator slices).
  • hideContentWhenLost in web AR options (viro-web-renderer 1.1.0).
  • Viro3DObject reads glTF KHR_texture_transform, KHR_node_visibility and KHR_animation_pointer, texture coordinate sets beyond the first, and emissive textures.

Changed

Advanced VPS

  • VPS calls go to ReactVision Spatial and need no address or credentials beyond the Editor session or rvApiKey and rvProjectId. Only image search and server-mode frames go to the VPS server, whose address ReactVision Spatial gives out (GET /vps/server).
  • uploadScanRecording sends the recording straight to storage and carries on in the background (iOS background URLSession, Android WorkManager). ReactVision Spatial keeps the recording only while it processes the scan, unless keepRecording is true.
  • ViroVPSScan.quality is typed and carries mesh_confidence (good, fair or poor, with reasons). ViroVPSScan also types outputs, keep_recording, app_user_id, queued_at, started_at, finished_at and mesh_quality, and ViroVPSMapDownload types geo. Code that builds ViroVPSScan objects by hand must add the first six.
  • uploadScanRecording reads credentials again just before its final call, so a credentials function is called twice per upload.

Meta Quest

  • The Editor scene menu opens on the left controller's menu button or a palm pinch, not Y. B closes it while it is open, and B dismisses an ALERT before it exits the scene.
  • The menu button and the palm pinch no longer exit a scene. They reach a ViroController's onClickState and onClick as source 13. Y goes back as B does.
  • Editor scene panels are drawn at a quarter of their size, look like Meta Horizon OS panels, and open centred where the wearer looks.
  • Editor scenes with plane detection use the room's planes (Space Setup) when the wearer has granted spatial data.
  • ViroXRSceneNavigator takes questPermissions. StudioSceneNavigator asks for no permissions at launch, and opens its headset view at once with a loading panel.
  • A co-location failure is explained by its error code in the indicator and the Quest HUD.
  • Expo plugin, Quest builds: minSdkVersion 32 and installLocation="auto", the Horizon OS SDK as <metavr:uses-metavr-sdk>, a network security config, excludeFromRecents and the 2D category on the launching activity, Meta's recommended configChanges, and no NFC.

Everywhere

  • minConfidence is documented per platform: a cutoff on iOS, a fusion weight on Android.
  • Calls that do nothing on a platform say so once (resetARSession and setWorldOrigin on Android, ViroARObjectMarker on Android and Quest, scene navigation in the web navigators). depthDebugEnabled is iOS only.
  • The scene navigators apply the style prop. Check the layout of any navigator you pass one.
  • ViroARSceneNavigator honours cloudAnchorProvider and geospatialAnchorProvider when provider is not set.
  • Android: startVideoRecording fails with error 1 when RECORD_AUDIO is denied, takeScreenshot keeps only the last path segment of its file name, and projectPoint / unprojectPoint on a stale view reject.
  • The iOS binaries are xcframeworks (ios/dist/ViroRenderer/ViroKit.xcframework, ios/dist/lib/ViroReact.xcframework), and the Expo plugin no longer excludes arm64 from simulator builds.
  • Scenes rendered by ViroARSceneNavigator remount once after upgrading.

Fixed

glTF and materials (virocore 3.1.0)

  • A GLB with animation pointers loads, and so does one whose samplers omit interpolation.
  • Emission shows on physically based materials, visionOS included.
  • Opaque glTF materials are opaque, masked ones are opaque where they pass the cutoff, and blended ones no longer hide what is behind them.
  • GLB characters whose rig animates a node above each joint move, skinned parts no longer vanish up close, a GLB with 8-bit indices draws, and skinned GLBs draw on the web.

Editor scenes

  • Editor scenes render on visionOS, in the ImmersiveSpace.
  • A scene stops downloading a map that keeps failing (four retries, then error until retryVps()).
  • An animation can follow one that started before its model loaded.
  • On Meta Quest: tap-placed models and the co-location frame stay in the room when the wearer recentres; only a trigger or pinch selects; a placement aimed at nothing lands along the controller's ray, and one aimed through a panel's text lands where it points; panels draw over scene objects in front of them, and a click between menu lines no longer reaches what is behind; long ALERTs fit their text and the view; wrapped lines no longer overlap; "No assets to display" appears in front of the wearer; content on a plane faces the wearer; the spatial data and microphone prompts open over the headset view; a failed photo, recording or scene change is reported in the headset; image-triggered assets that Quest cannot show are announced; text assets are sharp; an asset that fails to load no longer freezes the app.

Meta Quest renderer

  • Videos no longer abort the app or leak, pause when the headset sleeps, and pause while Horizon OS has input focus.
  • Content on a room surface stays on it after a recentre, and recenterTracking() no longer races the renderer.
  • A plane scene opened right after spatial data is granted gets its planes, and uses the plane nearest the wearer. A desk is classified as a table.
  • Hand input releases and presses once, a scene no longer crashes when its root changes as the headset view starts, and closing a scene logs no OpenXR errors.
  • The right eye clears its depth buffer, each eye's framebuffer is made once, and depth buffers are no longer written back to memory.
  • A top-level node in a ViroARScene no longer asks for a world anchor, switching AR scenes no longer logs a passthrough error, and AR hit tests no longer keep references to their results.

Rendering and input

  • A click goes to what is drawn on top, and a controller click on nothing reaches ViroController's onClick again.
  • Text no longer disappears after a front-culled material, and sits where its vertical alignment puts it.

iOS

  • iOS: maxMemoryMB: 0 and voxelSize: 0 keep the current value.
  • ViroKit.framework targets iOS 15.1, so apps targeting iOS 15 launch on iOS 15 devices.
  • A paused video no longer starts again on its own, and a closing scene no longer risks a crash from its videos' late callbacks.
  • ViroOrbitCamera orbits its focalPoint when it also has a position.

Android

  • Going back to a scene keeps its ViroController, and dropping a Viro view no longer logs an error.
  • A paused-then-played video keeps playing, and play and pause reach the player in the order they were made.
  • Stopping a scan recording no longer logs signalEndOfInputStream failed.
  • Hit tests always settle, and getRecordingStatus() returns "Unsupported" when the AR view cannot be found.

VPS and co-location

  • A VPS error the gateway answers itself keeps its code, and a VPS request with no answer fails with NETWORK_ERROR after 30 s (longer for downloads).
  • The app's own credentials go only to https or the local network (ENDPOINT_NOT_ALLOWED otherwise), on iOS and Android too.
  • uploadScanRecording() reports a native failure as a ViroVPSError and discards the zip, and a lost connection gives NETWORK_ERROR.
  • One failed poll no longer ends <ViroVPS> tracking.
  • The co-location hooks no longer stay working or throw on a native failure, room requests time out after 30 s, and a device id that failed once is asked for again.
  • <ViroSharedFrame> and <ViroARCloudAnchor> survive a source change. <ViroARCloudAnchor> takes onLocalizeProgress and maxAttempts.
  • ViroReplicationClient operations return whether they were sent.
  • rvCreateSharedFrame and rvJoinSharedFrame resolve { success: false } outside Quest.

API and types

  • Viro360Image takes stereoMode (steroMode is a deprecated alias), and sounds seek to fractional seconds.
  • ViroMaterial types colorWriteMask as an array, and ViroMaterials.createMaterials() tolerates a missing texture or semanticMask labels.
  • ViroAnimationDict types animation chains, and a color inside a chain is processed.
  • ViroARScene sets its view's ref, as ViroScene does.
  • Navigation: replace("key") on the top scene no longer throws, scene state no longer leaks between scenes on pop or replace, the navigators no longer modify viroAppProps, a scene's passProps cannot replace arSceneNavigator, and two pop() calls in one tick no longer pop the root scene.
  • Deprecation warnings are logged once.
  • The Expo plugin can run again over existing android/ and ios/ folders, and adds ARCore to the app's <queries> instead of replacing them.
  • react is a peer dependency only.
  • iOS and Android bridge hardening: downloads fail on a non-2xx status, credentials go only to https or a local host, promises settle on every path, and R8 keep rules ship with the library.

Full list: Upgrading to 3.1.0, the GitHub release and CHANGELOG.md.