How to install depth models with ViroReact on iOS

Monocular depth estimation enables AR occlusion and world-mesh generation on devices without LiDAR, using a CoreML model that infers depth from the camera image.

iOS only. The monocular path is iOS-exclusive. On Android, depth is provided by the ARCore Depth API.

Requirements: @reactvision/react-viro 3.0.2 (or later 3.x), iOS 14.0+, A12 Bionic or newer (Neural Engine).

How the SDK loads the model

VROMonocularDepthEstimator ships compiled inside ViroKit.framework, but the CoreML model file is not bundled. The framework only includes the body-mesh models (bodymesh.mlmodelc, hourglass_*.mlmodelc). The depth model must be downloaded and added to the app bundle separately.

At runtime, VROARSessioniOS searches for the model in two locations, in order: the framework bundle ([NSBundle bundleForClass:...]), then the app's main bundle ([NSBundle mainBundle]). It walks the following candidate list and uses the first .mlmodelc it finds:

NSArray<NSString *> *candidates = @[
  @"DepthAnythingV2_metric_indoor",  // DAv2 metric, Hypersim (recommended for indoor AR)
  @"DepthAnythingV2_metric_outdoor", // DAv2 metric, KITTI-trained
  @"DepthPro",                       // Apple DepthPro metric
  @"DepthAnythingV2",                // DAv2 relative (requires scale calibration)
];
// pathForResource:name ofType:@"mlmodelc"

The recommended model is DepthAnythingV2_metric_indoor (ViT-Small, Hypersim-trained, max_depth = 20 m, ~44 MB, ~90 ms / ~11 fps on A18 Pro). If no model is found, the SDK logs No depth model found in bundle and monocular occlusion stays inactive. No crash.

Download the model

The model ships as a prebuilt .mlmodelc archive. Download and unzip it. Do not recompile:

curl -L -o DepthAnythingV2_metric_indoor.mlmodelc.zip \
  "https://platform.reactvision.xyz/storage/v1/object/public/public_assets/DepthAnythingV2_metric_indoor.mlmodelc.zip"
unzip DepthAnythingV2_metric_indoor.mlmodelc.zip
# Produces: DepthAnythingV2_metric_indoor.mlmodelc/ (directory)

.mlmodelc is a compiled CoreML model stored as a directory, not a single file. Use it as-is.

Add the model to the app bundle

Expo (`expo prebuild`)

The showcase uses Expo (expo prebuild generates the native ios/ project), so the model goes into the app's main bundle:

Copy DepthAnythingV2_metric_indoor.mlmodelc into the generated ios/ folder.
In Xcode, drag the folder into the project and choose Add to target (app target).
Confirm it appears under Build Phases → Copy Bundle Resources.
Leave Copy items if needed unchecked if the file is already inside ios/.

Because .mlmodelc is a directory, Xcode treats it as a folder resource. After expo prebuild --clean this step must be repeated. It is not yet idempotent (candidate for a future config plugin).

Bare React Native

Copy DepthAnythingV2_metric_indoor.mlmodelc into your Xcode project (for example under the app target folder).
Add it to the app target and confirm it appears under Build Phases → Copy Bundle Resources.

Enable monocular depth from JS

Reference: showcase/components/ar-examples/ar-depth-test.tsx. Relevant ViroARSceneNavigator props:

<ViroARSceneNavigator
  autofocus
  worldAlignment="GravityAndHeading"
  occlusionMode="depthBased"     // enable depth-based occlusion
  preferMonocularDepth={true}    // force the ML model even on LiDAR devices
  depthDebugEnabled={false}      // debug: paint depth map full-screen
  monocularDepthScale={1.0}      // lower to 0.7–0.95 if model overestimates
  monocularDepthTargetFPS={5}    // inference rate; higher = more thermal load
  initialScene={{ scene: ARDepthTestScene }}
  style={styles.arView}
/>

Prop	Type	Default	Notes
`preferMonocularDepth`	bool	`false`	Forces the ML model even when LiDAR is present
`occlusionMode`	string	`"disabled"`	`"depthBased"` enables depth occlusion
`depthDebugEnabled`	bool	`false`	Paints the depth map full-screen
`monocularDepthScale`	float	`1.0`	Multiplies depth before occluding. Use `< 1.0` if the model overestimates (objects show through); `> 1.0` if it underestimates. Typical range: 0.7–1.3
`monocularDepthTargetFPS`	int	`5`	Self-throttles by thermal state: Critical→0, Serious→2, Fair→3

AR coordinate convention: in ViroReact, negative Z = in front of the camera. Place occlusion test objects at negative Z (e.g. a sphere at 1.5 m: position={[0, 0, -1.5]}).