Skip to main content

Desktop simulator

The simulator runs XR Blocks applications in a normal desktop browser. It uses the same scripts, targeting, selection, and world APIs as a real XR session. It does not emulate the browser WebXR API.

Start the simulator​

Show the simulator option in the XR button:

const options = new xb.Options();
options.xrButton.showEnterSimulatorButton = true;
await xb.init(options);

Autostart it in code or by URL:

options.formFactor = 'desktop';
http://127.0.0.1:8080/templates/14_simulator_setup/?formFactor=desktop

The simulator runtime is loaded only when startup is requested. On a desktop autostart path, await xb.init(options) waits for that load. Applications can also call:

await xb.init(options);
const simulator = await xb.core.startSimulator();

xb.core.simulator is undefined before startup completes. Use onSimulatorStarted() when a script must react to simulator startup.

Browser interface​

The settings, instructions, and hand-pose panels register automatically when Lit is available. No simulator add-on import is required. CDN applications need these mappings:

{
  "imports": {
    "lit": "https://esm.sh/lit@3.3.1",
    "lit/": "https://esm.sh/lit@3.3.1/"
  }
}

Disable panels before initialization when an application needs a clean canvas:

options.simulator.simulatorSettingsPanel.enabled = false;
options.simulator.instructions.enabled = false;
options.simulator.handPosePanel.enabled = false;

Control modes​

XR Blocks supplies five simulator modes:

  • SimulatorMode.USER: user movement and mouse selection.
  • SimulatorMode.POSE: navigation plus virtual hand-pose controls.
  • SimulatorMode.CONTROLLER: drive an active virtual hand or controller.
  • SimulatorMode.POINTER_LOCK: first-person pointer-lock navigation.
  • SimulatorMode.EDITOR: simulator scene editing.
options.simulator.defaultMode = xb.SimulatorMode.CONTROLLER;
options.simulator.modeToggle.enabled = true;

When mode toggling is enabled, the configured toggle key cycles through options.simulator.modeToggle.toggleOrder.

Automation preset​

Use the shared preset for automated or external browser runs:

const options = new xb.Options().enableAutomationMode({
  hideSimulatorUi: true,
  defaultMode: xb.SimulatorMode.POSE,
  enableHands: true,
  enableCamera: true,
});
await xb.init(options);

?xrAutomation=1 applies the default preset. ?debug=1 separately exposes window.xb and the initialization promise window.xbReady.

Scene environments​

Each environment points to one JSON manifest:

options.simulator.environments = [
  {name: 'Evaluation Room', manifestPath: './evaluation-room.json'},
];
options.simulator.activeEnvironmentIndex = 0;
evaluation-room.json
{
  "scenePath": "./room.glb",
  "scenePlanesPath": "./room-planes.json",
  "navMeshPath": "./room-navmesh.glb",
  "position": [0, 0, 0],
  "locations": {
    "wall-primary": {
      "description": "Clear wall point for ray interaction.",
      "position": [0, 1.4, -2]
    }
  },
  "objects": [
    {
      "id": "chair",
      "assetPath": "./chair.glb",
      "position": [0.5, 0, -1.5],
      "physics": "fixed",
      "detectObject": true,
      "label": "chair"
    }
  ]
}

Use scenePath or videoPath, not both. Paths can be manifest-relative, root-relative, or absolute URLs. Object physics can be false, fixed, or dynamic.

Load another environment at runtime with:

await xb.core.simulator.setEnvironment('Ad Hoc Room', './ad-hoc-room.json');

Read the optional named world-space locations from the active environment:

const locations = xb.core.simulator.getLocations();
const wall = locations['wall-primary'];
console.log(wall.description, wall.position);
options.simulator.navMesh.enabled = true;
options.simulator.navMesh.showDebugVisualizations = true;

The navmesh constrains and grounds the simulated user. It does not constrain hands or controllers. CDN applications that enable it need the three-pathfinding mapping shown in rollup.config.js and templates/14_simulator_setup/index.html.

Runtime physical objects​

const [chair] = await xb.core.simulator.objects.addObjects([
  {
    assetPath: './chair.glb',
    physics: 'fixed',
    detectObject: true,
    label: 'chair',
  },
]);

xb.core.simulator.objects.get([chair.id]);
xb.core.simulator.objects.removeObjects([chair.id]);
xb.core.simulator.objects.clear();

Use simulator ground truth through the normal world detector API:

options.enableObjectDetection();
options.world.objects.simulatorOverride = true;

const objects = await xb.world.objects.runDetection();

The override applies only in the simulator. Real XR sessions use the configured detector backend.

Hand reach and physical hands​

options.simulator.reachDistance.enabled = true;
options.simulator.reachDistance.radius = 0.75;
options.simulator.reachAngle.enabled = true;
options.simulator.reachAngle.angle = Math.PI;

options.simulator.handPhysics.enabled = true;

Physical hands require Rapier. leftHandOrigin and rightHandOrigin are camera-local shoulder anchors for reach limits and the physics tether. Set options.simulator.physics.enabled = false to disable simulator-owned collision without disabling the application's Rapier world.

See templates/14_simulator_setup, demos/sim_hand_poses, and the API reference for SimulatorOptions.