Embodied Control (Addon)

The embodied-control addon provides a programmatic user simulator for XR Blocks scenes. It can drive camera locomotion, left and right hand motion, hand poses, and WebXR-like select gestures through a local step API.

Use it directly for tests and samples that need deterministic movement without manual simulator controls.

---

Local Setup

import * as xb from 'xrblocks';
import {EmbodiedControl} from 'xrblocks/addons/embodied-control/index.js';

const embodied = new EmbodiedControl();

xb.add(embodied);
await xb.init();

embodied.applyControl({
  rightHand: {visible: true},
});

await embodied.step({
  durationMs: 250,
  control: {
    locomotion: {move: [0, 0, -0.25]},
    rightHand: {move: [0, 0, -0.12]},
  },
});

applyControl() applies the control immediately and leaves the normal frame loop alone. Use it in live apps, scripts, and demos where XR Blocks should keep rendering normally.

step() applies a control over a duration, advances the core frame loop, and resolves when movement is complete. By default the addon pauses the core after initialization, so frames advance only when step() is called. For visible demos, pass realTime: true so steps yield between frames and animate smoothly.

---

Compound Controls

Both applyControl() and step() use the same compound control schema. A control may combine locomotion and both hands:

await embodied.step({
  durationMs: 500,
  control: {
    locomotion: {
      move: [0, 0, -0.2],
      rotate: [0, 15, 0],
    },
    rightHand: {
      move: [0, 0, -0.18],
      selectStart: true,
    },
    leftHand: {
      move: [-0.05, 0, 0],
    },
  },
});

Locomotion move is [strafeMeters, riseMeters, forwardMeters], relative to the camera. Locomotion rotate is [pitchDegrees, yawDegrees, rollDegrees].

Hand move and rotate are relative to the simulator controller pose. Hand rotations can apply sparse simulator joint rotations in radians.

Use selectStart and selectEnd for pinch-like primary actions. In the simulator these call the semantic pinching APIs, which emit normal XR Blocks selectstart and selectend events. Raw pinching rotations only change the visual pose; they do not trigger selection.

---

High-Level Actions

EmbodiedControl also exposes target-aware movement helpers:

await embodied.teleportTo(cube, {distance: 1.2, faceTarget: true});
await embodied.lookAtTarget(cube, {velocity: 1.5});
await embodied.pointTo(1, cube, {velocity: 1.5});
await embodied.reachTo(1, cube, {velocity: 0.5});
await embodied.click(1);

All high-level methods return Promise<void> and resolve when movement or the gesture sequence is complete.

• teleportTo(target, options): Teleports the camera to coordinates or facing an object. Options: distance (default 1.5m), faceTarget (default true), and snapToGround (default false).
• lookAtTarget(target, options): Rotates the camera to look at the target. Options: velocity in radians/second; if omitted, snaps instantly in 1 frame.
• pointTo(handIndex, target, options): Rotates the controller locally in camera space to point directly at the target. Options: velocity in radians/second; if omitted, snaps instantly in 1 frame.
• reachTo(handIndex, target, options): Moves the controller position toward the target. Options: velocity in meters/second; if omitted, moves instantly.
• click(handIndex, options): Simulates click gesture press and release. Options: durationMs (default 200ms).

---

Local Tests

Await the control action, then assert against normal app state:

await embodied.pointTo(1, button);
await embodied.click(1);

expect(game.score).toBe(1);

Only one step may run at a time. If a second step is requested while another is active, EmbodiedControlBusyError is thrown.

---

Live App Control

For a normally running app, disable autoPause and call applyControl() from your script, UI, or scheduler:

const embodied = new EmbodiedControl({
  autoPause: false,
});

embodied.applyControl({
  locomotion: {rotate: [0, 10, 0]},
  rightHand: {selectStart: true},
});

applyControl() does not call core.stepFrame() and resolves no action result.

---

Sample

The sidebar-driven local demo lives at

samples/embodied_control/main.js. It disables manual simulator controls and runs locomotion, hand motion, and pinch-select steps through EmbodiedControl.

For source-level details, see

src/addons/embodied-control/README.md.