Spatial UI

XR Blocks includes one built-in spatial UI system. It provides flex layout, world-space cards, view-space overlays, text, images, icons, buttons, sliders, themes, validation, and model presentation.

Import UI components from xrblocks. The renderer starts with XR Blocks when a UI root enters the scene. Applications do not enable UI, initialize UIKit, register a renderer, or install a second UI raycaster.

Choose the root by coordinate space

Type	Coordinate space	Size and position	Use for
UICard	World space	size and object transforms use meters	Menus, tools, labels, model controls, and movable surfaces in the scene
UIOverlay	View space	Layout is relative to a private full-viewport container	HUDs, status, instructions, and view-fixed controls
UIPanel	Its UI parent	Descendant UIKit layout units	Nested rows, columns, sections, and visual grouping

UIPanel is not a scene root. Put it inside a UICard or UIOverlay. Use FollowHead on a card when content must remain a real world-space object that follows the viewer. Use an overlay when it must remain fixed to the 2D viewport.

World-space cards

import * as xb from 'xrblocks';

const card = new xb.UICard({
  size: {width: 0.6, height: 'auto'},
  manipulation: true,
  edge: true,
  style: {
    flexDirection: 'column',
    gap: 16,
    padding: 24,
    backgroundColor: '#202124',
    borderRadius: 24,
  },
});
card.position.set(0, 1.4, -1);

card.add(
  new xb.UIText({
    text: 'Welcome',
    style: {fontSize: 32, color: '#ffffff'},
  }),
  new xb.UIButton({
    label: 'Continue',
    onClick: () => console.log('Continue'),
  })
);

xb.add(card);
await xb.init(new xb.Options());

manipulation: true enables the card's standard face-camera move, scale, and corner resize behavior. Moving also scales the card with its distance from the viewer the way Android XR panels do, and a controller thumbstick pushes the card away or pulls it closer while you drag it. Moves stay between 0.75 and 5 meters from the viewer, Android XR's panel limits. These come only with manipulation: true. When you pass your own actions, opt in with translate: {scaleWithDistance: true, pushPull: true, minDistance: 0.75, maxDistance: 5}.

edge: true adds the outer hit band and requires translation or resize to be enabled. Two-source scale uses the card's scale action; the edge does not enable a missing action. An edge with only resize enabled resizes from its corners and does not move the card.

When the card's resize action is enabled, the corners of the edge band resize the card instead of moving it, like Android XR and Quest panels. Point and pinch with a ray, or pinch a corner directly with your hand. Resizing changes size in meters and reflows the layout; it does not scale the content. Resizing a card with height: 'auto' switches it to a fixed height, starting from its laid-out height.

By default a card never gets narrower than its content allows, such as its longest word or a row of buttons, and never shorter than its content needs at the current width, so narrowing a card makes it taller as text wraps. To let a card get smaller than its content, put the scrollable content in a UIScrollView (or set an explicit minSize.height to override the content floor):

const card = new xb.UICard({
  // A scroll view has no natural height, so use a fixed starting height.
  size: {width: 0.56, height: 0.6},
  edge: true,
  manipulation: true,
  children: [
    header,
    new xb.UIScrollView({
      style: {flexGrow: 1, flexBasis: 0, minHeight: 160},
      children: [body],
    }),
  ],
});

const card = new xb.UICard({
  size: {width: 0.6, height: 0.4},
  edge: true,
  manipulation: {
    actions: {
      translate: true,
      resize: {
        anchor: 'center', // or 'opposite' to keep the far corner in place
        preserveAspectRatio: false,
        minSize: {width: 0.3, height: 0.2},
        maxSize: {width: 1.2, height: 0.8},
      },
    },
  },
});

anchor: 'center' is the default and grows the card around its center. minSize defaults to 0.1 meters per axis, and maxSize is unbounded. When maxSize is smaller than the content needs, maxSize wins over the content floors, and a card with preserveAspectRatio keeps its proportions.

Use height: 'auto' when the card should fit its children, gaps, and padding. Keep the width fixed so text wrapping and percentage-width children have a stable horizontal constraint. Use a numeric height when the surface must stay at a fixed physical size.

View-space overlays

An overlay's world transform does not control its rendered position. Layout it inside the view-space container:

const overlay = new xb.UIOverlay({
  style: {
    width: 360,
    position: 'absolute',
    left: '50%',
    bottom: 24,
    transform: {translateX: '-50%'},
  },
  children: [new xb.UIText({text: 'Ready', style: {fontSize: 24}})],
});

xb.add(overlay);

Cards and overlays use the theme's surface appearance by default. Set appearance: 'none' for a transparent root used only for layout. UIPanel is a neutral nested element unless its style or theme role gives it an appearance.

Layout units and text

UI values have two separate unit systems:

Property	Meaning
Fixed UICard.size values and world transforms	Meters
UICard.size.height: 'auto'	Height calculated from child layout
Descendant numeric width, height, gap, padding, margin, and font size	UIKit layout units
Percentage strings	Percentage of the applicable parent layout size
auto	Content or flex layout chooses the size where supported
Numeric lineHeight	Multiplier of fontSize, like CSS unitless line height
lineHeight: 'Npx'	Explicit UIKit pixel value
lineHeight: 'N%'	Percentage of fontSize

These produce different line spacing:

new xb.UIText({
  text: 'Compact\nmultiline text',
  style: {fontSize: 24, lineHeight: 1.2}, // 1.2 * fontSize
});

new xb.UIText({
  text: 'Fixed\nmultiline text',
  style: {fontSize: 24, lineHeight: '32px'},
});

Use whiteSpace: 'pre-line' to preserve explicit line breaks. Combine a fixed height, bottom alignment, and clipped overflow for a newest-first transcript:

const transcript = new xb.UIText({
  text: 'You: Hello\n\nGemini: 你好',
  style: {
    width: '100%',
    height: 240,
    whiteSpace: 'pre-line',
    verticalAlign: 'bottom',
    overflow: 'hidden',
  },
});

UIText uses a system-font canvas when the default atlas lacks a glyph. The mounted UI element remains stable when text changes between native and Unicode rendering.

Semantic controls and telemetry

Use semantic controls instead of raw pointer listeners:

const status = new xb.UIText({text: 'Ready'});
const slider = new xb.UISlider({
  ariaLabel: 'Volume',
  min: 0,
  max: 1,
  step: 0.05,
  value: 0.5,
  onInput: (value) => (status.text = `Volume ${value.toFixed(2)}`),
  onChange: (value) => saveVolume(value),
});

onInput reports live captured changes. onChange reports one completed changed interaction. A canceled slider restores its starting value.

Application status, diagnostics, and telemetry are also UI semantics. Use a mounted UIText, UIPanel, or disabled control instead of drawing text into a canvas sprite:

const diagnostics = new xb.UIText({
  text: 'Tracking: waiting',
  style: {fontSize: 18, color: '#fbbc04'},
});

function showTrackingState(state) {
  diagnostics.text = `Tracking: ${state}`;
  diagnostics.style.color = state === 'ready' ? '#34a853' : '#fbbc04';
}

This keeps status readable, layout-aware, themeable, and available to the same interaction and scene-context systems as application UI.

Scrolling and editable text

UIScrollView is a nested vertical viewport, not a new scene root. Its children are ordinary retained UI elements. Use a finite viewport height and keep long rows from shrinking:

const history = new xb.UIScrollView({
  ariaLabel: 'History',
  style: {height: 240, gap: 12},
  children: [
    new xb.UIText({
      text: 'A longer entry...',
      style: {fontSize: 24, flexShrink: 0},
    }),
  ],
});
card.add(history);

scrollTop, clientHeight, scrollHeight, and maxScrollTop use UI layout units, not world meters. scrollTo(offset) clamps a requested offset; scrollBy(delta) returns whether a measured viewport moved. reveal(child) minimally reveals a descendant and requires a mounted layout. Read ready before operations that require measured layout.

Wheel input, ray dragging, and direct-hand dragging share the normal interaction pipeline. Dragging ordinary content past the movement threshold cancels its pending click. A slider or text selection keeps its own capture. Nested scrolling tries the nearest eligible field or viewport first; reaching the outer boundary does not turn the gesture into card scaling. Content outside the viewport is excluded from new pointer hits and visible-object context.

UITextInput supports single-line and multiline plain text:

const message = new xb.UITextInput({
  ariaLabel: 'Message',
  multiline: true,
  placeholder: 'Write a message',
  style: {height: 120, fontSize: 24},
  onInput: (value) => showDraftLength(value.length),
  onSubmit: (value) => addLocalMessage(value),
});
card.add(message);

Multiline mode is chosen at construction. Enter submits a single-line field and inserts a newline in a multiline field; Ctrl/Meta+Enter submits multiline text. IME composition must complete before Enter can submit. Submitting does not clear the field or call an AI provider automatically.

Native input/textarea editing owns the value, selection, clipboard, and composition behavior. onInput reports live user edits, onChange reports a changed editing session when focus ends, and onSubmit is a separate action. Programmatic value changes do not emit a user-input callback. Escape blurs without reverting entered text. A read-only field permits focus/selection/copying; a disabled field does not accept focus or edits.

Use focus() only when the field is ready. Removing, hiding, or disabling a field releases its editing ownership. Keyboard editing must not move the simulator camera. Operating-system keyboards and clipboard permissions remain browser-dependent, especially inside immersive sessions.

Editable text loads a private canvas-based presentation on demand and uses system fonts, following the existing Unicode UI rendering approach. It adds no dependency, font download, worker, or import-map entry. Ordinary UI does not download this chunk. Deploy the complete build/ directory so the private chunk remains available.

Canvas 2D and browser text layout provide the glyphs, caret, and selection geometry. Glyph availability and appearance depend on the device's system fonts. Read field.error for a reported module or rendering failure. A ready text field is not proof of native keyboard availability.

The existing keyboard addon can bind to a field:

import {Keyboard} from 'xrblocks/addons/virtualkeyboard/index.js';

const keyboard = new Keyboard({input: message});
card.add(keyboard);

Bound keys edit the field's caret/selection and preserve its focus. Reassign keyboard.input from another field's onFocus callback to share one keyboard. Use keyboard.open to show or hide the panel and update native-keyboard ownership immediately. While the keyboard is connected and visible, it requests that the bound field's browser software keyboard remain hidden; closing or detaching it restores normal behavior. The keyboard's unbound value/submit mode remains available. Other application-owned accessory controls can set object.xb.preserveTextFocus = true when interacting with them should not blur a field.

Custom keyboard integrations can use field.suppressNativeKeyboard(), which returns a release function. The request is scoped and reference-counted rather than a permanent field setting. It leaves native editing enabled and uses browser software-keyboard hints; it is not a guarantee that every headset runtime honors those hints.

Known Meta Quest limitation: the system keyboard may reopen or close while the in-panel keyboard is used. Prefer the native Quest keyboard and leave the panel keyboard closed when they conflict. The demo retains its keyboard controls for explicit testing; there is no automatic cross-browser detection of native-keyboard availability.

Text fields expose their accessible label and editing state to scene context, but not their draft value by default. An application can explicitly provide field.userData.semantic.text when it intends to disclose a value. This is not screenshot redaction: rendered text can still appear in captured images.

See samples/spatial_forms for scrolling history, both field modes, keyboard binding, and card/overlay composition without an AI backend. Rich text, general horizontal scroll views, and list virtualization are not included.

The larger demos/spatial_ui_lab gallery includes a local conversation, a searchable pattern library, multilingual notes, and editable/read-only/disabled field examples. Its navigation uses ordinary buttons; it does not introduce dropdown or toggle components.

Retained updates

Update public properties directly:

status.text = 'Complete';
button.disabled = true;
button.style.backgroundColor = 'rgba(66, 133, 244, 0.7)';
card.size.width = 0.72;

Content, nested styles, and size mutations update retained backend bindings. They do not require removing and recreating the complete UI tree. Add or remove children only when the application structure actually changes.

A UIButton with custom children must have an ariaLabel. Do not combine custom children with the label or icon convenience fields.

Visual, depth, and pointer participation

These controls are independent:

Concern	Control	Effect
Render the object tree	object.visible	Hides rendering and excludes the hidden branch from interaction
Visual alpha	UI style opacity or an alpha color	Changes appearance; does not disable input
Pointer blocking	UI style or object pointerEvents, set to auto or none	Includes or excludes the object branch from hit resolution
Logical interaction boundary	interactionEnabled or object.xb.interactionEnabled	Prevents ancestors past that boundary from becoming logical targets
Three.js depth testing	material.depthTest	Controls whether existing depth occludes a custom Three.js material
Three.js depth writing	material.depthWrite	Controls whether a custom material writes depth for later draws

A transparent object can still write depth and receive input. Set the control for the behavior you want instead of using transparency as a proxy for depth or interaction.

Themes change palette and structure

Select a built-in preset:

xb.ui.theme = 'grayGlass';
// Also: colorful, glimmer, glimmerOpaque, glimmerAmber, glimmerGreen

A theme contains colors, a shared border radius, and optional style roles for surface, panel, text, button, slider, image, and icon. A style role can change padding, gap, border width, radius, component height, hover state, and other layout or appearance properties. Theme changes are not limited to color.

Use xb.ui.setTheme(update) for a partial update. Use local element styles for intentional exceptions. Theme snapshots are validated and detached from the object passed by the application.

Model and placement composition

Use ModelViewer for a supported interactive glTF, splat, or existing THREE.Object3D. Use Placement scripts when a card must follow another object, follow the viewer, face the camera, orbit, or animate visibility. Attach placement scripts to a UICard root, not to a nested UIPanel.

Built-in manipulation suspends direct placement-script children while the user moves the root and rebases them when manipulation ends.

Extension boundary

Ordinary applications use the semantic classes exported from xrblocks. The UIKit renderer and files under src/ui/internal/ or build/internal/ are private implementation details.

There is currently no public low-level adapter for direct UIKit composition. A specialized addon can own a separate renderer behind its own public entry, but it must expose interaction surfaces through supported XR Blocks APIs and must not import the private built-in UI backend.

Validate layout

After the UI renderer completes a frame, validate one mounted root or all roots:

const report = xb.ui.validate(overlay);
if (!report.ok) console.table(report.issues);

The report identifies invalid layout, overflow, clipped text, and overlay surfaces outside the viewport. ready is false when no completed mounted layout is available.

See templates/01_spatial_ui for buttons, slider input, direct updates, and theme switching.