Skip to main content

Spatial UI

XR Blocks includes one built-in spatial UI system. It provides flex layout, world-space cards, view-space overlays, text, images, icons, buttons, toggles, dropdowns, sliders, progress bars, themes, validation, and model presentation.

Import UI components from xrblocks. The renderer starts with XR Blocks when a UI root enters the scene. Applications do not enable UI, initialize UIKit, register a renderer, or install a second UI raycaster.

Choose the root by coordinate space​

TypeCoordinate spaceSize and positionUse for
UICardWorld spacesize and object transforms use metersMenus, tools, labels, model controls, and movable surfaces in the scene
UIOverlayView spaceLayout is relative to a private full-viewport containerHUDs, status, instructions, and view-fixed controls
UIPanelIts UI parentDescendant UIKit layout unitsNested rows, columns, sections, and visual grouping

UIPanel is not a scene root. Put it inside a UICard or UIOverlay. Use FollowHead on a card when content must remain a real world-space object that follows the viewer. Use an overlay when it must remain fixed to the 2D viewport.

World-space cards​

import * as xb from 'xrblocks';

const card = new xb.UICard({
size: {width: 0.6, height: 'auto'},
manipulation: true,
edge: true,
style: {
flexDirection: 'column',
gap: 16,
padding: 24,
backgroundColor: '#202124',
borderRadius: 24,
},
});
card.position.set(0, 1.4, -1);

card.add(
new xb.UIText({
text: 'Welcome',
style: {fontSize: 32, color: '#ffffff'},
}),
new xb.UIButton({
label: 'Continue',
onClick: () => console.log('Continue'),
})
);

xb.add(card);
await xb.init(new xb.Options());

manipulation: true enables the card's standard face-camera move, scale, and corner resize behavior. Moving also scales the card with its distance from the viewer the way Android XR panels do, smooths the pointer ray so a pinch does not jolt the card and hand tremor does not shake it, and a controller thumbstick pushes the card away or pulls it closer while you drag it. The point you grab stays under the pointer as the card moves around you and scales with distance. Moves stay between 0.75 and 5 meters from the viewer. When you pass your own actions, translate: true gets the same move defaults, and a translate object overrides only the options it sets.

edge: true adds the outer hit band and requires translation or resize to be enabled. Two-source scale uses the card's scale action; the edge does not enable a missing action. An edge with only resize enabled resizes from its corners and does not move the card.

When the card's resize action is enabled, the corners of the edge band resize the card instead of moving it, like Android XR and Quest panels. Pointing at a side lights the band around the pointer; pointing at a corner shows a resize arc there, which thins while the corner is held. Point and pinch with a ray, or pinch a corner directly with your hand. Resizing changes size in meters and reflows the layout; it does not scale the content. The size eases toward the pointer, and dragging past minSize or maxSize stretches the card slightly past the limit until release, when it snaps back. The minimum size set by the card's content stays firm. A resize object overrides only the options it sets, so pass smoothing: false or overshoot: false to turn either off. Dragging to a new size switches automatic width and height dimensions to fixed sizes, starting from the laid-out dimensions.

By default a card never gets narrower than its content allows, such as its longest word or a row of buttons, and never shorter than its content needs at the current width, so narrowing a card makes it taller as text wraps. To let a card get smaller than its content, put the scrollable content in a UIScrollView (or set an explicit minSize.height to override the content floor):

const card = new xb.UICard({
// A scroll view has no natural height, so use a fixed starting height.
size: {width: 0.56, height: 0.6},
edge: true,
manipulation: true,
children: [
header,
new xb.UIScrollView({
style: {flexGrow: 1, flexBasis: 0, minHeight: 160},
children: [body],
}),
],
});
const card = new xb.UICard({
size: {width: 0.6, height: 0.4},
edge: true,
manipulation: {
actions: {
translate: true,
resize: {
anchor: 'center', // or 'opposite' to keep the far corner in place
preserveAspectRatio: false,
minSize: {width: 0.3, height: 0.2},
maxSize: {width: 1.2, height: 0.8},
},
},
},
});

anchor: 'center' is the default and grows the card around its center. minSize defaults to 0.1 meters per axis, and maxSize defaults to 10 meters per axis. When maxSize is smaller than the content needs, maxSize wins over the content floors, and a card with preserveAspectRatio keeps its proportions.

Each card dimension independently accepts a size in meters or 'auto'. Omitting size, including new xb.UICard(), makes both dimensions automatic. Omitted axes also default to 'auto': size: {width: 0.6} fixes only width, and size: {height: 0.3} fixes only height. Assigning a new card.size object uses the same defaults; assign card.size.width or card.size.height to change one axis while preserving the other. Automatic dimensions fit the child flex layout, gaps, and padding and update when content changes.

sizeBehavior
Omitted, or {width: 'auto', height: 'auto'}Fit width and height to content
{width: 0.6, height: 'auto'}Fixed width, content-driven height
{width: 'auto', height: 0.3}Content-driven width, fixed height
{width: 0.6, height: 0.3}Fixed width and height
const card = new xb.UICard({
style: {padding: 16, gap: 8},
children: [new xb.UIText({text: 'Fits its content'})],
});
card.size.width = 0.6;
card.size.width = 'auto';

Automatic width uses the content's natural width. Set a fixed width or style.maxWidth when long text should wrap at a known limit.

UI children and Three.js objects​

UICard, UIPanel, and other UI containers accept UI children such as UIText, UIButton, and UIImage. Their descendants participate in flex layout, not ordinary Three.js positioning. A UICard can also have TransformScript children such as FaceCamera; these affect placement rather than layout. Nested UICard and UIOverlay roots are not supported.

Put meshes, lights, and other Three.js objects beside the card under a shared THREE.Group or xb.Script. Move that parent to move the whole assembly:

const assembly = new THREE.Group();
const mesh = new THREE.Mesh(
new THREE.BoxGeometry(0.1, 0.1, 0.1),
new THREE.MeshStandardMaterial({color: 'orange'})
);
const card = new xb.UICard({
children: [new xb.UIText({text: 'Object details'})],
});
card.position.set(0, 0.15, 0);
assembly.add(mesh, card);
xb.add(assembly);

Use UIImage for an image inside a UI layout. Use ModelViewer for a supported 3D model view that acts like a 3D scene object.

View-space overlays​

An overlay's world transform does not control its rendered position. Layout it inside the view-space container:

const overlay = new xb.UIOverlay({
style: {
width: 360,
position: 'absolute',
left: '50%',
bottom: 24,
transform: {translateX: '-50%'},
},
children: [new xb.UIText({text: 'Ready', style: {fontSize: 24}})],
});

xb.add(overlay);

Cards and overlays use the theme's surface appearance by default. Set appearance: 'none' for a transparent root used only for layout. UIPanel is a neutral nested element unless its style or theme role gives it an appearance.

Layout units and text​

UI values have two separate unit systems:

PropertyMeaning
Fixed UICard.size values and world transformsMeters
UICard.size.width or .height: 'auto'Dimension calculated from child layout
Descendant numeric width, height, gap, padding, margin, and font sizeUIKit layout units
Percentage stringsPercentage of the applicable parent layout size
autoContent or flex layout chooses the size where supported
Numeric lineHeightMultiplier of fontSize, like CSS unitless line height
lineHeight: 'Npx'Explicit UIKit pixel value
lineHeight: 'N%'Percentage of fontSize

These produce different line spacing:

new xb.UIText({
text: 'Compact\nmultiline text',
style: {fontSize: 24, lineHeight: 1.2}, // 1.2 * fontSize
});

new xb.UIText({
text: 'Fixed\nmultiline text',
style: {fontSize: 24, lineHeight: '32px'},
});

Use whiteSpace: 'pre-line' to preserve explicit line breaks. Combine a fixed height, bottom alignment, and clipped overflow for a newest-first transcript:

const transcript = new xb.UIText({
text: 'You: Hello\n\nGemini: 你好',
style: {
width: '100%',
height: 240,
whiteSpace: 'pre-line',
verticalAlign: 'bottom',
overflow: 'hidden',
},
});

UIText uses a system-font canvas when the default atlas lacks a glyph. The mounted UI element remains stable when text changes between native and Unicode rendering.

Semantic controls and telemetry​

Use semantic controls instead of raw pointer listeners:

const status = new xb.UIText({text: 'Ready'});
const slider = new xb.UISlider({
ariaLabel: 'Volume',
min: 0,
max: 1,
step: 0.05,
value: 0.5,
onInput: (value) => (status.text = `Volume ${value.toFixed(2)}`),
onChange: (value) => saveVolume(value),
});

onInput reports live captured changes. onChange reports one completed changed interaction. A canceled slider restores its starting value.

Use UIToggle for an on/off setting. Like a slider, it draws no label, so give it an ariaLabel and put a UIText beside it:

const captions = new xb.UIToggle({
ariaLabel: 'Captions',
checked: true,
onChange: (checked) => setCaptionsVisible(checked),
});
const captionsRow = new xb.UIPanel({
style: {
flexDirection: 'row',
justifyContent: 'space-between',
alignItems: 'center',
},
children: [new xb.UIText({text: 'Captions'}), captions],
});

A completed press flips checked and calls onChange with the new value. Setting checked in code updates the switch without calling onChange. Scene context reports the toggle with role switch and its checked state.

In a toggle style, backgroundColor colors the track and color colors the thumb in both states. Put on-state colors under ':checked'; ':hover', ':active', and ':disabled' still apply on top of them:

const nightMode = new xb.UIToggle({
ariaLabel: 'Night mode',
style: {':checked': {backgroundColor: '#1e8e3e', color: '#ffffff'}},
});

Use UIDropdown to choose one value from a list of options. It needs an ariaLabel and at least one option; placeholder is shown while value is undefined:

const quality = new xb.UIDropdown({
ariaLabel: 'Video quality',
placeholder: 'Choose a quality',
options: [
{value: 'auto', label: 'Automatic'},
{value: 'hd', label: 'HD'},
{value: 'uhd', label: '4K', disabled: true},
],
value: 'auto',
onChange: (value) => setVideoQuality(value),
});

A completed press on the dropdown opens its list below it, or above it when the list would pass the bottom edge of its card, or of the view in an overlay. The list is drawn over sibling content and takes presses where it extends past the card, without changing the card's layout. A completed press on an enabled option selects it, calls onChange only when the value changed, and closes the list. A press from any source that starts anywhere else closes the list without choosing, and that press still reaches what it hit. Hiding, removing, or disabling the dropdown also closes it. open reports whether the list is showing; setting value in code, including to undefined, does not call onChange.

A list taller than 320 layout units scrolls, and it opens scrolled to show the selected option. The wheel and drags scroll the list before any scroll view around the dropdown.

Scene context reports the dropdown with role combobox, its selected option's value and label as text, and expanded. While it is open, a scrollable listbox holds its options with role option, and checked marks the selected one.

Style the closed dropdown with its style or the dropdown theme role. Option rows use the dropdownOption theme role, and the selected row also matches ':checked':

xb.ui.setTheme({
styles: {
...xb.ui.theme.styles,
dropdown: {borderRadius: 16},
dropdownOption: {':hover': {backgroundColor: '#3c4043'}},
},
});

A dropdown inside a UIScrollView is clipped to that view like its other content, so its list is cut off where it extends past the scroll viewport. Place dropdowns outside scrolling content when their lists need the room.

Application status, diagnostics, and telemetry are also UI semantics. Use a mounted UIText, UIPanel, or disabled control instead of drawing text into a canvas sprite:

const diagnostics = new xb.UIText({
text: 'Tracking: waiting',
style: {fontSize: 18, color: '#fbbc04'},
});

function showTrackingState(state) {
diagnostics.text = `Tracking: ${state}`;
diagnostics.style.color = state === 'ready' ? '#34a853' : '#fbbc04';
}

This keeps status readable, layout-aware, themeable, and available to the same interaction and scene-context systems as application UI.

Use UIProgress to show how much of a known-length task is done. It is display-only, so a press on it reaches its card like a press on text. Like a slider, it draws no label, so give it an ariaLabel and report the status in a UIText:

const download = new xb.UIProgress({ariaLabel: 'Download'});

function showDownload(fraction) {
download.value = fraction; // Clamped to min..max, 0..1 by default.
status.text = `Downloading ${Math.round(fraction * 100)}%`;
}

Pass min and max for another range, such as a file count; min must be less than max, and both are fixed at construction. The bar spans the width of a column; in a row, give it flexGrow: 1. Scene context reports it with role progressbar and its value, min, and max.

See demos/interactive-ml for a bar that follows model training.

Scrolling and editable text​

UIScrollView is a nested vertical viewport, not a new scene root. Its children are ordinary retained UI elements. Use a finite viewport height and keep long rows from shrinking:

const history = new xb.UIScrollView({
ariaLabel: 'History',
style: {height: 240, gap: 12},
children: [
new xb.UIText({
text: 'A longer entry...',
style: {fontSize: 24, flexShrink: 0},
}),
],
});
card.add(history);

To fit short content and scroll only longer content, use height: 'auto' with a maxHeight instead of a fixed height.

scrollTop, clientHeight, scrollHeight, and maxScrollTop use UI layout units, not world meters. scrollTo(offset) clamps a requested offset; scrollBy(delta) returns whether a measured viewport moved. reveal(child) minimally reveals a descendant and requires a mounted layout. Read ready before operations that require measured layout.

Wheel input, ray dragging, and direct-hand dragging share the normal interaction pipeline. Dragging ordinary content past the movement threshold cancels its pending click. A slider or text selection keeps its own capture. Nested scrolling tries the nearest eligible field or viewport first; reaching the outer boundary does not turn the gesture into card scaling. Content outside the viewport is excluded from new pointer hits and visible-object context.

UITextInput supports single-line and multiline plain text:

const message = new xb.UITextInput({
ariaLabel: 'Message',
multiline: true,
placeholder: 'Write a message',
style: {height: 120, fontSize: 24},
onInput: (value) => showDraftLength(value.length),
onSubmit: (value) => addLocalMessage(value),
});
card.add(message);

Multiline mode is chosen at construction. Enter submits a single-line field and inserts a newline in a multiline field; Ctrl/Meta+Enter submits multiline text. IME composition must complete before Enter can submit. Submitting does not clear the field or call an AI provider automatically.

Native input/textarea editing owns the value, selection, clipboard, and composition behavior. onInput reports live user edits, onChange reports a changed editing session when focus ends, and onSubmit is a separate action. Programmatic value changes do not emit a user-input callback. Escape blurs without reverting entered text. A read-only field permits focus/selection/copying; a disabled field does not accept focus or edits.

Use focus() only when the field is ready. Removing, hiding, or disabling a field releases its editing ownership. Keyboard editing must not move the simulator camera. Operating-system keyboards and clipboard permissions remain browser-dependent, especially inside immersive sessions.

Editable text loads a private canvas-based presentation on demand and uses system fonts, following the existing Unicode UI rendering approach. It adds no dependency, font download, worker, or import-map entry. Ordinary UI does not download this chunk. Deploy the complete build/ directory so the private chunk remains available.

Canvas 2D and browser text layout provide the glyphs, caret, and selection geometry. Glyph availability and appearance depend on the device's system fonts. Read field.error for a reported module or rendering failure. A ready text field is not proof of native keyboard availability.

The existing keyboard addon can bind to a field:

import {Keyboard} from 'xrblocks/addons/virtualkeyboard/index.js';

const keyboard = new Keyboard({input: message});
card.add(keyboard);

Bound keys edit the field's caret/selection and preserve its focus. Reassign keyboard.input from another field's onFocus callback to share one keyboard. Use keyboard.open to show or hide the panel and update native-keyboard ownership immediately. While the keyboard is connected and visible, it requests that the bound field's browser software keyboard remain hidden; closing or detaching it restores normal behavior. The keyboard's unbound value/submit mode remains available. Other application-owned accessory controls can set object.xb.preserveTextFocus = true when interacting with them should not blur a field.

Custom keyboard integrations can use field.suppressNativeKeyboard(), which returns a release function. The request is scoped and reference-counted rather than a permanent field setting. It leaves native editing enabled and uses browser software-keyboard hints; it is not a guarantee that every headset runtime honors those hints.

Known Meta Quest limitation: the system keyboard may reopen or close while the in-panel keyboard is used. Prefer the native Quest keyboard and leave the panel keyboard closed when they conflict. The demo retains its keyboard controls for explicit testing; there is no automatic cross-browser detection of native-keyboard availability.

Text fields expose their accessible label and editing state to scene context, but not their draft value by default. An application can explicitly provide field.userData.semantic.text when it intends to disclose a value. This is not screenshot redaction: rendered text can still appear in captured images.

See samples/spatial_forms for scrolling history, both field modes, and keyboard binding in a spatial card without an AI backend. Rich text, general horizontal scroll views, and list virtualization are not included.

The larger demos/spatial_ui_lab gallery includes a local conversation, a searchable pattern library, multilingual notes, and editable/read-only/disabled field examples. Its navigation uses ordinary buttons, and its notes example picks a template from a UIDropdown.

Retained updates​

Update public properties directly:

status.text = 'Complete';
button.disabled = true;
button.style.backgroundColor = 'rgba(66, 133, 244, 0.7)';
card.size.width = 0.72;

Content, nested styles, and size mutations update retained backend bindings. They do not require removing and recreating the complete UI tree. Add or remove children only when the application structure actually changes.

A UIButton with custom children must have an ariaLabel. Do not combine custom children with the label or icon convenience fields.

Visual, depth, and pointer participation​

These controls are independent:

ConcernControlEffect
Render the object treeobject.visibleHides rendering and excludes the hidden branch from interaction
Visual alphaUI style opacity or an alpha colorChanges appearance; does not disable input
Pointer blockingUI style or object pointerEvents, set to auto or noneIncludes or excludes the object branch from hit resolution
Logical interaction boundaryinteractionEnabled or object.xb.interactionEnabledPrevents ancestors past that boundary from becoming logical targets
Three.js depth testingmaterial.depthTestControls whether existing depth occludes a custom Three.js material
Three.js depth writingmaterial.depthWriteControls whether a custom material writes depth for later draws

A transparent object can still write depth and receive input. Set the control for the behavior you want instead of using transparency as a proxy for depth or interaction.

Themes change palette and structure​

Select a built-in preset:

xb.ui.theme = 'grayGlass';
// Also: colorful, glimmer, glimmerOpaque, glimmerAmber, glimmerGreen, highContrast

A theme contains colors, a shared border radius, and optional style roles for surface, panel, text, button, slider, toggle, progress, dropdown, dropdownOption, image, and icon. A style role can change padding, gap, border width, radius, component height, hover state, and other layout or appearance properties. Theme changes are not limited to color.

Use xb.ui.setTheme(update) for a partial update. Use local element styles for intentional exceptions. Theme snapshots are validated and detached from the object passed by the application.

On optical see-through displays, black contributes little or no visible light. The highContrast preset provides bright text, colored controls, and a solid dark-color backplate instead of a black or translucent surface. The built-in UI panels are shader-based and unlit. XR Blocks exposes the active blend mode but does not change the app's theme automatically:

if (xb.core.environmentBlendMode === 'additive') {
xb.ui.theme = 'highContrast';
}

core.environmentBlendMode is undefined outside an active XR session. Theme selection remains app-owned across session start and end. Explicit element styles still take precedence over theme styles. Keep text, controls, and hit targets generous; as a practical starting point for narrow glasses, target controls at least about 2 degrees wide (roughly 3.5 cm at 1 m).

To preview additive-style composition in an alpha-blend passthrough session, opt in before initialization:

const options = new xb.Options();
options.simulateAdditiveDisplay = true;
options.simulatedDimming = 0; // 0 = pure additive; 1 = block passthrough
xb.ui.theme = 'highContrast'; // Theme selection remains explicit.
await xb.init(options);

While an alpha-blend session is presenting, this draws a private, noninteractive mesh last. Its custom blend preserves destination RGB and writes zero alpha, approximating black-as-transparent additive output. It does not run in opaque or native additive sessions and does not emulate display brightness, optics, or field of view. The technique has been validated with a premultiplied projection layer in Quest Browser 152 on Quest 3.

simulatedDimming is read every frame, so an app can adjust it at runtime. It must remain between 0 and 1, inclusive. The default 0 writes zero alpha for pure additive-style composition; 1 writes full alpha and completely dims the passthrough behind the virtual scene.

Model and placement composition​

Use ModelViewer for a supported interactive glTF, splat, or existing THREE.Object3D. Use Placement scripts when a card must follow another object, follow the viewer, face the camera, orbit, or animate visibility. Attach placement scripts to a UICard root, not to a nested UIPanel.

Built-in manipulation suspends direct placement-script children while the user moves the root and rebases them when manipulation ends.

Extension boundary​

Ordinary applications use the semantic classes exported from xrblocks. The UIKit renderer and files under src/ui/internal/ or build/internal/ are private implementation details.

There is currently no public low-level adapter for direct UIKit composition. A specialized addon can own a separate renderer behind its own public entry, but it must expose interaction surfaces through supported XR Blocks APIs and must not import the private built-in UI backend.

Validate layout​

After the UI renderer completes a frame, validate one mounted root or all roots:

const report = xb.ui.validate(overlay);
if (!report.ok) console.table(report.issues);

The report identifies invalid layout, overflow, clipped text, and overlay surfaces outside the viewport. ready is false when no completed mounted layout is available.

See templates/01_spatial_ui for buttons, slider input, direct updates, and theme switching.