Spatial UI
XR Blocks includes one built-in spatial UI system. It provides flex layout, world-space cards, view-space overlays, text, images, icons, buttons, toggles, dropdowns, sliders, progress bars, themes, validation, and model presentation.
Import UI components from xrblocks. The renderer starts with XR Blocks when a
UI root enters the scene. Applications do not enable UI, initialize UIKit,
register a renderer, or install a second UI raycaster.
Choose the root by coordinate space
| Type | Coordinate space | Size and position | Use for |
|---|---|---|---|
UICard | World space | size and object transforms use meters | Menus, tools, labels, model controls, and movable surfaces in the scene |
UIOverlay | View space | Layout is relative to a private full-viewport container | HUDs, status, instructions, and view-fixed controls |
UIPanel | Its UI parent | Descendant UIKit layout units | Nested rows, columns, sections, and visual grouping |
UIPanel is not a scene root. Put it inside a UICard or UIOverlay.
Use FollowHead on a card when content must remain a real world-space object
that follows the viewer. Use an overlay when it must remain fixed to the 2D
viewport.
World-space cards
import * as xb from 'xrblocks';
const card = new xb.UICard({
size: {width: 0.6, height: 'auto'},
manipulation: true,
edge: true,
style: {
flexDirection: 'column',
gap: 16,
padding: 24,
backgroundColor: '#202124',
borderRadius: 24,
},
});
card.position.set(0, 1.4, -1);
card.add(
new xb.UIText({
text: 'Welcome',
style: {fontSize: 32, color: '#ffffff'},
}),
new xb.UIButton({
label: 'Continue',
onClick: () => console.log('Continue'),
})
);
xb.add(card);
await xb.init(new xb.Options());
manipulation: true enables the card's standard face-camera move, scale, and
corner resize behavior. Moving also scales the card with its distance from the
viewer the way Android XR panels do, smooths the pointer ray so a pinch does not
jolt the card and hand tremor does not shake it, and a controller thumbstick
pushes the card away or pulls it closer while you drag it. The point you grab
stays under the pointer as the card moves around you and scales with distance.
Moves stay between 0.75 and 5 meters from the viewer. When you pass your own
actions, translate: true gets the same move defaults, and a translate
object overrides only the options it sets.
edge: true adds the outer hit band and requires translation or resize to be
enabled. Two-source
scale uses the card's scale action; the edge does not enable a missing action.
An edge with only resize enabled resizes from its corners and does not move
the card.
When the card's resize action is enabled, the corners of the edge band resize
the card instead of moving it, like Android XR and Quest panels. Pointing at a
side lights the band around the pointer; pointing at a corner shows a resize
arc there, which thins while the corner is held. Point and pinch with a ray, or
pinch a corner directly with your hand. Resizing changes size in meters and
reflows the layout; it does not scale the content. The size eases toward the
pointer, and dragging past minSize or maxSize stretches the card slightly
past the limit until release, when it snaps back. The minimum size set by the
card's content stays firm. A resize object overrides only the options it
sets, so pass smoothing: false or overshoot: false to turn either off.
Dragging to a new size switches automatic width and height dimensions to fixed
sizes, starting from the laid-out dimensions.
By default a card never gets narrower than its content allows, such as its
longest word or a row of buttons, and never shorter than its content needs at
the current width, so narrowing a card makes it taller as text wraps. To let a card get
smaller than its content, put the scrollable content in a UIScrollView (or set
an explicit minSize.height to override the content floor):
const card = new xb.UICard({
// A scroll view has no natural height, so use a fixed starting height.
size: {width: 0.56, height: 0.6},
edge: true,
manipulation: true,
children: [
header,
new xb.UIScrollView({
style: {flexGrow: 1, flexBasis: 0, minHeight: 160},
children: [body],
}),
],
});
const card = new xb.UICard({
size: {width: 0.6, height: 0.4},
edge: true,
manipulation: {
actions: {
translate: true,
resize: {
anchor: 'center', // or 'opposite' to keep the far corner in place
preserveAspectRatio: false,
minSize: {width: 0.3, height: 0.2},
maxSize: {width: 1.2, height: 0.8},
},
},
},
});
anchor: 'center' is the default and grows the card around its center.
minSize defaults to 0.1 meters per axis, and maxSize defaults to 10 meters
per axis. When maxSize is smaller than the content needs, maxSize wins over
the content floors, and a card with preserveAspectRatio keeps its
proportions.
Each card dimension independently accepts a size in meters or 'auto'.
Omitting size, including new xb.UICard(), makes both dimensions automatic.
Omitted axes also default to 'auto': size: {width: 0.6} fixes only width,
and size: {height: 0.3} fixes only height. Assigning a new card.size object
uses the same defaults; assign card.size.width or card.size.height to change
one axis while preserving the other.
Automatic dimensions fit the child flex layout, gaps, and padding and update
when content changes.
size | Behavior |
|---|---|
Omitted, or {width: 'auto', height: 'auto'} | Fit width and height to content |
{width: 0.6, height: 'auto'} | Fixed width, content-driven height |
{width: 'auto', height: 0.3} | Content-driven width, fixed height |
{width: 0.6, height: 0.3} | Fixed width and height |
const card = new xb.UICard({
style: {padding: 16, gap: 8},
children: [new xb.UIText({text: 'Fits its content'})],
});
card.size.width = 0.6;
card.size.width = 'auto';
Automatic width uses the content's natural width. Set a fixed width or
style.maxWidth when long text should wrap at a known limit.
UI children and Three.js objects
UICard, UIPanel, and other UI containers accept UI children such as UIText,
UIButton, and UIImage. Their descendants participate in flex layout, not
ordinary Three.js positioning. A UICard can also have TransformScript
children such as FaceCamera; these affect placement rather than layout.
Nested UICard and UIOverlay roots are not supported.
Put meshes, lights, and other Three.js objects beside the card under a shared
THREE.Group or xb.Script. Move that parent to move the whole assembly:
const assembly = new THREE.Group();
const mesh = new THREE.Mesh(
new THREE.BoxGeometry(0.1, 0.1, 0.1),
new THREE.MeshStandardMaterial({color: 'orange'})
);
const card = new xb.UICard({
children: [new xb.UIText({text: 'Object details'})],
});
card.position.set(0, 0.15, 0);
assembly.add(mesh, card);
xb.add(assembly);
Use UIImage for an image inside a UI layout. Use ModelViewer for a supported
3D model view that acts like a 3D scene object.
View-space overlays
An overlay's world transform does not control its rendered position. Layout it inside the view-space container:
const overlay = new xb.UIOverlay({
style: {
width: 360,
position: 'absolute',
left: '50%',
bottom: 24,
transform: {translateX: '-50%'},
},
children: [new xb.UIText({text: 'Ready', style: {fontSize: 24}})],
});
xb.add(overlay);
Cards and overlays use the theme's surface appearance by default. Set
appearance: 'none' for a transparent root used only for layout. UIPanel is
a neutral nested element unless its style or theme role gives it an appearance.
Layout units and text
UI values have two separate unit systems:
| Property | Meaning |
|---|---|
Fixed UICard.size values and world transforms | Meters |
UICard.size.width or .height: 'auto' | Dimension calculated from child layout |
| Descendant numeric width, height, gap, padding, margin, and font size | UIKit layout units |
| Percentage strings | Percentage of the applicable parent layout size |
auto | Content or flex layout chooses the size where supported |
Numeric lineHeight | Multiplier of fontSize, like CSS unitless line height |
lineHeight: 'Npx' | Explicit UIKit pixel value |
lineHeight: 'N%' | Percentage of fontSize |
These produce different line spacing:
new xb.UIText({
text: 'Compact\nmultiline text',
style: {fontSize: 24, lineHeight: 1.2}, // 1.2 * fontSize
});
new xb.UIText({
text: 'Fixed\nmultiline text',
style: {fontSize: 24, lineHeight: '32px'},
});
Use whiteSpace: 'pre-line' to preserve explicit line breaks. Combine a fixed
height, bottom alignment, and clipped overflow for a newest-first transcript:
const transcript = new xb.UIText({
text: 'You: Hello\n\nGemini: 你好',
style: {
width: '100%',
height: 240,
whiteSpace: 'pre-line',
verticalAlign: 'bottom',
overflow: 'hidden',
},
});
UIText uses a system-font canvas when the default atlas lacks a glyph. The
mounted UI element remains stable when text changes between native and Unicode
rendering.
Semantic controls and telemetry
Use semantic controls instead of raw pointer listeners:
const status = new xb.UIText({text: 'Ready'});
const slider = new xb.UISlider({
ariaLabel: 'Volume',
min: 0,
max: 1,
step: 0.05,
value: 0.5,
onInput: (value) => (status.text = `Volume ${value.toFixed(2)}`),
onChange: (value) => saveVolume(value),
});
onInput reports live captured changes. onChange reports one completed
changed interaction. A canceled slider restores its starting value.
Use UIToggle for an on/off setting. Like a slider, it draws no label, so give
it an ariaLabel and put a UIText beside it:
const captions = new xb.UIToggle({
ariaLabel: 'Captions',
checked: true,
onChange: (checked) => setCaptionsVisible(checked),
});
const captionsRow = new xb.UIPanel({
style: {
flexDirection: 'row',
justifyContent: 'space-between',
alignItems: 'center',
},
children: [new xb.UIText({text: 'Captions'}), captions],
});
A completed press flips checked and calls onChange with the new value.
Setting checked in code updates the switch without calling onChange. Scene
context reports the toggle with role switch and its checked state.
In a toggle style, backgroundColor colors the track and color colors the
thumb in both states. Put on-state colors under ':checked'; ':hover',
':active', and ':disabled' still apply on top of them:
const nightMode = new xb.UIToggle({
ariaLabel: 'Night mode',
style: {':checked': {backgroundColor: '#1e8e3e', color: '#ffffff'}},
});
Use UIDropdown to choose one value from a list of options. It needs an
ariaLabel and at least one option; placeholder is shown while value is
undefined:
const quality = new xb.UIDropdown({
ariaLabel: 'Video quality',
placeholder: 'Choose a quality',
options: [
{value: 'auto', label: 'Automatic'},
{value: 'hd', label: 'HD'},
{value: 'uhd', label: '4K', disabled: true},
],
value: 'auto',
onChange: (value) => setVideoQuality(value),
});
A completed press on the dropdown opens its list below it, or above it when
the list would pass the bottom edge of its card, or of the view in an overlay.
The list is drawn over sibling content and takes presses where it extends past
the card, without changing the card's layout. A completed press on an enabled
option selects it, calls onChange only when the value changed, and closes
the list. A press from any source that starts anywhere else closes the list
without choosing, and that press still reaches what it hit. Hiding, removing,
or disabling the dropdown also closes it. open reports whether the list is
showing; setting value in code, including to undefined, does not call
onChange.
A list taller than 320 layout units scrolls, and it opens scrolled to show the selected option. The wheel and drags scroll the list before any scroll view around the dropdown.
Scene context reports the dropdown with role combobox, its selected option's
value and label as text, and expanded. While it is open, a scrollable
listbox holds its options with role option, and checked marks the
selected one.
Style the closed dropdown with its style or the dropdown theme role. Option
rows use the dropdownOption theme role, and the selected row also matches
':checked':
xb.ui.setTheme({
styles: {
...xb.ui.theme.styles,
dropdown: {borderRadius: 16},
dropdownOption: {':hover': {backgroundColor: '#3c4043'}},
},
});
A dropdown inside a UIScrollView is clipped to that view like its other
content, so its list is cut off where it extends past the scroll viewport.
Place dropdowns outside scrolling content when their lists need the room.
Application status, diagnostics, and telemetry are also UI semantics. Use a
mounted UIText, UIPanel, or disabled control instead of drawing text into a
canvas sprite:
const diagnostics = new xb.UIText({
text: 'Tracking: waiting',
style: {fontSize: 18, color: '#fbbc04'},
});
function showTrackingState(state) {
diagnostics.text = `Tracking: ${state}`;
diagnostics.style.color = state === 'ready' ? '#34a853' : '#fbbc04';
}
This keeps status readable, layout-aware, themeable, and available to the same interaction and scene-context systems as application UI.
Use UIProgress to show how much of a known-length task is done. It is
display-only, so a press on it reaches its card like a press on text. Like a
slider, it draws no label, so give it an ariaLabel and report the status in a
UIText:
const download = new xb.UIProgress({ariaLabel: 'Download'});
function showDownload(fraction) {
download.value = fraction; // Clamped to min..max, 0..1 by default.
status.text = `Downloading ${Math.round(fraction * 100)}%`;
}
Pass min and max for another range, such as a file count; min must be
less than max, and both are fixed at construction. The bar spans the width of
a column; in a row, give it flexGrow: 1. Scene context reports it with role
progressbar and its value, min, and max.
See demos/interactive-ml for a bar that follows model training.
Scrolling and editable text
UIScrollView is a nested vertical viewport, not a new scene root. Its children are ordinary retained UI elements. Use a finite viewport height and keep long rows from shrinking:
const history = new xb.UIScrollView({
ariaLabel: 'History',
style: {height: 240, gap: 12},
children: [
new xb.UIText({
text: 'A longer entry...',
style: {fontSize: 24, flexShrink: 0},
}),
],
});
card.add(history);
To fit short content and scroll only longer content, use height: 'auto' with a maxHeight instead of a fixed height.
scrollTop, clientHeight, scrollHeight, and maxScrollTop use UI layout units, not world meters. scrollTo(offset) clamps a requested offset; scrollBy(delta) returns whether a measured viewport moved. reveal(child) minimally reveals a descendant and requires a mounted layout. Read ready before operations that require measured layout.
Wheel input, ray dragging, and direct-hand dragging share the normal interaction pipeline. Dragging ordinary content past the movement threshold cancels its pending click. A slider or text selection keeps its own capture. Nested scrolling tries the nearest eligible field or viewport first; reaching the outer boundary does not turn the gesture into card scaling. Content outside the viewport is excluded from new pointer hits and visible-object context.
UITextInput supports single-line and multiline plain text:
const message = new xb.UITextInput({
ariaLabel: 'Message',
multiline: true,
placeholder: 'Write a message',
style: {height: 120, fontSize: 24},
onInput: (value) => showDraftLength(value.length),
onSubmit: (value) => addLocalMessage(value),
});
card.add(message);
Multiline mode is chosen at construction. Enter submits a single-line field and inserts a newline in a multiline field; Ctrl/Meta+Enter submits multiline text. IME composition must complete before Enter can submit. Submitting does not clear the field or call an AI provider automatically.
Native input/textarea editing owns the value, selection, clipboard, and composition behavior. onInput reports live user edits, onChange reports a changed editing session when focus ends, and onSubmit is a separate action. Programmatic value changes do not emit a user-input callback. Escape blurs without reverting entered text. A read-only field permits focus/selection/copying; a disabled field does not accept focus or edits.
Use focus() only when the field is ready. Removing, hiding, or disabling a field releases its editing ownership. Keyboard editing must not move the simulator camera. Operating-system keyboards and clipboard permissions remain browser-dependent, especially inside immersive sessions.
Editable text loads a private canvas-based presentation on demand and uses system fonts, following the existing Unicode UI rendering approach. It adds no dependency, font download, worker, or import-map entry. Ordinary UI does not download this chunk. Deploy the complete build/ directory so the private chunk remains available.
Canvas 2D and browser text layout provide the glyphs, caret, and selection geometry. Glyph availability and appearance depend on the device's system fonts. Read field.error for a reported module or rendering failure. A ready text field is not proof of native keyboard availability.
The existing keyboard addon can bind to a field:
import {Keyboard} from 'xrblocks/addons/virtualkeyboard/index.js';
const keyboard = new Keyboard({input: message});
card.add(keyboard);
Bound keys edit the field's caret/selection and preserve its focus. Reassign keyboard.input from another field's onFocus callback to share one keyboard. Use keyboard.open to show or hide the panel and update native-keyboard ownership immediately. While the keyboard is connected and visible, it requests that the bound field's browser software keyboard remain hidden; closing or detaching it restores normal behavior. The keyboard's unbound value/submit mode remains available. Other application-owned accessory controls can set object.xb.preserveTextFocus = true when interacting with them should not blur a field.
Custom keyboard integrations can use field.suppressNativeKeyboard(), which returns a release function. The request is scoped and reference-counted rather than a permanent field setting. It leaves native editing enabled and uses browser software-keyboard hints; it is not a guarantee that every headset runtime honors those hints.
Known Meta Quest limitation: the system keyboard may reopen or close while the in-panel keyboard is used. Prefer the native Quest keyboard and leave the panel keyboard closed when they conflict. The demo retains its keyboard controls for explicit testing; there is no automatic cross-browser detection of native-keyboard availability.
Text fields expose their accessible label and editing state to scene context, but not their draft value by default. An application can explicitly provide field.userData.semantic.text when it intends to disclose a value. This is not screenshot redaction: rendered text can still appear in captured images.
See samples/spatial_forms for scrolling history, both field modes, and keyboard binding in a spatial card without an AI backend. Rich text, general horizontal scroll views, and list virtualization are not included.
The larger demos/spatial_ui_lab gallery includes a local conversation, a searchable pattern library, multilingual notes, and editable/read-only/disabled field examples. Its navigation uses ordinary buttons, and its notes example picks a template from a UIDropdown.
Retained updates
Update public properties directly:
status.text = 'Complete';
button.disabled = true;
button.style.backgroundColor = 'rgba(66, 133, 244, 0.7)';
card.size.width = 0.72;
Content, nested styles, and size mutations update retained backend bindings. They do not require removing and recreating the complete UI tree. Add or remove children only when the application structure actually changes.
A UIButton with custom children must have an ariaLabel. Do not combine
custom children with the label or icon convenience fields.
Visual, depth, and pointer participation
These controls are independent:
| Concern | Control | Effect |
|---|---|---|
| Render the object tree | object.visible | Hides rendering and excludes the hidden branch from interaction |
| Visual alpha | UI style opacity or an alpha color | Changes appearance; does not disable input |
| Pointer blocking | UI style or object pointerEvents, set to auto or none | Includes or excludes the object branch from hit resolution |
| Logical interaction boundary | interactionEnabled or object.xb.interactionEnabled | Prevents ancestors past that boundary from becoming logical targets |
| Three.js depth testing | material.depthTest | Controls whether existing depth occludes a custom Three.js material |
| Three.js depth writing | material.depthWrite | Controls whether a custom material writes depth for later draws |
A transparent object can still write depth and receive input. Set the control for the behavior you want instead of using transparency as a proxy for depth or interaction.
Themes change palette and structure
Select a built-in preset:
xb.ui.theme = 'grayGlass';
// Also: colorful, glimmer, glimmerOpaque, glimmerAmber, glimmerGreen, highContrast
A theme contains colors, a shared border radius, and optional style roles for
surface, panel, text, button, slider, toggle, progress,
dropdown, dropdownOption, image, and icon. A style role can change
padding, gap, border width, radius, component height, hover state, and other
layout or appearance properties. Theme changes are not limited to color.
Use xb.ui.setTheme(update) for a partial update. Use local element styles for
intentional exceptions. Theme snapshots are validated and detached from the
object passed by the application.
On optical see-through displays, black contributes little or no visible light.
The highContrast preset provides bright text, colored controls, and a solid
dark-color backplate instead of a black or translucent surface. The built-in UI
panels are shader-based and unlit. XR Blocks exposes the active blend mode but
does not change the app's theme automatically:
if (xb.core.environmentBlendMode === 'additive') {
xb.ui.theme = 'highContrast';
}
core.environmentBlendMode is undefined outside an active XR session. Theme
selection remains app-owned across session start and end. Explicit element
styles still take precedence over theme styles. Keep text, controls, and hit
targets generous; as a practical starting point for narrow glasses, target
controls at least about 2 degrees wide (roughly 3.5 cm at 1 m).
To preview additive-style composition in an alpha-blend passthrough session, opt in before initialization:
const options = new xb.Options();
options.simulateAdditiveDisplay = true;
options.simulatedDimming = 0; // 0 = pure additive; 1 = block passthrough
xb.ui.theme = 'highContrast'; // Theme selection remains explicit.
await xb.init(options);
While an alpha-blend session is presenting, this draws a private, noninteractive mesh last. Its custom blend preserves destination RGB and writes zero alpha, approximating black-as-transparent additive output. It does not run in opaque or native additive sessions and does not emulate display brightness, optics, or field of view. The technique has been validated with a premultiplied projection layer in Quest Browser 152 on Quest 3.
simulatedDimming is read every frame, so an app can adjust it at runtime. It
must remain between 0 and 1, inclusive. The default 0 writes zero alpha
for pure additive-style composition; 1 writes full alpha and completely dims
the passthrough behind the virtual scene.
Model and placement composition
Use ModelViewer for a supported interactive glTF, splat, or
existing THREE.Object3D. Use Placement scripts when a card must
follow another object, follow the viewer, face the camera, orbit, or animate
visibility. Attach placement scripts to a UICard root, not to a nested
UIPanel.
Built-in manipulation suspends direct placement-script children while the user moves the root and rebases them when manipulation ends.
Extension boundary
Ordinary applications use the semantic classes exported from xrblocks. The
UIKit renderer and files under src/ui/internal/ or build/internal/ are
private implementation details.
There is currently no public low-level adapter for direct UIKit composition. A specialized addon can own a separate renderer behind its own public entry, but it must expose interaction surfaces through supported XR Blocks APIs and must not import the private built-in UI backend.
Validate layout
After the UI renderer completes a frame, validate one mounted root or all roots:
const report = xb.ui.validate(overlay);
if (!report.ok) console.table(report.issues);
The report identifies invalid layout, overflow, clipped text, and overlay
surfaces outside the viewport. ready is false when no completed mounted layout
is available.
See templates/01_spatial_ui for buttons, slider input, direct updates, and theme switching.