Class: XRDeviceCamera

Defined in: src/camera/XRDeviceCamera.ts:45

Handles video capture from a device camera, manages the device list, and reports its state using VideoStream's event model.

Extends

• VideoStream<XRDeviceCameraDetails>

Constructors

Constructor

new XRDeviceCamera(options): XRDeviceCamera

Defined in: src/camera/XRDeviceCamera.ts:77

Parameters

options

DeviceCameraOptions

The configuration options.

Returns

XRDeviceCamera

Overrides

VideoStream.constructor

Properties

aspectRatio?

optional aspectRatio: number

Defined in: src/video/VideoStream.ts:107

Inherited from

VideoStream.aspectRatio

---

height?

optional height: number

Defined in: src/video/VideoStream.ts:106

Inherited from

VideoStream.height

---

isXRScript

isXRScript: boolean = true

Defined in: src/core/Script.ts:99

Inherited from

VideoStream.isXRScript

---

loaded

loaded: boolean = false

Defined in: src/video/VideoStream.ts:104

Inherited from

VideoStream.loaded

---

rgbToDepthParams

rgbToDepthParams: RgbToDepthParams

Defined in: src/camera/XRDeviceCamera.ts:49

---

simulatorCamera?

optional simulatorCamera: SimulatorCameraSource

Defined in: src/camera/XRDeviceCamera.ts:48

---

state

state: StreamState = StreamState.IDLE

Defined in: src/video/VideoStream.ts:109

Inherited from

VideoStream.state

---

stream_

protected stream_: MediaStream | null = null

Defined in: src/video/VideoStream.ts:111

Inherited from

VideoStream.stream_

---

texture

texture: Texture

Defined in: src/video/VideoStream.ts:108

Inherited from

VideoStream.texture

---

video_

protected video_: HTMLVideoElement

Defined in: src/video/VideoStream.ts:112

Inherited from

VideoStream.video_

---

videoConstraints_

protected videoConstraints_: MediaTrackConstraints

Defined in: src/camera/XRDeviceCamera.ts:50

---

width?

optional width: number

Defined in: src/video/VideoStream.ts:105

Inherited from

VideoStream.width

---

xb?

optional xb: XBObjectOptions

Defined in: src/interaction/InteractionTypes.ts:34

Inherited from

VideoStream.xb

Accessors

isUsingXRCameraAccess

Get Signature

get isUsingXRCameraAccess(): boolean

Defined in: src/camera/XRDeviceCamera.ts:357

Whether the camera is using the WebXR Raw Camera Access API fallback.

Returns

boolean

---

video

Get Signature

get video(): HTMLVideoElement

Defined in: src/video/VideoStream.ts:113

Returns

HTMLVideoElement

Inherited from

VideoStream.video

Methods

captureSnapshot()

Call Signature

captureSnapshot(): Promise<Texture<unknown, TextureEventMap> | null>

Defined in: src/camera/XRDeviceCamera.ts:375

Captures a snapshot from the active camera source.

In the normal video path this resolves from getSnapshot immediately. In the WebXR Raw Camera Access fallback, the browser camera image is only valid during the XR frame that produced it, so this queues a one-shot GPU readback for the next updateXRCamera call and then resolves through getSnapshot. Concurrent camera-access calls share that next XR-frame readback, but each resolves with its own requested format. If no XR camera frame arrives within about one second, or the raw camera path is stopped, the promise resolves to null. Synchronous getSnapshot on that fallback path returns the most recently captured one-shot frame, or null when no capture has completed yet.

Returns

Promise<Texture<unknown, TextureEventMap> | null>

Call Signature

captureSnapshot(options): Promise<ImageData | null>

Defined in: src/camera/XRDeviceCamera.ts:376

Captures a snapshot from the active camera source.

In the normal video path this resolves from getSnapshot immediately. In the WebXR Raw Camera Access fallback, the browser camera image is only valid during the XR frame that produced it, so this queues a one-shot GPU readback for the next updateXRCamera call and then resolves through getSnapshot. Concurrent camera-access calls share that next XR-frame readback, but each resolves with its own requested format. If no XR camera frame arrives within about one second, or the raw camera path is stopped, the promise resolves to null. Synchronous getSnapshot on that fallback path returns the most recently captured one-shot frame, or null when no capture has completed yet.

Parameters

options

VideoStreamGetSnapshotImageDataOptions

Returns

Promise<ImageData | null>

Call Signature

captureSnapshot(options): Promise<string | null>

Defined in: src/camera/XRDeviceCamera.ts:379

Captures a snapshot from the active camera source.

In the normal video path this resolves from getSnapshot immediately. In the WebXR Raw Camera Access fallback, the browser camera image is only valid during the XR frame that produced it, so this queues a one-shot GPU readback for the next updateXRCamera call and then resolves through getSnapshot. Concurrent camera-access calls share that next XR-frame readback, but each resolves with its own requested format. If no XR camera frame arrives within about one second, or the raw camera path is stopped, the promise resolves to null. Synchronous getSnapshot on that fallback path returns the most recently captured one-shot frame, or null when no capture has completed yet.

Parameters

options

VideoStreamGetSnapshotBase64Options

Returns

Promise<string | null>

Call Signature

captureSnapshot(options): Promise<Texture<unknown, TextureEventMap> | null>

Defined in: src/camera/XRDeviceCamera.ts:382

Captures a snapshot from the active camera source.

In the normal video path this resolves from getSnapshot immediately. In the WebXR Raw Camera Access fallback, the browser camera image is only valid during the XR frame that produced it, so this queues a one-shot GPU readback for the next updateXRCamera call and then resolves through getSnapshot. Concurrent camera-access calls share that next XR-frame readback, but each resolves with its own requested format. If no XR camera frame arrives within about one second, or the raw camera path is stopped, the promise resolves to null. Synchronous getSnapshot on that fallback path returns the most recently captured one-shot frame, or null when no capture has completed yet.

Parameters

options

VideoStreamGetSnapshotTextureOptions

Returns

Promise<Texture<unknown, TextureEventMap> | null>

Call Signature

captureSnapshot(options): Promise<Blob | null>

Defined in: src/camera/XRDeviceCamera.ts:385

Captures a snapshot from the active camera source.

In the normal video path this resolves from getSnapshot immediately. In the WebXR Raw Camera Access fallback, the browser camera image is only valid during the XR frame that produced it, so this queues a one-shot GPU readback for the next updateXRCamera call and then resolves through getSnapshot. Concurrent camera-access calls share that next XR-frame readback, but each resolves with its own requested format. If no XR camera frame arrives within about one second, or the raw camera path is stopped, the promise resolves to null. Synchronous getSnapshot on that fallback path returns the most recently captured one-shot frame, or null when no capture has completed yet.

Parameters

options

VideoStreamGetSnapshotBlobOptions

Returns

Promise<Blob | null>

Call Signature

captureSnapshot(options): Promise<string | Texture<unknown, TextureEventMap> | Blob | ImageData | null>

Defined in: src/camera/XRDeviceCamera.ts:388

Captures a snapshot from the active camera source.

In the normal video path this resolves from getSnapshot immediately. In the WebXR Raw Camera Access fallback, the browser camera image is only valid during the XR frame that produced it, so this queues a one-shot GPU readback for the next updateXRCamera call and then resolves through getSnapshot. Concurrent camera-access calls share that next XR-frame readback, but each resolves with its own requested format. If no XR camera frame arrives within about one second, or the raw camera path is stopped, the promise resolves to null. Synchronous getSnapshot on that fallback path returns the most recently captured one-shot frame, or null when no capture has completed yet.

Parameters

options

VideoStreamGetSnapshotOptions

Returns

Promise<string | Texture<unknown, TextureEventMap> | Blob | ImageData | null>

---

dispose()

dispose(): void

Defined in: src/camera/XRDeviceCamera.ts:524

Disposes of all resources used by this stream.

Returns

void

Overrides

VideoStream.dispose

---

drawSnapshotSource_()

protected drawSnapshotSource_(context, width, height): void

Defined in: src/camera/XRDeviceCamera.ts:433

Draws the current snapshot source into context at the requested size. Subclasses may override this to provide pixels from another source.

Parameters

context

CanvasRenderingContext2D

width

number

height

number

Returns

void

Overrides

VideoStream.drawSnapshotSource_

---

getAvailableDevices()

getAvailableDevices(): MediaOrSimulatorMediaDeviceInfo[]

Defined in: src/camera/XRDeviceCamera.ts:326

Gets the list of enumerated video devices.

Returns

MediaOrSimulatorMediaDeviceInfo[]

---

getAvailableVideoDevices()

getAvailableVideoDevices(): Promise<MediaOrSimulatorMediaDeviceInfo[]>

Defined in: src/camera/XRDeviceCamera.ts:92

Retrieves the list of available video input devices.

Returns

Promise<MediaOrSimulatorMediaDeviceInfo[]>

A promise that resolves with an array of video devices.

---

getCurrentDevice()

getCurrentDevice(): MediaOrSimulatorMediaDeviceInfo | undefined

Defined in: src/camera/XRDeviceCamera.ts:333

Gets the currently active device info, if available.

Returns

MediaOrSimulatorMediaDeviceInfo | undefined

---

getCurrentDeviceIndex()

getCurrentDeviceIndex(): number

Defined in: src/camera/XRDeviceCamera.ts:350

Gets the index of the currently active device.

Returns

number

---

getCurrentTrackSettings()

getCurrentTrackSettings(): MediaTrackSettings | undefined

Defined in: src/camera/XRDeviceCamera.ts:343

Gets the settings of the currently active video track.

Returns

MediaTrackSettings | undefined

---

getDeviceIdFromLabel()

protected getDeviceIdFromLabel(label): string | null

Defined in: src/camera/XRDeviceCamera.ts:152

Parameters

label

string

Returns

string | null

---

getSnapshot()

Call Signature

getSnapshot(options): ImageData

Defined in: src/video/VideoStream.ts:264

Captures the current video frame.

Parameters

options

VideoStreamGetSnapshotImageDataOptions

The options for the snapshot.

Returns

ImageData

The captured data.

Inherited from

VideoStream.getSnapshot

Call Signature

getSnapshot(options): Promise<string | null>

Defined in: src/video/VideoStream.ts:265

Captures the current video frame.

Parameters

options

VideoStreamGetSnapshotBase64Options

The options for the snapshot.

Returns

Promise<string | null>

The captured data.

Inherited from

VideoStream.getSnapshot

Call Signature

getSnapshot(options): Texture

Defined in: src/video/VideoStream.ts:268

Captures the current video frame.

Parameters

options

VideoStreamGetSnapshotTextureOptions

The options for the snapshot.

Returns

Texture

The captured data.

Inherited from

VideoStream.getSnapshot

Call Signature

getSnapshot(options): Promise<Blob | null>

Defined in: src/video/VideoStream.ts:269

Captures the current video frame.

Parameters

options

VideoStreamGetSnapshotBlobOptions

The options for the snapshot.

Returns

Promise<Blob | null>

The captured data.

Inherited from

VideoStream.getSnapshot

---

handleVideoStreamLoadedMetadata()

protected handleVideoStreamLoadedMetadata(resolve, reject, allowRetry): void

Defined in: src/video/VideoStream.ts:174

Processes video metadata, sets dimensions, and resolves a promise.

Parameters

resolve

() => void

The resolve function of the wrapping Promise.

reject

(_) => void

The reject function of the wrapping Promise.

allowRetry

boolean = false

Whether to allow a retry attempt on failure.

Returns

void

Inherited from

VideoStream.handleVideoStreamLoadedMetadata

---

init()

init(): Promise<void>

Defined in: src/camera/XRDeviceCamera.ts:119

Initializes the camera based on the initial constraints.

Returns

Promise<void>

Overrides

VideoStream.init

---

initPhysics()

initPhysics(_physics): void | Promise<void>

Defined in: src/core/Script.ts:118

Enables depth-aware interactions with physics. See /samples/advanced/ballpit

Parameters

_physics

Physics

Returns

void | Promise<void>

Inherited from

VideoStream.initPhysics

---

initStream_()

protected initStream_(): Promise<void>

Defined in: src/camera/XRDeviceCamera.ts:163

Initializes the media stream from the user's camera. After the stream starts, it updates the current device index based on the stream's active track.

Returns

Promise<void>

---

onHoverEnter()

onHoverEnter(_event): void

Defined in: src/core/Script.ts:215

Called when a source starts hovering over this object.

Parameters

_event

HoverEvent

The hover source, target, surface, and intersection. Call event.stopPropagation() to stop bubbling to ancestor Scripts.

Returns

void

Inherited from

VideoStream.onHoverEnter

---

onHoverExit()

onHoverExit(_event): void

Defined in: src/core/Script.ts:221

Called when a source stops hovering over this object.

Parameters

_event

HoverEvent

The hover source, target, surface, and intersection. Call event.stopPropagation() to stop bubbling to ancestor Scripts.

Returns

void

Inherited from

VideoStream.onHoverExit

---

onHovering()

onHovering(_event): void

Defined in: src/core/Script.ts:227

Called while a source hovers over this object.

Parameters

_event

HoverEvent

The hover source, target, surface, and intersection. Call event.stopPropagation() to stop bubbling to ancestor Scripts.

Returns

void

Inherited from

VideoStream.onHovering

---

onKeyDown()

onKeyDown(_event): void

Defined in: src/core/Script.ts:158

Called on keyboard keypress.

Parameters

_event

KeyEvent

Event containing .code to read the keyboard key.

Returns

void

Inherited from

VideoStream.onKeyDown

---

onKeyUp()

onKeyUp(_event): void

Defined in: src/core/Script.ts:159

Parameters

_event

KeyEvent

Returns

void

Inherited from

VideoStream.onKeyUp

---

onLongSelect()

onLongSelect(_event): void

Defined in: src/core/Script.ts:152

Called when an object selection reaches the long-select delay.

Parameters

_event

LongSelectEvent

Returns

void

Inherited from

VideoStream.onLongSelect

---

onObjectGrabbing()

onObjectGrabbing(_event): void

Defined in: src/core/Script.ts:253

Called every frame a hand is grabbing this object.

Parameters

_event

ObjectGrabEvent

Returns

void

Inherited from

VideoStream.onObjectGrabbing

---

onObjectGrabEnd()

onObjectGrabEnd(_event): void

Defined in: src/core/Script.ts:258

Called when a hand stops grabbing this object. This ends built-in direct-touch manipulation without ending contact.

Parameters

_event

ObjectGrabEvent

Returns

void

Inherited from

VideoStream.onObjectGrabEnd

---

onObjectGrabStart()

onObjectGrabStart(_event): void

Defined in: src/core/Script.ts:249

Called when a hand starts grabbing this object (touching + pinching). A grab starts built-in direct-touch manipulation when enabled.

Parameters

_event

ObjectGrabEvent

Returns

void

Inherited from

VideoStream.onObjectGrabStart

---

onObjectLongSelect()

onObjectLongSelect(_event): void

Defined in: src/core/Script.ts:203

Called once when a captured selection is held for the long-select delay. Manipulation captures do not emit this callback.

Parameters

_event

LongSelectEvent

The controller and completed hold duration. Call event.stopPropagation() to stop bubbling to ancestor Scripts.

Returns

void

Inherited from

VideoStream.onObjectLongSelect

---

onObjectManipulate()

onObjectManipulate(_event): void

Defined in: src/core/Script.ts:209

Called for each phase of an automatic object manipulation. Call event.stopPropagation() to stop bubbling. Calling preventDefault() on a start event suppresses the automatic action.

Parameters

_event

ManipulationEvent

Returns

void

Inherited from

VideoStream.onObjectManipulate

---

onObjectSelectEnd()

onObjectSelectEnd(_event): void

Defined in: src/core/Script.ts:196

Called when a source stops selecting the object this Script represents.

Parameters

_event

SelectEndEvent

The completed state and end reason. Call event.stopPropagation() to stop bubbling to ancestor Scripts.

Returns

void

Inherited from

VideoStream.onObjectSelectEnd

---

onObjectSelectStart()

onObjectSelectStart(_event): void

Defined in: src/core/Script.ts:190

Called when a source starts selecting the object this Script represents.

Parameters

_event

SelectEvent

event.target is the logical object and event.source.controller identifies the controller. Call event.stopPropagation() to stop bubbling to ancestor Scripts.

Returns

void

Inherited from

VideoStream.onObjectSelectStart

---

onObjectTouchEnd()

onObjectTouchEnd(_event): void

Defined in: src/core/Script.ts:244

Called when a hand's index finger stops touching this object. This ends the default selection lifecycle after the touch callback.

Parameters

_event

ObjectTouchEvent

Returns

void

Inherited from

VideoStream.onObjectTouchEnd

---

onObjectTouching()

onObjectTouching(_event): void

Defined in: src/core/Script.ts:239

Called every frame that a hand's index finger is touching this object. The object remains selected during these frames unless touch selection was prevented when contact started.

Parameters

_event

ObjectTouchEvent

Returns

void

Inherited from

VideoStream.onObjectTouching

---

onObjectTouchStart()

onObjectTouchStart(_event): void

Defined in: src/core/Script.ts:233

Called when a hand's index finger starts touching this object. Direct touch starts the object's selection lifecycle by default. Call event.preventDefault() to handle contact without selecting.

Parameters

_event

ObjectTouchStartEvent

Returns

void

Inherited from

VideoStream.onObjectTouchStart

---

onSelect()

onSelect(_event): void

Defined in: src/core/Script.ts:144

Called whenever pinch / mouse click successfully completes, globally.

Parameters

_event

SelectEvent

The interaction source and completed target.

Returns

void

Inherited from

VideoStream.onSelect

---

onSelectEnd()

onSelectEnd(_event): void

Defined in: src/core/Script.ts:138

Called whenever pinch / mouse click discontinues, globally.

Parameters

_event

SelectEndEvent

The completed state and end reason.

Returns

void

Inherited from

VideoStream.onSelectEnd

---

onSelecting()

onSelecting(_event): void

Defined in: src/core/Script.ts:149

Called whenever pinch / mouse click is happening, globally.

Parameters

_event

SelectEvent

Returns

void

Inherited from

VideoStream.onSelecting

---

onSelectStart()

onSelectStart(_event): void

Defined in: src/core/Script.ts:132

Called whenever pinch / mouse click starts, globally.

Parameters

_event

SelectEvent

The interaction source and optional captured target.

Returns

void

Inherited from

VideoStream.onSelectStart

---

onSimulatorStarted()

onSimulatorStarted(): void

Defined in: src/core/Script.ts:124

Returns

void

Inherited from

VideoStream.onSimulatorStarted

---

onSqueeze()

onSqueeze(_event): void

Defined in: src/core/Script.ts:181

Called whenever gamepad trigger successfully completes, globally.

Parameters

_event

SelectEvent

event.source.controller identifies the controller.

Returns

void

Inherited from

VideoStream.onSqueeze

---

onSqueezeEnd()

onSqueezeEnd(_event): void

Defined in: src/core/Script.ts:170

Called whenever gamepad trigger stops, globally.

Parameters

_event

SelectEvent

event.source.controller identifies the controller.

Returns

void

Inherited from

VideoStream.onSqueezeEnd

---

onSqueezeStart()

onSqueezeStart(_event): void

Defined in: src/core/Script.ts:165

Called whenever gamepad trigger starts, globally.

Parameters

_event

SelectEvent

event.source.controller identifies the controller.

Returns

void

Inherited from

VideoStream.onSqueezeStart

---

onSqueezing()

onSqueezing(_event): void

Defined in: src/core/Script.ts:175

Called whenever gamepad is being triggered, globally.

Parameters

_event

SelectEvent

Returns

void

Inherited from

VideoStream.onSqueezing

---

onXRSessionEnded()

onXRSessionEnded(): void

Defined in: src/camera/XRDeviceCamera.ts:697

Returns

void

Overrides

VideoStream.onXRSessionEnded

---

onXRSessionStarted()

onXRSessionStarted(_session?): void

Defined in: src/core/Script.ts:121

Parameters

_session?

XRSession

Returns

void

Inherited from

VideoStream.onXRSessionStarted

---

physicsStep()

physicsStep(): void

Defined in: src/core/Script.ts:119

Returns

void

Inherited from

VideoStream.physicsStep

---

registerSimulatorCamera()

registerSimulatorCamera(simulatorCamera?): void

Defined in: src/camera/XRDeviceCamera.ts:520

Parameters

simulatorCamera?

SimulatorCameraSource

Returns

void

---

setDeviceId()

setDeviceId(deviceId): Promise<void>

Defined in: src/camera/XRDeviceCamera.ts:296

Sets the active camera by its device ID. Removes potentially conflicting constraints such as facingMode.

Parameters

deviceId

string

Device ID

Returns

Promise<void>

---

setFacingMode()

setFacingMode(facingMode): Promise<void>

Defined in: src/camera/XRDeviceCamera.ts:316

Sets the active camera by its facing mode ('user' or 'environment').

Parameters

facingMode

VideoFacingModeEnum

facing mode

Returns

Promise<void>

---

setRenderer()

setRenderer(renderer): void

Defined in: src/camera/XRDeviceCamera.ts:112

Sets the renderer reference, needed for WebXR camera access fallback.

Parameters

renderer

WebGLOrWebGPURenderer

Returns

void

---

setState_()

protected setState_(state, details): void

Defined in: src/video/VideoStream.ts:155

Sets the stream's state and dispatches a 'statechange' event.

Parameters

state

StreamState

The new state.

details

Additional data for the event payload.

VideoStreamDetails | XRDeviceCameraDetails

Returns

void

Inherited from

VideoStream.setState_

---

snapshotSourceAvailable_()

protected snapshotSourceAvailable_(): boolean

Defined in: src/camera/XRDeviceCamera.ts:427

Whether the current snapshot source has pixels available. Subclasses may override this to provide non-video sources while preserving getSnapshot's format handling.

Returns

boolean

Overrides

VideoStream.snapshotSourceAvailable_

---

stop_()

protected stop_(): void

Defined in: src/video/VideoStream.ts:343

Stops the current video stream tracks.

Returns

void

Inherited from

VideoStream.stop_

---

update()

update(_time?, _frame?): void

Defined in: src/core/Script.ts:113

Runs per frame.

Parameters

_time?

number

_frame?

XRFrame

Returns

void

Inherited from

VideoStream.update

---

updateXRCamera()

updateXRCamera(frame): void

Defined in: src/camera/XRDeviceCamera.ts:456

Updates the camera texture from the WebXR Raw Camera Access API. Must be called each frame from the render loop when in XR camera mode.

Parameters

frame

XRFrame

Returns

void

---

waitForFreshFrame()

waitForFreshFrame(timeoutMs): Promise<VideoFrameMetadata | null>

Defined in: src/video/VideoStream.ts:216

Waits for the next new video frame before returning, so a subsequent getSnapshot reads fresh pixels instead of whatever (possibly stale) frame the <video> element currently holds. Hidden or non-composited video elements — the normal situation inside an immersive XR session — can be throttled by the browser, in which case the held frame may be arbitrarily old.

Resolves with the frame's metadata (whose captureTime, when present, dates the pixels in the performance.now() timebase), or null when the signal is unavailable (requestVideoFrameCallback unsupported, no active media stream) or no frame arrived within timeoutMs.

Parameters

timeoutMs

number = 400

Returns

Promise<VideoFrameMetadata | null>

Inherited from

VideoStream.waitForFreshFrame