Class: ObjectDetector

Defined in: src/world/objects/ObjectDetector.ts:20

Detects objects in the user's environment using a specified backend. It queries an AI model with the device camera feed and returns located objects with 2D and 3D positioning data.

Extends

• Script

Constructors

Constructor

new ObjectDetector(): ObjectDetector

Defined in: node_modules/@types/three/src/core/Object3D.d.ts:95

This creates a new Object3D object.

Returns

ObjectDetector

Inherited from

Script.constructor

Properties

isXRScript

isXRScript: boolean = true

Defined in: src/core/Script.ts:53

Inherited from

Script.isXRScript

---

ux

ux: UX

Defined in: src/core/Script.ts:52

Inherited from

Script.ux

---

dependencies

static dependencies: object

Defined in: src/world/objects/ObjectDetector.ts:21

ai

ai: typeof AI = AI

aiOptions

aiOptions: typeof AIOptions = AIOptions

camera

camera: typeof Camera = THREE.Camera

depth

depth: typeof Depth = Depth

deviceCamera

deviceCamera: typeof XRDeviceCamera = XRDeviceCamera

options

options: typeof WorldOptions = WorldOptions

Methods

clear()

clear(): ObjectDetector

Defined in: src/world/objects/ObjectDetector.ts:241

Removes all currently detected objects from the scene and internal tracking.

Returns

ObjectDetector

Overrides

Script.clear

---

dispose()

dispose(): void

Defined in: src/core/Script.ts:196

Called when the script is removed from the scene. Opposite of init.

Returns

void

Inherited from

Script.dispose

---

get()

get(label): DetectedObject[]

Defined in: src/world/objects/ObjectDetector.ts:229

Retrieves a list of currently detected objects.

Parameters

label

null = null

The semantic label to filter by (e.g., 'chair'). If null, all objects are returned.

Returns

DetectedObject[]

An array of Object instances.

---

init()

init(__namedParameters): void

Defined in: src/world/objects/ObjectDetector.ts:54

Initializes the ObjectDetector.

Parameters

__namedParameters

ai

AI

aiOptions

AIOptions

camera

Camera

depth

Depth

deviceCamera

XRDeviceCamera

options

WorldOptions

Returns

void

Overrides

Script.init

---

initPhysics()

initPhysics(_physics): void | Promise<void>

Defined in: src/core/Script.ts:72

Enables depth-aware interactions with physics. See /demos/ballpit

Parameters

_physics

Physics

Returns

void | Promise<void>

Inherited from

Script.initPhysics

---

onHoverEnter()

onHoverEnter(_controller): void

Defined in: src/core/Script.ts:157

Called when the controller starts hovering over this object with reticle.

Parameters

_controller

Object3D

An XR controller.

Returns

void

Inherited from

Script.onHoverEnter

---

onHoverExit()

onHoverExit(_controller): void

Defined in: src/core/Script.ts:162

Called when the controller hovers over this object with reticle.

Parameters

_controller

Object3D

An XR controller.

Returns

void

Inherited from

Script.onHoverExit

---

onHovering()

onHovering(_controller): void

Defined in: src/core/Script.ts:167

Called when the controller hovers over this object with reticle.

Parameters

_controller

Object3D

An XR controller.

Returns

void

Inherited from

Script.onHovering

---

onKeyDown()

onKeyDown(_event): void

Defined in: src/core/Script.ts:109

Called on keyboard keypress.

Parameters

_event

KeyEvent

Event containing .code to read the keyboard key.

Returns

void

Inherited from

Script.onKeyDown

---

onKeyUp()

onKeyUp(_event): void

Defined in: src/core/Script.ts:110

Parameters

_event

KeyEvent

Returns

void

Inherited from

Script.onKeyUp

---

onObjectGrabbing()

onObjectGrabbing(_event): void

Defined in: src/core/Script.ts:187

Called every frame a hand is grabbing this object.

Parameters

_event

ObjectGrabEvent

Returns

void

Inherited from

Script.onObjectGrabbing

---

onObjectGrabEnd()

onObjectGrabEnd(_event): void

Defined in: src/core/Script.ts:191

Called when a hand stops grabbing this object.

Parameters

_event

ObjectGrabEvent

Returns

void

Inherited from

Script.onObjectGrabEnd

---

onObjectGrabStart()

onObjectGrabStart(_event): void

Defined in: src/core/Script.ts:183

Called when a hand starts grabbing this object (touching + pinching).

Parameters

_event

ObjectGrabEvent

Returns

void

Inherited from

Script.onObjectGrabStart

---

onObjectSelectEnd()

onObjectSelectEnd(_event): boolean

Defined in: src/core/Script.ts:150

Called when the controller stops selecting this object the script represents, e.g. View, ModelView.

Parameters

_event

SelectEvent

event.target holds its controller.

Returns

boolean

Whether the event was handled

Inherited from

Script.onObjectSelectEnd

---

onObjectSelectStart()

onObjectSelectStart(_event): boolean

Defined in: src/core/Script.ts:141

Called when the controller starts selecting this object the script represents, e.g. View, ModelView.

Parameters

_event

SelectEvent

event.target holds its controller.

Returns

boolean

Whether the event was handled

Inherited from

Script.onObjectSelectStart

---

onObjectTouchEnd()

onObjectTouchEnd(_event): void

Defined in: src/core/Script.ts:179

Called when a hand's index finger stops touching this object.

Parameters

_event

ObjectTouchEvent

Returns

void

Inherited from

Script.onObjectTouchEnd

---

onObjectTouching()

onObjectTouching(_event): void

Defined in: src/core/Script.ts:175

Called every frame that a hand's index finger is touching this object.

Parameters

_event

ObjectTouchEvent

Returns

void

Inherited from

Script.onObjectTouching

---

onObjectTouchStart()

onObjectTouchStart(_event): void

Defined in: src/core/Script.ts:171

Called when a hand's index finger starts touching this object.

Parameters

_event

ObjectTouchEvent

Returns

void

Inherited from

Script.onObjectTouchStart

---

onSelect()

onSelect(_event): void

Defined in: src/core/Script.ts:98

Called whenever pinch / mouse click successfully completes, globally.

Parameters

_event

SelectEvent

event.target holds its controller.

Returns

void

Inherited from

Script.onSelect

---

onSelectEnd()

onSelectEnd(_event): void

Defined in: src/core/Script.ts:92

Called whenever pinch / mouse click discontinues, globally.

Parameters

_event

SelectEvent

event.target holds its controller

Returns

void

Inherited from

Script.onSelectEnd

---

onSelecting()

onSelecting(_event): void

Defined in: src/core/Script.ts:103

Called whenever pinch / mouse click is happening, globally.

Parameters

_event

SelectEvent

Returns

void

Inherited from

Script.onSelecting

---

onSelectStart()

onSelectStart(_event): void

Defined in: src/core/Script.ts:86

Called whenever pinch / mouse click starts, globally.

Parameters

_event

SelectEvent

event.target holds its controller

Returns

void

Inherited from

Script.onSelectStart

---

onSimulatorStarted()

onSimulatorStarted(): void

Defined in: src/core/Script.ts:78

Returns

void

Inherited from

Script.onSimulatorStarted

---

onSqueeze()

onSqueeze(_event): void

Defined in: src/core/Script.ts:132

Called whenever gamepad trigger successfully completes, globally.

Parameters

_event

SelectEvent

event.target holds its controller.

Returns

void

Inherited from

Script.onSqueeze

---

onSqueezeEnd()

onSqueezeEnd(_event): void

Defined in: src/core/Script.ts:121

Called whenever gamepad trigger stops, globally.

Parameters

_event

SelectEvent

event.target holds its controller.

Returns

void

Inherited from

Script.onSqueezeEnd

---

onSqueezeStart()

onSqueezeStart(_event): void

Defined in: src/core/Script.ts:116

Called whenever gamepad trigger starts, globally.

Parameters

_event

SelectEvent

event.target holds its controller.

Returns

void

Inherited from

Script.onSqueezeStart

---

onSqueezing()

onSqueezing(_event): void

Defined in: src/core/Script.ts:126

Called whenever gamepad is being triggered, globally.

Parameters

_event

SelectEvent

Returns

void

Inherited from

Script.onSqueezing

---

onXRSessionEnded()

onXRSessionEnded(): void

Defined in: src/core/Script.ts:76

Returns

void

Inherited from

Script.onXRSessionEnded

---

onXRSessionStarted()

onXRSessionStarted(_session?): void

Defined in: src/core/Script.ts:75

Parameters

_session?

XRSession

Returns

void

Inherited from

Script.onXRSessionStarted

---

physicsStep()

physicsStep(): void

Defined in: src/core/Script.ts:73

Returns

void

Inherited from

Script.physicsStep

---

runDetection()

runDetection(): Promise<(undefined | null | DetectedObject)[]>

Defined in: src/world/objects/ObjectDetector.ts:83

Runs the object detection process based on the configured backend.

Returns

Promise<(undefined | null | DetectedObject)[]>

A promise that resolves with an array of detected DetectedObject instances.

---

showDebugVisualizations()

showDebugVisualizations(visible): void

Defined in: src/world/objects/ObjectDetector.ts:256

Toggles the visibility of all debug visualizations for detected objects.

Parameters

visible

boolean = true

Whether the visualizations should be visible.

Returns

void

---

update()

update(_time?, _frame?): void

Defined in: src/core/Script.ts:67

Runs per frame.

Parameters

_time?

number

_frame?

XRFrame

Returns

void

Inherited from

Script.update