Class: ObjectDetector
Defined in: src/world/objects/ObjectDetector.ts:47
Detects objects in the user's environment using a specified backend. It queries an AI model with the device camera feed and returns located objects with 2D and 3D positioning data.
Extends
Constructors
Constructor
new ObjectDetector():
ObjectDetector
Defined in: node_modules/@types/three/src/core/Object3D.d.ts:98
This creates a new Object3D object.
Returns
ObjectDetector
Inherited from
Properties
detectedObjects
detectedObjects:
DetectedObject<unknown>[] =[]
Defined in: src/world/objects/ObjectDetector.ts:76
The latest detected objects.
isXRScript
isXRScript:
boolean=true
Defined in: src/core/Script.ts:54
Inherited from
targetDevice
targetDevice:
string='galaxyxr'
Defined in: src/world/objects/ObjectDetector.ts:93
Target device profile used to look up RGB camera intrinsics and pose
for converting detection bounding boxes into world space. Defaults to
'galaxyxr'; auto-overridden to 'quest3' in init when the
Meta Quest browser is detected. Can be overridden manually before init.
ux
ux:
UX
Defined in: src/core/Script.ts:53
Inherited from
dependencies
staticdependencies:object
Defined in: src/world/objects/ObjectDetector.ts:48
ai
ai: typeof
AI=AI
aiOptions
aiOptions: typeof
AIOptions=AIOptions
camera
camera: typeof
Camera=THREE.Camera
depth
depth: typeof
Depth=Depth
deviceCamera
deviceCamera: typeof
XRDeviceCamera=XRDeviceCamera
options
options: typeof
WorldOptions=WorldOptions
renderer
renderer: typeof
WebGLRenderer=THREE.WebGLRenderer
Methods
clear()
clear():
ObjectDetector
Defined in: src/world/objects/ObjectDetector.ts:352
Removes all currently detected objects from the scene and internal tracking.
Returns
ObjectDetector
Overrides
Script.clear
dispose()
dispose():
void
Defined in: src/core/Script.ts:196
Called when the script is removed from the scene. Opposite of init.
Returns
void
Inherited from
get()
get<
T>(label):DetectedObject<T>[]
Defined in: src/world/objects/ObjectDetector.ts:338
Retrieves a list of currently detected objects.
Type Parameters
T
T = null
Parameters
label
null = null
The semantic label to filter by (e.g., 'chair'). If null, all objects are returned.
Returns
DetectedObject<T>[]
An array of Object instances.
init()
init(
__namedParameters):void
Defined in: src/world/objects/ObjectDetector.ts:99
Initializes the ObjectDetector.
Parameters
__namedParameters
ai
aiOptions
camera
depth
deviceCamera
options
renderer
Returns
void
Overrides
initPhysics()
initPhysics(
_physics):void|Promise<void>
Defined in: src/core/Script.ts:73
Enables depth-aware interactions with physics. See /demos/ballpit
Parameters
_physics
Returns
void | Promise<void>
Inherited from
onHoverEnter()
onHoverEnter(
_controller):boolean|void
Defined in: src/core/Script.ts:155
Called when the controller starts hovering over this object with reticle.
Parameters
_controller
An XR controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onHoverExit()
onHoverExit(
_controller):boolean|void
Defined in: src/core/Script.ts:161
Called when the controller hovers over this object with reticle.
Parameters
_controller
An XR controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onHovering()
onHovering(
_controller):boolean|void
Defined in: src/core/Script.ts:167
Called when the controller hovers over this object with reticle.
Parameters
_controller
An XR controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onKeyDown()
onKeyDown(
_event):void
Defined in: src/core/Script.ts:110
Called on keyboard keypress.
Parameters
_event
Event containing .code to read the keyboard key.
Returns
void
Inherited from
onKeyUp()
onKeyUp(
_event):void
Defined in: src/core/Script.ts:111
Parameters
_event
Returns
void
Inherited from
onObjectGrabbing()
onObjectGrabbing(
_event):void
Defined in: src/core/Script.ts:187
Called every frame a hand is grabbing this object.
Parameters
_event
Returns
void
Inherited from
onObjectGrabEnd()
onObjectGrabEnd(
_event):void
Defined in: src/core/Script.ts:191
Called when a hand stops grabbing this object.
Parameters
_event
Returns
void
Inherited from
onObjectGrabStart()
onObjectGrabStart(
_event):void
Defined in: src/core/Script.ts:183
Called when a hand starts grabbing this object (touching + pinching).
Parameters
_event
Returns
void
Inherited from
onObjectSelectEnd()
onObjectSelectEnd(
_event):boolean|void
Defined in: src/core/Script.ts:149
Called when the controller stops selecting this object the script represents, e.g. View, ModelView.
Parameters
_event
event.target holds its controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onObjectSelectStart()
onObjectSelectStart(
_event):boolean|void
Defined in: src/core/Script.ts:142
Called when the controller starts selecting this object the script represents, e.g. View, ModelView.
Parameters
_event
event.target holds its controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onObjectTouchEnd()
onObjectTouchEnd(
_event):void
Defined in: src/core/Script.ts:179
Called when a hand's index finger stops touching this object.
Parameters
_event
Returns
void
Inherited from
onObjectTouching()
onObjectTouching(
_event):void
Defined in: src/core/Script.ts:175
Called every frame that a hand's index finger is touching this object.
Parameters
_event
Returns
void
Inherited from
onObjectTouchStart()
onObjectTouchStart(
_event):void
Defined in: src/core/Script.ts:171
Called when a hand's index finger starts touching this object.
Parameters
_event
Returns
void
Inherited from
onSelect()
onSelect(
_event):void
Defined in: src/core/Script.ts:99
Called whenever pinch / mouse click successfully completes, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSelectEnd()
onSelectEnd(
_event):void
Defined in: src/core/Script.ts:93
Called whenever pinch / mouse click discontinues, globally.
Parameters
_event
event.target holds its controller
Returns
void
Inherited from
onSelecting()
onSelecting(
_event):void
Defined in: src/core/Script.ts:104
Called whenever pinch / mouse click is happening, globally.
Parameters
_event
Returns
void
Inherited from
onSelectStart()
onSelectStart(
_event):void
Defined in: src/core/Script.ts:87
Called whenever pinch / mouse click starts, globally.
Parameters
_event
event.target holds its controller
Returns
void
Inherited from
onSimulatorStarted()
onSimulatorStarted():
void
Defined in: src/core/Script.ts:79
Returns
void
Inherited from
onSqueeze()
onSqueeze(
_event):void
Defined in: src/core/Script.ts:133
Called whenever gamepad trigger successfully completes, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSqueezeEnd()
onSqueezeEnd(
_event):void
Defined in: src/core/Script.ts:122
Called whenever gamepad trigger stops, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSqueezeStart()
onSqueezeStart(
_event):void
Defined in: src/core/Script.ts:117
Called whenever gamepad trigger starts, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSqueezing()
onSqueezing(
_event):void
Defined in: src/core/Script.ts:127
Called whenever gamepad is being triggered, globally.
Parameters
_event
Returns
void
Inherited from
onXRSessionEnded()
onXRSessionEnded():
void
Defined in: src/core/Script.ts:77
Returns
void
Inherited from
onXRSessionStarted()
onXRSessionStarted(
_session?):void
Defined in: src/core/Script.ts:76
Parameters
_session?
Returns
void
Inherited from
physicsStep()
physicsStep():
void
Defined in: src/core/Script.ts:74
Returns
void
Inherited from
runDetection()
runDetection<
T>():Promise<DetectedObject<T>[]>
Defined in: src/world/objects/ObjectDetector.ts:212
Runs object detection or returns the ongoing detection promise.
- If continuous detection is started (has active clients), returns the promise for the next detection result.
- If continuous detection is not started, performs a one-off detection and returns the result. If a one-off detection is already in progress, returns the promise for that ongoing detection.
Type Parameters
T
T = null
Returns
Promise<DetectedObject<T>[]>
A promise that resolves with an
array of detected DetectedObject instances.
showDebugVisualizations()
showDebugVisualizations(
visible):void
Defined in: src/world/objects/ObjectDetector.ts:372
Toggles the visibility of all debug visualizations for detected objects.
Parameters
visible
boolean = true
Whether the visualizations should be visible.
Returns
void
start()
start(
client):void
Defined in: src/world/objects/ObjectDetector.ts:145
Starts continuous object detection for the given client. If this is the first client, starts the background detection loop.
Parameters
client
object
The client object requesting object detection.
Returns
void
stop()
stop(
client):void
Defined in: src/world/objects/ObjectDetector.ts:160
Stops continuous object detection for the given client. If this was the last client, stops the background detection loop.
Parameters
client
object
The client object that no longer needs object detection.
Returns
void
update()
update():
void
Defined in: src/world/objects/ObjectDetector.ts:168
Called per frame by the engine. If there are active clients, ensures the continuous object detection is running.
Returns
void