Class: Segmenter
Defined in: src/world/segmentation/Segmenter.ts:24
A Script that runs semantic segmentation on the device camera feed and returns a per-pixel category mask (SegmentationMask).
Mirrors HumanRecognizer / ObjectDetector, but without any depth or
world-space step, segmentation is a pure 2D camera-to-mask operation, so it
does not depend on the depth mesh or camera intrinsics.
Multiple concurrent calls to runSegmentation within the same async cycle are coalesced: only one MediaPipe inference is dispatched per cycle and its result is shared with all callers. The latest completed mask is also available synchronously via latestMask.
Extends
Constructors
Constructor
new Segmenter():
Segmenter
Defined in: node_modules/@types/three/src/core/Object3D.d.ts:98
This creates a new Object3D object.
Returns
Segmenter
Inherited from
Properties
isXRScript
isXRScript:
boolean=true
Defined in: src/core/Script.ts:54
Inherited from
ux
ux:
UX
Defined in: src/core/Script.ts:53
Inherited from
dependencies
staticdependencies:object
Defined in: src/world/segmentation/Segmenter.ts:25
deviceCamera
deviceCamera: typeof
XRDeviceCamera=XRDeviceCamera
options
options: typeof
WorldOptions=WorldOptions
Accessors
latestMask
Get Signature
get latestMask():
SegmentationMask|null
Defined in: src/world/segmentation/Segmenter.ts:65
The latest cached segmentation mask from the most recently completed
inference pass. Returns null until the first inference finishes.
Returns
SegmentationMask | null
Methods
dispose()
dispose():
void
Defined in: src/core/Script.ts:196
Called when the script is removed from the scene. Opposite of init.
Returns
void
Inherited from
init()
init(
__namedParameters):void
Defined in: src/world/segmentation/Segmenter.ts:50
Initializes an instance with XR controllers, grips, hands, raycaster, and default options. We allow all scripts to quickly access its user (e.g., user.isSelecting(), user.hands), world (e.g., physical depth mesh, lighting estimation, and recognized objects), and scene (the root of three.js's scene graph). If this returns a promise, we will wait for it.
Parameters
__namedParameters
deviceCamera
options
Returns
void
Overrides
initPhysics()
initPhysics(
_physics):void|Promise<void>
Defined in: src/core/Script.ts:73
Enables depth-aware interactions with physics. See /demos/ballpit
Parameters
_physics
Returns
void | Promise<void>
Inherited from
onHoverEnter()
onHoverEnter(
_controller):boolean|void
Defined in: src/core/Script.ts:155
Called when the controller starts hovering over this object with reticle.
Parameters
_controller
An XR controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onHoverExit()
onHoverExit(
_controller):boolean|void
Defined in: src/core/Script.ts:161
Called when the controller hovers over this object with reticle.
Parameters
_controller
An XR controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onHovering()
onHovering(
_controller):boolean|void
Defined in: src/core/Script.ts:167
Called when the controller hovers over this object with reticle.
Parameters
_controller
An XR controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onKeyDown()
onKeyDown(
_event):void
Defined in: src/core/Script.ts:110
Called on keyboard keypress.
Parameters
_event
Event containing .code to read the keyboard key.
Returns
void
Inherited from
onKeyUp()
onKeyUp(
_event):void
Defined in: src/core/Script.ts:111
Parameters
_event
Returns
void
Inherited from
onObjectGrabbing()
onObjectGrabbing(
_event):void
Defined in: src/core/Script.ts:187
Called every frame a hand is grabbing this object.
Parameters
_event
Returns
void
Inherited from
onObjectGrabEnd()
onObjectGrabEnd(
_event):void
Defined in: src/core/Script.ts:191
Called when a hand stops grabbing this object.
Parameters
_event
Returns
void
Inherited from
onObjectGrabStart()
onObjectGrabStart(
_event):void
Defined in: src/core/Script.ts:183
Called when a hand starts grabbing this object (touching + pinching).
Parameters
_event
Returns
void
Inherited from
onObjectSelectEnd()
onObjectSelectEnd(
_event):boolean|void
Defined in: src/core/Script.ts:149
Called when the controller stops selecting this object the script represents, e.g. View, ModelView.
Parameters
_event
event.target holds its controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onObjectSelectStart()
onObjectSelectStart(
_event):boolean|void
Defined in: src/core/Script.ts:142
Called when the controller starts selecting this object the script represents, e.g. View, ModelView.
Parameters
_event
event.target holds its controller.
Returns
boolean | void
Whether the event was handled. If true, the event will not bubble up.
Inherited from
onObjectTouchEnd()
onObjectTouchEnd(
_event):void
Defined in: src/core/Script.ts:179
Called when a hand's index finger stops touching this object.
Parameters
_event
Returns
void
Inherited from
onObjectTouching()
onObjectTouching(
_event):void
Defined in: src/core/Script.ts:175
Called every frame that a hand's index finger is touching this object.
Parameters
_event
Returns
void
Inherited from
onObjectTouchStart()
onObjectTouchStart(
_event):void
Defined in: src/core/Script.ts:171
Called when a hand's index finger starts touching this object.
Parameters
_event
Returns
void
Inherited from
onSelect()
onSelect(
_event):void
Defined in: src/core/Script.ts:99
Called whenever pinch / mouse click successfully completes, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSelectEnd()
onSelectEnd(
_event):void
Defined in: src/core/Script.ts:93
Called whenever pinch / mouse click discontinues, globally.
Parameters
_event
event.target holds its controller
Returns
void
Inherited from
onSelecting()
onSelecting(
_event):void
Defined in: src/core/Script.ts:104
Called whenever pinch / mouse click is happening, globally.
Parameters
_event
Returns
void
Inherited from
onSelectStart()
onSelectStart(
_event):void
Defined in: src/core/Script.ts:87
Called whenever pinch / mouse click starts, globally.
Parameters
_event
event.target holds its controller
Returns
void
Inherited from
onSimulatorStarted()
onSimulatorStarted():
void
Defined in: src/core/Script.ts:79
Returns
void
Inherited from
onSqueeze()
onSqueeze(
_event):void
Defined in: src/core/Script.ts:133
Called whenever gamepad trigger successfully completes, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSqueezeEnd()
onSqueezeEnd(
_event):void
Defined in: src/core/Script.ts:122
Called whenever gamepad trigger stops, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSqueezeStart()
onSqueezeStart(
_event):void
Defined in: src/core/Script.ts:117
Called whenever gamepad trigger starts, globally.
Parameters
_event
event.target holds its controller.
Returns
void
Inherited from
onSqueezing()
onSqueezing(
_event):void
Defined in: src/core/Script.ts:127
Called whenever gamepad is being triggered, globally.
Parameters
_event
Returns
void
Inherited from
onXRSessionEnded()
onXRSessionEnded():
void
Defined in: src/core/Script.ts:77
Returns
void
Inherited from
onXRSessionStarted()
onXRSessionStarted(
_session?):void
Defined in: src/core/Script.ts:76
Parameters
_session?
Returns
void
Inherited from
physicsStep()
physicsStep():
void
Defined in: src/core/Script.ts:74
Returns
void
Inherited from
runSegmentation()
runSegmentation():
Promise<SegmentationMask|null>
Defined in: src/world/segmentation/Segmenter.ts:104
Runs one segmentation pass over the current camera frame, or returns the result of the in-flight pass when one is already running. Multiple callers in the same async cycle share a single MediaPipe inference rather than each triggering their own.
Under normal usage consumers should poll latestMask (kept fresh by the automatic loop) rather than calling this directly.
Returns
Promise<SegmentationMask | null>
The mask, or null if the backend or camera frame is not ready.
update()
update(
time):void
Defined in: src/world/segmentation/Segmenter.ts:85
Continuous throttled loop driven by the engine frame tick.
Called every frame by ScriptsManager (via Core.update → scriptsManager.update).
Kicks off a fresh inference pass at most once per
options.segmentation.pollingIntervalMs milliseconds. The in-flight guard
prevents stacking: if a previous inference is still running the tick is
silently skipped rather than launching a second one.
After each completed inference latestMask is updated so all consumers in the same frame read the same cached result without each triggering their own MediaPipe run.
Parameters
time
number
Current timestamp in milliseconds, forwarded from the engine frame loop.
Returns
void