XR8.FaceController.pipelineModule()
XR8.FaceController.pipelineModule()
Description​
Creates a camera pipeline module that, when installed, receives callbacks on when the camera has started, camera proessing events, and other state changes. These are used to calculate the camera's position.
Parameters​
None
Returns​
Return value is an object made available to onUpdate as:
processCpuResult.facecontroller: { rotation, position, intrinsics, cameraFeedTexture }
| Property | Type | Description |
|---|---|---|
| rotation | {w, x, y, z} | The orientation (quaternion) of the camera in the scene. |
| position | {x, y, z} | The position of the camera in the scene. |
| intrinsics | [Number] | A 16 dimensional column-major 4x4 projection matrix that gives the scene camera the same field of view as the rendered camera feed. |
| cameraFeedTexture | WebGLTexture | The texture containing camera feed data. |
Dispatched Events​
faceloading: Fires when loading begins for additional face AR resources.
faceloading.detail : {maxDetections, pointsPerDetection, indices, uvs}
| Property | Type | Description |
|---|---|---|
| maxDetections | Number | The maximum number of faces that can be simultaneously processed. |
| pointsPerDetection | Number | Number of vertices that will be extracted per face. |
| indices | [{a, b, c}] | The list of indexes into the vertices array that form the triangles of the requested mesh, as specified with meshGeometry in XR8.FaceController.configure(). |
| uvs | [{u, v}] | The list of uv positions into a texture map corresponding to the returned vertex points. |
facescanning: Fires when all face AR resources have been loaded and scanning has begun.
facescanning.detail : {maxDetections, pointsPerDetection, indices, uvs}
| Property | Type | Description |
|---|---|---|
| maxDetections | Number | The maximum number of faces that can be simultaneously processed. |
| pointsPerDetection | Number | Number of vertices that will be extracted per face. |
| indices | [{a, b, c}] | The list of indexes into the vertices array that form the triangles of the requested mesh, as specified with meshGeometry in XR8.FaceController.configure(). |
| uvs | [{u, v}] | The list of uv positions into a texture map corresponding to the returned vertex points. |
facefound: Fires when a face first found.
facefound.detail : {id, transform, vertices, normals, attachmentPoints}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
| transform | {position, rotation, scale, scaledWidth, scaledHeight, scaledDepth} | Transform information of the located face. |
| vertices | [{x, y, z}] | Position of face points, relative to transform. |
| normals | [{x, y, z}] | Normal direction of vertices, relative to transform. |
| attachmentPoints | {name, position: {x,y,z}} | See XR8.FaceController.AttachmentPoints for list of available attachment points. position is relative to the transform. |
| uvsInCameraFrame | [{u, v}] | The list of uv positions in the camera frame corresponding to the returned vertex points. |
transform is an object with the following properties:
| Property | Type | Description |
|---|---|---|
| position | {x, y, z} | The 3d position of the located face. |
| rotation | {w, x, y, z} | The 3d local orientation of the located face. |
| scale | Number | A scale factor that should be applied to objects attached to this face. |
| scaledWidth | Number | Approximate width of the head in the scene when multiplied by scale. |
| scaledHeight | Number | Approximate height of the head in the scene when multiplied by scale. |
| scaledDepth | Number | Approximate depth of the head in the scene when multiplied by scale. |
faceupdated: Fires when a face is subsequently found.
faceupdated.detail : {id, transform, vertices, normals, attachmentPoints}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
| transform | {position, rotation, scale, scaledWidth, scaledHeight, scaledDepth} | Transform information of the located face. |
| vertices | [{x, y, z}] | Position of face points, relative to transform. |
| normals | [{x, y, z}] | Normal direction of vertices, relative to transform. |
| attachmentPoints | {name, position: {x,y,z}} | See XR8.FaceController.AttachmentPoints for list of available attachment points. position is relative to the transform. |
| uvsInCameraFrame | [{u, v}] | The list of uv positions in the camera frame corresponding to the returned vertex points. |
transform is an object with the following properties:
| Property | Type | Description |
|---|---|---|
| position | {x, y, z} | The 3d position of the located face. |
| rotation | {w, x, y, z} | The 3d local orientation of the located face. |
| scale | Number | A scale factor that should be applied to objects attached to this face. |
| scaledWidth | Number | Approximate width of the head in the scene when multiplied by scale. |
| scaledHeight | Number | Approximate height of the head in the scene when multiplied by scale. |
| scaledDepth | Number | Approximate depth of the head in the scene when multiplied by scale. |
facelost: Fires when a face is no longer being tracked.
facelost.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
mouthopened: Fires when a tracked face's mouth opens.
mouthopened.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
mouthclosed: Fires when a tracked face's mouth closes.
mouthclosed.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
lefteyeopened: Fires when a tracked face's left eye opens.
lefteyeopened.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
lefteyeclosed: Fires when a tracked face's left eye closes.
lefteyeclosed.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
righteyeopened: Fires when a tracked face's right eye opens.
righteyeopened.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
righteyeclosed: Fires when a tracked face's right eye closes.
righteyeclosed.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
lefteyebrowraised: Fires when a tracked face's left eyebrow is raised from its initial position when the face was found.
lefteyebrowraised.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
lefteyebrowlowered: Fires when a tracked face's left eyebrow is lowered to its initial position when the face was found.
lefteyebrowlowered.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
righteyebrowraised: Fires when a tracked face's right eyebrow is raised from its position when the face was found.
righteyebrowraised.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
righteyebrowlowered: Fires when a tracked face's right eyebrow is lowered to its initial position when the face was found.
righteyebrowlowered.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
lefteyewinked: Fires when a tracked face's left eye closes and opens within 750ms while the right eye remains open.
lefteyewinked.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
righteyewinked: Fires when a tracked face's right eye closes and opens within 750ms while the left eye remains open.
righteyewinked.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
blinked: Fires when a tracked face's eyes blink.
blinked.detail : { id }
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
interpupillarydistance: Fires when a tracked face's distance in millimeters between the centers of each pupil is first detected.
interpupillarydistance.detail : {id, interpupillaryDistance}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face. |
| interpupillaryDistance | Number | Approximate distance in millimeters between the centers of each pupil. |
When enableEars:true ear detection runs simultaneously with Face Effects and dispatches the following events:
earfound: Fires when an ear is first found.
earfound.detail : {id, ear}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face which the ear is attached to. |
| ear | String | Can be either left or right. |
earpointfound: Fires when an ear attachmentPoint is first found.
earpointfound.detail : {id, point}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face which the ear attachmentPoints is attached to. |
| point | String | Can be either leftHelix, leftCanal, leftLobe, rightHelix, rightCanal, or rightLobe. |
earlost: Fires when an ear is no longer being tracked.
earlost.detail : {id, ear}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face which the ear is attached to. |
| ear | String | Can be either left or right. |
earpointlost: Fires when an ear attachmentPoint is no longer being tracked.
earpointlost.detail : {id, point}
| Property | Type | Description |
|---|---|---|
| id | Number | A numerical id of the located face which the ear attachmentPoints is attached to. |
| point | String | Can be either leftHelix, leftCanal, leftLobe, rightHelix, rightCanal, or rightLobe. |
Example - adding pipeline module​
XR8.addCameraPipelineModule(XR8.FaceController.pipelineModule())