Application Settings: Live Reconstruction

Back To Page References


Reconstruction in motion capture is a process of deriving 3D points from 2D coordinate information obtained from captured images, and the Point Cloud is the core engine that runs the reconstruction process. The Reconstruction pane in Motive contains all of the settings and filter options to modify the engine's parameters, and it can be accessed under the View tab. These settings can be modified to optimize the quality of reconstruction depending on the conditions of the capture and what you're trying to achieve. You will find that this pane is used most often during Live mode, 2D playback, and when reconstructing a take in order to optimize the marker tracking for a given system.


POINT CLOUD

Multiple marker rays contributing to the reconstruction of a marker.

The Point Cloud reconstruction engine converts two dimensional point from camera images into coordinates in a three dimensional space through triangulation. All cameras should be calibrated for the engine to function properly (see Calibration). The triangulation of a marker occurs when a minimum of 2 rays intersect. Rays are generated from the objects present on a camera image and they resolve into a 3D point when the conditions defined by the reconstructions settings are met. These rays can be seen from the Perspective View pane when the tracked rays and untracked rays are enabled from the visibility Viewport16.png settings.

Due to inherent errors in marker tracking, rays generally do not converge perfectly on a single point in 3D space, so a tolerance value is defined. This tolerance, called the residual, represents one of the reconstruction constraints. If a ray could be defined as an infinite series of points aligned in a straight line, two or more rays that have points within the defined residual range (in mm) will form a marker.

Enable Point Cloud Reconstruction

Default: ON

This toggles real-time 3D reconstruction on and off.

It is recommended to turn this off if computer resource need to be dedicated to 2D recording.

OPTIONS

Reconstruction Pane.png

Residual (mm)

Default: 10.00 mm

The residual value sets the maximum allowable offset distance (in mm) between rays contributing to a single 3D point.

When the residual value is set too high, unassociated marker rays may contribute to marker reconstruction, and non-existing ghost markers may be reconstructed. When this value is set too low, the contributing rays within a marker could reconstruct multiple markers where there should only be one.


Choosing the Residual Value

Depending on the size of markers used, the contributing rays will converge with a varying tolerable offset. If you are working with smaller markers, set the residual value lower. If you're working with larger markers, set this value higher because the centroid rays will not converge as precisely as the smaller markers. A starting point is to set the residual value to the diameter of the smallest marker and go down from there until you start seeing ghost markers. For example, when 3 mm and 14 mm markers are captured in a same volume, set the residual value to less than 3 mm. The ghost markers can appear on larger markers if this value is set too low.

The residual can also be viewed as the minimum distance between two markers before they begin to merge. If two markers have a separation distance smaller than the defined residual (in mm), the contributing rays for each marker will be merged and only one marker will be reconstructed, which is undesirable. Remember that for a 3D point to be reconstructed, it needs to have at least two rays contributing to a marker depending on the Minimum Rays setting.

If calibration quality is not very good, you may need to set this value higher for increased tolerance. This will work only if your markers are further apart in the 2D views throughout the given marker motion. This is because there is more errors in the system. However, for best results, you should always work with a calibration with minimal error (See Calibration).

RC RayResiduals.png Reconstruction Residual.png


Minimum Angle (degrees)

Default: 5 degrees

The minimum allowable angle – in degrees from the marker's point of view – between the rays to consider them valid for marker reconstruction. This separation also represents the minimum distance required between the cameras.

For best results, cameras should be separated apart in order to capture unique views on the target volume. For example, if there are only two cameras, an ideal reconstruction would occur when the cameras are separated far enough so the rays converge with a 90 degree of an incident angle from the perspective of the reconstructed marker(s).

If you are working with a small number of cameras that are separated evenly around the capture volume, set this value lower (default 5 degree). On the other hand, if you have a large setup with numerous cameras placed close together, you can set this value a bit higher to ignore contributing rays that are coming from the cameras that are too close together.

Similar vantages obtained by the cameras in vicinity do not contribute unique positional data to the reconstruction, but they only increases the required amount of computation. Rays coming from very close cameras may increase the error in the reconstruction. Better reconstruction can only be achieved with a good, overall camera coverage (See Camera Placements).


Minimum Rays

Default: 2 rays

This sets the required minimum number of cameras that must see a marker for it to be reconstructed.

For a marker to be reconstructed, at least two or more cameras need to see the marker. The minimum rays setting defines the required number of cameras that must see a marker for it to be reconstructed. If you have 4 cameras and set this to 4, all cameras must see the marker; otherwise, the marker will not be reconstructed and the contributing rays will become the untracked rays.

When more rays are contributing to a marker, more accurate reconstruction can be achieved, but generally, you don't need all cameras in a setup to see a marker. If you have a lot of cameras capturing a marker, you can safely increase this setting to prevent false reconstructions which may come from 2 or 3 rays that happen to connect within the residual range. However, be careful when increasing this setting because a high number of minimum rays requirement may decrease the effective capture volume and increase the frequency of marker occlusions during capture.


Minimum Ray Length (m)

Default: 0.2 m

This sets the minimum distance, in meters, between a marker and a camera for the camera to contribute to the reconstruction of the marker. When ghost markers appear close to the camera lens, increase this setting to restrict the unwanted reconstructions in the vicinity. But for close-range tracking applications, this setting must be set low.


Maximum Ray Length (m)

Default: None — the calibration solver will set a suggested distance based on the wanding results, but this can still be adjusted by the user after calibration.

This sets the maximum distance, in meters, a marker can be from the camera to be considered for 3D reconstruction. In very large volumes with high resolution cameras, this value can be increased for a longer tracking range or to allow contributions from more cameras in the setup. This setting can also be reduced to filter out longer rays from reconstruction. Longer rays generally produce less accurate data than shorter rays.

When capturing in a large-size volume with a medium-size – 20 ~ 50 cameras – camera system, this setting can be adjusted for better tracking results. Tracking rays from cameras at the far end of the volume may be inaccurate for tracking markers on the opposite end of the volume, and the unstable rays may contribute to ghost marker reconstructions. In this case, lower the maximum ray length to restrict reconstruction contributions from cameras tracking at long distances. For captures vulnerable to frequent marker occlusions, adjusting this constraint is not recommended since more camera coverage is needed for preventing the occlusions. Note that lowering this setting can take a toll on performance at higher camera counts and marker counts because the solver has to perform numerous calculations per second to decide which rays are good.


Pixel Gutter (pixels)

Default: 0 pixels

Establishes a dead zone, in pixels, around the edge of the 2D camera image. Any 2D objects detected within this gutter will be discarded before calculating through the point cloud. In essence, it is a way of getting only the best data of the captured images, because markers seen at the edges of the camera sensor tend to have higher errors.

This setting can be increased in small amounts in order to accommodate for cases where lens distortions are potentially causing problem with tracking. Another use of the setting for limiting the amount of data going to the reconstruction solver, which may help when you have a lot of markers and/or cameras. Be careful adjusting this setting as the trimmed data can't be reacquired in post-processing pipelines.


Maximum 2D Objects

Default: 512 objects

The maximum number of 2D markers, per camera, that will be used for reconstruction. Increasing this will allow the cameras to detect more markers but with reduced performance as a tradeoff. In most cases, this setting should be left alone. If you have several hundreds of markers in a camera view, you may need to increase this setting if all the markers are not reconstructing. The absolute max that should be set is 1024.


Maximum Calculation Time (ms)

Default: 50 ms

Maximum amount of time Motive will spend attempting to reconstruct the point cloud per frame per camera group.The value is specified in milliseconds or 0.001 seconds.

Adjusting this value can help in situations where latency is causing issues with real time capture, and lowering the calculation time during recording can significantly decrease the likelihood of dropped frames during recording. However, reduced calculation times may also affect the accuracy of the data as a tradeoff.


2D OBJECT FILTER

Filter Type

Default: Size and Roundness

Toggles 2D object (Size and Roundness) filtering on or off.

This filter is very useful for filtering out extraneous reflections according to their characteristics (size and roundness) rather than blocking pixels using the masking tool or the Block Visible feature. Turn off this setting only when you want to use every 2D pixels above the brightness threshold from camera views. When there are extraneous or flickering reflections in the view, turn on the filter to specify and consider reflections only from markers. There are multiple filtering parameters to distinguish the marker reflections. Given that there are assumed marker characteristics, filtering parameters can be set. The size parameters can be defined to filter out extra-small or extra-large reflections that are most likely from extraneous sources other than markers. Non-circular reflections can be ignored assuming that all reflective markers have circular shapes. Note that even when applying the size and roundness filter, you should always Block Visible when you calibrate.


The size filter applied with a maximum size limit of 200 pixels. Any reflections bigger than 200 pixel-size is omitted from the Point Cloud reconstruction calculation.

Min Thresholded Pixels (pixels)

Default: 4 pixels

The minimum pixel size of a 2D object, a collection of pixels grouped together, for it to be included in the Point Cloud reconstruction. All pixels must first meet the brightness threshold defined in the Cameras pane in order to be grouped as a 2D object. This can be used to filter out small reflections that are flickering in the view. The default value for the minimum pixel size is 4, which means that more than 4 pixels need a group of pixels needs to be greater than 4 for a ray to be generated.


Max Thresholded Pixels (pixels)

Default: 2000 pixels

The maximum size of a 2D object, in pixels, in order for it to be included in point cloud reconstruction. Default is 2000 pixels which basically means that all of detected large reflections smaller than 2000 pixel-size will be included as a 2D object. Use this to filter out larger markers in a variable marker capture. For instance, if you have 4 mm markers on an actor's face and 14 mm markers on their body, use this setting to filter out the larger markers if the need arises.


The circle filter omitting non-circular reflections from a 2D camera view.

Circularity

Default: 0.6

The circularity value sets the 2D object filter sensitivity to the roundness of detected reflections. Valid range is between 0 and 1, with 0 being not round and 1 being perfectly round. This setting is relative to the resolution of the camera you are using. If you have lower resolution cameras, a marker will appear more pixelated and non-circular. If a marker is detected in only 5 pixels in a lower resolution camera at a set distance, the same marker will appear with more pixels from a high resolution camera and the captured reflection will have more circular perimeters.

When using lower resolution cameras to capture smaller markers at a long distance, you may need to lower the circularity filter value for the reflection to be considered as a 2D object from the camera view. Also, this setting may need to be lowered when tracking non-spherical markers in order to avoid filtering the reflections.


Intrusion Detection

Default: Normal

The intrusion detection feature allows cameras to recognize merged reflections and filter them out from the image. Merged reflections are susceptible to inaccurate centroid calculation and may result in jittering of the reconstructed markers. From 2D object images, both of the intruding and detected marker reflection will be omitted whenever there is an extraneous reflections in the vicinity, within the intrusion boundary. Available detection modes are None, Normal, and Aggressive, and by default, Normal intrusion detection mode is recommended.

Intrusion Detection Modes

  • None: Intrusion detection is disabled.
  • Normal (Default): Creates a circular boundary, an Intrusion Band, around the detected marker centroids. If any reflection intrudes the boundary, the marker is filtered out.
  • Aggressive: Uses a bigger intrusion boundary. More specifically, the marker is filtered out, whenever there is any extraneous reflection detected when marker reflection cells are extracted for the image mode processing.


Intrusion Band

Default: 0.5(pixels)

Sets the size of the intrusion boundary for the Normal intrusion detection mode. More specifically, the intrusion band value is added to the calculated radius of detected markers to establish a boundary, and any extraneous reflections intruding the boundary is considered as the intrusion. }}


ADVANCED OPTIONS

Reconstruction Bounds

Setting the reconstruction bound allows reconstruction only within the defined boundary of the capture volume. Starting from the origin, the user can set the minimum and maximum boundaries along the X/Y/Z axis to create a virtual volume. When the Bound Reconstruction is set to true, only the markers inside the defined volume will be reconstructed.

Bound Reconstruction

Default: False

Enables the reconstruction bound option. All 3D coordinates reconstructed outside of the defined volume is ignored.

Visible Bounds

Default: False

Enable to display the reconstruction volume boundaries.

Minimum/Maximum X/Y/Z (meter)

Sets the maximum and minimum boundaries of the reconstruction volume. All boundaries are referenced against the global origin.


Diameter Calculations

Default: OFF

This will enable the calculation of the diameter of a marker. Use this in conjunction with Min/Max Marker Size. Applicable to Point Cloud v1 only.


Two Ray Marker Optimization

Default: OFF

Enabling this feature improves the accuracy of reconstruction for markers with only two contributing rays by adding an extra evaluation step.

A ray may incorrectly contribute to a marker during capture, and when there are only two rays contributing to the marker, captured trajectory data may have noises. To apply the two ray marker optimization feature, the minimum rays setting should be set to two. If your project utilizes the minimum rays setting of two and require tracking of many markers, enable this setting for improved reconstruction. Applicable to Point Cloud v1 only.


Ray Ranking

Default: 4

This setting enables the Ray Ranking, which calculates quality of each ray to potentially improve the reconstruction. Setting this to zero means that ray ranking is off, while 1 through 4 set the number of the evaluation iterations; 4 being 4 iterations. Setting this value to the max of 4 will slow down the reconstruction process but will produce better results.

The Ray Ranking increases the stability of the reconstruction but at a heavy performance cost. The ray quality is analyzed by comparing convergence of rays that are contributing to the same marker. An average converging point is calculated, and each ray is ranked starting from the one closest to the converging point. Then, each ray is weighed differently in the Point Cloud reconstruction engine according to the assigned rankings.

This setting is useful especially when there are multiple rays contributing to a marker reconstruction. If you're working with small to medium marker counts, enabling this will not have an evident improvement on performance. Also, when precise real-time performance is required, disable this setting especially for a setup with numerous cameras.

Full Residual Threshold (%)

Default: 60%

This sets the maximum error, as a residual value percentage, that a ray can have for it to contribute fully to a reconstructed point. The default is 60%. Setting this to 0 means you don't want any rays to contribute fully as there is some error in all contributing rays. On the other hand, setting this to 100 means all rays will contribute fully. This setting is ignored if the Ray Ranking is turned off.

Zero Residual Threshold (%)

Default: 100%

Sets the maximum error, as a residual value percentage, that a ray can have for it to partially contribute to a reconstructed point. The default is 100% which allows all rays to contribute to some degree. However, if you decrease this setting down to 80% it could clean up noise in a 3D point by filtering the poor rays that are contributing to the reconstruction. If you only have a few cameras, keep this at 100%, you'll want every ray to contribute to the reconstruction.


Minimum Marker Size (mm)

Default: 0 mm

This sets the minimum marker size that will be reconstructed. If a marker has a calculated diameter smaller than this, the marker will not be reconstructed. Unlike the 2D filters, these setting do not affect the raw data when recording in Object Mode. This setting is given in mm. Applicable to Point Cloud v1 only.


Maximum Marker Size (mm)

Default: 0 mm

This sets the maximum marker size that will be reconstructed. If a marker has a calculated diameter larger than this value, the marker will not be reconstructed. Unlike the 2D filter, this setting does not affect the raw data when recording in Object Mode. This setting is given in mm. Applicable to Point Cloud v1 only.


Minimum Marker Spacing (mm)

Default: 0 mm

The minimum spacing feature filters markers that are closer to other markers than the defined spacing. Applicable to Point Cloud v1 only.


Exhaustive Ray Intersection Search

Default: True

Slower but potentially improve reconstruction with low camera count. When a reconstructed 3D marker has number of contributing rays less than minimum ray count setting, it is usually discarded, but with this setting enabled, the point cloud will continuously look for more rays that will intersect the 3D marker. Applicable to Point Cloud v1 only, other versions of the Point Cloud already have exhaustive search algorithm included.


Realtime Speed Boost

Default: OFF

This enables real-time speed boost which enhances system performance at the cost of accuracy. There are two type of enhancements, Type A and Type B. Type A modifies the Point Cloud algorithm to reduce processing time. Type B adds the utilization of system hardware. In some cases Type B will not help. If you get no performance boost when using Type B, use Type A only.

  • Type A: The 3D marker position is calculated with a least square function instead of averaging all the intersections of the rays. When there are a lot of rays contributing to a marker, the calculation time will be faster using this mode because it computes once for each ray, whereas all ray combinations are computed in the normal mode.
  • Type B: Enables the used of two threads to calculate the point cloud. First thread calculates point cloud with cameras 1, 3, 5... and the second thread calculates with cameras 2, 4, 6... After both threads are calculated, they are combined to get the result.
  • Type A and Type B: Combines above two methods for a faster reconstruction.


Small Marker Optimization

Default: OFF

This enables the optimization for small markers but decreases system performance slightly. Only use this if you are working with small markers that are close together, because this could adversely affect larger marker reconstruction. An instance where you will want to use this is when working with 3 to 5 mm markers. Note that this setting only works when the Realtime Speed Boost set to Type B or Type A & Type B. Applicable to Point Cloud v1 only.

TRAJECTORIZATION

Reconstruction Traject.png

After markers have been reconstructed in Motive, they need to be labeled in order to identify each marker in the volume. The trajectorization is a process of associating markers in multiple captured frames to be recognized as the same markers and maintain assigned marker labels within defined constraints.

Note: The term trajectorization refers to the reconstructing and auto-labeling process.


Prediction Radius (mm)

Default: 200 mm

To identify and label a marker from one frame to the next, a prediction radius must be set. If a marker location in the subsequent frame falls outside of the defined prediction radius, the marker will no longer be identified and become unlabeled. This is not the case when the marker is part of a tracked rigid body or skeleton. The prediction radius has no direct influence on reconstruction and is used only for marker labeling.

For capturing relatively slow motions with tight marker clusters, limiting the prediction radius will help maintaining precise marker labels throughout the trajectory. Faster motions will have a bigger frame to frame displacement value and the prediction radius should be increased. When capturing in a low frame rate settings, set this value higher since there will be bigger displacements between frames.


Create Missing Markers

Default: OFF

This option reconstitute the use of Untracked Rays - rays that are not contributing to any markers. Markers can often drop from reconstruction when Minimum Rays is set high for quality and the marker is occluded. The use of Untracked Rays can aid in solving a Skeleton or Rigid Body that is susceptible to occlusion.

Turn this option ON when tracking a Skeleton and/or a Rigid Body and when it's required to increase the Minimum Rays due to a noisy environment. A good indicator that you may need this is if markers are missing from your data that are required by the Asset - such as a flickering chest or finger marker.


Maximum Trajectory Gap

Default: 20

This setting sets the maximum allowable trajectory gap size. If a marker is hidden, or occluded, longer than the defined number of frames, then the labeled trajectory will be truncated and the marker will become unlabled.


Real Markers Only

Default: False

Track using only detected real markers without using virtual markers. When tracking skeletons and/or a rigid bodies, marker trajectory history is utilized to create a virtual marker to improve the trajectory.


Create Missing Markers

Default: False

This option reconstitutes the use of Untracked Rays – rays that are not contributing to reconstruction of markers. Untracked Rays appear when the ray is contributing to a marker that is not qualified for reconstruction because the defined minimum ray requirement is not satisfied. With this setting enabled, the untracked rays can be utilized to aid solving assets – skeleton or rigid body -- that is susceptible to marker occlusions.

The markers may be occluded from reconstruction when the minimum ray requirement is set too high. When capturing in a noisy environment and the minimum ray requirements need to be increased, this feature can be enabled to improve tracking of a skeleton and/or a rigid body. A good indicator that you may need this is if markers are missing from your data that are required by tracked assets - such as a flickering chest or finger marker.


Trajectorization Passes

Default: 1

The number of iterations for analyzing detected marker trajectories for maintaining constant marker labels. Increasing this setting can improve the marker auto-labeling, but more iterations will require more time and computation effort to complete the auto-labeling.


Rigid Body Assisted Labeling

Default: OFF

Rigid Body Assisted Labeling can be used to optimize the labeling of markers within a region defined by a rigid body. The first step in using this feature is to create a rigid body from markers that are visible and rigidly connected. The example shown in the figure below demonstrates this for hand tracking. Five white markers are selected on the top of the wrist - which is rigidly defined. The black markers on the fingers are not rigidly defined in any fashion but are within the boundary of the Rigid Body Assisted Labeler. Labeling continuity is improved for the markers on the fingers which are given automatic labels.

Tracking of organic or flexible objects - that do not have a tracking models like the face and hand, are good candidates for Rigid Body Assisted Labeling.

Rigid Body Assisted Labeling of markers on the hand results in less broken trajectories


Rigid Body Volume Radius

Default: 300 mm

The rigid body volume radius defines the region of space where the rigid body assisted labeling is applied. Increasing this radius will increase the trajectorization, or auto-labeling, time so care should be made when setting this property.


Maximum Assisted Labeling Gap

Default: 30 frames

The maximum gap frames property defines the maximum number of frames a marker can be hidden before it is truncated or unlabeled. Increase this value if larger gaps are to be anticipated. Increasing the assisted labeling gap will increase the processing time of reconstruction.


Prediction Radius

Default: 10 mm

The predication radius defines the size of the bounding region used to label markers. When labeling a marker from one frame to the next, a bounding region, relative to the rigid body, is created around each labeled marker. The labeling continuity is restricted to the bounding region from frame to frame. Increasing this can allow markers to swap if there are occlusions in the data. Decreasing this restricts labeling from frame to frame but may lead to an increase in broken trajectories.


Maximum Gap Frames

Default: 30 frames

The maximum gap frames property defines the maximum number of frames a marker can be hidden before it is truncated. Increase this value if larger gaps are to be anticipated. This will increase the processing time of trajectorization.


Discard External Markers

Default: OFF

Enabling this property will eliminate marker reconstructions outside of the region defined by the Rigid Body Volume.


Dynamic Translational Constraint

Default: None

Prevents the rigid body from moving/rotation more than specified amount per frame.

Max Translation (mm)

Default: 100

Distance for Dynamic Translation Constraint option.

Max Rotation (deg)

Default: 30

Angle for the Dynamic Rotation Constraint option.

Minimum Tracking Frames

Default: 20

Dynamic constraints are enabled when the rigid body is consecutively tracking more than this frame count.


Marker Filter Diameters

Default: False

Markers less than this diameter will not be used for rigid body tracking.

Minimum Diameter (mm)

Default: 10 Diameter used for Marker Filter Diameter option.


Back to Top