estimate_pose

Skill class for ai.intrinsic.estimate_pose skill.

The estimate_pose skill is one of Intrinsic’s main perception skills. It uses Intrinsic’s pose estimation algorithms to estimate the poses of one or multiple objects given a single camera image of the scene. The skill collects an image from the specified camera then runs the estimation algorithm using the specified pose estimator. The skill succeeds if the number of detections is equal or greater than the min_num_instances specified. The skill returns the pose, score and ID for each detected object instance. This skill does not update the pose of the detected object instances in the belief world. Instead, skills such as update_world or sync_product_objects can use the output of this skill to update the belief world. List of supported cameras:

• Any 2D color or intensity camera that supports the GenICam standard
• 3D cameras that produce a color/intensity image as well as a depth map To use multi-view pose estimation (i.e. pose estimation with >1 camera images), refer to estimate_pose_multi_view (here) instead. Pose estimator: One from the list of available, saved pose estimators. Mind that the pose estimator must be trained on a similar camera as the one selected. If a new pose estimator needs to be trained, follow the instructions here.

Prerequisites

estimate_pose requires a trained pose estimator in the solution (training guide). You need to set up and calibrate the perception equipment in your workcell before you can use this skill to detect objects and estimate their pose.

Usage Example

The estimate_pose skill outputs the estimated 3D pose(s) of the object(s) it was able to detect. Those poses are typically used as input to subsequent skills to plan the manipulation of object(s) or the scene in consideration of the objects’ respective pose.

Update the digital twin (belief world)

Instead of saving the detected poses in variables, they can be used to update the digital twin to match the configuration of the real or simulated world (from which the camera image has been acquired). To this end, use skills like update_world or sync_product_objects, which lets you transform objects in the belief world to their correct poses. Link estimate_pose output root_ts_target to the “A t b” input of the update_world or the root_ts_object input of the sync product objects skill, respectively.

To move a robot to the object, you can then directly use the updated frames of the object. In case the process flow shall not use a specific fixed object, skills like “echo world nodes” allow to query a range of objects and use them in further data flows.

The estimate_pose skill outputs the estimated 3D pose(s) of the object(s) it was able to detect. Those poses are typically used as input to subsequent skills to plan the manipulation of object(s) or the scene in consideration of the objects’ respective pose.

Parameters

pose_estimator

Id of the pose estimator.

min_num_instances

Minimal number of instances that must be detected. Skill fails if not enough instances can be found. Leaving this value unset means 0.

max_num_instances

Maximum number of instances that can be detected, which must be greater or equal to 'min_num_instances'. Leaving this value unset will take the default (if available) from the corresponding pose estimation config.

timeout_sec

Maximal allowed time in seconds to finish the estimate pose skill. This includes the creation of a camera, frame retrieval, pose estimator creation and the actual pose_estimate.

object_t_target

Transformation to apply to resulting root_t_target. Leave out to not apply any specific offset (corresponding to zero translation and no rotation).

occluders

Optional list of world objects which may occlude parts of the image. The provided objects will be rendered from the current camera view into an occlusion mask which the pose estimator uses to mask out the occluded regions. Note that not all pose estimators will actually use the mask, so for some pose estimators this parameter will have no effect.

image_intensity_corrections

List of image intensity corrections to be applied to the acquired image before running pose estimation. The corrections are applied in the order in which they are specified.

capture_data

If specified, the skill uses the capture_result_location within to retrieve an image captured earlier.

If not specified, the skill captures a new image directly from a camera.

Capabilities

camera

Resource with capability CameraConfig

Returns

estimates

Pose estimates returned by the pose estimator.

root_ts_target

Poses of the returned pose estimates. Note that this is redundant to the poses returned in 'estimates' but allows easier use in combination with other skills like sync_product_objects.

Error Code

The skill does not have error codes yet