ICON custom action guide

important

The ICON custom action API is not enabled by default.

An ICON custom action is an advanced feature to modify and extend the robot control stack. A custom action is a C++ class that implements a new real-time robot control behavior. It can read sensor inputs in real-time, implement a custom control law, and send control signals. While a custom action is running, the ICON robot controller invokes functions on that action in every control cycle.

Sine wave tutorial

In this tutorial, you learn to:

• Upload a custom action to the ICON server
• Look at and modify the code for SineWavePluginAction, an action that generates a sine wave trajectory
• Run SineWavePluginAction in simulation

Prerequisites

This tutorial assumes the following setup steps and connection to industrial PC:

• Install Bazel and Git
• Have the SDK repository https://github.com/intrinsic-ai/sdk/tree/main/ checked out in a local directory
• Create the hostname entry for xfa.lan
• Have a direct Ethernet connection from your terminal to the non-realtime network (of the uplink port) of the Intrinsic industrial PC.
• Start an Intrinsic application in simulation
• Open the Robot Control panel to visualize the simulated robot

Warning: Run this tutorial only in a simulated environment. Verify that the title bar of the frontend says "Running in simulation".

Load the sine wave plugin

An ICON custom action is stored as a binary plugin, a shared object .so file. The ICON server can load one or more custom action plugins during startup. When the ICON server runs, a custom action can be started and executed as part of any action-reaction graph, in the same way as pre-installed actions like the joint move action intrinsic.point_to_point_move.

In this example, you load the custom action sine_wave_action. Its source code is located in the SDK repository in intrinsic/icon/control/c_api/external_action_api/sine_wave_plugin_action.cc.

First, compile the existing example as a shared object plugin.

bazel build //intrinsic/icon/control/c_api/external_action_api:sine_wave_plugin.so

Then, upload the plugin file to the ICON server and restart the server.

bazel run //intrinsic/platform/file_upload:file_upload_cli upload $PWD/bazel-bin/intrinsic/icon/control/c_api/external_action_api/sine_wave_plugin.so
# Check the output of this to confirm that the upload worked (the list of files
# in the output should contain sine_wave_plugin.so)
bazel run //intrinsic/platform/file_upload:file_upload_cli list
# ICON only loads plugins on startup, not while it is running, so restart ICON:
bazel run //intrinsic/icon/tools:restart_server -- --instance robot_controller

note

robot_controller is the default resource name of the real-time control service. If the real-time control service has a different name, pass that name with the flag --instance.

Restarting the server may take several minutes. You can verify that the sine_wave_action has been loaded successfully by running the following terminal command:

bazel run //intrinsic/icon/tools:list_actions -- --instance robot_controller

It should output "sine_wave_action" among the list of existing, pre-installed actions.

Run the sine wave example

Now, you can run an example program that calls the custom action. The source code of the example program is available in intrinsic/icon/control/c_api/external_action_api/move_sine_wave.cc.

Feel free to modify any part of the example program to see how it affects the robot.

While the example program runs on the developer machine, the custom action plugin runs as part of the ICON server in real-time.

bazel run //intrinsic/icon/control/c_api/external_action_api:move_sine_wave -- --instance robot_controller

While the example is running, you can observe the simulated robot performing a sine wave motion with multiple joints in the web frontend. This trajectory is completely defined by the custom sine_wave_action in the real-time control loop.

More specifically, the Control function of the custom action in intrinsic/icon/control/c_api/external_action_api/sine_wave_plugin_action.cc calculates the setpoints for each joint as follows:

  eigenmath::VectorNd position_reference(
      current_state_->starting_position.size());
  for (size_t i = 0; i < current_state_->starting_position.size(); ++i) {
    double sine_value = std::sin(current_state_->time_since_start * 2. * M_PI *
                                 params_.frequencies[i]);
    double sine_cubed = sine_value * sine_value * sine_value;
    position_reference(i) = current_state_->starting_position(i) +
                            params_.amplitudes[i] * sine_cubed;
    // Clamp position reference to joint limits. This doesn't prevent
    // exceeding the other limits (velocity, acceleration, jerk)!
    position_reference(i) =
        std::clamp(position_reference(i), default_limits_.min_position(i),
                   default_limits_.max_position(i));
  }
  return feature_interfaces->joint_position->SetPositionSetpoints(
      JointPositionCommand(position_reference));

ICON executes this code in real-time. While this example does not read sensor inputs, the custom action API generally allows accessing sensor measurements and formulating new real-time control laws. This is an extension point of ICON to allow additional robot behaviors and sensor-driven robot control. (This example action cannot produce a smooth trajectory when starting on a moving robot. It assumes that it starts from a steady state.)

Modify the sine wave example

A custom action has multiple interfaces to interact with the robot controller. It is possible to add parameter types, define streaming input and output, modify the implementation (control law) and add state variables. In this tutorial, you make a small change to the control law implementation.

In the provided example, the sine wave trajectory starts in positive joint angle direction. For experimentation, you can change the direction by editing the file.

edit intrinsic/icon/control/c_api/external_action_api/sine_wave_plugin_action.cc

To modify the control law to start the motion in the opposite direction, modify the Control function and add a negative sign to the line where sine_cubed is computed:

double sine_cubed = -sine_value * sine_value * sine_value;

After saving the change, compile and upload the modified custom action again by running the same commands again.

bazel build //intrinsic/icon/control/c_api/external_action_api:sine_wave_plugin.so
bazel run //intrinsic/platform/file_upload:file_upload_cli upload $PWD/bazel-bin/intrinsic/icon/control/c_api/external_action_api/sine_wave_plugin.so
bazel run //intrinsic/icon/tools:restart_server -- --instance robot_controller

You can observe the modified trajectory by running the example program:

bazel run //intrinsic/icon/control/c_api/external_action_api:move_sine_wave -- --instance robot_controller

Develop a custom action

To verify that a custom action is configured and loaded correctly, you can print detailed signatures of all actions with the command

bazel run //intrinsic/icon/tools:list_actions -- --show_details --instance robot_controller

This prints detailed signatures, including parameter types and state variables, of all actions, including custom actions that were loaded.

You should see this entry, which corresponds to the sine wave action:

sine_wave_action
Waves a Sine!
Compatible Parts

arm
Fixed Parameters
Message Type: xfa.icon.external_actions.SineWaveFixedParams
Streaming Inputs
number
a number that is echoed back via the 'number' state variable
Streaming Output
time_since_start
Reports the time since the Action was started.
State Variables
time_since_start (double)
seconds since the Action was started.
number (double)
reports the latest value received via the streaming input 'number'.

When modifying or writing a custom action, remember that ICON runs the following functions in the real-time control loop:

• OnEnter()
• Sense()
• Control()
• GetStateVariable()

The implementations of these functions need to be real-time safe. That is, they must always finish computation within a single control cycle, even in error cases. 1000Hz is a common control frequency, which means all of these four functions should take well under one millisecond to finish, even in exceptional or error cases.

Among other requirements, real-time safe code shouldn't allocate or re-allocate memory, or make any system calls. Follow the real-time safe programming guide when making changes to a custom action.

Custom actions should handle most computation in their Create() function, particularly long-running computations like motion planning. The Create() function does not run in real-time, and can take as long as it needs to finish.

The action needs to take into account that the robot state has changed between Create and OnEnter, and that the robot can be in arbitrary motion while OnEnter is called.

Cleanup

Remember that you uploaded the plugin to the server as an .so file. The file remains stored on the server until you remove it, even if you restart the ICON server executable or reboot the operating system. To clean up after this tutorial, you can delete the file with the following command:

bazel run //intrinsic/platform/file_upload:file_upload_cli remove sine_wave_plugin.so