Real-time safe C++ programming

Large parts of ICON, the control and hardware layers, need to operate as a real-time system. They need to compute setpoints and control hardware within fixed deadlines under normal operation.

Both custom actions and custom hardware modules run within the real-time system. They need to follow a real-time safe programming style to avoid exceeding deadlines, causing hardware timeouts, or controlling discontinuous motion.

note

Skills that use the ICON client and developer tools that use icon::Client run outside the real-time system and don't need to follow these precautions.

note

Real-time programming is only needed to extend custom actions and hardware modules in ICON. It is an advanced feature that requires familiarity with C++, knowledge of C++ memory management, and knowledge of C++ multithreading and synchronization.

Avoid dynamic memory allocation

One of the largest differences between real-time safe programming and regular programming is that real-time critical sections of the code may not allocate or free memory on the heap. Dynamic memory management functions like malloc and realloc have no deterministic deadline. This also includes the new operator and other functions that allocate on the heap. Effectively, many common data types cannot be used in real-time contexts.

There are two main alternatives:

1. Create the data outside the real-time context and treat it as constant in the real-time context.
2. Use data types without dynamic allocation and with a fixed maximum size.

Similarly, file operations can block real-time operations. The alternative is to delegate reading and writing files to a non-real-time thread and use real-time safe communication and buffers to pass the file content.

Avoid because of blocking memory allocation or file operation	Real-time safe alternative
std::vector	If the size is known, use std::array or C arrays. Otherwise, consider xfa::FixedVector. If no resize is needed, absl::Span may be a viable interface for functions.
std::exception, absl::Status, absl::StatusOr	icon::RealtimeStatus, icon::RealtimeStatusOr. Another alternative is to return absl::StatusCode, std::optional or only bool.
std::string	Prefer absl::string_view in interfaces if no modification is needed. Otherwise, consider icon::FixedString<kMaxSize>.
std::string::append, absl::StrCat	icon::FixedStrCat
Eigen::VectorXd, Eigen::MatrixXd	For read-only vector operations or vector modification without re-sizing, consider absl::Span. Also, fixed-size Eigen::Vector6d, Eigen::Matrix4d or fixed maximum size eigenmath::VectorNd (or xfa::FixedVector for larger sizes) are allowed.
Most standard containers such as std::map. Even read operations like std::map::find can dynamically allocate.	(No direct replacement available.)
Protobuf or other types of serialization	Only Flatbuffers (generated with cc_fbs_library) are allowed. Trivially-copyable C++ structs are another alternative.
Abseil logging, LOG(INFO) and related	Use XFA_RT_LOG instead.

Synchronization within timeouts

Most mutex and synchronization primitives, including std::mutex::lock and absl::MutexLock, don't have an upper bound deadline and therefore don't fulfill real-time requirements.

For communication over real-time threads and processes, options are

• icon::RealtimeClock to wait for the next control tick.
• std::atomic to signal and poll.
• icon::BinaryFutex to signal and wait with a defined timeout.
• icon::SharedMemoryManager to pass messages.

For synchronization across the real-time/non-real time boundary, options are

• absl::Notification to signal and wait on the non-real-time side, or icon::BinaryFutex if repeated notifications are needed.
• xfa::RealtimeWriteQueue to pass messages from real-time to non-real-time.

Avoid other blocking operations

In addition to dynamic memory management, file operations and synchronization, other potential causes for blocking a real-time thread are:

• Device and bus communication need to set timeouts and operate with fixed-size buffers for send and receive operations.
• When you create a thread, you must set the priority and scheduler policy. A common mistake is to inherit the priority from the parent thread, which may be too high. Then, the child thread uses excessive resources and block the real-time thread as an effect.
• All libraries and frameworks need to follow similar standards.

Additional style practices

Intrinsic follows the Google C++ Style. For code that can run in a real-time thread (including custom actions and custom hardware modules), an additional practice is to annotate which functions should be real-time safe and which shouldn't. Unless it is clear from context, each function declaration should have a comment whether the function is real-time safe.

To avoid accidental function calls from real-time safe to not real-time safe functions, you can create a runtime analyzer xfa::icon::RealTimeGuard in the real-time thread. RealTimeGuard marks its scope, including all function calls in its scope, as real-time critical. If subsequent function calls reach the macro INTRINSIC_ASSERT_NON_REALTIME, the process exits fatally and prints logs about the unsafe function call. You can place INTRINSIC_ASSERT_NON_REALTIME in a function to disallow calls from a real-time context.