Whether it's an autonomous vehicle transporting passengers and goods or cobots working closely with humans, embodied AI generates enormous amounts of data for enterprises to collect and retain.
Data comes from many sources, such as vision systems, microphones, environmental sensors and torque and force sensors. When you consider that an autonomous vehicle can produce up to 5 terabytes per hour from various sources, it's easy to see how data retention can become an overwhelming headache.
Embodied AI data presents real business value for ongoing training and behavioral fine-tuning, not to mention the potential for data monetization. It also poses serious business risks regarding privacy, security and cost. Long-term robotic management requires careful data storage and retention practices -- understanding what data to save, how to save it most efficiently, how to protect it and how to apply practical retention and deletion practices.
Physical AI data collection challenges
Storing and processing vast amounts of data is certainly not a new issue for AI systems, but physical and embodied AI platforms pose an array of new challenges, such as the following:
- Data quality. Because data quality problems in a robot might result in a serious crash or fatal injury, data must meet the highest quality standards. For example, if a robot relies on video or audio data, it might not operate well in areas where vision is obscured or where excessive noise drowns out important sounds.
- Data processing. Robots depend on extremely frequent real-time data that must be stored, processed and acted upon in milliseconds. The robot's data storage subsystem must be fast enough to keep pace with the data and deliver it for processing within incredibly tight performance windows.
- Data storage. Most robots cannot store all the data they produce, especially when robots are small, light and subject to tight power limitations. Some use cases might allow offloading some or all the data from the robot to a data center or the cloud. In other cases, robot storage designs might include a rotating first-in, first-out buffer that overwrites the oldest data that is no longer necessary.
- Data compression and reduction. Robots can apply compression and reduction to extend data storage capacity. For example, applying a compression algorithm to a raw video image can significantly reduce its data size, thereby increasing the robot's data storage capacity. However, compressed data cannot be decompressed, making it less valuable for subsequent analysis or training.
- Data selection. Omitting unnecessary data can extend available storage capacity, but the omitted data is gone forever once it's processed. Store data required to meet compliance requirements, analyze it later for additional fine-tuning, or package and monetize it. Make sure the process complies with AI explainability requirements -- by discarding old data, the enterprise no longer has a complete data set to reconstruct the AI’s state when it makes a questionable decision.
- Cloud storage cautions. While it's easy to imagine cloud storage as a good option for long-term robot data storage, cloud storage should be used with caution. Large volumes of data can incur recurring costs, and data transfers can add to the cloud storage bill. Deciding what to send to the cloud and how to retain it is critical.
Robot data retention fundamentals
Businesses capture, organize and store robot data for two overarching reasons: learning and compliance. Learning includes simulation, debugging and fine-tuning or optimizing robot behaviors. Compliance ensures that the robot's behaviors and decision-making are explainable and trustworthy -- or, if not, at least that it can determine why an undesirable or unintended behavior occurred. Consequently, the most pressing question to consider is data retention.
Today, there are no universally accepted retention periods for robot data. Retention depends on the type of data, the associated industry and the organization's operational requirements. Business, technology and compliance leaders should collaborate closely to understand the available data types and determine the most appropriate retention periods. However, there are several common guidelines to use as a baseline:
- Decision-making data. The data a robot uses to make decisions, such as video, audio and other sensory data, can be retained for 90 days by default. This allows ample time for corrective training or incident review. However, the data can be retained longer if deemed legally or operationally necessary.
- Log data. A robot typically produces error and maintenance logs. These logs should generally be kept for anywhere from one to three years. Ideally, the retention period should align with the robot's predictive maintenance and performance review windows. For example, if an industrial robot needs to be overhauled every two years, retain its log data for the same period.
- Traceability data. If the robot is involved in producing or inspecting components for highly regulated industries, the data related to part inspection should match the regulatory lifespan of the associated part. Such retention periods can be five to ten years, using standards like ISO 9001, FDA 21 CFR Part 11, IATF 16949 or AS9100.
- Personal data. Some robots collect sensitive or personally identifiable information (PII) such as faces, voices and a person’s movements. PII is protected by a wide range of regulatory standards, such as the GDPR. Collect and retain such data only as long as necessary for its specific purpose, apply appropriate data security measures and cap data retention at 30 to 90 days.
Differing data requirements for robots
Robot data needs can vary depending on intended goals. Operation, troubleshooting, forensics, training and monitoring all have different needs regarding data granularity, latency, retention and purpose. Consequently, data retention depends on how it will be used. Consider the following:
- Operational data. Operational data refers to data used to make immediate decisions and keep systems running smoothly. Operational data can also enforce service-level agreements, support alerts or handle errors. Such data is typically ultra-low-latency and contains vast amounts of granular content on current states and real-time telemetry. Operational data tends to be short-lived and is retained up to several hours depending on specific business and regulatory needs.
- Troubleshooting data. Troubleshooting data is used to quickly identify and remediate active symptoms. Data access can tolerate greater latency, from seconds up to 60 minutes, and typically contains details about system state changes, recent events, errors and traces. Troubleshooting data is also generally short-lived, retained for hours to a few days.
- Root cause data. Although root cause analysis is a type of troubleshooting, the effort falls into a separate category that details a deep-dive forensic investigation after an outage, anomaly or accident. Data used in root cause analysis is typically not latency-sensitive and is accessed asynchronously, hours or days after the event. The data includes full payloads with state transitions, infrastructure or system events, triggers and logs. This kind of data set is typically archived or preserved around the event and copied or exported to protected storage for review.
- Training data. Training data is unconcerned with latency and is collected and curated into massive historical data sets using feature matrices, annotated labels and split data. This data is archived long-term with careful version control and tracking.
- Monitoring data. Long-term monitoring data is vital for tracking data or model drift, as well as for detecting encroaching bias. Data latency is not a factor, and collected data is typically aggregated periodically for regular analysis. The data set usually involves simplified time-series data, statistical details and service-level compliance trends. Monitoring data might be retained for months or even years to support capacity planning and compliance requirements.
Data management considerations for robots
Robot data management involves capturing, organizing, storing and synchronizing data from physical and embodied AI platforms. Although robot users don't affect the robots' underlying design, they often exercise broad discretion over the data captured, its format, how it's correlated, how it's protected and where it's stored. Here are some considerations:
- Minimize data collection and retention. Not all data is important. Review the entire suite of data that a robot generates. Decide which robot data should be retained and how long. Then decide how best to cache and store the data for its designated retention period.
- Focus on data synchronization. A robot's reactions and real-world decision-making depend on time relationships across multimodal data. Apply accurate timestamps to data so that cause-and-effect relationships are well correlated. When an engineer seeks to examine data, they need only add a time reference to view the complete suite of pertinent data.
- Use the most appropriate data storage format. Traditional raw data storage might be inadequate for high-throughput, real-time streaming data. Select a storage format designed for robotic data capture, such as ROS bags or the message capture format for multimodal robotics, which handles heterogeneous data streams with low recording latency and native viewing support in tools like Foxglove. Other formats include Apache Parquet and Arrow as columnar storage formats suited for tabular telemetry, data lakes and fast metadata querying, as well as Zarr and Lance, which provide chunked array and multimodal formats that support random access and evolving data schemas -- popular for robot training data.
- Use tiered storage and edge filtering. Uploading all robot data to the data center in real time taxes network bandwidth and might be impossible for remote or autonomous robots. Instead, first cache the data locally on the robot and then offload it to centralized storage based on real-world events, triggers and conditions. Offloaded data typically resides on a high-performance tier for prompt evaluation and further processing, and is eventually deleted or relegated to archival storage or even tape for the retention period.
- Consider data storage holistically. Robots present a particular data challenge in both volume and location. So much data across so many locations can pose security risks or lead to retention policies being accidentally overlooked. Track each data location carefully to maintain a comprehensive picture of the entire robot data storage environment.
- Ensure data version control and reproducibility. Robot data is typically treated like code because in many cases, the data set is used much like code to assess, train or reliably reproduce an event or condition. Each captured data set should include versioning details such as collection configurations, episode identifications, calibration conditions and validation details. Such additional metadata helps engineers easily find data sets and compare events.
- Select data visualization and analytics tools. Data means little without a convenient way to inspect or review the data streams for individual robots and entire robot fleets. For example, tools such as Foxglove offer open source data visualization and debugging for ROS messages, MCAP files and 3D robot models expressed as .URDF files. Scale AI and Encord provide human-in-the-loop engines for video, 3D and physical AI data sets; ARES offers an open source robot data platform with a structured database, embedded lookups and multimodel annotation; and Extend Robotics and ReductStore offer platforms supporting real-time multimodel data recording, data synchronization and time-series telemetry for robots.
- Implement comprehensive data security. Because robots can collect sensitive or protected information, use encryption to safeguard data. Implement strong data access controls such as identity and access management and maintain immutable access logs that detail each data access event for security monitoring and compliance audits.
Stephen J. Bigelow, senior technology editor at TechTarget, has more than 30 years of technical writing experience in the PC and technology industry.