Robotics needs real-time edge AI: Why and where

How edge AI works

Edge AI provides the components and software that run ML models and other AI software on hardware devices, such as an embodied AI robot, rather than exchanging data with a centralized computing platform. The process of building edge AI is straightforward, often starting with a large, sophisticated ML model in a cloud environment, trained on massive data sets. That establishes the baseline model, which is then compressed, reduced and simplified to make it far smaller and less resource intensive.

The simplified model can then be deployed onto an edge device, such as a car, drone, smartphone or embodied AI robot. Once deployed, the model can run continuous inference using real-world sensor data to make real-time decisions or predictions on the edge device. It's still possible to exchange data with a centralized computing infrastructure when more analytical power or access to more detailed data is needed, but the networking requirements are less stringent, and no sensitive data needs to be exchanged in real time.

Edge AI in robotics consists of three primary elements:

1. Computing. The edge AI system requires specialized computing hardware, including GPUs and often NPUs and TPUs, intended to handle inference math at extremely high speed and energy efficiency. The system might also require storage as part of the computing package to retain important information such as camera images, GPS locations and decision contexts -- all of which can provide insight into how the device made its decisions.
2. Software. Edge AI runs very fast, simplified models and other AI software components to execute software in an energy-efficient way, extend battery life and avoid latency.
3. Sensors. Edge AI devices can employ a broad array of sensory devices to help AI perceive the real world. Sensor packages can include cameras; microphones; GPS receivers; LiDAR; radar; forward-looking infrared cameras; temperature, pressure and humidity reading instruments; and combined inertial measurement units for orientation, speed and acceleration. The computing package processes sensor data locally in real time and stores it for more detailed analysis.

Edge AI robotics use cases

Edge AI provides a wide range of potential robotics applications, enabling fast decision-making and zero-to-low latency computing, which are critical for real-world operation. Potential use cases include the following:

• Industrial management and maintenance. Sensors can detect environmental conditions and performance characteristics on factory floors, warehouses or embodied AI platforms. The collected data can be used to adjust environmental and performance characteristics in real-time to prevent damage and inventory waste.
• Environmental safety. Fixed cameras and vision systems on embodied AI platforms can monitor work areas and alert workers when they enter potentially hazardous workspaces.
• Quality control. Cameras can inspect parts and materials and spot defects during each step of the manufacturing process.
• Inventory monitoring. Cameras can track parts and materials to better automate inventory management and improve counting accuracy. Embodied AI platforms used to pick inventory can continuously count remaining items and provide frequent inventory updates. The data can then drive automated ordering systems to support just-in-time inventory control methodologies for more efficient cost management.
• Driver assistance capabilities. Cars incorporate an ever-growing variety of sensors and cameras to assist drivers with blind spot alerting, lane keeping, emergency braking, hazardous road conditions and other evolving automotive features.
• Autonomous vehicles. Self-driving cars and drones rely on camera, LiDAR, radar and inertial data to operate safely and avoid obstacles in situations where extremely fast processing and response times are critical.
• Patient telemetry. Medical patients can use an array of wearable medical devices to monitor heart performance, check blood glucose and administer medicines. The devices can process medical data locally to alert the patient directly or transmit alerts to medical providers.
• Farming and agriculture. Various embodied AI platforms, such as drones and field robots, can map terrain, monitor soil conditions to guide fertilization and irrigation, and identify and remove weeds without herbicides, while autonomous tractors can harvest crops.
• Utilities. Energy producing systems such as solar and wind power can use edge AI to predict failures and schedule preventive maintenance, while sensors combined with weather data can improve the performance of solar and wind systems.

Edge AI implementation best practices

The approach to implementing edge AI can be as varied as the organization and guided by the unique needs of the industry and its associated regulatory obligations. But there are several common guidelines to effectively and efficiently implement edge AI in robotics, including the following:

• Start with high-end models. Traditional, high-end models are trained and validated on powerful server farms using historical and simulated data. Once the model is simplified and optimized for inference, it can be deployed in the edge AI environment.
• Minimize models for the edge. Huge, sophisticated models have no place at the edge because they need too much computing capacity and energy to justify the nuances they provide. Reduce model complexity by converting floating point weights to integers and removing outliers to reduce memory demands. Smaller models handle inference well, require far less computing capacity and power, and respond with near-zero latency.
• Keep edge data temporary. Edge AI input from cameras and sensors can generate enormous amounts of contemporaneous data. Decide which data and how much of it the edge platform should store. Retain only the minimum amount of data needed for further analysis or to meet regulatory obligations and discard the remaining data to avoid unnecessary storage demands on the edge platform.
• Consider hardware demands. It's important to match the edge hardware to the task. Too much computing power at the edge wastes capacity and energy, while too little can lead to excessive latency -- highly undesirable in real-time systems. Some platforms will benefit from specialized AI accelerators such as TPUs, while other platforms will focus on highly integrated controller chips purpose-built for specific edge applications.
• Understand edge limitations. Constraints are an important aspect of edge AI system design. Remote power can be limited and require frequent nearby battery charging, network connectivity can be intermittent or absent, and environmental conditions can affect an edge device's performance and reliability. Device designs should be optimized for the specific use case.
• Incorporate edge device management. An edge environment can contain dozens, hundreds or even thousands of edge devices, from smart sensors to independent embodied AI robots. Consider how all these devices will be monitored and managed for issues such as location tracking, operational integrity, model drift and software updates.
• Monitor edge security. Edge AI poses additional security challenges due to the number of devices and potential for physical injury. Implement edge AI systems with features like signed firmware updates, encrypted models and secure startup sequences. Security is critical to prevent hackers and attackers from taking control of edge AI devices and causing harm to humans.

Stephen J. Bigelow, senior technology editor at TechTarget, has more than 30 years of technical writing experience in the PC and technology industry.