Remote device management of connected robots

A device management system supports features such as remote provisioning, software upgrades, command and control, configuration updates, state and health monitoring, diagnostics and debug.

In IoT and connected device systems, remote device management from a server or cloud is one of the most critical features, but often the most sidelined too. Because the device management solution does not result into an actual tangible or visible feature of the system, product management often tends to ignore it during the system development. Later when devices get rolled out in the field, the importance of device management becomes clear. For example, a critical software fix requires complicated procedures to upgrade the software to field-deployed IoT endpoints when a device management system is not implemented.

Autonomous mobile robots (AMR) are increasingly internet connected. Indoor AMRs have access to Wi-Fi. Outdoor AMRs have access to cellular. There may be cases where the connectivity is restricted to on-premises LAN for additional security. Invariably, AMRs are connected.

One can look at connected AMRs as connected IoT devices on wheels, autonomously navigating in the field they are deployed in. They need to be remotely managed. Typical device management features are required; however, as a result of the robot’s autonomous mobility, there are some unique challenges and use cases that require specific add-on features for remote management. The key question that drives remote management solutions is how does one remotely manage provision, diagnose and give software updates to AMRs on the field?.

Similar requirements hold true for industrial robots with robotic arms. They have increasing numbers of sensors, multi-CPU configurations and connectivity. Although their motion is restricted to the configuration space surrounding them, there are a vast number of aspects that need to be remotely managed, including states, sensors, power or battery health, algorithms, software and firmware.

This article dives into typical requirements for remote device management of AMR’s and industrial robots and serves as a guide for product management teams of existing and new robot OEMs, software solution providers for AMRs and industrial robotics.

Key functional features that admins should remember for remote management of robots are:

  1. Provisioning and authentication. This involves robot on-boarding, registration and authentication with the remote management server.
  2. Command and Control. Features such as factory reset or reboot.
  3. Configuration updates: Sending offline generated revised site maps, dynamic map edits specific to a site, updating other configuration parameters, which get fined tuned for better operation over the period.
  4. Monitoring and diagnostics. These allow the capture of log files from the robot and upload them to the remote management server. Files include real-time streaming of sensor data, algorithm data, event data and error logs to the server for monitoring state and diagnostics. All of them need to have timestamps associated with the data.
  5. Software updates and upgrades. In robots there are various host and slave CPUs or connectivity modules, robot applications and update agents. All need to be updated for bug fixes, new features and more. Software updates and upgrades also include rollbacks or downgrades when the new update has created issues. Software updates need to be done for various components, including robot host CPU application, agents which do the updates, robot operating system (ROS) middleware, host OS, microcontrollers (MCUs) firmware, Wi-FI or cellular module, computer vision with AI and machine learning models, and containers.
  6. Multi-tenant remote robot management system server with role-based access control. Manage a fleet of robots based on various roles and permissions set in the system. Fleets of robots themselves are organized in various groups with different permissions.
  7. Fleet management: The remote management features need to be provided for a group of robots, a group within a fleet of robots and multi-fleet management.
  8. Robot off-boarding, de-registration and disable.

In addition to above functional features, there are many that are not part of the core functionality, but instead focus on security, adaptivity, analytics, location, scheduling, monitoring, data storage and log data.

  1. Security. Security is the underlying theme for all the features. Without secure authentication and provisioning and a secure data exchange mechanism, there is just no way an end customer will trust a remote robot management solution.
  2. Adaptivity to connection quality. One specific issue which often arises for AMRs and industrial robots that are connected over wireless is the unpredictable quality of wireless connection. AMRs require adaptability to variable connection quality. The Wi-FI or cellular wireless connectivity link may not be reliable. Signal strengths will vary across the environment. There could also be high data consumption cost when connected over cellular. The remote robot management solution needs to handle this condition like a managed service throttling the data and retry when there are connectivity issues. Software updates need to happen when there is minimal business impact.
  3. Analytics. Data from a fleet or individual robot is analyzed based on data streamed stored on servers.
  4. Locate and search robots. Robots have a specific ID for a specific customer and site based on state or and other attributes and view the information, state, configuration and data.
  5. Schedule over-the-air jobs. Scheduling is based on campaigns and view the status of different jobs.
  6. Monitor various API calls.
  7. Visualize insights and data.
  8. Easy to navigate, responsive GUI.
  9. Playback of captured log and diagnostics data. In the robot simulations’ environment storing data in rosbag files — rosbag is a file format supported by ROS — allows data to be played back into ROS simulations with the correct timestamp.
  10. Meta data-based search and remote management.
  11. Map upload, combination and push back. AMR’s in the field may upload separate Maps to the Server if mapping operations which are combined on the Server and then pushed back to the Robots.

Associated with each robot’s data are various types of meta information including current location, site, customer and robot ID. Use this meta data to quickly search AMRs and then complete remote management.

Importance of interfacing to middleware

Remote management systems will invariably involve an agent which resides on the robot, typically on the host CPU. The agent should provide a flexible handshake with the robot application for all features of remote management.

ROS is a very popular middleware in robotics. ROS1 is and will remain de-facto for a while but ROS2 is also coming up. Both provide various publish and subscribe topics and services for command and control over which the remote management agent needs to interface to the robot’s primary application. There are robots in the field and organizations developing new robots which don’t use ROS. The ability to interface to robots native application is important. ROS provides a bridge feature to handshake between a Non-ROS application and a ROS application.

Remote debugging AMRs and robotic arm robots in the field.

AMRs and robotic arms’ motion is possible as a result of sensors and algorithms that utilize the raw sensor data. Sensors are susceptible to noise, intrinsic randomness and statistical variations.

AMRs monitor live sensor data, view it in the cloud or remote server and record live data on the server side based on time parameters, such as start capture of a select number of sensors for next 10 mins or show last 10 mins of chosen sensor and algorithm data.

The sensor data can include point clouds from depth cameras, GPS, estimated pose, dynamic linear and angular velocities, accelerations, various events, various motor parameters, live video streams from various cameras and odometry. Many of these get categorized under telemetry.

The same holds true for various algorithms that are continuously generating various states and processed outputs. Algorithms for AMRs include localization, mapping, slam computer vision and AI. Algorithms for industrial robotic arms include motion planning, inverse kinematics, computer vision, sensor fusion and more.

Challenges with streaming large raw sensor or algorithm processed data to a remote server

The bandwidth limitations and usage can be a problem, especially when streaming large amounts of raw sensor or processed data originating from lasers, depth and vision cameras, other sensors and algorithm processed data. Capturing data on robots for upload later to a server can be a better approach. However, when a technician is remotely debugging robots, live streaming is unavoidable.

Some end customers are not comfortable with public cloud connectivity. On-premises data streaming has high value with features on similar lines as a cloud deployed system.

Offline robots due to connectivity loss

Connectivity may not be available or possible in areas where industrial robotics and AMRs are deployed. For example, AMRs can get deployed in remote sites — such as mining fields — where there is no cellular connectivity. Satellite connectivity is costly. Connectivity loss should not result in robot malfunctioning and its remote management should remain active, but in a different form. It is essential to support capture logs, streaming diagnostics, algorithm data from the field robot and software updates via USB or local ethernet LAN as part of remote management. Push captured data on the robot to the management server at a later point. There are limits to how much data can be stored locally on the robot, but maintaining even the last 10 seconds of data can make a big difference to diagnose what went wrong when bad situations arise, such as an AMR crashed or an industrial robot failed to pick and place.

Black box for AMRs

With the large amount of data that can get captured locally for logging, diagnostics and later upload, there is an overall need for a black box similar to flight recorder black boxes. Cost is a large factor to having an independent black box which can survive crashes and transmit beacon signals at periodic intervals upon a crash to receivers used for remote monitoring. This will become increasingly important.

Remote management 24/7

Robots are running 24/7 in the field at indoor warehouses and outdoor environments. Having a strong remote robot management solution, combined with 24/7 managed services is peace of mind to the customer who has deployed the fleet of AMRs. The criticality of managed services should not be underestimated.

This article focused on priority features. There are other non-functional features not necessarily covered.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.

Data Center
Data Management