Getty Images/iStockphoto


An introduction to IoT logging types and practices

Collecting IoT data logs can help IT teams spot deployment issues and security alerts. For smooth operations, admins should identify what data they need and use the right hardware.

Admins who oversee IoT devices use logging software to yield insights about device performance. The goal of using IoT logging software and devices is to ensure efficient, effective and safe operations.

Logging is a way for IT administrators who manage IoT deployments to use any collected data to gauge how the IoT deployments are running. For a successful logging workflow, IT teams should figure out what log types are the most important and how the log data should be used.

"Logging in IoT is the act of keeping a record of the various things and actions an IoT device does. A log can be nearly anything, including status changes, commands sent to the device, sensor readings, error codes, bugs, network changes, authentication attempts or configuration changes," said Geoff Weathersby, director for IoT at consulting firm Protiviti.

IoT log data collection and analysis can help admins better understand the health and performance levels of their connected devices and overall IoT deployments.

That work takes a strategy -- one where admins must identify what data they need and what purpose the data is for and then collect, manage and analyze the required logging data. Furthermore, admins must have tools to clean data before use.

Centrally managed logging systems for IoT devices help decrease downtime, troubleshoot fixes and identify general malfunctions of log connection issues, said Kateryna Dubrova, research analyst on ABI Research's IoT networks and services team.

Why logging data is important

IoT data logging is similar to non-IoT device logging, said Adonya Ourshalimian, vice president of product management at Theorem. Both can provide valuable insights that businesses can use to improve or optimize operations.

Just as other IT systems use different log types for specific data, IoT also has different types of data to log. These logs detail whether devices are on or off, connection status, or whether devices fail or experience errors.

Admins should pay attention to logging data because it provides insights into IoT deployment health, which can be hard to get otherwise.

"IoT devices are generally low-cost, low-power devices intended to be deployed in large numbers for long periods of time with minimal direct interaction. Practically, what this means is that you will want to maximize your access to the device's history and status without negatively impacting device performance, battery life or data costs," Weathersby said.

IoT logging types

To ensure the right data gets to the right users and it can benefit the organization, it's important to know what types of data are available and how to classify any logs. The following are the different IoT logging types:

  • Authentication logs show whether registered users are logged in or not and might show login attempts, including failed ones; this type of log is sometimes called an access log or permission log because it shows who has device access.
  • Configuration logs track devices' various characteristics, such as device settings and how they've changed over time.
  • Error logs detail when something goes wrong or fails with device or network operation.
  • Memory dump logs provide details on when and how devices fail. They are also known as crash reports.
  • Status logs share if a device is online, offline, transferring or in error; these logs provide high-level overviews of each device's overall status.

Devising an IoT logging strategy

IoT data logging serves various purposes and can support initiatives such as preventative maintenance, production monitoring and troubleshooting, IoT deployment optimization, and quick root cause analysis and security alerts.

To establish a logging strategy, admins should consider what logging data they need based on the outcomes they're trying to achieve, such as safeguarding IoT device reliability, calibrating data accuracy or increasing security. Then, admins must determine how to gather, manage, visualize and analyze that data -- and if there are any available software options that can help.

"Before beginning a deployment, you need to think through your logging strategy. Once deployed, logging is one of the main tools to know how well your devices are performing in the field. Logging allows you to monitor, debug and evaluate IoT deployments. Because of the quantity and location of IoT devices, it's not practical to physically monitor or examine deployed IoT devices with any regular frequency," Weathersby said.

Collecting IoT log data is a challenge, and it's generally more complicated than gathering log data from other IT infrastructure, as IoT devices generate astronomical amounts of data.

"[Connected] devices differ from servers as they do not have enough onboard memory to store IoT logs for any prolonged period. All devices will need hand-operated intervention in case of idle time or lack of memory to store the log. In addition, keeping logs locally requires a physical link to a remote computer or bridge to upload or download data," Dubrova said.

Other challenges that affect IoT data collection include intermittent or unstable connections with IoT devices, network congestion, firewall issues and storage limitations.

When it comes to data analysis and collection, Ourshalimian said IoT admins can use commercially available monitoring software and dashboards to hold, visualize and analyze logging data. But they may decide to custom build data collection capabilities and dashboards to gain the insights that are most relevant to specific organization deployments.

Weathersby recommended that tech teams specify what logging data they need, as collecting all IoT data is not typically a smart move; not only does it make it difficult to find useful data, but it can also get expensive wherever it is stored.

"When setting up an IoT deployment, it's easy to just say, 'Log everything, and we'll figure it out later.' The challenge comes when deploying devices at scale. Usually, onboard storage on an IoT device is fairly small to keep device costs down, and log files can quickly fill up depending on how active an IoT device is. This means the logs must regularly be transferred to the cloud. Depending on the connectivity technology, there could be additional costs for cellular data usage, cloud data transfer [or] cloud storage," Weathersby said.

Next Steps

How to select the right IoT database architecture

Dig Deeper on Internet of things platform

Data Center
Data Management