Getty Images/iStockphoto
How to create a network monitoring system
Most organizations buy network monitoring systems from vendors, but those that choose to build in-house must follow a detailed set of steps to ensure a comprehensive platform.
A network monitoring system is a tool or set of tools used to track information about a network and its devices. Network monitoring systems collect network-relevant information about other devices connected to the network, such as servers and laptops.
For each entity that a network monitoring system monitors, it observes one or more of the following features:
- Network behavior. Traffic flowing through the network devices.
- Network performance. The efficiency of traffic flowing through the network.
- Device configurations. The way devices manage traffic flow and regulate access to themselves.
Network monitoring systems enable network teams to manage a network more proactively and realistically prepare for capacity planning. They can also more effectively and efficiently troubleshoot and mitigate network problems when they occur.
Network teams must decide how to acquire a network monitoring system. These choices include deciding whether to deploy the system on-premises or in the cloud; going with an imperative, command-based approach; or using a configurational, state-based approach. One of the most fundamental choices, though, is whether to build or buy a network monitoring system.
Organizations that choose to build network monitoring systems have specific use cases and purposes for building the platform. This article evaluates those reasons and compares the process of building a network monitoring system to buying tools from vendors.
Considerations for building a network monitoring system
Reasons to consider building a network monitoring tool in-house come down to three factors:
- Cost. Organizations believe they can build something sufficient to their needs at a lower cost.
- Scope, customization and control. Organizations might build their own tools to keep the platform more focused on the set of functions they need and to deliver the capabilities securely.
- Trust and risk. Organizations want to have a network management system with minimal added risk.
How to create a network monitoring system
An organization that wants to build a secure network monitoring system that meets its monitoring needs should follow a few steps. The process of creating a network monitoring system is as follows:
- Define the requirements.
- Choose a programming language.
- Use AI to simplify the process.
- Follow coding standards.
- Follow proper design principles.
- Secure the network monitoring system.
1. Define the requirements
Before building the network monitoring system, the organization must define the core functional areas of performance, behavior and configuration that the network management system needs to address. Specify the list in order of importance to provide guidance for the tool's structure and a rough order of feature development.
Define core requirements for the objects that the network management system will monitor. This documentation can include everything on the network or some specific subsets of equipment.
2. Choose a programming language
If an organization is building a network management system without prebuilt code, it must pick a programming language. To avoid starting entirely from scratch, choose an open source package to mine for code and strategies or to use as a base to further develop your system. If an organization chooses to go this route, the network team should be aware of the licensing terms that apply.
Network teams should also reference materials on the creation of other network management systems. It's helpful to look at the code of other open source systems, even if you decide not to use any of them. Open source systems provide a wealth of examples for how to -- or how not to -- build a system.
It's also important to ensure all members of the network team who will use the system are involved in the design and testing cycles. Ongoing collaboration with end users helps ensure the tool works well for their needs.
3. Use AI to simplify the process
If an organization has access to AI tools that support programming, the network team should use AI for coding assistance. Network teams can use AI to generate the basic framework for the monitoring system, which typically includes the following:
- Files.
- Skeleton functions.
- Partially specified objects.
- Data structures.
- Interfaces.
The organization can use an AI tool to implement these required components based on detailed guidance on the structure and function. However, before asking AI to build anything, the developers should have a clear understanding of what they want it to build. They should also feed the AI example code of the current network monitoring code to demonstrate how it can create the system.
Also, if the AI has been trained on identifying potential security threats, the network team should ask the AI to examine the code for any security issues. Many problems are typically introduced in a codebase early in its life, and the longer they remain, the harder it is to fix them.
AI can help network teams with the grunt work of coding the network monitoring system and handle the subtleties of application security, like validating inputs before use or bounds-checking an array of operations. This can reduce development cycle times and enable the team to focus their attention on important questions of function.
4. Follow coding standards
The network team should adhere to naming and other coding standards as they create the network monitoring system. If their organization already has guidelines for in-house application development, the team should use them as much as possible. If their organization doesn't have any standards, it should develop some at this point. Following standards makes it easier for teams to collaborate and helps later developers understand the system's construction.
A code management system also helps with areas like branch management and versioning. In addition, the network team should apply change management principles to the network monitoring system once the team starts to use it.
5. Follow proper design principles
The network must follow good design principles throughout the process. In the case of creating a network monitoring system from scratch, it means building a cluster of modules that each deliver a specific function -- or a tightly related set of functions -- that work against shared databases.
This means the network team must design APIs for the components. The design should consider the need for additional functions added to the system over time. Also, the network team should use a database that can deliver the necessary performance at scale.
6. Secure the network monitoring system
Network teams should prioritize the security of the network monitoring system as they build the tool. As soon as the network team has release candidates for the monitoring system, they must subject each version of the software to full security scans and red-teaming attacks before deploying them in production. The guiding principle should be that, if a piece of the system doesn't work securely, the part doesn't work at all.
The purchasing alternative
If an organization decides not to create a network monitoring system in-house, it can buy monitoring tools from network vendors. This is the case for many organizations, as most don't create network monitoring systems from scratch. Vendors are the ultimate authority on the capabilities, performance and configuration of their own equipment.
Enterprises can also buy tools from management vendors that sell management tools and services rather than network equipment. These companies are experts in building management tools and tools for a broad range of equipment. Over time, most organizations come to have several tools from several vendors in place.
Most organizations also bring in one or more open source network monitoring tools. This reduces the effort and expense of building their own tools, as well as the expense of buying another tool. For smaller organizations in particular, open source tools are at the core of monitoring operations. In larger organizations, however, open source tools are typically used to provide some specific set of functions, such as basic uptime monitoring.
The reality of building a network monitoring system
Organizations sometimes decide to create their own network monitoring tool rather than buy one or rely on open source, believing it to be more cost-effective and secure. If an organization decides to invest in the developer and network staff time required for this, IT must follow the same software development practices as it would for a piece of customer-facing software.
It might be tempting for an organization to think it can build a better tool when faced with some poorly designed network monitoring systems or tools. However, creating a network monitoring system isn't a trivial project. It's imperative for organizations to follow best practices to deliver resilient, performant and secure network monitoring systems that suit their needs and purposes.
John Burke is CTO and a research analyst at Nemertes Research. Burke joined Nemertes in 2005 with nearly two decades of technology experience. He has worked at all levels of IT, including as an end-user support specialist, programmer, system administrator, database specialist, network administrator, network architect and systems architect.
