What is network monitoring?
Network monitoring is a complex and vital topic for any company. Knowing its benefits and sub-areas is crucial to getting started right.
Network monitoring is one of the processes that make the larger network management area. Network monitoring is a multifaceted endeavor that at its basic deals with keeping track of what is happening within a network in order to discover issues, act upon problems, optimize performances, and network efficiency. This is done through a network monitoring software that takes care of collecting metrics, analyzing and presenting them to network administrators, so action can be taken, whenever necessary.
What was once a simple task for the limited extensions of networks of old has become staggeringly complex as networks became larger. Different network monitoring protocols have been created along with the tools to effectively monitor a network to help the administrators wade through the numerous hosts and devices to monitor. Network monitoring metrics collected through these protocols and network monitoring services that make use of them are the core of what is network monitoring.
Network monitoring does not only include monitoring the health and performance of a network, though. It includes subtopics like network performance monitoring, network security monitoring and plenty of others. All or most of these are supported by a network monitoring system, granting the administrator a holistic view of the infrastructure.
Who is network monitoring for?
Simply put, monitoring is like getting the pulse of an infrastructure. Anybody who wants to prevent, know, and act on possible issues and malfunctions, is naturally interested in network monitoring. This includes all who are in charge of operating the various hosts and devices that make a network infrastructure, not just those who set them up. Be it a real or virtual one, anybody who is tasked with maintaining or operating a network-connected device is within the spectrum of those who should consider monitoring it.
In larger companies, a specific department may be devoted to network monitoring. Often the security team also does network monitoring, since ensuring that a network operates securely is also done through constantly monitoring it. Network security monitoring is an important branch of monitoring, as a network has to be both performing well and be secure from outside attackers to be considered fully “healthy”.
Why is network monitoring important?
A high performance network is at the core of companies' IT, and not only that. Ensuring that the network infrastructure operates to the best of its capabilities, without bottlenecks, hiccups, or errors, is vital for the whole company. To offer its products or services, and for its employees to efficiently work on them, an efficient network is crucial. Regardless of the company’s industry, maintaining business operations necessitates a fully functioning network. A slow or unreliable network causes disruptions and the loss of image and customers.
Network monitoring is there to prevent all this. Or, at the very least, to quickly act when issues arise. Without any monitoring system at work that can do network performance monitoring and monitor the network in general, it is impossible to identify potential problems in advance. The administrators are left to react to problems after they have already occurred, and even worse, without accurate monitoring it becomes a guesswork of where and why the issue occurred, with no guarantee of prevention.
Benefits of network monitoring
The benefits of network monitoring are easy to imagine. Firstly, a network monitoring service can give the administrator an overview, hopefully the most comprehensive possible, of the whole network. Knowing where the components are, how they are working, and when they are not is a clear benefit that needs no explanation.
A correlated benefit of network monitoring is in creating a topology of your infrastructure. By monitoring all your hosts, it is possible to create a map of all the hardware and software present in your network. This, along with monitoring the performance of each component, can tell you where more powerful hardware is necessary, where it is not scaling properly, where traffic congestions are and so on. This will give you the important benefit of knowing where the infrastructure needs to be updated or upgraded without actively having to check it, piece by piece.
A more subtle benefit of network monitoring is the increased security. By having a complete view of the hardware and software running, it is easy to notice all the occurring changes. When using a real time monitoring-capable tool, these changes can be immediately seen and measures taken in case of compliance and security violations. Network monitoring is therefore important not only on a practical level, but also on a legal and security one.
How to monitor networks?
Monitoring networks is done with the use of various network protocols and a suite of monitoring tools that commonly include at least one couple made of a service and a manager. More commonly, multiple services and managers. The service is also known as an agent, which is either pre-installed on the network device or has to be installed and configured manually. The manager is the software that collects the data from across the network and analyzes them, creating reports, dashboards, and handling alerts. This manager is the central monitoring software, usually installed on a network monitoring server. Agents are often present but not always, as monitoring through vendor APIs is a recent development of network monitoring that does not require an agent.
The job of monitoring the network comprises collecting metrics and sending them to the manager, which takes care of configuring the whole network monitoring system. The configuration includes analyzing and presenting the data, determining under what conditions to trigger an alert, setting up actions to automatically act on events and more. How to do it all depends on the network monitoring tool of choice.
Not only monitoring but managing the hosts and relative services is usually possible through a monitoring solution. These include configuration capabilities that allow the administrator to remotely set up and modify the configuration of network devices, but not only. If in response to an event or not, managing your network is often done through the use of a monitoring manager.
What components need to be monitored?
It is easy to think too long about what needs to be monitored when setting up a solution. Given that usually every device has to be set up for monitoring, and that the more monitored devices, the more data is collected, it is understandable to try to skip some devices. Every new amount of data is a burden in terms of both computing and human time, and some administrators may find it reasonable to try to minimize the monitoring to what matters to them the most. But this means excluding important pieces of your infrastructure, that can both be the cause and the explanation of future issues.
Therefore, the answer to what components need to be monitored is: all of them. Monitoring the entire IT infrastructure should always be attempted – and preferably with an all-in-one approach. Only by taking a holistic approach to network monitoring is it possible to have a complete, and by consequence more accurate, view of what is happening on a network. Ignoring part of the infrastructure creates the risk of having tiny blind spots in your monitoring, apparently innocuous. Yet, as everything is interconnected, these ignored spots will sooner or later be the cause of problems and disruptions, which could have been avoided if they had been monitored from the beginning.
Going into specifically what components to monitor, network monitoring is generally about checking devices like routers, switches, access points, firewalls, sensors, UPSs, network printers, etc. These are only the bare minimum, though. All connected hosts, regardless of them running Windows, Linux, Unix or else, will have to be included in network monitoring, along with virtual and cloud servers. A modern network monitoring service is naturally capable of monitoring all these types of devices, real or virtual.
What metrics need to be monitored?
The most obvious parameters to monitor are those which are a direct sign of a not-so healthy network: packet drop rates, error rates, and low bandwidth. Along with these, the state of the ports of your network devices also needs to be monitored through appropriate port monitoring. These metrics will give you the basics of if and how the network itself is working.
In coordination with the above, knowing hardware-specific metrics like CPU temperature and usage, occupied and free RAM, fan speeds, and power supply’s voltages are vital to know the overall health of network devices. Along with the traffic metrics cited before, these will give a mostly complete idea of what is causing or could cause problems.
Next, more service-related metrics are useful to collect. For firewalls, their general status, the number, and type of open ports, and the active connections. For VPN monitoring, the status of the tunnels, the level of availability are the minimum to be aware of. For software in general, their version, their status (running, installed, stopped etc.), and eventual error messages are the first metrics to take into consideration.
Modern infrastructures have both wired and wireless connections. WLAN monitoring means monitoring the status of all access points, their signal strength, the noise levels, and full list of connected devices. Since a functioning WLAN is highly dependent on external influences, it is essential for a company to include the infrastructure components for the wireless environment in the monitoring.
What is network performance monitoring?
Network performance monitoring is preoccupied with the actual performances of the network, on how efficiently it operates, and in identifying bottlenecks. It aids network administrators and analysts in gathering network data, allowing them to measure performance variables and identify potential issues or risks. A monitoring tool analyzes the performance metrics, identifying where bottlenecks and congestions are, and helping in increasing the throughput of the network once these are fixed.
Network performance monitoring is not only focused on fixing problems, but also on ensuring that the flow of information in your infrastructure is moving as fast as possible, and with no interruptions. Otherwise, they can cause anything from minor inconveniences, like intermittent disconnections, up to a network slowing down to a crawl, effectively making it unusable. By implementing performance monitoring in your larger network monitoring efforts, it is possible to know where and why the network is struggling, and take action.
What is network traffic monitoring?
Network traffic is the sum of data moving across a network at a given moment. All packets going through a network constitute the overall traffic.
Monitoring the traffic is a key element of both network security and performance monitoring. Performance, because knowing the origin and type of the packets will inform you better about the network’s requirements and shortcomings. Security, because it is an important indication of possible malware, intrusions, and unauthorized usage of the network.
Network traffic monitoring is often, but not necessarily, network real-time monitoring as well. With a constant visualization of the traffic flow, it is easier to see from where the traffic originates and where it goes, which is basically what traffic monitoring is all about. While simple packet capturing is sufficient to understand how traffic behaves on a network, network flow monitoring is specifically designed to inform you of this. Protocols like NetFlow have been developed to have a clear and immediate view of network traffic’s trends. By extracting samples from the overall traffic, you can monitor network traffic more efficiently compared to indiscriminately capturing every packet going through all your hosts, which is a rather expensive effort.
What is network security monitoring?
The last, but certainly not the least, subset of network monitoring is network security monitoring. With the increased reliance on networks of companies in every industry, the quality and quantity of data that is exchanged through them also increased. Sensitive data are present on any network and need to be protected. Ensuring their safety from unauthorized eyes is of primary importance.
Network security monitoring is about monitoring components and metrics that may indicate a less than optimal security. Connected devices, tentative of access through a not authorized device, frequent connection attempts on a closed port, unusual traffic coming from an otherwise quiet host and many other metrics like these may suggest something suspicious is happening in your network. Most network monitoring solutions, Checkmk included, integrate these checks in order to guarantee that security is kept at maximum levels. With an adequate alert system, intrusions and possible holes in your network are quickly caught and reported.
Getting started with network monitoring
With all the benefits and importance of network monitoring, it is clear that the question should not be why, but how. There are countless network monitoring services, with plenty of different features and focuses. To choose one, it is important to be aware of what they can monitor, what network monitoring protocols are supported, and how they monitor the network. Technical differences, like an agentless or agent-based monitoring solution or using vendor APIs to collect metrics, have to be considered as well before settling with one particular network monitoring software.
These are the fundamental, technical, deciding aspects for getting started with network monitoring. We will discuss them one by one in the following sections.
How to find the best network monitoring tool
While the absolute best network monitoring tool is an imaginary concept, a few features have to be present to consider a monitoring system the “best” one.
Often, network monitoring’s necessities fall into five main parameters. The first, monitoring the health and performance of your network, means having at least SNMP support in your monitoring service. This is a fundamental protocol in monitoring that is specifically designed to check the overall health of network devices, and can also manage them.
The second is monitoring the flow of your network traffic. A tool that supports protocols like NetFlow, IPFIX, sFlow is optimal here, as these protocols are all about tracking the general directions of network traffic. These protocols are insightful for upgrade planning, performance monitoring, and disruption-prevention.
The third is an active tool that can monitor your network for issues, in as close to real time as possible, and generate alerts on events. This is often one of the most sought after features, and with good reasons. Being able to continuously monitor an infrastructure and be at ease knowing that an alert will be generated when problems are found is a key parameter in network device and service monitoring.
Fourth, a monitoring tool that can present the collected data in an easy-to-read manner is an often underrated feature. Visual dashboards, network topology map, and a few clicks to highlight the issue are important aspects of a modern monitoring system that are not always supported. By accessing the collected information more rapidly, it is quicker to understand issues and to take a decision about them.
Fifth and last parameter, a network monitoring service that can do IP monitoring and scanning is important to automate the discovery step in monitoring new and existing infrastructure. It is helpful to keep the topology of your network updated in a semi-automatic way.
What network monitoring protocols do exist?
The exchange of metrics for network monitoring is done through the use of a network monitoring protocol. There are many of them, not all of which are supported by all network monitoring systems and are also no longer in use.
The major one is still SNMP. It includes not only monitoring capabilities but it can also manage the supported devices. It is a widespread protocol that is already pre-installed on many network routers, switches and more. SNMP monitoring is a vast topic in itself and has a dedicated page.
After SNMP, in order of usage if not importance, are the family of flow-based protocols: NetFlow, sFlow, IPFIX. These are designed to analyze the network traffic “flows” and are thus especially useful in network performance monitoring and security monitoring. When referring to network flow monitoring, usually one of these protocols is implemented.
All these protocols are OS-agnostic, working equally well in Linux network monitoring and Windows network monitoring. A specific protocol for Windows network monitoring is WMI, with capabilities that mimic those of SNMP and adding a few more features. Under Linux, log analysis through an implementation of the Syslog protocol or accessing each device with SSH are very rudimentary methods of monitoring.
For network traffic monitoring, RMON is often implemented as a traffic sniffer and analyzer. Being an extension of SNMP, RMON is found wherever SNMP is too, and comes pre-installed on many network devices.
Agentless monitoring with SNMP
SNMP does make use of agents in its monitoring. It may therefore seem counterintuitive to talk of agentless monitoring with SNMP. The reason is one of semantics. While an SNMP agent is usually pre-installed on many network devices nowadays, it is often configurationless, without required steps to start monitoring with it (other than perhaps activating it). The SNMP manager can poll it effortlessly, without the network administrator having to do anything.
In this specific situation, SNMP monitoring is considered agentless as the agent is transparent to the human, not requiring any tinkering with, nor any pre-configuration. It makes SNMP act like the agent does not exist at all, as with a proper agentless monitoring solution. Thus, while technically being an agent-based monitoring protocol, SNMP may also be considered agentless.
Monitoring via vendor APIs
Nowadays, some vendors do not use any of the classic network monitoring protocols and implement custom APIs in their devices. These are highly-specific but highly-customizable ways for monitoring a network. Generally, they are RESTful APIs that can be accessed with a simple
PUT HTTP request. Lightweight and programmable, these vendor APIs have a steeper initial learning curve by being non-standardized, but then have the advantage of being easily automated.
By utilizing an API, network engineers can create scripts and programs that can be interconnected and react, automatically, to monitoring events. This is essentially how most software functions, but network engineers use it to make physical and virtual changes to network infrastructure instead. Vendor APIs can expose all sorts of metrics, matching in quantity what usual network monitoring protocols like SNMP can.
A single API to monitor does not exist, but efforts to standardize have been made. OpenConfig being one of them. Currently, most of these APIs are still wildly different from each other and require a learning phase before being of use. Being highly specific to a single vendor limits their usage for network monitoring, but by themselves these APIs show a great deal of potential.
Best Practices for network monitoring
How to monitor a network depends on the specifics of a network and is hardly identical on two different networks. Still, a few best practices for network monitoring apply to any situation and company.
Why you should monitor all network ports
Port monitoring consists of checking the status and health of the physical ports on network devices. This means monitoring your router and switch ports, but is not limited to these. Port monitoring adds another layer of protection against disruptions and intrusions that generic network monitoring cannot offer.
This is because by monitoring your hardware ports, it is possible to discover faulty ones, and replace or disconnect them before they cause an interruption to your network traffic. Duplex mismatches and configuration problems can be identified through a port showing a high rate of packet loss and errors. Without port monitoring, this would have eluded the attention of any network administrator.
Problems related to the cables, like a failing or a poorly connected one, can be detected through port monitoring. For example, corrupted packets and connections being intermittently up and down can be caused by the cables, and not necessarily the port itself. Yet through port monitoring these problems can be inferred.
Security-wise, a rogue user connected to one of your devices can be discovered through accurate port monitoring. Knowing the status of each of your network ports means identifying those that are in use, those that are not, and those that should not be. A security audit may require a port scan, which a network monitoring system that includes port monitoring can comply with. Thus, port monitoring is a component of network security monitoring.
How can IP monitoring help
IP monitoring scans IP addresses to identify and include devices to monitor. This means that after installation, the monitoring tool automatically starts a scan of the entire network, or of a specific IP range or subnet, and then automatically integrates the devices found into the monitoring process. In the ideal case, during the scanning process the monitoring software recognizes the type of device and from which manufacturer it comes and, based on this knowledge, automatically includes the relevant metrics into the monitoring.
But that is not all. IP monitoring, once the first scan is done, helps in monitoring the changes relative to IP addresses in your network. Registered domains that point, or not, to the correct address, can be caught by a monitoring software that supports IP monitoring. Any change that regards IP addresses is within the sphere of IP monitoring.
As a consequence of proper IP monitoring, a monitoring software can take care of setting meaningful threshold values as standard, the exceeding or falling below of which will trigger an alarm or notification. Rules can be applied to edge cases or specific, perhaps temporary, necessities of a particular host.
IP monitoring helps in many cases, but especially in two. For inexperienced administrators, it provides a layer of automation that can greatly ease the burden of monitoring and the worries related to it. By leaving the identification step to the network monitoring system through an accurate IP scanning, errors and shadow IT are more easily avoided. When monitoring large networks, with hundreds or thousands of hosts, IP monitoring can set up the monitoring quickly and efficiently, allowing administrators to start monitoring the infrastructure in a short time.
How to monitor large networks
Large networks have an order of magnitude of complexity that makes them a separate category in the world of network monitoring. Network device monitoring may take a long time in a large network with thousands of hosts, and an excess of alerts is definitely a common problem. While it may be tempting to skip parts of the infrastructure, foregoing a holistic view, it is a poor choice. Ignoring may initially be convenient, but it leads to all sorts of future risks that a company is in no position to accept.
For the monitoring of large and complex networks with different locations and differently connected components, a monitoring software that works with a rule-based configuration is quite suitable. With a few simple steps, administrators can use rules to define a policy, such as monitoring only the error rate of all access ports, for monitoring numerous similar devices. Using rules, it is possible to apply the same setting to multiple hosts, immensely reducing the workload in configuring your monitoring setup. Alerts can be more focused, and useful, when using rules: for instance, a terminal that is regularly switched off will be set to not trigger an alert, as it is not an exceptional case but routine behavior.
A policy can be applied to a large network with a handful of well-crafted rules. A rule-based monitoring solution then handles the monitored systems based on this policy. Changing the policy can be done anytime, with a few steps, and applied to numerous devices all at once. Exceptions are also possible at any time and are documented via rules. Automation via rules also makes it easier and less error-prone to include new hosts in the monitoring.
Rule-based monitoring is not strictly necessary when dealing with a moderate amount of hosts. Small companies may operate without such a solution, using a standard network monitoring service instead. But for large infrastructures, rules can be the best solution to monitor the network.
Network topology for a holistic view
A monitoring software that is able to detect all components in a network or in a specific IP range, and retrieve the data required for the monitored metrics, enables the administrator to obtain a holistic view of even complex network infrastructures. Via IP monitoring, a complete view of the network can be presented in an easy-to-read dashboard, facilitating the identification of errors.
The network topology is often given as an overview map, showing how the network is connected, virtually and physically. The network manager can navigate this map, clicking on the specific host that needs immediate attention. Other network monitoring tools display the network in a tree structure or as a table. The table display has the advantage that several pieces of information in a condensed form can be viewed at a glance.
Not only the network topology can be visualized in a holistic view. Some network monitoring tools support viewing performance parameters graphically as well. The current bandwidth, the status of individual ports, and abnormal error rates can all be shown in a graphical way, allowing the network administrator to quickly identify patterns, such as expected or unexpected performance peaks.
All this is possible by implementing your network monitoring solution holistically, considering all the hardware and software, without exceptions.
Network documentation to combat shadow IT
Shadow IT is a term referring to the use of IT-related hardware or software by a department or individual without the knowledge of the IT or security group within the organization. It can encompass cloud services, software, and hardware. The problem with shadow IT is that these hardware and software components are often consumer products. These usually do not have the necessary security features, or are not provided with the required security patches by their producers, so that such hardware or software can quickly prove to be a gateway for cyberattacks into the corporate network.
One of the common cases of shadow IT is the use of a cloud service without the knowledge of the corporate IT department. Sensitive data can end up on an unmonitored cloud, perhaps unintentionally, violating compliance rules.
A monitoring tool that implements a holistic view of the network can combat shadow IT. By scanning the infrastructure, a network documentation is automatically created, which includes anything, permitted or not. The result is not only a topology of the network infrastructure, but also direct information about all hardware components and software solutions in the network.
Any good monitoring tool has an inventory function. This allows to document the devices and software versions present on a network, and this data can be passed to a third-party solution, such as a license management system. This is not only useful for checking the compliance of every component on the network but also to alert the administrator of smaller changes, like updating a service to an unsupported version, that can pose a security risk.
Once the network documentation is obtained, it is easy to discover connected hardware or installed software that should not be used. Without network monitoring, this would have been a tedious and manual check of each host.
Monitoring the network includes many aspects, with lots of possible shortcomings, but more benefits to reap once everything is set up. There is certainly no tool to fit all, and compromises must be made. But a complete network monitoring solution cannot easily do without IP monitoring, real-time monitoring, flow, and SNMP monitoring, packed and presented in a modern user interface. These are all features that make the life of network administrators easier and hard to renounce.
Making an inventory of your network, a complete topology, and including security checks through port monitoring and relative alerts, are far from secondary features. In the end, monitoring is not only keeping an eye on the infrastructure, but borders with managing it. To do both is fundamental to have the most complete view of your infrastructure that can be achieved in a relatively uncomplicated way. Most network monitoring tools go into this direction, easing the work of network administrators and empowering them to have the highest control on what is happening on their networks.
Checkmk does include them all, making it a complete solution for your network monitoring needs. Whether you are ok with the simplicity of the Raw Edition or prefer the enhanced support of the Enterprise Edition is totally up to you to decide. Both versions offer a great deal of features to make network monitoring easy, effective, and accurate.
Open source network monitoring refers to monitoring tools that are open source. For instance, Checkmk is open source. This does not always mean they are free, though: most open source solutions come with a cost that is asked in exchange for better support, quicker updates, and additional features. Choosing a free open source network monitoring is perfectly viable, like with the Checkmk Raw Edition, as long as it is acceptable to handle issues and configuration on your own.
Linux network monitoring refers to monitoring hosts running Linux, in one of its many distributions. Linux itself comes with a series of simple commands for monitoring networks that are standard not only under Linux, but also under many Unix-like operating systems. Htop, tcpdump, netstat are present in virtually all Linux systems and offer rudimentary network monitoring capabilities. Contemporary monitoring tools support Linux monitoring with advanced features and a much friendlier user interface.
Windows network monitoring is monitoring Windows hosts. Monitoring tools unequivocally support monitoring Windows systems through the use of a few protocols, like SNMP or WMI. Performances, services, and the event log are taken into consideration when doing Windows monitoring, similarly to other operating systems.
In monitoring, a flow is defined as a group of metrics about traffic. These flows are grouped in a special packet that includes a number of different pieces of information, such as the IP address of the sender and receiver, the source and destination ports, Layer 3 protocol types, the classification of the service, and the router or switch interface. Network flow monitoring is done with a few specific network protocols, like NetFlow and IPFIX, that specify how these packets should be created and transmitted.