Analyze Network Traffic with Network Flows
Monitoring of network flow provides a in-depth analysis of the network and shows who is talking to whom, and over which protocols.
Network flow: A look into the meta-information
Most network monitoring tools already provide many interesting insights into a network. How extensive this is depends primarily on the information that the respective software receives from the various network components. Since most monitoring solutions work with SNMP, they depend on the data provided by the SNMP agent. This provides – if the protocol is implemented correctly – a very good insight into the network utilization, e.g. the bandwidth on the different network interfaces, the status and/or the utilization of the devices in the system.
Administrators who require a more in-depth analysis of their network should therefore use monitoring software that offers more advanced functions, such as examining and monitoring the network flow. Network flow is meta-information derived from the data traffic. A flow data record thus provides information about who is talking to whom in the network, and over which protocols. It provides information about who is using which port, and how much bandwidth each application is using on each port. It thus enables more thorough root cause analysis, helping to identify bottlenecks in the network more quickly.
Network flow also allows packet inspection. The network flow data sets are very tightly consolidated, but such packet insights require access to the raw packets – which can only be obtained via a mirror port or a Network Tap. However, these can usually also be connected as a data source for a network flow analysis.
Top talker and top listener in your network
With the help of the network flow, it is possible to obtain detailed user data from the used switches and routers – provided these support a network flow protocol. This way, the monitoring solution in operation can list top talkers and top listeners in the network, as well as breaking down network usage by source, destination address, protocol or application. In addition, the administrator also receives further information on what is happening in the network.
The flow data is provided via various protocols, such as NetFlow or sFlow. Cisco originally developed NetFlow with the intention of simplifying the creation of access control lists. However, the data collected has proven to be very useful for additional analysis, so the network manufacturer has developed the protocol further.
Although other analysis techniques from other network manufacturers, such as J-flow from Juniper Networks, NetStream from Huawei or Rflow from Ericsson, are now available, Cisco's technique is the most common. With IPFIX, the IETF (Internet Engineering Task Force) has also introduced an industry standard for the export of flow data records based on NetFlow version 9. Many producers now support this protocol with their devices.
With flow examination, the administrator is able to know who is doing what, when and where in the network, and how the data traffic flows in their infrastructure. It also enables the administrator to detect any anomalies that may indicate malware or other undesirable network events.
The data also helps with capacity planning in the network. Finally, an inspection of network flow can be used to determine how much network bandwidth an application requires. On the one hand, this helps to assess the impact of certain applications on the network. For example, peaks can be quickly identified and connection bottlenecks in the network optimized.
On the other hand, the information is also useful for the implementation of a Quality of Service (QoS) in a network environment. QoS concerns the prioritization of data traffic. This means that the data packets of a business-critical application which depends on low latency – for instance VoIP (Voice over IP) – are given ‘priority’ over data packets from a less latent application, such as emails.
How network flow works
A flow packet contains various types of meta-information, such as the IP address of the sender and the receiver, the source and destination ports, Layer 3 protocol types, the classification of the service, and the router or switch interface. The protocol groups all packets with the same contents into a flow and then summarizes the packets and bytes together at the end.
Network components that support a network flow protocol create and send such a flow packet via a push method. A collector then gathers these data records. Depending on which protocols it supports, this collector converts these data into standardized values, thus preparing them for examination. The collector then forwards the data to a central instance, which then displays the processed data graphically, making it available in the form of a GUI for deeper insights.
Network flow monitoring with Checkmk
With Checkmk 2.0 the monitoring solution is extended by the possibility of network flow monitoring. For this purpose tribe29, the company behind Checkmk, integrates the ntop network flow monitoring solution from the company of the same name into tribe29’s own software. With the aim of providing a uniform look and feel, from Checkmk 2.0 onwards the network flow data from the ntop servers should be easily accessible via Checkmk monitoring – without the user having to leave the software interface. Watch our presentation about the ntop integration in Checkmk.
Netflow monitoring with ntop
The ntop architecture
The network flow monitoring of ntop is based on two components: the collector called nprobe, and the ntopng analysis and visualization console.
In nprobe, traffic data is collected from switches and routers on the network by collecting their NetFlow, sFlow or IPFIX records. The collector then prepares this data for analysis and sends it to ntopng, which provides a web-based, graphical overview of the data.
By visualizing the data, a user can gain the insight into his network infrastructure that they need. In addition to flow data, nprobe can also be fed with ‘raw data’, i.e. data packets not aggregated into a flow from a mirror port or network tap, to perform a deep packet inspection.
Visualisation of the network traffic
ntop provides the network administrator with insights into their own network that go far beyond the information obtained via SNMP. This includes:
- Network flow analysis, such as top talkers, etc.
- Deep performance monitoring, such as delay, round-trip times, etc.
- Threat detection support, by quickly detecting threads such as DDoS attacks.
With ntop it is possible to view the top talkers of a port, check where the data is going, who or what is using the most bandwidth of a port, and which is the most frequent destination address of a port. The administrator can also view the data traffic in real time and, for example, examine the most active interfaces and top applications. Furthermore, it is possible to analyze historical traffic data to, for example, identify anomalies or trends.
The network flow monitoring in ntop also provides administrators with more detailed information on the network’s hosts. Various filters in the dashboard allow you to view numerous details for a host, such as traffic, packets, ports, peers or applications. This makes it possible to use the host information provided by Checkmk to enrich additional details from ntop’s flow monitoring.
For this purpose, each host overview in Checkmk will contain an ntopng icon for viewing the additional flow data from ntop. This is made possible by the hardware and software inventory in Checkmk, which can search for interfaces such as ntop’s host view.
As already mentioned, ntop also provides the function Deep Package Inspection (nDPI). Deep Package Inspection allows ntop to break down which application protocols are used by a specific IP address over a specific port. In this way, it is possible to trace which applications a user is using, such as Microsoft Office 365, Citrix, SSH or Checkmk. According to its own description, ntop supports over 250 different application protocols. Since more and more network traffic is also encrypted via SSL, nDPI can support encrypted connections and check for their encryption certificate using a decoder for SSL certificates from clients and servers. In this way, application protocols for Citrix Online or Apple iCloud can also be identified, which otherwise remain undiscovered in encrypted data traffic.
Alerts: Detecting when something is wrong
With network flow monitoring, it is of course not only possible to analyze network traffic. It is also possible to get an overview of all alerts in ntop. The alert dashboard in Checkmk provides a table of active alerts, such as when thresholds have been exceeded. Another table provides an overview of past alerts that are no longer active (past alerts). The third available table of the dashboard lists the flow alerts. In this category, ntop reports anomalous or suspicious data flows. These alerts do not appear in the active or past alerts table. This helps the administrator to get an overview of all the problem areas in his network.
Ready to explore the full feature set of Checkmk?
Download the free trial and see it in action.
