Optimize cloud workloads with effective Azure monitoring

Azure monitoring includes a handful of broad areas to monitor. We will start with the broader areas, namely performance and security, and then move on to the more specific areas, like networks, computing, storage, and databases.

Among the options for monitoring an Azure cloud, Azure Monitor holds a central place. It is the software that aggregates multiple sources of metrics for monitoring, and we will encounter it the most when talking about Azure monitoring. In case of Azure performance monitoring, Azure Monitor uses Application Insights, an extension that allows it to perform application performance monitoring (APM).

This extension includes not only the classic metrics such as error rates, used memory, CPU usage, HTTP request rate and more, but also pulls in telemetry data and application logs, creating a rather complete view of Azure application monitoring and performance.

Azure performance monitoring thus uses a combination of sources to get information about events and statuses, to generate statistical evaluation of the past performance and to predict future trends. Azure Monitor has a specific dashboard with multiple widgets and views that focuses on the various aspects of Azure performance monitoring.

Third-party Azure monitoring tools are quite similar in that regard. The difference is that while Azure Monitor is integrated in Azure clouds and is a proprietary solution, other tools like Checkmk are external and use REST APIs and agents to gather the data from Azure services and resources. Such an external software can be locally installed or in the cloud along with the other Azure services.

Azure Monitor with Application Insights is the first interface that an administrator can use to monitor the performance of Azure services. However, it is not very flexible and limited to Azure clouds. For larger infrastructures and more advanced needs, a separate Azure monitoring tool is a necessity. But for quick checks and moderate infrastructure, Azure Monitor is a sufficient starting point.

Security is a key area in all monitoring, and Azure monitoring is no exception. Ensuring that data and resources are accessed by only authorized people and that nothing has been tampered with is an extremely important aspect in every infrastructure.

Azure offers a few tools for Azure security monitoring, and the most important is Microsoft Sentinel. It is a cloud-native solution that collects data across users, devices, and applications, both on-premises and on the cloud. With Sentinel, it is possible to investigate threats with the help of artificial intelligence, to detect previously undetected ones, and to report incidents. It is a Microsoft-only solution, integrating data sources coming from both Azure and local applications, like Microsoft Defender or Office 365.

Monitoring tools like Azure DDoS Protection and Azure Rights Management (RMS), on the other hand, are only suited for Azure. As their name suggests, they are small Azure monitoring tools that undertake one specific task within Azure security monitoring. For monitoring roles and policies, Azure Governance Visualizer provides administrators with a graphical view of their status.

As in all other areas of Microsoft Azure monitoring, these tasks, and more, can also be accomplished with non-Azure tools that support Azure cloud monitoring. Checkmk and similar tools can draw from multiple data sources and have many Azure monitoring metrics at their disposal for maintaining a secure Azure infrastructure.

Azure network monitoring deals with checking that the virtual networks within an Azure cloud are working efficiently. Monitoring these networks is vital when making sure that any cloud-based infrastructure is optimally performing and preventing issues. Azure has a couple of tools to perform Azure network monitoring.

The most obvious choice is another extension to Azure Monitor named Network Insights. It provides a visual representation of topologies, health, and metrics for all the available network resources, without much configuration effort. The tool is split into a few monitoring components, going from diagnosing the traffic and connectivity, to simply showing network topology and general health. Through it, a few other tools can be accessed for deeper insight, such as Connection Monitor and Traffic Analytics.

Specifically for virtual networks monitoring, there is Azure Network Watcher. It is another tool integrated in Azure clouds, and it makes it possible to perform a series of diagnostics to check network traffic filtering and routing problems, to diagnose outbound connections, and to capture packets from a VM. Mostly, it is an Azure network monitoring tool focused on virtual machines and how they are interconnected, but with not a lot of insights outside this area.

A cloud infrastructure would not be of great use if it did not offer computing possibilities. Running containers, applications, and serverless functions on a cloud is one of its primary uses. Azure computing monitoring includes a series of tools and efforts that monitor the performance and health of these Azure services.

Azure has a couple of extensions for Azure Monitor: Azure VM Insights and Azure Container Insights. As their names suggest, they deliver metrics and logs to Azure Monitor from virtual machines and containers, and present them in their dashboard. These metrics come from different types of VMs and containers, such as Linux or Windows virtual machines, Azure Functions, mobile and web apps on Azure clouds, Azure Kubernetes Service (AKS), OpenShift and generic container instances. There are multiple use cases for all these computing Azure resources, but broadly speaking, all of them have the metrics CPU, memory and disk usage, network traffic used, and the number of active connections in common.

By monitoring these active Azure resources it is possible to understand how much of the infrastructure is used, how much is under stress, or unused, and to then accordingly scale it up or down. Since there are a few, complex services, it is necessary to deploy a third-party Azure monitoring tool instead of the integrated Azure tools. Monitoring Azure clouds with an advanced Azure monitoring service or on-premises tool is advantageous when dealing with a larger set of metrics and data, which can prove strenuous on less performing tools.

The data for applications reside in Azure Storage services such as Blob Storage and Managed Disks, and they need to be monitored for data corruption and to avoid going over their space limits. One of the simplest tools to keep an eye on storage in Azure clouds is another extension for Azure Monitor called Storage Insights.

With this extension, you can observe Azure Storage services performance, capacity, and availability in a straightforward interface. You can also include all storage-related resources within an Azure cloud account in the monitoring, and receive a unified dashboard across all storage services on a single infrastructure.

While Storage Insights offers insightful metrics like latency, transaction type and errors, number of failures, and total capacity used by each data service, it is not advanced enough to serve all use cases. As far as Azure monitoring services go, it is fine, but it only has limited customizability and can’t serve more complex demands. A third-party Azure monitoring software can scale much better and offer more monitoring power to ensure that the data fueling your Azure cloud applications are safe and performing.

The second side of data in an infrastructure are databases. When dealing with Azure-based infrastructures, there are many choices, like MySQL, PostgreSQL, Azure SQL, MongoDB, MariaDB, and Cassandra. All have their pros and cons, which go well beyond the scope of this monitoring guide. In terms of Azure database monitoring, Azure itself offers a few tools for Azure Monitor that help cloud administrators keep track of the health and performance of each type of database.

As by now is unsurprising, Azure Monitor integrates monitoring database services with dedicated extensions for each type of supported database. Azure SQL Insights is still in the preview phase (as of 2023) but it is already capable of extracting plenty of useful metrics from Azure SQL databases. Metrics such as sessions, requests, performance counters, I/O, wait statistics and more can all be checked with Azure SQL Insights within the larger interface of Azure Monitor. Other types of databases have their extensions within the Azure monitor dashboard.

A particularity of Azure database services is Cosmos DB: a globally-distributed NoSQL database that is designed to provide high availability, scalability and low-latency access. If used, it can be monitored with its dedicated extension called Cosmos DB Insights. Since it is an additional service, it is important to take care of its health and efficiency, just like any other important resource in an Azure cloud. Metrics such as number of requests, data usage, failed requests, index usage and more are part of Cosmos DB metrics, similar to other databases.