Azure is Microsoft's answer to the cloud which allows users to use the concepts of infrastructure as a service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS).
Even though Microsoft manages the operation of the underlying hardware and software components,it is still necessary and useful to monitor your Azure cloud services. However this must be done in a different way than by using a regular Checkmk agent.
From version 1.5.0p12 Checkmk supports monitoring of Azure through its own special agents, as well as with a number of check plug-ins. Currently these are as follows:
- Microsoft Azure Agent Info
- Microsoft Azure SQL Databases
- Microsoft Azure SQL Databases: CPU utilization
- Microsoft Azure SQL Databases: Connections
- Microsoft Azure SQL Databases: Deadlocks
- Microsoft Azure SQL Databases: Storage
- Microsoft Azure SQL Databases: Throughput units
- Microsoft Azure Storage Accounts
- Microsoft Azure Storage Accounts: Data flow
- Microsoft Azure Storage Accounts: Performance
- Microsoft Azure Virtual Machines
- Microsoft Azure Virtual Machines: Summary
- Microsoft Azure Virtual Network Gateways
- Microsoft Azure Webserver
As we continue to evolve support for Azure monitoring, more plug-ins will be added. Please also note that up to Version 1.6.0 it is still possible that structural changes to the implementation may be made and you might need to adjust your configuration again.
2. Preparing Azure for Checkmk
2.1. Create the App
To monitor Azure with Checkmk, you will need your subscription ID and your tenant ID (also known as ‚Directory ID‘).
First, register Checkmk Monitoring as an app so that you can work with the Azure API. The option for this can be found in Azure portal at Azure Active Directory ➳ App registration ➳ New application registration:
Assign a name of your choice. In the example we use my-check-mk-app. This is only informative. The reference to the app is instead made via a UUID which you will see in a later step. The Application type must remain set to Web app/API. The Home page like the name is just informative. It is best to specify the URL used to access your Checkmk server.
After the creation select the new app from the list of apps. If it does not appear in the list, then query Select My apps on All apps. In the details for the app you'll also find the Application ID you'll need later. The Object-ID is not required.
2.2. Assigning permissions to the App
In order for your new app to have access rights to the monitoring data, you must assign them here. On the left of the main navigation page select the item All services and there the point Subscriptions:
Select your subscription from the list here. In the navigation of this page go to Access Control (IAM) and from there to Add role assignment:
Now enter the Reader role and Assign access to the value Azure AD user, group, or service principal, and Select your new app’s name:
2.3. Create a key for the app
Now you need a key (a secret) with which Checkmk API can log on at the API. Enter this under the properties of the app again under Settings. The entry is called Keys. For once you will not find no button to add this or anything similar. For a change Microsoft would like you to enter enter a name of your choice here in the Key description field. We chose my-check-mk-key in this example. Do not forget to make a selection in EXPIRES that is suitable for you.
The Safe button has now turned blue. Save the settings with this. Then your key will be displayed:
Attention: the note Copy the key value. You will not be able to retrieve it after you leave this blade. is meant seriously. This is the only time the key is ever displayed! Note it for later reference.
The setup under Azure is now complete and you should now have the following four pieces of information:
- Your Subscriptions-ID
- Your Tenant-ID (also known as the ‚Directory-ID‘).
- The Application-ID (Client-ID) for the App my-check-mk-app
- The secret of the key my-check-mk-key to this app
If you do not have your tenant ID at hand, find it by hovering over your login name in the pop-up help under Directory: default directory ....:
3. Setting up monitoring in Checkmk
Even if you are not dealing with a physical host in Azure, create a host for your Azure directory in Checkmk. The host name you can define at will. Important: Because Azure is a service and therefore does not have an IP address or DNS name (access is granted by the agent itself), you must set the IP Address Family to No IP.
3.2. Configuring the Azure-Agent
Since Azure cannot be queried through the regular Checkmk agent now you set up the Azure Special Agent – which is also called data source program. In this situation Checkmk does not contact the destination host over TCP port 6556 as usual, instead it calls a utility that communicates with the target system via Azure’s application-specific API.
To do this, under Host & Service Parameters ➳ Datasource Programs ➳ Microsoft Azure create a rule whose conditions apply exclusively to the Azure host that has just been created. There you will find the input fields for the IDs and the secret:
Here you can also select the resource groups or resources that you want to monitor. If you have not checked explicitely specified groups, all resource groups are automatically monitored.
If you now perform a service discovery on the Azure host, only a single service called Azure Agent Info should be detected:
If access to the API does not work (because of a wrong ID or bad permissions, for example), an error message from the Azure API appears in the status text of Azure Agent Info:
3.3. Make resource groups available as hosts
For clarity, Azure monitoring in Checkmk has been designed so that each Azure resource group is represented by a (so to speak) logical host in Checkmk. This is done with the help of a piggyback procedure. This piggyback will take data from the Azure host using special agents, and within Checkmk redirect it to these resource group hosts.
The resource group hosts do not automatically appear in Checkmk. Place these hosts either manually or - from Version 1.6.0 - optionally with the new Dynamic Configuration Daemon (DCD). Important – when doing so the names of the hosts must exactly match the names of the resource groups - and it is also case-sensitive! If you are uncertain about the exact spelling of the groups’ names, you can do this directly from the Azure Agent Info service on the Azure host.
By the way: with the auxiliary script find_piggy_orphans from the Treasures Directory you will find all of the Piggyhosts for which there are data but which have not yet created as a host in Checkmk:
OMD[mysite]:~$ share/doc/check_mk/treasures/find_piggy_orphans Glastonbury Woodstock
Configure the resource group hosts without an IP address (analogous to the Azure host), and select No Agent as the agent.
Now if you perform a service discovery on one of these resource group hosts you will find there are additional services that specifically relate to this resource group:
Choose different names for the resource group hosts
Tip: If you want to freely choose the names of the resource group hosts, with the Host & Service parameters ➳ Access to agent ➳ Hostname translation for piggybacked hosts rule you can define a conversion of resource groups to hosts.
3.4. Virtual machines (VMs)
When you use Azure to monitor virtual machines, they simultaneously serve as your normal host in Checkmk – you can use the Azure services associated with those VMs instead of the resource group hosts associated directly with the VM hosts in Checkmk. Do this in the Azure rule by choosing the Map data relating to VMs option. Map data to the VM itself. For this to work the VM’s Checkmk host in monitoring must have exactly the same name as the corresponding VM in Azure.
3.5. Rate limit for API queries
Currently the API queries that Checkmk needs for monitoring Azure (as opposed to AWS) are free – however there is a limit to the number of queries permitted per time period (‚Rate Limit‘). Per Application ID the limit is 12,000 read requests per hour.
Due to the structure of the API, Checkmk requires at least one or more queries per requested resource. Therefore the total number of queries scales linearly with the number of resources being monitored. If the query limit is reached or exceeded, the query fails with a HTTP code 429 (too many requests), and the Check_MK service for the Azure host is flagged as critical.
This rate limit is the realization of Azure's so-called ‚token bucket’ algorithm. It all starts with you having a ‚credit‘ of 12,000 remaining queries. Each query consumes one of these. Simultaneously 3.33 queries per second are added to the credit. The output of the service Azure Agent Info lets you see how many queries are currently left.
Specifically, this means that:
- If your query rate is sufficiently low, the available queries are always just under 12,000.
- If your rate is too high, the credit will slowly go down to 0 and then errors will occur sporadically in the query.
In this case, you can reduce the polling rate by querying fewer polling resource groups or resources, or by reducing the check interval in the active check Check_MK on the Azure host. This is possible with the Monitoring Configuration ➳ Normal check interval for service checks rule.
So that you can react in time, the Azure Agent Info service monitors the number of remaining queries and warns you in advance. By default the warning threshold is 50% and the critical threshold at 25% for remaining queries.