From 1.6.0 Checkmk includes the new concept of Labels. A host can contain any number of Labels, and these are very similar to Tags:
- Labels are ‘attached’ to hosts in the same way as tags.
- Labels can be used in the same way as tags as conditions for rules.
- Labels are constructed similarly to tags on the key:value principle.
So why is there a new concept here? Well - the IT world is changing and becoming much more dynamic. Cloud and Container systems like Azure, AWS and Kubernetes autonomously create and delete objects – which correspond to hosts in Checkmk. In these technologies tags/labels play a big role because they make the connections between the supervised objects and their functions. The host names, on the other hand, are increasing random and meaningless.
Checkmk can create such dynamic hosts automatically with the new Dynamic Configuration Daemon and gets the information about the already existing labels/tags. These you can then use for rule conditions, searches, evaluations, and other tasks.
Of course the question arises – why we do not simply map such dynamic labels to the existing concept of host tags. And in fact that is also a very very obvious conclusion at first look. However host tags have a very important property which is very difficult and complicated: existing tags and tag groups and tags are attached rigidly to Checkmk. Everything is well defined. Every host has exactly one tag from every group. Everyone can count on it. Typing errors in the spelling of tags cannot occur – as with hosts, which do not stick to the scheme – because its compliance is strictly controlled by WATO. This is very important and useful in very heterogeneous environments with many thousands of managed hosts.
By contrast, dynamic labels from Kubernetes and Co are quasi-‘freeform’. And even if they do follow a scheme, this is unknown to Checkmk. In addition you might be monitoring several different platforms, which in turn use labels in very different ways.
That is why a new Checkmk-Labels concept has been introduced which suits this growing dynamic. Of course you can also use the labels even if you don’t use connections to cloud environments.
Here are the features of Labels:
- Labels do not have to be predefined anywhere. There is no fixed scheme for labels. Everything is free form. Everything is allowed.
- Each host can have as many labels as you like. These can be maintained manually, defined via rules, or created automatically.
- Labels are structured according to the key: value principle. Each host may only have one value per key. So a host that has the label foo:bar cannot at the same time have foo:bar2.
- Unlike the host properties, both the key and the value - except for the colon - may contain any characters.
- There is no distinction between ID and title (display name). The name/key of the label is at the same time its ID and display name.
Labels have the following tasks:
- They form a basis for conditions in configuration rules (for example, all hosts with the label os:windows should be monitored this way).
- It is very easy to store additional information or comments about a host (for example, Location:RZ 74/123/xyz ) and to display this in views for example.
- Labels are passed to Grafana over the new connector and can be used there for filtering, aggregation, etc.
2. Creating Labels
2.1. Explicit Labels
A host can be assigned labels in three different ways. The first of these is simple: In the mask in which you create a host, you can arbitrarily define as many labels as desired:
To enter the labels, activate the checkbox attribute as usual, click on the Add some label text, and there define your labels. Enter the labels according to the Name:value scheme and close each label by pressing enter.
You can edit an existing label by double-clicking it in its text, or delete it with the little cross.
Note: Both the name and the value of a label may include any character - except the colon! However you should think carefully about the use of characters and upper/lower case, since if you later define conditions via labels then the spelling of both the name and the value must be strictly observed.
Of course labels can also be inherited from a folder. Like other attributes, labels can be in subfolders or at the host, then as needed be overwritten again - individually in fact. If for example, the label location:munich is set in the folder, this will be inherited by all hosts in this folder which do not themselves already have the label location defined. Other labels a host may have will remain unaffected.
Explicitly-defined labels of hosts or folders appear violet in the list of hosts:
2.2. Creating labels via rules
As usual in Checkmk, attributes can also be applied to hosts and services by rules. This will make you independent of the folder structure. This also applies to the labels. There is a rule set for this function Host labels.
The following rule adds the hw:real label to all hosts in the Bayern folder which have the host feature Hardware type is real hardware:
You may have noticed that labels cannot be used in the conditions for this rule! This is not a mistake, but intentional, and it avoids recursive dependencies and other anomalies.
2.3. Automatic Labels
The third way labels can be created is fully automatically. Various data sources such as the special agents for monitoring cloud and container systems, as well as the agent plug-in of the hardware/software inventory automatically generate labels. Such labels are displayed in orange.
The nice thing is that you do not have to configure anything at all. As soon as these data sources are active, the corresponding labels arise. Here are some details:
The inventory system in Checkmk finds static information about the hardware, the operating system and the software installed on a host. Depending on that labels can be generated automatically. These can be found in the Inventory Tree view in the Software ➳ Applications ➳ Check_MK ➳ Discovered host labels path:
Labels on objects in Amazon Web Services (AWS) (AWS uses the term tags for this) are automatically taken over by Checkmk.
Labels are also called Tags in Azure. Details about labels when monitoring Azure with Checkmk can be found in its own article.
Above all, Kubernetes works intensively with labels - which as well are known there as such. A distinction is made between automatic labels (e.g., pod_id, pod_name and pod_namespace), and those that the administrator itself assigns. Both types migrate as automatic labels directly after Checkmk.
2.4. Labels in the discovery check
If you have enabled the discovery check – which is the default for new installations – it will warn you when new host labels are found that have not yet been added to the WATO properties of a host. This will look like this, for example:
You have two options for responding to this warning. The first is to add the new labels by calling the service configuration of the host in WATO and updating the configuration of the labels with the Update host labels button. The discovery check will then be OK again the next time you run it (in up to two hours), even if you have not yet activated the changes.
If this affects many hosts at once, you will certainly not want to visit the service configuration for each one. The best way to do this is to run Bulk discovery and select the Add unmonitored services and new host labels mode.
The second way to get the discovery check green is to reconfigure it so that it no longer prompts for new labels. To do this, go to the ruleset Monitoring Configuration ➳ Inventory and Check_MK settings ➳ Periodic service discovery and edit the existing rule. There you will find the option Severity of new host labels:
2.5. Sequence of label assignment
Theoretically, the same label may be defined with different values simultaneously in multiple sources. That's why there is the following order of priority:
- First of all, explicit labels, i.e. those that you define via WATO directly at the host or folder.
- In the second place are labels that are created by rules.
- In the last place are automatic labels.
These precedence rules give you the ultimate control over the labels.
3. Labels as conditions in rules
An important function of labels is the same as with tags, namely their ‘Use’ condition in rules. This is especially true with automatically-generated labels, because they perform their monitoring fully-automatically according to information from AWS, Azure, Kubernetes and Co.
The following example shows a rule condition that applies if, and only if the host has the label state:bavaria but not the label environment:test:
You can use both labels and tags in a rule. These will be automatically AND-linked. The rule only applies if both conditions are met simultaneously.
Please note that the exact spelling of the labels is important. Since labels are freeform and WATO therefore cannot know exactly which labels really exist, it cannot recognize typing errors. If that causes isolated problems it may be more effective if you work with tags, since these work with selection boxes instead of with text input.
4. Labels in Views
So far we have only talked about the configuration. The labels are also visible in the monitoring itself. This starts with the host details:
Since the labels are also clickable, they are not just for appearance: With a click you will then be forwarded to a search for all hosts with this one label. You can also do something similar in the Views’ search function – here there is a new search box that will enable you to for search for labels. The entry is made here using an interactive search for all existing labels:
5. Service Labels
Services can also have labels. These are similar to the host labels, however with a few small differences:
- You cannot define service labels explicitly. These can only be created by rules (Service labels), or automatically.
- You cannot currently formulate any conditions via service labels, however this will soon be possible.
6. Labels in Grafana
For Grafana Datasource is currently being developed with which you can access the historical metrics of Checkmk directly from Grafana. If you use these Grafana automatically receives the information about all host and service labels. This allows you to more easily group Checkmk metrics and work with templates in Grafana.