Ep. 51: Automated host creation in Checkmk

[0:00:00]	Map your Cloud objects automatically via auto-registration and automated host creation.
[0:00:15]	Welcome to the Checkmk channel in the last video we configured the push mode and registered our agents in order to use this configuration option this was a manual task where we first installed the agent on the target host and after that we use the cmk_agent ctl to commands in order to register the agent against the agent receiver.
[0:00:42]	Today I will show you how to automate this process using auto-registration and automated host creation. So without further due let's get started.
[0:00:49]	For this demo I'm using a Checkmk Cloud Edition version 2.20p6, and I am also using some EC2 instances, around 10 of them. All with the same tag, I'll be using this tag in order to run a remote command on these ten EC2 instances. All of them are based on Ubuntu and the goal will be that these machines have recently been launched via the AWS console, and they have no agent installed on them.
[0:01:19]	Our goal will be to install the agent on these machines and after the agent has been installed, they will be created automatically, their registration will also happen automatically. So let's move on to the Checkmk web UI and configure the prerequisites.
[0:01:41]	As the first prerequisite, we will configure the user that we will be needing for auto-registration. Why we need this special user? It is because in Checkmk 2.2 we have introduced a very specific role that has been implemented just for the agent registration. It has a very limited permission which basically separates it from Checkmk administration tasks that's why it is recommended to use the agent registration user role.
[0:02:18]	You can simply select the automation secret, I paste in my automation secret you can also generate your own by clicking on this dice and copy the password and use that to download the agents or perform any kind of tasks that you would like to do with this auto-register role. So I have already configured it now we can simply save it.
[0:02:40]	The next prerequisite is to set up a folder wherein we will be storing all the EC2 instances. I name the folder as AWS host, I don't have any IP address for them, so I'll just set it no IP. For the API integration Checkmk agent can stay with the default value and then the most important configuration option is the connection mode which will be “push” in this case. Of course, we will also want to bake the agent, so we will click on the vacation packages, select on bake a generic agent package for this folder, because all the hosts that will be part of this folder will get this package and any new host that will be added to this folder will also get this package automatically.
[0:03:29]	We do not have any SNMP, we do not need any piggyback data so let's simply save the folder. And now we can go and proceed with the agent registration configuration. This is in two steps. One registration configuration has to be done on the Checkmk server side which is under the setup menu and the second one has to be performed for the agent.
[0:03:59]	This will be done via the Agent Bakery and that's why we chose that option bake generic agent package for this folder because we want the second configuration, this one to be applied to all the hosts in that folder.
[0:04:13]	Let's proceed with the agent registration on the server side, we can click on agent registration. Click on add rule. It is divided into different sections, the first section is properties where it asks to define a rule ID, a rule title, and then comes the matching criteria box, here you basically specify which labels are to be accepted when the checking key server receives a request for registration from an agent.
[0:04:47]	These agent labels that you see here are used exclusively for the auto-registration, and they are quite different from the labels otherwise used in Checkmk to identify the hosts and services.
[0:05:02]	By default, two agent labels are defined that can be used out of the box which is the 'hostname-simple' and 'os -family', 'hostname-simple' basically gives you a unique name without the domain name component and the OS family tells you which operating system this agent has, whether it's a Windows or whether it's a Linux server.
[0:05:32]	And last but not the least you can also create your own label by defining a custom label in this value here and choose the appropriate value. You have multiple options inside the value where you can say whether my key access, whether the key doesn't exist, whether the key is equal to a particular value or whether the key is equal to a value where I can also define a regular expression. So for my tutorial today I'll just simply use the key hostname-simple.
[0:06:07]	And in addition to that I will also use a custom label which will be my_autoregister_label, and I have told that the value should just exist. The next section is actions, here we define whether we want to create the hosts or whether we don't want to create the host. So I just simply select create host.
[0:06:32]	The next section is about hostname computation where you basically decide how the names should look like when they are created automatically in Checkmk. And here the $cmk/hostname-simple is the agent label that I got from the above agent labels that I defined. And when my hosts are created they are going to use this hostname template. If I have some upward cases that can also do the case translations, I can convert them to upper/lower, or I can also choose not to convert anything. If I want to do explicit hostname mapping like I receive a host A and I want to change it to host B, then I can do that. If I will have multiple hosts and I want to do the translation of the host names in a single step then I can also use multiple regular expressions.
[0:07:25]	But you can find more information about these settings by clicking on the inline help where you get the option on how to use the regular expressions, how to use the match groups and everything. Let's hide the inline help and then the last but not the least section is host creation.
[0:07:45]	Here you basically define which folder you would like to create the host I want to choose AWS host here, and then you can define different attributes. I will stick to the default attribute which is the monitoring agent set to API integration or else Checkmk agent. And simply save this rule.
[0:08:08	The second part of the configuration is the agent side agent controller registration. Which is under the agent rules. So let's click here, click on add rule, and here it has only one configuration element that is the agent controller, auto-registration, where basically it asks for information on which side this agent controller should register to. So I will use this standalone Checkmk site that I'm using in this tutorial.
[0:08:43]	Or otherwise if you are using distributed monitoring then please try to put the same host name or IP address of that corresponding site. The second setting is the agent receiver port, here you have two options first of all if you want to determine your agent receiver Port you can simply log into your Checkmk command line and type omd config show and 'grep' for the word “receiver”.
[0:09:18]	You will see that the agent receiver port is on, and your agent receiver port for your Checkmk site is 8000. If there are multiple sites then this number will keep on increasing. So let's go back to our Checkmk web UI, either I can configure the agent receiver port here directly or I can also choose to detect it automatically. With the second option I just need the port 8000 to be open between the target host and the monitoring server.
[0:09:53]	But if I choose the detect automatically option then the target host should first go to the Apache on Port 80 or 443 and then determine the agent receiver port. So in that case I have to open two or three ports. The next setting is which site I would like to register with, by default it's the cloud site and then under the authorization I can select the auto-register user that I used for doing this auto-registration. And then, here you have to define the label based on which the filtering should happen or the matching criteria that you define should match.
[0:10:36]	So my auto-register label has a value and in the matching criteria I have chosen that the key should just exist. If you already have some existing connections where the same host is being monitored by multiple sites you can choose to either keep those existing connections or you can simply say 'No'.
[0:10:55]	And on my host are fresh host and I don't have any other connection, so I decided to keep this set to 'No' and now I will simply assign this rule to the AWS host folder and save it. Let's go to the setup agents where we can see this agent configuration for that we need to click on “Bake agents” and now we have the baked agent configuration available.
[0:11:23]	Let's also activate the changes because we have created a user we have created a folder modified the attributes of the folder, and then we have created these two rules. So if we go back again to the agents, this is the package now we need to deploy on all our target hosts that means those ten EC2 instances. As I already told those are based on Ubuntu operating system, I can simply download this package and either use Ansible or a parallel SSH copy to these machines or I can simply use the AWS service in order to do that.
[0:12:04]	So let's move on to the AWS console and use the AWS systems manager service in order to install the agent on these ten EC2 instances. For that I have also prepared the command. This command is based on our rest API endpoint where it's a simple curl that basically puts the package into this file, and it includes some headers and this is the main REST API endpoint where I have the Checkmk site IP, the site name, the whole API endpoint, the AWS host folder is even included, OS type is Linux Deb and agent type is generic and after that I simply install the package here.
[0:12:51]	So we can copy these two commands and directly go to the AWS console, in the search we can search for systems manager, open this in a new tab, click on run command and after that simply search for “shell”, select the AWS run shell script come a little bit below, paste these two commands and simply assign this command to the tag which we assign to all our EC2 instances. So if the key was name and the value of the tag was automated-push-mode, so we can simply copy it from here and paste it in the value.
[0:13:49]	Click on “Add” here and let's go for the down, nothing else to has to be changed here and simply click on "run". As soon as I click run, you can see that the command ID has been generated, and it says that these commands were successfully sent. We can simply refresh this page. And you can already see that the command has been executed against those 10 EC2 instances.
[0:14:20]	Now we can go back to the Checkmk web UI, it takes few minutes for the agents to be created here. So the hosts are already in, we can directly click on these 10 changes to see what has happened. We can already see that using the auto-register user 10 EC2 instances with that label hostname-simple that we chose in the matching criteria they have been added.
[0:14:48]	And let's reload the page again and here we can see that the services which are obtained via the agent have also been saved. And maybe in the next refresh the changes have also been activated. So if we click on the number of hosts here, we can see that they already have those standard agent checks that it gets from the Checkmk agent.
[0:15:18]	It gets the CPU load checks, disk IO checks, all the file system, mount options, systemd checks, and if you have baked the agent with any agent plug-in or any local checks then those checks will also be executed by the agent and shown here in the services view.
[0:15:45]	So that's it for today thanks for watching please don't forget to like and subscribe. I will see you around.