Ep. 16: Working with 'Bulk Actions' in Checkmk
| [0:00:00] | Welcome back to the Checkmk channel in this episode, we're taking a look at a topic requested by our viewers namely how to work with bulk actions. |
| [0:00:18] | With bulk actions, you can perform certain actions on a large number of hosts so now let's take a look on how that works. We are back in our test site which we have been using throughout the series. |
| [0:00:31] | For now let's go to setup, hosts and then into the servers folder. So here we have six hosts and you see a checkbox at the beginning of each line, and you can use that to select one or multiple hosts and you can use the checkbox at the top to select or deselect all of them. |
| [0:00:55] | For now, let's check the four database servers and then go to the Hosts menu. Now here under unselected host, you see all the bulk actions that we can perform on these hosts so you can move them to another folder, you can delete them, edit the attributes. |
| [0:01:15] | For now, let's just edit the attribute. Here you see an overview of all the attributes that we can edit in bulk. And it's important to know that you will only edit a certain attribute when you check the box of that line. |
| [0:01:38] | And if a box is already checked like here at IP that means that all the selected hosts are sharing the same value for that attribute. You can also see which are using the default value for a certain attribute or you can see if the value differs for a certain attribute, so that means that the host we have selected do not share the same value for this attribute. But for now, let's edit an attribute. |
| [0:02:09] | Let's edit networking segment. So we check the box and then we choose a value that we want to give. So let's choose the WAN. And now if we save that, if we press save then the WAN value for the networking segment will be applied to all or selected hosts. |
| [0:02:40] | In Checkmk, there is a difference between explicitly setting the value of an attribute or leaving it empty so it gets the default value or it inherits the value of a folder, when you edit a single host and remove the value of an attribute and it will fall back to the default value or the inherited value. |
| [0:03:02] | When doing bulk edits this is not possible. You can only set values but you cannot remove them. So if I bulk edit these hosts again and I would remove this checkbox that doesn't mean that I will unset the value of this attribute it just means that I don't want to edit it. |
| [0:03:29] | So if I were to save it everything will remain the same. The reverse is true for the bulk removal of explicit attributes action. So let's select the same four hosts and go to the host menu and click on remove explicit attribute settings. |
| [0:03:55] | And you'll see that the list is a lot shorter here that's because only attributes show up that have a value explicitly set. So you see that the criticality has been set on two of the four hosts while the IPv4 address has been set on all of the selected hosts and of course also the networking segment that we have just set. |
| [0:04:17] | So now let's remove the networking segment for these four hosts and press save. And if I now look into a single host here you'll see, that the networking segment is again the default value 'Local network'. |
| [0:04:38] | So this is a powerful way to reset our inheritance and make sure that it worked properly again. Let's say that I want all hosts in a folder to start inheriting an attribute directly from the folder. If I were to have hundreds of hosts in that folder that have the value for that attribute explicitly set and it would be a painful process to reset or remove that attribute on each of them. |
| [0:05:06] | But this way we can easily make sure that all hosts will inherit the value that we want. |
| [0:05:14] | So another important action we can perform in bulk is the adding of missing services to a host or remove them if they have disappeared and you already know this feature from earlier episodes and you can easily go there by clicking on this yellow box. |
| [0:05:36] | And then you will see an overview of all the services that are currently being monitored on this host. |
| [0:05:44] | Since we are talking about bulk actions in this episode, I want to show you how you can do this for multiple hosts at once. Like with other bulk actions you first have to select the host you want to perform the action on. So once again let's select our database hosts and then go to the host menu and here you find this item discover services. In this dialogue, we can exactly configure what we want to do. |
| [0:06:16] | The currently selected mode is to add unmonitored services and new host label that means it will only add these two things and it will not remove any vanished services. Automatically removing vanished services can potentially be risky. |
| [0:06:37] | For example, let's say that you have a problem with one of your file systems and it vanishes. |
| [0:06:42] | In that case, you don't want it to be automatically removed from your monitoring because that way you wouldn't know that there is a problem, instead you want to be notified about the issue. |
| [0:06:53] | If I would have configured remove vanished services then it would have been simply kicked from the monitoring altogether and that's not without risk. So here you can choose if you only want new services to be added or the inverse if you only want to remove vanished services. |
| [0:07:13] | And with this third option, you can do both at the same time, the fourth option will let you do a service discovery from scratch. So basically it will remove everything and then re-discover everything and then the fourth option will only discover new host labels. Then there are some additional options for example this one only include hosts that failed on previous discovery. |
| [0:07:42] | When you do a bulk or a CSV import of hundreds of hosts and do a service discovery at the same time then it could be that a few of these hosts are not reachable at a time. So to prevent that you have to find and select all these hosts manually, you can simply select this checkbox and it will only perform the service discovery on the host that previously failed. |
| [0:08:06] | And then there is another option to only run the service discovery on host where the discovery check has failed. |
| [0:08:13] | This is a check that warns you when there are undiscovered services on your host that are currently not being monitored. When you are ready simply click start and this will trigger a background job and this can take a while depending on the number of hosts that you are rediscovering. |
| [0:08:32] | When it's ready the state like here we'll go to finished and down here you'll see a summary of everything that has been discovered. So it discovered a total of four hosts, a total of eight host labels has been discovered and the total of 116 services have been discovered. |
| []0:08:56 | So now let's go back to our host configuration. So there is another way you can perform a service discovery in bulk and that is for the entire folder. So in the host menu, there is this section in this folder and it also has an item discover services when you click it you'll see a similar screen that you saw before, the only change is that there is a new checkbox include all subfolders and if you check this it will perform the host discovery or the service discovery recursively, so if there were any subfolders here then it would also do the service discovery for all those subfolders. |
| [0:09:44] | Another bulk action that you can perform on hosts is Renaming. |
| [0:09:53] | While this might not sound very spectacular, anyone who has done this before knows that in Checkmk this is actually quite a complex undertaking. And that's because the hostname is also the key for historical data and the logs and the databases. |
| [0:10:10] | So when you rename a host, Checkmk actually has to go through all these logs and databases and update the hostname. In order that you don't lose any historical data after renaming. |
| [0:10:23] | If this were not the case you could have simply deleted or removed the host and then added again with the new hostname. But often you want to keep data regarding availability notifications and so on. |
| [0:10:36] | And that's why the renaming action works in such a way that it will basically rename the entire past of a host to the new hostname in order that you have complete historical data. |
| [0:10:49] | So it's more complex than it sounds and it also requires your Checkmk site to be stopped in order that the host hostname can be updated in a database consistently. |
| [0:11:02] | So if you were to rename a large number of hosts and this could be quite a time-consuming undertaking because for every host the Checkmk site has to be stopped and started again and that's why there is a bulk action to do this. |
| [0:11:15] | You can find the bulk renaming in the host menu here, and you'll see that the include all subfolders is already pre-selected. And this would mean that it would perform the renaming recursively for all hosts and all subfolders. |
| [0:11:43] | Important is this hostname matching and this is where you configure which hosts need to be renamed. If you leave this empty then it will apply to all hosts. |
| [0:11:58] | This field will accept a regular expression and if you want to read more on the topic then you can find an article in our documentation. |
| [0:12:07] | Important to know is that this will match from the start of the hostname. So I can for example say DB_server and now it will match on all hosts starting with DB_server. |
| [0:12:25] | Now we need to configure the renaming operation we cannot simply give all hosts the same name because hostnames are unique. So now let's configure one by clicking 'Add renaming'. |
| [0:12:42] | In this first drop-down, you'll see some pre-configured operations you can choose from. For example, case translation and with this one, you can convert all host names to either upper or lower case and this can be quite useful after an import if the case of all the hostnames are not consistent. |
| [0:13:04] | So let's convert all our hostnames to uppercase now. You can simply click 'Bulk rename', and before it will actually apply the rename you'll see an end result that it would be. If I click 'Cancel' I can go back and change my configuration. |
| [0:13:30] | We can for example add a second rule. So add renaming and this time we'll add a suffix and we'll add a suffix of '_test'. |
| [0:13:45] | And if I then once again go to the example so both rename you'll see that every hostname will have a suffix '_test'. But it's not an upper case yet and that's because we added the suffix after the case transformation. |
| [0:13:59] | To change that we can simply drag the rule above the case translation. So if we go to the example now, you'll see that everything is in caps. |
| [0:14:18] | Then there is a more extensive operation you can perform, more extensive than the case translation and the suffix and that one is the 'Regular expression substitution'. So let's first remove the other two. |
| [0:14:39] | So this one is a bit more complex, but I will show you a good example of what you can do with it. Instead of naming our hosts 'DB_server' and I followed by a number I want to name them 'db - number - server', all lowercase. So to do that I would have to extract a portion of the hostname and reuse that in the new hostname. And to do this I would need to create a subgroup in regular expression and in my case I want to extract the part that's always at the end after DB_server. |
| [0:15:25] | So how would I write that. So I want to capture everything after 'DB_server', and you could write that like this. So every host that starts with 'DB' or contains 'DB_server' and then you have this group and what is in the parentheses is what we want to capture and " .* ", basically means anything or everything. And this will work fine because we know that we have only four hosts and all of them have a number after 'DB_server'. |
| [0:16:10] | But if we would for example have a host that was called only 'DB_server' without a number then this can potentially go wrong because then there is nothing to capture. And the renaming of the new host will probably not work how we want it to work. |
| [0:16:31] | So we could write this in a safer way we could for example write " ^DB_server(\d+)$ " parentheses to start the group and then we want to capture "\d+" and then a "$" sign to say that's the end of the string. So what is written here is the string starts with 'DB_server' and then we want to capture this group '\d' means any digit and "+" means one or more. So it will capture any hostname starting with DB_server and followed by one or more digits and then the end of the string. |
| [0:17:26] | And now we need to configure with what we want to replace it. And like we said we wanted to replace it with "db-\1-server". The "\1" will be replaced with whatever we captured here in our original hostname. |
| [0:17:44] | So let's take a look on how this will look. And now you see that all our hostnames will be renamed to 'db-number-server'. |
| [0:18:06] | And if you are happy with the end result you can simply click confirm. And once I do that the renaming procedure will start and this is done in a background job. |
| [0:18:22] | And the whole procedure may take a bit of time because like I said before Checkmk will need to be stopped to consistently rename all the host names and all the logs and databases. And that's why you should avoid doing this in a productive system when the system is required to be functioning. |
| [0:18:42] | The whole process can take somewhere from a few seconds to a few hours depending on how many hosts you want to rename and how old your monitoring system is. |
| [0:18:54] | And afterwards, your monitoring will automatically be started again. So let's press Confirm. |
| [0:19:01] | Like I said (it's being) now a background job is being started and while it's running you'll find the current status or what is being done here in the progress info. |
| [0:19:13] | And once it's done the state here will go from running to finished. Like now. Then here in the result, you'll see where exactly all the renaming has taken place. |
| [0:19:28] | And that's how easy it is to rename or bulk rename a number of hosts. And that's everything I wanted to show you regarding bulk actions. |
| [0:19:40] | I hope this video was helpful to you. If so, please subscribe to our channel and like the video. I hope to see you in the next episode. |
Want to know more about Checkmk? Join us for our Introduction to Checkmk Webinar
