Werk #12907: check_mk_agent: Fix issues with systemd

When deploying the checkmk agent on systemd based systems, users often encountered problems when hitting the limit of 3 connections per source (the monitoring server in this case).

The problem of hitting the connection limit was caused by systemd not terminating the connection when the agent finished. Instead systemd waited for all child processes to finish (in particular the started cache refreshes). As a result, the connection was kept alive while the plugins executed that had been deliberately configured to run in the background.

This is solved by adding a second service to checkmks systemd units:

check-mk-agent@.service: The preexisting service. It is triggered by a remote connection of the monitoring server to the socket. This service no longer triggers the asynchronous jobs. It only outputs the available results (which may be waiting in cache files).

check-mk-agent-async.service: The new service. It consists of an agent process kept running in the background. It checks every 60 seconds wether new asynchronous jobs need to be started, to update the caches for cached plugins (and run the real time checks, see below). The created monitoring data is stored on disk.

The check-mk-agent@.service triggered by the socket will then output the prepared cached information.

Known limitations:

Unfortunately this is not the only problem when using systemd: The real-time checks are still not working. This issue will be fixed separately.

To the list of all Werks