Werk #17290: Synthetic Monitoring: Re-work Keyword Monitoring
| Component | Checks & agents | ||
| Title | Synthetic Monitoring: Re-work Keyword Monitoring | ||
| Date | Sep 17, 2024 | ||
| Level | Prominent Change | ||
| Class | New Feature | ||
| Compatibility | Incompatible - Manual interaction might be required | ||
| Checkmk versions & editions |
|
Version 2.3 offered the option to monitor the runtime of Robot Framework (RF) keywords. This was possible via the option Maximum Keyword runtime in the ruleset Robotmk test. However, this option was hard to use because the resulting metric names were not robust to modifications of the test definition. Moving, adding or deleting keywords resulted in shifting metric names, which rendered the recorded metric histories hard to interpret.
To address this issue, the monitoring of keywords (formerly as an accessory to test monitoring) has been expanded into an independent Key Performance Indicator (KPI) monitoring in version 2.4. The option Maximum Keyword runtime has been removed from the Robotmk test ruleset. Instead, Checkmk now offers two distinct mechanisms for monitoring KPIs:
-
Pattern-based approach: There is a new service discovery ruleset called Robotmk KPI discovery. This ruleset offers the option to discover keywords based on regular expressions. However, contrary to before, the matched RF keywords must be unique per RF test. Furthermore, each matched keyword now results in a separate service.
-
Marker-based approach: To overcome the limitation of unique keywords per RF test, version 2.4 additionally introduces the option to mark RF keywords directly in the test definitions as KPIs to be monitored by Checkmk. To this end, we introduce a dedicated RF keyword library. For now, this library exposes one keyword only, namely
Monitor Subsequent Keyword Runtime. This keyword marks the subsequent keyword as a KPI that will be monitored with a separate service by Checkmk. Users can optionally specify an identifier for this KPI that will be used in the corresponding service name. This enables monitoring multiple KPIs that are based on the same underlying RF keyword.
In both cases, thresholds for the KPI runtime are configurable via the new ruleset Robotmk KPI monitoring.
These changes are marked as incompatible because no automatic migration from 2.3 to 2.4 is possible. When updating to 2.4, the patterns previously configured below the Maximum Keyword runtime option will be dropped.