What is synthetic monitoring?
Synthetic monitoring is a practice in monitoring that simulates user interactions with a system. This can be in the form of an app or website, or any type of software that is exposed to end users. The data generated from the simulated interactions is collected and analyzed to evaluate how the users behave and how the system responds to users’ input and actions.
A common use of synthetic website monitoring is to understand and optimize a website, but synthetic monitoring is a much wider practice, encompassing many areas. It is used to emulate desktop/OS operations, for image pattern recognition, or to perform HTTP checks over a network. This type of monitoring is done through the use of synthetic tests: a series of scripts and configuration files that tell the monitoring tool how to perform synthetic testing.
These are of great usefulness for IT administrators in testing all possible use cases that a piece of software, or more generally a system, is subject to, and seeing if there is room for optimization, reasons for changes, or actions that need to be taken to fix issues.
Synthetic monitoring is also referred to as proactive monitoring as it does not wait for events to happen but creates scenarios to verify how the system behaves. It can validate systems’ performance and availability in an active manner. In contrast, classic monitoring efforts that analyze only real data are referred to as passive monitoring.
How does synthetic monitoring work?
Broadly speaking, synthetic tests are created by QA (quality assurance) engineers to test a website, application, or system. Depending on the adopted framework and synthetic monitoring tool, these scripts take many forms and syntaxes. Synthetic monitoring solutions execute the tests and report on their results to the monitoring tool. The data is analyzed to assess the system’s behavior. If a need is identified, the system is improved or modified accordingly. New synthetic tests are written to further verify the changes and possibly continue improving the quality of the system.
Often synthetic testing comes in the form of an automated task that would typically be performed by engineers. It is normally also possible to run synthetic tests manually, even if it is rarer. Most occurrences of manual intervention happen when tests are updated to reflect changes in the monitored systems’ behavior and UI.
Synthetic monitoring has synthetic testing at its core, as it all revolves around correctly writing and executing the tests for your own use cases. In most synthetic monitoring solutions, there are synthetic tests possible for a myriad of scenarios, with a wide coverage of any company’s needs, within the limits of what the vendor has built into the software. Whether you need to test an end-user clickstream through your e-commerce platform, or monitor shopping transactions or even online gaming, synthetic tests are possible.
What are the types of synthetic monitoring?
By now, it is easy to see that there are multiple types of synthetic monitoring. They differentiate themselves in the use case they are testing for or the type of system they need to test. Let’s see what types exist and for what they are used.
Uptime monitoring
Also known as availability monitoring, it helps to track the availability and responsiveness of a web application. Synthetic tests may involve simply using a ping or a GET request to specific resources to test their accessibility, or creating multiple concurrent requests to monitor the responsiveness and performance of your web application.
Synthetic website monitoring
As it is easy to guess from the name itself, synthetic website monitoring tests the availability and navigability of a website. Not only may simple uptime monitoring tests be performed but also more complex use cases of a hypothetical user navigating through the website are possible with synthetic website monitoring. Response times, page load time, and any website performance metrics can be checked through synthetic tests of this type.
Synthetic transaction monitoring
Synthetic transaction monitoring goes a bit beyond synthetic website monitoring, and it is more specialized. It is used to test both websites and web apps in order to ensure bug-free functionalities, validate form submissions, check that business-critical operations like shopping cart checkout are free from issues, and more. Tests like these are often run at regular intervals from servers around the world to ensure the complete availability of all the critical functions of your web-based applications and websites.
Real user monitoring vs synthetic monitoring: what is the difference?
A word must be said on real user monitoring vs. synthetic monitoring. The terms are often confused. Real user monitoring is based on the actual data and interactions performed by end users on a system, regardless of what that is. It is therefore real-world data, unlike synthetic monitoring, which is computer-generated, a simulation of the possible behavior of real users.
Thus, synthetic vs real user monitoring do not differ in their final objective, which is still monitoring a system to ensure its optimal performance and operability, but they do use completely different datasets. It is not necessary to choose between real user monitoring vs synthetic monitoring, though. Both can be, and are, implemented at the same time, even by the same tools. Sometimes synthetic monitoring is set up only on systems that are in the testing phase, not yet in production; sometimes it is run in parallel with real user monitoring on systems that are live already. The choice falls on engineers, and your specific reasons for monitoring.
What are the benefits of synthetic monitoring?
There are clear benefits to implementing synthetic monitoring. One is validating a system’s performance. In what is sometimes called synthetic performance monitoring, tests are created to check a system’s performance under stress, and judge whether modifications are needed before it goes live. Or, simply to monitor the overall performance so the modifications do not negatively impact end users or internal workloads. This is invaluable in gathering insights that may be harder to obtain from real users, as some edge cases may be too rare to provide enough meaningful data.
With synthetic transaction monitoring, it is possible to establish a baseline benchmark. By observing how the application behaves under a series of synthetic tests, it is possible to set up a benchmark of how it should perform under normal conditions. Once the application is in production, and throughout its use, it is then useful to compare it to its performance benchmark and be made immediately aware of any decaying performance.
Not just synthetic performance monitoring benefits exist. An important asset that synthetic monitoring can give you is the ability to monitor the accessibility of a system. By simulating the behavior of users who require specific accessibility features, insight can be gleaned on how accessible a UI or system is in general.
Similarly, with synthetic monitoring, it is possible to simulate the experience that a user far away from your datacenter will have with a system. Developers and QA engineers can determine types of requests a given user is likely to make, or they themselves can initiate requests under certain conditions, such as routing users in a way that reflects how they’d be routed to a specific geographical location.
Synthetic monitoring main challenges
Synthetic monitoring is not without its challenges. For instance, many of the difficulties in setting up a synthetic monitoring system lie in preparing the right synthetic tests. For instance, in many infrastructures nowadays, a mix of cloud and on-premises assets are used. The larger the distributed environment is, the more challenging it is for DevOps or engineering teams to simulate every possible scenario.
Errors can be missed if a certain location or situation is not predicted and tested accordingly. Without previously knowing with accuracy the path users take on your web app, for example, synthetic testing becomes a guessing game. Some work should be done before starting to implement synthetic monitoring software.
Synthetic monitoring requires a specific skill set from engineers, and not all are knowledgeable about it. Different testing languages and frameworks use slightly different ways to write tests and establish a working synthetic monitoring system. Without already having specialized knowledge in your teams, monitoring synthetically may become difficult to set up and time-consuming.
Following even minor UI changes in client-facing applications can cause testing scripts to fail, triggering alerts and notifications and the inevitable bouts of worry for system administrators. It is important to have a good alerting/notification setup that avoids needless triggering, causing “alert fatigue.” While resolving most failures is often trivial, synthetic monitoring still has a long way to go before claiming to be a “set it once and forget it” solution.
Conclusion
Despite the challenges, it is clear that synthetic monitoring is an incredibly useful tool in the arsenal of companies that want to optimize the reliability of their infrastructure. Monitoring tools have been moving towards integrating it or pairing up with specific synthetic monitoring frameworks. Checkmk is no exception.
Checkmk has included the power of the Robot Framework into the already automation-heavy features of the Cloud and Enterprise editions. Following up on the results of synthetic tests from Checkmk is now as easy as setting up a normal monitoring agent, with integrated views into the general Checkmk dashboards. A step further towards making synthetic monitoring nearly effortless.