Implementing API monitoring for an efficient infrastructure

What is API monitoring?

What is API testing?

How API monitoring works

The easiest way to perform API monitoring is by calling each endpoint and checking that they reply, return the correct data, and are acceptably well-performing. That’s API endpoint monitoring in a nutshell, the bulk of how an API is monitored. There are more advanced solutions for endpoint monitoring though, like writing synthetic tests that emulate how users poll an API with custom-written robots. That’s part of synthetic monitoring that goes much beyond just monitoring APIs but is often used to have a more automatic and personalized way to do it.

Regardless of the possibilities offered by your API monitoring tool of choice, there are a few important areas that need to be checked when performing API monitoring.

Availability: if one or more of the API endpoints is not even replying, that’s the primary issue that needs to be dealt with. Alerts that fire once any endpoint stops responding are vital to set up in API endpoint monitoring.

Performance: even if all the endpoints return calls correctly, an API is of little use if its performance is not up to standard. HTTP return codes should be verified to be correct, and response times should be monitored to ensure that there is no degradation that may signal a growing problem. Performance differences between development and production environments should be also considered, as they may provide insight on bugs.

Data validation: an API is of no use if the endpoints return different data than expected or show random behaviors. They must be consistent with what users and services depending on the API expect to. Validating the returned data with a set of valid data is easily done with most API monitoring tools, and can be quickly performed with synthetic tests as well.

Those are the basics of API monitoring. Whether you are monitoring a REST API or another type, these three aspects should not be ignored in any attempt at proper API monitoring.

Benefits of API monitoring

API monitoring, and its most common “child” of REST API monitoring, can help you not only with your own APIs but also with integrations. API monitoring tools can give you visibility into third-party and partner APIs, like for example those of cloud services you may be using, or of a payment system for your e-commerce site. This helps in not only noticing issues that are outside your scope but to hold your partners accountable.

Similarly, if you are using an external API and relying on it to provide you a specific set of data, API endpoint monitoring can identify changes in features immediately. If any endpoint of the API changes, replying differently to calls or being completely renamed, you will be informed and can act upon it, avoiding service disruptions. Multi-step user journeys often rely on multiple APIs, chaining requests one after another. If any of these APIs introduces changes or experiences a temporary disruption, business-critical workflows may be interrupted. Monitoring each API endpoint is key to avoid small to large disruptions.

API security is an important topic as well. Often malicious actors take advantage of vulnerabilities and can tamper with API endpoints, or listen to your calls to the API. With API monitoring you can ensure authentication is working and that the security of the APIs you are connecting to is not compromised by monitoring for anomalous behavior. That can mean a security breach happened upstream and you should be very careful on what data you communicated with the API.

Lastly, REST API monitoring is of particular importance for developers to create services that depend on others, and ensuring that the API is efficient and fully functional is key for those services to continue working. If you have developed such an API, making sure that there are no issues with it is key to salvage your reputation as an API provider.

Key API metrics to keep an eye on

Whether you are doing REST API monitoring or generically any type of API monitoring, a few key metrics are common to any API. We briefly touched the subject earlier but it is important to state the metrics in full now.

Availability and uptime should be your first concern. An API needs to be working, and monitoring that it is available at all times is the very first metric to check. Alongside it, monitoring the error rate of each endpoint tells you how often an API does not return the intended results, due to authentication or network errors, or a code bug. This may signify a problem with the API or in the services that the API calls and must be further investigated.

Performance-related metrics are also vital to routinely be checked to be sure that the API is performing adequately. Those are, at a minimum, the CPU and memory usage of the servers receiving the API calls, and the response latency of the API as a whole. Large spikes of usage or latency are better be checked soon to ensure they do not hide growing problems.

Also important to monitor is the overall API consumption. This is the sum of requests per minute or second that the API handles. This is important for capacity, informing you if the API has enough resources to serve all its calls, or if a cyberattack is underway, causing, for instance,  a spike of requests.

Challenges of API monitoring

Monitoring APIs can become quite complex, posing an increasing challenge for IT administrators. Firstly, there are multiple types of APIs. REST API monitoring may be the most popular, but not the only one you may need to consider. GraphQL is a recent addition that allows a more granular control of data requests, which presents quite a few added problems to consider when monitoring it. Architecture-wise, APIs may be monolithic or based on microservices, or serverless computing, each with their own set of worries for IT administrators. A monolithic API may become a choke point for the infrastructure if performance is not closely monitored, but it is also easier to monitor instead of hundreds of microservices that are surely more agile and scalable, but rather complex.

Sometimes APIs are created in such a way to not expose part of their information, mainly in order to protect proprietary data. Part of the infrastructure may not be exposed as an API at all, or not provide a full picture of its performance. API monitoring has then to consider how much you can check from outside, if it is a third-party API you are monitoring.

Rarer, but a possibility nonetheless, are monitoring solutions that are heavy on resources or they are not rightly weighted against the requirements of the software. This may slow down monitoring servers or fill up network traffic that is also shared with your API infrastructure.

Thus, while doing API monitoring a decay of performance may be noticed that is not due to the APIs themselves but only affects them. Much depends here on how your monitoring infrastructure is designed, in truth.

Siloed data is a more frequent challenge of API monitoring. It happens when different types of data are used from different parts of the system. If an API is trying to connect devices using not the same format or type of data, errors may happen and trick the IT administrators into thinking that the API itself has bugs. These errors should be prevented or resolved before they reach the API monitoring phase, ideally in the API testing one.

API monitoring and Synthetic monitoring

API monitoring can be done with plenty of tools, and it is mostly a reactive monitoring endeavor: the endpoints of an API are recursively checked for availability, errors, and correctness. A more active way to do API endpoint monitoring is by using synthetic tests, exploiting the power of synthetic monitoring. With a capable solution like Checkmk Synthetic Monitoring you can simulate requests and actions from around the globe to your API. Actual requests and operations done usually by end users or developers to an API can be simulated with tests, run from your API monitoring of choice.

Synthetic monitoring helps throughout the API development phases: both during API testing and later when API monitoring. Having the possibility to develop your own tests, customized to your intended usage of the API, is invaluable in discovering bugs and honing the experience you offer with your API. Or simply to have more granular control on how you are monitoring a third party your infrastructure depends on.

Checkmk Synthetic Monitoring includes the flexibility of the Robot Foundation’s test framework, a staple in the open source community for writing synthetic tests, to have efficient synthetic monitoring powers for API monitoring and more.

FAQ