What is AWS database monitoring?

Monitoring AWS databases is generally done in a few different ways. The first step is usually CloudWatch, which is already integrated in the AWS cloud and therefore a good start when setting up a monitoring infrastructure. It offers a set of dashboards and can monitor not only all the AWS databases, but also some on-premises ones, as well as plenty of other services and resources in the AWS cloud.

CloudWatch is not the only option, though. AWS X-Ray can monitor requests to MySQL and PostgreSQL-based self-hosted or Amazon databases, like Amazon RDS and Aurora. AWS SQL databases can then view all the requests made to it, and generate an application-centric view of the performance of your databases, providing an overview of how the AWS databases are being used for the administrators.

For a more open-ended approach, AWS exposes many metrics through their REST APIs. These can be exploited with custom-made scripts or, more commonly in medium/large environments, with third-party AWS database monitoring tools like Checkmk. These tools simplify gathering the right metrics and creating custom dashboards, alerts, reports and more.

Last but definitely not least, CloudWatch or third-party monitoring agents can also be used for more granular monitoring of databases on AWS and other resources. Many monitoring tools use both agents and the REST APIs to complete what can be monitored. A comprehensive AWS database monitoring system will benefit from having the highest number of different metrics and data collected in a unified view as possible.

AWS database options are aplenty, and not all monitoring efforts are equally worthwhile. Next, we will take a look at how the different Amazon databases are to be monitored, since there are usually slight differences.

Into the category of AWS SQL databases falls Amazon Aurora and Amazon RDS. Aurora is a MySQL and PostgreSQL-compatible relational database service, while RDS is a fully-managed service to set up your relational database of choice. Monitoring AWS SQL databases is very similar, so the collected metrics do not differ much. But still, there are a few differences you should be aware of.

Keeping an eye on the usual basic metrics like CPU utilization, memory usage, disk I/O, database connections, query execution time is obvious and should be included in any AWS database monitoring set up. In case of AWS RDS monitoring, if your databases are spread over multiple AZ (Availability Zones), replication logs are also important to consult.

Additionally, Aurora is an AWS serverless database and exposes a few specific metrics over RDS. Namely, cluster health metrics like the reading and writing number of operations. These can signal how well the underlying clusters of your Aurora databases are working. For instance, replication lag tells you how well replication is working at cluster-level. The single volume metrics like active transactions, deadlocks, queue depth, or login failure counts should be taken into consideration as well.

There are a few AWS NoSQL databases among the numerous AWS database options. You will probably face ElastiCache or DynamoDB the most. The former has a few metrics that are obviously related to cache, like hit ratio and evictions, and that are important to track with CloudWatch or your AWS database monitoring tool of choice.

DynamoDB is a managed NoSQL database service similar to Amazon Aurora, and when monitoring it you should focus on similar or equivalent metrics. Read and write capacity utilization, throttled requests, consumed provisioned throughput, latency, overall storage usage, and more.

In both ElastiCache and DynamoDB, the classic metrics related to CPU, memory, network, disk utilization and performance should be regularly checked as well. Again, either with CloudWatch for a simpler look or with a third-party AWS database monitoring tool like Checkmk.

AWS Graph databases are part of the AWS NoSQL databases. An example is Amazon Neptune, which is fully-managed by AWS and cloud-native. These Amazon databases work differently than other NoSQL databases, but their monitoring metrics do not diverge much from what we have seen with ElastiCache and DynamoDB. Monitoring key utilization metrics like CPU usage, memory utilization, disk I/O and space, query latency, database connections and so on is as much a necessity with an AWS graph database as with any other NoSQL databases.

In case of a serverless Amazon Neptune though, it is important to keep track of NCU (Neptune capacity unit) related metrics, like serverless database capacity or NCU utilization. The NCU is a basic unit for serverless Neptune databases, consisting of a block of 2 GiB of memory, an associated virtual processor capacity (vCPU), and necessary network resources. They give information about the health of the respective Neptune instance.

Regardless of the AWS database options you choose, there is a set of best practices to follow for an efficient monitoring system. First, consider the factors that most matter to your organization. Whether it is performance, availability, scalability, or security. This way, you will probably pay more attention to a set of metrics that is more relevant for your environment.

CloudWatch is not to be underestimated. While limited in configurability and customizability, CloudWatch can give you a helpful overview in only a few seconds, making it easy to understand if something is amiss or wrong. A more advanced AWS database monitoring solution, like Checkmk, can be used to investigate further and troubleshoot when necessary. It is not advisable to limit yourself to CloudWatch only, as it hardly fits every company's needs.

If you think about what we said about the specific metrics of every type of Amazon database, you quickly realize that most of them have a lot of common metrics that should not be ignored. However, they also have a few individual metrics that inform cloud administrators about their specific functional areas. Whether you are using an AWS SQL database or an AWS NoSQL database, knowing what their typical metrics are is important to set up an efficient monitoring solution.

Set up alarms and notifications based on your thresholds. CloudWatch can notify you through Amazon SNS, a push notification service that can notify on many devices and systems. Even though it comes at a cost, it can be of great help for emergency notifications. Otherwise, a separate AWS database monitoring tool will provide you with many options to inform administrators of overstepped thresholds and bring their attention to the right database. Checkmk allows you great control on when and how exactly you should be notified, and is an invaluable tool in avoiding notification fatigue.

If you have database replication or multi-Availability Zone (AZ) deployments for high availability or disaster recovery, monitoring replication lag, replication status, and failover events ensures data consistency and availability.