Success with Bigtable Observability

The capacity to comprehend the state of a database, including its performance, health, and security, is referred to as observability. Observability is essential in any database, but it is extremely important when working at scale with a database service like Cloud Bigtable.

Because Bigtable is a managed service, users have access to the suite of monitoring tools and metrics that we provide, and we are constantly listening to user feedback to discover what challenges you’re attempting to address and what tooling we might build to better the development and debugging experience.

In the following article, we’ll look at some new Bigtable tools and metrics that enhance popular tools like Key Visualizer and Cloud Monitoring. We’ll explore what problems they can help diagnose and how to remedy them so that Bigtable performs optimally.

Using query statistics, you can identify wasteful queries.

Bigtable’s low latency is a crucial feature; at the 99th percentile, you may conduct single-digit millisecond searches. However, there will always be some high-latency queries in any large-scale system. These are classified into two types: queries that are intrinsically slow due to the amount of data handled and queries that are slow due to external factors.

You can’t optimize queries that are intrinsically slow by adding more nodes or processing power. A common Bigtable example is counting all the rows in a table, a query that requires scanning the full table. If there is now considerable traffic on the table, a query that is otherwise performant may become delayed due to the external condition.

Bigtable query analytics have been added to assist you in diagnosing queries that are inherently sluggish. You can learn more about the query results by running them through the cbt CLI or the Go Client library and enabling query metrics.

The ratio of data seen to data returned is a critical piece of information contained in query analytics. If your query sees significantly more data than it returns, it is a poor performer since it read rows it didn’t require.

Among the solutions for these slower searches are:

• Narrow the scope of your query – The best strategy to limit the number of rows shown and speed up your queries is to use a prefix scan or set the start and end keys for a query.

The ratio of data seen to data returned is a critical piece of information contained in query analytics. If your query sees significantly more data than it returns, it is a poor performer since it read rows it didn’t require.

Among the solutions for these slower searches are:

• Narrow the scope of your query – The best strategy to limit the number of rows shown and speed up your queries is to use a prefix scan or set the start and end keys for a query.

• Making changes to your schema – If you have a sophisticated query that is used infrequently, you may be able to accept slower performance; but, if your common queries are doing sluggish table scans, you may wish to rewrite your schema to be more optimum.

• Denormalize data – Putting the same data in numerous rows or tables with different row key designs to allow multiple access patterns is one method to alter your schema. This may add complexity to your program, but depending on your use case, it may be simple to include.

• Archive your frequently used queries – Caching slow queries is a last resort for dealing with them. To reduce response time, you might retain the results of your queries in a cache service for 30 minutes and send queries there instead of Bigtable. This will be most useful when you have slow queries whose results are required several times.

High-granularity metrics can help you with spiky workloads.

Bigtable applications function best when traffic is consistent or varies slowly. However, given Bigtable’s vast scale, abnormalities may emerge and impact performance. Spikey traffic is an example of an irregularity since it is too sudden to rescale for and might be difficult to diagnose.

We’ve included new high-granularity metrics for CPU utilization and request count that provide the most value in a five-second period each minute. This visualization will show you how this reveals spikes in those parameters.

CPU

This fine-grained max CPU usage statistic can help you identify spiky workloads that were previously averaged out. If your program was experiencing unexpected delay but the CPU appeared to be functioning normally, there could have been hot tablets caused by processing spikes. You may now more quickly identify hot activity and then use tools like the hot tablet tool and the key visualizer to precisely analyze the problem.

the number of requests

Another indicator that can be used to explain latency fluctuations is request count. The high granularity statistic allows you to spot peaks in your request count and more accurately diagnose latency changes.

If your bursty traffic is uncommon, you can implement some rate limiting – perhaps a request is being spammed and should be limited regardless of latency implications. If this happens frequently, you may need to change your row key format to better distribute the load across the database. Sometimes your workload will be spiky and you won’t be able to adjust it, but this statistic will provide you confidence that the services you’re expecting are functioning well.

Table statistics provide access to metadata.

Table stats provide a summary of data from a Bigtable table. They can be beneficial when debugging performance issues or determining the source of storage charges.

Table stats are able to offer rapid insights into some table information for a high level perspective of how data is stored, which is useful if your firm has numerous developers working within the same instance. Among the new metrics are logical data in bytes, the average number of columns per row, and row count.

The number of columns per row can indicate whether the table is utilized as a key value store or a wide-table store. If you have time-series data, row count can be a useful data point. This indicator isn’t real-time and is normally approximately a week behind, so you won’t see it in your newer tables.