Google Logs Explorer Modify Cloud SQL for PostgreSQL Metrics

Google Cloud Logs explorer

For PostgreSQL-compatible workloads, Google Cloud offers multiple options: the Spanner, AlloyDB for PostgreSQL, and Cloud SQL for PostgreSQL. Monitoring several aspects of your databases’ performance, such as locking conditions and the frequency of specific errors appearing in the database log, is necessary to ensure that they are operating at peak efficiency in a production setting. Additionally, keeping an eye on your database’s error log for problems that might be harming your users, like deadlocks, job failures, and shifts in database statistic trends, is a wonderful method to take better care of your database.

You can view and examine logs from your Cloud SQL for PostgreSQL or AlloyDB instances using Google Logs Explorer, a Google Cloud product. Additionally, you can construct custom metrics with Logs Explorer. Basically, you can use Google Logs Explorer to monitor and publish a new statistic for any entries that are published into postgres.log. You can designate the precise error or condition from that log that you would like to monitor or receive alerts for.

Additionally, Google cloud provide instructions on how to utilise Log Explorer to generate new custom metrics that will improve the monitoring of your PostgreSQL database that is hosted on Cloud SQL or AlloyDB.

Google Logs Explorer

Use the Logs Explorer to view logs

An overview of the Google Cloud console’s Google Logs Explorer is given in this paper. Log entries saved in log buckets can be retrieved, viewed, and analysed using this tool. You can troubleshoot issues by viewing and examining individual log entries as well as a series of log entries. The Google Cloud CLI and the Logging API can also be used to read log data.

Upgrading the log bucket to utilise Log Analytics and then running queries from the Log Analytics tab of the Google Cloud dashboard will allow you to do aggregate operations on your log entries, such as counting the number of log entries that have a particular pattern. For viewing logs in log buckets that have been updated to use Log Analytics, you can still use the Google Logs Explorer.

Before starting

Request that your administrator provide you access to the following IAM roles so that you can examine log entries:

• Use Logs Viewer (roles/logging.viewer) on your project to view the log entries in the _Required bucket and those in the _Default view on the _Default bucket.
• Use the Private Logs Viewer on your project (roles/logging.privateLogViewer) to see the log entries in the _Required and _Default buckets.
• Logs View Accessor (roles/logging.viewAccessor) on the project containing the user-defined bucket can be used to view log entries saved in a log view on a log bucket. See Control access to a log view for details on how to gain access to a particular log view.
• Use your project’s Logs Field Accessor (roles/logging.fieldAccessor) to examine restricted LogEntry fields in a bucket.

It’s possible that other established roles or bespoke roles can grant you the necessary permissions.

Start now

Take the following actions to start using the Google Logs Explorer:

1. Navigate to the Google Logs Explorer page in the Google Cloud console:

Go to Logs Explorer

2. If you found this page using the search box, choose the result with the subtitle “Logging.”

Choose the relevant Google Cloud project, folder, or group.

The AWS connector project, which connects your AWS account to Google Cloud services, is where you may find your log entries if you use Amazon Web Services Elastic Compute Cloud (AWS EC2).

Logs Explorer user interface

You can see log entries, parse and analyse them, and set query parameters using the Google Logs Explorer interface.

The following sections of the Google Logs Explorer are described in depth on this page:

1. Action toolbar
2. Query pane
3. Results toolbar
4. Log fields pane
5. Histogram
6. Query results pane

Action toolbar

These are some of the things you can accomplish with the Action toolbar features:

• Refine scope: Select which log items to get for display from log buckets. You can filter the fetched log items by writing queries. Refer to Refine scope for additional details.
• Discover: View the connections to the pertinent subjects and documentation.
• Provide a link: To make sharing a query easier, create a shortened URL for the query and copy it to your clipboard. The copied URL has the corresponding absolute time range, such as 7:49:37 PM to 8:49:37 PM, which is represented by the time range of your query.

Query pane

Utilise the following features of the Query pane:

• Query tab: Utilise the following tools to create and improve queries:
  • Time-range selector: Choose the time period that you wish to view in the log entries.
  • Find log entries that match your search terms or phrases using the search-text field.
  • Create queries based on Resource, Log name, and Severity using the filter options.
  • Query-editor field: Use the Logging query language to create complex queries.
• View your most recent queries by clicking on the “Recent” tab. Refer to Utilise recent inquiries for additional details.
• Saved tab: Access queries you have saved and queries shared with you by other Google Cloud project participants.
• View suggested queries on the Suggested tab, which are determined by the resources in your Google Cloud project.
• Library tab: View and execute queries offered by Google according to your use cases.
• Save: Store queries on the Saved tab so they can be viewed and executed later.
• Clear query: In the Query pane, clear and reset the choices you made while creating a query.
• View log entries as they are stored in log buckets by streaming logs.
• Run query: After creating your queries in the Query pane, execute them.

Turn on Show query if you are unable to see the query-editor field. Any search phrases you provide in the search-text box are assessed as part of your query expression and also show up in the query-editor field.

Click Run query once you’ve reviewed your query. Logs that correspond with your search are displayed in the Query Results window. The query expression also affects the Histogram and Log fields panels.

Results Toolbar

You can access the following options from the Results toolbar:

• Logs field: From your Logs Explorer page layout, show or hide the Logs field pane.
• Histogram: Adjust the Google Logs Explorer page layout to display or conceal the Histogram pane.
• Examine the outcomes: Use Log Analytics to aggregate and chart the results of your query. This button appears only if there is a log bucket in the scope you have chosen that has been updated to use Log Analytics.
• Measurement creation: Establish a log-based metric.
• Make an alert Establish a policy for log-based alerts.
• Go to the present: To add the current time to the query results, force a refresh. The query is executed with a default time range of one hour if the time-range selection uses a custom range and an end time is specified. If not, it executes the query and refreshes with the current duration or start date. Keep in mind that when you utilise this feature, the query expression remains unchanged.
• Additional actions To construct a sink or administer log-based alerting policies, use these additional options.

The log fields pane

A quick and effective approach to hone a query is through the Log fields pane, which also provides a high-level summary of log data. The pane displays log entries that have been divided into various dimensions based on the fields included within these entries.

The Log fields pane is filled in with the query’s results whenever you run it in the query-editor field. The numbers of log entries for every supported dimension are shown in this pane. You can choose just one option for each dimension.

Histogram

The distribution of logs across time can be seen with the Histogram pane. When you execute a query, the histogram is updated, which facilitates the process of identifying patterns in your log data and resolving issues.

Features of Histograms

Histogram bars: A time range is shown by each bar in the histogram. A three-color breakdown of the log-severity levels recorded within each bar’s time period is displayed in each bar.

The following log severities are represented by the colours:

• Blue: Default, Debug, Info, and Notice are examples of low severities.
• Yellow: Moderate levels of severity, like Caution.
• Red: Extremely serious conditions, including emergency, critical, alert, and error.

There is a menu with choices to analyse your logs on each bar of the histogram.

• Time controls: Enable you to change the logs’ time range that appears in the Query results window.
• Timeline: This displays the logs’ time range, shown by histogram bars, that are shown in the query results window. You can better understand the logs you’re viewing inside the wider time span of your query by using the timeline.

Query results

Use the Query results window to see the results of your queries. You may inspect the specifics of each log entry, combine log entries together, and analyse log entries to identify trends in your logs to assist you in troubleshooting your applications.

Set up the column for time

Your log entries’ timestamps are shown in the Query results pane’s Time column. You can set the Time column to display only specific portions of the timestamp. This increases the amount of horizontal space in the log entry, allowing you to view more information.

Click the arrow_drop_down Additional time choices menu in the Time column, then choose one of the following options to choose which portions of the timestamp to display:

• Timing, date, and time zone
• Time and date (default)
• Time alone