Cloud Logging
Completely controlled, real-time log management that includes exabyte-scale storage, search, analysis, and alerting.
Overview of Cloud Logging
An overview of Cloud Logging, a real-time log-management system that supports storage, search, analysis, and monitoring, is given in this article. Log data from Google Cloud resources is automatically gathered by Cloud Logging. Cloud Logging may receive log data from your apps, on-premise resources, and services from other cloud providers.
Additionally, alerting settings may be set up to notify you when specific types of events are detected in your log data by Cloud Monitoring. You can choose where your log data is kept for security or regulatory purposes.
Gather logs from third-party software and your apps
By instrumenting your application using a client library, you may gather logs from apps you create. However, instrumenting your application isn’t always required. For instance, under some circumstances, logs sent to stdout or stderr can be sent to your Google Cloud project using the Ops Agent.
By installing the Ops Agent and setting it up to write logs from your third-party apps, such as nginx, to your Google Cloud project, you may also get log data from those programs.
Troubleshoot and analyze logs
Using the Logs Explorer or Log Analytics pages in the Google Cloud interface, you may see and examine your log data. Both interfaces allow you to browse and query logs, but they offer different features and employ different query languages.
Monitor your logs
Cloud Logging may be set up to alert you when specific types of events take place in your logs. These alerts might be triggered when a trend is found in your log data or when a certain pattern shows up in a log entry.
Log storage
Configuring the log storage location is not required. All logs received by your Google Cloud project are automatically stored in a Cloud Logging log bucket by default. For instance, all of the logs that Compute Engine creates are automatically saved for you if it is a part of your Google Cloud project. You may, however, set up many features of your log storage, including which logs are kept, which are deleted, and where the logs are kept, if necessary.
Log categories
The purpose of log categories is to provide a description of the logging data that is at your disposal; they are not exclusive of one another:
The logs generated by your Google Cloud services are called platform logs. You may better understand the Google Cloud services you’re utilising and diagnose and troubleshoot problems with the aid of these logs.
Though they are produced by Google-provided software components that operate on your systems, component logs resemble platform logs. GKE, for instance, offers software components that consumers may operate in their own data centre or virtual machine. The user’s Google Cloud project receives the logs that are produced from their Google Kubernetes Engine instances. GKE offers user help by using the logs or their metadata.
Data model for logs
The dimensions across which you may query your log data depend on the data model that Cloud Logging employs to arrange it. For instance, you may query your data using the log’s name as it is a named collection of distinct items. Similar to this, you may create queries that only return log entries when the value of a LogEntry field satisfies specific requirements since each log is made up of log entries that are represented as LogEntry objects. For instance, only log entries with the severity column set to ERROR can be shown.
Features
Logs Explorer
Flexible query statements allow you to search, sort, and query logs. You may also store your searches, see sophisticated histogram visualizations, and use basic field explorers.
Custom logs and an API for ingestion
Use the public write APIs to write any custom log from on-premises or another cloud.
Alerting logs
With Cloud Monitoring, you can set up alerts for log-based metrics or individual messages in your logs.
Logs Analytics
BigQuery-powered Log Analytics provides access to platform and workload logging data that is fed into Cloud Logging. To query your logs and do sophisticated analytics, use SQL. Additionally, the log data is readily accessible in BigQuery, allowing you to correlate your logs with other business data.
Logs retention
Using the Logs Router, set criteria for distinct logs and varied retention durations for logs in various log buckets.
Logs-based metrics
Log data may be used to build metrics, which can then be visualized and dashboards created in Cloud Monitoring.
Audit logging
Access audit logs, which include 400 days of free data retention, document all admin and data access activities inside Google Cloud.
Third-party integrations
Configure Logs Router to export the logs and use Pub/Sub to integrate with external systems.
Logs archival
Logs may be readily exported into Cloud Storage to be stored for a longer period of time at a lesser cost.
Error Reporting
By automatically searching your logs for exceptions and cleverly combining them into relevant error categories, error reporting helps you spot issues through the clutter.
Log buckets and views
Log buckets offer a superior log storage solution that enables you to partition or centralize your logs according to your requirements. Then, using typical IAM controls, employ log views to define which logs a user should be able to access.
Advantages
Get started immediately
Platform logs are safely saved without requiring any setup and are consumed from Google Cloud services. Workload logs from VMs are recorded by the Ops Agent, and GKE workload logs are automatically recorded.
Quickly resolve issues
To help you debug problems across all of your services, Cloud Logging is linked with Cloud Monitoring, Error Reporting, and Cloud Trace. Set up log alerts to keep you informed of significant occurrences.
Real-time insights
Build Cloud Monitoring dashboards using log-based data and identify abnormalities quickly with real-time ingestion. For deeper insights, Log Analytics leverages BigQuery‘s capability in conjunction with Cloud Logging.
Cloud logging pricing
- You can manage your consumption and expenses with Google Cloud Observability’s price.
- The cost of Google Cloud Observability products is determined by use or data volume.
- There are no upfront costs or obligations when you begin using the free data use allotments.
- $300 in free credits are given to new clients to use for cloud logging.
