Overview of the Flow Analyzer
Without writing intricate SQL queries to analyze VPC Flow Logs, you can quickly and effectively comprehend your VPC traffic flows with Flow Analyzer. With a 5-tuple granularity (source IP, destination IP, source port, destination port, and protocol), Flow Analyzer enables you to conduct opinionated network traffic analysis.
Flow Analyzer, created with Log Analytics and driven by BigQuery, allows you to examine your virtual machine instances’ inbound and outgoing traffic in great detail. It enables you to keep an eye on, troubleshoot, and optimize your networking configuration for improved security and performance, which helps to guarantee compliance and reduce expenses.
Data from VPC Flow Logs that are kept in a log bucket (record format) are examined by Flow Analyzer. You must choose a project with a log bucket containing VPC Flow Logs in order to use Flow Analyzer. Network monitoring, forensics, real-time security analysis, and cost optimization are all possible with VPC Flow Logs.
The fields contained in VPC Flow Logs are subjected to searches by Flow Analyzer.
The following tasks can be completed with Flow Analyzer:
- Create and execute a basic VPC Flow Logs query.
- Create a SQL filter for the VPC Flow Logs query (using a WHERE statement).
- Sort the query results based on aggregate packets and total traffic, then arrange the results using the chosen attributes.
- Examine the traffic at specific intervals.
- See a graphical representation of the top five traffic flows over time in relation to the overall traffic.
- See a tabular representation of the resources with the most traffic combined over the chosen period.
- View the query results to see the specifics of the traffic between a given source and destination pair.
- Utilizing the remaining fields in the VPC Flow Logs, drill down the query results.
How it operates
A sample of network flows sent from and received by VPC resources, including Google Kubernetes Engine nodes and virtual machine instances, are recorded in VPC Flow Logs.
The flow logs can be exported to any location supported by Logging export and examined in Cloud Logging. Log analytics can be used to perform queries that examine log data, and the results of those queries can subsequently be shown as tables and charts.
By using Log Analytics, Flow Analyzer enables you to execute queries on VPC Flow Logs and obtain additional information about the traffic flows. This includes a table that offers details about every data flow and a graphic that shows the largest data flows.
Components of a query
You must execute a query on VPC Flow Logs in order to examine and comprehend your traffic flows. In order to view and track your traffic flows, Flow Analyzer assists you in creating the query, adjusting the display settings, and drilling down.
Traffic Aggregation
You must choose an aggregation strategy to filter the flows between the resources in order to examine VPC traffic flows. The following is how Flow Analyzer arranges the flow logs for aggregation:
- Source and destination: this option makes use of the VPC Flow Logs’ SRC and DEST data. The traffic is aggregated from source to destination in this view.
- Client and server: this setting looks for the person who started the connection. The server is a resource that has a lower port number. Because services don’t make requests, it also views the resources with the gke_service specification as servers. Both directions of traffic are combined in this shot.
Time-range selector
The time-range picker allows you to center the time range on a certain timestamp, choose from preset time options, or define a custom start and finish time. By default, the time range is one hour. For instance, choose Last 1 week from the time-range selector if you wish to display the data for the previous week.
Additionally, you can use the time-range slider to set your preferred time zone.
Basic filters
By arranging the flows in both directions based on the resources, you may construct the question.
Choose the fields from the list and enter values for them to use the filters.
Filter flows that match the chosen key-value combinations can have more than one filter expression added to them. An OR operator is used if you choose numerous filters for the same field. An AND operator is used when selecting filters for distinct fields.
For instance, the following filter logic is applied to the query if you choose two IP address values (1.2.3.4 and 10.20.10.30) and two country values (US and France):
(Country=US OR Country=France) AND (IP=1.2.3.4 OR IP=10.20.10.30)
The outcomes may differ if you attempt to alter the traffic choices or endpoint filters. To see the revised results, you have to execute the query one more.
SQL filters
SQL filters can be used to create sophisticated queries. You can carry out operations like the following by using sophisticated queries:
- Comparing the values of the fields
- Using AND/OR and layered OR operations to construct intricate boolean logic
- Utilizing BigQuery capabilities to carry out intricate operations on IP addresses
BigQuery SQL syntax is used in the SQL filter queries.
Query result
The following elements are included in the query results:
- The highest data flows chart shows the remaining traffic as well as the top five largest traffic flows throughout time. This graphic can be used to identify trends, such as increases in traffic.
- The top traffic flows up to 10,000 rows averaged during the chosen period are displayed in the All Data Flows table. The fields chosen to organize the flows while defining the query’s filters are shown in this table.
