Big Query Omni cross-cloud MVs (Materialized Views)

Customers often ask how to make Big Query cross-cloud analytics simple and affordable as more companies adopt multi-cloud data infrastructures. Today, Google are excited to release the public preview of Big Query Omni cross-cloud materialized views to aid customers with cross-cloud analytics. Cross-cloud MVs let clients simply generate a summary materialized view on GCP from basic data assets on another cloud. Cross-cloud MVs are automatically and progressively maintained as base tables update, requiring only a small data transfer to sync the materialized view on GCP. The industry-first, cost-effective, and scalable capability allows clients to do seamless, efficient, and economical cross-cloud analytics.

Organizations need cross-cloud materialized views(Cross-cloud MVs). Why?

Customer demand for cross-cloud MVs is rising as they want to do more with their data across cloud platforms while keeping huge data assets in distinct clouds. Today, cloud analytics requires copying or replicating huge datasets across cloud providers, making it difficult. This is a laborious and expensive data transfer method. Customers want to optimize data operations and save money by integrating cross-cloud MVs.

Key client use cases where cross-cloud MVs can simplify workflows and reduce costs include:

Predictive analytics: Companies want to use Google Cloud’s cutting-edge AI/ML with Vertex AI. The ability to easily develop ML models on GCP using cross-cloud MVs and Google’s large language fundamental models like PaLM 2 and Gemini excites clients to explore novel data interactions. Cross-cloud MVs seamlessly ingest and aggregate data across a customer’s multi-cloud settings to use Vertex AI and Google Cloud’s LLMs.

Summary of cross-cloud or cross-region data for compliance: Emerging privacy use cases require raw data to stay in the originating country due to data sovereignty restrictions. Aggregating, summarizing, and rolling up data can solve cross-regional or cross-cloud data sharing and collaboration. This processed data, which meets privacy standards, can be replicated across regions for sharing and consumption by team and partner companies and updated gradually through cross-cloud MVs.

Marketing analytics: Companies commonly combine cloud data. CRM, user profile, and transaction data on one cloud are often integrated with campaign management or advertisements data in Google advertisements Data Hub. This connectivity is essential for privacy-safe consumer segmentation, campaign management, and marketing analytics. Sometimes only a subset or summary of user profile and transaction data from another cloud is needed to link Google Ads or campaign data. Customers want these interactions to be efficient and give data governance controls.

Business analytics near real-time: Business intelligence (BI) dashboards and reporting technologies enable real-time insights. These analytical applications are essential because they combine data from multiple sources. These dashboards need hourly, daily, or weekly updates to reflect the latest business data. Cross-cloud MVs refresh data assets independent of location, guaranteeing relevant and timely insights. Together with GCP’s strong Looker platform and semantic models, these features give users new insights and value.

BigQuery Omni’s cross-Cloud MV solution Benefits

BigQuery Omni’s cross-Cloud MV solution includes unique capabilities and benefits:

Simplicity: Cross-cloud MVs make data merging and analysis easier regardless of cloud location. They simplify running and managing complex analytics pipelines, large-scale data duplication, and regularly changing data.

Significant cost reduction: It dramatically minimizes data egress costs across clouds by simply transmitting incremental data when needed.

Automatic refresh: Cross-cloud MVs automatically refresh and incrementally update according on user preferences for convenience.

Unified governance: BigQuery Omni secures and governs materialized views in both clouds. This feature is essential for local and cross-cloud analytics.

Single pane of glass: The BigQuery user interface makes defining, querying, and managing cross-cloud MVs easy.

Industry/Customer scenarios

Cross-cloud MVs benefit several sectors and consumer scenarios, as seen below:

Healthcare: Data scientists in one department want to send daily or weekly data summaries from AWS to Google Cloud (BigQuery) for aggregate analytics and model building.

Media and entertainment: A marketing analyst wants to weekly combine, de-duplicate, and segment Ads Whiz data from AWS with Google Cloud listener and audience data to enhance audience reach.

Telecom: A data analyst wants to periodically centralize AWS log data and Ads server streaming data for revenue targeting.

Education: Data analysts must combine AWS product instrumentation data with Google Cloud enterprise data. With new items on their platform, they want to streamline their ETL process and cost issues with cross-cloud MVs.

Retail: A marketing analyst must securely link Azure user profile data to Ads Data Hub campaign data. New retail users enter the system daily, thus they employ cross-cloud MVs for combined analysis to keep data current.

Embrace cross-cloud analytics’ future

Google enable enterprises to break down cloud silos and use their rich, evolving data in near-real-time in Google Cloud using cross-cloud MVs. This revolutionary technology is changing cross-cloud analytics and multi-cloud architectures, giving customers more flexibility, cost-effectiveness, and actionable insights. Google deliver rich and actionable insights to data consumers faster and easier using BigQuery Omni cross-cloud analytics and Looker agile semantics.

Learn more

Watch the demo, read the public documentation, and sample the product to learn how cross-cloud MVs may improve your organization’s cross-cloud analytics.