Highlights:
.The 49th International Conference on Very Large Data Bases (VLDB) will be held in Vancouver, Canada, from August –September 2023.
.Intel presents eight co-authored papers in the main conference’s research, industrial, and demonstrations tracks.
.The Second International Workshop on Composable Data Management Systems (CDMS), held alongside the VLDB Conference, accepted Intel researchers’ paper.
.Intel Labs is proud to recognize Nesime Tatbul, a senior research scientist at Intel’s Parallel Computing Lab (PCL), for receiving a Distinguished Associate Editor award at this year’s conference for their outstanding service to PVLDB Volume 16.
.This year’s International Conference on Very Large Data Bases (VLDB) will be held in Vancouver, Canada, August –September . Research talks, tutorials, demonstrations, and workshops at the conference will cover data management, database, and information systems research, which are the technological foundations of 21st-century applications.
Intel And Collaborators :
Intel is pleased to share eight co-authored contributions in the main conference’s research, industrial, and demonstrations tracks. These publications include results from research projects with academic and industry collaborators like MIT DSAIL and Meta. The Second International Workshop on Composable Data Management Systems (CDMS), held alongside the VLDB Conference, accepted Intel researchers’ paper. This paper introduces Gluten, an Intel-led open-source software project with industrial contributors.
In addition to the works presented, Intel Labs is proud to recognize Nesime Tatbul, a senior research scientist at Intel’s Parallel Computing Lab (PCL), for receiving one of this year’s conference awards for Distinguished Associate Editors for their outstanding service to PVLDB Volume 16.
Read about Intel’s co-authored publications below:
Research Track
Strong Query-Driven Cardinality Estimation under Workload Changes
Historical query logs teach query-driven cardinality estimation models. Their low storage, fast inference and training, and adaptability to any query make them lightweight. Due to workload drift, such models can perform poorly if the query pattern or data changes. This makes them unreliable and hard to deploy.
In this paper, researchers examine why workload drift makes models unpredictable and modify query representation and neural network training to make query-driven models robust to workload drift. First, they randomly mask table or column features during training to simulate workload drift in queries involving unseen tables or columns. This forces the model to predict with missing query information, using robust features based on current DBMS statistics that are useful even when query or data drift occurs.
Second, join bitmaps use sideways information passing to make sampling-based features consistent across joins. Both ideas can be adapted to handle data updates, they conclude. The paper generalizes across workloads and databases better than previous works. Using our techniques on a simple workload (JOBLight-train) with 40𝑘 synthetic queries of up to 3 tables, our model can generalize to the more complex Join Order Benchmark, which includes queries with up to 16 tables, and improve query runtimes by 2× over PostgreSQL.
The work also shows robustness with data updates and other workloads. Researchers discuss situations where we expect and see improvements and more difficult workload drift scenarios where these techniques do not improve over PostgreSQL. Standard query-driven models can perform much worse than PostgreSQL in the most difficult scenarios, but the proposed models never do.
Video stream extract-transform-load
Social media, self-driving cars, and traffic cameras generate large, cheap video streams. Storage and querying such large amounts of video is prohibitively expensive. This paper suggests treating large-scale video analytics as data warehousing: Video is easy to produce but needs to be converted into an application-specific, queryable format.
The Video Extract-Transform-Load problem is defined similarly. V-ETL systems must reduce the cost of running a user-defined job and guarantee throughput to keep up with data production. Researchers propose Skyscraper, a V-ETL-specific system, because no current system meets both needs. Skyscraper can execute any video ingestion pipeline and adapt its sampling rates and resolutions to the content to reduce cost without compromising quality.
Skyscraper uses buffering and cloud bursting to handle workload peaks caused by expensive processing configurations and cheap on-premises compute. In experiments, Skyscraper significantly reduced V-ETL ingestion costs compared to current SOTA system adaptations while providing robustness guarantees that these systems lack.
A Persistent Learned Index for Non-Volatile Memory with High Performance and Instant Recovery
NVM is an alternative to next-generation main memories. The majority of NVM tree indices use B+-tree-like structures. This work considers integrating learned indexes into NVM to improve NVM-aware indices. Two challenges arise from such integration: (1) Learned indices use huge nodes to achieve a flat structure, while existing NVM indices use small nodes to accelerate insertions with crash consistency.
(2) Learning indices’ node structures are not NVM-friendly, causing multiple NVM block misses. PLIN is a new persistent learned index proposed in this paper. An NVM-aware data placement strategy, locally unordered and globally ordered leaf nodes, a model copy mechanism, and a hierarchical insertion strategy make PLIN novel. PLIN is also proposed for the instant-recovery NVM-only architecture.
Researchers also present optimistic concurrency control and fine-grained locking mechanisms to scale PLIN to concurrent requests. PLIN was compared to APEX, PACtree, ROART, TLBtree, and Fast&Fair in real persistent memory experiments with different workloads. On average, PLIN outperforms its competitors in insertion and query performance by 2.08x and 4.42x, respectively. PLIN can recover from a system crash in approximately 30 seconds.
Similarity Search with Compressed Indices in One Blink
Vectors represent data today. Similarity search—finding vectors similar to a query among millions and billions—is a common problem with many applications. Billion-scale similarity search is best with graph-based indices. Due to their random-access memory pattern, they cannot reach their full potential.
New methods and systems for faster and smaller graph-based indices are presented here. To improve search performance with fast similarity computations and a reduced effective bandwidth, researchers introduce Locally-adaptive Vector Quantization (LVQ), a novel vector compression method that uses per-vector scaling and scalar quantization to reduce memory footprint and barely affect accuracy.
When combined with a new high-performance computing system for graph-based similarity search, LVQ sets the new performance and memory footprint standards. LVQ outperforms the second-best alternatives for billions of vectors by (1) 20.7x in low-memory throughput with a 3x memory footprint reduction, and (2) 5.8x in high-throughput with 1.4x less memory.
Industrial Track
Learned SQL Database Query Optimization AutoSteer
In any SQL database with tunable optimizer knobs, AutoSteer, a learning-based solution, automatically optimizes queries. AutoSteer adds new features to the Bandit optimizer (Bao) like automated hint-set discovery to reduce integration effort and improve usability in monolithic and disaggregated SQL systems.
The team successfully applied AutoSteer to five popular open-source database engines with diverse query optimizers: PostgreSQL, PrestoDB, SparkSQL, MySQL, and DuckDB. A detailed experimental evaluation with public benchmarks (JOB, Stackoverflow, TPC-DS) and Meta’s PrestoDB production workload followed.
The evaluation shows that AutoSteer can outperform these engines’ native query optimizers (e.g., PrestoDB by 40%) and match Bao-for-PostgreSQL with reduced human supervision and increased adaptivity by replacing Bao’s static, expert-picked hint-sets with automatically discovered ones. Researchers provide an open-source AutoSteer implementation and a visual tool for query optimization experts.
TPCx-AI: An Industry Standard Benchmark for AI and ML Systems
AI and ML have been around for years, but new hardware and model training and inference techniques have greatly improved their performance. Even experts struggle to find good deployments with the growing number of algorithms, systems, and hardware solutions. Researchers and industry experts have developed benchmark suites for AI and ML applications and systems to address this challenge.
None of the existing benchmarks measure end-to-end ML deployment performance, but they help compare AI applications. Many have been rigorously developed by academia and industry, but no benchmark is standardized. The first industry standard benchmark for end-to-end machine learning deployments, the TPC Express Benchmark for Artificial Intelligence (TPCx-AI), is introduced in this paper. The first AI benchmark to represent common ML and AI pipelines is TPCx-AI. The TPCx-AI software kit includes a data generator, driver, and two workload implementations, one based on Python libraries and one on Apache Spark.
The complete benchmark and scale factor results are presented. TPCx-AI’s main contributions are a novel unified data set that covers structured and unstructured data, a fully scalable data generator that can generate realistic data from GB to PB scale, and a diverse and representative workload using different data types and algorithms that covers real ML workloads like data integration, processing, training, and inference.
A General-Purpose, Modular, and Heterogeneous Acceleration Toolkit for Data Analytical Engines
Data processing engines benefit from query compilation and hardware acceleration. In recent years, many studies have explored and adopted these methods. Some engines still don’t use them for various reasons.
An often-cited reason is that these techniques make engines too complicated to maintain. Because these techniques lack widely accepted architectures and libraries, adoption often requires starting from scratch and a lot of effort. This paper proposes Intel Big Data Analytic Toolkit (BDTK), an open-source C++ acceleration toolkit library for analytical data processing engines. BDTK supports query compilation and hardware accelerators with lightweight, easy-to-connect, reusable components with interoperable interfaces.
BDTK query compilation uses vectorized execution and data-centric code generation for high performance. BDTK helps engines optimize query compilation and hardware accelerators to optimize performance bottlenecks with less engineering effort.
Demos Track
Insight into Steered Query Optimizers
Steered query optimizers correct traditional query optimizers’ planning errors by giving them per-query hints. QO-Insight, a visual tool for exploring steered query optimizers’ query execution traces, is introduced in this paper. Visual inspection and analysis allow database administrators and experts to gain qualitative insights and improve performance from steered query optimizers, which are often seen as black boxes.
CDMS Workshop
Gluten Open-Source Software Project: Lakehouse-Era Java-based Query Engine Modernization
Every year, exponential data growth and machine learning’s appetite for processing it transform the data management industry. In response, the data lakehouse architecture emerged. The transformative lakehouse architecture and the need to allow a diverse set of query engines to access lakehouse data are driving refactoring in these query engines. Composable data management system is industry’s response. This paper introduces the Gluten OSS project, a CDMS implementation. Gluten is a Java Native Interface (JNI) bridge that lets Java-based query engines use native acceleration libraries like Meta’s Velox OSS project to accelerate processing.
