Google’s Role in Supporting DAOS Foundation Launch

Advancing AI Storage: DAOS Foundation and Google

Google are pleased to announce that Google Cloud has joined the DAOS Foundation as a founding member, which was established today by the Linux Foundation. The world-record-breaking open-source DAOS storage stack, which supports workloads for next-generation HPC and AI/ML, will see faster investment and development thanks to the creation of the DAOS Foundation in partnership with partners.

In order to promote a new paradigm in storage for high performance computing, Intel Corporation started investing in DAOS in 2012. (HPC). Notable innovations like the following have been made possible by the combination of fully-distributed metadata management, a key-value store architecture, and the absence of reliance on kernel modifications:

Remarkable improvements in performance for I/O, throughput, and metadata operations; simpler integration of containers and Kubernetes; greater architectural flexibility; and reduced infrastructure overhead (no longer requiring metadata servers).

The enormous influence DAOS has had on the field of scientific computing is shown in the impending implementation at Argonne National Laboratories, which is intended to deliver >25TB/s of throughput for HPC.

In addition, Google just introduced Parallelstore, a DAOS-based scratch storage solution that outperforms rival Lustre scratch solutions with read-throughput performance up to 6.3 times faster. HPC and large language models (LLMs) have high demands on throughput, IOPS, MDOPS, and ultra-low latency. Client requests for AI/ML training based on big data sets, tiny files, and low-latency operations are perfectly aligned with DAOS’s design and performance features.

“As a founding member of the DAOS Foundation, Google is thrilled,” Google Storage VP/GM Sameet Agarwal stated. “Our new storage service, Parallelstore, is built on this industry-leading architecture, and we are thrilled to accelerate the growth of HPC and AI/ML workloads in the cloud.”

The Linux Foundation, which offers a framework for shared governance and ongoing support of the open source community, will create the DAOS Foundation. Companies have worked well together, and Google Cloud is eager to keep making this commitment.