Increase server memory bandwidth using CXL memory to boost AI and HPC workloads.
AI and HPC workloads demand a lot of memory bandwidth and, to a lesser extent, capacity. CXL “type-3” memory extension modules boost server memory capacity and bandwidth by using the CXL protocol via the processor’s PCIe ports.
The experimental results of employing Micron’s CXL modules to boost memory bandwidth for HPC and AI applications are discussed in this study. To the best of my knowledge, this is the first study to present actual data experiments using eight CXL E3.S (x8) Micron CZ122 devices on the Intel Xeon 6 processor 6900P (formerly codenamed Granite Rapids AP) with 128 cores and Micron DDR-5 memory running at 6400 MT/s on each of the CPU’s 12 DRAM channels.
To increase overall system bandwidth, the eight CXL memories were configured as a single NUMA configuration using a software-based page-level interleaving method between DDR5 and CXL memory nodes, which is compatible with the Linux kernel v6.9+. CXL’s ability to expand memory bandwidth is crucial for improving the efficiency of tasks including artificial intelligence (AI) and high-performance computing (HPC). CXL memory extension increases mixed read/write capacity by up to 38% and read-only bandwidth by 24%. The geometric mean of performance speedups for AI and HPC workloads is 24%.
What Is CXL memory?
Compute Express Link (CXL) memory connects accelerators, memory, CPUs, and other devices quickly.
Memory Sharing
Memory sharing across CPUs, GPUs, and FPGAs is made possible via CXL. This makes it possible to share resources, which may result in:
- Better results
- Decreased complexity in the software stack
- Reduced system costs overall
Memory Extension
The extension of CXL memory offers more capacity and bandwidth.
Memory pooling
CXL memory pooling enables access to numerous CXL memory devices by multiple computing nodes. This can increase scalability and performance, save money, and improve resource efficiency.
Industry standard
- Each and every major CPU, device, and datacenter operator has accepted CXL, an open industry standard.
- CXL was created to solve the “memory wall” issue, which arises when the speed at which DRAM memory is being improved outpaces the rate at which microprocessor performance is improving.
Micron CXL Memory Expansion Modules Increase Intel Xeon 6 System Memory Bandwidth
OVERVIEW
Important computational tasks requiring significant processing and memory resources are included in high-performance and AI workloads. These workloads are commonly used in simulations, scientific research, and data-intensive applications like as DNA sequencing, weather forecasting, and computational fluid dynamics. AI is essential for analyzing massive information and propelling advancements in a variety of sectors, in addition to HPC.
In Retrieval Augmented Generation (RAG), for instance, LLM inference and vector search are essential tasks because they provide quick access to pertinent data and improve the caliber of produced replies, which increases the accuracy and contextual awareness of AI interactions. The performance of AI and HPC workloads on the Intel Xeon 6 processor 6900P series, which is currently in full production, in conjunction with Micron CZ122 CXL devices is examined in this study through experimental work done by Micron and Intel.
By leveraging CXL memory extension to increase system memory bandwidth beyond local DRAM modules, the study measures the performance benefits of using Micron CZ122 devices in HPC/AI applications. CXL’s ability to expand memory bandwidth is crucial for improving HPC and AI job performance.
Although the main goal of CXL has been to increase memory capacity, its benefits for applications requiring a lot of bandwidth still need to be fully investigated and measured in actual CXL-capable systems, making use of as many PCIe lanes as feasible. Optimizing the capabilities of each memory tier in terms of memory bandwidth can be difficult since local DRAM and CXL memory have different unique bandwidth characteristics based on the read/write ratio of workloads. This is accomplished by optimization using a software-based weighted interleaving technique that is present in the popular Linux kernel release.
The table below provides further information about the platform.
| Feature | Specification |
|---|---|
| Platform | Intel Avenue City |
| CPU Family | Intel Xeon 6 6900P series |
| CPU Cores | 128 |
| Native DRAM | Micron DDR5-64GB (6400MTs) (12 modules ~ 768 GB) – HEX mode |
| CXL Memory | Micron CZ122 – 128GB 8 (8 modules E3.S form factor ~ 1TB) |
FINAL RESULTS
When utilized on systems with Intel’s 6th Generation Xeon processors, Micron’s CZ122 CXL memory modules in software level ratio-based weighted interleave configuration dramatically increase memory bandwidth for HPC and AI applications, according to the experimental results reported in this study.
Important conclusions from include this study
- It increased bandwidth, the combination of native DDR5-6400 memory and CXL-based memory extension significantly improves system performance.
- A key element in attaining these performance improvements is the optimization of the DRAM:CXL ratios.
- The potential for CXL technology to significantly improve artificial intelligence and high-performance computing applications.
The results of this study highlight how CXL may greatly enhance system performance and efficiency in demanding situations. In order to address the growing computational needs for HPC and AI workloads, future research and development activities should keep investigating and improving this integration, opening the door for even more advancements in hybrid memory systems.
