MPI Intel Library: Reliable & Scalable Cluster Communication

The MPI Intel Library: Provide cluster messaging that is adaptable, effective, and scalable.

One Library with Multiple Fabric Support

Using the open source MPICH specification, the Intel MPI Library is a multibaric message-passing library. Utilize the library to develop, manage, and test sophisticated, intricate programs that run more efficiently on Intel-based and compatible HPC clusters.

• Create programs that can execute on a variety of cluster interfaces that you specify at runtime.
• Deliver optimal end-user performance as soon as possible without requiring alterations to the operating system or software.
• Use automatic tuning to get the optimal latency, bandwidth, and scalability.
• Deploy on the most recent optimized fabrics and link to a single library to shorten time to market.

Download as Part of the Toolkit

The Intel oneAPI HPC Toolkit includes the MPI Intel Library. Obtain the necessary tools to evaluate, improve, and produce scalable apps.

Download the Stand-Alone Version

The Intel MPI Library can be downloaded separately. You can select your favorite repository or get binaries from Intel.

Features

OpenFabrics Interface (OFI) Support

Communication services are exposed and exported to High Performance Computing(HPC) applications using this optimized framework. APIs, provider libraries, kernel services, daemons, and test apps are important parts.

OFI is used by the Intel MPI Library to manage all communications.

• Permits a more efficient route that concludes with data communications from the application code.
• Enables runtime underlying fabric tweaking using basic environment parameters, including network-level tools like multirail for more bandwidth.
• Aids in providing the best possible performance for large-scale solutions built on Cornelis and Mellanox InfiniBand networks.

Increased communication throughput, lower latency, streamlined program design, and a shared communication infrastructure are the outcomes.

Scalability

The high-performance MPI 3.1 and 4.0 standards are implemented on many fabrics by this library. This enables you to rapidly achieve optimal application performance without requiring significant changes to the operating systems or software, even if you update to new interconnects.

• Tracing hybrid multithreaded MPI applications for best performance on multicore and manycore Intel architectures is made possible by thread safety.
• The process manager mpiexec.hydra provides improved start scalability and is:
  • A process management tool for initiating concurrent tasks
  • That is built to natively support several network protocols, including SSH, RSH, PBS, Slurm, and SGE
• Integrated cloud support for Google Cloud Platform, Microsoft Azure, and Amazon Web Services

Performance and Tuning Utilities

You can get optimal performance out of your apps with the help of two more features.

Interconnect Independence

For fast interconnects via OFI, the library offers an accelerated, universal, multibaric layer that works with the following configurations:

• Sockets for Transmission Control Protocol (TCP)
• Communal memory
• Remote Direct Memory Access (RDMA)-based interconnects, such as Ethernet and InfiniBand

Read more on New Intel Ethernet E830 Network Adapters Improve Performance

It does this by minimizing the memory footprint by dynamically establishing the connection only when required. Additionally, it automatically selects the quickest mode of transportation.

• Create MPI code without regard to the fabric, knowing that it will function well on any network you specify at runtime.
• Only the necessary amount of memory should be allocated by using a two-phase communication buffer-enlargement capability.

Application Binary Interface Compatibility

An application’s low-level interface between two program modules is called an application binary interface (ABI). It dictates the size, arrangement, and alignment of data types in addition to how functions are invoked. Applications that are compatible with ABI follow the same set of runtime naming guidelines.

ABI compatibility with current MPI-1.x and MPI-2.x applications is provided via the MPI Intel Library. To benefit from the library’s performance enhancements, you can use its runtimes without recompiling, even if you’re not ready to switch to the new 3.1 and 4.0 standards.

A collection of MPI performance metrics for point-to-point and global communication activities over a range of message sizes is called Intel MPI Benchmarks. To obtain results for a specific subset, run each of the supported benchmarks or provide a single executable file in the command line.

The benchmark data that was produced completely describes:

• Cluster system performance, including throughput, network latency, and node performance
• Efficiency of the MPI implementation

To get the best performance, you can adjust the library’s extensive collection of default parameters or use them exactly as is. Use mpitune to modify your cluster or application parameters if you wish to fine-tune them beyond the defaults. Then, repeatedly change and fine-tune the parameters until you get the optimal performance.

MPI Intel Library: Windows OS Developer Guide

MPI Tuning: The Intel MPI Library offers a tool called Autotuner

Autotuner

Autotuning could enhance the performance of an application that spends a lot of time in MPI collective operations. Autotuner has a very minimal overhead and is simple to use.

The tuning scope of the autotuner tool is the I_MPI_ADJUST_ family are MPI collective operation algorithms. Tuning is restricted by the autotuner to the cluster configuration as it stands (fabric, number of ranks, number of ranks per node). Performance may be enhanced simply by turning on the autotuner, which operates while an application is running. A new tuning file with MPI collective operations tailored to the requirements of the application can also be created and supplied to the I_MPI_TUNING_BIN variable.

Intel MPI Library Spec