Intel Arc PyTorch: How Intel Enables Generative AI

Intel Arc GPUs Enhance Generative AI with PyTorch

Intel Arc PyTorch

As seen by its latest foray into Generative AI (GenAI) technologies, Intel has long been at the forefront of technical innovation. As AI-powered gaming became more popular, Intel aimed to provide a simple and easy-to-use GenAI inferencing solution for AI PCs with Intel’s newest GPUs. Intel successfully created AI Playground, an open source application that exhibits sophisticated GenAI workloads, by using PyTorch as the foundation for development efforts.

The Business Difficulty

Intel’s objective was to provide an easily navigable GenAI inferencing solution specifically designed for Intel-powered AI PCs. It realised that its newest line of client GPUs needed to demonstrate the power of the newest GenAI workloads. In order to solve this, it used PyTorch to create the open source starter application AI Playground, which comes with a thorough developer reference sample on GitHub.

This application uses retrieval-augmented generation (RAG) characteristics to seamlessly combine picture creation, image enhancement, and chatbot functions into a single, easy-to-use installation package. In addition to showcasing the capabilities of various AI workloads, this project acts as an ecosystem-wide teaching tool, assisting developers in making the most of the Intel Arc GPU product family for cutting-edge AI applications. To speed up inferencing, this method makes use of Intel Arc Xe Cores and Xe Matrix Extensions (XMX).

How PyTorch Was Used by Intel

The main AI framework for AI Playground is called PyTorch. Intel makes considerable use of PyTorch’s eager mode, which is ideal for its generative models’ dynamic and iterative nature. This method allows us to quickly prototype and iterate on sophisticated AI capabilities while simultaneously improving its development workflow. Intel has produced a solid reference sample that demonstrates the possibilities of GenAI on Intel GPUs in a single, coherent application by utilising PyTorch’s formidable capabilities.

Using PyTorch to Address AI Challenges

By offering a strong training and inference framework tailored for both discrete and integrated Intel Arc GPU product lines, PyTorch has been crucial in helping tackle its AI difficulties. It was essential to select PyTorch over other frameworks or APIs. Other choices might have narrowed its feature set and considerably prolonged its time to market by requiring more custom development or one-off solutions. It took use of PyTorch’s adaptability and user-friendliness to free up its team to concentrate on experimentation and innovation rather on infrastructure. By maximising computational efficiency and facilitating smooth scaling on Intel hardware, the inclusion of Intel Extension for PyTorch significantly improved performance and made sure that its program operated more quickly and effectively.

The Advantages of PyTorch Use

The vast PyTorch ecosystem, which links us to a vibrant and collaborative developer community, is the main advantage of choosing PyTorch for us. This partnership has made it easier to integrate the newest GenAI capabilities into AI Playground by enabling the smooth deployment of important features from already-existing open source projects. We achieved this with surprisingly little re-coding, guaranteeing that these cutting-edge functionalities are easily available on Intel Arc GPUs.

Intel’s PyTorch Optimisations

Accelerate the deployment of AI from research to production.

Optimise the Performance of PyTorch on Intel Hardware

A well-liked framework for AI and machine learning for both research and industrial applications is PyTorch. Deep learning applications whose computationally demanding training and inference push the boundaries of available hardware resources frequently employ this open source toolkit.

Prior to being upstreamed into open source PyTorch, Intel offers its most recent features and optimisations as the Intel Extension for PyTorch.

The Intel Extension for PyTorch can be used with a few lines of code to:

• Benefit from the latest recent PyTorch hardware and software optimisations from Intel.
• Reduce model size and inference computing burden by automatically combining various precision data types.
• Use APIs to add your own performance customisations.

In order to optimise the PyTorch framework for Intel technology, Intel also collaborates closely with the open source PyTorch project. Together with the extension, these PyTorch optimisations are a part of the whole range of Intel AI and machine learning development tools and resources.

Qualities

PyTorch Open Source Driven by Intel’s Optimisations

• With Intel’s open source contributions, you can get the finest PyTorch training and inference performance on Intel CPU or GPU hardware.
• PyTorch 2.5 adds support for Intel GPU prototypes.
• To parallelise and speed up PyTorch tasks, utilise the Intel Deep Learning Boost, Intel Advanced Vector Extensions (Intel AVX-512), and Intel Advanced Matrix Extensions (Intel AMX) instruction set capabilities.
• Use PyTorch’s oneAPI Collective Communications Library (oneCCL) bindings to conduct distributed training.

PyTorch Optimisations and Features with the Intel Extension

• With only minor code modifications, implement the most recent performance enhancements not yet available in PyTorch.
• Use an Intel CPU or GPU to run PyTorch.
• Reduce the computational effort and model size by automatically switching between float32 and bfloat16 operator datatype precision.
• Manage thread runtime features including asynchronous task launching and multistream inference.

Enhanced Implementation with the OpenVINO Toolkit

• To reduce model size and speed up inference, import your PyTorch model into OpenVINO Runtime.
• Target Intel CPUs, discrete or integrated GPUs, NPUs, or FPGAs right away.
• Use the OpenVINO model server to optimise inference in cloud, container-based, or microservice applications. Scale using gRPC or REST inference services and the same architecture API as KServe for inference execution.

Specifications