A Simple Overview of the Intel TensorFlow Extension. A diverse, high-performing, deep learning extension plugin is the Intel Extension for TensorFlow.
Enables users to freely plug an XPU into TensorFlow, displaying the processing power within Intel’s hardware, and upstream multiple optimizations into open source TensorFlow. It is based on the TensorFlow Pluggable Device interface, which brings Intel CPUs, GPUs, and other devices into the Intel TensorFlow open source community for AI workload acceleration.
Let’s examine everything, including how you might accelerate your AI workloads using the extension.
What is Intel TensorFlow Extension?
The Framework
This is precisely what the Intel Extension for TensorFlow is: an addition to the open-source, stock Intel TensorFlow library that is specially tuned for optimal performance on Intel Architecture. Because of its compatibility with the stock TensorFlow public API, developers may keep the same user experience while making just minor code modifications.
The extension includes
- TensorFlow’s public API can be extended with a custom API. Under the itex.ops namespace, users can import and utilise the custom API.
- Good performance is made possible without requiring code modifications due to Intel Advanced Feature and Extension Management. Additionally, performance can be improved by using straightforward front-end Python APIs and tools. For specific application cases, this can be accomplished with small code modifications (~2–3 lines).
- XPU Engine, which integrates Intel GPU into the Intel TensorFlow ecosystem and offers device runtime and graph optimization. Deeper performance optimization on Intel CPU hardware is another benefit of this engine. To accommodate various situations, the user can install the Intel GPU backend or CPU backend independently.
Features
A number of features are available in the Intel Extension for TensorFlow to boost AI performance.
- Operator Optimization: Uses the Intel oneAPI DPC++/C++ Compiler to implement all GPU operators and optimize CPU operators. By default, operator optimization can be applied without the need for any configuration.
- Conv2D+ReLU and Linear+ReLU are two examples of graph optimization techniques that help fuse a given op pattern to a new single op for improved performance. Users don’t need any extra settings to implement this by default.
- Advanced Auto Mixed Precision (AMP): Makes models run quicker and use less memory during training and inference by using lower-precision data types like bfloat16 and float16. The extension offers an Advanced Auto Mixed Precision feature for improved performance and is completely compatible with the Keras mixed precision API in stock TensorFlow.
- Simple Python API: For many application scenarios, frontend Python APIs and utilities offer extra performance optimizations that can be achieved with small code modifications.
- GPU Profiler: To monitor TensorFlow model performance on the Intel GPU. Export ZE_ENABLE_TRACING_LAYER=1, export UseCyclesPerSecondTimer=1, and export ENABLE_TF_PROFILER=1 are the three environment variables that enable the profiler. Refer to GPU Profiler for further details.
- CPU Launcher (Experimental): Optimising configuration settings helps to increase performance. But there isn’t a single setup that works best for all topologies. Users must experiment with various combinations. A CPU launcher is provided by the Intel Extension for TensorFlow to automate these configuration settings, relieving users of laborious tasks.
- INT8 Quantisation: TensorFlow INT8 quantisation solution compatibility and the same user experience are provided by the Extension in collaboration with Intel Neural Compressor.
- XPUAutoShard on GPU (Experimental): TensorFlow graphs and input data are automatically sharded using this capability. To optimise hardware utilisation, these shards are installed on GPU devices.
- OpenXLA Support on GPU (Experimental): To build the Intel GPU backend for OpenXLA experimental support, the extension uses PJRT, a consistent Device API, as the supporting device plugin.
Google created Intel TensorFlow Serving, which serves as a link between applications that require the use of trained machine learning models and the models themselves. It preserves efficiency and scalability while streamlining the deployment and serving of models in a production setting.
Getting Started
Setting up
Hardware Needed: The Intel GPU and CPU are supported via the Intel Extension for TensorFlow.
Code Example
This code example will show users how to use the Intel AI Analytics Toolkit to run a Intel TensorFlow inference task on both the GPU and CPU. The actions listed below are carried out:
- Use the Intel Extension for TensorFlow’s (resnet50 inference sample).
- Make use of TensorFlow Hub’s resnet50v1.5 pretrained model.
- Inference with Intel Caffe using images GitHub.
- To operate on an Intel CPU and GPU, use a distinct conda environment.
- Examine oneDNN verbose logs to confirm CPU or GPU use.
- Use the Jupyter Notebook and Intel Developer Cloud to test the code sample.
What Comes Next?
You are also welcome to contribute to the project. In order to help you plan, develop, implement, and scale your AI solutions, it also encourage you to look into and integrate Intel’s other AI/ML Framework optimizations and end-to-end portfolio of tools into your AI workflow. Additionally, you can learn about the unified, open, standards-based oneAPI programming model that serves as the basis for Intel’s AI Software Portfolio.
