IBM and Intel partnership
IBM Empowers Enterprises to Scale AI. Intel Gaudi 3 AI accelerators make IBM Cloud hybrid cloud solutions affordable, scalable, and easy to implement.
Executive Summary
Intel and IBM have helped customers and partners achieve their goals since 1981, when the first IBM personal PC was powered by the 8088 CPU. This long-term collaboration has spurred innovation and produced great outcomes across technology generations. IBM Cloud now offers Intel Gaudi 3 AI accelerators, continuing their collaboration. This service helps organizations expand AI and innovate cost-effectively while prioritizing openness, security, and resiliency. This partnership will feature Intel Gaudi 3 AI accelerators in IBM’s watsonx AI and data platform in Q2 2025.
Challenge
AI can boost business efficiency and competitiveness. GenAI can answer 97% of customer questions, generate 60% of software development material, and increase HR efficiency by 8x.
Due to cost and scalability problems, the time and effort required to choose and deploy infrastructure that fulfils performance criteria, and security, compliance/governance, and resilience requirements, organizations struggle to accomplish such tantalizing ambitions. When organizations implement AI workloads in the hybrid cloud, these issues multiply. Enterprises need reliable partners to maximize GenAI.
Solution
IBM and Intel have a longstanding partnership to create AI systems with low total cost of ownership (TCO) and enable an open ecosystem to grow enterprise AI.
IBM Cloud is the first enterprise cloud service provider to provide Intel Gaudi 3 AI accelerators. IBM Cloud’s full-stack GenAI service includes Intel Gaudi 3 AI accelerators.
AI-specific architecture and functionality in Intel Gaudi 3 accelerators: These devices enable an open development architecture and address the growing demand for GenAI, big model inferencing, and model fine-tuning. Intel Gaudi 3 accelerators support multi-model large language models (LLMs) and retrieval-augmented generation (RAG), simplifying interaction with IBM’s watsonx and data platform. For deep neural network inference, Intel Gaudi 3 accelerators provide matrix math engines, tensor computing processors, high-bandwidth memory, and Ethernet connections.
Expected Business Outcomes
- Competitive AI performance: Intel Gaudi 3 accelerators have 4x the computation, 2x the networking bandwidth, and 1.5x the memory bandwidth of Intel Gaudi 2. They also have 128 GB of HBM @ 3.7 TB/sec to boost GenAI performance.
- Budget-friendly design: Intel Gaudi 3 AI accelerators on IBM Cloud can generate over 5,000 tokens per second for IBM’s granite-8b model on a single card, allowing over 100 concurrent users with less than 20 milliseconds inter-token latency. This results in ~50 average-length emails.
- Enterprises may scale from a single node (eight accelerators) with 9.6 TB/s throughput to a 1,024-node cluster (8,192 accelerators) with 9.830 PB/s. To save costs, many industry-standard and high-capacity Ethernet switches and other supporting infrastructure are used for scaling.
IBM’s granite-8b model has a linear performance scaling factor of over 5,000 tokens per second, according to tests. To successfully meet AI computing demand, enterprises require this scalability.
- Support open development: IBM has led open-source technology development for over 20 years. IBM Cloud uses Linux and Kubernetes for its hypervisor, developer tools, blockchain, and AI. Operating, hardening, scaling, and contributing to open-source are IBM’s specialties.
Intel Gaudi 3 accelerators support many AI applications. They are compatible with PyTorch, the open framework used for most GenAI development, making integration and migration easy. Developers may easily use Intel Gaudi 3 AI accelerators’ superior AI innovation capabilities to minimize development time and code maintenance.
Intel Gaudi 3 accelerators are helping IBM address the problems organizations face in delivering GenAI workload processing power at a cost-effective entry point. Intel Gaudi 3 accelerators power AI workloads to reduce TCO and boost performance.
By running GenAI workloads on Intel Gaudi 3 accelerators, enterprises can access new AI business opportunities, including in highly regulated industries, to test, innovate, and deploy AI inferencing solutions more cost-effectively, scaling enterprise AI with optimized price/performance. Adding Intel Gaudi 3 AI accelerators to IBM Cloud Virtual Servers for Virtual Private Cloud (VPC) helps x86-based organizations run applications quickly and securely, improving user experiences.
A Closer Look at Intel Gaudi 3 AI Accelerators and IBM Cloud Synergy
Chatbots, virtual assistants, code creation, natural language translations, and text summarization and paraphrasing are enterprise GenAI applications. Intel Gaudi 3 accelerators are excellent for GenAI applications, as mentioned above. The benefits of utilising IBM go beyond hardware selection.
Enterprises can benefit from Intel Gaudi 3 accelerators and IBM’s lengthy experience in providing an enterprise cloud platform for heavily regulated sectors including financial services, government, healthcare, and telco. IBM hybrid cloud by design can also benefit retail and media.
Designed for Security and AI Governance
Security is crucial for AI and sensitive company data. To use GenAI without exposing their proprietary data and ideas to the Internet, enterprises must ensure data security.
Responsible AI deployment requires end-to-end AI lifecycle tracking employing automated methods for clarity, monitoring, and cataloguing. Governance is essential for AI model trust and transparency. The governance process lets companies lead, manage, and monitor AI across business processes. IBM offers an end-to-end AI platform for enterprises seeking secure AI.
IBM Cloud offers security-focused hybrid cloud AI infrastructure:
- Customized training datasets.
- Unclosed models.
- Stacking hybrid training and inference.
Hybrid Cloud by Design
AI requires data, but complicated and compartmentalized IT architectures make it hard for enterprise IT professionals to gather the proper assets. GenAI struggles in distributed or heterogeneous environments:
- Different IT stacks perform multi-model workloads.
- A hybrid multicloud infrastructure requires expensive resources.
- Enterprise workflows complicate model and data governance.
- Heterogeneous settings impede scalability and replicability, and programs stay in pilot too long.
- Distributed data limits quality and access.
- Disconnected data can reduce AI efficiency, revenue, and security.
To solve these problems, you must realize that hybrid cloud and AI are one and the same. Organizations must be hybrid by design.
IBM Cloud’s data-focused, platform-oriented, and AI-infused capabilities enable organizations worldwide to exploit hybrid cloud and AI. AI workloads have varied requirements, hence one design is not suitable for them. Proper data management and a strategic hybrid cloud architecture may assist IT directors make informed decisions for their companies to make data and AI easier to access and execute.
IBM’s hybrid cloud strategy combines cloud knowledge with strong hyperscaler and software vendor connections and Red Hat OpenShift, an open hybrid cloud platform.
Flexible Deployment Options
Customers can deploy AI workloads on Intel Gaudi 3 accelerators on IBM Cloud in several ways:
- A purpose-built AI server on IBM Cloud Virtual Private Cloud (VPC) for traditional and GenAI workloads, including Red Hat Enterprise Linux (RHEL) AI workloads, is available.
- Bring your own Watsonx licence: In Q2 2025, IBM Cloud VPC clients can deploy IBM watsonx.ai on their Intel Gaudi 3-based virtual server for AI stack control.
- IBM Cloud provides deployable architectures (DA) design modules for developers and operations teams to easily deploy new features and system changes without manual intervention. IBM Cloud clients can immediately integrate Intel Gaudi 3 capabilities with watsonx software, IBM Cloud Virtual Server for VPC, and Red Hat OpenShift and Kubernetes Service DAs. The DAs will be available in 2H 2025.
In Q2 2025, IBM Cloud will deliver Intel Gaudi 3 as a worker node for Red Hat OpenShift AI clusters and Kubernetes Service for clients using managed containerized infrastructure.
