Monday, February 17, 2025

The Omniverse Robotics, Autonomous Vehicles With Physical AI

Into the Omniverse: OpenUSD Workflows Promote Robotics and Autonomous Vehicles with Physical AI.

The purpose of NVIDIA Cosmos pretrained world foundation models is to produce realistic virtual environments for the development of physical AI.

The next AI revolution is physical AI. Physical AI models are used to power autonomous devices like robots and self-driving automobiles because they can comprehend instructions and observe, interact, and carry out complex operations in the real environment.

Physical AI models can comprehend the world and produce actions, much like large language models can analyze and produce text. This requires that these models be taught in simulation settings to understand geometric and spatial relationships, cause and effect, and physical dynamics like as gravity, friction, or inertia.

In order to create realistic virtual worlds, or digital twins, that can be used to train physical AI with previously unheard-of accuracy and detail, world leaders in software development and professional services are utilising NVIDIA Omniverse, powered by OpenUSD, to develop new products and services that will speed up the development of AI and controllable simulations.

Generate Exponentially More Synthetic Data With the Omniverse and NVIDIA Cosmos

In order to further integrate the Omniverse into physical AI applications like robots, autonomous vehicles, and vision AI, NVIDIA unveiled generative AI models and blueprints at CES.

NVIDIA Cosmos, a platform of cutting-edge tokenizers, guardrails, accelerated video processing pipeline, and generative world foundation models, was one of these announcements. Its goal is to speed up the development of physical AI.

NVIDIA Cosmos
Image Credit To NVIDIA

The process of creating physical AI models is expensive, time-consuming, and resource-intensive, requiring a great deal of testing and real-world data. Developers can easily create vast volumes of photoreal, physics-based synthetic data to train and assess AI for robotics, autonomous vehicles, and machines with Cosmos’ world foundation models (WFM), which forecast future world states as movies based on multimodal inputs. Additionally, developers can enhance the quality and efficiency of Cosmos WFMs for particular physical AI use cases or refine them to create downstream world models.

Cosmos and Omniverse combine to form a potent artificial data multiplication engine. After creating 3D situations in Omniverse, developers can input the outputs into Cosmos to produce controlled films and variations. By producing exponentially more training data encompassing a range of surroundings and interactions, this can significantly speed up the development of physical AI systems, such as robots and driverless cars.

By ensuring that the data in these situations is consistently represented and easily integrated, OpenUSD improves the simulations’ realism and efficacy.

Among the first to use Cosmos are prominent robotics and automotive businesses, such as 1X, Agile Robots, Agility Robotics, Figure AI, Foretellix, Fourier, Galbot, Hillbot, IntBot, Neura Robotics, Skild AI, Virtual Incision, Waabi, and XPENG, as well as the massive ridesharing company Uber.

Listen to the NVIDIA AI Podcast episode with Ming-Yu Liu, vice president of research at NVIDIA, to find out more about how world foundation models will progress physical AI.

See Cosmos in Action for Physical AI Use Cases

By offering a uniform platform for creating, honing, and implementing large-scale AI models across several applications, Cosmos WFMs are transforming industries. Generative physical AI and simulation can be used by businesses in the automotive, industrial, and robotics industries to boost innovation and operational effectiveness.

  • Humanoid robots: To train humanoid robots through imitation learning, developers can create enormous synthetic motion datasets with the aid of the NVIDIA Isaac GR00T Blueprint for synthetic motion synthesis. By using GR00T processes, users can record human behaviour and use Cosmos to greatly expand the dataset’s size and diversity, strengthening it for use in training real-world AI systems.
  • Autonomous vehicles: To speed up their pipelines, AV engineers can replay driving data, create new ground-truth data, and conduct closed-loop testing using the Omniverse Sensor RTX application programming interfaces for autonomous vehicle (AV) simulation. Cosmos speeds up the creation of physical AI models for autonomous cars by enabling developers to create artificial driving scenarios that amplify training data by orders of magnitude. NVIDIA and Uber, the world’s largest ridesharing company, are collaborating to speed up autonomous mobility. When paired with Cosmos and NVIDIA DGX Cloud, rich driving statistics from Uber can assist AV partners in more effectively developing more robust AI models.
  • Industrial environments: Before being deployed in factories and warehouses, Mega is an Omniverse Blueprint for creating, testing, and refining physical AI and robot fleets at scale in a USD-based digital duplicate. The blueprint enables high-fidelity sensor simulation at scale by rendering multisensor data from any kind of intelligent machine simultaneously using the Omniverse Cloud Sensor RTX APIs. By creating artificial edge case scenarios to increase training data, Cosmos can improve Mega and greatly increase the reliability and effectiveness of robot training in simulation. One of the first companies to employ Mega to push warehouse automation in retail, consumer packaged goods, parcel services, and other areas was KION Group, a supply chain solutions firm.

Get Plugged Into the World of OpenUSD

Watch the clip of NVIDIA CEO Jensen Huang’s CES talk to learn more about Cosmos, and start using the Cosmos WFMs that are currently available on Hugging Face and the NVIDIA NGC catalogue under an open model license. To learn more about Cosmos WFMs and physical AI workflows, tune in to the livestream on Wednesday, February 5.

The NVIDIA Deep Learning Institute has released a free self-paced Learn OpenUSD curriculum for 3D developers and practitioners that will help you continue to optimize OpenUSD workflows.

Drakshi
Drakshi
Since June 2023, Drakshi has been writing articles of Artificial Intelligence for govindhtech. She was a postgraduate in business administration. She was an enthusiast of Artificial Intelligence.
RELATED ARTICLES

Recent Posts

Popular Post

Govindhtech.com Would you like to receive notifications on latest updates? No Yes