NVIDIA Launches New NIM Microservices for Robotics Simulation in Isaac Lab and Isaac Sim, OSMO NVIDIA Robot Cloud Compute Orchestration Service, Teleoperated Data Capture Workflow, and More, Advancing the Development of NVIDIA Humanoid Robotics.
Humanoid Robot
NVIDIA today announced that in an effort to spur worldwide progress in humanoid robots, it is offering a range of services, models, and computing platforms to the top robot makers, AI model developers, and software developers in the world. These resources will be used to design, develop, and construct the next generation of humanoid robotics.
Among the products are the OSMO NVIDIA orchestration service for managing multi-stage robotics workloads, the new NVIDIA NIM microservices and frameworks for robot simulation and learning, and an AI- and simulation-enabled teleportation workflow that enables developers to train robots with minimal human demonstration data.
Humanoid Robot Development
Quickening the Process of Development NVIDIA NIM and OSMO NVIDIA NIM microservices enable developers to cut down on deployment times from weeks to minutes by offering pre-built containers that are driven by NVIDIA inference software. NVIDIA Isaac Sim is a reference application for robotics simulation based on the NVIDIA Omniverse platform. With the help of two new AI microservices, robotics will be able to improve simulation processes for generative physical AI.
Synthetic motion data is produced by the MimicGen NIM microservice using teleoperated data that has been captured by spatial computing devices such as Apple Vision Pro. The Robocasa NIM microservice in OpenUSD, a general-purpose framework for creating and interacting in 3D environments, creates robot tasks and simulation-ready environments.
With the help of distributed computing resources, either on-premises or in the cloud, customers can coordinate and scale complicated robotics development workflows with OSMO NVIDIA, a managed service that is currently accessible in the cloud.
NVIDIA OSMO
Robot training and simulation workflows are greatly streamlined by OSMO NVIDIA, which reduces deployment and development cycle duration’s from several months to a few days. A variety of tasks, such as creating synthetic data, training models, performing reinforcement learning, and putting software-in-the-loop testing into practice at scale for industrial manipulators, autonomous mobile robots, and humanoids, may be visualised and managed by users.
Humanoid Robots
Enhancing Data Capture Processes for Developers of Humanoid Robots
The amount of data needed to train foundation models for humanoid robots is enormous. Teleportation is one method of gathering data about human demonstration, but it’s getting more and more expensive.
At the SIGGRAPH computer graphics conference, an NVIDIA AI– and Omniverse-enabled teleportation reference workflow was presented. It enables academics and AI developers to produce vast quantities of synthetic motion and perception data from a small number of remotely recorded human demonstrations.
First, a limited number of teleoperated demonstrations are recorded by developers using Apple Vision Pro. Next, they use NVIDIA Isaac Sim to simulate the recordings, and the MimicGen NIM microservice is used to create synthetic datasets from the recordings.
By using both synthetic and actual data to train the Project GR00T humanoid foundation model, developers are able to minimise expenses and time. They then create experiences to retrain the robot model using the Robocasa NIM microservice in Isaac Lab, a robot learning platform. OSMO NVIDIA sends computing jobs to various resources in a smooth manner throughout the workflow, sparing the developer weeks’ worth of administrative effort.
The business Fourier, which makes all-purpose robot platforms, recognises the advantages of creating artificial training data through simulation technology.
NVIDIA Humanoid Robots
Increasing NVIDIA Humanoid Developer Technologies’ Access
To facilitate the development of humanoid robotics, NVIDIA offers three computing platforms: NVIDIA Jetson Thor humanoid robot processors, which are used to run the models; NVIDIA AI supercomputers for training the models; and NVIDIA Isaac Sim, which is based on Omniverse and allows robots to learn and hone their skills in virtual environments. For their unique requirements, developers have access to and can use all or any portion of the platforms.
Developers can get early access to the new products as well as the most recent iterations of NVIDIA Isaac Sim, NVIDIA Isaac Lab, Jetson Thor, and Project GR00T general-purpose humanoid foundation models through a new NVIDIA Humanoid Robot Developer Program.
Among the first companies to sign up for the early-access program are 1x, Boston Dynamics, ByteDance Research, Field AI, Figure, Fourier, Galbot, LimX Dynamics, Mentee, Neura Robotics, RobotEra, and Skild AI.
Availability
Developers will soon get access to NVIDIA NIM microservices and can access OSMO NVIDIA and Isaac Lab by enrolling in the NVIDIA Humanoid Robot Developer Program.
Project GR00T
Advantages
Early Access to Models of the Humanoid Foundation: A collection of foundation models for humanoid robots is called Project GR00T. These models allow robots to mimic human gestures, comprehend natural language, and learn new skills quickly thanks to NVIDIA-accelerated training and mult-modal learning.
Utilise OSMO NVIDIA Managed Service for Free: For scaling complicated, multi-stage, and multi-container robotics workloads across on-premises, private, and public clouds, OSMO NVIDIA is a cloud-native orchestration platform.
First Access to the New ROS Libraries for NVIDIA Isaac: A collection of NVIDIA GPU-accelerated ROS 2 libraries called Isaac ROS helps to speed up the creation and functionality of AI robots.
Early Access to Simulation and Learning Frameworks for Robots: Robots can learn through imitation and reinforcement using the simulation program Isaac Lab.
NVIDIA’s Generalist Robot 00 Technology
NVIDIA GR00T Project
The goal of Project GR00T is to provide a general-purpose foundation model for humanoid robots that may be used to generate robot actions based on past interactions and multimodal instructions. This sophisticated model is modular, containing systems for low-level quick, precise, and responsive motion as well as high-level planning and reasoning.
NVIDIA’S Project GROOT
- Research on Project GROOT is currently underway. To receive updates and availability information, please sign up below.
- NVIDIA-Powered Next-Gen Robots: Creating Robotic Systems Using Three Computers
- Every component of NVIDIA’s three-computer robotics stack is used in Project GR00T.
- This incorporates NVIDIA Jetson Thor and Isaac ROS for rapid robot runtime, NVIDIA AI and DGX for model training, and NVIDIA Isaac Lab for reinforcement learning.
DGX from NVIDIA
NVIDIA DGX Cloud is a comprehensive AI platform designed for developers that offers scalable capacity based on the most recent NVIDIA architecture, co-engineered with top cloud service providers globally.
Isaac ROS
NVIDIA Isaac Laboratory
This small reference application is essential for training robot foundation models and is based on the NVIDIA Omniverse platform, which is specifically optimised for robot learning. It can train any kind of robot embodiment and optimises for imitation, transfer learning, and reinforcement.
NVIDIA Isaac ROS on Jetson Thor is a set of AI models and accelerated computation packages intended to simplify and accelerate the creation of cutting-edge AI robotics applications. Isaac ROS, which is designed for quick humanoid thought and movement, is used on Jetson Thor for humanoids.
How It Operates
Function of the Model
Humanoid embodiment’s may execute commands from multi-modal instructions and respond to their surroundings in real time by self-observing thanks to Project GR00T.
Growth and Instruction
The methodology speeds up the development of humanoid robots by using multi-modal learning from a range of data sources, such as instructions, videos, demonstrations, and imitation learning.
For large-scale Project GR00T training, NVIDIA Isaac Lab is the best choice. Moreover, NVIDIA OVX systems for simulation, NVIDIA IGX systems for hardware-in-the-loop validation, and NVIDIA DGX systems for training can all have their workflows coordinated by using the OSMO NVIDIA robot cloud computing orchestration service.
