Nvidia Alpamayo is a proprietary reasoning model designed for autonomous driving applications. The model is built on the open-source Cosmos 2.5 Reason foundation and subsequently post-trained for self-driving vehicle stacks. During development and testing at headquarters, the real-time autonomous driving stack runs on a cluster of 16 VR Ruben systems.
—
HDMI® Technology is the foundation for the worldwide ecosystem of HDMI-connected devices; integrated with displays, set-top boxes, laptops, audio video receivers and other product types. Because of this global usage, manufacturers, resellers, integrators and consumers must be assured that their HDMI® products work seamlessly together and deliver the best possible performance by sourcing products from licensed HDMI Adopters or authorized resellers. For HDMI Cables, consumers can look for the official HDMI® Cable Certification Labels on packaging. Innovation continues with the latest HDMI 2.2 Specification that supports higher 96Gbps bandwidth and next-gen HDMI Fixed Rate Link technology to provide optimal audio and video for a wide range of device applications. Higher resolutions and refresh rates are supported, including up to 12K@120 and 16K@60. Additionally, more high-quality options are supported, including uncompressed full chroma formats such as 8K@60/4:4:4 and 4K@240/4:4:4 at 10-bit and 12-bit color.
—
While the Alpamayo model can fit onto smaller graphics processing units such as the GB300, executing the full simulation and autonomous driving stack in real time currently requires high-performance hardware. The high computing demand during testing stems from simulating multiple camera views and generating the surrounding environment in real time. Once the system is deployed on production physical vehicles, only the Alpamayo reasoning model is executed, eliminating the need for real-time world generation on the vehicle itself.
Nvidia Alpamayo is currently deployed on test vehicles at the company headquarters. The autonomous driving model is designed to scale down to smaller chips for commercial vehicles. To train the autonomous model, the system uses simulated environments to accumulate driving experience. This training process is supported by Omni Dreams, a world-generation model built on the open-source Cosmos 2.5 Predict model.
Omni Dreams generates synthetic environment data to train the autonomous driving model under diverse road conditions, including rain, snow, and varying traffic densities. The simulation runs locally on a DGX Station using the Alpasim platform. This local simulation generates virtual environments in real time, allowing test vehicles to navigate maps without collision constraints, thereby augmenting physical road testing.
The core models Alpamayo and Omni Dreams remain proprietary. However, their underlying backbone models, Cosmos 2.5 and Cosmos 3, are open source. This allows developers to utilize the open backbone models to train custom autonomous systems for specific deployment requirements. Nvidia is currently updating its autonomous driving stack to leverage the recently released Cosmos 3 model.
