NVIDIA's next-generation rack-scale system made its benchmark debut on September 16, and the headline multiplier is large: Vera Rubin NVL72 posted up to 3.7x the throughput of the shipping GB300 NVL72 on the Qwen3-VL workload in MLPerf Inference v6.1. The asterisk matters as much as the number. Vera Rubin entered under MLCommons' preview category, which covers hardware that is not yet generally available, so the result is a forward-looking signal rather than something a buyer can order today.
Key takeaways
- Vera Rubin NVL72 delivered up to 3.7x GB300 NVL72 throughput on Qwen3-VL and up to 2.5x on DeepSeek-R1, entering MLPerf as a preview-category system rather than an available one.
- A 288-GPU GB300 NVL72 submission spanning four racks reached 99% scaling efficiency in the offline DeepSeek-R1 scenario, with throughput growing nearly linearly from a single-rack baseline.
- Software optimization alone lifted GB300 Qwen3-VL results up to 1.6x over MLPerf v6.0 six months earlier, on unchanged silicon.
What Vera Rubin actually submitted
NVIDIA confined its preview entry to two of the round's heaviest workloads. On Qwen3-VL it ran vLLM with the company's open-source Dynamo serving framework and reported up to 3.7x GB300 throughput across the offline, server and interactive scenarios. On DeepSeek-R1 it switched to the TensorRT-LLM library and landed at up to 2.5x. The narrow entry is itself informative: a preview submission is a choice about where a platform looks strongest, and two reasoning-and-vision workloads are where rack-scale interconnect pays off most.
The company attributes the jump to co-design rather than raw clock speed. Rubin's revised Tensor Cores and Transformer Engine accelerate both the prefill and decode halves of inference, while NVFP4 quantization shrinks the memory footprint of weights, attention and the KV cache. Both submissions leaned on disaggregated serving β splitting prefill from decode onto separate resources β plus wide expert parallelism across the mixture-of-experts layers that DeepSeek-R1 and Qwen3-VL both use.
Cloud provider Nebius filed its own Vera Rubin NVL72 preview results in the same round, an early sign of how far outside NVIDIA's walls the hardware has already travelled. NVIDIA also submitted Jetson AGX Thor results on the round's brand-new Edge-Agentic test, running Qwen3.6-27B through TensorRT Edge-LLM.
Why 99% scaling matters more than peak throughput
The less quotable result may be the more useful one for anyone sizing a cluster. NVIDIA scaled a DeepSeek-R1 submission from one GB300 NVL72 rack of 72 GPUs to four racks of 288 GPUs and measured 99% scaling efficiency in the offline scenario.
That figure answers a question peak throughput cannot: whether the fourth rack earns its cost. Efficiency at that level means the marginal rack behaves almost like the first one, which is the assumption every capacity plan quietly depends on. Where scaling decays instead, the shortfall compounds with every rack added, and the operator pays for silicon that sits waiting on the network. NVIDIA credits its sixth-generation NVLink and NVLink Switch fabric, which it says carries 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet, for keeping the disaggregated serving techniques viable at rack scale. On the WAN 2.2 text-to-video test, the same rack-scale setup produced 0.65 720p videos per second at 5.7 seconds each, which NVIDIA measures as 9x the throughput of a single node.
Software is compounding as fast as silicon
Buried under the Rubin numbers is a result that costs nothing to adopt. GB300 NVL72 performance on Qwen3-VL improved up to 1.6x between v6.0 and v6.1, a six-month gap, through lower KV cache precision, additional kernel fusion, better kernels and disaggregated serving. The hardware did not change. For an operator who bought a GB300 rack last spring, that is a free capacity increase delivered entirely through the serving stack, and it reframes how a depreciation schedule should be read. NVIDIA says optimization continued past the submission deadline, with unverified post-submission gains on GPT-OSS-120B and DLRMv3.
One caveat belongs on a separate line. NVIDIA's claim of 30x better agentic performance than GB300 comes from SemiAnalysis AgentX preview testing, not from MLPerf, and has not been through MLCommons peer review.
What the wider v6.1 round shows
MLCommons called the round a participation record, with 30 submitting organizations including six first-timers, and roughly 486 datacenter and edge results across 120 systems. The suite added two tests: an end-to-end RAG pipeline spanning embedding, retrieval, re-ranking and generation, and the Edge Agentic Inference test aimed at multi-turn coding workloads on constrained devices.
MLCommons also reported that the best per-accelerator DeepSeek-R1 server result is now 5.7x better than a year ago. That compounding is the backdrop to the buildout that has pushed hyperscalers into orders like Amazon's tripling of its NVIDIA commitment to two million GPUs.
Outlook
Two shifts are worth tracking. More than half of v6.1 submitters used MLCommons' new API-centric harness, the foundation of MLPerf Endpoints, which the consortium says will replace Inference for datacenter benchmarking. And Vera Rubin's numbers stay provisional until the platform moves out of preview into the available category, where competitors get to submit against it on equal terms. Until then the honest reading of a 3.7x preview figure is a direction of travel, not a purchase decision β and the 1.6x that arrived in a software update is the part already on the table.
FAQ
Is Vera Rubin NVL72 available to buy?
Not yet. It was submitted under MLPerf's preview category, which MLCommons reserves for systems that are not generally available at submission time. Preview results are peer-reviewed but are not directly comparable to available-category entries that customers can deploy now.
Does the 3.7x figure apply to every workload?
No. It is the best case on Qwen3-VL across offline, server and interactive scenarios. On DeepSeek-R1, using TensorRT-LLM instead of vLLM with Dynamo, the gain over GB300 NVL72 was up to 2.5x. NVIDIA submitted Vera Rubin results on only those two benchmarks.
What is new in MLPerf Inference v6.1?
Two benchmarks joined the suite: an end-to-end RAG test covering a full retrieval and generation pipeline, and an Edge Agentic Inference test for multi-turn agentic coding on edge devices. The round also added speculative decoding support in the interactive scenario for several tasks.
