Nemotron 3 Ultra: Open 550B Agents at 1M Context

NVIDIA released Nemotron 3 Ultra on June 4, 2026 as a 550-billion-parameter mixture-of-experts model with 55 billion active parameters and a one-million-token context window. The official technical announcement pairs the weights with training data, recipes, reinforcement-learning environments, and evaluation assets—an unusually broad openness claim for a model at this scale.

The architecture combines Mamba and Transformer layers, LatentMoE routing, multi-token prediction, and NVFP4 quantization. NVIDIA reports up to five times the throughput of comparable open models and up to 30 percent lower task-completion cost on agent benchmarks. These figures come from NVIDIA or named external endpoints and depend strongly on Blackwell hardware, precision, batching, and the comparison pool.

Architecture in operational terms

The model stores 550B parameters but activates 55B for a token. LatentMoE is intended to reduce the communication and memory cost of selecting experts. Hybrid Mamba-Transformer layers give most of the sequence processing a recurrent, linear-time path while retaining attention where precise retrieval is useful.

The one-million-token window targets repository-scale and long-running agents. Multi-token prediction proposes more than one future token to increase decode speed. NVFP4 compresses weights and compute for Blackwell-class accelerators. Each mechanism addresses a different bottleneck: capacity, context, generation, or hardware throughput.

An architecture diagram does not establish end-to-end performance. Serving a 550B checkpoint still requires a substantial GPU system, optimized collectives, cache capacity, and compatible kernels. The active count should not be mistaken for total memory footprint.

Benchmark and systems snapshot

NVIDIA's public materials report a mix of accuracy and efficiency values:

These values should be consumed as a source-specific profile. Terminal-Bench 2.0 is not version 2.1. RULER measures controlled long-context behavior, not an entire million-token business workflow. Elo changes with the compared systems. Efficiency maxima do not all occur on one workload.

The openness package is unusually broad

NVIDIA describes open weights, pretraining and post-training data, recipes, software, and RL environments. OpenMDW 1.1 and related assets let teams inspect more of the development chain than a weight-only release. This supports audit, specialization, and reproducibility.

Every asset has its own license and provenance. “Open” is not one switch. A production team should inventory weights, datasets, code, containers, and third-party components separately. Training data may carry restrictions that differ from the model license.

The first-party model card should be pinned by revision and hash. Quantized checkpoints need their own acceptance results. The open recipe is valuable even when an organization cannot afford to pretrain or serve the full model.

Long context must earn its cost

A million tokens can hold a large codebase, long legal record, or extended agent trace. Hybrid sequence layers are designed to keep that context computationally feasible. NVIDIA reports strong RULER performance, which indicates retrieval remains effective in controlled tests.

Real workloads contain duplicated files, stale instructions, conflicting updates, and irrelevant logs. Test facts at different depths, changes late in the prompt, multi-file references, and compaction. Measure prefill time and cache memory. A nominal context that adds minutes before the first action may not be economical.

Long context also expands the prompt-injection surface. Treat retrieved files as untrusted data, separate instructions from content, and restrict tools. The model should cite which file and region support a decision.

Throughput and NVFP4

NVIDIA's five-times throughput claim reflects hardware-software co-design. NVFP4 reduces memory traffic and increases tensor-core utilization on Blackwell. A later NVIDIA quantization report says an NVFP4 checkpoint can reach up to 5.9x the inference throughput of a GLM-5.1 FP4 comparison in decode-heavy runs while largely matching BF16 accuracy.

That result cannot be transferred to H100, Apple silicon, or arbitrary serving engines. Quantization sensitivity differs by task and language. Long-context prefill may have a different bottleneck from decode. Production tests should plot accuracy against throughput across batch and context lengths.

Task cost is even more useful than tokens per second. NVIDIA reports up to 30 percent savings on selected agent completions because Nemotron uses fewer total tokens and fewer tokens per turn. Confirm this using the same task, acceptance rubric, and infrastructure pricing.

Agent specialization and transparent data

Nemotron is designed as a base for domain agents. NVIDIA uses multi-teacher on-policy distillation, drawing feedback from more than ten domain-specific teachers. The open data and recipes allow organizations to extend the model for legal, engineering, or enterprise workflows.

Specialization can improve utility and introduce blind spots. Fine-tuning should preserve a general regression suite and safety tests. Synthetic data needs lineage, filtering, and quality rubrics. A domain score can rise while instruction following or multilingual performance falls.

The release's later RTL-coding work illustrates the potential: a specialized agent reported 97.1 percent across Verilog categories. That is evidence about the combined ACE-RTL system and tuned workflow, not the raw June checkpoint alone.

Deployment reality

A 550B model is not a desktop release. Organizations need a distributed serving plan, high-bandwidth networking, model-parallel strategy, failure recovery, and observability. The NVFP4 checkpoint lowers cost but still represents serious infrastructure.

Start by reproducing a small reference evaluation in BF16 or a trusted high-precision endpoint. Then test NVFP4 at target concurrency and context. Record throughput per user, time to first token, power, task completion, and error variance. Pin CUDA, drivers, inference engine, and quantization toolchain.

The open-model operations guide provides the artifact and rollout controls; our frontier evaluation guide provides the comparison manifest. Both are necessary at this scale.

Best fit and verdict

Nemotron 3 Ultra is best suited to organizations building private, customizable, long-running agents with access to substantial NVIDIA infrastructure. It is also a research asset for teams studying open post-training, long context, and efficient sparse serving. Smaller models will be more economical for routine work.

Procurement should also distinguish the research value of the open package from the runtime value of the checkpoint. A team may use the data, environments, or recipes to improve a smaller internal model even when serving 550B is impractical. That reuse is a concrete benefit that ordinary hosted-model benchmarks do not measure.

The release deserves major coverage because openness extends beyond weights and because the architecture is designed around task cost rather than parameter spectacle. The benchmark table supports a serious pilot but does not establish universal leadership. The strongest decision evidence will come from reproducing quality and cost on the exact hardware, quantization, and agent loop intended for production.

Source notes — reviewed 2026

• NVIDIA's Nemotron 3 Ultra technical launch supplies the date, architecture, openness, and efficiency claims.
• NVIDIA Research's Nemotron 3 Ultra page records benchmark and architecture context.
• The official BF16 model card is the first-party checkpoint and specification source.
• NVIDIA's NVFP4 checkpoint report documents quantization methodology and throughput claims.
• The official NVIDIA NeMo repository provides the training and deployment software lineage.