The Token Trade-Off Behind Nemotron 3.5 Bolt’s Compact Size

🔍 Read the full analysis: The Token Trade-Off Behind Nemotron 3.5 Bolt’s Compact Size on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get smart everyday buys delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

NVIDIA released Nemotron 3.5 Bolt 74B-A9B on October 15, an open-weight model that scores 21.4 on Artificial Analysis’ Intelligence Index v4.3.2, close to the 22.9 recorded by NVIDIA’s 550-billion-parameter Nemotron 3 Ultra. But the benchmark comparison comes with a notable cost: Artificial Analysis reports that Bolt generated about 82,000 output tokens per index task, nearly three times Ultra’s total, while ranking Bolt fourth among models with fewer than 100 billion total parameters.

Bolt is a mixture-of-experts model with 74 billion total parameters and 9 billion active for each token, according to the source report. It accepts text and images, produces text, supports reasoning and has a 256,000-token context window. NVIDIA released its weights for self-hosting. Artificial Analysis evaluated the reasoning version.

The score puts Bolt 1.5 points below Nemotron 3 Ultra, despite Bolt having about one-seventh as many total parameters and one-sixth as many active parameters. NVIDIA’s Nemotron 3 Super, released in March with 120 billion total and 12 billion active parameters, scored 12.8 on the same index, according to the report. These index scores summarize performance across evaluated tasks; they do not establish that the models perform equally well in every use case.

Bolt’s results vary by task. The report says it scored 15.5% on AutomationBench-AA, compared with 3.0% for Ultra, and 912 versus 876 Elo on AA-Briefcase. Ultra scored higher on long-context evaluation AA-LCR, 79% to Bolt’s 70%, and on several knowledge-oriented tests. Both models scored zero on Terminal-Bench 4.0, according to the supplied results. The report also gives Bolt a score of −19 on AA-Omniscience, which penalizes confident incorrect answers; the supplied material does not include a full explanation of that result.

Bolt’s Trade-Off: Compute for Time

The headline score makes Bolt relevant to organizations looking for an open-weight model they can host themselves, particularly those that need image input or want to document a model’s origin. Artificial Analysis ranks it first among US models in its under-100-billion-parameter class, according to the report. That is a narrower distinction than leading the class overall: the report ranks Bolt fourth, behind Qwen3.8 27B at 33.7, K2 Horizon MoVA 36B-A4B at 25.3 and G9v3-39A5B at 21.8.

The token count affects how useful the benchmark result may be in practice. Artificial Analysis reports about 82,000 output tokens per index task, against 28,000 for Nemotron 3 Ultra. More generated tokens can mean longer waits when running a model locally. The source offers illustrative arithmetic, not measured Bolt speeds: at a sustained 100 tokens per second, 82,000 tokens would take about 14 minutes; at 50 tokens per second, about 27 minutes. Actual throughput depends on hardware, software and workload.

That leaves a practical divide between batch work and interactive use. Long-running tasks that can be queued may tolerate extensive reasoning output; a person waiting for an answer may not. Bolt’s 9-billion active-parameter configuration suggests a potential compute advantage per generated token compared with a much larger dense model, but no measured speed is supplied to show whether that advantage offsets the output length.

How Bolt Compares Within Its Class

Bolt was one of two NVIDIA open-weight releases on October 15, 2026. The other, Nemotron 3.5 Super VL, drew more attention in the source report, which describes it as the highest-scoring permissively licensed open-weight model from the US. Bolt is the first model in NVIDIA’s new Bolt family, according to the same report.

The class comparison changes how to read Bolt’s near-flagship score. Qwen3.8 27B leads the listed under-100-billion-parameter models with 33.7, while K2 Horizon MoVA scores 25.3. The report says MoVA uses half Bolt’s total parameters and fewer than half its active parameters, and produces fewer tokens per evaluated task. Bolt’s distinguishing features in the comparison include image input and its position as the highest-ranked US model in that size class, not the class’s top overall score.

Artificial Analysis has no hosted API provider listed for Bolt on the model page described in the report. That means the source provides no API price, speed measurement or cost per task. It also says the license is categorized as permissive, but does not give the license name. Those details matter to organizations weighing deployment, redistribution and operating costs.

““For batch jobs that run overnight, that is fine. For anything a person waits for, it is not.””

— Thorsten Meyer, ThorstenMeyerAI.com

Speed, Cost and Knowledge Gaps

Real-world throughput has not been reported in the supplied material. The token-rate examples are calculations based on assumed speeds, not tests of Bolt on particular hardware. Without measured throughput, hardware details or a hosted API price, users cannot derive a reliable wait time or cost per task from the benchmark output count alone.

The report also leaves open why Bolt’s output is so long, how much comes from its reasoning configuration, and whether shorter-answer settings would preserve its evaluated performance. Its AA-Omniscience score of −19 is reported without enough detail here to determine which errors drove it or how the result translates into specific applications. The exact license name is also not identified.

Benchmark rankings and results reflect the tests and configurations cited, not a guarantee of performance in a particular deployment. It is also unclear from the supplied material whether NVIDIA or Artificial Analysis will publish additional evaluations, including direct speed comparisons or results for other Bolt configurations.

What Deployment Tests Must Show

The next useful evidence will be independent measurements of Bolt’s speed and output length on specified hardware, alongside comparable tests of Nemotron 3 Ultra and other models in the class. Such results could show whether Bolt’s lower active-parameter count translates into faster local inference despite its higher token use.

Organizations considering the model will also need the license terms, deployment requirements and workload-specific evaluations. In particular, agent developers should test multi-step tool workflows, while teams handling long documents or knowledge-heavy tasks should examine the weaker results cited in the report. Until pricing and throughput are available, the cost and responsiveness of a real deployment remain open questions.

Key Questions

What did NVIDIA release?

NVIDIA released Nemotron 3.5 Bolt 74B-A9B on October 15, 2026. The open-weight mixture-of-experts model has 74 billion total parameters, 9 billion active per token, and accepts text and images.

How close is Bolt to Nemotron 3 Ultra?

Artificial Analysis’ Intelligence Index v4.3.2 gives Bolt 21.4 and Nemotron 3 Ultra 22.9, a 1.5-point difference. The overall scores do not mean the models match on every task.

Is Bolt the highest-ranked model under 100 billion parameters?

No. The source report places Bolt fourth in that class and first among US models in the class. It lists Qwen3.8 27B, K2 Horizon MoVA 36B-A4B and G9v3-39A5B ahead of it.

Why does Bolt’s output-token count matter?

Artificial Analysis reports about 82,000 output tokens per index task. More output can increase local inference time, but actual speed depends on hardware and measured throughput, which the supplied report does not provide.

Can users access Bolt through a hosted API?

The report says Artificial Analysis lists no hosted API provider for Bolt on the model page it reviewed. It therefore gives no hosted price or cost per task; the weights are available for self-hosting, according to the source.

Source: ThorstenMeyerAI.com