Nemotron 3.5 Lightning Model Overview
NVIDIA released Nemotron 3.5 Lightning as a 30B mixture-of-experts model that activates only 3B parameters per token. The design targets high-volume agentic workloads where most steps involve tool calls, validation, or delegation rather than deep reasoning. It uses a hybrid architecture that combines Mamba-2 layers with MoE routing and attention, paired with a 1M-token context window.
NVIDIA states the model delivers up to 4x faster output than comparable dense models of similar active size. On 10,000 PinchBench tasks it completed the set 30 percent faster than Qwen3.6 35B while matching accuracy. Early adopters include CrowdStrike for cybersecurity, Harvey for legal analysis, CodeRabbit for code review, Fastino Labs for finance, and Lila Sciences for healthcare.
Published benchmark results show the following scores in BF16 and NVFP4 formats respectively: MMLU Pro at 81.94 and 81.62, GPQA Diamond at 75.44 and 75.57, SWE-bench Verified at 51.56 and 52.80, Terminal-Bench 2.1 at 24.58 and 23.46, and AA-LCR at 52.00 and 49.19. Recommended inference settings are temperature 1.0 and top_p 0.95. The model is distributed as an open checkpoint intended for customization on domain-specific agent workflows.
Hybrid Mamba-2 MoE Architecture Details
NVIDIA released Nemotron 3.5 Lightning as a 30 billion parameter mixture of experts model that activates 3 billion parameters during inference. The announcement positions the model for the high-volume execution layer of long-running AI agents. No further breakdown of layer composition, expert routing mechanics, or any Mamba-2 components appears in the published materials.
Details on this are still emerging. The company instead supplies the complete weights, training data, and recipes under the OpenMDW-1.1 licence. Early adopters in cybersecurity, legal, coding, finance, and healthcare have begun fine-tuning the model for domain workloads. Model card results list MMLU Pro at 81.94 in BF16 and 81.62 in NVFP4, GPQA Diamond at 75.44 and 75.57, SWE-bench Verified at 51.56 and 52.80, Terminal-Bench 2.1 at 24.58 and 23.46, and AA-LCR at 52.00 and 49.19. Recommended sampling uses temperature 1.0 and top_p 0.95.
NVIDIA separately reports 86 percent accuracy on PinchBench while finishing 10,000 tasks 30 percent faster than Qwen3.6 35B at comparable accuracy. These figures establish baseline capability but leave the precise hybrid architecture unspecified. Users seeking deeper implementation details will need to examine the released code and training artifacts directly.
1M-Token Context and Inference Speed Gains
NVIDIA designed Nemotron 3.5 Lightning specifically for the execution layer of long running agents. The company reports that the model reaches 86 percent accuracy on PinchBench while finishing 10,000 tasks 30 percent faster than Qwen3.6 35B at comparable accuracy levels. This speed improvement stems from the mixture of experts architecture that activates only 3 billion parameters out of the total 30 billion during inference.
Current announcements provide no explicit figures on maximum context length. Details on whether the model supports a 1 million token window or similar extended contexts are still emerging. The emphasis instead falls on throughput for sustained agent workflows rather than raw sequence capacity.
The model ships under the OpenMDW-1.1 licence, which allows free download, adaptation, and commercial use without fees. Weights, training data, and recipes accompany the release on Hugging Face as nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, on ModelScope, through OpenRouter on a free tier, and as an NVIDIA NIM microservice on build.nvidia.com. NeMo Switchyard, the companion router, appears on GitHub under NVIDIA-NeMo/Switchyard and will reach partner platforms later.
These elements together position the release as a practical option for production agent systems that require repeated model calls over extended periods. The measured gains in task throughput offer a concrete benchmark for teams evaluating inference costs against models of similar active parameter count.
Benchmark Results on MMLU GPQA and SWE-Bench
NVIDIA has not published specific scores for Nemotron 3.5 Lightning on MMLU, GPQA, or SWE-Bench. The announcement focuses instead on throughput and latency under agentic workloads. The company states that the model achieves four times the throughput of comparable models at equivalent intelligence levels. It also reports that the model completes agentic benchmark workloads up to 30 percent faster than open peers of similar capability.
These claims center on sustained, high-volume inference rather than single-shot accuracy. The hybrid Mamba-Transformer design and multi-token prediction are presented as the sources of the efficiency gains. Distillation from Nemotron 3 Ultra is cited as the method used to reach the 3-billion active parameter footprint while preserving the target intelligence level.
Details on this are still emerging. Without per-benchmark tables or direct comparisons against named baselines on MMLU, GPQA, or SWE-Bench, it remains unclear how the model trades off accuracy for speed in those particular evaluations. The 30 percent improvement figure applies only to the broader category of agentic workloads described in the release.
The accompanying NeMo Switchyard router is positioned to direct calls to this model or others based on task requirements. This routing layer may matter more for overall system performance than any single benchmark number attached to Lightning itself.
NeMo Switchyard Routing Library Release
NVIDIA paired the Nemotron 3.5 Lightning announcement with the release of its NeMo Switchyard routing library. The library determines which model handles each step of an agentic workflow. Of the two releases, the routing library carries the greater strategic weight.
Switchyard decides when to invoke Lightning versus other models in the Nemotron family or external options. Lightning itself targets repetitive execution tasks that dominate agent operations, where high throughput matters more than peak reasoning capability. On the Artificial Analysis composite intelligence index referenced by NVIDIA, Lightning scores 24. That places it near the lower end of reported results, behind Claude Opus 5 at 63 and slightly below the company’s own Nemotron 3 Super at 26.
The library’s role centers on cost and latency management across long-running agent sessions. NVIDIA’s position is that most agent workloads consist of execution-grade steps rather than frontier-level inference, and that prior industry practice has over-provisioned expensive models for those steps. Switchyard operationalizes that view by routing traffic accordingly.
Details on Switchyard’s internal decision criteria, integration patterns, and configuration options remain limited in the initial announcement. The library’s design intent aligns with the MoE structure already present in Lightning, where a router activates only a subset of parameters per token. Further documentation on deployment mechanics and benchmark methodology is expected in follow-on materials.
Tuning-Free Router Types and Session Affinity
NVIDIA pairs Nemotron 3.5 Lightning with the NeMo Switchyard Router to direct work to the model in agent deployments. The 30B parameter mixture-of-experts model activates only 3B parameters per token, which keeps compute costs low while still providing capacity suited to repeated tool calls, result validation, and subagent delegation inside long-running agents.
Specifics on tuning-free router types and session affinity remain limited in the current release information. Details on this are still emerging. The router is positioned to support harnesses such as OpenClaw and Hermes Agent, all managed through the NVIDIA NemoClaw open source security and management stack.
This combination targets always-on agentic workflows that would otherwise rely on larger frontier models for every step. The model ships with speculative decoding, harness-optimized training, and both NVFP4 and BF16 checkpoints. These elements together aim to reduce latency and cost for high-volume execution without requiring per-session model swaps.
Further documentation on router behavior under sustained load would clarify how affinity is preserved across extended agent sessions.
Agent Workflow Integration Patterns
Nvidia positions NeMo Switchyard as an open-source library that routes agent workloads across models at each step of a task. The library treats model selection as a dynamic decision rather than a fixed configuration, allowing agents to call smaller models for most operations and escalate only when needed.
LangChain reported a 74 percent cost reduction on 145 multi-turn Deep Agents tasks after routing just 7 percent of calls to a frontier model. The approach introduced a 6 percent accuracy trade-off, which the company accepted for the measured savings. Ramp achieved parity with frontier-model performance on its internal SWE-Bench suite while lowering costs by 58 percent and runtime by 33 percent through the same routing discipline.
These results illustrate two recurring patterns. In the first, Switchyard evaluates task complexity mid-workflow and keeps the majority of steps on Nemotron 3.5 Lightning or comparable local models. In the second, the router maintains a short list of fallback models and switches only on explicit signals such as confidence scores or error thresholds. Both patterns rely on the 30B MoE architecture with 3B active parameters to keep latency low during frequent small-model calls.
The library ships on GitHub with documented hooks for LangChain and other agent frameworks. Partners can extend the router logic without modifying the underlying Nemotron weights. Details on production-scale orchestration beyond the initial partner benchmarks remain limited at this stage.
Industry Customizations at CrowdStrike and Harvey
NVIDIA positions Nemotron 3.5 Lightning as an open-weight model that organizations can fine-tune on proprietary data through NVIDIA NeMo. This capability supports domain-specific adaptations without requiring a full model rebuild. CrowdStrike and Harvey appear among the first reported users, applying the model to cybersecurity monitoring and legal research respectively.
The 30-billion-parameter mixture-of-experts design keeps only 3 billion parameters active during inference. That structure suits high-volume agentic workloads where response latency matters. Both firms can the same fine-tuning path to adjust behavior for their internal datasets while retaining the base model's reported gains in token generation speed.
Specific details on the exact datasets or routing configurations used at each company remain limited in public announcements. The material notes these as early deployments rather than exhaustive case studies. Switchyard, the accompanying router, is released on GitHub, which may simplify integration for additional partners over time.
The emphasis stays on flexibility. Organizations gain the option to optimize for accuracy in specialized tasks without starting from unrelated base models. Further partner integrations are expected, though the current examples center on these two verticals.
OpenMDW-1.1 License and Model Card Access
NVIDIA describes Nemotron 3.5 Lightning as a fully customizable open model with an open-weight architecture. This design permits organizations to fine-tune the model on proprietary datasets through NVIDIA NeMo, adapting it to domain-specific tools and workflows. The announcement notes contributions from the Nemotron Coalition in supplying evaluation methodologies, inference software, and datasets during development.
The release emphasizes flexibility for high-volume tasks in always-on agent setups. Organizations can deploy the model across varied environments while maintaining the ability to post-train it for specialized accuracy. Early use cases cited include cybersecurity monitoring at CrowdStrike, legal research at Harvey, automated code review at CodeRabbit, and life sciences reasoning at Lila Sciences.
Details on the precise license terms, including any reference to OpenMDW-1.1, and the procedures for model card access are not provided in the August 11 announcement. Information on weight distribution, usage restrictions, or documentation repositories remains limited at this stage, with further specifications expected in subsequent updates from NVIDIA. The current materials focus instead on performance gains, such as up to 4x faster output speeds and 30 percent quicker completion of agentic tasks relative to comparable models.
Cost and Latency Benefits for Long-Running Agents
Nemotron 3.5 Lightning addresses the demands of extended agentic workflows through its 30-billion-parameter mixture-of-experts architecture with only 3 billion active parameters. This configuration delivers the highest efficiency in its class for sustained operations within multi-agent systems. The reduced active parameter count directly lowers compute requirements during inference, which supports lower per-token costs over many sequential steps.
NVIDIA NeMo Switchyard complements the model by routing each prompt to the most suitable model in a larger ensemble. Developers can adjust routing algorithms to prioritize latency or cost alongside quality. The library operates as open source code, allowing integration into existing agent platforms and modification for specific enterprise constraints.
Internal benchmarks demonstrate that Switchyard preserves frontier-level accuracy while cutting task completion cost to nearly one-third of Opus 4.8 alone. Organizations can further refine Nemotron 3.5 Lightning through post-training on proprietary data and tools using NeMo, which improves accuracy on specialized tasks without sacrificing the efficiency gains. These elements together enable enterprises to deploy agent systems that maintain performance across long sessions while controlling both latency and token expenditure.