Changes confirmed high confidence

NVIDIA's Audex-30B-A3B unifies audio and text in one decoder without regressing text intelligence

A 30B/3B-active hybrid Mamba-Transformer MoE handles ASR, translation, TTS, audio generation and speech-to-speech while preserving its text backbone's reasoning — but under a non-commercial license.

NVIDIA released Nemotron-Labs-Audex-30B-A3B on July 6, 2026, a unified audio-text language model whose single decoder spans speech recognition, translation, text-to-speech, audio generation and speech-to-speech generation — while, the authors report, preserving the text intelligence of its backbone with marginal or no regression. The release was published as an arXiv paper with model checkpoints, and covered by MarkTechPost.

Context

Multimodal expansion usually carries a tax: models fine-tuned into audio or vision often lose capability on their original text tasks. That tradeoff has long forced practitioners to run separate text and speech stacks in production. Audex targets the tradeoff directly, building on NVIDIA's Nemotron-Cascade-2 backbone with a hybrid Mamba-Transformer mixture-of-experts architecture — 30B total parameters with about 3B active per token.

What changed

Per the arXiv abstract: "Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research."

Key details from the paper and coverage:

Why it matters

If the no-regression claim holds up under independent testing, a single small-footprint decoder could replace chained ASR-to-LLM-to-TTS pipelines, cutting latency, deployment complexity and the compounding-error risk that plagues cascaded voice agents. The 3B-active parameter count puts the model within reach of single-GPU deployment, which matters for on-device and latency-sensitive speech applications. The research signal is broader than one model: it suggests modality unification and capability preservation are not mutually exclusive at this scale.

The license, however, draws a clear line between this and the month's permissive releases. Audex checkpoints are non-commercial — a research artifact, not an enterprise building block. That contrasts with the permissive licensing trend elsewhere in July's open-weight wave: Z.ai's GLM-5.2 — a 753B-total / ~40B-active MoE per the Hugging Face model card (some Z.ai materials list 744B) — ships under MIT, and multiple Chinese labs shipped Apache-2.0 flagships. Teams building commercial voice products cannot simply adopt Audex weights; they would need a license conversation with NVIDIA or a permissively licensed alternative.

Details

The unified-decoder design means the same weights serve understanding and generation in both modalities, rather than bolting a speech encoder and vocoder onto a text model. Reported text-side numbers (91.2% AIME 2025, 86.4 MMLU-Redux) are meant to evidence the no-regression claim against the backbone; the 6.82 average WER on OpenASR is the headline speech-recognition figure. Audex-2B offers a lighter entry point for experimentation. All benchmark numbers are vendor-reported from the paper; no independent evaluation had appeared at publication.

The training mix — 157.4B audio tokens against 320.5B text tokens — is itself instructive: the recipe keeps text data dominant, which is consistent with the paper's capability-preservation thesis. The hybrid Mamba-Transformer design, increasingly common in NVIDIA's Nemotron-Labs line, trades some transformer uniformity for inference efficiency at long sequences, a property that matters disproportionately for audio, where tokenized speech runs far longer than equivalent text. The release also fits a wider July pattern of architecture diversification — the same week, NVIDIA published a tri-mode autoregressive/diffusion language model family, and ICML's top awards went to diffusion research — suggesting the industry is actively exploring alternatives to pure left-to-right decoding.

Limitations and caveats

All performance figures are first-party results from the NVIDIA paper; independent reproduction was pending at publication. The non-commercial license sharply limits enterprise and product use despite the open checkpoints. "State-of-the-art" claims in the abstract are the authors' own characterization. This report relies on the arXiv abstract and trade coverage; the full paper's eval methodology was not independently audited.

Sources

*Update note: This post was last reviewed on 2026-07-22. Watch for independent speech and reasoning evaluations and any change to the checkpoint license terms.*

Sources

Drafted with AI assistance from source briefs; reviewed for citation completeness and label accuracy.