Diffusion LLMs and the Main Types of AI Models

Most AI chatbots write the way you type on your phone: one word after another, left to right. A diffusion LLM works differently. It starts with a rough, mostly blank draft of the whole answer and then cleans up many words at the same time over a handful of passes. The approach is getting more attention: today (October 19) Inception launched Mercury 3, which it calls "the most capable diffusion-based LLM released to date."

Below we cover how diffusion LLMs work, what you can actually try, and where they sit among the other kinds of language model you'll keep hearing about.

What a diffusion LLM actually does

A standard model is autoregressive. It predicts the next token (roughly a word piece), adds it to the text, and repeats. As Ars Technica explains, each word has to wait for every word before it.

Diffusion borrows the idea behind image generators like Stable Diffusion. Text is made of separate tokens, so instead of adding visual "noise," these models hide tokens behind mask placeholders. The model begins with the answer fully masked and fills it in over several "denoising" steps, gradually revealing every part of the response (Ars Technica). Inception compares it to "an editor revising a full draft at once" rather than a typewriter (Inception).

Why it's fast. Every pass updates many tokens at once, so you need far fewer passes than one per word. Google says DiffusionGemma moves the main bottleneck from memory bandwidth to raw compute (Google). In practice, most current systems don't generate an entire long answer in one go. They write it in blocks: block by block in order, with diffusion inside each block. DiffusionGemma, for example, works on 256-token "canvases" (Google AI docs).

Other perks. Because the model can look at the whole block in both directions, it can fix its own mistakes mid-generation. It is also suited to tasks that don't run neatly left to right, like in-line editing and code infilling (filling a gap in the middle of existing code) (Google, Google DeepMind).

Trade-offs. Quality isn't automatically on par. Google says DiffusionGemma's "overall output quality is lower than standard Gemma 4" and still recommends standard Gemma 4 when you need maximum quality (Google). Speed numbers from different companies are measured in different ways, so they're hard to compare. An independent roundup by BenchLM ranks only models with comparable hosted measurements and leaves the rest unranked (BenchLM). Researchers also note that safety training built for autoregressive models doesn't automatically carry over to diffusion models (Backfill Radar survey).

Diffusion LLMs you can look at today

Inception Mercury 3 (commercial, hosted). Launched today. Inception says it runs at 1,068 tokens per second on NVIDIA GPUs, has a 1M-token context window and costs $0.20/$0.70 per million input/output tokens, with 50% off at launch. It's available in Inception's chat, its API, and through OpenRouter, Baseten and Vercel (Inception). Vendor claims. Inception also says Mercury 3 beats GPT-6 Luna and Claude Haiku 5.5 on its benchmarks; as BenchLM noted, there were no independent scores at launch.
Inception Mercury 2 (Feb 2026). Inception describes it as the first reasoning diffusion LLM, citing 1,009 tokens/sec on NVIDIA Blackwell GPUs and an OpenAI-compatible API (Inception). Vendor claims. Inception also ships Mercury Edit 2, a diffusion model for next-edit code suggestions (Inception blog).
Google DiffusionGemma (open weights, Apache 2.0, June 2026). An experimental 26B mixture-of-experts model with about 3.8B parameters active at a time. It accepts text, images and video as input. Google says it reaches "up to 4x faster" generation and more than 1,000 tokens/sec on one H100, and that it fits in 18 GB of VRAM when quantized (Google, Hugging Face). Vendor claims. It runs in Transformers, vLLM, SGLang and MLX (Google Developers Blog).
Google Gemini Diffusion (experimental demo). Google reports a sampling speed of 1,479 tokens/sec plus 0.84 seconds of overhead. In its own table, results are mixed against Gemini 2.0 Flash-Lite: ahead on some coding tests, behind on GPQA Diamond and multilingual tests (Google DeepMind). Vendor claims. There's no general API (BenchLM).
LLaDA (research, open). An 8B masked-diffusion model from Renmin University and Ant Group, trained from scratch. Its authors report it is competitive with LLaMA3 8B (paper).
Dream 7B (research, open). From HKU and Huawei. It starts from an autoregressive model and adapts it to diffusion (paper, survey).

The main categories of language model

One thing to know first: these labels overlap. A single model can be a mixture-of-experts, multimodal, open-weight reasoning model all at once. DiffusionGemma is diffusion, MoE, multimodal and open-weight (Google AI docs). So think of these as features you can mix and match, not separate boxes.

Autoregressive transformers (the standard chat model)

What it is: The Transformer architecture, generating one token at a time. This is the default design behind nearly every mainstream chatbot. Designed for: General-purpose writing, chat and coding, with a huge ecosystem of tools built around it. Pick it when: You want the most mature, best-supported option. Google, for one, still calls standard Gemma 4 "the standard for high-quality production outputs" next to its diffusion experiment (Google).

Reasoning ("thinking") models

What it is: Models trained, often with reinforcement learning, to work through a problem step by step before giving a final answer (OpenAI, DeepSeek-R1 paper). Designed for: Math, tricky code, planning and multi-step agent work. Examples: OpenAI's o1 line and DeepSeek-R1. Thinking modes are also showing up in other categories: Mercury 2 is a reasoning diffusion model, and DiffusionGemma has a configurable thinking mode (Inception, Google AI docs). Pick it when: Getting the answer right matters more than speed or cost. Extra thinking takes time and tokens, a trade-off Inception itself points out (Inception).

Mixture-of-experts (MoE)

What it is: A very large model split into many "expert" sub-networks, with only a few switched on for each token (Mixtral paper). Designed for: The knowledge of a big model at a fraction of the compute per token. Examples: Ling-3.1-flash has 560B total parameters, 25B active (our coverage). Microsoft's MAI-Base-2 has 2.8T total, 80B active (our coverage). DiffusionGemma uses 8 of its 128 experts at a time (Hugging Face). Pick it when: You want strong quality at lower serving cost. Note that the whole model still has to fit in memory, even though only part of it runs per token.

Small and on-device models

What it is: Compact models built to run on phones, laptops or a single consumer GPU. Designed for: Privacy, offline use, low latency and no per-request API bill. Examples: Gemma 4 E2B and E4B. Google lists E2B at about 2.3B effective parameters and E4B at 4.5B, both handling text, image and audio input (Gemma 4 model card). Android exposes Gemma 4 through AICore (Android Developers Blog). Pick it when: Data has to stay on the device, or you're building features into apps and games that must work offline.

Multimodal and "omni" models

What it is: Models that take in images, audio or video, not just text. "Omni" models can also respond in speech, in real time (Qwen2.5-Omni paper, OpenAI GPT-4o). Designed for: Screenshots, documents, voice assistants and video understanding. Pick it when: Your input isn't plain text. Check which directions are supported: DiffusionGemma, for example, takes images and video but outputs only text (Google AI docs).

Diffusion LLMs

What it is: Covered above. These models draft and refine blocks of tokens in parallel instead of writing one token at a time. Designed for: Speed-critical work such as voice agents, search pipelines, fast agent sub-steps, in-line editing and code infilling (Inception, Google). Pick it when: Response time is the bottleneck. Test quality on your own tasks first.

State-space and hybrid models (Mamba-style)

What it is: Mamba and similar "state-space" layers keep a fixed-size running summary of the text instead of an ever-growing memory of every past token. Hybrid models mix many of these layers with a few regular attention layers (NVIDIA Nemotron 3). Designed for: Long documents and high throughput with less memory. Examples: NVIDIA's open Nemotron 3 Super (120B total, 12B active, up to 1M context) (NVIDIA tech report), IBM Granite 4.0 (InfoWorld), and AI21's Jamba (paper). Pick it when: You're processing very long inputs or serving at scale and memory is the constraint.

Code-specialized models

What it is: Models trained or tuned mainly on code and developer workflows. Designed for: Writing code, autocomplete, next-edit suggestions and agentic coding. Examples: Qwen3-Coder (Qwen), Inception's diffusion-based Mercury Edit 2 (Inception blog), and DiffuCoder, a research diffusion model from Apple and HKU researchers (paper). Pick it when: Coding is the main job, especially inside an IDE where speed matters on every keystroke.

Embedding and retrieval models

What it is: These models don't chat. They convert text (or images) into lists of numbers so software can find similar content. Designed for: Search, recommendations and RAG, where a chatbot looks up your documents before it answers. Examples: Qwen3-Embedding (open weights) and Google's gemini-embedding-001 (API) are near the top of the MTEB leaderboard (MTEB snapshot via GTM Directory, OpenRouter). Pick it when: You need an AI to search your docs, wiki or game lore. Usually you'd pair one with a chat model.

Open-weight vs. closed (a distribution choice, not an architecture)

What it is: Open-weight models let you download the model files and run them yourself, under the terms of a license. Closed models are only available through the vendor's app or API. Examples: DiffusionGemma (Apache 2.0) (Google) and Ling-3.1-flash (MIT) (our coverage) are open-weight. Mercury 3 and Gemini Diffusion are hosted only (Inception, Google DeepMind). Pick open when: You need to control the model, fine-tune it, keep data private or run it offline. Pick closed when: You want frontier quality without running servers.

Quick comparison

Category	Core idea	Best for	Example
Autoregressive transformer	One token at a time	General chat, writing	Standard Gemma 4
Reasoning	Thinks before answering	Math, hard code, agents	DeepSeek-R1
Mixture-of-experts	Only some experts run per token	Big-model quality, lower compute	Ling-3.1-flash
Small / on-device	Compact enough for phones and PCs	Privacy, offline	Gemma 4 E2B
Multimodal / omni	Images, audio, video in (and speech out)	Voice, screenshots, video	Qwen2.5-Omni
Diffusion LLM	Parallel draft and refine	Low latency, editing, infilling	Mercury 3, DiffusionGemma
State-space / hybrid	Fixed-size memory plus some attention	Long context, throughput	Nemotron 3 Super
Code-specialized	Tuned for code	IDEs, coding agents	Qwen3-Coder
Embedding	Text to vectors	Search, RAG	Qwen3-Embedding
Open vs. closed	How it's distributed	Control vs. convenience	DiffusionGemma vs. Mercury 3

What to weigh

Latency vs. quality: Does your app need answers in under a second (voice, game NPCs, autocomplete), or is it fine to wait for a model to think?
Your own evals: Vendor speed and benchmark numbers are measured differently. Test on your own prompts.
Input types: Text only, or images, audio and video too?
Context length: How much text you need to feed in at once.
Where it runs: Cloud API, your own GPU, or the user's device. Check the memory you'll need.
License and data: Open-weight terms, privacy requirements, and whether you need to fine-tune.
Ecosystem: Tool calling, JSON output, SDK and framework support. Newer architectures may have less of it.
Cost: Per-token price vs. the cost of hosting it yourself.