Abstract

Transformer attention scales quadratically with sequence length, which has made long-sequence modeling expensive enough that practitioners routinely truncate inputs or pay steep hardware bills. State space models (SSMs), particularly the Mamba architecture and its successors, offer a different trade-off: linear-time sequence mixing via learned recurrences, with competitive quality on language, audio, and genomics benchmarks. As of early 2026 the SSM family is not yet dominant, but it has accumulated enough production deployments and ablation studies to be taken seriously as a first-class alternative rather than a research curiosity.

How Mamba Differs from Prior SSMs

Earlier SSMs like S4 and H3 fixed their transition matrices at initialization, limiting their ability to do content-dependent routing of information across time steps. Mamba’s key contribution, introduced by Gu and Dao in late 2023, is selective state space: the model learns input-dependent matrices B, C, and delta at every layer, allowing it to gate which tokens influence future state. This makes Mamba behave more like a soft, recurrent attention than a fixed filter bank. The Mamba-2 paper tightened the connection further, showing that the selective SSM kernel is a specific instance of the structured masked attention (SMA) framework, which let the authors fuse SSM layers into standard GPU matrix kernels with near-FlashAttention efficiency.

Where SSMs Win and Where They Still Lose

On sequences beyond 32k tokens, Mamba-class models consistently outperform equivalently-sized Transformers in throughput and memory footprint, often by 3-5x in tokens-per-second at inference. This advantage compounds with sequence length. The gap closes or reverses on short sequences where attention’s constant overhead matters less. Retrieval-heavy tasks that require precise, arbitrary-position lookups remain a weakness: SSMs propagate information through a fixed-size state vector, so a fact buried 100k tokens back may be compressed out even if theoretically reachable. Hybrid architectures, alternating SSM layers with sparse attention layers (as in Jamba from AI21 Labs), attempt to get both properties without paying the full quadratic cost.

Practical Deployment Considerations

The main friction at adoption is tooling maturity. FlashAttention has years of production hardening behind it; the equivalent triton kernels for Mamba-2 are newer. Teams at Cartesia and Together AI have published inference servers with SSM support, and Hugging Face Transformers now includes Mamba checkpoints, but quantization and speculative decoding support lag behind the Transformer ecosystem. For teams working with genomics sequences, audio waveforms, or very long documents, the throughput gains are real enough to justify the tooling overhead. For standard sub-8k text tasks, the switch is harder to justify today.