Nunchaku 4-bit Diffusion Inference Finally Lands in Diffusers

Most quantization techniques are paper tigers. They compress your model weights down to save precious video RAM, only to dequantize them right back on the fly during compute time. The memory footprint shrinks, sure, but your actual inference speed stays pathetically flat or takes a latency penalty. It is a frustrating compromise that leaves serious builders hanging out to dry.

Then SVDQuant and the Nunchaku engine changed the rules by actually tackling both weights and activations at 4-bit precision, speeding up the entire denoising loop instead of just saving disk space. But there was a massive catch. Running these checkpoints meant juggling a completely separate, fragile inference library that made local integration feel like a minor engineering marathon.

That friction just vanished. You can now load a Nunchaku checkpoint using a standard from_pretrained call right inside Diffusers. No custom pipeline classes. No excruciating local CUDA compilation cycles. The runtime handles pulling down the necessary NVFP4 kernels automatically from the Hub on the very first execution, which is how developer tooling should always work.

On modern hardware like an RTX 5090. Pushing out a crisp 1024x1024 image takes roughly 1.7 seconds while drinking a mere 12 GB of VRAM instead of the usual 24 GB. Of course, you need a Blackwell architecture GPU to squeeze every drop of results out of those NVFP4 formats. Though reliable INT4 variants remain available if you're running older cards.

This update represents a massive win for anyone trying to run heavy diffusion models locally without melting their hardware or spending half a day fighting dependency hell. It strips away the unnecessary complexity and lets us focus on building actual products instead of wrestling with brittle custom runtimes.