Google's DiffusionGemma proves you don't need to train from scratch to build a text diffusion model

Google DeepMind has retrofitted the Gemma 4 model into DiffusionGemma, utilizing less than 10% of the original training budget. DiffusionGemma generates 256 tokens in parallel, achieving a speed of about 1,500 tokens per second. However, its performance on reasoning tasks still lags behind the original autoregressive model. This development highlights the potential for enhancing existing models rather than starting from scratch, saving time and resources for AI researchers.