Riffusion

Riffusion is a fascinating, open-source AI project that generates music utilizing a highly unconventional method: it creates audio by generating visual images. Specifically, Riffusion is a fine-tuned version of the popular Stable Diffusion image generation model. Instead of generating pictures of cats or landscapes, it generates "spectrograms"—visual representations of the frequency spectrum of sound over time. It then translates these generated images back into audible sound waves, creating completely original musical loops. Because it relies on image generation architecture, the prompting process is incredibly unique and highly experimental. Users can type prompts like "jazzy saxophone over a heavy hip-hop beat" or "typing on a vintage typewriter," and the model will attempt to visually construct the audio representation of that sound. While the resulting audio is often low-fidelity and chaotic compared to dedicated audio models, Riffusion represents a groundbreaking, highly creative approach to synthetic media generation that has captivated the open-source developer community. **Pricing:** Riffusion is an entirely free and open-source project. Because its underlying code and model weights are publicly available, developers and technical users can download, modify, and run the software locally on their own hardware without any subscription fees, licensing costs, or usage restrictions.