ACE-Step is an open-source foundation model for music generation. Give it a text prompt (genre, mood, instruments) and optional lyrics, and it produces a complete song with vocals and instrumentation. It also supports lyric-to-vocal, remixing and editing workflows, which makes it a building block for music apps as well as a creative tool.
The project’s site at ace-step.club covers the architecture, hardware benchmarks, applications and a live demo.
How It Works
ACE-Step combines diffusion-based generation with a Deep Compression AutoEncoder and a lightweight linear transformer. This design keeps generation fast while holding melody, rhythm and structure together across a full-length track, an area where many faster models lose coherence.
Key Features
- Text to music — Describe a style and get a full song, up to about 4 minutes long.
- Lyrics to vocals — Add your own lyrics and have them sung.
- Remix and edit — Rework existing generations instead of starting over.
- Open weights — Apache-2.0 code, weights on Hugging Face and GitHub.
- Extensible — Fine-tune custom styles and voices; works with Hugging Face Diffusers and ComfyUI.
Performance
Speed is measured as Real-Time Factor (RTF) on a single GPU with batch size 1:
| GPU | Time for 1 min of audio (27 steps) | Time for 1 min of audio (60 steps) |
|---|---|---|
| NVIDIA A100 80GB | 2.2 s | 5.0 s |
| NVIDIA RTX 4090 | 4.0 s | 9.0 s |
| NVIDIA RTX 3090 | 5.0 s | 11.0 s |
| NVIDIA RTX 2080 Ti | 7.0 s | 16.0 s |
Getting Started
- Try the live demo on the website to hear what it produces.
- Clone the GitHub repository and install the requirements.
- Run
python app.pyto launch the local web interface. - Fine-tune or plug it into ComfyUI or Diffusers for custom pipelines.
Use Cases
- Songwriters drafting demos from lyrics
- Background music for videos, games and podcasts
- Researchers studying music generation
- Developers building music features into apps
ACE-Step is a strong choice if you want a fast, open music model you can run, study and customize yourself.