← Wiki
2026-11-12 hub hubmusicaudiogenerationmidistemsace-step

Music AI

This is the running page for everything we do with AI and music. It starts from what is already in our files: transcription and stem splitting are daily tools, generation is so far reading only. Each line says which.

Generation (text or lyrics to song)

Transcription (audio to MIDI), local

The audio-to-midi skill in the SuperDuper DSP plugin (project-superduper-dsp). Three models, picked by material, measured on Apple Silicon (figures from the skill, same 3:12 recording):

Model Hears Time for 3:12 Licence note
MuScriptor medium full mix, one MIDI track per instrument, GM drums 52 s code MIT, weights CC BY-NC 4.0 (fine for our own film, not for a shipped product)
Basic Pitch polyphony in one track, no drums 35 s Apache-2.0
SwiftF0 (our Core ML conversion) one line, the loudest 1 s (about 190x realtime) CC-BY-4.0 source

Measured on a Gesaffelstein teardown (40 s): MuScriptor tempo 102.994 against 102.985 by hand, kick grid with 0 missing hits over 51 beats; but every velocity comes out as 100, durations are wrong (bass note median 0.26 of a beat against 0.93 real), and instrument labels are guesses. Trust onsets and grid, not length or dynamics.

Stem splitting

solo:stem-split: Demucs htdemucs, four stems (vocals, drums, bass, other), local; about 4 minutes per 20 minutes of audio on Apple Silicon with the mps device. Used for the Content ID radio scrub in creative-video-sound. ACE Music offers its own stem splitter and MIDI export, but only on paid plans (ace-music-v1).

Rhythm and structure

Making and mixing

Open questions

  1. Can ACE-Step 1.5 produce background tracks good enough for Life2Film, and what does it cost in time and memory on the M5? Nobody here has run it.
  2. Does ACE Music v1 beat Suno v6 on our own prompts, or only on Artificial Analysis’s mix? The published gap sits inside the error bars.
  3. Does generated audio survive our own tools, i.e. does audio-to-midi recover the notes of a generated track as well as it does a recorded one? That would also test ACE’s built-in MIDI export.
  4. Licensing: which generators give commercial rights on a free or paid plan, and which weights allow it. ACE Music’s free tier gives none.
  5. When the open ACE Music v1 Flash arrives (promised for late 2026), is it a usable local model? Unconfirmed until released.

Related