The authenticity crisis

Is AI music becoming better than the real thing?

AI-generated music is getting good. Scary good. We've gone from robotic-sounding MIDI files to full-blown songs with vocals, complex instrumentation, and emotional weight in just a few years. Platforms like Suno and Udio can create tracks that can be, to the average listener, indistinguishable from something on the radio. This rapid progress has created an interesting problem: how can we even tell the difference anymore? And what does it mean if we can't?

A new research paper (from anonymous authors and currently under double blind review) dives headfirst into this issue; its findings are impressive (and a little unsettling).

Melody or machine

The paper, titled "Melody or Machine: Detecting Synthetic Music with DualStream Contrastive Learning," tackles the challenge of detecting AI-generated music. The researchers point out that as AI music generators get better and more diverse, our tools for spotting them are falling behind. An AI detector trained to spot a song from Suno 4.5 might be completely fooled by a track from Udio's latest model.

To solve this, they did two things:

1. They built a better dataset. They created a massive new benchmark called Melody or Machine (MoM), with over 130,000 songs from a wide variety of AI models (both public and private) and a huge collection of real human-made songs.

2. They designed a smarter detector. They developed a new model called CLAM (Contrastive Learning for Audio Matching) that's much better at generalizing and catching fakes from new, unseen AI generators. It cleverly looks for subtle inconsistencies between different parts of a song, like the vocals and the instruments, which are often a tell-tale sign of AI generation.

The titans of AI music

The study pits its detector against music from the current heavyweights in the AI music scene. If you’re working in the music industry and haven’t played with these tools yet, do so now; you're missing out on a glimpse of the future.

The big three right now are:

• Suno. Suno is typically viewed as the fan favorite, generating melodies that closely fit “popular” music at the cost of some audio fidelity. They’ve made a lot of progress with instrumental fidelity, but their vocal quality even in their newest model still has a very “AI”-feel to it.

• Udio. Udio is known for its high fidelity in instrumentals and vocals. The songs it generates often have a professional, studio-grade clarity. It’s the platform of choice if you want something that sounds technically pristine (but may need some finagling to get a solid song structure in place).

• Producer.ai (formerly Riffusion). Riffusion recently rebranded to Producer.ai, launching a new heavily ChatGPT-inspired interface on top of a new model. The new model’s output seems to be a mix of Suno and Udio’s with solid song structure and a higher-than-average fidelity.

Here are some examples of what they can do (snagged two generations for each prompt):

Producer.ai

Prompt: cinematic, epic fantasy score, soaring strings, choir

Udio

Prompt: A soulful blues song about a robot who lost its charging cable

Suno

Prompt: Upbeat indie pop song for a summer road trip, female vocals

Table 5

This is where the study gets interesting. To test how "good" the AI music was, the researchers had humans do blind, head-to-head comparisons between different songs, asking them "Which song sounds more realistic?". They used an ELO rating system, like in chess, to rank the models based on these human preferences.

The results are in Table 5 of the paper, and they're a bit of a bombshell:

According to their human listeners, music from Producer.ai and Udio was perceived as more realistic and higher quality than actual human-created songs.

But does this mean AI has surpassed human creativity? It's not that simple. "Realistic" and "high quality" are subjective. AI models are trained on vast datasets of existing music, and they're exceptionally good at creating music that is technically perfect. The notes are perfectly on pitch, the rhythm is perfectly in time, and the production is clean. But does technical perfection equal "better" music?

Music is more than just a collection of correct notes. It's about the subtle imperfections, the unexpected emotional shifts, the shared human experience baked into a performance. Can an AI truly replicate the soul of a blues guitarist who's lived the stories they're singing about? Or the raw energy of a punk band thrashing out their frustrations in a garage? Probably not (yet).

The inevitable consequences

Whether it's "better" or not is almost a moot point. The fact that AI can generate music that is good enough has massive consequences, echoing the concerns we've seen with AI's impact on stock music.

• Copyright chaos. The music industry is already fighting back (nothing new). Major labels are suing Suno and Udio, alleging they trained their models on copyrighted songs without permission. This is the central battleground for generative AI right now.

• Market saturation. When anyone can generate dozens of high-quality songs in minutes, the music market will become incredibly saturated. It's already hard for new artists to get noticed; this will make it exponentially harder.

• The authenticity problem. If we can't tell the difference between human and AI art, does the connection we feel to it change? Part of the joy of music is connecting with the artist. If the artist is an algorithm, is that connection lost?

The "Melody or Machine" study is a crucial piece of work (though, to be noted, it is still pending peer review). It proves that AI music is already a force outperforming human-made music in some respects. The CLAM model is a necessary tool in a world where the lines are blurring.

We need to decide what we value in music: technical perfection, or human expression, flaws and all. Because soon, we might have to choose.