How Does AI Help Beethoven? – A Conversation with Walter Werzowa.

drawn image of calendar icon

March 7, 2024

hand drawn writing icon

Anastassia Lauterbach

Walta Wezowa portrait photo in black and white

In 1824, Beethoven premiered his last orchestral work, Symphony No. 9 in D minor. But, before his death three years later in 1827, he had begun work on a 10th symphony.

All that remained of Beethoven’s 10th Symphony was fragmentary sketches of the first movement which he had notably started before his death in 1827. However, these fragments were turned into a complete piece of music using AI technology.

I talked to the composer behind the Beethoven X AI Project, Walter Werzowa:

Lauterbach: Could you please talk about how you started your journey in music, and explain what brought you to where you are today?

Werzowa: I decided that music was my world quite early in life. I studied architecture and was very good at it. But something called me to music, and studying it was a big challenge. I love challenges. My initial struggles made me a very different musician. When I went through postgraduate studies at USC, my peers told me that my different approach in music was a direct ticket to unemployment. For a while, I was confused and ashamed of that. It became my most significant gift once I realised that different is a prerequisite of unique. The world is looking for unexpected new solutions. I worked with Spielberg and many incredible, monumental artists. I could see their appreciation for a different angle, that fresh perspective opening hidden doors and uncovering looming horizons.

Lauterbach: Do you have any advice for people looking for alternative problem-solving approaches?

Werzowa: I love Brian Eno’s card game called Oblique Strategies. These cards are helpful whenever your creativity flow is challenged. I am not referring just to artistic endeavours. It is all about opening up the mind. Imagine you have a complex mathematical problem. Instead of pondering it for hours, you might use Eno’s card deck with nonsensical questions. They change your perspective. Suddenly, you might feel like you have just returned from a walk in wonderful nature. You might see the problem from a different angle and ultimately solve it.

Lauterbach: Do you believe that music can support problem-solving?

Werzowa: When we listen to melancholic music, our brain produces prolactin, the hormone mothers secrete in high quantities while giving birth. Health, music, and creativity are connected. And techno music without lyrics can help focusing.

Lauterbach: How did you come to the Beethoven’s Tenth Symphony project?

Werzowa: Matthias Röder, MD of the von Karajan Institute, called me and asked whether I wanted to compose a symphony with Beethoven. I took it for a joke first. And then, I got terrified. I doubted I could ever master something like that. Over two years, I worked with an incredible team, learning unexpected things about art and hacking the process of creativity. I remember meeting the project’s Chief Data Scientist, Ahmed Elgammal, for the first time and telling him how we had to teach the AI all the rules about music. He smiled and said: ‘No, we couldn’t and shouldn’t do this.’ And I thought: ‘Oh, my God, he doesn’t understand the essence of music!’ A data scientist explained to a musician that all the team needed was to directly learn from Beethoven and Beethoven alone. Beethoven’s triumphs and his “mistakes/ground breaking abilities” were part of his unique musical humanity. When Ahmed said it, something happened within me. Everything was perfectly clear all of a sudden. The unexpected, the rule-breaking and acting outside of frameworks carved in traditional books led to a new space. It helped me to understand creativity from a totally different angle.

Creativity is a human choice. I could picture Beethoven walking through the vineyards in Vienna, having hundreds of melodies in his head. He always carried a sketchbook to write down what he had just heard or dreamed about. Later, leafing through the book, he might have combined two or three of these melodies. Such creative accidents – or choices – might have led to what we – two hundred years later – describe as Beethoven’s innovative genius.

When we talk about understanding music today, we include Shostakovich’s Ninth Symphony and the creations of Schönberg into the mix. But if we want to create with Beethoven, we can’t get confused by what happened after he died. The data set to train AI included only Beethoven and the music that influenced him.

We generated our first results and tested our hypotheses while arranging several themes for the string quartet. We performed in front of an audience of journalists and musical critics. Upon the performance, we asked whether they believed the music could be attributed to Beethoven. We got fortunate. These people applauded. We understood we were on the right track that evening. Even the usual naysayers couldn’t find arguments against our project. In a sense, we gave Beethoven a neutral, objective partner – the generative AI.

Lauterbach: When I think about synthetic data in the context of Generative AI, I wonder what their nature means for innovation and novelty.
You recreated a specific historical context when you fed AI with all of Beethoven’s creations and topped it up with the music he knew during his lifetime. This context represents a closed dataset. What happens if we apply similar procedures to different fields, industries and branches of art?

What do you think about creating something new in the world influenced by always backward-looking synthetics? Or is new just an unexpected combination of something already existing?

Werzowa: That’s such an important question that I am pondering about all the time. My insignificant human mind prefers to believe in indefinite possibilities of creation. I think quantum leaps are possible, and these will bring us to new heights. Schönberg and Picasso were such Quantum Leap artists. We don’t understand where their ideas came from and what hidden forces they were connected to. If we apply quantum physics to music, we can assume that they might have received signals from parallel worlds.

Lauterbach: David Deutsch, a famous quantum physicist, believes our reality is a multiverse. Are you thinking along his lines?

Werzowa: I believe that from a scientific and spiritual point of view, there are multiple worlds out there, interfering with each other and stimulating each other. Our creativity may be partly enhanced by the existence of the multiverse. Beethoven had a powerful initial four-node theme when he did his Fifth Symphony. He repeated it over and over in slight variations, to keep it memorable and interesting. However, we still don’t know where the initial spark came from. He might have heard a dog’s bark next door. We have zero information, what might have inspired him to write this simple theme. His synthesis – what we have today, the end product – is simply mind-blowing. As long as there are humans, we won’t run out of novelties, even in the age when we use AI with synthetic data to augment the real world.

Lauterbach: Could you please talk about the daily challenges when you worked with AI on the Tenth Symphony? How did you overcome these?

Werzowa: Our daily routine was changing depending on where we were with our work. Initially, we built the data set, including annotating musical pieces and integrating libraries of music that might have influenced Beethoven. The research piece of the project was humongous, with countless books we could put our hands on and conversations we had with music historians.

Developing the algorithms was a more straightforward part. It was interesting to see how Python – the language we used for computing – understood music. It was so different from how humans perceived it. For an AI, music is an absolute time and an endless function string.
The AI gifted me with invaluable musical encounters. Every morning around 5 a.m., I rushed into the studio to listen to the new MIDI files. Some of them were awful. But some made me cry, as their themes were so touching, beautiful, and unexpected. I felt like climbing Mount Everest, standing at the top, and gazing at everything around me. Working with AI was a profoundly emotional experience. I had to make choices without being pretentious, without comparing myself to Beethoven himself, and reminding myself that even this innovator had to choose among multiple ideas and impulses. I had to remember that Beethoven took themes from one symphony and plugged them into another, just like it was the case from his Seventh to his Ninth.

Lauterbach: You tell a story of an incredibly fruitful collaboration between humans and machines. Why do you think people fear AI so much?

Werzowa: Hollywood did an incredible job making us fearful of the future with awful AIs looking like Terminators. We need to shift the paradigm. Working with technology isn’t about squashing it or prevailing over it. The experience can make us happy. I remember feeding AI all the data, imagining like I was in a dialogue with the machine. Some days, I was angry and confused about the intermediate steps. But overall, the project brought happiness to me. I did not perceive the experience to be a competition between me and the AI. I thought I was getting better in my artistic expression. I thought about an incredible chance to look from the inside into Beethoven’s creative process as if transported into his room in Vienna, listening to his piano.

I wish more people had positive collaborative experiences with AIs. Maybe fear is an initial emotion with any technology. We might review the time when the first phone appeared, or the first TV entered our households. And we need to understand what we lose and what we gain whenever there is the next technological leap to embrace.

Gutenberg introduced the printing of books, which was an incredible innovation as so many people could read all of a sudden. But human memory suffered from it, most probably. Before Gutenberg, we had to remember the books by heart and recite the Bible. We didn’t need to hone our memory once there were books around.

Lauterbach: How can we reduce fears of technology in the society?

Werzowa: Universal Basic Income might be an excellent way to mitigate fears. Telling positive stories about technological impact on social changes is essential, too. Our society transformed, and technology played an important role in it. AI can help us develop and apply solutions to save this planet. We should proactively address its positive impact.

Lauterbach: Technology is a great equalizer. It offers access to knowledge and enables cross-border communication and collaboration. It can enhance creativity. Indeed, famous artists use AI avatars to perform in front of audiences in different time zones. At the same time, Hollywood writers and actors go on strike because studios use Generative AI, putting pressure on loans and freelancers’ fees.
How do you think about achieving excellence in the age of AI?

Werzowa: Excellence comes to those who know life struggles, rise above challenges, and show grit in pursuing their calling. Finding a purpose isn’t always easy. Many difficult and even hazardous jobs should be done by machines, not humans. But those holding on to these jobs fear AI the most. I don’t want to sound harsh, but maybe “just getting by” is a sedative for humans, preventing them from moving on. Imagine a lady working every day of the week as a cashier. Could it be that some might feel trapped there, or afraid of a change?
I met incredible moviemakers, composers, and actors, many of whom started as dishwashers. Being overprotective of society doesn’t support innovation. We can see what over-regulation and protectionism do to Europe. The European answer to everything is regulation, even if they regulate something they did not build or don’t even have.

My fifth “Romy and Roby” book will be dedicated to using generative AI in Music creation. My mind is currently set on finishing “The Unfinished Symphony” by Schubert, which happens to be one of my favourite pieces of Romanticism. By 2028, when the fifth volume will be created, I will be an expert on Schubert. You can’t work with AI well if you don’t have a superb understanding of the domain – and this time, it will be the music of Schubert and his contemporaries. I am curious to learn whether there was a motive or the composer for not finalizing his symphony. Maybe we can find out together?

romy and roby and the secrets of sleep book cover

Book 1

Romy & Roby And the Secrets Of Sleep.


Submit a Comment

Your email address will not be published. Required fields are marked *

Select your currency