I’m an old-school musician, since I was a child I admired people who could make music. If I saw someone on a street corner, in a bar, in a theatre, in a park or anywhere else making musical notes on an instrument, I was the kid who would gawk at the performer. It seemed like magic to me, not just because of the sound but because of what that person was making me feel.
The importance of musical creativity
There are other animals capable of solving simple problems, but we humans take this creative capacity to unsuspected limits and we are the only species capable of creating art. All of us are capable of it, although some are much better than others. We are all capable of painting, writing a story, sculpting a statue or writing a melody if we care to learn the techniques. Naturally, only a few are like Beethoven, Velázquez, Lorca or Michelangelo, but even making a ‘churro’, we are light years away from what any other animal would do. Try to teach a chimpanzee, the closest relative to us, the techniques of oil painting and let him see what he does and compare it with any work of any mediocre artist. Creativity is what identifies us as humans.
Until very recently, computers were much better at calculations than we are, but they were incapable of being creative and many of us thought that this ability could never be replicated in a machine (the eternal discussion of the existence of the soul). The closest thing a machine could do in music was to interpret melodies and chords given to it by a human through midi and honestly, I was never a big fan of styles where machines replace musicians because the sound seems very cold to me. That perfection in the note, placed at the exact moment, with the exact intensity gives it an unreal character. The same thing happens to me with human performers.
Until very recently, everything played with virtual instruments sounded artificial to me, I’m not a Taliban, it’s fun on a dance floor and for socialising at a party, but I wouldn’t wear it to drive a long journey or enjoy at home, savouring every nuance of the song.
About 10 years ago now, on a musicians’ forum, news came out that a university in Poland had trained an AI with hundreds of thousands of metal songs and set it to composing and the songs it came up with were uploaded to a website for the public to listen to, and I rushed over to check it out, and I wasn’t too happy about it.
As I had no idea where we were on this issue, before I started writing, I spent some time watching videos, reading articles and even experimenting with AIs like Suno, Udio and Sonauto. I have tested them with musical styles that I knew they were not ‘comfortable’ with because I already knew that the more ‘inorganic’ styles where the virtual instrument and the ‘cold’ sound prevail, computers play with an advantage so I experimented with rock, blues, heavy metal, jazz, country, flamenco and even jotas and now I am about to write what I have learned these days from the point of view of a musician and a music consumer.
What can an AI do musically?
After a lot of research, I have come to the conclusion that AI can be used in four types of work:
- Cloning a voice
- Composing a track
- Mixing and mastering
- Multi-purpose tool for studying, learning and in the creative process.
Cloning a voice
It has been in the news in recent times that tracks have been uploaded on the net ‘sung’ with the voices of well-known artists without them having been involved in the recording. In fact, there are already websites and applications that allow you to use these voices for whatever you want.
So far, none of the artists who have been involved in these experiments have given their approval. The closest to agreeing, as far as I know, has been Grimes, who has proposed that, if someone uses her voice, she should be paid 50% of the revenue generated, all the others have expressed their absolute refusal to use their voices without permission, and it makes perfect sense.
This new situation opens up a new debate, so far, it is not so clear that voices do not have ‘image rights’. A person’s ‘image’ is everything that identifies them, not just their face. The voice, like the face, is a personal and unique feature of each person, if you use someone’s voice to create anything, just as if you use their face, you are irremediably associating them with that creation and it doesn’t take much technical, legal or moral explanation to understand that that person has something to say about it and decide if they want or are interested in being part of it.
Legislation will have to catch up with this new reality, although I am sure it will.
Currently, the management companies that we musicians use to upload our music to streaming platforms have applications tracking these platforms and if they detect that someone has used a track from one of their clients, they automatically request that the monetisation be removed and diverted to the author. I’ve had my manager remove my own monetisation because I uploaded a new video without notifying them.
Music composition
Here we enter one of the most philosophically controversial parts. As I said at the beginning of the article, until recently, only humans were able to create music, and this new capacity of computers takes away the only characteristic that differentiated us from the rest of living beings: creativity.
To develop this section, I have not only documented myself in experiments of others uploaded to YouTube or articles published on the internet, but I have also spent a good while experimenting with three well-known platforms. Suno, Udio and Sonauto, testing them with different styles.
The first impression is that it has improved a lot since the time I went to the website of that Polish university to listen to the result of that experiment more than 10 years ago. What I heard, there was a bit of everything, but some of it was undoubtedly quite worthy.
The way the AI learns and creates is based on the analysis of the data it has been given in the training process, in this case, hundreds of thousands of songs of all styles. The AI analyses them, extracts patterns and uses them in its own creations, and that is exactly the same way we humans learn.
All musicians learn to compose by playing the pieces of our favourite composers. When we write, consciously or unconsciously, our musical tastes and experiences are reflected in our creations. Everything new comes from something previous and even when we force our creativity and create a new style, that new style comes from the combination and/or development of one or more previous ones, a friend once told me ‘everything that is not tradition, is plagiarism’.
At the moment the AI creates music from already known styles, we don’t know if it will be able to evolve music from an already known style to a new one, for example from blues to rock’n’roll, from rock’n’roll to heavy metal and from heavy metal to trash metal. Time will tell.
After many hours of fiddling with these three platforms, I have come to several conclusions.
The sound it delivers from the compositions created is not very good in the sense of production, mixing and mastering, but we’ll get to that later. It sounds like a low quality mp3, although for many people who are used to listening to music through their mobile phone’s mini speakers this won’t be a problem.
The AI is undoubtedly developed in the US and so some styles are better suited to it than others. During the experiment, I found that it struggled more as I asked it to play lesser known styles from the land of Coca-Cola. If I asked him for a blues, he did a pretty good job, when I asked for a flamenco he did a pop rumba, but when I asked for a Castilian jota, he did a pop song with a touch of country. Without a doubt, he had no idea what I was asking for, so he went for pop, which is the ‘standard’, and as in the description I asked him to talk about a traffic fine, he must have interpreted ‘traffic’ as road and that led him to country.
We are not yet dealing with a virtual Mozart. Depending on the style, as I said, it depends on whether they are well-known styles in the US, he does better or worse, but I haven’t heard anything ‘sublime’ yet. However, I don’t hear it in a lot of human-made music either. In fact, I’ve heard stuff in AI that is better than some of the human-made stuff that has been successful.
It works better the more explicit you are in your description, for example, if you give it just one sentence to make the lyrics, it will make a very flat and obvious lyric, if you ask it to use metaphors, onomatopoeias or metonymies (this costs it a bit more) then the lyrics become a bit more interesting.
I get the impression that he doesn’t deal specifically with each part of the song. A song in its simplest version consists of two parts, the base and the chorus. That would be the least worked form and as we complicate it, we add more parts such as intros, changes, bridges, instrumental parts or even several different bases and choruses. Each of these parts has a function, the chorus is the main part, where the idea of strength is shown, it must have an easy to remember and repetitive melody. On the other hand, we have the base, which is the story that leads us to that idea of strength. The bridges and changes are transitions that serve to facilitate the passage from one to another or to generate a tension that drives the next part. Each of these parts should be written with their specific function in the song as a whole in mind, and yet I get the impression that for the AI, everything is a chorus. In some cases, it sounds good, but in many others it does not. The structure used in most of the examples has been intro + base + chorus.
Instruments that can be well replicated by midi sound better, in styles with instruments like the electric guitar, which doesn’t sound good this way, in most cases, it sounds like ‘keyboards with overdrive’.
Another important aspect of using AI as a composer is that of copyright. I understand that there are people who are getting into the business of releasing songs through AI, uploading them to streaming music platforms and monetising them.
Mixing and mastering
When I started to do more research, this was the part of the whole music creation where I put the most enthusiasm. Of all the creative process, there is a part that is less known to the general public and yet fundamental to the final result: mixing and mastering.
Whether an album is produced by one producer or another is the same as if the same film is directed by one director or another. We know that the final result will be completely different.
For me it’s one of the most difficult parts and the one that gives me the most headaches. The times I’ve had to do that job I’ve felt frustrated. It takes a lot of creative ability and a lot of knowledge of music theory, wave physics and even aural perception. When you delegate the job to someone else and they do it well, it’s a relief, they’ve more than earned their paycheck.
There are applications that use AI for mastering and mixing work, especially the mastering part, and some platforms for uploading your music to the web offer it to you, but the result is quite disappointing. They ask you what musical style you want and they apply pre-established formulas that make everything sound too homogeneous. Each piece of music should have its own personality and should be treated individually, you need sensitivity and artistic understanding. That’s why, in point 1 of the previous section, I already warned that AI-generated music sounded to me like a low-quality mp3 of flat production quality. It looks like I’m going to have to deal with that heavy and unrewarding part of it for a while longer.
Besides, a producer’s job is not only limited to mixing and mastering, as I said, he is a director, he decides which takes are good, which ones are repeated and, above all, he gives indications to the musicians on how to interpret and even corrects parts of the composition to improve it. This, for the moment, is not done by the AI either, which only works with what you give it already done.
However, the AI has something else to offer in this area that is, in my opinion, very interesting and useful. It cannot replace the producer, but there are applications and plugins that use AI to help the producer make decisions, analysing waves and frequencies, looking for inconsistencies in the mix and giving the information in a more visual and clear way so that the producer can do his job more efficiently. The truth is that the examples I have seen have surprised me quite a lot and represent a great advance in this aspect, although here, thank God, humans are still essential if you want something of quality.
Multi-purpose tool
Of all the things I have learned from researching the uses of AI, this is what I liked the most and that is why I have left it for the end. When we hear about AI in music, it is easy to think of an artificial intelligence that makes music, but its application as a tool to help in the study and creation process is not so obvious.
During my apprenticeship I have seen and used really useful applications that make the musician’s work much easier. There are applications that give you a song and separate the different instruments and the voice by tracks. With it you can isolate the part you are interested in to study it and also eliminate one or several instruments to practice playing over them.
For study and analysis there are also applications that give you directly the score or the chord progression of the instruments you request for a given piece.
There are AI applications in which you give it a recorded voice and you can ask it to do backing vocals in thirds, fifths and octaves of what you have given it, ideal for use in your mixes if you don’t have a backing vocal group at hand. Other applications give them a piece of music and they help you by including instruments that you don’t have access to by following your instructions.
Another example are AIs where you give it an incomplete recording and it gives you several options to finish it, which is very useful when you find yourself in a creative rut with a half-done track that you don’t know how to solve. I’m not saying that the AI will finish the job for you, because as we’ve said before, it’s still not the best composer we can turn to, but it can give you hints and ideas on how to do it yourself.
There are applications that improve the sound of old and damaged recordings, allowing you to recover works. There are thousands of examples and I could go on and on for pages and pages.
Perhaps the biggest news about this was The Beatles’ last song “Now and Then” released on 2 November 2023, 44 years after Lennon’s death and 23 years after Harrison’s death. This song was written and recorded on a demo by John Lennon in 1977. In 1995, 15 years after Lennon’s death, his bandmates decided to release it on a compilation album, but the poor quality of the recording made it unfeasible. 29 years later, artificial intelligence was able to clean up and improve the sound of Lennon’s demo, separate the instruments into tracks and so the living members, Paul and Ringo were able to re-record their parts and arrangements over John’s parts and include George’s 1995 recordings. AI allowed us to enjoy The Beatles’ last song when biology itself had denied us.
So is AI a threat or an ally to the musician?
Back to the beginning of the article, here I am, in front of the computer with mixed feelings, trying to decide which I give more weight to. Artificial intelligence is a very powerful tool, not only for music. Having given computers the ability to be creative opens up a new reality of opportunities and threats in all aspects of life, from engineering to finance and of course art in general and music in particular.
AI is a tool and as such, it is neither good nor bad, it depends on the use we make of it. Throughout the article I have described what, in my opinion, are the strengths and weaknesses of this technology. It is the use we make of it and, therefore, we are the ones who have the last word.
The main advantage of AI is that it breaks down our limits. Until the advent of this technology, musicians were tremendously limited by our technical ability and access to resources. You could have a great idea in your head, but whether you could execute it or not depended on whether you were able to execute it, whether your technical level allowed you to do so. I could imagine a piece with a bagpipe passage, but if I am not able to play the bagpipes, then if you are a musician with resources, a professional musician with a sufficient budget, you could hire bagpipers, but what if not? You might be lucky enough to meet one who will do you the favour, but … what if not?
Even if you were limited to the instruments you had access to and were able to play, sometimes your own technical inability prevented you from executing the idea you had in your head. Until now, the only solution was to practice until you got it, which was not bad at all because it forced you to improve.
And here a question arises: what is better in this case: to force yourself to practice until you achieve it or to ask the AI to do it for you and lose that opportunity? And this is an issue that worries me. Creativity and skills improve with practice and having a tool that gives you what you need immediately and effortlessly can make you involute as an artist and as a person. An example of skills that are lost if they are not practised. When I was young, I didn’t have a tuner and I had to tune to the landline tone which is a standard ‘A’ at 440 Hz. I would pick up the landline and use that signal to tune the fifth string of the guitar by ear and from then on all the other strings, and thanks to having to do it that way, I developed a very good ear for recognising notes. When I wanted to play a song, I would listen to it a couple of times and I knew what chords and notes were being used, but one day my father turned up at home with an electronic tuner and I stopped using the phone as a reference. The result was that I gradually lost the ability to pick out the songs I liked by ear.
Our abilities improve with practice and are lost if they are not practised often enough. If we leave the creative work to a machine that gives us the desired result immediately, we can’t do it.
