Humans teach AI to learn, and AI gives back to humans to create inspiration.
Palestrina PalA Java-based analysis tool helps students detect grammatical, stylistic and rule inconsistencies in imitating Renaissance composer Palestrina's style.
After the paper was finished, Huang eagerly introduced it to the teachers at the University of Southern California, Thornton Conservatory of Music. She excitedly told them that this tool could help them teach and enable students to improve their imitative composing level better and faster in the process of practice.
But the teachers rejected Huang's idea. They thought that although such tools were useful, they were shortcuts for students, which would affect the training of basic skills. "He said it wasn't us, it was your idea, it wasn't ours. Huang recalls.
But Hang, who has a dual background of computer and music, did not give up because of the teacher's negation. She thought the traditional teaching method was too boring. In the process of learning all kinds of composing rules over and over again, the dull teaching has to some extent eroded the students'inspiration for music creation. Huang believes that cumbersome rule learning and error-checking assignments can be accomplished in a more efficient way through computer-aided learning, and that students should be more directly exposed to more musical things. "What we learn in class, or from composers, is how to express what we want to express, to find out what we want to say. "
As a result, she identified the path of technology-assisted music creation. After receiving his bachelor's degree in computer science and music composition from the University of Southern California, Huang received his master's and doctoral degrees in related fields in MIT Media Lab and Harvard University, and eventually joined Google Brain.Magenta TeamHe is engaged in the research of machine learning and music creation.
First AI Doodle
The First Bach Birthday Commemoration Doodle Based on AI Technology。 Behind this 48-hour Doodle landing on Google's home page is the machine learning algorithm model that Huang started studying when he joined Google Magenta three years ago.Coconet。
The animation effect of Bach Doodle Google
The interactive Bach Doodle is more like a game. On the five-line spectrum, the user clicks on the mouse to compose a piece of music at will and clicks on Harmonize. After a few seconds, the machine learning algorithm behind Doodle will generate a Bach-style Concerto based on the melody just entered by the user.
The idea of creating a Bach creative Doodle originally came from Doodle's team. A year and a half ago, the Doodle team wanted to create a creative and interactive music theme. They chose Bach, a famous German composer. "Bach is a composer that everyone admires very much. If we can create with Bach, it will be a wonderful process." Huang said to the geek park. With this in mind, Doodle's team has found a Macinta team dedicated to machine learning and artistic creation within Google, which coincides with Huang, who is using Bach music to train Coconet's multi-functional model.
Huang joined Google Brain's newly established Agnta team as an intern three years ago to study single-line melody creation. But Huang believes that music should be created in parallel with multiple lines, which is a subject that needs to be overcome. Although the problem was not solved by Agita at that time, the research has begun to take shape. Huang then ended her internship in Magenta and visited the Montreal Institute of Learning Algorithms (MILA) at the University of Montreal as a graduate student. A year later, she returned to Magenta as a resident AI researcher and continued Coconet's research.
It was also at this time that Doodle's team came to the door and decided to work with the Maagenta team to build this interactive Doodle commemorating Bach. For them, the biggest problem is how to make the long-term research projects available to ordinary users, providing a simple and easy interactive experience. It took Coconet about 45 seconds to generate two bars of music on a computer, which obviously could not meet the needs of users. So the Doodle and Maagenta teams adopted TensorFlow. js, a machine learning acceleration architecture that runs on front-end browsers, from Google PAIR. After adjusting the adapter, Coconet runs at a speed ranging from 4 to 8 seconds, with a minimum of 2 seconds. In addition, in the case that TensorFlow.js cannot be enabled, Bach Doodle will use the machine learning processing unit TPU developed by Google to speed up the processing as much as possible.
The Model of 306 Music Training
Machine Learning Assisted authoring is not a new thing. In the past few years, many research institutes and technology companies have released AI applications similar to script writing and news writing. But unlike music, there is only one single line in linear narrative, and music is multi-line parallel. In this multi-line parallel structure, Bach's works are the most typical and basic samples.
"Bach can teach us how to connect these lines. Each line has its own personality, but they go together in harmony. Every student who studies music starts from here, so we also begin to teach AI how to create from here. "Huang explained.
Bach Doodle interactive interface Google
Magenta's team chose Bach's 306 hymns as training data for the Coconet model. "Bach hymns are mostly four parts, each part has its own melodic line, and when played together, they can create rich harmony effects. This concise structure makes it a good training data for machine learning model. Lauren Hannah-Murphy, project manager of Bach Doodle's Google AI teamIntroducing blogHe wrote.
No matter Bach's works are suitable for training, it is almost impossible to build a reliable machine learning model with only 300 pieces of music data volume. Huang knows this very well, so in view of the lack of data, they chose a more sophisticated training method.
Google CEO Sundar Pichai forwards and shares a screenshot of Bach Doodle Twitter
Traditional machine learning algorithm models are trained by feeding data into the model at one time. This learning process is simple, but the result of output is that the model can only understand how the work smoothly transits from the beginning to the end, but can not deduce the content from the end to the middle.
Coconet's training method is to erase a part of a complete music, leaving the AI model to calculate the middle part according to erasing the contents of both sides of the part, and then compare it with the original one, and correct it a little until it is closest to Bach's style. In this way, 300 pieces of music can produce more training samples.
"It's like a lot of times when we write music or articles, not just from beginning to end, but there are many revisions, many beats, and the way this model is created is the same," Huang said. "First we write a rough draft, then we slowly decide on the details of each place, and then we revise it. So it can work with you and modify each other. "
Bach Doodle Static Map Google
The Technical Nature of Music
Of course, Coconet can only produce short melodic chords, which "can't see" long lines of structure, such as deep ideas that musicians want to express in the music, which is the part of machine learning at this stage that can't be perceived anyway.
So when it comes to the limitations of technology, Houang, who has studied computers and music for many years, knows better than anyone else. From the very beginning, what she wanted was an assistant composer and a tool to provide more inspiration.
Asked whether the tools behind Bach Doodle are more suitable for experienced composers or weak beginners, Huang said that different people use tools in different ways. For beginners, simple tools may enable them to discover their hidden musical talents and thus enjoy music and embark on this path. But for experienced composers, Coconet will offer some ideas that are different from the composer's own. After entering original music materials, Coconet may be able to harvest "something very different".
Anna Huang's introduction page on Google AI
"Those different things, I think, are somewhat precious, because many times the composer is afraid, afraid that one day he suddenly lost his inspiration. Huang said that models like Coconet provide composers with a source of inspiration and an additional choice.
Huang grew up in Hong Kong and received strict art education. Often she hated music because of the dullness of the instrument examination. But once she watched a zither show, she fell in love with the instrument. "This is what I want. It's the way I feel I can express my emotions naturally." "Over the next few years, the study of Guzheng made Huang gradually begin to compose his own music, but she soon found that learning to compose was a very difficult process. At first, she did not know much about chords, so she learned very slowly. In a few months, only a few chords were learned, so the possibility of music creation was limited.
Comparatively, computer is an inherent skill for Huang. Her father is a university computer professor. For her, it is "something you can explore without going to school." As her understanding of computer and music deepened, Huang began to think about doing something with computer in music. "Computers can bring me closer to music. "Music is very emotional, but it has a rational side and a very technical part," she said. When learning music, you also need to learn some techniques, which can make it easier for you to find what you want to express. "
"Just like learning a foreign language, it can help you run faster. At first, you don't know how to express it. It can help you do better. It can make you feel more interesting, fun and want to express it. "
Topic Source: Google Official