Creating Songs You’ve Never Heard Before Through Evolutionary Computation

Everyone has songs they love: when they want to calm down, cheer themselves up, or concentrate. Music stays close to our emotions in each moment and moves our feelings. On the other hand, even songs we love can begin to lose their freshness after we listen to them again and again. What if it were possible to create a brand-new song that you had never heard before, yet matched your tastes and current mood perfectly? By selecting just a few favorite songs that make you feel “healed” or “energized,” the system captures the characteristics they share and generates a new piece just for you. Professor Noriko Otani of the Faculty of Media Information Studies is working on research into this kind of automatic composition, one that stays close to each individual’s sensibilities.

Many of the generative AI systems widely used today rely on a technology called deep learning, learning features and regularities from massive amounts of data. In the case of music, they generate new pieces based on enormous collections of songs. By contrast, the automatic composition system developed by Professor Otani and her colleagues requires only two or three songs. For example, when someone selects a few songs that they feel make them more energetic, the system extracts features such as key, tempo, chord progression, melodic movement, and rhythm. It then creates a new song the person has never heard before while reflecting those features. One distinctive point is that it does not require large amounts of data or high-performance computers, and can run on an ordinary personal computer. “The target of our work is a specific individual. Instead of preparing large-scale servers or tens of thousands, hundreds of thousands of songs, we use the kind of computer many people already have, extract features from just two or three songs that person likes, and create a new song.”

The composition itself is handled by “evolutionary computation,” Professor Otani’s area of expertise. Evolutionary computation is a general term for methods that imitate the mechanisms of biological evolution on a computer in order to search for optimal solutions to difficult problems, and it includes various approaches. One representative method is the “genetic algorithm.” As the name suggests, it imitates on a computer the process by which living organisms evolve over generations. First, many candidates that might serve as answers to a problem are created at random, and each is evaluated according to how well it fits the conditions. Candidates with high evaluations are retained, combined, and partially altered as generation after generation is repeated, gradually moving closer to a better answer. In automatic composition, the system first prepares many pieces by randomly arranging notes, then evaluates whether they satisfy minimal music theory and how much they contain the features of the input songs. By repeatedly changing generations while preserving highly rated pieces, it searches for a new song that matches the person’s sensibilities. Even when the same songs are input, the initially generated candidates are random, so the resulting composition differs each time.

Professor Otani uses a method called “symbiotic evolution,” a type of genetic algorithm. The problem is divided into several parts, and two populations evolve simultaneously: one that searches for good parts themselves, and another that searches for good combinations of those parts. Applying the same idea to music, the system treats a two-measure unit as a “motif,” a building block of a piece, and searches both for good motifs and for the best order in which to combine them into a complete song. Professor Otani compares this mechanism, in which parts and the whole evolve at the same time, to a sports team. “To win in baseball or soccer, each individual player needs to be excellent. But simply gathering excellent players does not necessarily mean the team will win. The combination matters too.”

When she first began this research, Professor Otani believed that good music could not be created unless music theory was followed strictly. She read specialized books and spoke with collaborators who were experts in music. In the world of engineering, definitions and rules are clear. In music, however, there are many exceptions: “basically this is the rule, but sometimes it is not.” There are songs that move people’s hearts even when they deviate from theory. This led her to shift her thinking. She decided to keep the music theory built into the system to the minimum necessary, and beyond that to emphasize the features of the input songs. “At first, I thought we had to follow music theory properly. But there are many wonderful songs that do not follow it. So I came to think that we should observe only the minimum rules, and use the features of songs that the person likes as clues.”

This mechanism has also been applied not only to full musical pieces, but also to corporate “sound logos.” Employees of the client company are asked to freely sing the company name with their own melodies, and those voices are transcribed into musical scores. Features such as key, tempo, syllable placement, and melody are extracted from each person’s way of singing, and evolutionary computation is used to create a new melody. Unlike something composed from scratch by a specific composer, the result is a one-of-a-kind sound logo that reflects the image of the company held by the people who work there, as well as the characteristics of their singing.

In recent years, she has also been working on research that connects composition with everyday activities. The starting point was coffee, which Professor Otani herself loves. In hand-drip coffee, there are steps for brewing delicious coffee, such as waiting for a fixed amount of time after pouring hot water. However, it is surprisingly difficult to measure the time while following each step. So she created music that allows people to brew coffee at the proper timing naturally, without looking at a clock, simply by working in time with the progression of the song. The same idea has expanded to music that supports tooth brushing, making cup soup, and interval walking. Instead of treating listening to music itself as the goal, the idea is to use music as a tool to make ordinary everyday actions more enjoyable and easier to perform.

The applications of evolutionary computation are not limited to music. For example, Professor Otani has also worked on research to find delivery sequences for multiple households that minimize carbon dioxide emissions, rather than simply minimizing travel distance. As a vehicle unloads packages, it becomes lighter and consumes less fuel. Should heavier packages be delivered first, or should distance be prioritized? Evolutionary computation is used to search for better answers among these intricately intertwined conditions. She has also conducted research on policies to reduce future municipal spending in regional cities facing population decline. The same technology of “searching for a better answer” is being applied to problems that may seem unrelated at first glance.

At the root of this wide-ranging research is “analogy,” a subject that has interested Professor Otani since her student days. There are moments when an unfamiliar mechanism suddenly becomes understandable by replacing it with something already known. At university, she studied how such “comparisons” affect human learning. After completing graduate school, she worked at a company on information retrieval research. Later, when she returned to university, her mentor suggested that evolutionary computation, which reproduces biological evolution on a computer to search for answers, might be the “ultimate analogy.” Humans take mechanisms found in nature and transfer them to computers, then apply those mechanisms to music, logistics, and urban issues.

Now that generative AI has spread rapidly and we live in a society where anyone can use artificial intelligence, Professor Otani believes what matters is the knowledge and judgment of the humans using it. In her laboratory, she values the attitude of “first moving your own hands and trying it.” When something is inconvenient, students do not simply look for an available service; they make it themselves. Rather than feeling as if they understand something through dialogue with AI alone, they actually build, run, and fail. Only through that process do new questions come into view. In the laboratory, programs developed by the students themselves are used for checking schedules, requesting proofreading of papers, managing books, and other tasks. At the annual summer camp, teams develop systems, and the most user-friendly one is adopted for actual laboratory operations. Instead of writing programs only for research, students find inconveniences close at hand and use technology to solve them. That experience also becomes practical learning for them. “I am not denying the use of generative AI itself. But if you are going to use it, I think it is meaningless unless you first think properly about the right way to use it.”

What Professor Otani aims to create is artificial intelligence that can generate new songs suited to each individual’s sensibilities, or add one more option that humans might never have thought of. The optimal answer is not waiting somewhere from the beginning. It is found by trying and combining various possibilities, and sometimes by causing mutation. That process overlaps with the way we discover new joys and insights in everyday life. Just as living organisms have evolved, the answers that can enrich our lives may also be sleeping among countless possibilities.