Real, artificial brains make magical music

Bill Manaris (left) and Yiorgos Vassilandonakis

Wade Spees // The Post and Courier

Performing arts students Chee-Hang See (left) and Amy Tan play the keyboards while two computerized Monterey Mirrors capture and respond to the musicians’ styles in the Simons Center Recital Hall at the College of Charleston.

Pianists Amy Tan and Chee-Hang See often perform with other musicians, but this week they took the stage with two laptop computers.

In a campus concert, the College of Charleston performing arts students demonstrated the Monterey Mirror, an interactive music performance system developed by a team led by computer science professor Bill Manaris.

Tan and See, both seniors, played something on their keyboards while the computers "listened." Then the computers briefly "thought" about what the human musicians had played, captured their styles and played back something that was similar, although not exactly the same.

Tan, See and their computers created an interactive performance, merging human skill and talent with artificial intelligence.

"It's like a whole different instrument," Tan said of the Monterey Mirror.

Manaris said the system is an example of how computer science can be merged with the arts. The college this fall launched a new computing and the arts major, he said. Already, more than 30 undergraduate students have signed on.

Chris Starr, chairman of the college's computer science department, said the department is trying to engage more students in computer science by merging it with other disciplines, such as business, analytics and music.

"We're attracting a new kind of student that is technically competent and creative," Starr said, "not just one or the other."

He also said he thinks computer science now is a "foundation discipline," much like literacy was in the 19th and 20th centuries. "There isn't a single discipline that doesn't have a software competency aspect," Starr said.

It's not only the students who are crossing disciplinary lines. Manaris is working with music professor and composer Yiorgos

Vassilandonakis to stage performances using the Monterey Mirror.

Vassilandonakis composed Tan's and See's performance. "The piece was about interacting," he said. "It was a low-level competition."

Composing is a challenge when the Monterey Mirror is involved, he said, because nobody knows exactly what the computer will produce. But it works, he said. "It's kind of like a script. The material is the same, but the essence is different."

Vassilandonakis said he has always been interested in computers. And Manaris also is a musician who plays the guitar.

Manaris, who is from Greece, said, "I came to this country with two suitcases and a guitar." He pursued a career in computer science, he said, but the College of Charleston, which is a liberal arts school, allowed him to merge his interests.

The computer science field traditionally has focused on the less creative "left brain," he said. "We left the right brain less explored." He's trying to change that.

Manaris already is thinking about the next steps for the Monterey Mirror.

Now, a person playing a musical instrument can use the system to jam with a computer. But he's working on adapting the system to the human voice.

He's also looking into programming it to capture and reflect the styles of famous musicians, he said. "You could be jamming with Bach."