Merton Simpson, a Charleston artist and musician who rose to prominence in the New York City art world, died Saturday in Manhattan. He was 84.

His younger brother Carl and sister-in-law Harriet Simpson said he had been ill for some years.

Simpson operated his own gallery that featured objects from Africa. African art clearly influenced his own work, which combined elements of abstract expressionism. He was a part of the Spiral collective, a group of black artists striving in the 1960s to reconcile the creative process with the civil rights movement and black identity.

Born in Charleston in 1928, Simpson grew up in a large family that loved good music. He suffered from diphtheria and rheumatic fever as a child and did not begin his schooling until the fifth grade. He became a proficient player of reed instruments and a frequent jazz performer, and by the time he was a teenager he was playing music and making art regularly.

“I was always sort of torn between the two,” he told Al Murray in 1968 for an oral history project run by the Smithsonian’s Archives of American Art.

Simpson learned music at the famed Jenkins Orphanage. “Everybody wanted to go there just because it was such a good place to get involved with music,” Simpson said. “Charleston was at that time a kind of musical center for jazz.”

Still a teenager, hampered by segregation, Simpson drew the attention of William Halsey, an important Charleston painter who ran the Gibbes Gallery at the time.

“He sort of took me under his wing, so to speak, and taught me a bit about drawing, constructing a picture, mixing paints, and the general sort of things needed to be an artist.”

Angela Mack, executive director of the Gibbes Museum of Art, organized a 1995 exhibition of Simpson’s work, and the artist brought his band to the opening to play a set. The Gibbes Museum collection includes a few of Simpson’s paintings.

Mack said it was likely an early job at a framing shop in New York that exposed Simpson to abstract expressionists and other artists active at the time.

“It was through those relationships that he got his feet wet and established his reputation in New York,” she said.

Artist Jonathan Green called Simpson “legendary,” a major figure in the art world, “the sort of person many people in Charleston don’t know about.”

Simpson’s most recent show in South Carolina was the 2011 exhibit at the Greenville County Museum of Art called “Confrontations.” It was a series of large canvases showing pairs of mask-like faces, a reference to the standoffs between police and blacks during the 1960s.

Mark Sloan, director of the Halsey Institute of Contemporary Art, said that series, and others, displayed a jazz-like improvisational and expressive style.

“Some were kind of lyrical, some were sort of brash and noisy, some were harmonious and subtle,” Sloan said. “To me, he was taking the rage that he and others were feeling at that time and channeling it in a really interesting way, a very compelling visual way.”

Sloan recalled sitting once in the New York gallery with Simpson, surrounded by an amazing collection of African art, as the phone rang continuously. Simpson apologized, explaining that he was telephoned by people from around the world, often at odd hours. Then he answered the calls.

“In the span of a short visit I heard him speak four languages,” French, two African dialects and English, Sloan said, adding that a compliment was quickly belittled.

“Oh I just know enough to get around,” Simpson told him.