A lone protester stood outside City Hall at one of Charleston’s best-known intersections Monday, holding signs calling Mayor Joe Riley a racist and offering his views to passing tourists in horse-drawn carriages.

Jerome Smalls, 62, is a founder of People United to Live and Let Live, and a longtime protester. He picketed Riley’s re-election headquarters in 2007, and the campaign office of then-Councilman Arthur Christopher in 1983, for example.

Smalls claims Riley’s policies played a large role in the demographic changes seen on the Charleston peninsula, where census data shows the black population dropped by 63 percent from 1980 to 2010. During those years the white population increased by 37 percent, switching the peninsula from a large black majority to a population two-thirds white.

“He’s just so racist that he doesn’t think anything is wrong,” said Smalls, who said on one of his signs that Charleston’s 10-term mayor “brings out the Denmark Vesey in me.”

Ironically, Riley has supported fundraising efforts to create a new memorial in a city park to Vesey, who was hanged in 1822 — wrongly, after an unfair trial, some say — for plotting a slave rebellion in Charleston.

Riley said he’s familiar with Smalls and his criticism. The mayor said the city has produced or aided the production of thousands of affordable housing units.

“Any city is always dynamic, and subject to free enterprise, and if people are selling homes and moving to other neighborhoods, that’s a normal activity that goes on,” Riley said.

Rising downtown rents, along with a broad national trend of blacks moving to suburban areas while whites return to city centers, have often been cited as factors on the peninsula.

Some, including Smalls, also cite the 1992 demolition of the publicly owned Ansonborough Homes housing project, on what is now Concord Park. The project was flood-damaged by Hurricane Hugo in 1989, then demolished in 1992 because of pollution discovered in the soil — the area became a federal Superfund site — but Smalls says that was a lie created to move blacks out and build the South Carolina Aquarium.

The city is currently pursuing a long-standing plan to redevelop portions of Concord Park, which includes creating 62 units of subsidized housing.

“The important thing is, this is America and that man can stand on the corner protesting,” Riley said.