fighters entered Tadmur, the Arabic name for the modern city with thousands of residents at the Palmyra site, after days of fighting with Syrian government forces in the area over the past week, the London-based Syrian Observatory for Human Rights said.
Militants controlled most of the city by the evening and had taken over Tadmur's prison, the observatory said. Meanwhile, Syrian government forces reportedly have retreated to the city's security headquarters, and clashes with ISIS are ongoing.
Palmyra's centuries-old remains of temples and other structures are a UNESCO World Heritage Site
in the Homs countryside northeast of Damascus.
UNESCO Director-General Irina Bokova said she was deeply concerned about reports of the battles.
"The fighting is putting at risk one of the most significant sites in the Middle East, and its civilian population," she said.
"I reiterate my appeal for an immediate cessation of hostilities at the site," Bokova said. "I further call on the international community to do everything in its power to protect the affected civilian population and safeguard the unique cultural heritage of Palmyra.
"Finally, it is imperative that all parties respect international obligations to protect cultural heritage during conflict, by avoiding direct targeting, as well as use for military purposes."
The Islamist militants of ISIS have captured parts of Syria
in campaigns of violence and barbarity, slaughtering not only those who fight them but also conquering civilians who don't adhere to their extreme brand of Islam
As it has conquered territory, it has destroyed archaeological sites, claiming that it considers all religious shrines idolatrous.
Syria's state-run news agency, SANA, reported that heavy clashes were taking place in Tadmur's northern neighborhoods, but that security forces repelled ISIS in the rest of the city.
'Would be a loss for the entire world'
Between the first and second centuries A.D., Palmyra "stood at the crossroads of several civilizations," with its art and architecture mixing Greek, Roman and Persian influences, UNESCO says.
The city already was a caravan oasis when Romans overtook it in the mid-first century A.D. Its importance grew as a city on the trade route linking the Roman Empire to Persia, India and China, UNESCO says.
British historian and novelist Tom Holland described Palmyra as "an extraordinary fusion of classical and Iranian influences intermixed with various Arab influence as well."
The destruction of Palmyra wouldn't just be a tragedy for Syria, it would be a loss for the entire world, he told CNN this month.
"This isn't just about Middle Eastern history; these are the wellsprings of the entire global culture," he said. "Mesopotamia, Iraq, Syria, this is the wellspring of global civilization. It really couldn't be higher stakes in terms of conservation."
This is not the first time ancient sites have been destroyed during Syria's brutal four-year civil war
, in which ISIS is one of the belligerents.
Notable casualties include 11th century crusader castle Crac des Chevaliers; its walls were severely damaged by regime airstrikes in 2013.
Aleppo Souk, a formerly thriving part of Syria's economic and social life, was severely damaged in a fire in 2012.
"What is distinctive and horrendous about (ISIS') mode of operation is that they are deliberately going out of their way to destroy (ancient artifacts)," Holland said.
ISIS propaganda claims the Islamist militants are destroying idols or false gods and following in the footsteps of the Prophet Mohammed, who smashed statues in Mecca.
But ISIS also sells artifacts for huge profits.
"They have networks that allow them to traffic in cultural treasures. They have made tens of millions of dollars selling artworks," said London School of Economics professor Fawaz Gerges, author of the forthcoming book, "ISIS: A Short History."