Kiir had earlier demanded an immediate end to the fighting between his soldiers and those loyal to the rival vice president. The civil disputes have left more than 150 dead across the capital city of Juba since fighting broke out on Thursday.
"I'm informing all the troops that have been fighting to respect the ceasefire and to remain where they are now," Machar said on Eye Radio, a U.S.-funded radio station that broadcasts in the country.
Despite an apparent halt in fighting in the capital on Tuesday, the Red Cross said that food and other supplies are running low for tens of thousands of citizens taking shelter at places like humanitarian centers and churches.
"The needs are enormous," said Jurg Eglin, head of the Red Cross mission in the country. "... It is by no means back to normal."
A representative of the president said that the ceasefire was holding Tuesday, and that Kiir "doesn't want the peace to collapse."
"All the soldiers have returned to their barracks," besides those who are manning checkpoints around government institutions and other important areas, said Kiir's representative, Ateny Wek Ateny.
Following an overnight lull, fighting had resumed Monday morning with the sound of gunfire blasting through parts of Juba, according to Shantal Persaud, acting spokeswoman for the U.N. Mission in South Sudan. Should the truce fail to hold, fighting threatens to throw the country back into civil war.
Fighting first broke out Thursday with skirmishes between the Sudan People's Liberation Army, loyal to Kiir, and soldiers backing Machar. A resurgence Sunday and Monday came after a halt when the country celebrated the fifth anniversary of its independence from Sudan on Saturday.
The U.S. State Department announced on Monday a reduction in staff at the U.S. Embassy in Juba as bloody violence in the capital spirals, leaving scores of people dead, including two Chinese U.N. peacekeepers.
The United Nations Security Council, which held a closed door meeting Sunday, expressed "shock and outrage" at attacks on civilians and U.N. compounds, saying they may constitute war crimes.
An estimated 83,000 people are seeking refuge in churches, schools and outside the peacekeeping base of the U.N. mission, according to the organization.
The Red Cross and its teams managed to deliver food to about 3,700 sheltering civilians and patients early this week, as well as medical supplies and drugs to two hospitals for the injured -- but relief workers have felt free to move only in small parts of the city and much more needed to be done.
"We are starting to respond, but these are drops into a ocean of big needs," Eglin said.
Earlier this week, humanitarian workers were stopped and threatened at checkpoints when trying to deliver assistance, unable to reach the most needy, said Chaloka Beyani, the U.N. special rapporteur on the human rights of internally displaced persons.
International law requires that parties in conflict allow humanitarian workers to have safe passage, Beyani said Monday. The threats to humanitarians in the western city of Wau are unconscionable, his office said in a statement Monday. Two weeks ago, fighting in Wau displaced at least 70,000, according to the United Nations.
Relief groups reducing staff in Juba
How many have been killed in fighting between the factions is unclear. One estimate puts the death toll close to 150, while other reports indicate more than 270 have been killed. CNN is working to confirm an exact death toll.
The United Nations said it was deeply concerned over reports that civilians were prevented from seeking refuge at U.N. compounds and "protection of civilians" sites.
About 7,000 civilians have sought refuge at U.N. sites in Juba, but eight people were killed and 59 others were injured at or near the sites in the last week, the United Nations said.
Even though the humanitarian need is great, some relief organizations said Tuesday they were evacuating some of their staff members from Juba because of security fears, chartering planes for them at the city's airport.
The International Medical Corps' hospital in Juba was hit by shelling Monday, injuring no one but forcing workers to relocate critical patients to another facility, the group said.
"The hospital has been without water for three days and is running out of fuel, which will leave us without power or an ambulance soon," IMC spokeswoman Rebecca Gustafson said Tuesday. "We have been unable to move around the city because of the fighting so our hospital is running with fewer staff, and our medicines and medical supplies are slowly dwindling."
The IMC is relocating some staff to places outside South Sudan to support the group's efforts remotely, "but a core team will remain in country so our lifesaving work can continue," Gustafson said.
The Red Cross, Medicins Sans Frontieres and the International Organization for Migration also saidTuesday that some of their workers were leaving Juba because of security concerns, but each group said they were retaining some staff in the capital.
'Sudden and serious'
The U.S. State Department said the security situation in Juba on Sunday had seen a "sudden and serious deterioration," with clashes between government and opposition forces breaking out into "general fighting."
Indian External Affairs Minister Sushma Swaraj said in a tweet that Indian nationals were being evacuated and warned others not to travel to the country. There are many Indian nationals working at the U.N mission there.
The Ugandan military is also working on a plan to evacuate its citizens, Lt. Col. Paddy Ankunda told CNN. They are still assessing the security situation, he told CNN.
Koro Bessho, Japan's ambassador to the United Nations, confirmed the death of a Chinese soldier, while Chinese state broadcaster CCTV reported the death of a second Chinese peacekeeper.
The country is nearly out of money because its funds come almost exclusively from oil revenue -- the value of which has plummeted. People have become desperate. In lieu of payment, government soldiers have reportedly been allowed to rape women, a U.N. report said
South Sudan gained independence in 2011 after 98% of the population voted to break away from Sudan. The East African nation quickly fell into civil war that took on ethnic undertones.
In December 2013, soldiers from Kiir's Dinka ethnic group tried to disarm Nuer soldiers perceived to be loyal to Machar. Soldiers targeted Nuer civilians in the ensuing fighting, Human Rights Watch said.
At least 50,000 were killed and more than 2 million displaced in the civil war, and nearly 5 million people faced severe food shortages.
Under a peace deal signed in August, Kiir became the president of the country, and Machar the first vice president.
Kate Almquist Knopf, director of the Africa Center for Strategic Studies, said last week that the recent violence may represent "a total breakdown of command and control in Juba."
"We need to watch carefully for whether a cycle of reprisal killings by either side begins in the next few days," she said.