MARDAN, Pakistan (CNN) -- A family of 18 Pakistani men, women and children trudges down a dirt road toward a refugee camp.
These children are among the thousands of refugees this week at the Jalozai camp in western Pakistan.
Adolescent girls carry infants on their hips, while the men lug bundles of belongings on their backs.
"Come, stay close to me," said one woman wrapped in brightly colored robes, speaking to three children trailing behind her.
"This one is empty," a white-bearded Pakistani police officer tells the family, pointing toward a tent.
The women and children scramble under the canvas flap, as Salar Khan explains what led his family to flee to Mardan.
"Mortars destroyed three houses in my village," he said. "It was dangerous. A piece of shrapnel almost pierced my child's leg."
Khan said his family left their home Wednesday morning in Sultanwas, a town in Buner district.
Now, they are living in Mardan's rapidly growing tent city of more than 1,400 other displaced Pakistanis. Five days ago, it was an empty field. Watch as CNN's Ivan Watson tours a refugee camp »
Khan's family has joined tens of thousands of other Pakistanis fleeing south to escape the escalating conflict between the military and Taliban militants in northwestern Pakistan. Meanwhile, columns of Pakistani troops in military trucks head in the opposite direction, hauling field guns north toward the conflict zone.
Pakistani families have fled the area any way they can: on foot, by hitching rides on the back of trucks and by stowing their belongings on the roofs of cars.
As fighting has spread from the districts of Buner and Lower Dir to the Taliban stronghold in the Swat Valley, camps for displaced people are cropping up across northwest Pakistan.
The United Nations said the new exodus is exacerbating an already existing humanitarian crisis. Since August, the U.N. has registered more than 500,000 Pakistanis forced to flee their homes by fighting in other northwestern parts of the country.
"Last year ... 4 million people worldwide lost their homes, out of which you have half a million displaced in Pakistan," said Manuel Bessler, a top U.N. official in Islamabad.
Bessler spoke on a rooftop, overlooking the sprawling Jalozai refugee camp in western Pakistan.
Until recently, the camp housed refugees from neighboring Afghanistan. The Afghans are now gone, replaced by more then 49,000 Pakistanis.
Administrators are preparing space for 35,000 others. With help from U.N. agencies, the Pakistani government and other aid organizations, residents get access to medical care, children's schools and training programs to teach them how to rebuild their damaged homes if and when they get to return.
Tensions have been building in the Jalozai camp. Two months ago, Pakistani police shot and killed one demonstrator after residents protested, blocking roads, throwing stones and demanding compensation for homes damaged by the fighting.
This week, a crowd of several hundred agitated men gathered at the entrance, angry about a delay of several days in the monthly distribution of food aid. Some accused camp administrators of corruption, allegations that aid workers have denied.
"The wheat we've been given is substandard, and people are getting sick instead of being fed," said one man named Gulzada.
"Our houses have been destroyed," said another man called Anwar. "There's no tea, no sugar, no wheat, no lentils. All that we have are the clothes we are wearing."
A fresh wave of displaced Pakistanis will only aggravate tensions, said Bessler, the U.N. official.
"This is a factor that is destabilizing not only in the camp but in the country as a whole," he warned.
Only a fraction of the hundreds of thousands of displaced Pakistanis are ending up in camps. Many more have settled with host families or have resorted to paying rent in other cities.
The influx of ethnic Pashtuns from northwest Pakistan upset the delicate demographic balance last month in the port city of Karachi. That led to ethnic clashes between Pashtuns and the resident Muhajir community, resulting in the deaths of more than 30 people.
Many more Pakistanis are unable to leave the conflict area, according to Sebastian Brack, a spokesman for the International Committee of the Red Cross in Islamabad.
"There is a serious humanitarian crisis under way," Brack said. "There is serious fighting going on. There will be massive displacement. Because of the curfew, [many] have not been able to leave yet."
In this moment of crisis, some homeless Pakistanis are turning to a higher power.
"Whenever it is God's will, we will go back to our homes," says Mohammed Munir, an elderly man who fled with his family from the Buner district to the new camp in Mardan three days ago. "And we pray to Allah that he will protect us. It's up to Allah. We can't do anything."
The man kneeled and prayed in the grass outside the entrance of a tent that his family now calls home.
All About Pakistan • The Taliban • United Nations