The government is shutting down the camps because of "very heavy" economic, security and environmental burdens, senior Interior Ministry official Karanja Kibicho said in a statement.
"Kenya, having taken into consideration its national security interests, has decided that hosting of refugees has come to an end," Kibicho said, pointing to threats, such as the terror group Al-Shabaab.
Kenya announced the closure of refugee camps last year
for the same reasons but backed down in the face of international pressure.
At the time, government officials were not clear where they expected the refugees to go, other than somewhere into Somalia and out of Kenya. Kibicho's statement didn't address the question of where the refugees would go.
While it is not immediately clear if or when a closure might happen, the Interior Ministry said it has already disbanded the Department of Refugee Affairs as a first step.
Most of the residents in Dadaab come from Somalia, which has been torn by civil war. The Kakuma camp used to be home for South Sudanese refugees, but in recent years houses Congolese, Rwandans and Burundians.
'I don't think I can go back to Somalia'
Several refugees interviewed by CNN said they didn't want to return to their homeland.
A 20-year-old Somali refugee said living in Dadaab is hard, but leaving will also be difficult
"Dadaab is like being in a cage and now when they ask us to go home ... first of all, which home is that?" he said.
"My parents have been in Kenya for 25 years, I was born in this camp and now I want to join university, which I can't because they have given me an alien card but then they actually closed down the department of refugees.
"My card might not be valid in Nairobi anymore. My movement will be restricted. I don't know what to do but I will see how to go about it."
He said his parents lost all their land and property during the war. Sending people back with $100 or $1,000 as an incentive to start life over again won't work.
"I agree no one has to live in a refugee camp all their life," he said. "It's not correct morally, but then if you've left your home 25 years ago, going back is like going back to a new place."
Shidad Abdinoor, 43, a Somali refugee and father of five at the Kakuma camp, said he may go to another East African nation.
"I don't think I can go back to Somalia," he said. "I hear there is a government, but a government that is bombed and attacked every now and then."
Life in the camp is hard, he said, "but sometimes it would be better to go home than get killed in the camp, a foreign land."
Miyat Chang, a refugee for 18 years, says Kakuma is his home. He also will ask to be sent to another African nation.
"We all came here because home was dangerous for our life," he said. "I cannot go back to South Sudan, even if they force me."
He'd like to become a citizen of Kenya.
"I think If someone stays in a country for long, they can be a citizen," he said.
Amnesty International criticizes decision
The government called on the international community to "collectively take responsibility" for the humanitarian needs of the hundreds of thousands who may be affected by this.
Amnesty International blasted what it called a "reckless decision" by Kenya.
Kenya's decision is "an abdication of its duty to protect the vulnerable and will put thousands of lives at risk," said Muthoni Wanyeki, Amnesty's regional director for East Africa, the Horn and the Great Lakes.
He said Kenya would be violating its obligations under international law.