"We really hold the military leadership accountable for what's happening with the Rakhine area," Tillerson told an audience in Washington, referring to the area in western Myanmar where thousands of Rohingya have been killed or forced to flee in recent months.
"What's most important to us is the world can't just stand idly by and be witness to the atrocities that are being reported in that area," he said.
According to the United Nations, hundreds of thousands of Rohingya Muslims -- many of them women and children -- have fled to Bangladesh since the outbreak of violence at an average of almost 20,000 a day.
In a report released last week, the global body said the actions taken against the Rohingya have been "well-organized, coordinated and systematic, with the intent of not only driving the population out of Myanmar but preventing them from returning to their homes."
The United States has accused the security forces in Myanmar, also known as Burma, of conducting a campaign of violence against the Rohingya ethnic group after a series of alleged attacks by Rohingya militants on government border posts.
Myanmar's government has blamed terrorists for the violence that has led to the current humanitarian crisis.
Rohingya Muslims, a minority group inside the predominantly Buddhist country are considered to be among the world's most persecuted people. Straddling the border between Myanmar and Bangladesh, the Rohingya are effectively stateless as the two governments maintain their rightful home is in the other country.
Nobel laureate Aung San Suu Kyi, Myanmar's de-facto leader has been criticized for failing to effectively stop the violence, but the country's military continues to wield considerable power within the country after the former military junta that formerly ruled Myanmar permitted elections in 2015 and a return to civilian rule.
The military still retains a considerable amount of influence within the government since 25% of the seats in the country's parliament are reserved for them.
Refugees International described the actions of the military in Myanmar as "crimes against humanity" in a report released earlier this month.
Tillerson laid a good deal of the responsibility for resolving the crisis with the military leadership in Myanmar following recent media reports of indiscriminate killing, burned Rohingya villages and thousands of refugee camps situated in squalid and dire circumstances.
"If these reports are true, someone is going to be held to account for that," Tillerson said. "It's up to the military leadership of Burma to decide what direction they want to play in the future of Burma."
The United States announced last month that it would provide a humanitarian aid package
to the Rohingya worth nearly $32 million dollars.
While the State Department has been able to get a few of its personnel from the embassy in Myanmar into Rakhine state to assess the situation, Tillerson encouraged greater access for humanitarian agencies such as the Red Cross, Red Crescent as well as other agencies from the UN.