CNN  — 

President Donald Trump tweeted Sunday morning that he has ordered for the National Guard to start the process of withdrawing from Washington, DC, after days of peaceful protests.

“They will be going home, but can quickly return, if needed. Far fewer protesters showed up last night than anticipated!” Trump tweeted.

Roughly 5,000 National Guard troops had been called to patrol the nation’s capital – 3,900 out-of-state National Guard troops from 11 states, and 1,200 DC-based Guard troops – – amid ongoing protests over the police killing of George Floyd, a black man who died at the hands of a white officer in Minneapolis.

Trump did not specify in the tweet whether he meant all National Guard forces should be withdrawn or if he was just referring to out-of-state troops ordered to leave. The presence of those out-of-state forces had been a point of contention between Washington officials and the administration.

Secretary of the Army Ryan McCarthy said Sunday that out-of-state National Guard would start going home at 5 p.m. ET, on Sunday. The plan to withdraw the out-of-state Guard forces was drawn up over the last 48 hours given the peaceful nature of the most recent demonstrations.

The Pentagon, McCarthy said, did not want to use the active duty troops to quell unrest in the nation’s capital but said that the invocation of the Insurrection Act that would allow for the use of such troops was “heavily discussed” within the Trump administration.

“They were on the outskirts cause we didn’t want to do it. The Department of Defense didn’t want to do it because we knew once we went to that escalation, it’s very, very difficult,” he told reporters on a phone call, referring to the some 1,600 active duty soldiers that had been flown to bases in the DC area.

“We did everything we could to not cross that line,” McCarthy added.

Maj. Gen. William Walker, commander of the DC National Guard, previously told CNN that the out-of-state troops would begin leaving as early as Monday.

In a Friday letter to the President, DC Mayor Muriel Bowser argued that the additional law enforcement are “inflaming” and “adding to the grievances” of people protesting in the wake of Floyd’s death.

“The protestors have been peaceful, and last night, the Metropolitan Police Department did not make a single arrest. Therefore, I am requesting that you withdraw all extraordinary law enforcement and military presence from Washington, DC,” Bowser wrote in the letter, adding that she had ended the state of emergency in DC related to the protests.

Trump attacked the mayor in response, claiming in a Friday tweet that she was “fighting” with the Guard and warning that if she didn’t treat the service members “well” he would bring in a “different group of men and women.”

The following day, Bowser joined thousands of protesters on the streets of the nation’s capital on Saturday.

“We should all be watching what’s happening in Washington, DC, because we don’t want the federal government to do this to any other Americans,” Bowser said while walking through downtown DC with protesters on Saturday.