That description isn't fair, Temer told CNN in an exclusive TV interview on Monday.
"There isn't a coup in this country. There isn't any attempt to violate the Constitution," Temer said, adding that 62% of Brazilians are in favor of Rouseff's impeachment.
As Temer spoke with CNN from Jaburu, the heavily guarded vice presidential palace designed by famed Brazilian architect Oscar Niemeyer, there was a constitutional law book open on his desk.
"What conspiracy am I leading? Do I have the power to convince 367 members of congress? More than half of Brazil's population," he said. "I think the President is wrong on this point as well."
Rousseff and Temer, a 75-year-old attorney known as a backroom negotiator, forged an alliance with support from his Brazilian Democratic Movement Party. Together, they rose to power with the blessing of Rousseff's mentor, former President Luiz Inacio Lula da Silva.
But that alliance is the latest casualty in a rapidly fracturing political landscape as a growing political scandal shakes Brazil's government
just months before the country is set to host the Olympics
His plan: Getting Brazil 'back on track'
Temer is the man who will take the reins in Brazil for at least 180 days while Rousseff defends herself if the country's Senate votes in favor of holding an impeachment trial.
If Rousseff is found guilty, Brazilian law says Temer must fulfill the remainder of her term, which runs through the end of 2018.
"My proposal is one of national reconciliation, of national pacification," Temer told CNN. "I've already said this many times, but what I propose is a government that could act as a national salvation. That would unify all the parties, including those on the opposition side. I think this is the only way to get out of this crisis. ... My goal will be, with the support of the country's political forces, to form a good Cabinet to advise me and guarantee governability, to help the economy recover (and) put the country back on track, for a smooth election in 2018."
And what about the Summer Olympics, which Rio de Janeiro is set to host in August? Could the Games take a hit as the country faces chapter after chapter of a deepening political crisis?
"I don't think so," Temer said. "You know that the Olympic Games are being very well organized as a joint effort between the Rio state government, the Rio municipal government and with the participation of the federal government. I have no doubt the Olympic Games will be a success, just like they have been in other parts of the world."
Rousseff is a highly unpopular President with an approval rating near 10% thanks to the worst recession in decades and a bribery scandal involving state-run oil company Petrobras and dozens of politicians in her party and governing coalition.
The President herself is not implicated in the scandal, but millions of Brazilians have taken to the streets to demand her ouster over the institutional corruption and tanking economy. Last week the country's lower house overwhelmingly approved a motion for her impeachment
. She's accused of breaking budgetary laws by borrowing from state banks to cover a shortfall in Brazil's deficit and pay for popular social programs.
Part of the challenge for Temer will be that many politicians from his party have also been charged in the bribery scandal. Temer is not under investigation, but his name has been mentioned in some of the testimonies.
He told CNN that he had nothing to do with the bribery scandal.
Temer's first step if the impeachment process goes forward?
"I want to regain the trust of the Brazilian people and all the sectors of society," he told CNN.
But that's no easy matter.
A majority of Brazilians have also said they think the vice president should also be impeached and would prefer new elections.
Temer said his country's constitution doesn't allow its vice president to be impeached. But he knows people are closely watching the steps their government takes.
"I am aware that if I do become the President, I, too, could be processed for any political wrongdoing," he said.