Organizers had defended Sano's stylized "T" as recently as last Friday against accusations of copying from Belgian designer Olivier Debie, who had designed a similarly stylized logo for the Theatre de Liege in his country.
Sano denied the allegations, saying he had never seen the Belgian design and that the red circle in his design was meant to evoke the Japanese flag.
But with criticism mounting, Tokyo organizing committee head Toshio Muto announced at a press conference Tuesday that the logo had been withdrawn by Sano. He said the committee would "respect the decision" and develop a new design.
"Using the logo that is not supported by the public is not quite suitable (for the success of the Olympics)," he said.
Muto added that he hoped a new logo, "loved and supported by the public," would be chosen soon.
In the meantime, the Tokyo Olympics would use a logo from a past bid.
Designer issues apology
Kenjiro Sano, who did not appear at the conference, posted an apology on his website on Tuesday.
While insisting he "never copied or plagiarized" the design, Sano said he "failed to properly handle jobs other than the logo."
That referred to accusations aimed at some of his other designs, including a tote bag found to have used others' images without permission -- a controversy that emerged as Internet users began to dig into Sano's previous work.
Sano said he made the decision to scrap the Tokyo logo in part to "protect his family and staff from criticism and bashings," adding "I have reached my limit where I can no longer endure as a human being."
This isn't the first controversy to hit the Tokyo Olympics.
In July, Prime Minister Shinzo Abe announced the country would scrap plans for an expensive new stadium
for the 2020 showpiece.
Abe said the design was dropped due to skyrocketing costs, which rose to 250 billion yen ($2.02 billion) at one point.
The design, by Iraqi-British architect Zaha Hadid, had drawn scathing criticism for its apparent likeness to a bike helmet or a toilet seat.