Armed officers from the Metropolitan Police's counter-terrorism command unit swooped on the suspect Thursday, as part of an "ongoing operation."
The 27-year-old man was arrested at 2.22pm, a few meters from the scene of a terror attack last month
that left four people and the attacker dead.
Witnesses said the man was pinned to the ground on Parliament Street, within sight of the Houses of Parliament and an array of British government buildings. Downing Street, the official residence of the British Prime Minister, is a few minutes' walk away.
Forensics officers examined the scene outside the London Underground station at Westminster and recovered the suspect's backpack and at least two knives.
A Scotland Yard statement said: "The 27-year-old man was arrested in Parliament Street, junction with Parliament Square, by armed officers from the Met's Specialist Firearms Command.
"He was arrested on suspicion of possession of an offensive weapon and on suspicion of the commission, preparation and instigation of acts of terrorism. Knives have been recovered from him.
"He is being detained under the Terrorism Act and is in custody in a south London police station."
Police added on Twitter that the man was "arrested as part of an ongoing operation by the Counter Terrorism Command."
Traffic was briefly halted on Whitehall, the street that leads from Parliament Square to Downing Street.
Eyewitnesses described a large police presence as the man was arrested. Ian Moss, who was traveling on a bus through Whitehall at the time, told the Press Association that the man did not appear to have put up a struggle. "He had police pointing arms at him though."
A CNN journalist nearby saw vehicles backed up along Westminster Bridge, but roads reopened within two hours of the arrest and life in the area appeared back to normal.
The gates of Parliament have also reopened after a temporary closure.
British Prime Minister Theresa May thanked the security services, saying their swift response showed they were on alert.
"I would say that we owe a huge debt of gratitude to these people," May said.
May was not at 10 Downing Street during the arrest, her office told CNN.
Police said there had been no injuries and that there was no immediate threat in the aftermath of the arrest.