At the Etihad Stadium, Manchester City -- who are the current Premier League champions and whose Middle Eastern owners have spent close to $1.5 billion assembling the most expensive squad in the league's history -- lost to second tier Middlesbrough 2-0.
Frank Lampard looked to have secured a last minute draw but his free kick instead hit the post. A few seconds later Middlesbrough scored a second, securing a famous victory.
But arguably the bigger shock took place In west London.
At Stamford Bridge Chelsea, the current Premier League leaders who have looked imperious all season, took an early two goal lead against Bradford City, who play in English football's third tier.
An easy victory looked on the cards. But Chelsea somehow conspired to throw away their lead as Bradford City scored an incredible four goals without reply. It was the first time Chelsea have conceded four goals against opposition two leagues below them since the 1950s.
Before the game Chelsea coach Jose Mourinho said it would be "a disgrace" if his team lost Saturday's game. He didn't mince his words after the match either.
"I am ashamed," Mourinho told Sky Sports. "The players should feel exactly the same."
The English FA Cup has long been famous for its tradition of seeing smaller teams upsetting the odds and beating more established opposition.
"In this moment I feel happy for them. I am honest," Mourinho said of Bradford City's performance. The Portuguese coach, who has won the UEFA Champions League twice, visited Bradford City's dressing room after the game and congratulated the team on their performance.
"This is beauty of the FA Cup," he added.
"This happens here more than any other country."
Elsewhere it wasn't a good day for the top half of the English Premier League.
Southampton, who are currently third, were also knocked out after losing 3-2 to Crystal Palace whilst Tottenham Hotspur, in sixth, lost 2-1 to recently promoted Leicester City. Swansea, in ninth place, lost to Blackburn 3-1.
On Friday fifth placed Manchester United narrowly avoided losing to Cambridge United, who play in English football's fourth tier.
Louis Van Gaal's expensively assembled team could not break down the fourth division team, who featured former Manchester United academy player Luke Chadwick, once a contemporary of David Beckham, Paul Scholes and Ryan Giggs.
The match finished 0-0, securing Cambridge United -- whose entire yearly wage bill is the same as Argentina international Angel Di Maria's monthly paycheck -- a replay at Old Trafford and a place in the fifth round draw.
"The atmosphere in the stadium with the crowd was fantastic," Van Gaal told the BBC after the game.
"I can only give the credit to Cambridge United ... every aspect of the match was against us; the pitch, the referee, everything."
Holders Arsenal through
Sunday's action went more to form, with holders Arsenal winning 3-2 at second-flight Brighton. Goals from Theo Walcott, Mesut Ozil and Tomas Rosicky put the Gunners 3-1 ahead although Sam Baldock's late second for the home side led to a few late scares.
Fellow EPL sides Aston Villa and West Ham also progressed with 2-1 and 1-0 wins over Bournemouth and Bristol City respectively.