Douglas Costa scored the winning spot-kick for Bayern after Sven Bender and Sokratis Papastathopoulos failed to convert for Dortmund.
The two best sides in German football over recent years could not be separated after 90 minutes of normal time and 30 minutes of extra time ended goalless.
Dortmund striker Pierre-Emerick Aubameyang missed a guilt-edged chance that could have won the cup for Dortmund late on in regular time.
But Guardiola was given the perfect send off thanks as successful penalties from Arturo Vidal, Robert Lewandowski, Thomas Mueller and Costa ensured a 4-3 shootout win.
Guardiola leaves Bayern to take over at moneyed English Premier League club Manchester City this summer.
And the former Barcelona coach appeared to have tears in his eyes as he was tossed in the air by celebrating Bayern players.
"Anything can happen in a penalty shootout, so we are delighted," Guardiola said in comments carried by the AFP news agency after the game.
"It has been an amazing three years at this club. I will miss the players so much."
Although he has failed to to win the Champions League during his three-year stint in charge of Bayern, Guardiola has claimed the Bundesliga title three times, the German cup twice as well as triumphs at the the FIFA World Club Cup and in the UEFA Super Cup.
Guardiola will be replaced at Bayern by former AC Milan, Chelsea and Real Madrid boss, Carlo Ancelotti over the summer.
Zlatan scores as PSG win French Cup
Swedish striker, Zlatan Ibrahimovic, scored twice as Paris Saint Germain completed the French treble by beating Marseille 4-2 in the final of the Coupe de France.
Ibrahimovic was playing his last game for PSG and scored a 47th minute penalty to give the Parisians a 2-1 lead after teammate Blaise Matuidi's early strike was canceled out for Marseille by Florian Thauvin.
Edinson Cavani made it 3-1 to PSG on 57 minutes before Ibrahimovic grabbed his second eight minutes before the end.
Michy Batshuayi scored a late consolation for Marseille but that did little to dampen the spirits of the Parisian players or crowd.
PSG has now won the Coupe de France on 10 occasions, including three times in the past seven years.
Morata leaves it late for Juve
Juventus put the seal on the Italian double by defeating AC Milan after extra in the Coppa Italia final in Rome.
Substitute Alvaro Morata scored the only goal of the game after 110 minutes to ensure Juventus became the first team to win back to back doubles in Italy.
Defeat for Milan means it fails to qualify for European competition next season. That honor will now go to Sassuolo which finished sixth in Serie A.
Milan could only finish in seventh position this season and needed to win the Coppa Italia to have any hope of qualifying for the Europa League.