An ordinary Brazil seemed on course for a second successive draw, which would have made for an especially nervy final group match against Serbia on June 27, but in the 91st minute Coutinho stabbed home against tiring opponents and Neymar added gloss to the scoreline in the 97th minute.
It was stirring finale and a victory which puts Brazil top of Group E with four points, though the South Americans may be usurped by Serbia who play Switzerland later on Friday. A draw against Serbia could be enough to put Brazil through to the last 16.
The magnitude of the result was evident in Tite's celebrations -- the Brazil coach jolted out of the dugout and giddily fell onto the pitch when Coutinho put his team ahead -- while after the final whistle Neymar could be seen on his knees sobbing.
Neymar's first goal of the tournament, scored seconds before the final whistle, moves the Paris Saint-Germain star above Romario into third on Brazil's all-time scoring list with 56 goals in 87 internationals. Only Ronaldo (62) and Pele (77) are above him in the exalted record books.
Neymar fit to start
After much conjecture over Neymar's fitness -- the world's most expensive player hobbled out of a training session on Tuesday -- the Brazilian started the match, though it took some time for him to shine.
Fouled 10 times in the opening 1-1 draw against Switzerland, the most fouls on a player at a World Cup match for 20 years, he had to endure much the same in the St. Petersburg stadium.
However, moving to the left helped the playmaker, and his teammates, and after 30 minutes the Brazilians flicked a switch and improved after a static opening, though Costa Rica's players proved to be obdurate opponents.
In the first half, Coutinho's sublime ball over the top of Costa Rica's defense set Neymar free, but the 26-year-old's poor first touch meant he was unable to test Keylor Navas in Costa Rica's goal.
Neymar's rustiness was perhaps understandable given that his first competitive outing since breaking a bone in his right foot in February was against Switzerland.
Despite Brazil's dominance in possession, Costa Rica twice threatened their more celebrated opponents.
Celso Borges should have done better than to drag his shot wide after being put through by a Cristian Gamboa cut-back, while patience from Marcos Urena just before the break would have kept him onside and through on goal.
After the break, Navas was immediately called into action. Gabriel Jesus' header hit the crossbar and Coutinho went close with the follow-up shot, while a brilliant point-blank save from the Real Madrid goalkeeper prevented Neymar from breaking the deadlock.
Through on goal with just the goalkeeper to beat, Neymar had another chance to put his team ahead, but the former Barcelona forward curled his effort wide from 20 yards.
Costa Rica were holding firm and even survived a penalty scare in the 78th minute.
Refree Bjorn Kuipers initially awarded a spotkick for a Giancarlo Gonzalez foul on Neymar, but after watching replays of the incident he correctly deemed that Gonzalez's touch of the Brazilian's shirt would not have been enough for Neymar to tumble over.
In the end, it mattered little as Coutinho came to the rescue, directing Firmino's header across goal beyond Navas from four yards, while Neymar tapped in Douglas Costa's cross to put the former champions on firmer footing towards the last 16 and Costa Rica out of the competition.