With nothing separating the sides in a tense, even game, the Roma defender stepped up and curled his shot over the wall and inside the near post after 56 minutes, leaving Keylor Navas helpless.
With Serbia president Aleksandar Vucic reportedly promising to gift the squad a €10 million ($11.6M) prize if they bring the World Cup home, Kolarov's strike could prove to be priceless with difficult games against Brazil and sixth-ranked Switzerland still to come in Group E.
Yet Serbia, appearing in a first World Cup finals since South Africa 2010, will be wary of allowing complacency to creep in.
Eight years ago they were eliminated at the group stage, despite defeating eventual semifinalists Germany.
Moment of magic
Defeat for Costa Rica ended their run of five World Cup games without loss, having exited Brazil 2014 on penalties in the quarterfinal stage after drawing with Holland.
But it was Los Ticos that started strongest in Samara, taking advantage of a snoozing Serbia defense with a quick corner, but Giancarlo Gonzalez's header went well over when unmarked from four yards out.
With neither side able to stamp their authority on the match, it felt like a moment of magic would need to be conjured to separate the teams.
Sergej Milinković-Savić, the Lazio midfielder who could be in the middle of this summer transfer window's biggest tug of war, almost provided it.
The 23-year-old met Branislav Ivanovic's cross into the box in spectacular style, but his bicycle kick was well saved and gathered at the second attempt by Navas.
Savic and strike partner Aleksandar Mitrovic looked a threat to Costa Rica's defense every time they went forward and the pair created the best opening of the game soon after the restart.
Mitrovic latched onto Savic's first time through ball but couldn't find a way past the imposing figure of Navas, the Real Madrid keeper tipping the shot wide.
But it wasn't long before Serbia found the breakthrough and it was Kolarov's wand of a left foot which provided it.
After Mitrovic was fouled, Kolarov stepped up to the free-kick, all of 30 yards from goal, and struck it an inch inside the post, leaving even a keeper of Navas' quality grasping at thin air.
That was the third goal scored directly from a free-kick at Russia 2018, already equaling the amount from Brazil four years ago.
With Kolarov in that type of form, it might not be the last.