"I want to kiss him tonight," said the Dutchman, after two goals and an assist from Memphis turned United's Champions League playoff against Club Brugge on its head, following a Michael Carrick own goal that gifted the Belgian visitors an early lead.
"I'm very happy for him because a player needs that," added United's manager as he reflected on Memphis' first goals for United.
"Hopefully he shall continue to score. I have confidence in him. He showed what he can do tonight. He scored two goals and got the assist for the other."
Van Gaal was even happier that Memphis believed he could have done more in the game.
"When you score two goals and the assist also, normally in the world you are the hero," said Van Gaal. "For a trainer-coach, it's different. He's never satisfied. I like that.
"You have to know what you are doing and also what you can do and then your desire to improve is higher. And that's Memphis."
Two fine individual efforts -- the first including a flick of the ball over a defender's head -- were rounded off when the 21-year-old turned provider in the final seconds, sending a ball into the box that Marouane Fellaini headed home to give United a two-goal cushion for next Wednesday's second leg.
Memphis might have netted a hat-trick, but in the opening 45 minutes he selflessly chose to pass instead of shoot, and in the second half he blazed a shot over the crossbar after being set up by Wayne Rooney's deft back-heel.
"I'm happy that I scored -- and of course I'm happy that we won," said the $34 million signing. "But I must admit I'm a bit disappointed because I could have scored the last two chances and got a hat-trick.
"I will think about that tonight and forget about it tomorrow."
Van Gaal had appeared uncharacteristically tense in the run-up to his side's playoff with their Belgian opponents.
The outspoken Dutchman, a Champions League winning coach with Ajax in 1995 and runner-up with Bayern Munich in 2010, bristled at questions regarding tactics, team selection and the form of his players when speaking with reporters Monday.
"The pressure is high," he said, revealingly, when asked about his pre-match emotions.
After watching his side battle to a hard-fought 3-1 victory at Old Trafford Tuesday, however, the 64-year-old coach can breathe easier for the time being.
Van Gaal had even gone as far as to suggest the two-legged tie would constitute the biggest games since he replaced David Moyes at the helm of the Old Trafford club last year.
Speaking at the end of Tuesday's entertaining first-leg, he seemed content with the win, yet was keen to stress that the tie was nowhere near over.
"Because of that last goal (we can be) more confident," said United's Dutch manager.
"(Fellaini's goal) makes a difference because 2-1 is a very difficult result. We deserved much more because we created a lot of chances."
Brugge strike first
Although on paper one of the easier teams United could have faced when the playoff draw was made two weeks ago -- the others being CSKA Moscow, Lazio, Monaco and Rapid Vienna -- the Blaue-Zwart proved stubborn opposition throughout.
It even had the temerity to strike first.
Victor Vazquez's eighth-minute free-kick from the angle deflected off the shin of United midfielder Michael Carrick, deceiving goalkeeper Sergio Romero to trickle into the home side's net.
The home crowd was stunned while Brugge's voluble traveling support could hardly believe its luck. Yet it was a lead that would last only four minutes.
Depay controlled Carrick's cross field pass with his chest deep in Brugge territory before dancing past three defenders and placing the ball beyond Sebastien Bruzzese.
At this stage it looked like a goal-fest could be on the cards.
But there were to be few more goalmouth incidents until just before half-time when Depay picked up Daley Blind's pass on the right hand corner of the Brugge box.
The 21-year-old summer signing from Dutch champions PSV Eindhoven took a touch, shifted it on to his right foot and curled a precise strike around the despairing Bruzzese for his second goal of the night.
Brugge was pipped to the Belgian title by Gent last season, but it's players would seldom have encountered a customer as tricky Depay on domestic duty.
The Dutchman was repeatedly allowed to cut inside from the left onto his stronger right foot with the Belgian club's defenders seemingly at a loss at how to stop him.
Depay again tested Bruzzese at the beginning of the second half then missed a golden opportunity to notch his hat-trick on the hour, blazing over after being set up by Wayne Rooney's exquisite touch.
The game was to tilt towards United yet further when Brugge was reduced to 10 men on 79 minutes.
Brandon Mechele hauled down substitute Javier Hernandez with referee Deniz Aytekin swiftly waving a second yellow card at the defender.
And United made its numerical advantage in the third minute of injury time. Depay, again allowed time on the ball, swung over a cross that Marouane Fellaini was able to nod beyond Bruzzese.
It was a goal that could prove crucial in deciding the tie, and relaxing one Louis Van Gaal.
Champions League roundup
In the evenings other playoff ties, Lazio claimed a vital home victory over Bayer Leverkusen after Keita Baldé Diao's goal ensured a slender 1-0 victory.
A late Islam Slimani strike gave Sporting Club de Portugal a 2-1 home victory over CSKA Moscow in Lisbon while BATE Borisov overcame 10-man Partizan Belgrade in Minsk thanks to a solitary Mikhail Gordeychuk goal.
FC Astana of Kazakhstan triumphed 1-0 at home to APOEL Nicosia of Cyprus courtesy of a Baurzhan Dzholchiyev strike.