The 27-year-old Northern Irishman said he was "taken aback" by the reaction and criticism to his round at the president's Trump International course in Florida.
McIlroy released a statement Friday to counter claims he was a "fascist" and a "bigot" for playing with the controversial president.
But that hasn't been the end of the matter and the round with Trump featured prominently in the former world No. 1's news conference ahead of his PGA Tour comeback from injury in Mexico Tuesday.
"It's not as if we were speaking foreign policy out there, we were talking about golf and the grass he was putting on the greens," McIlroy said.
During a vitriolic campaign and a raucous start to his term, Trump has done little to reach beyond his base of deeply committed voters, with his brash, Twitter-fueled approach rocking the nation's politics.
However, as he sought to stabilize his administration after a tumultuous five weeks in office, Trump struck a more conventional presidential posture in his first address to Congress Tuesday.
"[You] respect the office even if you don't respect the guy who's in it," McIlroy told reporters ahead of the WGC-Mexico Championship which begins Thursday.
Lunch with Tiger
McIlroy, who has been out for nearly two months with a rib injury, also revealed he had lunch with Tiger Woods last week, saying of the former world No. 1: "Mentally, he's in a good place."
The 41-year-old Woods has not played since pulling out of last month's Dubai Desert Classic with back spasms, following nearly 17 months out because of multiple back surgeries.
"He's struggled with his body over the past couple of years, unfortunately it just won't allow him to do what he wants to do," McIlroy told reporters.
"It's tough but I know he's working hard to get back. It just takes time. Even if he plays eight to 10 times year it's a bonus for all of us."
McIlroy, who said he had played with President Bill Clinton and "spent time with President George W. Bush," said the opportunity to play golf with a sitting US president was unique.
"To see 20 secret service agents and 30 cops and snipers in the trees was just a surreal experience for me," he said.
"I was little bit taken aback by the blow back but I get why, it's a tough position to be in."
Trump was a frequent and vocal critic of President Barack Obama's golf habit, regularly slamming the former president for playing golf with many pressing issues before the country.
"I was just doing what I felt was respectful," the four-time major winner added. "If the president of the US phones you up and wants to play golf with you, I wasn't going to say no.
"I don't agree with everything he says. I'm not American, I can't change the way the political system is. I can't vote and even if I could vote I don't think I would have. I enjoyed myself, I had a good time.
"I'm sorry if I pi**ed people off but I felt like I was in a position where I couldn't do anything but say yes."
While McIlroy has been on the sidelines, Dustin Johnson has moved to world No. 1 and a number of his contemporaries near the top of the rankings have won, including Jordan Spieth, Hideki Matsuyama and Rickie Fowler.
McIlroy, though, says his injury may have been a "blessing in disguise."
Third-ranked McIlroy can overtake Johnson at the top if he wins at Mexico's Club de Golf Chapultepec and the American finishes worse than a two-way tie for third.
The World Golf Championships event had previously been held at Trump's Doral course in Miami, but moved to Mexico after sponsor Cadillac did not renew its contract.