The Australian is known for his fiery temperament and fell foul of chair umpire Carlos Ramos after loudly asking a ballboy for a towel during his win Sunday over Italy's Marco Cecchinato.
Ramos ruled he had overstepped the mark with his demand but Krygios disagreed in an expletive-filled rant during a first set tiebreaker.
He drew attention to an incident involving world number one Novak Djokovic at the Rome Masters last week where the Serbian escaped sanction for an altercation with umpire Carlos Bernardes -- and accused Ramos of "unbelievable bias."
Still clearly seething, Kyrgios settled down to take the breaker 8-6 and it was the same margin in the second set tiebreak.
He eventually closed out a 7-6 7-6 6-4 victory on Court One at Roland Garros after two hours 21 minutes.
The 17th seed will next play Dutch lucky loser Igor Sisjling for a place in the last 32, the furthest he has reached in the clay court grand slam.
The volatile Kyrgios told gathered reporters that he felt hard done by but did acknowledge umpires have difficult tasks.
"For me that time I didn't get mad at all. I just said it a little bit loud. The crowd was going on. I just felt like it was a bit rough.
"It's not an easy job out there, especially five-set matches, to concentrate, I think they do a good job.
"I think with someone like me that's pretty emotional it can frustrate me a little bit, but I felt like I dealt with it pretty well," he added.
After a lengthy rain delay, sixth seed Kei Nishikori, who could meet Rome winner Andy Murray in the quarterfinals, took a two sets lead over Simone Bolelli of Italy before the players were called off again.
In the women's draw, two-time grand slam champion Petra Kvitova was a first day winner having to battle hard to beat Danka Kovinic 6-2 4-6 7-5, but 2014 finalist Simona Halep was among a number of players kept waiting by the delays as her match was canceled.