(CNN) -- Basketball Hall of Famer and former Los Angeles Clippers general manager Elgin Baylor told CNN's "Anderson Cooper 360" that he had arguments with Donald Sterling about his views on race.
"He would say, 'Well, that's the way I feel," Baylor said in his first national television remarks about Sterling since the scandal unfolded. "Occasionally it might come up about players."
Baylor, who lost a lawsuit alleging he was fired by the team because of his age, said Sterling never directed racist language at him. But when asked if the longtime owner who now is banned for life from the league was a racist, Baylor didn't pause.
"Of course he is," Baylor said in an interview telecast on Wednesday. "There's no doubt in my mind now."
Baylor's wife, Elaine, elaborated.
"He's not running around with a white robe on," she told Cooper. "He's not that blatant. He's a very smart man, but he communicates how he feels and what he wants, especially to people who work for him. So you know where he is and what he wants."
Elgin Baylor said that he feels vindicated by the racist remarks that were released in an audio recording on TMZ's website last month even though he lost in court. Baylor's suit originally included the accusation that he was canned because of racism but that was taken out of the lawsuit.
"Justice has been served," he said. "Now they know what Donald is like, and the things I said before about Donald are absolutely true."
One of those things Baylor had related was how Sterling on "many occasions" would bring women into the locker room and look at the players in the showers.
"Look at those beautiful black bodies. That's what he used to say," the NBA legend said.
Baylor said he told Sterling several times to quit before the owner finally did so.
The former GM, who spent 22 years directing player acquisitions, also told CNN how Sterling asked him to go out early in his career and scout playgrounds in the inner cities to see if he could find any talented players.
The NBA is in the process of forcing Sterling, who has owned the team since it was in San Diego in 1981, to sell the franchise. Baylor foresees a potential legal fight.
"Donald likes the limelight, whether it's good or bad," he said.
Baylor was the Clippers GM during many of their worst seasons. He joined the team in 1986 after a career in which he scored 23,149 points, which puts him 33rd on the list of all-time scorers in the NBA, and a stint as the coach of New Orleans Jazz. The Clippers were the joke of the league, though, and only made the playoffs three times.
He was named NBA executive of the year in 2006 when the Clippers made the second round of the postseason, but two years later, he said he was fired. The team said he resigned.
The audio recordings brought back memories, Baylor said.
"It didn't surprise me ... at all," he said. "That's the Donald Sterling I know."