(CNN) -- Gary Coleman's parents grieve "more and more" every day since their son died four weeks ago, but they said they hold no anger toward the actor's ex-wife over how she handled his death.
Shannon Price, 24, ordered doctors to end Coleman's life support just a day after he fell into a coma, despite his living will's direction that he be kept alive at least 15 days.
Sue and Willie Coleman appeared on HLN's "The Joy Behar Show" on Tuesday to talk about their son's life and the controversy surrounding his sudden death.
"We're not holding any animosity towards her because, like my wife said, she was young and inexperienced," Willie Coleman told Behar.
Price, whom they never met, did not call Coleman's parents after he suffered a brain hemorrhage from a fall in his home May 26 or before he was removed from life support two days later, the Colemans said.
His parents said they felt like they were robbed of a chance to say a last goodbye to their son.
"I hurt more and more every day," Willie Coleman said. "More and more."
The Colemans spoke sadly about their estrangement from their son, who they adopted when he was four days old. The estrangement began when he became a legal adult.
"When Gary turned 18 years old, you know how you do a horse?" Willie said. "We were put out to pasture."
Willie Coleman said his son told him then that "I just feel that you were holding me in prison."
"He wanted to be his own man, do things his own way, so we tried to encourage him the best we could, we tried to guide him as much as we could, but it was his life," Sue Coleman said.
The Colemans, who had helped guide the child actor's career to that point, left California and returned to the middle-class lives they had left behind in Chicago a decade earlier.
"Gary took on a whole new branch of individuals that wanted to manage his career and also manage his money," his father said.
Although Gary Coleman sued his parents for mismanaging his money, resulting in a $1 million judgment for him in 1993, there was always love between them, they said.
Willie Coleman said he has a handful of letters from his son that offer him comfort.
"Every once and a while, I pull those letters out and read," he said. "In each of those letters is ended with 'Hey, guys. I love you guys and I know you love me too.'"
His father wants Coleman to be remembered by fans as "the sweet little gentle boy that we raised."
The parents are staying out of the bitter legal fight that is brewing over Coleman's estate.
A Provo, Utah, lawyer was named temporary special administrator to control the estate while a Utah judge decides a will dispute between ex-wife Shannon Price and former business associate Anna Gray.
