(CNN) -- In his prime, about five years ago, James Arthur Ray could draw a crowd of thousands. He was on the New York Times Best Sellers list. He had Oprah's stamp of approval.
To get the best out of people, the self-help guru was known for pushing them past their limits.
But in October 2009, Ray pushed too far. Three people died.
And Ray was sentenced to 20 months in prison for negligent homicide.
Just months out of prison, Ray, 56, sat down with CNN's Piers Morgan on Monday night to express remorse.
"I think the most difficult thing I can ever imagine is investing your entire life in helping people, and then finding them getting hurt," he said. "It's just the antithesis of anything that I had ever stood for or wanted. And so that anguish has continued every single day since that moment."
The death's took place at a sweat lodge ceremony during the Spiritual Warrior weekend at the 70-acre Angel Valley retreat outside Sedona.
It was Ray's recklessness, prosecutors argued, that took the lives of Kirby Brown, 38, of Westtown, New York; James Shore, 40, of Milwaukee; and Lizbeth Marie Neuman, 49, of Prior Lake, Minnesota. At least 15 others who took part in the ceremony became ill.
The lodge, made of willow trees and branches and covered with tarpaulins and blankets, was heated to a perilously high temperature, causing the participants to suffer dehydration and heatstroke, prosecutors said.
They also said Ray didn't monitor the temperature inside the lodge or the well-being of participants and was indifferent to those having trouble.
Ray, sometimes seeming on the verge of tears, admitted that he made mistakes.
"Was I arrogant? Yes. I have that characteristic, I can be arrogant. And I think there's a lot of hubris that comes in my former business. You know, people flying all over the world and asking me how to have a better life," he said. "It tends to go to your head. You know? You tend to think you've got all the answers, and so you get humbled."
But not everybody may be convinced that Ray has changed.
Family members of Brown, Neuman and Shore have said they will keep an eye on him should he attempt to rebuild his self-help empire.
They are upset at some of the practices in the Arizona ceremony that participants shaved their heads, meditated in the desert for 36 hours without food and water and then were told they would symbolically die and be reborn in the sweat lodge ritual, according to court testimony.
Ray said he will not do a sweat lodge again. He said he's lost all the millions he earned doing that work.
"If I could trade places with any of the three, James Kirby or Liz, I would do it," Ray said.
Ann O'Neill contributed to this report
Watch Piers Morgan Live weeknights 9 p.m. ET. For the latest from Piers Morgan click here.